PCVAE: Generating Prior Context for Dialogue Response Generation

PCVAE: Generating Prior Context for Dialogue Response Generation

Zefeng Cai, Zerui Cai

Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence
Main Track. Pages 4065-4071. https://doi.org/10.24963/ijcai.2022/564

Conditional Variational AutoEncoder (CVAE) is promising for modeling one-to-many relationships in dialogue generation, as it can naturally generate many responses from a given context. However, the conventional used continual latent variables in CVAE are more likely to generate generic rather than distinct and specific responses. To resolve this problem, we introduce a novel discrete variable called prior context which enables the generation of favorable responses. Specifically, we present Prior Context VAE (PCVAE), a hierarchical VAE that learns prior context from data automatically for dialogue generation. Meanwhile, we design Active Codeword Transport (ACT) to help the model actively discover potential prior context. Moreover, we propose Autoregressive Compatible Arrangement (ACA) that enables modeling prior context in autoregressive style, which is crucial for selecting appropriate prior context according to a given context. Extensive experiments demonstrate that PCVAE can generate distinct responses and significantly outperforms strong baselines.
Keywords:
Natural Language Processing: Language Generation
Machine Learning: Autoencoders
Natural Language Processing: Dialogue and Interactive Systems