An Empirical Study of Multitask Learning to Improve Open Domain Dialogue Systems
Paper i proceeding, 2023

Autoregressive models used to generate responses in open-domain dialogue systems often struggle to take long-term context into account and to maintain consistency over a dialogue. Previous research in open-domain dialogue generation has shown that the use of auxiliary tasks can introduce inductive biases that encourage the model to improve these qualities. However, most previous research has focused on encoder-only or encoder/decoder models, while the use of auxiliary tasks in encoder-only autoregressive models is under-explored. This paper describes an investigation where four different auxiliary tasks are added to small and medium-sized GPT-2 models fine-tuned on the PersonaChat and DailyDialog datasets. The results show that the introduction of the new auxiliary tasks leads to small but consistent improvement in evaluations of the investigated models.

open-domain dialogue

multitask learning

auxiliary tasks

Författare

Mehrdad Farahani

Chalmers, Data- och informationsteknik, Data Science och AI

Richard Johansson

Göteborgs universitet

Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa)

1736-8197 (ISSN) 1736-6305 (eISSN)

347-357
978-99-1621-999-7 (ISBN)

24th Nordic Conference on Computational Linguistics (NoDaLiDa)
Tórshavn, Faroe Islands,

Ämneskategorier

Språkteknologi (språkvetenskaplig databehandling)

Datavetenskap (datalogi)

Mer information

Senast uppdaterat

2024-04-03