Overview of the Ninth Dialog System Technology Challenge: DSTC9

• Published in: IEEE/ACM transactions on audio, speech, and language processing Vol. 32; pp. 4066 - 4076
• Main Authors: Gunasekara, Chulaka, Kim, Seokhwan, D'Haro, Luis Fernando, Rastogi, Abhinav, Chen, Yun-Nung, Eric, Mihail, Hedayatnia, Behnam, Gopalakrishnan, Karthik, Liu, Yang, Huang, Chao-Wei, Hakkani-Tur, Dilek, Li, Jinchao, Zhu, Qi, Luo, Lingxiao, Liden, Lars, Huang, Kaili, Shayandeh, Shahin, Liang, Runze, Peng, Baolin, Zhang, Zheng, Shukla, Swadheen, Huang, Minlie, Gao, Jianfeng, Mehri, Shikib, Feng, Yulan, Gordon, Carla, Alavi, Seyed Hossein, Traum, David, Eskenazi, Maxine, Beirami, Ahmad, Cho, Eunjoon, Crook, Paul A., De, Ankita, Geramifard, Alborz, Kottur, Satwik, Moon, Seungwhan, Poddar, Shivani, Subba, Rajen
• Format: Journal Article
• Language: English
• Published: IEEE 2024
• Subjects: Accuracy; Artificial intelligence; chatbots; computational intelligence; computational linguistics; human computer interaction; human-machine systems; Mathematical models; Measurement; natural language generation; natural language processing; Oral communication; personal voice assistants; Proposals; Speech processing; Task analysis
• ISSN: 2329-9290; 2329-9304
• DOI: 10.1109/TASLP.2024.3426331

This paper introduces the Ninth Dialog System Technology Challenge (DSTC-9). This edition of the DSTC focuses on applying end-to-end dialog technologies for four distinct tasks in dialog systems, namely, 1. Task-oriented dialog Modeling with Unstructured Knowledge Access, 2. Multi-domain task-oriented dialog, 3. Interactive evaluation of dialog and 4. Situated interactive multimodal dialog. This paper describes the task definition, provided datasets, baselines, and evaluation setup for each track. We also summarize the results of the submitted systems to highlight the general trends of the state-of-the-art technologies for the tasks.