Search Results - "Mahamood, Saad"

Automatic Metrics in Natural Language Generation: A Survey of Current Evaluation Practices by Schmidtová, Patrícia, Mahamood, Saad, Balloccu, Simone, Dušek, Ondřej, Gatt, Albert, Gkatzia, Dimitra, Howcroft, David M, Plátek, Ondřej, Sivaprasad, Adarsa

“…Automatic metrics are extensively used to evaluate natural language processing systems. However, there has been increasing focus on how they are used and…”
Get full text

On the Role of Summary Content Units in Text Summarization Evaluation by Nawrath, Marcel, Nowak, Agnieszka, Ratz, Tristan, Walenta, Danilo C, Opitz, Juri, Ribeiro, Leonardo F. R, Sedoc, João, Deutsch, Daniel, Mille, Simon, Liu, Yixin, Zhang, Lining, Gehrmann, Sebastian, Mahamood, Saad, Clinciu, Miruna, Chandu, Khyathi, Hou, Yufang

“…At the heart of the Pyramid evaluation method for text summarization lie human written summary content units (SCUs). These SCUs are concise sentences that…”
Get full text

Needle in a Haystack: An Analysis of High-Agreement Workers on MTurk for Summarization by Zhang, Lining, Mille, Simon, Hou, Yufang, Deutsch, Daniel, Clark, Elizabeth, Liu, Yixin, Mahamood, Saad, Gehrmann, Sebastian, Clinciu, Miruna, Chandu, Khyathi, Sedoc, João

“…To prevent the costly and inefficient use of resources on low-quality annotations, we want a method for creating a pool of dependable annotators who can…”
Get full text

Generating affective natural language for parents of neonatal infants by Mahamood, Saad Ali

“…The thesis presented here describes original research in the field of Natural Language Generation (NLG). NLG is the subfield of artificial intelligence that is…”
Get full text

Automatic Construction of Evaluation Suites for Natural Language Generation Datasets by Mille, Simon, Dhole, Kaustubh D, Mahamood, Saad, Perez-Beltrachini, Laura, Gangal, Varun, Kale, Mihir, van Miltenburg, Emiel, Gehrmann, Sebastian

“…Machine learning approaches applied to NLP are often evaluated by summarizing their performance in a single number, for example accuracy. Since most test sets…”
Get full text

Underreporting of errors in NLG output, and what to do about it by van Miltenburg, Emiel, Clinciu, Miruna-Adriana, Dušek, Ondřej, Gkatzia, Dimitra, Inglis, Stephanie, Leppänen, Leo, Mahamood, Saad, Manning, Emma, Schoch, Stephanie, Thomson, Craig, Wen, Luou

“…We observe a severe under-reporting of the different kinds of errors that Natural Language Generation systems make. This is a problem, because mistakes are an…”
Get full text

Neonatal Intensive Care Information for Parents An Affective Approach by Mahamood, S., Reiter, E., Mellish, C.

“…Based upon qualitative work done with former Neonatal Intensive Care Unit parents, we propose a potential user model to estimate the level of stress/anxiety…”
Get full text

Missing Information, Unresponsive Authors, Experimental Flaws: The Impossibility of Assessing the Reproducibility of Previous Human Evaluations in NLP by Belz, Anya, Thomson, Craig, Reiter, Ehud, Abercrombie, Gavin, Alonso-Moral, Jose M, Arvan, Mohammad, Braggaar, Anouck, Cieliebak, Mark, Clark, Elizabeth, van Deemter, Kees, Dinkar, Tanvi, Dušek, Ondřej, Eger, Steffen, Fang, Qixiang, Gao, Mingqi, Gatt, Albert, Gkatzia, Dimitra, González-Corbelle, Javier, Hovy, Dirk, Hürlimann, Manuela, Ito, Takumi, Kelleher, John D, Klubicka, Filip, Krahmer, Emiel, Lai, Huiyuan, van der Lee, Chris, Li, Yiru, Mahamood, Saad, Mieskes, Margot, van Miltenburg, Emiel, Mosteiro, Pablo, Nissim, Malvina, Parde, Natalie, Plátek, Ondřej, Rieser, Verena, Ruan, Jie, Tetreault, Joel, Toral, Antonio, Wan, Xiaojun, Wanner, Leo, Watson, Lewis, Yang, Diyi

“…We report our efforts in identifying a set of previous human evaluations in NLP that would be suitable for a coordinated study examining what makes human…”
Get full text

GEMv2: Multilingual NLG Benchmarking in a Single Line of Code by Gehrmann, Sebastian, Bhattacharjee, Abhik, Mahendiran, Abinaya, Wang, Alex, Papangelis, Alexandros, Madaan, Aman, McMillan-Major, Angelina, Shvets, Anna, Upadhyay, Ashish, Yao, Bingsheng, Wilie, Bryan, Bhagavatula, Chandra, You, Chaobin, Thomson, Craig, Garbacea, Cristina, Wang, Dakuo, Deutsch, Daniel, Xiong, Deyi, Jin, Di, Gkatzia, Dimitra, Radev, Dragomir, Clark, Elizabeth, Durmus, Esin, Ladhak, Faisal, Ginter, Filip, Winata, Genta Indra, Strobelt, Hendrik, Hayashi, Hiroaki, Novikova, Jekaterina, Kanerva, Jenna, Chim, Jenny, Zhou, Jiawei, Clive, Jordan, Maynez, Joshua, Sedoc, João, Juraska, Juraj, Dhole, Kaustubh, Chandu, Khyathi Raghavi, Perez-Beltrachini, Laura, Ribeiro, Leonardo F. R, Tunstall, Lewis, Zhang, Li, Pushkarna, Mahima, Creutz, Mathias, White, Michael, Kale, Mihir Sanjay, Eddine, Moussa Kamal, Daheim, Nico, Subramani, Nishant, Dusek, Ondrej, Liang, Paul Pu, Ammanamanchi, Pawan Sasanka, Zhu, Qi, Puduppully, Ratish, Kriz, Reno, Shahriyar, Rifat, Cardenas, Ronald, Mahamood, Saad, Osei, Salomey, Cahyawijaya, Samuel, Štajner, Sanja, Montella, Sebastien, Shailza, Jolly, Shailza, Mille, Simon, Hasan, Tahmid, Shen, Tianhao, Adewumi, Tosin, Raunak, Vikas, Raheja, Vipul, Nikolaev, Vitaly, Tsai, Vivian, Jernite, Yacine, Xu, Ying, Sang, Yisi, Liu, Yixin, Hou, Yufang

“…Evaluation in machine learning is usually informed by past choices, for example which datasets or metrics to use. This standardization enables the comparison…”
Get full text

NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation by Dhole, Kaustubh D, Gangal, Varun, Gupta, Aadesh, Mahendiran, Abinaya, Mille, Simon, Shrivastava, Ashish, Tan, Samson, Wu, Tongshuang, Hovy, Eduard, Dusek, Ondrej, Ruder, Sebastian, Anand, Sajant, Aneja, Nagender, Barthe, Lisa, Boyle, Connor, Brun, Caroline, Cabezudo, Marco Antonio Sobrevilla, Cahyawijaya, Samuel, Chapuis, Emile, Che, Wanxiang, Choudhary, Mukund, Clauss, Christian, Colombo, Pierre, Cornell, Filip, Das, Mayukh, Dixit, Tanay, Dopierre, Thomas, Dray, Paul-Alexis, Dubey, Suchitra, Di Giovanni, Marco, Goyal, Tanya, Gupta, Rishabh, Han, Sang, Harel-Canada, Fabrice, Honore, Antoine, Jindal, Ishan, Joniak, Przemyslaw K, Kleyko, Denis, Kovatchev, Venelin, Krishna, Kalpesh, Kumar, Ashutosh, Langer, Stefan, Lee, Seungjae Ryan, Liang, Kaizhao, Liu, Zhexiong, Lukyanenko, Andrey, Marivate, Vukosi, de Melo, Gerard, Meoni, Simon, Meyer, Maxime, Mir, Afnan, Muennighoff, Niklas, Mun, Timothy Sum Hon, Murray, Kenton, Namysl, Marcin, Obedkova, Maria, Pfister, Jan, Plant, Richard, Prabhu, Vinay, Qin, Libo, Raji, Shahab, Rajpoot, Pawan Kumar, Rinberg, Roy, Roux, Claude, S, Vasconcellos P. H, Sai, Ananya B, Schmidt, Robin M, Scialom, Thomas, Sefara, Tshephisho, Shamsi, Saqib N, Shen, Xudong, Shi, Haoyue, Shi, Yiwen, Siegel, Nick, Sileo, Damien, Simon, Jamie, Singh, Chandan, Soni, Priyank, Sorensen, Taylor, Soto, William, Srivastava, Aman, Sun, Tony, Tabassum, A, Tan, Fiona Anting, Teehan, Ryan, Tiwari, Mo, Tolkiehn, Marie, Wang, Athena, Wang, Zijian, Wang, Gloria, Wang, Zijie J, Wei, Fuxuan, Wilie, Bryan, Winata, Genta Indra, Wu, Xinyi, Wydmański, Witold, Xie, Tianbao, Yaseen, Usama, Yee, Michael A, Zhang, Yue

“…Data augmentation is an important component in the robustness evaluation of models in natural language processing (NLP) and in enhancing the diversity of the…”
Get full text

The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics by Gehrmann, Sebastian, Adewumi, Tosin, Aggarwal, Karmanya, Ammanamanchi, Pawan Sasanka, Anuoluwapo, Aremu, Bosselut, Antoine, Chandu, Khyathi Raghavi, Clinciu, Miruna, Das, Dipanjan, Dhole, Kaustubh D, Du, Wanyu, Durmus, Esin, Dušek, Ondřej, Emezue, Chris, Gangal, Varun, Garbacea, Cristina, Hashimoto, Tatsunori, Hou, Yufang, Jernite, Yacine, Jhamtani, Harsh, Ji, Yangfeng, Jolly, Shailza, Kale, Mihir, Kumar, Dhruv, Ladhak, Faisal, Madaan, Aman, Maddela, Mounica, Mahajan, Khyati, Mahamood, Saad, Majumder, Bodhisattwa Prasad, Martins, Pedro Henrique, McMillan-Major, Angelina, Mille, Simon, van Miltenburg, Emiel, Nadeem, Moin, Narayan, Shashi, Nikolaev, Vitaly, Niyongabo, Rubungo Andre, Osei, Salomey, Parikh, Ankur, Perez-Beltrachini, Laura, Rao, Niranjan Ramesh, Raunak, Vikas, Rodriguez, Juan Diego, Santhanam, Sashank, Sedoc, João, Sellam, Thibault, Shaikh, Samira, Shimorina, Anastasia, Cabezudo, Marco Antonio Sobrevilla, Strobelt, Hendrik, Subramani, Nishant, Xu, Wei, Yang, Diyi, Yerukola, Akhila, Zhou, Jiawei

“…We introduce GEM, a living benchmark for natural language Generation (NLG), its Evaluation, and Metrics. Measuring progress in NLG relies on a constantly…”
Get full text