Search Results - "Kapoor, Sayash"

Leakage and the reproducibility crisis in machine-learning-based science by Kapoor, Sayash, Narayanan, Arvind

“…Machine-learning (ML) methods have gained prominence in the quantitative sciences. However, there are many known methodological pitfalls, including data…”
Get full text

Corruption-tolerant bandit learning by Kapoor, Sayash, Patel, Kumar Kshitij, Kar, Purushottam

“…We present algorithms for solving multi-armed and linear-contextual bandit tasks in the face of adversarial corruptions in the arm responses. Traditional…”
Get full text

The limitations of machine learning models for predicting scientific replicability by Crockett, M J, Bai, Xuechunzi, Kapoor, Sayash, Messeri, Lisa, Narayanan, Arvind

Get full text

Considerations for governing open foundation models by Bommasani, Rishi, Kapoor, Sayash, Klyman, Kevin, Longpre, Shayne, Ramaswami, Ashwin, Zhang, Daniel, Schaake, Marietje, Ho, Daniel E, Narayanan, Arvind, Liang, Percy

“…Different policy proposals may disproportionately affect the innovation ecosystem…”
Get full text

REFORMS: Consensus-based Recommendations for Machine-learning-based Science by Kapoor, Sayash, Cantrell, Emily M, Peng, Kenny, Pham, Thanh Hien, Bail, Christopher A, Gundersen, Odd Erik, Hofman, Jake M, Hullman, Jessica, Lones, Michael A, Malik, Momin M, Nanayakkara, Priyanka, Poldrack, Russell A, Raji, Inioluwa Deborah, Roberts, Michael, Salganik, Matthew J, Serra-Garcia, Marta, Stewart, Brandon M, Vandewiele, Gilles, Narayanan, Arvind

“…Machine learning (ML) methods are proliferating in scientific research. However, the adoption of these methods has been accompanied by failures of validity,…”
Get full text

Leakage and the Reproducibility Crisis in ML-based Science by Kapoor, Sayash, Narayanan, Arvind

“…The use of machine learning (ML) methods for prediction and forecasting has become widespread across the quantitative sciences. However, there are many known…”
Get full text

How large language models can reshape collective intelligence by Burton, Jason W., Lopez-Lopez, Ezequiel, Hechtlinger, Shahar, Rahwan, Zoe, Aeschbach, Samuel, Bakker, Michiel A., Becker, Joshua A., Berditchevskaia, Aleks, Berger, Julian, Brinkmann, Levin, Flek, Lucie, Herzog, Stefan M., Huang, Saffron, Kapoor, Sayash, Narayanan, Arvind, Nussberger, Anne-Marie, Yasseri, Taha, Nickl, Pietro, Almaatouq, Abdullah, Hahn, Ulrike, Kurvers, Ralf H. J. M., Leavy, Susan, Rahwan, Iyad, Siddarth, Divya, Siu, Alice, Woolley, Anita W., Wulff, Dirk U., Hertwig, Ralph

“…Collective intelligence underpins the success of groups, organizations, markets and societies. Through distributed cognition and coordination, collectives can…”
Get full text

Promises and pitfalls of artificial intelligence for legal applications by Kapoor, Sayash, Henderson, Peter, Narayanan, Arvind

“…Is AI set to redefine the legal profession? We argue that this claim is not supported by the current evidence. We dive into AI's increasingly prevalent roles…”
Get full text

CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmark by Siegel, Zachary S, Kapoor, Sayash, Nagdir, Nitya, Stroebl, Benedikt, Narayanan, Arvind

“…AI agents have the potential to aid users on a variety of consequential tasks, including conducting scientific research. To spur the development of useful…”
Get full text

The Foundation Model Transparency Index v1.1: May 2024 by Bommasani, Rishi, Klyman, Kevin, Kapoor, Sayash, Longpre, Shayne, Xiong, Betty, Maslej, Nestor, Liang, Percy

“…Foundation models are increasingly consequential yet extremely opaque. To characterize the status quo, the Foundation Model Transparency Index was launched in…”
Get full text

AI Agents That Matter by Kapoor, Sayash, Stroebl, Benedikt, Siegel, Zachary S, Nadgir, Nitya, Narayanan, Arvind

“…AI agents are an exciting new research direction, and agent development is driven by benchmarks. Our analysis of current agent benchmarks and evaluation…”
Get full text

Foundation Model Transparency Reports by Bommasani, Rishi, Klyman, Kevin, Longpre, Shayne, Xiong, Betty, Kapoor, Sayash, Maslej, Nestor, Narayanan, Arvind, Liang, Percy

“…Published in AIES 2024 Foundation models are critical digital technologies with sweeping societal impact that necessitates transparency. To codify how…”
Get full text

The worst of both worlds: A comparative analysis of errors in learning from data in psychology and machine learning by Hullman, Jessica, Kapoor, Sayash, Nanayakkara, Priyanka, Gelman, Andrew, Narayanan, Arvind

“…Recent arguments that machine learning (ML) is facing a reproducibility and replication crisis suggest that some published claims in ML research cannot be…”
Get full text

The Foundation Model Transparency Index by Bommasani, Rishi, Klyman, Kevin, Longpre, Shayne, Kapoor, Sayash, Maslej, Nestor, Xiong, Betty, Zhang, Daniel, Liang, Percy

“…Foundation models have rapidly permeated society, catalyzing a wave of generative AI applications spanning enterprise and consumer-facing contexts. While the…”
Get full text

Weaving Privacy and Power: On the Privacy Practices of Labor Organizers in the U.S. Technology Industry by Kapoor, Sayash, Sun, Matthew, Wang, Mona, Jaźwińska, Klaudia, Watkins, Elizabeth Anne

“…We investigate the privacy practices of labor organizers in the computing technology industry and explore the changes in these practices as a response to…”
Get full text

Towards a Framework for Openness in Foundation Models: Proceedings from the Columbia Convening on Openness in Artificial Intelligence by Basdevant, Adrien, François, Camille, Storchan, Victor, Bankston, Kevin, Bdeir, Ayah, Behlendorf, Brian, Debbah, Merouane, Kapoor, Sayash, LeCun, Yann, Surman, Mark, King-Turvey, Helen, Lambert, Nathan, Maffulli, Stefano, Marda, Nik, Shivkumar, Govind, Tunney, Justine

“…Over the past year, there has been a robust debate about the benefits and risks of open sourcing foundation models. However, this discussion has often taken…”
Get full text

The Responsible Foundation Model Development Cheatsheet: A Review of Tools & Resources by Longpre, Shayne, Biderman, Stella, Albalak, Alon, Schoelkopf, Hailey, McDuff, Daniel, Kapoor, Sayash, Klyman, Kevin, Lo, Kyle, Ilharco, Gabriel, San, Nay, Rauh, Maribeth, Skowron, Aviya, Vidgen, Bertie, Weidinger, Laura, Narayanan, Arvind, Sanh, Victor, Adelani, David, Liang, Percy, Bommasani, Rishi, Henderson, Peter, Luccioni, Sasha, Jernite, Yacine, Soldaini, Luca

“…Foundation model development attracts a rapidly expanding body of contributors, scientists, and applications. To help shape responsible development practices,…”
Get full text

A Safe Harbor for AI Evaluation and Red Teaming by Longpre, Shayne, Kapoor, Sayash, Klyman, Kevin, Ramaswami, Ashwin, Bommasani, Rishi, Blili-Hamelin, Borhane, Huang, Yangsibo, Skowron, Aviya, Yong, Zheng-Xin, Kotha, Suhas, Zeng, Yi, Shi, Weiyan, Yang, Xianjun, Southen, Reid, Robey, Alexander, Chao, Patrick, Yang, Diyi, Jia, Ruoxi, Kang, Daniel, Pentland, Sandy, Narayanan, Arvind, Liang, Percy, Henderson, Peter

“…Independent evaluation and red teaming are critical for identifying the risks posed by generative AI systems. However, the terms of service and enforcement…”
Get full text

On the Societal Impact of Open Foundation Models by Kapoor, Sayash, Bommasani, Rishi, Klyman, Kevin, Longpre, Shayne, Ramaswami, Ashwin, Cihon, Peter, Hopkins, Aspen, Bankston, Kevin, Biderman, Stella, Bogen, Miranda, Chowdhury, Rumman, Engler, Alex, Henderson, Peter, Jernite, Yacine, Lazar, Seth, Maffulli, Stefano, Nelson, Alondra, Pineau, Joelle, Skowron, Aviya, Song, Dawn, Storchan, Victor, Zhang, Daniel, Ho, Daniel E, Liang, Percy, Narayanan, Arvind

“…Foundation models are powerful technologies: how they are released publicly directly shapes their societal impact. In this position paper, we focus on open…”
Get full text

Balanced News Using Constrained Bandit-based Personalization by Kapoor, Sayash, Keswani, Vijay, Vishnoi, Nisheeth K, Celis, L. Elisa

“…We present a prototype for a news search engine that presents balanced viewpoints across liberal and conservative articles with the goal of de-polarizing…”
Get full text