About this video Over deze video
Abstract: As large language models integrate into daily workflows—from personal assistants to workplace tools—they handle sensitive information from multiple sources yet struggle to reason about what to share, with whom, and when. In this talk, we explore critical gaps in LLMs' privacy reasoning through complementary benchmarks. First, ConfAIde [ICLR2024 Spotlight] reveals that even advanced models like GPT-4 inappropriately disclose private information in contexts where humans would maintain boundaries. Second, we extend this analysis, in CIMemories [ICLR2026], to persistent memories—an increasingly adopted personalization feature—showing failures in handling compositional secrets with multiple attributes and contextual cues. We then present a data minimization framework [ICLR 2026] that formally defines the least privacy-revealing disclosure that maintains task utility. Our experiments show frontier models can tolerate up to 85% data redaction without losing functionality, yet they lack awareness of what information they actually need—leading to systematic oversharing. We conclude with techniques for restoring performance when privacy measures are applied, offering a path toward AI systems that respect contextual privacy norms while remaining useful.
Niloofar Mireshghallah is a Member of Technical Staff at humans&, working on building AI systems that model the long-term social good of people. Beginning Fall 2026, she will join Carnegie Mellon University as an Assistant Professor jointly appointed in the Language Technologies Institute (LTI) and the Department of Engineering & Public Policy (EPP), and will be a core member of CyLab. Previously, she was a Research Scientist in the Alignment group at Meta's Fundamental AI Research (FAIR) lab until November 2025, working on privacy-preserving AI systems and LLM safety. Before that, she was a post-doctoral scholar at the Paul G. Allen School of Computer Science & Engineering at the University of Washington, advised by Yejin Choi and Yulia Tsvetkov. She received her Ph.D. in Computer Science from UC San Diego in 2023. Her research focuses on privacy-preserving AI systems, LLM policy and ethics, contextual integrity in AI, and AI for science and health. Niloofar's work has been recognized with the Tinker Academic Research Compute Grant (2025), Modal Academic Research Compute Grant (2025), NCWIT Collegiate Award (2020), finalist distinction in the Qualcomm Innovation Fellowship (2021), the Rising Star in Adversarial ML Award (2022), and selection for the Rising Stars in EECS workshop (2022).
Website: https://mireshghallah.github.io/
This session is brought to you by the Cohere Labs Open Science Community - a space where ML researchers, engineers, linguists, social scientists, and lifelong learners connect and collaborate with each other. We'd like to extend a special thank you to Manuel Villanueva and Damani Leads of ourPrivacy, Security and Policy group for their dedication in organizing this event.
If you’re interested in sharing your work, we welcome you to join us! Simply fill out the form at https://forms.gle/ALND9i6KouEEpCnz6 to express your interest in becoming a speaker.
Join the Cohere Labs Open Science Community to see a full list of upcoming events (https://tinyurl.com/CohereLabsCommunityApp).
