I am a tenured faculty (equivalent to full professor) at CISPA Helmholtz Center for Information Security. I sometimes chime in iDRAMA Lab for the memes.

Research Areas

  • Trustworthy Machine Learning, with a focus on LLMs (Safety, Privacy, and Security)
  • Misinformation, Hate Speech, and Memes
  • Social Network Analysis

I’m always looking for motivated students and postdocs to join my group. If you are interested, please write me an email (zhang@cispa.de).

Awards

  • Best paper finalist at CSAW Europe 2024
  • Best paper finalist at CSAW Europe 2023
  • Best paper award honorable mention at CCS 2022
  • Busy Beaver teaching award nomination for seminar “Privacy of Machine Learning” at Saarland University (2022 Winter)
  • Busy Beaver teaching award nomination for advanced lecture “Machine Learning Privacy” at Saarland University (2022 Summer)
  • Busy Beaver teaching award for seminar “Privacy of Machine Learning” at Saarland University (2021 Winter)
  • Distinguished reviewer award at TrustML Workshop 2020 (co-located with ICLR 2020)
  • Distinguished paper award at NDSS 2019
  • Best paper award at ARES 2014

What’s New

  • [3/2025] Rui Wen has successfully passed his Ph.D. defense! Congratulations, Dr. Wen!
  • [3/2025] Yixin Wu and Xinyue Shen are named MLCommons Rising Stars 2025!
  • [1/2025] Xinyue Shen is named KAUST Rising Star in AI 2025！
  • [1/2025] I’ll serve as an Area Chair of KDD 2025!
  • [1/2025] Zheng Li joined Shandong University as a full professor!
  • [1/2025] Zeyang Sha joined Ant Financial as a senior algorithmic engineer (Ant Star)!
  • [12/2024] I’ll serve as an Area Chair of ICML 2025!
  • [10/2024] Zeyang Sha has successfully passed his Ph.D. defense! Congratulations, Dr. Sha!
  • [10/2024] One paper “Towards Understanding Unsafe Video Generation” got accepted in NDSS 2025!
  • [9/2024] Our paper “DE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Generation Models” is in the best paper finalists of CSAW Europe 2024!
  • [9/2024] Xinyue Shen received the Abbe Grant from the Carl-Zeiss-Stiftung Foundation and is selected as a Young Researcher to attend the 11th Heidelberg Laureate Forum!
  • [9/2024] One paper “The Death and Life of Great Prompts: Analyzing the Evolution of LLM Prompts from the Structural Perspective” got accepted in EMNLP 2024!
  • [9/2024] One paper “ModScan: Measuring Stereotypical Bias in Large Vision-Language Models from Vision and Language Modalities” got accepted in EMNLP 2024!
  • [9/2024] One paper “Reconstruct Your Previous Conversations! Comprehensively Investigating Privacy Leakage Risks in Conversations with GPT Models” got accepted in EMNLP 2024!
  • [9/2024] Zheng Li won ERCIM WG STM Best Ph.D. Thesis Award 2024!
  • [8/2024] One paper “Understanding Data Importance in Machine Learning Attacks: Does Valuable Data Pose Greater Harm?” got accepted in NDSS 2025!
  • [8/2024] One paper “Membership Inference Attacks Against In-Context Learning” got accepted in CCS 2024!
  • [8/2024] One paper “Image-Perfect Imperfections: Safety, Bias, and Authenticity in the Shadow of Text-To-Image Model Evolution” got accepted in CCS 2024!
  • [8/2024] One paper “BadMerging: Backdoor Attacks Against Model Merging” got accepted in CCS 2024!
  • [8/2024] One paper “ZeroFake: Zero-Shot Detection of Fake Images Generated and Edited by Text-to-Image Generation Models” got accepted in CCS 2024!
  • [7/2024] One paper “SeqMIA: Sequential-Metric Based Membership Inference Attack” got accepted in CCS 2024!
  • [7/2024] We received an unrestricted gift from Google!
  • [5/2024] We released a technical report “Voice Jailbreak Attacks Against GPT-4o” on how to jailbreak GPT-4o with voice!
  • [5/2024] One paper “Instruction Backdoor Attacks Against Cutomized LLMs” got accepted in USENIX Security 2024!
  • [5/2024] One paper “MGTBench: Benchmarking Machine-Generated Text Detection” got accepted in CCS 2024!
  • [5/2024] We released SecurityNet, a large-scale dataset containing more than 1000 models for evaluating attacks and defenses in the field of trustworthy machine learning!