Yinpei Dai

Welcome! My name is Yinpei Dai, and I am a fourth-year Ph.D. student at Situated Language and Embodied Dialogue (SLED) Lab, University of Michigan. I'm super fortunate to work closely with Prof. Joyce Chai and Prof. Nima Fazeli. Before I started my PhD, I was a senior algorithm engineer at Alibaba DAMO Academy. I received my M.S. and B.S. degrees from Tsinghua University. My main interest is Embodied AI. I am especially interested in language-guided manipulation, interactive navigation, and human-robot dialogue systems.
X (Twitter) G. Scholar Github LinkedIn

Email: daiyp [at] umich [dot] edu

08/2025

🎯 Our papar AimBot is accepted to CoRL 2025! Let's equip your VLA models with scope reticles for more precise manipulation! Check out more details here.

01/2025

🚀 Our paper RACER is accepted by ICRA 2025. It's a framework to train visuomotor policies with rich language instructions and failure recovery behaviors.

09/2024

🚀 Our paper Language-Teachable Decision Transformer is accepted by EMNLP 2024! We trained RL agents that are more aligned with human instructions.

01/2024

🎉 Our papar ORION is accepted to ICRA 2024! It's an agentic system that enables human-robot interactive personalized navigation. Check out more details here.

09/2023

🎉 Our papar SpokenWoz is accepted by NeuIPS 2023. A large-scale speech-text benchmark for spoken task-oriented dialogue agents.

06/2023

🏆 We won 🥇 First Place ($500,000) in the first Amazon Alexa Prize SimBot Challenge! 🎉 Read our technical report here. (Media coverage: U-M, Amazon Science)

08/2022

I started my Ph.D. at University of Michigan!

06/2022

🎉 Our SPACE series on pre-trained dialogue models have been open-sourced! It's a model family including GALAXY (AAAI 2022), SPACE-2 (COLING 2022) and SPACE-3 (SIGIR 2022). Check out our code, here.

AimBot: A Simple Auxiliary Visual Cue to Enhance Spatial Awareness of Visuomotor Policies

Yinpei Dai*, Jayjun Lee*, Yichi Zhang, Ziqiao Ma, Jed Yang, Amir Zadeh, Chuan Li, Nima Fazeli, Joyce Chai
Conference on Robot Learning (CoRL) 2025
Webpage • PDF • Code

RACER: Rich Language-Guided Failure Recovery Policies for Imitation Learning

Yinpei Dai*, Jayjun Lee*, Nima Fazeli, Joyce Chai
★ Best Overall Award @ UM AI Symposium 2024 ★
The 3rd Workshop on Language and Robot Learning (LangRob) @ CoRL 2024
IEEE International Conference on Robotics and Automation (ICRA) 2025
Webpage • PDF • Code

Bootstrapping Visual Assistant Modeling with Situated Interaction Simulation

Yichi Zhang, Run Peng, Yinpei Dai, Lingyun Wu, Xuweiyi Chen, Qiaozi Gao, Joyce Chai
Conference on Language Modeling (CoLM) 2025
Webpage • PDF

Training Turn-by-Turn Verifiers for Dialogue Tutoring Agents: The Curious Case of LLMs as Your Coding Tutors

Jian Wang, Yinpei Dai, Yichi Zhang, Ziqiao Ma, Wenjie Li, Joyce Chai
Findings of the Association for Computational Linguistics (ACL) 2025
PDF • Code

Think, Act, and Ask: Open-World Interactive Personalized Robot Navigation

Yinpei Dai, Run Peng, Sikai Li, Joyce Chai
IEEE International Conference on Robotics and Automation (ICRA) 2024
PDF • Code • Video

Teaching Embodied Reinforcement Learning Agents: Informativeness and Diversity of Language Use

Jiajun Xi, Yinong He, Jianing Yang, Yinpei Dai, Joyce Chai
Empirical Methods in Natural Language Processing (EMNLP) 2024
PDF • Code

Toward satisfactory public accessibility: A crowdsourcing approach through online reviews to inclusive urban design

Lingyao Li, Songhua Hu, Yinpei Dai, Min Deng, et al.
Journal of Computers, Environment and Urban Systems, 2024
PDF

SEAGULL: An Embodied Agent for Instruction Following through Situated Dialog

Yichi Zhang, Jianing Yang, Keunwoo Yu, Yinpei Dai, Shane Storks, et al.
Alexa Prize SimBot Challenge Proceedings 2023
★ 🏆 World Champion ★
PDF • U-M CSE News • Amazon Science

SpokenWOZ: A Large-scale Speech-text Benchmark for Spoken Task-oriented Dialogue Agents

Shuzheng Si, Wentao Ma, Haoyu Gao, Yuchuan Wu, Ting-En Lin, Yinpei Dai, et al.
Neural Information Processing Systems (NeurIPS) 2023
PDF • Website

Task-Oriented Dialogue System as Natural Language Generation

Weizhi Wang, Zhirui Zhang, Junliang Guo, Yinpei Dai, Boxing Chen, et al.
International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) 2022
PDF

SPACE-3: Unified Dialog Model Pre-training for Task-Oriented Dialog Understanding and Generation

Wanwei He*, Yinpei Dai*, Min Yang, Jian Sun, et al.
International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) 2022
PDF • Code

CGoDial: A large-scale benchmark for chinese goal-oriented dialog evaluation

Yinpei Dai, Wanwei He, Bowen Li, et al.
Empirical Methods in Natural Language Processing (EMNLP) 2022
PDF • Data

SPACE-2: Tree-Structured Semi-Supervised Contrastive Pre-training for Task-Oriented Dialog Understanding

Wanwei He*, Yinpei Dai*, Binyuan Hui, et al.
International Conference on Computational Linguistics (COLING) 2022
PDF • Code

GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-Supervised Learning and Explicit Policy Injection

Wanwei He*, Yinpei Dai*, Yinhe Zheng, Yuchuan Wu, Zheng Cao, et al.
AAAI Conference on Artificial Intelligence (AAAI) 2022
PDF • Code

UIUC CSL Student Conference

2023

EMNLP Workshop on RoboNLP

Cornell Robotics Seminar

MILA Robot Learning Seminar

UPenn GRASP SFI Seminar

RSS Workshop on Symmetries in Robot Learning (video)

RSS Workshop on Generalizable Manipulation Policy Learning: Paradigms and Debates

CVPR Workshop on 3D Scene Understanding for Vision, Graphics, and Robotics

ICRA Workshop on Embracing Contact: Making Robots Physically Interact with our World

ICRA Round-Table Discussion with Science Robotics

ICRA Workshop on Life-Long Learning with Human Help

Panel on Northwestern Grand Challenges in Robotics (video)

Guest Lecture at Columbia COMS 4733: Computational Aspects of Robotics

Stanford Vision and Learning Lab

2022

CoRL Workshop on Language and Robot Learning (video)

NeurIPS Workshop on Robot Learning (video)

NeurIPS Workshop on Habitat Rearrangement Challenge

Guest Lecture at Columbia IEOR E6617: Machine Learning and High-Dimensional Data

Princeton Robotics Seminar

RSS Workshop on Implicit Representations (video)

CVPR Tutorial on Vision-Based Robot Learning

MIT CSL Seminar (video)

Guest Lecture at Columbia COMS 4733: Computational Aspects of Robotics

2021

CVPR Workshop on 3D Vision and Robotics

Guest Lecture at Stony Brook University CSE525: Introduction to Robotics

Guest Lecture at NYU CSCI-UA 74: Big Ideas in Artificial Intelligence

2020

RSS Workshop on Self-Supervised Robot Learning

RSS Workshop on Robotics Retrospectives (video)

2019

Amazon Research Robotics Symposium

2018

RE·WORK Deep Learning for Robotics Summit

Guest Lecture at NCTU Robotics Seminar: Robotic Manipulation

2025

Advanced Robotics (AR) Best Survey Paper Award

2024

IEEE Robotics and Automation (RAS) Early Career Award

Human-Robot Interaction (HRI) Best Paper Award (Technical Track)

IEEE International Conference on Intelligent Robots and Systems (IROS) RoboCup Best Paper Award

2023

Conference on Robot Learning (CoRL) Best Student Paper Award

Associate Editor, IEEE Robotics and Automation Letters (RA-L)

Robotics: Science and Systems (RSS) Best Demo Paper Award Finalist

Google AI blog post "Modular Visual Question Answering via Code Generation"

Demo, Robotics: Science and Systems (RSS) - Large Language Models on Robots (memories)

Co-Organizer, RSS Workshop on Articulate Robots

IEEE International Conference on Robotics and Automation (ICRA) Outstanding Learning Paper Award

Associate Editor, IEEE International Conference on Robotics and Automation (ICRA)

Demo, Google I/O - Code as Policies with PaLI and PaLM 2 (memories)

Google AI blog post "Visual Language Maps for Robot Navigation"

Co-Organizer, AAAI Tutorial on Everything You Need to Know about Transformers

Google Research PaLM-E debut - "An Embodied Multimodal Language Model" (20+ derived YouTube videos)

Google AI blog post "PaLM-E: An Embodied Multimodal Language Model"

2022

Conference on Robot Learning (CoRL) Special Innovation Award

Area Chair, Conference on Robot Learning (CoRL)

Co-Organizer, CoRL Workshop on Pre-training Robot Learning

Co-Organizer, CoRL Workshop on Inductive Bias in Robot Learning

Google AI blog post "Robots That Write Their Own Code"

Google AI@ Event Code as Policies debut (memories) - AXIOS, TechCrunch, ZDNet (60+ articles)

Session Chair, Robotics: Science and Systems (RSS)

Co-Organizer, 2nd RSS Workshop on Scaling Robot Learning

Sponsor, Google Research Scholar Program

Co-Organizer, CVPR Tutorial on Vision-Based Robot Learning

Session Chair, IEEE International Conference on Robotics and Automation (ICRA)

Co-Organizer, ICRA Workshop on Scaling Robot Learning

AXIOS article "Unleash All This Creativity: Google AI's Breathtaking Potential"

CNET video "Google’s Most Advanced Robot Brain Just Got a Body"

Google Research PaLM-SayCan debut - Wired, Washington Post, The Verge (260+ articles)

Google AI blog post "Robot See, Robot Do"

Princeton News article "In Picking Up Trash, Robots Pick Up New Approaches to Work"

2021

Conference on Robot Learning (CoRL) Best Paper Award Finalist

Google AI blog post "Decisiveness in Imitation Learning for Robots"

Session Chair, IEEE International Conference on Robotics and Automation (ICRA)

Co-Organizer, RSS Workshop on Advancing AI and Manipulation for Robotics

Area Chair, Conference on Robot Learning (CoRL)

Google AI blog post "Learning to Manipulate Deformable Objects"

Sponsor, Google Research Scholar Program

Japan Factory Automation (FA) Foundation Paper Award

Google AI blog post "Rearranging the Visual World"

2020

IEEE Transactions on Robotics (T-RO) Best Paper Award

Conference on Robot Learning (CoRL) Best Paper Presentation Award Finalist

Google AI blog post "Visual Transfer Learning for Robotic Manipulation"

IEEE International Conference on Robotics and Automation (ICRA) Best Paper in Automation Award Finalist

Google AI blog post "Learning to See Transparent Objects"

Mentor, Google CS Research Mentorship Program

Sponsor, Google Faculty Research Awards

2019

New York Times article "A New Lab Full of Fast Learners" (100+ articles)

Google AI blog post "Learning to Assemble and to Generalize"

Robotics: Science and Systems (RSS) Best System Paper Award

Google AI blog post "Unifying Physics and Deep Learning with TossingBot"

2018

Honored to be a recipient of the Princeton SEAS Award for Excellence

Demo, Google TGIF - TossingBot (memories)

IEEE International Conference on Intelligent Robots and Systems (IROS) Best Cognitive Paper Award Finalist

Amazon Robotics Best System Paper Award

Honored to be a recipient of the NVIDIA Fellowship

MIT News article "Robo-Picker Grasps and Packs"

2017

1st place winners (stow) at the worldwide Amazon Picking Challenge 2017 with Team MIT-Princeton

2016

3rd place winners at the worldwide Amazon Picking Challenge 2016 with Team MIT-Princeton

Co-Organizer, CVPR Workshop on 3D Deep Learning

2015

Honored to be a recipient of the Gordon Y.S. Wu Fellowship in Engineering and Wu Prize

2015+

Reviewer, T-RO, RSS, CoRL, IJRR, RA-L, ICRA, NeurIPS, CVPR, IROS, ECCV, ICCV

Reviewer, SIGGRAPH, PR Journal, Eurographics, TMM, TIP, CASE, TCSVT

Member, IEEE