Xuanlin (Simon) Li

I am a 4th-year PhD student at UCSD CSE, advised by Prof. Hao Su (2021-). Previously I was an undergraduate majoring in Mathematics and Computer Science at UC Berkeley (2017-2021). I was also an undergraduate research assistant at Berkeley Artificial Intelligence Research, where I was advised by Prof. Trevor Darrell.

Resume (Oct'24)  /  GitHub  /  Google Scholar  /  LinkedIn  /  Twitter  /  Email

profile photo

Research

I am primarily interested in Embodied AI, Vision-Language, and Robotics. My major research interests include (1) building vision-language models and robotic agents with universal, open-world (2D & 3D) perception and reasoning capabilities that can be efficiently and effectively deployed for real world applications; (2) scaling up training data, learning-from-demonstration algorithms, and benchmarks for generalizable and robust robotic manipulation in the real world.

I have been a major contributor of the SAPIEN Manipulation Skill Challenge (ManiSkill). I've also lead the benchmark on evaluating real-world generalist robot manipulation policies in simulation (Simpler-Env).

(* = equal contribution)
project image

Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation


Xuanlin Li, Tong Zhao, Xinghao Zhu, Jiuguang Wang, Tao Pang, Kuan Fang
Preprint
website /

project image

Evaluating Real-World Robot Manipulation Policies in Simulation


Xuanlin Li*, Kyle Hsu*, Jiayuan Gu*, Karl Pertsch^, Oier Mees^, Homer Rich Walke, Chuyuan Fu, Ishikaa Lunawat, Isabel Sieh, Sean Kirmani, Sergey Levine, Jiajun Wu, Chelsea Finn, Hao Su^^, Quan Vuong^^, Ted Xiao^^
Conference on Robot Learning (CoRL) 2024
paper / website / code /

project image

Open X-Embodiment: Robotic Learning Datasets and RT-X Models


Contributor & Author
IEEE International Conference on Robotics and Automation (ICRA) 2024
paper / website /

project image

PartSLIP++: Enhancing Low-Shot 3D Part Segmentation via Multi-View Instance Segmentation and Maximum Likelihood Estimation


Yuchen Zhou*, Jiayuan Gu*, Xuanlin Li , Minghua Liu, Yunhao Fang, Hao Su
Preprint
arxiv /

project image

Unleashing the Creative Mind: Language Model As Hierarchical Policy For Improved Exploration on Challenging Problem Solving


Zhan Ling, Yunhao Fang, Xuanlin Li, Tongzhou Mu, Mingu Lee, Reza Pourreza, Roland Memisevic, Hao Su
Preprint
arxiv /

project image

Distilling Large Vision-Language Model with Out-of-Distribution Generalizability


Xuanlin Li*, Yunhao Fang*, Minghua Liu, Zhan Ling, Zhuowen Tu, Hao Su
International Conference on Computer Vision (ICCV) 2023
arxiv / code / poster /

project image

Deductive Verification of Chain-of-Thought Reasoning


Zhan Ling*, Yunhao Fang*, Xuanlin Li, Zhiao Huang, Mingu Lee, Roland Memisevic, Hao Su
Neural Information Processing Systems (NeurIPS) 2023
arxiv / code /

project image

OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understanding


Minghua Liu*, Ruoxi Shi*, Kaiming Kuang*, Yinhao Zhu, Xuanlin Li, Shizhong Han, Hong Cai, Fatih Porikli, Hao Su
Neural Information Processing Systems (NeurIPS) 2023
arxiv / website / code /

project image

Live Fitness Coaching as a Testbed for Situated Interaction


Sunny Panchal, Apratim Bhattacharyya, Guillaume Berger, Antoine Mercier, Cornelius Bohm, Florian Dietrichkeit, Reza Pourreza, Xuanlin Li, Pulkit Madan, Mingu Lee, Mark Todorovich, Ingo Bax, Roland Memisevic
Neural Information Processing Systems (NeurIPS) 2024
paper /

project image

On the Efficacy of 3D Point Cloud Reinforcement Learning


Zhan Ling*, Yunchao Yao*, Xuanlin Li, Hao Su
Preprint
arxiv / code /

project image

Reparameterized Policy Learning for Multimodal Trajectory Optimization


Zhiao Huang, Litian Liang, Zhan Ling, Xuanlin Li, Chuang Gan, Hao Su
International Conference on Machine Learning (ICML) 2023 (Oral)
arxiv / website / code /

project image

Frame Mining: a Free Lunch for Learning Robotic Manipulation from 3D Point Clouds


Xuanlin Li*, Minghua Liu*, Zhan Ling*, Yangyan Li, Hao Su
Conference on Robot Learning (CoRL) 2022
arxiv / website / video / code /

project image

ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills


Jiayuan Gu†, Fanbo Xiang†, Xuanlin Li*, Zhan Ling*, Xiqiang Liu*, Tongzhou Mu*, Yihe Tang*, Stone Tao*, Xinyue Wei*, Yunchao Yao*, Xiaodi Yuan, Pengwei Xie, Zhiao Huang, Rui Chen, Hao Su
ICLR 2023
arxiv / website / code / implementation /

project image

ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations


Tongzhou Mu*, Zhan Ling*, Fanbo Xiang*, Derek Yang*, Xuanlin Li*, Stone Tao, Zhiao Huang, Zhiwei Jia, Hao Su
Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track, 2021
arxiv / website / video / code / implementation /

project image

Discovering Non-Monotonic Autoregressive Orderings with Variational Inference


Xuanlin Li*, Brandon Trabucco*, Dong Huk Park, Yang Gao, Michael Luo, Sheng Shen, Trevor Darrell
International Conference on Learning Representations (ICLR) 2021
arxiv / video_transcripts / code / poster / slides /

project image

Improving Policy Optimization with Generalist-Specialist Learning


Zhiwei Jia, Xuanlin Li, Zhan Ling, Shuang Liu, Yiran Wu, Hao Su
International Conference on Machine Learning (ICML) 2022
arxiv / website / code /

project image

Regularization Matters in Policy Optimization - An Empirical Study on Continuous Control


Zhuang Liu*, Xuanlin Li*, Bingyi Kang, Trevor Darrell
International Conference on Learning Representations (ICLR) 2021 (Spotlight)
arxiv / video / code / poster / slides /




Other Projects

These include coursework, side projects and unpublished research work.

project image

Inferring the Optimal Policy using Markov Chain Monte Carlo


Brandon Trabucco, Albert Qu, Xuanlin Li, Ganeshkumar Ashokavardhanan
Berkeley EECS 126 (Probability and Random Processes)
2018-12-10
arxiv /

Final course project for EECS 126 (Probability and Random Processes) in Fall 2018.

Experiences

Hillbot.ai, Research Intern, Feb. 2024 - Jun 2024, Sep. 2024 - Now
Boston Dynamics AI Institute, Research Intern, Jun. 2024 - Sep. 2024
Qualcomm AI Research, Research Intern, Mar. 2023 - Sep. 2023
Berkeley Artificial Intelligence Research, Undergraduate Research Assistant, 2019 - 2021

Services

Honors and Awards

Jacobs School of Engineering PhD Fellowship, UC San Diego CSE, 2021
Arthur M. Hopkin Award, UC Berkeley EECS, 2021
EECS Honors Program & Mathematics Honors Program, UC Berkeley


Design and source code from Jon Barron's website