Xuanlin (Simon) Li

I am a 3rd-year PhD student at UCSD CSE, advised by Prof. Hao Su (2021-). Previously I was an undergraduate majoring in Mathematics and Computer Science at UC Berkeley (2017-2021). I was also an undergraduate research assistant at Berkeley Artificial Intelligence Research, where I was advised by Prof. Trevor Darrell.

Resume (Nov.23)  /  GitHub  /  Google Scholar  /  LinkedIn  /  Twitter  /  Email

profile photo


I am primarily interested in Embodied AI, Vision-Language, and Robotics. My long-term goal is to build vision-language models and policies with universal, open-world (2D & 3D) perception and reasoning capabilities that can be efficiently and effectively deployed for real world applications. For example, by integrating these models into large-scale robotic learning systems, we empower robots to acquire generalizable skills and excel in diverse challenging tasks.

I'm also a major contributor of the SAPIEN Manipulation Skill Challenge (ManiSkill).

(* = equal contribution)
project image

Evaluating Real-World Robot Manipulation Policies in Simulation

Xuanlin Li*, Kyle Hsu*, Jiayuan Gu*, Karl Pertsch^, Oier Mees^, Homer Rich Walke, Chuyuan Fu, Ishikaa Lunawat, Isabel Sieh, Sean Kirmani, Sergey Levine, Jiajun Wu, Chelsea Finn, Hao Su^^, Quan Vuong^^, Ted Xiao^^
paper / website / code /

project image

Open X-Embodiment: Robotic Learning Datasets and RT-X Models

ICRA 2024
paper / website /

project image

PartSLIP++: Enhancing Low-Shot 3D Part Segmentation via Multi-View Instance Segmentation and Maximum Likelihood Estimation

Yuchen Zhou*, Jiayuan Gu*, Xuanlin Li , Minghua Liu, Yunhao Fang, Hao Su
arxiv /

project image

Unleashing the Creative Mind: Language Model As Hierarchical Policy For Improved Exploration on Challenging Problem Solving

Zhan Ling, Yunhao Fang, Xuanlin Li, Tongzhou Mu, Mingu Lee, Reza Pourreza, Roland Memisevic, Hao Su
arxiv /

project image

Distilling Large Vision-Language Model with Out-of-Distribution Generalizability

Xuanlin Li*, Yunhao Fang*, Minghua Liu, Zhan Ling, Zhuowen Tu, Hao Su
International Conference on Computer Vision (ICCV) 2023
arxiv / code / poster /

project image

Deductive Verification of Chain-of-Thought Reasoning

Zhan Ling*, Yunhao Fang*, Xuanlin Li, Zhiao Huang, Mingu Lee, Roland Memisevic, Hao Su
Neural Information Processing Systems (NeurIPS) 2023
arxiv / code /

project image

OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understanding

Minghua Liu*, Ruoxi Shi*, Kaiming Kuang*, Yinhao Zhu, Xuanlin Li, Shizhong Han, Hong Cai, Fatih Porikli, Hao Su
Neural Information Processing Systems (NeurIPS) 2023
arxiv / website / code /

project image

Situated Real-time Interaction with a Virtually Embodied Avatar

Sunny Panchal, Guillaume Berger, Antoine Mercier, Cornelius Bohm, Florian Dietrichkeit, Xuanlin Li, Reza Pourreza, Pulkit Madan, Apratim Bhattacharyya, Mingu Lee, Mark Todorovich, Ingo Bax, Roland Memisevic
CVPR 2023 Embodied AI Workshop (Preprint)
paper /

project image

On the Efficacy of 3D Point Cloud Reinforcement Learning

Zhan Ling*, Yunchao Yao*, Xuanlin Li, Hao Su
arxiv / code /

project image

Reparameterized Policy Learning for Multimodal Trajectory Optimization

Zhiao Huang, Litian Liang, Zhan Ling, Xuanlin Li, Chuang Gan, Hao Su
International Conference on Machine Learning (ICML) 2023 (Oral)
arxiv / website / code /

project image

Frame Mining: a Free Lunch for Learning Robotic Manipulation from 3D Point Clouds

Xuanlin Li*, Minghua Liu*, Zhan Ling*, Yangyan Li, Hao Su
Conference on Robot Learning (CoRL) 2022
arxiv / website / video / code /

project image

ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills

Jiayuan Gu†, Fanbo Xiang†, Xuanlin Li*, Zhan Ling*, Xiqiang Liu*, Tongzhou Mu*, Yihe Tang*, Stone Tao*, Xinyue Wei*, Yunchao Yao*, Xiaodi Yuan, Pengwei Xie, Zhiao Huang, Rui Chen, Hao Su
ICLR 2023
arxiv / website / code / implementation /

project image

ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations

Tongzhou Mu*, Zhan Ling*, Fanbo Xiang*, Derek Yang*, Xuanlin Li*, Stone Tao, Zhiao Huang, Zhiwei Jia, Hao Su
Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track, 2021
arxiv / website / video / code / implementation /

project image

Discovering Non-Monotonic Autoregressive Orderings with Variational Inference

Xuanlin Li*, Brandon Trabucco*, Dong Huk Park, Yang Gao, Michael Luo, Sheng Shen, Trevor Darrell
International Conference on Learning Representations (ICLR) 2021
arxiv / video_transcripts / code / poster / slides /

project image

Improving Policy Optimization with Generalist-Specialist Learning

Zhiwei Jia, Xuanlin Li, Zhan Ling, Shuang Liu, Yiran Wu, Hao Su
International Conference on Machine Learning (ICML) 2022
arxiv / website / code /

project image

Regularization Matters in Policy Optimization - An Empirical Study on Continuous Control

Zhuang Liu*, Xuanlin Li*, Bingyi Kang, Trevor Darrell
International Conference on Learning Representations (ICLR) 2021 (Spotlight)
arxiv / video / code / poster / slides /

Other Projects

These include coursework, side projects and unpublished research work.

project image

Inferring the Optimal Policy using Markov Chain Monte Carlo

Brandon Trabucco, Albert Qu, Xuanlin Li, Ganeshkumar Ashokavardhanan
Berkeley EECS 126 (Probability and Random Processes)
arxiv /

Final course project for EECS 126 (Probability and Random Processes) in Fall 2018.


Qualcomm AI Research, Research Intern, Mar. 2023 - Sep. 2023
Berkeley Artificial Intelligence Research, Undergraduate Research Assistant, 2019 - 2021


Honors and Awards

Jacobs School of Engineering PhD Fellowship, UC San Diego CSE, 2021
Arthur M. Hopkin Award, UC Berkeley EECS, 2021
EECS Honors Program & Mathematics Honors Program, UC Berkeley

Design and source code from Jon Barron's website