Xuanlin (Simon) Li

I am a 3rd-year PhD student at UCSD CSE, advised by Prof. Hao Su (2021-). Previously I was an undergraduate majoring in Mathematics and Computer Science at UC Berkeley (2017-2021). I was also an undergraduate research assistant at Berkeley Artificial Intelligence Research, where I was advised by Prof. Trevor Darrell.

Resume (Nov.23) / GitHub / Google Scholar / LinkedIn / Twitter / Email

Research

I am primarily interested in Embodied AI, Vision-Language, and Robotics. My long-term goal is to build vision-language models and policies with universal, open-world (2D & 3D) perception and reasoning capabilities that can be efficiently and effectively deployed for real world applications. For example, by integrating these models into large-scale robotic learning systems, we empower robots to acquire generalizable skills and excel in diverse challenging tasks.

I'm also a major contributor of the SAPIEN Manipulation Skill Challenge (ManiSkill 1&2).

(* = equal contribution)

	Open X-Embodiment: Robotic Learning Datasets and RT-X Models Contributor In Submission paper / website /
	PartSLIP++: Enhancing Low-Shot 3D Part Segmentation via Multi-View Instance Segmentation and Maximum Likelihood Estimation Yuchen Zhou, Jiayuan Gu, Xuanlin Li , Minghua Liu, Yunhao Fang, Hao Su Preprint arxiv /
	Unleashing the Creative Mind: Language Model As Hierarchical Policy For Improved Exploration on Challenging Problem Solving Zhan Ling, Yunhao Fang, Xuanlin Li, Tongzhou Mu, Mingu Lee, Reza Pourreza, Roland Memisevic, Hao Su Preprint arxiv /
	Distilling Large Vision-Language Model with Out-of-Distribution Generalizability Xuanlin Li, Yunhao Fang, Minghua Liu, Zhan Ling, Zhuowen Tu, Hao Su International Conference on Computer Vision (ICCV) 2023 arxiv / code / poster /
	Deductive Verification of Chain-of-Thought Reasoning Zhan Ling, Yunhao Fang, Xuanlin Li, Zhiao Huang, Mingu Lee, Roland Memisevic, Hao Su Neural Information Processing Systems (NeurIPS) 2023 arxiv / code /
	OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understanding Minghua Liu, Ruoxi Shi, Kaiming Kuang, Yinhao Zhu, Xuanlin Li, Shizhong Han, Hong Cai, Fatih Porikli, Hao Su Neural Information Processing Systems (NeurIPS) 2023* arxiv / website / code /
	Situated Real-time Interaction with a Virtually Embodied Avatar Sunny Panchal, Guillaume Berger, Antoine Mercier, Cornelius Bohm, Florian Dietrichkeit, Xuanlin Li, Reza Pourreza, Pulkit Madan, Apratim Bhattacharyya, Mingu Lee, Mark Todorovich, Ingo Bax, Roland Memisevic CVPR 2023 Embodied AI Workshop (Preprint) paper /
	On the Efficacy of 3D Point Cloud Reinforcement Learning Zhan Ling, Yunchao Yao, Xuanlin Li, Hao Su Preprint arxiv / code /
	Reparameterized Policy Learning for Multimodal Trajectory Optimization Zhiao Huang, Litian Liang, Zhan Ling, Xuanlin Li, Chuang Gan, Hao Su International Conference on Machine Learning (ICML) 2023 (Oral) arxiv / website / code /
	Frame Mining: a Free Lunch for Learning Robotic Manipulation from 3D Point Clouds Xuanlin Li, Minghua Liu, Zhan Ling, Yangyan Li, Hao Su Conference on Robot Learning (CoRL) 2022* arxiv / website / video / code /
	ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills Jiayuan Gu†, Fanbo Xiang†, Xuanlin Li, Zhan Ling, Xiqiang Liu, Tongzhou Mu, Yihe Tang, Stone Tao, Xinyue Wei, Yunchao Yao, Xiaodi Yuan, Pengwei Xie, Zhiao Huang, Rui Chen, Hao Su ICLR 2023 arxiv / website / code / implementation /
	ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations Tongzhou Mu, Zhan Ling, Fanbo Xiang, Derek Yang, Xuanlin Li, Stone Tao, Zhiao Huang, Zhiwei Jia, Hao Su Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track, 2021* arxiv / website / video / code / implementation /
	Discovering Non-Monotonic Autoregressive Orderings with Variational Inference Xuanlin Li, Brandon Trabucco, Dong Huk Park, Yang Gao, Michael Luo, Sheng Shen, Trevor Darrell International Conference on Learning Representations (ICLR) 2021 arxiv / video_transcripts / code / poster / slides /
	Improving Policy Optimization with Generalist-Specialist Learning Zhiwei Jia, Xuanlin Li, Zhan Ling, Shuang Liu, Yiran Wu, Hao Su International Conference on Machine Learning (ICML) 2022 arxiv / website / code /
	Regularization Matters in Policy Optimization - An Empirical Study on Continuous Control Zhuang Liu, Xuanlin Li, Bingyi Kang, Trevor Darrell International Conference on Learning Representations (ICLR) 2021 (Spotlight) arxiv / video / code / poster / slides /

Other Projects

These include coursework, side projects and unpublished research work.

Inferring the Optimal Policy using Markov Chain Monte Carlo

Brandon Trabucco, Albert Qu, Xuanlin Li, Ganeshkumar Ashokavardhanan
Berkeley EECS 126 (Probability and Random Processes)
2018-12-10
arxiv /

Final course project for EECS 126 (Probability and Random Processes) in Fall 2018.

Experiences

Qualcomm AI Research, Research Intern, Mar. 2023 - Sep. 2023
Berkeley Artificial Intelligence Research, Undergraduate Research Assistant, 2019 - 2021

Services

Challenge Organizer:
- SAPIEN Manipulation Skill Challenge 1
- SAPIEN Manipulation Skill Challenge 2
Conference Reviewer:
- Computer Vision: CVPR 2022-2023, ECCV 2022, ICCV 2023
- Machine Learning: NeurIPS 2022-2023, ICML 2022-2023, ICLR 2022, 2024
- Robotics: ICRA 2024

Honors and Awards

Jacobs School of Engineering PhD Fellowship, UC San Diego CSE, 2021
Arthur M. Hopkin Award, UC Berkeley EECS, 2021
EECS Honors Program & Mathematics Honors Program, UC Berkeley

Design and source code from Jon Barron's website

Xuanlin (Simon) Li

Research

Open X-Embodiment: Robotic Learning Datasets and RT-X Models

PartSLIP++: Enhancing Low-Shot 3D Part Segmentation via Multi-View Instance Segmentation and Maximum Likelihood Estimation

Unleashing the Creative Mind: Language Model As Hierarchical Policy For Improved Exploration on Challenging Problem Solving

Distilling Large Vision-Language Model with Out-of-Distribution Generalizability

Deductive Verification of Chain-of-Thought Reasoning

OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understanding

Situated Real-time Interaction with a Virtually Embodied Avatar

On the Efficacy of 3D Point Cloud Reinforcement Learning

Reparameterized Policy Learning for Multimodal Trajectory Optimization

Frame Mining: a Free Lunch for Learning Robotic Manipulation from 3D Point Clouds

ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills

ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations

Discovering Non-Monotonic Autoregressive Orderings with Variational Inference

Improving Policy Optimization with Generalist-Specialist Learning

Regularization Matters in Policy Optimization - An Empirical Study on Continuous Control

Other Projects

Inferring the Optimal Policy using Markov Chain Monte Carlo

Experiences

Services

Honors and Awards