Robot Learning Project

Jan 1, 2018

Robot Learning project is focused on training a robot with VR or MR approaches.

Autonomous Robot Machine Learning

Zhenliang Zhang

Research Scientist of AI

My research interests include wearable computing, machine learning, Cognitive Reasoning, and mixed/virtual reality.

Publications

[ICLR 2024] Bongard-OpenWorld: Few-Shot Reasoning for Free-form Visual Concepts in the Real World

In this paper, We introduce Bongard-OpenWorld, a new benchmark for evaluating real-world few-shot reasoning for machine vision.

Rujie Wu, Xiaojian Ma, Zhenliang Zhang, Wei Wang, Qing Li, Song-Chun Zhu, Yizhou Wang

[ICLR 2024] Bongard-OpenWorld: Few-Shot Reasoning for Free-form Visual Concepts in the Real World

[Engineering 2023] The tong test: Evaluating artificial general intelligence through dynamic embodied physical and social interactions

The Tong test describes a value- and ability-oriented testing system that delineates five levels of AGI milestones through a virtual environment with DEPSI, allowing for infinite task generation.

Yujia Peng, Jiaheng Han, Zhenliang Zhang, Lifeng Fan, Tengyu Liu, Siyuan Qi, Xue Feng, Yuxi Ma, Yizhou Wang, Song-Chun Zhu

[Engineering 2023] The tong test: Evaluating artificial general intelligence through dynamic embodied physical and social interactions

[Engineering 2023] A reconfigurable data glove for reconstructing physical and virtual grasps

We present a reconfigurable data glove design to capture different modes of human hand-object interactions, which are critical in training embodied artificial intelligence (AI) agents for fine manipulation tasks.

Hangxin Liu, Zeyu Zhang, Ziyuan Jiao, Zhenliang Zhang, Mingchen Li, Chenfanfu Jiang, Yixin Zhu, Song-Chun Zhu

[Engineering 2023] A reconfigurable data glove for reconstructing physical and virtual grasps

[IROS 2020] Graph-based hierarchical knowledge representation for robot task transfer from virtual to physical world

We study the hierarchical knowledge transfer problem using a cloth-folding task, wherein the agent is first given a set of human demonstrations in the virtual world using an Oculus Headset, and later transferred and validated on a physical Baxter robot.

Zhenliang Zhang, Yixin Zhu, Song-Chun Zhu

[IROS 2020] Graph-based hierarchical knowledge representation for robot task transfer from virtual to physical world

[VR 2020] Extracting and transferring hierarchical knowledge to robots using virtual reality

We study the knowledge transfer problem by training the task of folding clothes in the virtual world using an Oculus Headset and …

Zhenliang Zhang, Jie Guo, Dongdong Weng, Yue Liu, Yongtian Wang

[VR 2020] Extracting and transferring hierarchical knowledge to robots using virtual reality

[ICRA 2019] High-fidelity grasping in virtual reality using a glove-based system

This paper presents a design that jointly provides hand pose sensing, hand localization, and haptic feedback to facilitate real-time …

Hangxin Liu, Zhenliang Zhang, Xu Xie, Yixin Zhu, Yue Liu, Yongtian Wang, Song-Chun Zhu

[ICRA 2019] High-fidelity grasping in virtual reality using a glove-based system

[TURC 2019] VRGym: A virtual testbed for physical and interactive AI

We propose VRGym, a virtual reality testbed for realistic human-robot interaction. Different from existing toolkits and virtual reality environments, the VRGym emphasizes on building and training both physical and interactive agents for robotics, machine learning, and cognitive science.

Xu Xie, Hangxin Liu, Zhenliang Zhang, Yuxing Qiu, Feng Gao, Siyuan Qi, Yixin Zhu, Song-Chun Zhu

[TURC 2019] VRGym: A virtual testbed for physical and interactive AI