Puze Liu 刘普泽

Journal Articles

1. 

   2026 Nature Machine Intelligence

   A robot operating system framework for using large language models in embodied AI

   An open-source bridge between large language models and ROS turns natural-language goals into robust robot behaviors, while learning and refining new skills through feedback.

   该开源框架连接大语言模型与 ROS，将自然语言目标转化为可靠的机器人行为，并能通过反馈持续学习和优化新技能。

   Abstract摘要 Project DOI

   Autonomous robots capable of turning natural-language instructions into reliable physical actions remain a central challenge in artificial intelligence. Here we show that connecting a large language model agent to the robot operating system enables a versatile framework for embodied intelligence, and we release the complete implementation as freely available open-source code. The agent automatically translates large language model outputs into robot actions, supports interchangeable execution modes (inline code or behaviour trees), learns new atomic skills via imitation, and continually refines them through automated optimization and reflection from human or environmental feedback. Extensive experiments validate the framework, showcasing robustness, scalability and versatility in diverse scenarios and embodiments, including long-horizon tasks, tabletop rearrangements, dynamic task optimization and remote supervisory control. Moreover, all the results presented in this work were achieved by utilizing open-source pretrained large language models.

2. 

   2025 IEEE Transactions on Robotics (T-RO)

   Safe Reinforcement Learning on the Constraint Manifold: Theory and Applications

   A principled framework projects arbitrary reinforcement-learning actions onto a constraint manifold, enabling safe, high-dimensional learning directly on a real robot air-hockey platform.

   该框架将强化学习动作投影到约束流形的安全空间中，使机器人能够在真实空气曲棍球平台上完成安全的高维在线学习。

   Abstract摘要 arXiv PDF Project DOI

   Integrating learning-based techniques, especially reinforcement learning, into robotics is promising for solving complex problems in unstructured environments. However, most existing approaches are trained in well-tuned simulators and subsequently deployed on real robots without online fine-tuning. In this setting, the simulation’s realism seriously impacts the deployment’s success rate. Instead, learning with real-world interaction data offers a promising alternative: not only eliminates the need for a fine-tuned simulator but also applies to a broader range of tasks where accurate modeling is unfeasible. One major problem for on-robot reinforcement learning is ensuring safety, as uncontrolled exploration can cause catastrophic damage to the robot or the environment. Indeed, safety specifications, often represented as constraints, can be complex and non-linear, making safety challenging to guarantee in learning systems. In this paper, we show how we can impose complex safety constraints on learning-based robotics systems in a principled manner, both from theoretical and practical points of view. Our approach is based on the concept of the Constraint Manifold, representing the set of safe robot configurations. Exploiting differential geometry techniques, i.e., the tangent space, we can construct a safe action space, allowing learning agents to sample arbitrary actions while ensuring safety. We demonstrate the method’s effectiveness in a real-world Robot Air Hockey task, showing that our method can handle high-dimensional tasks with complex constraints. Videos of the real robot experiments are available on the project website.

3. 

   2025 IEEE Robotics and Automation Letters

   Adaptive control based friction estimation for tracking control of robot manipulators

   An adaptive controller estimates joint friction online and compensates for it during motion, improving manipulator tracking without relying on a fixed friction model.

   自适应控制器在运动过程中在线估计并补偿关节摩擦，无需固定摩擦模型即可提升机器人操作臂的轨迹跟踪精度。

   PDF DOI
4. 

   2024 IEEE Transactions on Robotics (T-RO)

   Fast Kinodynamic Planning on the Constraint Manifold With Deep Neural Networks

   A neural planner generates dynamically feasible motions on constrained manifolds at constant inference time, supporting rapid replanning in demanding robot air-hockey scenarios.

   神经规划器以固定推理时间生成满足动力学与流形约束的运动轨迹，可在高动态机器人空气曲棍球任务中快速重规划。

   Abstract摘要 PDF DOI

   Motion planning is a mature area of research in robotics with many well-established methods based on optimization or sampling the state space, suitable for solving kinematic motion planning. However, when dynamic motions under constraints are needed and computation time is limited, fast kinodynamic planning on the constraint manifold is indispensable. In recent years, learning-based solutions have become alternatives to classical approaches, but they still lack comprehensive handling of complex constraints, such as planning on a lower dimensional manifold of the task space while considering the robot’s dynamics. This article introduces a novel learning-to-plan framework that exploits the concept of constraint manifold, including dynamics, and neural planning methods. Our approach generates plans satisfying an arbitrary set of constraints and computes them in a short constant time, namely the inference time of a neural network. This allows the robot to plan and replan reactively, making our approach suitable for dynamic environments. We validate our approach on two simulated tasks and in a demanding real-world scenario, where we use a Kuka LBR Iiwa 14 robotic arm to perform the hitting movement in robotic air hockey.

Conference Papers

1. 

   2025 Conference on Robot Learning (CoRL)

   Morphologically Symmetric Reinforcement Learning for Ambidextrous Bimanual Manipulation

   Symmetry-aware reinforcement learning transfers skills across mirrored morphologies, allowing a bimanual robot to manipulate dexterously with either arm.

   利用形态对称性的强化学习可在镜像结构间迁移技能，使双臂机器人能够使用任一手臂完成灵巧操作。

   PDF Project
2. 

   2024 8th Annual Conference on Robot Learning

   Handling Long-Term Safety and Uncertainty in Safe Reinforcement Learning

   The method explicitly models long-term uncertainty and safety, helping reinforcement-learning agents remain reliable beyond short planning horizons.

   该方法显式建模长期不确定性与安全约束，使强化学习智能体在超越短期规划范围后仍能保持可靠。

   PDF
3. 

   2024 Proceedings of the 38th International Conference on Neural Information Processing Systems

   A Retrospective on the Robot Air Hockey Challenge: Benchmarking Robust, Reliable, and Safe Learning Techniques for Real-World Robotics

   A large-scale real-robot benchmark distills the practical lessons of the Robot Air Hockey Challenge, revealing when hybrid learning and prior knowledge outperform purely data-driven systems.

   该真实机器人基准总结了空气曲棍球挑战赛的实践经验，揭示了融合先验知识与学习的方法何时优于纯数据驱动系统。

   Abstract摘要 PDF

   Machine learning methods have a groundbreaking impact in many application domains, but their application on real robotic platforms is still limited. Despite the many challenges associated with combining machine learning technology with robotics, robot learning remains one of the most promising directions for enhancing the capabilities of robots. When deploying learning-based approaches on real robots, extra effort is required to address the challenges posed by various real-world factors. To investigate the key factors influencing real-world deployment and to encourage original solutions from different researchers, we organized the Robot Air Hockey Challenge at the NeurIPS 2023 conference. We selected the air hockey task as a benchmark, encompassing low-level robotics problems and high-level tactics. Different from other machine learning-centric benchmarks, participants need to tackle practical challenges in robotics, such as the sim-to-real gap, low-level control issues, safety problems, real-time requirements, and the limited availability of real-world data. Furthermore, we focus on a dynamic environment, removing the typical assumption of quasi-static motions of other real-world benchmarks. The competition’s results show that solutions combining learning-based approaches with prior knowledge outperform those relying solely on data when real-world deployment is challenging. Our ablation study reveals which real-world factors may be overlooked when building a learning-based solution. The successful real-world air hockey deployment of best-performing agents sets the foundation for future competitions and follow-up research directions.

4. 

   2023 Proceedings of the IEEE International Conference on Robotics and Automation (ICRA)

   Safe Reinforcement Learning of Dynamic High-Dimensional Robotic Tasks: Navigation, Manipulation, Interaction

   A unified safe-exploration formulation handles complex, learned collision constraints and transfers to real navigation, manipulation, and human–robot interaction tasks.

   统一的安全探索方法可处理由数据学习的复杂碰撞约束，并在真实机器人的导航、操作与人机交互任务中完成验证。

   Abstract摘要 arXiv BibTeX PDF

   Safety is a crucial property of every robotic platform: any control policy should always comply with actuator limits and avoid collisions with the environment and humans. In reinforcement learning, safety is even more fundamental for exploring an environment without causing any damage. While there are many proposed solutions to the safe exploration problem, only a few of them can deal with the complexity of the real world. This paper introduces a new formulation of safe exploration for reinforcement learning of various robotic tasks. Our approach applies to a wide class of robotic platforms and enforces safety even under complex collision constraints learned from data by exploring the tangent space of the constraint manifold. Our proposed approach achieves state-of-the-art performance in simulated high-dimensional and dynamic tasks while avoiding collisions with the environment. We show safe real-world deployment of our learned controller on a TIAGo++ robot, achieving remarkable performance in manipulation and human-robot interaction tasks.

   @inproceedings{ICRA_2022_ReDSDF_ATACOM,
     title = {Safe Reinforcement Learning of Dynamic High-Dimensional Robotic Tasks: Navigation, Manipulation, Interaction},
     author = {Liu, Puze and Zhang, Kuo and Tateo, Davide and Jauhri, Snehal and Hu, Zhiyuan and Peters, Jan and Chalvatzaki, Georgia},
     booktitle = {Proceedings of the IEEE International Conference on Robotics and Automation (ICRA)},
     publisher = {IEEE},
     year = {2023},
     image = {/assets/img/hri_real.gif},
   }

5. 

   2022 Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

   Regularized Deep Signed Distance Fields for Reactive Motion Generation

   ReDSDF learns smooth, multi-scale distance fields for articulated bodies, enabling fast collision reasoning and reactive motion around people in shared workspaces.

   ReDSDF 学习面向关节物体的平滑多尺度距离场，使机器人能够在共享空间中快速进行碰撞推理与人机协作运动。

   Abstract摘要 BibTeX PDF Code

   Autonomous robots should operate in real-world dynamic environments and collaborate with humans in tight spaces. A key component for allowing robots to leave structured lab and manufacturing settings is their ability to evaluate online and real-time collisions with the world around them. Distance-based constraints are fundamental for enabling robots to plan their actions and act safely, protecting both humans and their hardware. However, different applications require different distance resolutions, leading to various heuristic approaches for measuring distance fields w.r.t. obstacles, which are computationally expensive and hinder their application in dynamic obstacle avoidance use-cases. We propose Regularized Deep Signed Distance Fields (ReDSDF), a single neural implicit function that can compute smooth distance fields at any scale, with fine-grained resolution over high-dimensional manifolds and articulated bodies like humans, thanks to our effective data generation and a simple inductive bias during training. We demonstrate the effectiveness of our approach in representative simulated tasks for whole-body control (WBC) and safe Human-Robot Interaction (HRI) in shared workspaces. Finally, we provide proof of concept of a real-world application in a HRI handover task with a mobile manipulator robot.

   @inproceedings{IROS_2022_ReDSDF,
     title = {Regularized Deep Signed Distance Fields for Reactive Motion Generation},
     author = {Liu, Puze and Zhang, Kuo and Tateo, Davide and Jauhri, Snehal and Peters, Jan and Georgia, Chalvatzaki},
     booktitle = {Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
     publisher = {IEEE},
     year = {2022},
     image = {/assets/img/tiago_sdf.gif},
   }

6. 

   2022 Proceedings of the 5th Conference on Robot Learning (CoRL)

   Robot Reinforcement Learning on the Constraint Manifold

   Best Paper Award Finalist

   Constraint-manifold exploration embeds known robot models and safety limits into reinforcement learning, improving sample efficiency while guaranteeing safe exploration.

   约束流形探索将机器人模型与安全限制嵌入强化学习，在保证探索安全的同时提高了样本效率。

   Abstract摘要 BibTeX PDF Supp Code Website

   Reinforcement learning in robotics is extremely challenging due to many practical issues, including safety, mechanical constraints, and wear and tear. Typically, these issues are not considered in the machine learning literature. One crucial problem in applying reinforcement learning in the real world is Safe Exploration, which requires physical and safety constraints satisfaction throughout the learning process. To explore in such a safety-critical environment, leveraging known information such as robot models and constraints is beneficial to provide more robust safety guarantees. Exploiting this knowledge, we propose a novel method to learn robotics tasks in simulation efficiently while satisfying the constraints during the learning process.

   @inproceedings{CORL_2021_ATACOM,
     title = {Robot Reinforcement Learning on the Constraint Manifold},
     author = {Liu, Puze and Tateo, Davide and Bou-Ammar, Haitham and Peters, Jan},
     booktitle = {Proceedings of the 5th Conference on Robot Learning (CoRL)},
     pages = {1357--1366},
     year = {2022},
     editor = {Faust, Aleksandra and Hsu, David and Neumann, Gerhard},
     volume = {164},
     series = {Proceedings of Machine Learning Research},
     publisher = {PMLR},
     comments = {Best Paper Award Finalist},
     image = {/assets/img/manifold.gif},
   }

7. 

   2021 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

   Efficient and Reactive Planning for High Speed Robot Air Hockey

   Best Entertainment and Amusement Paper Award Finalist

   A fast, reactive planning stack pushes general-purpose robot arms to execute precise hits and competitive play in a highly dynamic air-hockey environment.

   快速反应式规划系统使通用机器人操作臂能够在高动态空气曲棍球环境中完成精准击球与对抗。

   Abstract摘要 BibTeX HTML PDF DOI

   Highly dynamic robotic tasks require high-speed and reactive robots. These tasks are particularly challenging due to the physical constraints, hardware limitations, and the high uncertainty of dynamics and sensor measures. To face these issues, it’s crucial to design robotics agents that generate precise and fast trajectories and react immediately to environmental changes. Air hockey is an example of this kind of task. Due to the environment’s characteristics, it is possible to formalize the problem and derive clean mathematical solutions. For these reasons, this environment is perfect for pushing to the limit the performance of currently available general-purpose robotic manipulators. Using two Kuka Iiwa 14, we show how to design a policy for general-purpose robotic manipulators for the air hockey game. We demonstrate that a real robot arm can perform fast-hitting movements and that the two robots can play against each other on a medium-size air hockey table in simulation.

   @inproceedings{IROS_2021_Air_Hockey,
     title = {Efficient and Reactive Planning for High Speed Robot Air Hockey},
     author = {Liu, Puze and Tateo, Davide and Bou-Ammar, Haitham and Peters, Jan},
     year = {2021},
     booktitle = {2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
     pages = {586-593},
     doi = {10.1109/IROS51168.2021.9636263},
     comments = {Best Entertainment and Amusement Paper Award Finalist},
     image = {/assets/img/air_hockey_iros.jpg},
   }