Gairdao Community

英文摘要：In robot learning, scaling training datasets across diverse embodiments and environments has become a dominant paradigm for learning generalizable robot policies. These policies are commonly trained via behavior cloning to imitate actions from pre-collected demonstrations. However, since robot actions are tied to the dynamics of the data collection robot, different robots may require different actions to achieve the same motion. This discrepancy hinders both policy training and deployment across diverse robots. To address this, we propose using Cartesian state delta as a universal action representation across robots, and introduce State Prediction and Adaptive Command Execution (SPACE) framework. SPACE handles robot dynamics variation at three levels: across different embodiments, across hardware units of the same embodiment, and within a single robot during operation. It consists of two components: (i) a Cartesian state delta policy that predicts geometric end-effector displacement, and (ii) Action Adapter, which converts the predicted Cartesian state delta into robot-specific control commands. Experiments show that SPACE substantially outperforms policies that directly predict control commands when learning from data collected across different embodiments and across hardware units of the same embodiment. SPACE also remains robust under dynamics shifts at deployment, including changes in control frequency, object weight, and controller gains. The project page is available at this http URL.

2. Beyond Monotonic Progress: Retry-Supervised Value Learning for Robot Imitation

超越单调进展：面向机器人模仿的重试监督值学习

AI 总结：针对机器人模仿学习中演示数据包含错误与修正行为的问题，提出ReTVL框架，利用重试事件作为稀疏监督学习对错误敏感的值函数，通过全局进度校准与局部偏好学习捕捉错误-恢复结构，优化下游行为克隆。

链接：https://arxiv\.org/abs/2606\.24633

作者：Xinyao Qin, Junjie Lu, Kaixin Wang, Chuheng Zhang, Sinjae Kang, Kimin Lee, Min Xu, Bin Liang, Jun Yang, Li Zhao

英文摘要：Human demonstrations for robot imitation learning often contain mistakes and corrective behaviors, such as imprecise grasps, object misalignment, unstable contact, and repeated attempts. While these segments are commonly treated as noisy or suboptimal data, they provide valuable evidence about when execution deviates from a desirable path and how task feasibility can be restored. However, existing reward and value models often rely on monotonic progress assumptions, which capture coarse task advancement but may overlook local execution errors and corrective behaviors in imperfect demonstrations. In this work, we propose ReTVL (ReTry-Supervised Value Learning), a framework for learning mistake-sensitive value functions from mixed-quality robot demonstrations by leveraging retry events as sparse supervision. ReTVL captures the local degradation-and-recovery structure around mistakes by combining global progress calibration with local pairwise preference learning induced by sparsely annotated retry keypoints. The learned value model is then used to reweight demonstration chunks for downstream behavior cloning, reducing the influence of harmful execution errors while preserving useful corrective behaviors. Experiments on real-robot manipulation tasks show that ReTVL produces more fine-grained value estimates than progress-based baselines and improves imitation learning from imperfect demonstrations.

3. World Value Models for Robotic Manipulation

机器人操作的世界价值模型

AI 总结：提出世界价值模型（WVM），融合世界模型与价值估计，通过时序建模评估数据质量，在标准基准和次优轨迹基准上达到SOTA，提升策略学习性能。

链接：https://arxiv\.org/abs/2606\.24742

机构：Peking University（北京大学）； Tsinghua University（清华大学）

作者：Zhihao Wang, Jianxiong Li, Yu Cui, Yuan Gao, Xianyuan Zhan, Junzhi Yu, Xiao Ma

英文摘要：Generalist value models play a pivotal role in scaling robotic policy learning from large-scale, mixed-quality data. Mathematically, accurate value estimation demands deep temporal understanding, requiring models to both ground the current belief using historical context and plan over future outcomes. However, most existing robotic value models are built on Vision-Language Model (VLM) backbones that are pretrained primarily on static or temporally sparse visual observations, lacking the requisite temporal modeling capabilities for value estimation. Unlike VLMs, world models naturally excel at temporal modeling and future planning, making them ideal foundations for learning generalizable value functions. Driven by this insight, we marry world models with value estimation to construct a new generalist robotic value model, World Value Model (WVM), that offers accurate task progressions to assess data quality. On standard benchmarks, WVM delivers state-of-the-art (SOTA) Value-Order Correlation (VOC) results. Complementing standard evaluation suites that contains only expert data, we further introduce Suboptimal-Value-Bench, a multi-embodiment benchmark consisting of 800 suboptimal trajectories with high-fidelity, human-labeled frame annotations. Our evaluations show that WVM maintains its SOTA performance on Suboptimal-Value-Bench, establishing its robustness in handling both expert and suboptimal data. When deployed for policy learning, WVM improves manipulation performance across various policy extraction approaches in both simulated and real-world deployment, providing robust guidance for learning from mixed-quality data.

2. 运动规划、控制与动力学 | 5 篇

4. TurboMPC: Fast, Scalable, and Differentiable Model Predictive Control on the GPU

TurboMPC：GPU上快速、可扩展且可微分的模型预测控制

AI 总结：提出TurboMPC，一种完全运行在GPU上的可微分MPC求解器，通过SQP、ADMM内求解器、隐式微分和JAX-CUDA协同设计，在约束规划、人形模仿学习和强化学习中实现高达15倍和58倍的加速，并在实车最小时间竞赛中通过贝叶斯优化调参显著提升性能。

链接：https://arxiv\.org/abs/2606\.24039

机构：Dartmouth College（达特茅斯学院）； Toyota Research Institute（丰田研究所）

作者：Gabriel Bravo-Palacios, Jianghan Zhang, Zachary Pestrikov, Brian Plancher, Thomas Lew

英文摘要：Robotics increasingly relies on GPUs for parallel simulation, large-scale learning, and neural-network inference. For model predictive control (MPC) to scale with this paradigm, solvers must run efficiently on this hardware while remaining fast, differentiable, and compatible with expressive MPC formulations used in robotics. We present TurboMPC, a differentiable MPC solver that runs entirely on the GPU and supports state and control inequality constraints, implicit integrators, cross-time-coupled costs, and slack variables. TurboMPC combines sequential quadratic programming (SQP), an alternating direction method of multipliers (ADMM) inner solver, implicit differentiation, and a co-designed JAX-CUDA implementation for efficiency and ease of use. In simulation, we validate TurboMPC on constrained planning, humanoid imitation learning, and reinforcement learning with neural-network cost function tasks, achieving up to $15\times$ and $58\times$ speedups over state-of-the-art CPU and GPU differentiable solvers, respectively. We deploy TurboMPC on a full-scale car for minimum-time racing and find that batched, GPU-accelerated tuning of MPC parameters via Bayesian optimization yields significantly faster driving than a hand-tuned baseline. TurboMPC also scales to planning horizons of over $8000$ knot points while maintaining control of the vehicle. We open-source TurboMPC at: this https URL

5. DynaWM: Dynamics-Aware Distillation with World Model and Momentum Targets for Smooth Locomotion over Continuous Stairs

DynaWM: 基于世界模型和动量目标的动力学感知蒸馏实现连续楼梯上的平滑运动

AI 总结：提出DynaWM框架，通过世界模型正则化增强地形编码，并利用动量目标编码器稳定知识蒸馏，使双足轮式机器人在连续楼梯上实现高适应性和平滑运动。

链接：https://arxiv\.org/abs/2606\.24089

机构：School of Mechatronical Engineering, Beijing Institute of Technology（北京理工大学机电学院）

作者：Haidong Hou, Zhangguo Yu, Hengbo Qi, Jianlin Zhang

英文摘要：Recent advances in control have enabled bipedal-wheeled robots to traverse slopes and single-step obstacles, yet long staircase traversal remains challenging as current teacher-student frameworks suffer from weakened dynamics-aware representations and incomplete terrain geometry encoding. To bridge this gap, we propose DynaWM, a dynamics-aware representation learning framework. To enhance terrain encoding capability and enable transparent assessment, we introduce a world model as a regularizer to enforce forward-dynamics awareness, preserving comprehensive terrain geometry while facilitating hierarchical encoding visualization. To stabilize knowledge transfer, we employ a momentum target encoder to provide consistent distillation targets, preventing dimensional collapse from non-stationary teacher updates. Evaluation of the learned representations through Principal Component Analysis (PCA) visualization and quantitative metrics reveals that our encoder hierarchically captures terrain geometry with higher terrain encoding capability, leading to enhanced terrain adaptability and motion smoothness. Experimental results in simulation and real hardware demonstrate that our method achieves superior terrain adaptability and motion smoothness, enabling bipedal-wheeled robots to overcome diverse continuous stairs, as shown in Fig. 1.

6. Grounding Generative Policies in Physics: Optimization-Guided Diffusion for Robot Control

将生成式策略基于物理：面向机器人控制的优化引导扩散

AI 总结：提出推理时优化框架，通过将扩散引导建模为约束优化问题，在采样中施加物理可行性约束，无需重训练即可提升机器人任务成功率。

链接：https://arxiv\.org/abs/2606\.24208

机构：ETH Zurich（苏黎世联邦理工学院）； EPFL（洛桑联邦理工学院）

作者：Sabrina Bodmer, René Zurbrügg, Tifanny Portela, Hao Ma, Alexandre Didier, Marco Hutter, Colin Jones, Melanie Zeilinger

英文摘要：Diffusion models sample effectively from high-dimensional, multimodal distributions, but their outputs may violate deployment constraints. For task-space robot policies, generated grasps, waypoints, or trajectories can be distributionally valid yet infeasible, violating reachability, collision-avoidance, or closed-loop executability requirements. This embodiment gap limits zero-shot deployment across robots, even when the task-space behavior itself is transferable. We propose an inference-time optimization framework that couples the behavior generation to physical feasibility by formulating diffusion guidance as a constrained optimization problem. Our key insight is to replace the sampling perturbation in the backward process with an optimized correction, allowing hard constraints or soft penalties to be imposed during sampling without the need to retrain the diffusion model, while keeping samples close to the learned prior. We evaluate the method on dexterous grasp synthesis with reachability and collision-avoidance constraints, and dynamic manipulation with controller-level trackability constraints. Across settings and robot embodiments, optimization-guided denoising matches the feasibility of projection- and gradient-guidance baselines while better preserving grasp quality, and improving controller-level executability and task success, with task success improving by up to 20pp. on dexterous grasping and 23pp. on visuomotor manipulation over the best baseline.

7. FT-WBC: Learning Fault-Tolerant Whole-Body Control for Legged Loco-Manipulation

FT-WBC：学习腿式操作任务的容错全身控制

AI 总结：针对腿式机械臂在致动器故障下易失稳的问题，提出FT-WBC框架，通过故障估计器和姿态适应模块实现容错全身控制，提升生存率和操作空间。

链接：https://arxiv\.org/abs/2606\.24466

机构：The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））； The University of Hong Kong（香港大学）

作者：Yudong Zhong, Pengfei Mai, Sikai Guo, Jiahang Cao, Zhihai Bi, Qiuyue Liu, Ziyan Feng, Jinni Zhou, Jun Ma

英文摘要：Legged manipulators combine the mobility of legged platforms with the manipulation capability of robotic arms. However, arm-induced Center-of-Mass shifts and dynamic disturbances make the system more prone to instability under actuator failures, potentially leading to falls, task failures, or safety risks. Existing fault-tolerant control methods mainly focus on locomotion alone, leaving the coupled problem of whole-body stability and arm reachability in fault-tolerant loco-manipulation largely unaddressed. To bridge this gap, we propose FT-WBC, a fault-tolerant loco-manipulation framework for robust whole-body control of legged manipulators under actuator failures. FT-WBC adopts a decoupled upper- and lower-body policy architecture and introduces two key modules: a Fault Estimator (FE) and a Posture Adaptation Module (PAM). The FE predicts faulty joints from lower-body proprioceptive histories, while the PAM uses this fault information to adapt the base posture plan generated by the arm policy, converting potentially unstable posture requests into safe and executable base posture commands. Through this fault-aware posture adaptation mechanism, FT-WBC synthesizes compensatory gaits under actuator failures and preserves as much arm workspace as possible while maintaining whole-body stability. Simulation and real-world experiments show that FT-WBC significantly improves survival rate and workspace under weakening or locked failures, and transfers zero-shot to a real legged manipulator in the real world.

8. Optimization-based Safe Trajectory Planning for Autonomous Ground Vehicle in Multi-Floor Scenarios

基于优化的多楼层场景自动驾驶地面车辆安全轨迹规划

AI 总结：提出一种针对多楼层场景的轨迹规划框架，包括基于广义Voronoi图和多目标算法的任务规划模块，以及采用优化方法和热启动分层规划生成高质量轨迹的轨迹规划模块，并通过仿真验证可行性。

链接：https://arxiv\.org/abs/2606\.24631

机构：Beijing Institute of Technology（北京理工大学）； Tsinghua University（清华大学）

作者：Zishang Xiang, Runda Zhang, Runqi Chai, Kaiyuan Chen, Senchun Chai, Yuanqing Xia

英文摘要：The development of trajectory planning strategies for autonomous ground vehicles (AGVs) represents a prevailing research interest within the domain of intelligent transportation systems. This paper introduces a trajectory planning framework tailored for multi-floor scenarios. The framework consists of two main modules: the task planning module and the trajectory planning module. The task planning module involves a strategic selection phase, where a task planning strategy based on generalized voronoi diagrams (GVD) and multi-objective algorithms is proposed to select the floor exits for each floor. The trajectory planning module utilizes optimization-based methods to generate high-quality trajectories, and a warm-started hierarchical planning framework is designed to ensure rapid convergence. Additionally, for handling complex obstacle constraints, a correlation constraint calculation method is designed for reducing obstacle constraints in trajectory planning. Finally, the feasibility and effectiveness of the proposed framework are verified through simulations.

3. 操作、抓取与灵巧手 | 5 篇

9. Enforcing Human-like Kinematics in Dexterous Piano Playing via Adversarial Posture Regularization

通过对抗姿态正则化在灵巧钢琴演奏中强制执行类人运动学

AI 总结：提出对抗姿态正则化（APR）方法，利用少量非对齐的人类演奏数据，通过对抗目标匹配策略姿态分布与人类先验，使高自由度灵巧手在物理仿真中弹钢琴时呈现更自然的姿势。

链接：https://arxiv\.org/abs/2606\.23848

机构：Shanghai Jiao Tong University（上海交通大学）； The University of Hong Kong（香港大学）； Shanghai AI Laboratory（上海人工智能实验室）

作者：Bin Qiu, Yanming Shao, Guanyu Cai, Yao Mu

英文摘要：Reinforcement learning can train bimanual dexterous hands to play piano in physics simulation with high note accuracy, but for high-DoF dexterous hands, relying solely on task rewards or IK inversion often leads to unnatural postures and joint overextension. We propose \textit{Adversarial Posture Regularization (APR)}. It avoids expensive, song-aligned expert demonstration data and instead uses a small amount of casual human playing data. By matching the distribution of the posture of the policy with the human prior through an adversarial objective, APR encourages more human-like hand shapes. Meanwhile, we collect and release unstructured hand motion data of piano playing using a consumer-grade Meta Quest 3, and retarget the key motion information to the Shadow Hand. Finally, we achieve significantly better performance than prior methods on all three human-likeness metrics (cPSI, BSE, and FAC) as well as in visual quality. Project repository: this https URL.

10. RE4: Transformation-aware Imitation of Object Interactions Using Manipulation Modes

RE4: 使用操作模式进行物体交互的变换感知模仿

AI 总结：提出RE4框架，通过轻量级自监督姿态估计和操作模式感知的检索、变换、重规划与执行，在保持可解释性的同时提升物体交互模仿学习的性能和鲁棒性。

链接：https://arxiv\.org/abs/2606\.24403

机构：Australian National University（澳大利亚国立大学）

作者：Arsh Chawla, Rahul Shome

英文摘要：Object interaction tasks have been a focus of advances in imitation learning. End-to-end methods, dominated by diffusion and flow-based variants have shown leaps in performance while sacrificing interpretability. Object-centric and pose-informed variants have had a role in learning from demonstration in manipulation tasks. In this paper, we revisit a few modern imitation learning benchmarks for object interactions, with the aim of composing a framework that repurposes principled theories of manipulation, preserving both performance and interpretability. For image observations, lightweight training is proposed for model-free pose estimation of the target object, using self-supervision over the demonstration data available for imitation learning. This information is then used to inform a manipulation mode-aware retrieval of a demonstration, a mode-aware transformation, a replan step that connects to the retrieval point while preserving mode constraints, and finally rolling out the transformed demonstration. These compose four key steps of the proposed RE4 framework, evaluated over state-based and image-based benchmarks in Push-T and Robomimic. An adversarial benchmark that evaluates sparse data regions of image-based Push-T showcases the robustness, further bolstered by indications from low-data regime experiments. The current work shows promise in using simple interpretable building blocks to learn manipulation skills.

11. NoContactNoWorries: Estimating Contact through Vision and Proprioception for In-Hand Dexterous Manipulation

NoContactNoWorries: 通过视觉和本体感觉估计手内灵巧操作的接触

AI 总结：提出一种基于Transformer的多模态框架，融合RGB-D视觉和机器人本体感觉推断二值接触状态，作为手物交互的伪触觉信号，支持下游强化学习实现手内物体重定向并泛化至新物体。

链接：https://arxiv\.org/abs/2606\.24450

机构：Robotics Research Center (RRC), International Institute of Information Technology (IIIT), Hyderabad, India（印度海得拉巴国际信息技术研究所机器人研究中心）； Department of Computer Science, The University of Manchester（曼彻斯特大学计算机科学系）

作者：Soham Patil, Avirup Das, Sourabh Bhosale, Spandan Roy

英文摘要：Perceiving physical contact is fundamental to dexterous manipulation. While robots often rely on dedicated hardware tactile sensors, humans exhibit a remarkable ability to infer contact by integrating visual information with an innate sense of their body's pose and movement. Inspired by this embodied perceptual skill, we investigate whether a robot can learn to infer contact from vision, an approach that also offers a scalable alternative to tactile hardware specifically for binary contact estimation, which faces practical challenges in cost, fragility, and integration. We present NoContactNoWorries, a transformer-based multimodal framework that fuses RGB-D vision with the robot's proprioception to infer binary contact states as a pseudo-tactile signal for hand-object interactions. We validate by training a single contact prediction model on multiple objects and show that the inferred contact signal supports downstream reinforcement learning agents for in-hand object reorientation, generalizing to novel objects. Experiments in both simulation and on a real-world robot validate our approach, highlighting the feasibility of inferring contact from vision and proprioception. Project Page: this https URL

12. Enabling Robust Cloth Manipulation via Inference-Time Simulator-in-the-Loop Refinement

通过推理时仿真器在环优化实现鲁棒的布料操作

AI 总结：提出基于仿真器在环优化的布料操作框架，通过合成数据训练的真实到仿真模块和稀疏网格滚动结合先验引导MPPI，从单RGB输入实现鲁棒操作。

链接：https://arxiv\.org/abs/2606\.24552

机构：National University of Singapore（新加坡国立大学）； Shanghai Jiao Tong University（上海交通大学）

作者：Xin Liu, Yulin Li, Ziming Li, Pengyu Jing, Zhenhao Huang, Bingyang Zhou, Ziqiu Zeng, Siyuan Luo, Chenkun Qi, Fan Shi

英文摘要：Simulator-in-the-loop optimization offers a promising inference-time mechanism for robot manipulation. It uses a physical simulator as a backend rollout engine to evaluate candidate trajectories in parallel and refine nominal actions online, a paradigm proven effective in rigid-body manipulation where state and contact are relatively tractable. We bring this paradigm to real-world cloth manipulation from a single RGB input through three pillars. (i) We design a scalable synthetic-data generation and inference-time rollout pipeline built on FLASH, a deformable-object simulator that provides a practical balance among physical fidelity, numerical stability, and rollout efficiency. (ii) We develop a real-to-sim module, trained purely on synthetic data, that maps a single RGB observation to simulation-compatible cloth state by fusing pretrained visual features with learnable canonical tokens. (iii) We perform online planning by coupling a sparse-mesh rollout backend with prior-guided MPPI, anchored at an offline-distilled policy trajectory, preserving manipulation-relevant deformation and contact while enabling sufficient parallel rollout batches. Real-robot experiments show higher success rates and stronger robustness than baseline methods.

13. TACTFUL: Tactile-Driven Exploration For Object Localization and Identification in Confined Environments

TACTFUL: 受限环境中的触觉驱动探索用于物体定位与识别

AI 总结：提出无视觉触觉探索框架TACTFUL，使多指机器人通过动态奖励策略平衡全局探索与局部表面细化，实现物体自主发现与识别，平均重建误差0.015米，成功率77%。

链接：https://arxiv\.org/abs/2606\.24712

机构：Amazon Fulfillment Technologies & Robotics（亚马逊履约技术与机器人）； Siebel School of Computing and Data Science, University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校西贝尔计算与数据科学学院）； Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）

作者：Shivani Kamtikar, Chung Hee Kim, Camilla Tabasso, Tye Brady, Joshua Migdal, Taskin Padir

英文摘要：Humans effortlessly locate and identify objects by touch alone, even without vision. In contrast, robotic systems rely heavily on vision and struggle with autonomous tactile exploration and object identification. We present TACTFUL, a vision-free tactile exploration framework that enables a multi-fingered robot to autonomously explore confined workspaces, discover objects through contact, and identify them via tactile reconstruction. Trained entirely on real hardware without simulation, our system learns a single policy that balances global workspace exploration with local surface refinement through a dynamic reward schedule. Our results demonstrate that tactile sensing, when paired with structured learning, can serve as an effective primary modality for object-level reasoning, achieving 77% success with 0.015 m average reconstruction error and outperforming baseline approaches on real-world objects.

4. 导航、定位与SLAM | 2 篇

14. NavWM: A Unified Navigation World Model for Foresight-Driven Planning

NavWM：一种用于前瞻性规划的统一导航世界模型

AI 总结：提出NavWM统一导航世界模型，通过潜在世界令牌提取几何与语义先验，结合锚点多模态轨迹预测框架生成多样化动作空间，利用视觉前瞻评估最优路径，在多种机器人数据集上显著提升未来状态生成与零样本导航成功率。

链接：https://arxiv\.org/abs/2606\.24101

机构：Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）； Beihang University（北京航空航天大学）； School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）

作者：Yanghong Mei, Longteng Guo, Ming-Ming Yu, Guiyu Zhao, Xingjian He, Jing Liu

英文摘要：Conventional visual navigation policies often struggle with myopic decision-making and mode collapse in complex environments. While world models offer a promising alternative, existing paradigms typically isolate perception, generation, and control, failing to capture their shared spatio-temporal dynamics. In this paper, we propose NavWM, a unified navigation world model that seamlessly integrates latent world reasoning, multimodal action prediction, and controllable visual generation. At its core, NavWM leverages latent world tokens to distill geometric and semantic priors, endowing the agent with robust structural understanding. To overcome the limitations of deterministic policies, we introduce an anchor-based multimodal trajectory forecasting framework that generates a diverse action space. This inherent diversity explicitly empowers the generative world model to act as a robust closed-loop planner, utilizing visual foresight to evaluate and select the optimal path. Extensive experiments across diverse robotics datasets demonstrate that NavWM significantly advances the state-of-the-art, delivering remarkable improvements in both high-fidelity future state generation and zero-shot navigation success.

15. Vision-Language Model Reasoning for Contextual Semantic Mapping in Intralogistics

面向内部物流的上下文语义建图中的视觉-语言模型推理

AI 总结：提出结合SLAM、SAM、实例聚类和VLM多视角推理的上下文语义建图流程，零样本推断物体可移动性，实现语义分类98.93% mIoU和可移动性估计89.17% mAcc。

链接：https://arxiv\.org/abs/2606\.24814

机构：Institute for Material Handling and Logistics, Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院物流与物料搬运研究所）

作者：Marvin Rüdt, Hao Pang, Constantin Enke, Zäzilia Seibold, Kai Furmans

英文摘要：Autonomous mobile robots operating in intralogistics environments rely on geometric maps for localization and navigation, but lack semantic understanding of objects and their contextual properties. We present a contextual semantic mapping pipeline that combines SLAM-based geometric mapping, SAM-based instance segmentation, instance clustering, and VLM multi-view reasoning to produce a contextual semantic map representation encoding geometric structure, object class, and object movability. By aggregating observations across multiple viewpoints and querying a VLM in a zero-shot, open-vocabulary setting, the pipeline infers contextual object properties--here demonstrated through movability--without requiring task-specific training or predefined object categories. We evaluate three VLMs under two prompting strategies and conduct a component-wise analysis of the pipeline. The proposed pipeline achieves 98.93 % mIoU for semantic classification and 89.17 % mAcc for object movability estimation. Component analysis identifies VLM reasoning as the primary bottleneck for contextual understanding and instance clustering as the main limitation for panoptic performance. The resulting semantic map supports context-aware filtering and robust navigation in dynamic intralogistics environments.

5. 人机交互与协作机器人 | 1 篇

16. Legible and Intuitive Multi-modal Robot State and Intent Communication Validated in Online and Real-world Studies

可读且直观的多模态机器人状态与意图通信：在线和真实世界研究验证

AI 总结：针对移动非人形机器人，通过在线和真实世界实验，系统比较了低表达单模态LED与高表达多模态（凝视、手势、语音）通信策略，发现多模态更可读直观，且真实环境中可读性和信心下降。

链接：https://arxiv\.org/abs/2606\.24445

机构：Chair of Perception for Intelligent Systems, Munich Institute of Robotics and Machine Intelligence (MIRMI), Technical University of Munich (TUM)（慕尼黑工业大学慕尼黑机器人与机器智能研究所智能系统感知教席）； Centre for Applied Autonomous Sensor Systems (AASS), Örebro University（厄勒布鲁大学应用自主传感器系统中心）； Robotics Institute Germany (RIG)（德国机器人研究所）

作者：Tim Schreiter, Jens V. Rüppel, Andrey Rudenko, Martin Magnusson, Achim J. Lilienthal

英文摘要：Effective robot-to-human communication can increase transparency and trust, reduce uncertainty, and contribute to safer collaboration in shared workspaces. Designing and validating an effective robot communication strategy is challenging due to the varying and often limited communication modalities across robots, differences in how diverse recipients interpret messages, and the underexplored virtual-to-real gap in studies of communication legibility. We present a systematic, large-scale comparative validation of existing communication strategies for a mobile non-humanoid robot across message types and settings (online and in-person). Based on the prescribed message types in the existing standards for industrial robots, we realize and compare a low-expressive, unimodal LED-based strategy with a highly expressive, multimodal one that leverages robotic gaze, gestures, and voice. For each strategy, we analyze the communication of a turning intention, an attention request, error status, whether the robot is stuck, and whether it is functioning normally. We evaluate these strategies in replicated online and in-person experiments. We find strong evidence that highly expressive multimodal communication is perceived as more legible and intuitive than unimodal LED-based communication. Comparing the online and real-world study findings, we observe a notable decrease in overall legibility, particularly for signaling with LEDs. Similarly, confidence in message interpretation decreases during the real-world evaluation.

6. 具身智能与视觉语言动作模型 | 4 篇

17. RoBoSR: Structured Scene Representations for Embodied Robotic Reasoning

RoBoSR：面向具身机器人推理的结构化场景表示

AI 总结：提出RoBoSR，一种基于语义对象中心场景图的中间结构表示，将操作建模为逐步状态转换，实现因果推理和长期任务规划，并在零样本泛化中优于提示方法和经典TAMP基线。

链接：https://arxiv\.org/abs/2606\.24338

作者：Kewei Hu, Wanchan Yu, Fangwen Chen, Jing Jiajian, Zimeng Li, Ying Wei, Tianhao Liu, Michael Zhang, Hanwen Kang

英文摘要：Despite rapid progress, embodied reasoning under real-world variability remains challenging. Existing approaches rely on demonstration-driven sequential biases, limiting flexibility in open-ended and long-horizon tasks that require structured reasoning over evolving states. We introduce RoBoSR, an intermediate structural representation that formulates manipulation as step-wise state transitions over semantically grounded, object-centric scene graphs. By modeling object states and their spatial relations at the perception-action interface, RoBoSR disentangles high-level task reasoning from raw inputs and enables structured reasoning over preconditions, effects, and goal states. This representation endows the agent with causal reasoning capability, enforcing subtask dependencies and supporting coherent long-horizon task planning. To learn such structure-aware reasoning, we construct Manip-Cognition-1.6M, an open-world dataset that jointly supervises scene understanding, instruction interpretation, and subtask planning across diverse tasks. Across several benchmarks and real-world demonstrations, our method consistently outperforms prompting-based methods and classical TAMP baselines in zero-shot generalization and long-horizon tasks. The results underscore structured intermediate representations as a critical inductive bias for scalable embodied reasoning.

18. Supervise What Survives: Geometry-Guided VLA Adaptation from Synthetic Robot Videos

监督幸存信息：基于几何引导的合成机器人视频VLA适配

AI 总结：提出GRA方法，从合成视频中提取几何信息（2D末端执行器路径点）监督视觉表征，仅用真实演示训练动作头，在真实机器人任务中优于伪动作基线。

链接：https://arxiv\.org/abs/2606\.24448

机构：Show Lab, National University of Singapore（新加坡国立大学 Show Lab）

作者：Danze Chen, Yanzhe Chen, Qiming Huang, Zhijun Cao, Chen Gao, Mike Zheng Shou

英文摘要：Vision-Language-Action (VLA) models require large-scale video-action pairs, yet real teleoperation remains scarce. While generated robot videos offer a scalable alternative, existing methods treat them as real robot data by recovering pseudo-actions from synthesized pixels. We argue that deriving low-level control from generated visuals is a mismatched abstraction. A video captures only \emph{geometry}: the spatial trajectory representing the \emph{where} of a task. A real demonstration captures \emph{control}: the exact motor commands representing the \emph{how}. Human-to-robot video generation preserves these unequally: the visible geometry survives the generation process, while the underlying control signals are lost. This \textbf{Asymmetric Preservation Principle} dictates a clean rule: this surviving geometry should solely supervise visual perception, leaving control to real demonstrations. Following this principle, we propose \textbf{GRA} (\textbf{G}eometry-guided \textbf{R}epresentation \textbf{A}lignment), which extracts the geometric content as future 2D end-effector waypoints, computed from the source human video through pose estimation, retargeting, simulation, and calibrated projection, and routes them to the VLA vision backbone via an auxiliary 2D head. The action head is trained on real demonstrations only. During fine-tuning, the waypoint loss persists as a \textbf{spatial representation anchor} that prevents the backbone from losing its geometric grounding. On real-robot tasks, GRA outperforms pseudo-action baselines under matched data budgets and narrows the gap to policies trained with substantially more real demonstrations, suggesting that correctly routed geometry bridges generated videos to robot policies more reliably than recovered actions.

19. G$^3$VLA: Geometric inductive bias for Vision-Language-Action Models

G$^3$VLA：视觉-语言-动作模型的几何归纳偏置

AI 总结：提出G$^3$VLA模块，通过射线嵌入、投影位置编码和跨视图融合为VLA模型注入相机几何先验，无需深度传感器，在多个基准和真实机器人上提升空间敏感任务性能。

链接：https://arxiv\.org/abs/2606\.24472

机构：New York University Shanghai（上海纽约大学）； Technical University of Denmark（丹麦技术大学）； MBZUAI - Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）； New York University Abu Dhabi（纽约大学阿布扎比分校）

作者：Yue Peng, Yongzhe Zhao, Artur Habuda, Khuyen Pham, Yanheng Zhu, Tran Nguyen Le, Fares Abu-Dakka, Li Guo

英文摘要：Vision-language-action (VLA) models have made rapid progress in generalist robot manipulation by harnessing semantic knowledge from pretrained vision-language backbones, but their visual tokens remain grounded in 2D image coordinates rather than the calibrated geometry of the robot's cameras -- a mismatch especially pronounced in multi-camera setups, where views are coupled by known intrinsics and extrinsics yet processed as independent images. We propose G$^3$VLA, a camera-aware geometric module that injects calibrated structure into the visual-token stream of a pretrained VLA without altering its action space or imitation objective, combining intrinsic-conditioned ray embeddings, projective positional encoding (PRoPE), and bidirectional cross-view fusion. Geometric supervision is provided either from ground-truth point maps when available, or from confidence-gated $\pi^3$X teacher predictions, requiring no depth sensors or manual annotations. Instantiated on $\pi_0$, G$^3$VLA yields consistent gains across the LIBERO suites, RoboCasa24, RoboTwin2.0, and real-robot settings, with the largest improvements on spatially and object-sensitive tasks. We further validate on $\pi_{0.5}$ and GR00T 1.5, with results suggesting that geometric transfer is most effective when geometry-aware tokens have direct access to the action generation pathway. Our project page is at this https URL

20. InSight: Self-Guided Skill Acquisition via Steerable VLAs

InSight: 通过可引导的VLA实现自主技能获取

AI 总结：提出InSight框架，通过将VLA模型在基本动作层面变得可引导，实现自主技能获取，包括自动分割演示为基本动作和VLM引导的数据飞轮，无需人类演示即可学习新技能。

链接：https://arxiv\.org/abs/2606\.24884

机构：Stanford University（斯坦福大学）； Princeton University（普林斯顿大学）

作者：Maggie Wang, Lars Osterberg, Stephen Tian, Ola Shorinwa, Jiajun Wu, Mac Schwager

英文摘要：Vision-language-action (VLA) models can learn manipulation skills from demonstrations, but their capabilities are bounded by the skills in the training data. We present InSight, a framework that unlocks autonomous skill acquisition by rendering VLAs steerable at the primitive-action level (e.g., "move gripper to the bowl", "lift upward", "pour the bottle"). InSight consists of two primary stages: (1) an automated segmentation pipeline that partitions demonstrations into labeled primitives via VLM plan decomposition and end-effector poses to enable VLA primitive steerability, and (2) a VLM-guided data flywheel that identifies missing primitives required to accomplish a novel task, autonomously attempts demonstrations of the missing primitives with VLM-proposed low-level control, and automatically labels, stores, and integrates successful demonstrations into the VLA training set. We evaluate InSight across simulation and real-world manipulation tasks, including block flipping, drawer closing, sweeping, twisting, and pouring, without any human demonstrations of these target skills. Once learned, these primitives can be composed to execute novel, long-horizon tasks without additional human demonstrations. Our findings demonstrate that primitive steerability provides a practical foundation for continual skill acquisition in VLA policies. Project website: this https URL.

7. 多机器人与群体系统 | 4 篇

21. Topological Online Learning for Displacement-based Formation Control

基于位移的编队控制中的拓扑在线学习

AI 总结：提出拓扑在线学习（TOLD）框架，通过实时调整交互权重最小化编队失真，理论保证有向图下渐近一致性与有界失真，仿真和实物实验验证了有效性。

链接：https://arxiv\.org/abs/2606\.23901

机构：Indian Institute of Science（印度科学学院）； ArtGarage, Indian Institute of Science（印度科学学院ArtGarage）

作者：Saksham Sharma, Shubhankar Gupta, Sumant A Gunagi, Suresh Sundaram

英文摘要：This paper addresses the problem of robust formation control by introducing Topological Online Learning for Displacement-based (TOLD) formation control, a real-time edge-level adaptation framework. Unlike conventional node-level robust controllers that regulate individual robot inputs without modifying the interaction topology, TOLD updates the interaction topology weights online to directly minimize formation distortion. Two strategies are proposed under the TOLD formation control framework: Online Gradient Flow (OGF) with unconstrained weights and Online Exponential Gradient Flow (OExpGF) with non-negative convex weights. Theoretical analysis establishes that, for single-integrator agents over directed graphs, OExpGF guarantees asymptotic consensus, while OGF ensures bounded formation distortion. Simulations with twelve robots under intermittent disturbances show 1.2%-33.14% median cumulative Root Mean Distortion Error reduction when augmenting TOLD with node-level controllers. Hardware experiments with Crazyflie 2.0 quadrotors demonstrate over 62% (OGF) and 31.4% (OExpGF) reduction in median formation distortion compared to fixed-weight consensus.

22. The Evaluation Cost of Task Specialization in Evolutionary Multi-Robot Systems

进化多机器人系统中任务专业化的评估成本

AI 总结：研究在进化多机器人系统中，任务专业化与通用行为相比所需的评估预算成本，发现随着系统规模增大，更少的预算即可使专业化表现优于通用化。

链接：https://arxiv\.org/abs/2606\.24191

机构：University of Konstanz（康斯坦茨大学）； Centre for the Advanced Study of Collective Behaviour（集体行为高级研究中心）； Department of Computer and Information Science（计算机与信息科学系）； University of Technology Nuremberg（纽伦堡工业大学）； Department of Computer Science and Artificial Intelligence（计算机科学与人工智能系）

作者：Paolo Leopardi, Heiko Hamann, Jonas Kuckling, Tanja Katharina Kaiser

英文摘要：Task specialization can improve the efficiency of multi-robot systems (MRSs). Previous works have investigated the emergence of task-specialist robot controllers through evolutionary optimization and have argued that task specialization is more likely to evolve when subtask behaviors are readily available as building blocks. However, the available evaluation budget must be distributed across all subtasks, whereas a single generalist behavior can exploit the entire budget for its own optimization. We present a cost-benefit analysis of evolving task-specialist versus generalist behaviors in a foraging scenario here. In a physics-based robotics simulator, we study the total evaluation budget required to evolve task-specialist behaviors that outperform generalist behaviors across MRS sizes. We show that with increasing MRS size, a lower total evaluation budget is sufficient to evolve specialists that outperform generalists.

23. Varying Bundle Size Reactive Multi-Task Assignment using Selective Cost Estimation for Multi-Agent Systems

基于选择性成本估计的多智能体系统变束大小反应式多任务分配

AI 总结：提出一种分布式两阶段多保真束生成框架，通过低保真启发式搜索束空间，仅对最有希望候选进行高保真路径规划，实现反应式多机器人任务分配，提高拍卖方法性能。

链接：https://arxiv\.org/abs/2606\.24462

机构：Luleå University of Technology（吕勒奥理工大学）

作者：Niklas Dahlquist, Shridhar Velhal, George Nikolakopoulos

英文摘要：This paper presents a scalable framework for multi-robot task allocation in complex environments where estimating task execution costs is computationally expensive. While combinatorial auction-based approaches offer reliable solutions, the exponential complexity of bundle generation typically renders them intractable for real-time reactive applications, particularly when accurate path planning is required for cost validation. We address this through a distributed, two-stage multi-fidelity bundle generation approach. Agents utilize a local search tree guided by a low-fidelity heuristic (such as euclidean distance) to rapidly explore the bundle space, applying high-fidelity path planning only to the most promising candidates in a best-first manner. These refined bids are then submitted to a central coordinator that solves a set packing problem to ensure global feasibility and maximize the overall utility. Simulation results in multiple environments demonstrate that the framework is able to improve the performance of reactive auction-based task allocation. Overall, the presented framework is shown to enable reactive task allocation with dynamic bundle sizes in multiple settings without exposing the agents' state and internal cost estimation models.

24. Decentralized Pose Graph Riemannian Optimization for Object-based Multi-Robot SLAM

基于对象的分布式位姿图黎曼优化用于多机器人SLAM

AI 总结：提出一种完全分布式的黎曼优化框架，通过共识机制解耦对象级多机器人SLAM中的位姿图优化问题，并利用近似牛顿法加速收敛，在SE(d)流形上保持几何一致性，显著降低通信开销并提升精度。

链接：https://arxiv\.org/abs/2606\.24489

机构：College of Control Science and Engineering, Zhejiang University（浙江大学控制科学与工程学院）； Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University（同济大学上海智能自主系统研究院）； Division of Decision and Control Systems, School of EECS, KTH Royal Institute of Technology（瑞典皇家理工学院电气工程与计算机科学学院决策与控制系统系）

作者：Yixian Zhao, Yan Huang, Yang Xu, Liang Li, Jinming Xu

英文摘要：Pose graph optimization (PGO) is a key back-end component for state estimation in networked multi-robot simultaneous localization and mapping (SLAM). In object-based multi-robot SLAM, the problem becomes more tightly coupled because robots must jointly estimate both their trajectories and the poses of persistent objects observed by multiple agents. Existing decentralized solutions often assume that the communication graph closely matches the physical interaction topology, which is restrictive in realistic deployments where communication is sparse, intermittent, or time-varying. This paper presents a fully decentralized Riemannian optimization framework for object-based multi-robot PGO that decouples the coupled estimation problem via a consensus mechanism, enabling flexible communication topologies. To improve convergence under limited communication budgets, we further develop a distributed approximate-Newton scheme that exploits local second-order information while operating directly on the SE(d) manifold to preserve geometric consistency, and we establish the convergence to Riemannian first-order stationary points and provide a local condition-number analysis explaining the benefit of approximate second-order information over first-order Riemannian descent. The resulting method reduces iteration count and communication overhead without sacrificing estimation accuracy. Extensive evaluations on public benchmarks, large-scale simulations, and real-world multi-robot experiments demonstrate improved accuracy, runtime efficiency, scalability across network topologies, and robustness to communication failures.

8. 无人车、无人机与移动机器人 | 1 篇

25. SlipSense: Multimodal Sensing for Online Slip Detection in Legged Robots

SlipSense: 用于足式机器人在线滑移检测的多模态传感

AI 总结：提出SlipSense框架，通过定制轻量化传感器足和LSTM模型，实现四足机器人基于力的在线滑移检测，早期滑移检测精度达24.1mm，准确率85.9%。

链接：https://arxiv\.org/abs/2606\.24350

机构：School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）； Institute for Infocomm Research, Agency for Science Technology and Research（资讯通信研究院，科技研究局）

作者：Iris Szu-Yao Liu, Chien Chern Cheah, Meng Yee Michael Chuah

英文摘要：Legged robots rely on accurate ground interaction awareness to traverse variable terrains, such as slippery surfaces. Existing slip detection methods often rely on kinematics and proprioception, which lack the sensitivity to detect early-stage slips that occur prior to catastrophic instability. Thus, this paper presents SlipSense, a novel framework for online force-based slip detection using a custom lightweight sensorized foot for quadrupeds to detect slip. The framework integrates a multimodal sensor design with a LSTM-based model to infer ground reaction forces and detect slip-indicative anomalies during locomotion. The proposed framework is deployed on a Unitree Go1 quadruped to demonstrate blind online slip detection over a slippery terrain. Our method detects early-stage slips down to an average displacement of 24.1 +/-6.4mm with an overall accuracy of 85.9%. This represents a 3.3-fold finer detection resolution and a 24% relative accuracy improvement over a standard kinematic baseline that uses foot velocity inferred through state estimation. The work in this paper serves as a foundation for force-aware gait adaptation in legged robotic locomotion, allowing future controllers to estimate terrain friction and adjust constraints, thus improving the overall stability of the system.

9. 软体机器人与硬件设计 | 1 篇

26. PDS Joint: A Parametric Double-Spiral Joint Tailored for Dexterous Hands

PDS关节：一种专为灵巧手设计的参数化双螺旋关节

AI 总结：提出参数化双螺旋（PDS）柔性关节，通过阿基米德和对数螺旋模板实现多方向刚度调控，结合嵌入式感应与学习校准实现大变形下关节状态估计，并集成至开源灵巧手验证抓取与安全交互。

链接：https://arxiv\.org/abs/2606\.24377

机构：Beijing Institute of Technology（北京理工大学）

作者：Haoyang Li, Yibo Wen, Yixiang Fan, Yiheng Xu, Yufeng Yue

英文摘要：Compliant joints can embed safety and adaptability into dexterous hands, but achieving large-stroke anthropomorphic motion while maintaining joint-specific, directiondependent stiffness and reliable proprioception remains challenging. This paper presents the PDS joint, a parametric doublespiral (PDS) compliant joint that enables systematic shaping of directional stiffness across multiple deformation modes, including flexion/extension, abduction/adduction, and pronation/supination. We instantiate the joint using Archimedean and logarithmic spiral templates for different hand joints and introduce an asymmetry ratio to tailor stiffness distributions for both grasp stability and hyperextension resistance. To make the joint practically usable under large deformation, we co-design embedded inductive proprioception and propose a learningbased calibration pipeline that maps raw inductive signals to joint states using ArUco-marker tracking. Experiments characterize the stiffness landscapes across geometric parameters and demonstrate a non-monotonic dependence of lateral support on asymmetry, indicating the importance of principled parameter tuning. For joint-state estimation in the most challenging abduction/adduction motion, a learned multilayer-perceptron (MLP) mapping reduces the error compared with conventional curve fitting by 41.6%. Finally, we integrate the proposed joints into an open-source dexterous hand as a demonstration platform, on which the hand grasps a set of nine everyday objects and performs safe, contact-rich human-involved interactions.

10. 仿真、数据集与评测 | 3 篇

27. Sim-to-Real Betting on the E-Process: Bringing "simulators" to anytime-valid confidence sequences

基于E过程的仿真到现实投注：将“模拟器”引入随时有效置信序列

AI 总结：将仿真到现实性能估计与投注方法及安全随时有效推断相结合，利用缩放模拟器生成高效可靠的均值估计证书，特别适用于机器人性能测试。

链接：https://arxiv\.org/abs/2606\.24038

机构：Iowa State University（爱荷华州立大学）

作者：Yujia Chen, Bowen Weng

英文摘要：This note describes an integration of the sim-to-real performance estimate with betting (from Chen et al.) and the safe anytime-valid inference (from Ramdas et al.). Using the scaled simulators. The method produces efficient, reliable certificates for the mean estimate, an approach that is especially valuable in robot performance testing. This note gives a primary, self-contained account of the construction; preliminaries of the respective methods are kept at a minimum, and one shall refer to the original works for full detail. Some synthetic examples demonstrating the proposed algorithm can be found at this https URL.

28. MinInter: Minimizing Trajectory Interpolation During Data Augmentation for Imitation Learning

MinInter：在模仿学习的数据增强中最小化轨迹插值

AI 总结：提出MinInter方法，通过选择需要最少插值的源演示来生成高质量合成轨迹，在MimicGen基准的12个操作任务中显著提升数据生成成功率和策略成功率。

链接：https://arxiv\.org/abs/2606\.24078

机构：School of Automation and Intelligent Manufacturing, Southern University of Science and Technology（南方科技大学自动化与智能制造学院）； Pengcheng Laboratory（鹏城实验室）； Carnegie Mellon University（卡内基梅隆大学）

作者：Qingyang Wang, Xingang Liu, Changwei Yao, Zikai Ouyang, Junwei Liu, Haibo Lu, Wei Zhang

英文摘要：Imitation learning enables robots to acquire complex manipulation skills from demonstrations, but its effectiveness is limited by the cost of collecting high-quality data. Trajectory-level data augmentation methods alleviate this challenge by recombining expert demonstrations under varied initial states. However, such methods typically insert interpolations or other non-expert transition segments between disjoint parts, and such non-expert segments could reduce the quality of the generated data. This paper introduces Minimizing Interpolation (MinInter), an effective trajectory selection method that, for each sampled initial configuration, chooses the source demonstration requiring the least interpolation to form a complete trajectory. By explicitly minimizing interpolations during data generation, MinInter produces higher-quality synthetic demonstrations while remaining compatible with existing data generation frameworks. Experiments on 12 manipulation tasks with 26 variants from the MimicGen benchmark show that MinInter consistently improves both data generation success rates and policy success rates, with the largest gains on contact-rich, long-horizon and high-variance settings. Compared to the recent SkillGen framework, MinInter achieves higher policy success rates despite its conceptual simplicity, underscoring the value of interpolation minimization for data augmentation.

29. ArtiTwinSplat: Interactable Digital Twin Reconstruction via Gaussian Splatting from RGB-D videos

ArtiTwinSplat：基于RGB-D视频的高斯泼溅实现可交互数字孪生重建

AI 总结：提出ArtiTwinSplat框架，利用3D高斯泼溅从RGB-D视频自动构建可交互的铰接物体数字孪生，无需CAD模型或人工标注，支持实时渲染与交互操作。

链接：https://arxiv\.org/abs/2606\.24628

机构：ETH Zürich（苏黎世联邦理工学院）； Microsoft（微软）； University of Bonn（波恩大学）

作者：Pranjal Mishra, René Zurbrügg, Max Wilder-Smith, Marco Hutter, Marc Pollefeys, Zuria Bauer, Hermann Blum

英文摘要：Deploying robots in unstructured real-world environments needs accurate, interactive models of the objects. Constructing these models at scale remains a critical bottleneck for robotic system integration. We present ArtiTwinSplat, a framework that automatically constructs articulated, photo-realistic digital twins of objects directly from RGB-D videos, requiring no CAD models, simulation assets, or manual annotations. Our method is built on 3D Gaussian Splatting that preserve geometric fidelity and photometric realism, coupled with an unsupervised articulation discovery pipeline that recovers part structure and joint kinematics from observed motion alone. With tracking and optimization stages our method provides stable, queryable digital twins that support real-time rendering, viewpoint control, and interactive manipulation. Unlike prior methods confined to simulation, ArtiTwinSplat operates directly on real-world observations and produces twins that are immediately usable by downstream robot planning and learning systems. This method offers a practical, scalable pathway toward digital twin construction, lowering the integration barrier for articulated object manipulation in embodied AI and human-robot collaboration contexts.

11. 安全、鲁棒性与可信机器人 | 3 篇

30. Verifiable Foundation Models for Robot Safety

机器人安全的可验证基础模型

AI 总结：提出FEARL框架，将策略分解为大控制器和小安全模块，使形式化验证仅应用于安全模块，从而在保持控制器表达能力的同时实现可验证的机器人安全控制。

链接：https://arxiv\.org/abs/2606\.23754

机构：University of California, Irvine（加州大学尔湾分校）

作者：Davide Corsi, Kyungmin Kim, Roy Fox

英文摘要：Deploying foundation models for robot control raises a central challenge: the expressive power that enables rich, multimodal perception also makes these models opaque and difficult to analyze formally, rendering them intractable for existing verification tools. In this paper, we present FEARL (Foundation-Enabled Assured Robot Learning), a framework that addresses this tension through a modular architectural decomposition. FEARL separates the policy into a large Controller (C) responsible for high-dimensional perception and task reasoning, and a small Safety module (S) that receives low-dimensional observations from dedicated safety sensors together with a bounded context embedding from C and produces the final action. Since many robot safety requirements, such as collision avoidance and workspace boundary constraints, can be expressed over these safety sensor observations, formal verification can be applied to S rather than to the full foundation-model backbone. This makes formal analysis tractable with existing tools while preserving the Controller's expressive power for task reasoning. To show that the decomposed policy remains capable of solving diverse tasks, we evaluate FEARL on three simulated robotic domains using multiple Controller backbones and training procedures, including pretrained off-the-shelf vision-language-action models. We further transfer the learned policy from one of our simulated tasks to a physical robot, suggesting that the low-dimensional safety interface supports practical sim-to-real transfer.

31. Engineering Reliable Autonomous Systems: Challenges and Solutions

工程可靠自主系统：挑战与解决方案

AI 总结：本文报告了2024年Lorentz中心研讨会关于工程可靠自主系统的讨论，聚焦验证与验证技术、实际系统工程和安全软件架构三大主题，提出了挑战目录和解决路径。

链接：https://arxiv\.org/abs/2606\.23760

机构：University of Manchester（曼彻斯特大学）； University of Nottingham（诺丁汉大学）； University of Modena and Reggio Emilia（摩德纳大学）； University of Aberdeen（阿伯丁大学）； Utrecht University（乌得勒支大学）； University of Milano-Bicocca（米兰比可卡大学）； University of York（约克大学）； University of Malta（马耳他大学）； ENSTA | Institut Polytechnique de Paris（巴黎高等先进技术学院）； Vanderbilt University（范德堡大学）； ONERA（法国航空航天研究院）； Politecnico di Milano（米兰理工大学）； Kernkonzept GmbH（Kernkonzept有限公司）； Lancaster University Leipzig（兰卡斯特大学莱比锡校区）； Gran Sasso Science Institute（格兰萨索科学研究所）； Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）； University of Oslo（奥斯陆大学）； Southwest University（西南大学）； Aarhus University（奥胡斯大学）； Newcastle University（纽卡斯尔大学）； KU Leuven（鲁汶大学）； Maynooth University（梅努斯大学）

作者：Marie Farrell, Matt Luckcuck, Angelo Ferrando, Rafael C. Cardoso, Natasha Alechina, Marco Autili, Diana Benjumea Hernandez, Luciana Brasil Rebelo dos Santos, Daniela Briola, Ana Cavalcanti, Christian Colombo, Louise A. Dennis, Clare Dixon, Michael Fisher, Mario Gleirscher, Taylor Johnson, Charles Lesire, Livia Lestingi, Sven Linker, Brian Logan, Colin Paterson, Fabio Papacchini, Patrizio Pelliccione, Pedro Ribeiro, Maike Schwammberger, Silvia Lizeth Tapia Tarifa, Hazel Taylor, Jim Woodcock, Mengwei Xu, Yi Yang, Huan Zhang

英文摘要：Engineering reliable autonomous systems is an important and growing topic in computer science. As autonomous systems become more prevalent, easy-to-use techniques for building them reliably are increasingly important. This workshop report captures and expands on the discussions at the Lorentz Center Workshop "Engineering Reliable Autonomous Systems" (ERAS), held from 10 to 14 June 2024. The workshop was co-organised by the organisers of the Workshop on Formal Methods for Autonomous Systems (FMAS) and the Workshop on Agents and Robots for reliable Engineered Autonomy (AREA). It brought together members of the FMAS and AREA communities, industry practitioners, and representatives from sectors where autonomous systems pose distinctive engineering challenges. The workshop focused on three main research topics: techniques for verification and validation of autonomous systems; engineering real-world autonomous systems; and software architectures for safe autonomous systems. Its main outcome is a catalogue of challenges in these areas and, most importantly, a pathway to solutions. Some challenges can already be tackled by techniques that are well known in academia but have not yet become regularly used in practice. Other challenges remain unresolved and require further research. This roadmap is intended to support future research and industrial collaboration.

32. Explaining Failures of Cyber-Physical Systems with Actual Causality

用实际因果性解释信息物理系统的故障

AI 总结：提出利用实际因果性框架解释信息物理系统故障，解决理论空白并给出两种实用算法，在神经网络控制的自动驾驶汽车上验证。

链接：https://arxiv\.org/abs/2606\.24546

机构：Rice University（莱斯大学）

作者：Khen Elimelech, Tom Yaacov, David A. Kelly, Hana Chockler, Moshe Y. Vardi

英文摘要：Modern autonomous Cyber-Physical Systems (CPSs), such as self-driving cars, face increasingly complex demands, and yet are expected to act reliably. The black-box nature often characterizing such systems, especially those relying on neural components, makes it impossible to fully verify the system behavior prior to deployment. Unfortunately, unexpected failures-when the system does not comply with its specification-are inevitable and may have catastrophic implications. To improve trust in the system and facilitate future mitigation after a failure occurs, it is important to try to derive an explanation for the unexpected system behavior. This paper introduces the novel concept of leveraging the framework of actual causality for CPS failure explanation. Up until now, this framework was only used to derive explanations in the context of simple systems, such as image classifiers. This paper addresses the theoretical gaps and provides the guidance needed to allow for correct explanation derivation in the CPS domain. Beyond the theoretical contribution, the paper presents two novel, practical, system-agnostic explanation derivation algorithms, allowing to prioritize either explanation optimality or derivation efficiency. The approach is demonstrated and evaluated in the context of a neural-network-controlled autonomous car, designed to avoid collisions.

原文链接: https://www.leiphone.com/category/private/I8j6ZfIp71LtKj6y.html

- Copy Link
  
  Twitter
  
  Facebook
  
  LinkedIn