What is MiniCPM-Robot?

MiniCPM-Robot is a lightweight, open-source embodied AI Vision-Language-Action (VLA) model series developed by Moonshot Intelligence, comprising the 1.5B-parameter MiniCPM-RobotManip general-purpose manipulation model and the 0.9B-parameter MiniCPM-RobotTrack tracking and navigation model, complemented by the PhyAI inference framework. The models leverage Visual Token Compression and Streaming Context Memory techniques, enabling long-range fine-grained robotic arm operations and indoor/outdoor target tracking for robotic dogs. They support on-device local execution under weak network conditions with inference latency as low as 120ms, placing them among the top tier in mainstream benchmarks.

Main Features of MiniCPM-Robot

• MiniCPM-RobotManip: Controls robotic arms to perform long-haul task execution (e.g., making sandwiches), supports multi-camera input, and features context memory capability to retain task progress and historical actions.

• MiniCPM-RobotTrack: Enables robotic dogs to continuously track designated targets based on natural language instructions, supporting single-target tracking, multi-person interference tracking, and fuzzy target tracking, all operable locally under weak or offline network conditions.

Technical Principles of MiniCPM-Robot

• Visual Token Compression: Filters out irrelevant visual information unrelated to tasks, reducing computational load per inference cycle.
• Streaming Context Memory: Reuses previously processed contextual data; new frames require only incremental computation, preventing explosive growth in computation for long-duration tasks.
• Self-Evolving Data Pipeline: Actively exposes model weaknesses in simulation and real-world environments, collects error scenarios for expert correction and closed-loop retraining.

Follow WeChat and reply "open source" to join the AI Open Source Project Community

How to Use MiniCPM-Robot

• Environment Setup:

  • Clone the GitHub repository and install dependencies.

  • Prepare hardware: robotic arm (RobotManip) or Unitree Go2 Edu robotic dog (RobotTrack).

• Model Loading: Download corresponding model weights from Hugging Face. Load models via the PhyAI framework, completing quantization and operator optimization.
• Camera Integration: Configure wrist-mounted camera and main-view camera (RobotManip) or the robotic dog’s built-in camera (RobotTrack).
• Instruction Input: Input natural language commands (e.g., “follow the person in black” or “make a sandwich”).
• One-Click Launch: Run the open-source script; the model receives visual and linguistic inputs, outputs robotic action commands, and executes them.

Core Advantages of MiniCPM-Robot

• Minimal Parameters, Top-Tier Performance: With only 1.5B parameters, it matches larger models like π0.5 in LIBERO, Calvin, and RoboTwin2 benchmarks; achieves ~53 points on RMBench for context memory, far exceeding π0.5’s ~10 points.

• Inference Latency Halved: Single-frame decision latency on H100 is ~120ms—approximately half that of π0.5 (~234ms).

• Truly On-Device Execution: Robotic dogs achieve stable tracking at over 5Hz using only local compute power, continuing operation even when disconnected from the network.

Project Links for MiniCPM-Robot

• Official Website: https://github.com/OpenBMB/MiniCPM-Robot
• Hugging Face Model Hub:
  • https://huggingface.co/openbmb/MiniCPM-RobotManip
  • https://huggingface.co/openbmb/MiniCPM-RobotTrack

Competitive Comparison with Similar Models

Application Scenarios of MiniCPM-Robot

• Food Preparation: Dual-arm robots complete long-haul sandwich-making tasks such as retrieving bread, adding lettuce, ham, and eggs upon instruction.
• Exhibition Guiding: Robots deliver offline tours, answer free-form questions, and plan routes using local knowledge bases without internet connectivity.
• Facility Patrol: Robotic dogs identify unauthorized vehicle parking, road anomalies, and public facility issues, processing sensitive visuals locally without uploading to the cloud.
• Warehouse Logistics: Robots execute long-haul tasks like shelf retrieval and material handling; automatically retry and recover upon failure.
• Indoor/Outdoor Following: Robotic dogs maintain continuous tracking of target individuals in weak-network environments such as elevators and underground parking garages based on natural language commands.