Skip to content

🕹️ PND Adam Teleoperation

A real-time motion mapping framework designed for the Adam series humanoid robots, delivering natural and smooth teleoperation via high‑precision motion‑capture systems.

Supports Noitom Motion Capture System and Meta Quest 3S as input sources.

📋 Supported Models

Model Noitom Mocap VR Headset
Adam SP ✅ In progress
Adam Pro ✅ In progress
Adam U ✅ ✅

Demo

🏗️ System Architecture

The system uses a distributed architecture:

  • Motion Input Layer: Noitom motion‑capture system (Windows) or Meta Quest 3S VR device
  • Control Core Layer: pnd_teleoperation and pnd-control running on the Adam robot’s onboard NUC
  • Data Pipeline: Real‑time communication based on ROS 2 Humble

teleop arch

⚙️ Environment Setup

The Adam robot’s built‑in NUC is pre‑installed with the full runtime environment.
If deploying on another device, configure the following:

sudo apt-get update && sudo apt-get install -y \
    build-essential \
    coinor-libipopt-dev \
    gfortran \
    liblapack-dev \
    pkg-config \
    swig \
    git \
    cmake \
    python3 \
    python3-pip \
    git-lfs \
    --install-recommends

cd /tmp && \
git clone https://github.com/casadi/casadi.git casadi && \
cd casadi && \
git checkout 3.6.7 && \
mkdir build && \
cd build && \
cmake -DWITH_PYTHON=ON -DWITH_IPOPT=ON -DWITH_OPENMP=ON -DWITH_THREAD=ON .. && \
make -j$(nproc) && \
sudo make install && \
sudo ldconfig

💃 Motion Capture Setup

Motion_capture

Launch Axis Studio, complete sensor connection and calibration.
Enable BVH broadcast and send the UDP motion stream to the target robot IP.

Refer to the Video

BVH Broadcast Key Points:

  • Local Address: The IP of the computer running Axis Studio
  • Target Address: The IP of the Adam robot
  • Network Requirement: Both devices must be on the same LAN

Axis broadcast config

Notice

  • After calibration, keep both feet fixed when beginning teleoperation
  • To change position, disconnect teleoperation, move, then re‑calibrate

Prerequisites:

  • VR device and Adam robot must be on the same LAN
  • pnd_teleoperation is running
  • Python dependencies installed 
cd ~/Documents/pnd_teleoperation
pip install uv --user
uv sync

Steps:

  1. Open https://192.168.XX.XX:8443 in the Meta Quest 3S browser
  2. Click Start Controller Tracking
  3. Foxglove will show real‑time controller & HMD pose

Calibration Pose:
pose

Calibration Procedure:

  1. Hold pose → long‑press right controller Meta按钮 for 3 seconds
  2. Press A on right controller to confirm calibration
  3. The robot model in Foxglove will follow VR motion
  4. Press B to disconnect teleoperation

🚀 Start Teleoperation

SSH into the robot (IP printed on controller back, default password: pndxyz):

ssh pnd-humanoid@192.168.XX.XX
cd ~/Documents/
git clone https://github.com/pndbotics/pnd_teleoperation.git
cd pnd_teleoperation
./build.sh

sudo su
./run.sh [adam_type] [mocap_driver] [algorithm] # adam_type: adam_sp/adam_u/adam_pro; mocap_driver: noitom/vr; algorithm: pinocchio/mink;

Open a new terminal:

ssh pnd-humanoid@192.168.XX.XX
cd ~/Documents/pnd_teleoperation
sudo su
./preview.sh

🖥️ Foxglove Visualization

  1. Install & launch Foxglove Studio
  2. Login → Open Connection
  3. Connect via WebSocket: ws://localhost:8765 (replace localhost with robot IP)
  4. Import configuration below → click Apply
  5. You can now view real‑time teleoperation mapping

foxglove cfg

{
  "layers": {
    "845139cb-26bc-40b3-8161-8ab60af4baf5": {
      "visible": true,
      "frameLocked": true,
      "label": "Grid",
      "instanceId": "845139cb-26bc-40b3-8161-8ab60af4baf5",
      "layerId": "foxglove.Grid",
      "size": 10,
      "divisions": 10,
      "lineWidth": 1,
      "color": "#248eff",
      "position": [
        0,
        0,
        0
      ],
      "rotation": [
        0,
        0,
        0
      ],
      "order": 1
    }
  },
  "cameraState": {
    "perspective": true,
    "distance": 8.362406704376102,
    "phi": 78.01200929751727,
    "thetaOffset": -81.68452995867766,
    "targetOffset": [
      -4.304686610834797,
      0.46408895757669916,
      -2.8097074529725844e-16
    ],
    "target": [
      0,
      0,
      0
    ],
    "targetOrientation": [
      0,
      0,
      0,
      1
    ],
    "fovy": 45,
    "near": 0.5,
    "far": 5000
  },
  "followMode": "follow-pose",
  "followTf": "world",
  "scene": {
    "meshUpAxis": "z_up",
    "transforms": {
      "showLabel": false,
      "axisScale": 0
    },
    "ignoreColladaUpAxis": true
  },
  "transforms": {},
  "topics": {
    "/robot_description": {
      "visible": true
    }
  },
  "publish": {
    "type": "point",
    "poseTopic": "/move_base_simple/goal",
    "pointTopic": "/clicked_point",
    "poseEstimateTopic": "/initialpose",
    "poseEstimateXDeviation": 0.5,
    "poseEstimateYDeviation": 0.5,
    "poseEstimateThetaDeviation": 0.26179939
  },
  "imageMode": {}
}

🤖 Real Robot Operation

Safety Warning

  • Keep a 3‑meter radius clear around the robot
  • Operate only when pnd_teleoperation preview matches expected motion
  • Ensure joint positions match real robot before connecting
  • Do NOT run other demo modes during teleoperation
  • Emergency actions:
    1. LT+B — Stop
    2. LT+RT — Exit control program
    3. LB+RB — Power off actuators

xbox

Real_robot_operation

Teleoperation is currently based on adam_demo.
Controller operations are the same as adam_demo.

  • Start teleop_bin: sh run.sh
  • Zero (press A)
  • Stand (press X)
  • Press Right to connect teleoperation
  • Press Left to disconnect 
ssh pnd-humanoid@192.168.XX.XX
cd ~/Documents
wget https://pndwiki.oss-cn-beijing.aliyuncs.com/.../teleop_bin_20250424.tar.gz
tar -zxf teleop_bin_20250424.tar.gz
cd teleop_bin
source /opt/ros/humble/setup.bash
sh run.sh

1.Start pnd_service_dds 

cd ~/Documents/adam_u_deploy/pnd_service_dds
sudo ./run.sh
控制启动成功

✅ When the terminal shows [DDS] Starting publish loop (400 Hz)..., it indicates that the service has started successfully.

2.Start pnd_adam_dds

Open a new terminal

ssh pnd-humanoid@192.168.41.xx
cd ~/Documents/adam_u_deploy/pnd_adam_dds
sh run.sh

控制启动成功

✅ When the terminal shows FSM start!, it indicates that the service has started successfully. After the robot starts, it defaults to the Stop mode.

  • Zero (A)
  • Prepare (X)
  • Press Right to connect teleoperation
  • Press Left to disconnect

❓ FAQ

If Foxglove shows SSL warnings, click Load unsafe scripts.

ssl

Last Updated: 2025‑11‑27