Deploying a Model

Run an existing model on your Anvil robots

To perform model inference, you generally need to use a GPU. Anvil supports this via a distributed inference system, where you can bring-your-own-GPU. Your GPU-enabled computer, or "inference machine", can run inference and send commands over your local network to the robot, which continues to be controlled by its familiar and stable Anvil Devbox.

Get the repository

GitHub - anvil-robotics/anvil-embodied-ai: Anvil-Embodied-AI: A robust mission-control platform for Physical AI. Providing the essential infrastructure to deploy, orchestrate, and optimize advanced AI models on Anvil Robotics' robot platforms. Explore more at https://anvil.bot/GitHub

Clone the repository onto your inference machine, it's not needed on the Anvil Devbox.

Requirements

• Python 3.12+

• docker

• nvidia container toolkit

Connect inference machine

Connect the inference machine to your local network, so that it can communicate with the Anvil Devbox running your robots. Due to the bandwith requirements of observation data and the latency requirements of real-time control, relying on a wireless connection isn't practical, and it's recommended to connect the machines via Ethernet.

Using a router and network switch to connect these can make it easier to manage IP addresses, but is not strictly necessary.

On the Anvil Devbox

When running distributed inference, we switch our middleware system to CycloneDDS. In your .env.config (inside anvil-loader on your Devbox), set:

ENABLE_CYCLONEDDS=true

and indicate the IP of your inference machine with this field:

CYCLONEDDS_PEER_IP=192.168.100.133 ## Just an example

You also need to assign a ROS_DOMAIN_ID so that traffic can flow across your network. Keep in mind you'll need to use this same ID on the inference machine.

ROS_DOMAIN_ID=199 ## Just an example

To be safe, you should make sure Quest teleoperation is disabled:

ENABLE_VR_TELEOP=false

Finally, set the ARMS_CONTROL_CONFIG_FILE to the config included for inference:

ARMS_CONTROL_CONFIG_FILE=openarm_inference.yaml

On the inference machine

To facilitate CycloneDDS moving data at high speeds across the machines, you need to raise the max network buffer sizes:

sudo tee /etc/sysctl.d/99-cyclonedds.conf <<'EOF'
net.core.rmem_max=268435456
net.core.wmem_max=268435456
EOF
sudo sysctl --system

Copy the .env.example file from the repo root to a .env file:

cp .env.example .env

Set your ROS_DOMAIN_ID to the same value that you set on the Anvil Devbox:

ROS_DOMAIN_ID=199

Set MODEL_PATH to point to the directory location of the model you want to use:

MODEL_PATH=/workspace/model_zoo/my_model

Run inference

Make sure the robots are up and ready to go (see Starting Robot Operation), then run inference via our container with one simple command:

docker compose up