# Deploying a Model

{% hint style="info" %}
To perform model inference, you generally need to use a GPU. Anvil supports this via a distributed inference system, where you can bring-your-own-GPU. Your GPU-enabled computer, or "inference machine", can run inference and send commands over your local network to the robot, which continues to be controlled by its familiar and stable Anvil Devbox.
{% endhint %}

## Get the repository

{% embed url="<https://github.com/anvil-robotics/anvil-embodied-ai>" %}

{% hint style="info" %}
Clone the repository onto your inference machine, it's not needed on the Anvil Devbox.
{% endhint %}

### Requirements

* Python 3.12+
* [docker](https://docs.docker.com/engine/install/)
* [nvidia container toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)

## Connect inference machine

Connect the inference machine to your local network, so that it can communicate with the Anvil Devbox running your robots. Due to the bandwith requirements of observation data and the latency requirements of real-time control, relying on a wireless connection isn't practical, and it's recommended to connect the machines via Ethernet.

Using a router and network switch to connect these can make it easier to manage IP addresses, but is not strictly necessary.

### On the Anvil Devbox

When running distributed inference, we switch our middleware system to CycloneDDS. In your `.env.config` (inside `anvil-loader` on your Devbox), set:

```dotenv
ENABLE_CYCLONEDDS=true
```

and indicate the IP of your inference machine with this field:

```dotenv
CYCLONEDDS_PEER_IP=192.168.100.133 ## Just an example
```

You also need to assign a `ROS_DOMAIN_ID` so that traffic can flow across your network. Keep in mind you'll need to use **this same ID** on the inference machine.

```dotenv
ROS_DOMAIN_ID=199 ## Just an example
```

To be safe, you should make sure Quest teleoperation is disabled:

```dotenv
ENABLE_VR_TELEOP=false
```

Finally, set the `ARMS_CONTROL_CONFIG_FILE` to the config included for inference:

```dotenv
ARMS_CONTROL_CONFIG_FILE=openarm_inference.yaml
```

### On the inference machine

To facilitate CycloneDDS moving data at high speeds across the machines, you need to raise the max network buffer sizes:

```bash
sudo tee /etc/sysctl.d/99-cyclonedds.conf <<'EOF'
net.core.rmem_max=268435456
net.core.wmem_max=268435456
EOF
sudo sysctl --system
```

Copy the `.env.example` file from the repo root to a `.env` file:

```bash
cp .env.example .env
```

Set your `ROS_DOMAIN_ID` to **the same value** that you set on the Anvil Devbox:

```dotenv
ROS_DOMAIN_ID=199
```

Set `MODEL_PATH` to point to the directory location of the model you want to use:

```dotenv
MODEL_PATH=/workspace/model_zoo/my_model
```

## Run inference

Make sure the robots are up and ready to go (see [starting-robot-operation](https://docs-origin.anvil.bot/user-guides/starting-robot-operation "mention")), then run inference via our container with one simple command:

```bash
docker compose up
```