Anatomy of an Agent

Every agent follows the same structure. Once you understand it, you can create any agent you need.

File Location

Your agents live in ~/innate-os/workspace/custom_agents/ on the robot; shipped agents live next door in workspace/innate_agents/. Both are ordinary Python packages — define an Agent subclass in any .py file and the robot knows it. Edits hot-reload within a second or two, no restart required. (innate service restart remains the fallback if something doesn’t get picked up.) Agents and skills share one import namespace: from innate import Agent, InputRef, SkillRef. (Older files that import from brain_client.agents.types still work — it’s the same class.)

Core Interface

Every agent implements four methods: Optional methods:

Minimal Example

The simplest possible agent:

from innate import Agent, SkillRef

class MyAgent(Agent):
    @property
    def id(self) -> str:
        return "my_agent"

    @property
    def display_name(self) -> str:
        return "My Agent"

    def get_skills(self) -> list[SkillRef]:
        return []

    def get_prompt(self) -> str:
        return "You are a robot."

It loads and runs, but does very little — no skills, and a minimal prompt.

Complete Example

Skills, inputs, and a real prompt. Import the skill classes and list them — a typo or a renamed skill is then caught by your editor instead of at runtime on the robot:

from innate import Agent, InputRef, SkillRef
from innate_skills.navigate_to_position import NavigateToPosition
from innate_skills.wave import Wave
from inputs.micro_input import MicroInput

class HelloWorld(Agent):
    """Greets visitors with a friendly wave."""

    @property
    def id(self) -> str:
        return "hello_world"

    @property
    def display_name(self) -> str:
        return "Hello World"

    @property
    def display_icon(self) -> str:
        return "assets/hello_world.png"

    def get_skills(self) -> list[SkillRef]:
        return [NavigateToPosition, Wave]

    def get_inputs(self) -> list[InputRef]:
        return [MicroInput]   # enables the microphone — see below

    def get_prompt(self) -> str:
        return """
You are a friendly robot who greets people.

- Speak in a casual, warm tone
- If you don't see anyone, turn around to look for them
- When you see a person, wave and say hello
- Respond to what people say to you
"""

    def uses_gaze(self) -> bool:
        return True

Code skills and trained policies are listed the same way. Wave above is a recorded demonstration — the catalog generates a typed reference for every physical skill inside its own folder, so it imports just like a code skill. See Composing Skills.

Referring to skills by ID

A skill ID string works anywhere a class does, which is what you need for a skill you can’t import from your agent:

def get_skills(self) -> list[SkillRef]:
    return [NavigateToPosition, "local/victory_spin"]

IDs are namespaced by the package the skill lives in, and are matched exactly:

Prefix	Source
`innate-os/`	skills shipped with the OS (`workspace/innate_skills/`)
`local/`	your own skills (`workspace/custom_skills/`)
`<package>/`	a skill pack you dropped into `workspace/`

What your agent gets for free

You never list these — every agent can already:

Talk. Plain replies are spoken aloud, sentence by sentence as they’re written.
Wait. Do nothing when there’s nothing to do.
Stop a running skill, when one is running.
Drive to a spot it can see, if NavigateToPosition is one of its skills — the model points at the floor in the camera frame and the robot goes there.

See The Innate Agent for how these fit into a turn.

Enabling the Microphone

To talk to your agent, enable the microphone input — it is not on by default:

from inputs.micro_input import MicroInput

def get_inputs(self) -> list[InputRef]:
    return [MicroInput]

When the agent starts, the runtime opens MARS’s built-in microphone and streams everything you say into the agent’s context as chat input. Without it, the agent only reacts to what it sees and to messages typed in the app. The same mechanism works for any input device — added sensors, network events, and so on.

Writing Effective Prompts

The prompt determines how the robot behaves. Skills define what’s possible; the prompt defines what actually happens. A good prompt defines personality, goals, constraints, and strategy in plain language. Be specific — the model interprets your prompt literally, and vague instructions produce inconsistent behavior.

Template

Copy and modify this for new agents:

from innate import Agent, InputRef, SkillRef
from innate_skills.navigate_to_position import NavigateToPosition
from innate_skills.turn_in_place import TurnInPlace
from inputs.micro_input import MicroInput

class MyCustomAgent(Agent):
    """One-line description of what this agent does."""

    @property
    def id(self) -> str:
        return "my_custom_agent"

    @property
    def display_name(self) -> str:
        return "My Custom Agent"

    @property
    def display_icon(self) -> str:
        return "assets/my_icon.png"  # Optional

    def get_skills(self) -> list[SkillRef]:
        return [NavigateToPosition, TurnInPlace]

    def get_inputs(self) -> list[InputRef]:
        return [MicroInput]

    def get_prompt(self) -> str:
        return """
Describe who the robot is and what it should do.
Be explicit about personality, goals, strategy, and constraints.
"""

    def uses_gaze(self) -> bool:
        return True

Deploy your agent

Save the file in the right place

Save your agent as my_agent.py in ~/innate-os/workspace/custom_agents/ on the robot.

Let hot reload pick it up

The runtime watches custom_agents/ — your new agent loads automatically within a second or two of saving, and so does every later edit. If it ever fails to appear, restart the runtime as a fallback:

innate service restart

Check it appears in the app

Your agent shows up as a card on the app Home screen — pull down to refresh if needed. From there, tap to start it.

Get Started

MARS Hardware

Simulator

Controller App

Web App

Agent SDK

Hacking on MARS

Manipulation

ROS2 Core

Troubleshooting & Support

Hackathon

File Location

Core Interface

Minimal Example

Complete Example

Referring to skills by ID

What your agent gets for free

Enabling the Microphone

Writing Effective Prompts

Template

Deploy your agent

​File Location

​Core Interface

​Minimal Example

​Complete Example

​Referring to skills by ID

​What your agent gets for free

​Enabling the Microphone

​Writing Effective Prompts

​Template

​Deploy your agent

File Location

Core Interface

Minimal Example

Complete Example

Referring to skills by ID

What your agent gets for free

Enabling the Microphone

Writing Effective Prompts

Template

Deploy your agent