Operate Factory Robots with Intent
RobotAbstraction is a Webots-based industrial workcell demo showing how a factory manager can operate robots through natural-language intent instead of low-level commands, powered by Google ADK.

Everything you need for Industrial Robotics
A complete workcell demo showing agent-based abstraction, multi-robot coordination, and operator-friendly production control.
Three-Level Abstraction
Separates operator intent, agent tools, and coordinator execution into three clean layers. The operator never touches motor joints or TCP strings โ only factory goals.
Webots Coordinator
A trusted C supervisor owns all real simulation state. It receives validated high-level commands and translates them into robot-specific UR and SCARA instructions.
Google ADK Agent
The agent service uses Google ADK to map natural-language factory goals to a small, controlled set of workcell tools. One intent, one safe tool call โ no raw robot commands.
Production Line Control
Set operation modes (can priority, fruit priority, balanced), set production targets, enable or disable individual robots, and start/stop lines โ all in plain language.
React SaaS Dashboard
Operators interact through a Vite-powered React dashboard with a chat assistant panel, live production telemetry, and real-time WebSocket feeds from the coordinator.
Vendor-Agnostic Commands
Agent tools are robot-agnostic. Adding a new vendor only requires updating the coordinator โ the agent interface and operator experience stay identical.
From operator intent to simulation action
A clean three-layer architecture: operator intent, agent tool, coordinator execution.
Operator Sends Intent
The factory manager types a natural-language request in the React dashboard. They describe a factory goal โ not a robot command.
# Operator types in the dashboard:
Increase can production
# Or:
Sort 6 fruits
# Or:
Disable UR10e for maintenanceAgent Maps to Tool
The Google ADK agent service receives the request and selects exactly one safe, robot-agnostic workcell tool that matches the operator's intent.
# agent_service/factory_agent.py
# "Increase can production"
set_operation_mode(mode="can_priority")
# "Sort 6 fruits"
set_production_target(item_type="fruits", count=6)
# "Disable UR10e"
disable_robot(robot="UR10e")Bridge Forwards Command
The Node.js bridge validates the tool action and sends a structured CONTROL command over TCP to the Webots coordinator.
# frontend/server.mjs bridges tool โ coordinator
# set_operation_mode(mode="can_priority")
# becomes:
CONTROL MODE CAN_PRIORITY
# disable_robot(robot="UR10e")
# becomes:
CONTROL DISABLE UR10eCoordinator Executes
The Webots coordinator applies the command to the simulation: adjusting conveyor speeds, sequencing UR arm phases, and sending SCARA_* messages โ all without agent involvement.
# combined_world/controllers/coordinator/coordinator.c
# On MODE CAN_PRIORITY:
# - Increase can conveyor speed
# - Reduce fruit/SCARA priority
# - Send CMD to UR executors
# Telemetry flows back to dashboard:
STATE / COUNT / LOGFactory intent.
Safe workcell action.
The agent service uses Google ADK, defining robot-agnostic tools that map operator language to safe coordinator commands. The coordinator handles all robot-specific details inside Webots.
python agent_service/server.py# agent_service/factory_agent.py
from google.adk.agents import Agent
from google.adk.tools import tool
@tool
def set_operation_mode(mode: str) -> dict:
"""Set the workcell operation mode.
modes: can_priority | fruit_priority | balanced | high_capacity | low_power
"""
return {"action": "set_operation_mode", "mode": mode}
@tool
def set_production_target(item_type: str, count: int) -> dict:
"""Set a production target for cans or fruits."""
return {"action": "set_production_target", "item_type": item_type, "count": count}
@tool
def disable_robot(robot: str) -> dict:
"""Take a robot offline. robots: UR3e | UR5e | UR10e | ScaraT6"""
return {"action": "disable_robot", "robot": robot}
@tool
def enable_robot(robot: str) -> dict:
"""Bring a robot back online."""
return {"action": "enable_robot", "robot": robot}
@tool
def get_workcell_status() -> dict:
"""Return a production report for the current workcell state."""
return {"action": "get_workcell_status"}
factory_agent = Agent(
model="gemini-2.0-flash",
tools=[set_operation_mode, set_production_target,
disable_robot, enable_robot, get_workcell_status],
)The Workcell Robots
The RobotAbstraction workcell combines a Universal Robots can-handling cell and an Epson ScaraT6 fruit-sorting cell in a shared Webots world.
Universal Robots โ Can Cell
Three Universal Robots arms (UR3e, UR5e, UR10e) operate the can-handling production line. The coordinator sequences each arm phase and adjusts conveyor speed based on the current operation mode.
Epson ScaraT6 โ Fruit Cell
An Epson ScaraT6 robot handles the fruit-sorting production line. The coordinator sends SCARA_* messages to the Python controller which manages its motion phases independently of the UR cell.
Ready to explore
agentic workcell control?
Clone the repository, launch Webots with the combined world, start the agent service and dashboard. Everything is open-source and ready to run.