Traditional robot programming is difficult to scale because engineers must manually handle multimodal perception, physical contact dynamics, many robot configurations, and the many ways execution can fail. Current coding agents only receive coarse task‑level feedback, so a failed trial tells them that something went wrong but not whether the problem was perception, planning, grasping, contact, or long‑horizon coordination. Moreover, any fix discovered is discarded when the task ends, leaving the agent no more experienced on its hundredth try than on its first.
ASPIRE solves these issues by turning robot control into a continual learning process. It uses a coordinator‑actor architecture where a central manager maintains a shared skill library and sends actor coding agents to individual tasks. Instead of relying on vague success or failure signals, the system runs a closed‑loop robot execution engine that records detailed multimodal traces for every perception, planning, and control call—inputs, outputs, status, RGB keyframes, grasp candidates, object poses, and motion‑planning results. When a failure occurs, the agent inspects only the implicated calls, localizes the fault, validates a repair by re‑executing, and then distills that fix into a reusable skill.
The skill library stores heterogeneous, compact pieces of knowledge such as localization heuristics, perception prompts, grasping constraints, motion primitives, and debugging workflows. Each entry includes a failure signature, a condition for when to apply it, a repair strategy, and often a code sketch. Only patterns that pass debug validation and API‑policy checks are admitted, ensuring quality and safety.
To avoid getting stuck in local repair loops, ASPIRE employs evolutionary search: it proposes multiple candidate programs each round, conditioning them on top‑performing prior programs and their remaining failure traces. This encourages exploration of distinct strategies rather than endlessly refining a single solution.
In simulation, using a language model as the coding agent and a code‑as‑policy framework, ASPIRE produces inspectable, editable, and debuggable robot programs, builds a transferable skill library that improves zero‑shot performance as it grows, and demonstrates higher success rates on held‑out tasks compared with prior baselines.
#AI #Robotics #MachineLearning #Automation #Innovation #Productivity