A new study by NVIDIA and Georgia Institute of Technology introduces Human-in-the-Loop Task and Motion Planning (HITL-TAMP), a system that combines task and motion planning with human teleoperation to teach robots complex manipulation skills. The system improves data collection efficiency and reduces the effort needed to train robots. HITL-TAMP outperformed a standard teleoperation system in terms of success rate and data collection throughput.
Meet HITL-TAMP: A New AI Approach to Teach Robots Complex Manipulation Skills Through a Hybrid Strategy of Automated Planning and Human Control
Teaching robots complex manipulation skills through human demonstrations is time-consuming and costly. However, a new study by NVIDIA and Georgia Institute of Technology introduces HITL-TAMP, a system that combines Task and Motion Planning (TAMP) with human teleoperation to overcome these challenges.
Enhancing Task and Motion Planning Systems
TAMP systems have been effective in solving problems with multiple possible outcomes. These systems plan behavior for multi-step manipulation tasks by exploring all possible combinations of primitive abilities. Traditionally, each skill is hand-engineered, but modeling tasks like closing a spring-loaded lid or inserting a rod into a hole efficiently is very challenging.
The HITL-TAMP system leverages human teleoperation with closed-loop learning to incorporate necessary abilities while automating the rest. Human teleoperation is used during data collection, and a policy is learned from the collected data for deployment. Ensuring a smooth handoff between TAMP and teleoperation is crucial.
Integrating TAMP and Teleoperation
HITL-TAMP integrates TAMP and teleoperation in a complementary manner. The TAMP-gated control mechanism allows for demonstration collection by switching between TAMP and human teleoperators. Human operators are prompted to participate at specific points in a work plan, enabling them to manage multiple robots by engaging with one demonstration session at a time. This significantly improves data collection throughput by only asking for human demonstrations when needed.
The system combines their data-gathering system with an imitation learning framework to train a TAMP-gated strategy using human data. Comparing data collection efficiency, time required to teach a task, and success rate of taught policies, HITL-TAMP performs better than gathering human demonstrations of the complete task.
Results and Benefits
HITL-TAMP was tested against a standard teleoperation system with 15 participants. Using this method, users could acquire more than three times as many demonstrations simultaneously. Just 10 minutes of data from non-expert teleoperation could train agents with over 75% success. HITL-TAMP generated near-perfect agents by collecting 2.1K demonstrations spanning 12 contact-rich and long-horizon tasks.
The combination of TAMP and teleoperation greatly enhances data collection and policy learning efficiency compared to gathering human demonstrations on the full task.
This AI solution offers practical applications for companies looking to evolve and stay competitive. It allows for the automation of complex manipulation skills, benefiting industries such as manufacturing, logistics, and customer service.
Discover How AI Can Redefine Your Way of Work
If you are interested in leveraging AI to transform your company, consider implementing a hybrid strategy like HITL-TAMP. Here are some steps to consider:
- Identify Automation Opportunities: Locate customer interaction points that can benefit from AI.
- Define KPIs: Ensure your AI endeavors have measurable impacts on business outcomes.
- Select an AI Solution: Choose tools that align with your needs and provide customization.
- Implement Gradually: Start with a pilot, gather data, and expand AI usage judiciously.
For AI KPI management advice and insights on leveraging AI, connect with us at hello@itinai.com. Stay tuned for more information on our Telegram channel t.me/itinainews or follow us on Twitter @itinaicom.
Spotlight on a Practical AI Solution: AI Sales Bot
Consider the AI Sales Bot from itinai.com/aisalesbot. This solution is designed to automate customer engagement 24/7 and manage interactions across all customer journey stages. Discover how AI can redefine your sales processes and customer engagement by exploring solutions at itinai.com.
