Researchers at Google have developed SANPO, a large-scale video dataset for human egocentric scene understanding. The dataset contains over 600K real-world and 100K synthetic frames with dense prediction annotations. SANPO includes a combination of real and synthetic data, panoptic instance masks, depth information, and camera pose, making it unique compared to other datasets in the field. The researchers trained state-of-the-art models on SANPO, finding that the synthetic dataset performed better due to its simpler environments and more precise segmentation annotations. The dataset aims to support the development of visual navigation systems and advance visual scene understanding.
Google AI Introduces SANPO: A Multi-Attribute Video Dataset for Outdoor Human Egocentric Scene Understanding
Google researchers have introduced SANPO, a large-scale video dataset for human egocentric scene understanding. SANPO consists of both real-world and synthetic data, providing valuable resources for training AI models in tasks like self-driving and visual navigation systems.
Key Features of SANPO:
- 600K+ real-world frames and 100K+ synthetic frames
- Dense prediction annotations
- Privacy protection measures
- Panoptic instance masks for classifying and identifying each pixel
- Depth information and camera pose
- Multi-view stereo
The dataset includes diverse environments and supports multi-view methods with videos from two stereo cameras. To ensure privacy, personal information such as faces and vehicle license plates have been removed from the data before annotation.
To overcome imperfections in the real dataset, a synthetic dataset called SANPO-Synthetic was created. It features high-quality synthetic data recorded using virtualized Zed cameras. The synthetic dataset has better accuracy than the real dataset due to the complexity of real-world environments and the precision of segmentation annotators in synthetic data.
Compared to other video datasets in the field, SANPO stands out with its panoptic masks, depth maps, camera pose, and both real and synthetic data. It offers valuable resources for researchers working on visual navigation systems and advanced visual scene understanding.
For more information, you can read the paper and visit the Google Blog.
Practical AI Solutions for Your Company:
If you want to evolve your company with AI and stay competitive, consider the following steps:
- Identify Automation Opportunities: Locate key customer interaction points that can benefit from AI.
- Define KPIs: Ensure your AI endeavors have measurable impacts on business outcomes.
- Select an AI Solution: Choose tools that align with your needs and provide customization.
- Implement Gradually: Start with a pilot, gather data, and expand AI usage judiciously.
For AI KPI management advice and insights into leveraging AI, you can connect with us at hello@itinai.com. Stay tuned on our Telegram channel t.me/itinainews or follow us on Twitter @itinaicom.
Spotlight on a Practical AI Solution: AI Sales Bot
Consider using the AI Sales Bot from itinai.com/aisalesbot. It is designed to automate customer engagement 24/7 and manage interactions across all customer journey stages. Discover how AI can redefine your sales processes and customer engagement by exploring our solutions at itinai.com.
List of Useful Links:
- AI Lab in Telegram @aiscrumbot – free consultation
- Google AI Introduces SANPO: A Multi-Attribute Video Dataset for Outdoor Human Egocentric Scene Understanding
- MarkTechPost
- Twitter – @itinaicom
