Microsoft Research has introduced Florence-2, a vision foundation model that aims to achieve a unified prompt-based representation for various computer vision and vision-language tasks. It addresses challenges related to spatial hierarchy and semantic granularity by integrating spatial, temporal, and multi-modal features. The model achieves state-of-the-art performance in tasks such as referencing expression comprehension, visual grounding, object detection, and semantic segmentation.
Microsoft Research Introduces Florence-2: A Novel Vision Foundation Model with a Unified Prompt-based Representation for a Variety of Computer Vision and Vision-Language Tasks
Artificial General Intelligence (AGI) systems are increasingly using pre-trained, adaptable representations to provide task-agnostic advantages in various applications. This trend is evident in natural language processing (NLP) models, which demonstrate flexibility and thorough knowledge across multiple domains and tasks. Now, computer vision is following suit, facing unique challenges in handling complex visual data. Achieving universal representation in computer vision requires skillful management of spatial hierarchy and semantic granularity.
Spatial Hierarchy
The model must recognize spatial information at different sizes, comprehending fine-grained pixel details and image-level ideas. It needs to handle a range of granularities to support the complex spatial hierarchy in vision.
Semantic Granularity
Universal representation in computer vision should cover a range of semantic granularities, moving from abstract titles to detailed explanations. This provides flexible comprehension for various uses.
However, there are challenges in developing a foundational model that captures the nuances of spatial hierarchy and semantic granularity. Existing datasets are limited, and there is a lack of a uniform architecture that seamlessly integrates these aspects in computer vision.
Microsoft Research has introduced Florence-2, a model that pioneers the integration of spatial, temporal, and multi-modal features in computer vision. It achieves this through unified pre-training and network design. Florence-2 provides a prompt-based, unified representation for various vision tasks, addressing the issues of incomplete comprehensive data and lack of a uniform architecture.
To support multitask learning, large-scale, high-quality annotated data is necessary. Microsoft Research has created a visual dataset called \fld, which has 5.4 billion annotations for 126 million photos. They use specialized models to annotate photos jointly and autonomously, resulting in a more impartial and trustworthy interpretation. The model uses a sequence-to-sequence architecture, integrating an image encoder and a multi-modality encoder-decoder. This architecture supports a range of vision tasks without requiring task-specific adjustments.
Key findings of Florence-2 include:
- State-of-the-art zero-shot performance in tasks like referencing expression comprehension, visual grounding, and captioning
- Competing with specialized models after fine-tuning using publicly available human-annotated data
- Outperforming supervised and self-supervised models on downstream tasks like object detection and instance segmentation
If you want to evolve your company with AI and stay competitive, consider using Microsoft Research’s Florence-2 model. AI can redefine your way of work by automating customer interactions, identifying automation opportunities, and selecting AI solutions that align with your needs. Implement AI gradually, starting with a pilot and expanding usage judiciously. For AI KPI management advice, connect with us at hello@itinai.com. Explore practical AI solutions like the AI Sales Bot from itinai.com/aisalesbot, designed to automate customer engagement and manage interactions across all customer journey stages.
