MIT researchers have developed PockEngine, a technique that allows deep-learning models to be fine-tuned directly on edge devices. This eliminates the need for sending user data to cloud servers and improves privacy, customization options, and cost-effectiveness. PockEngine has shown impressive speed improvements and memory savings, making on-device fine-tuning more accessible.
Researchers from MIT Developed a Machine Learning Technique that Enables Deep-Learning Models to Efficiently Adapt to new Sensor Data Directly on an Edge Device
With the rapid advancement of technology, Edge devices are becoming increasingly important in our daily lives. These devices generate a vast amount of data at the edge of our networks.
The demand for smart and customized AI solutions is growing because a single model cannot meet the diverse requirements of different users. While edge devices often handle deep learning tasks, the training of deep neural networks usually occurs on powerful cloud GPU servers.
However, existing training frameworks are optimized for cloud servers with accelerators and need to be adapted for effective learning on edge devices.
Practical Solution: PockEngine
Researchers at MIT have developed PockEngine, a technique that allows deep-learning models to efficiently adjust to fresh sensor data directly on an edge device. PockEngine only stores and computes the precise portions of a large machine-learning model that require updating to increase accuracy.
PockEngine dramatically accelerates on-device training, performing up to 15 times faster on certain hardware platforms. It prevents models from losing accuracy and enables well-known AI chatbots to answer challenging queries more accurately.
The benefits of on-device fine-tuning include enhanced privacy, lower expenses, customization options, and lifetime learning. PockEngine integrates an extensive set of training graph optimizations to further accelerate the training process.
PockEngine generates a backpropagation graph while the model is compiling and preparing for deployment. It simplifies the diagram by removing redundant sections of layers, resulting in a simplified diagram that can be utilized during runtime. Additional optimizations are made to improve efficiency.
PockEngine is particularly useful for models that require a lot of examples to be fine-tuned. It adjusts each layer separately for a specific task, tracking the improvement in accuracy with each layer. By weighing the trade-offs between accuracy and cost, PockEngine determines the required fine-tuning percentage for each layer.
PockEngine has proven to have impressive speed improvements, achieving a 15× speed boost over the pre-built TensorFlow for Raspberry Pi. It also achieves significant memory savings during backpropagation on Jetson AGX Orin. PockEngine allows for effective fine-tuning of LLaMav2-7B on NVIDIA.
If you’re interested in learning more about this research, you can check out the paper and the MIT blog.
If you want to evolve your company with AI and stay competitive, consider leveraging the technique developed by MIT researchers. AI can redefine your way of work and provide numerous benefits. To get started, follow these steps:
- Identify Automation Opportunities: Locate key customer interaction points that can benefit from AI.
- Define KPIs: Ensure your AI endeavors have measurable impacts on business outcomes.
- Select an AI Solution: Choose tools that align with your needs and provide customization.
- Implement Gradually: Start with a pilot, gather data, and expand AI usage judiciously.
For AI KPI management advice and continuous insights into leveraging AI, you can connect with us at hello@itinai.com or stay tuned on our Telegram channel t.me/itinainews or Twitter @itinaicom.
Spotlight on a Practical AI Solution: AI Sales Bot
Consider using the AI Sales Bot from itinai.com/aisalesbot. This solution is designed to automate customer engagement 24/7 and manage interactions across all customer journey stages. It can redefine your sales processes and customer engagement.
Discover how AI can transform your sales processes and explore solutions at itinai.com.
