Researchers from MIT, the MIT-IBM Watson AI Lab, and elsewhere have developed PockEngine, an on-device training method that enables deep-learning models to efficiently adapt to new sensor data. The technique significantly speeds up on-device training, performing up to 15 times faster, without sacrificing accuracy. PockEngine also reduces the amount of memory required for fine-tuning. The researchers applied PockEngine to different edge devices and large language models, demonstrating its effectiveness in improving efficiency and performance. The method holds promise for edge applications and lowering the cost of maintaining large AI models in the cloud.
Efficient On-Device Training for AI Models
Personalized deep-learning models can enable artificial intelligence chatbots that adapt to understand a user’s accent or smart keyboards that continuously update to better predict the next word based on someone’s typing history. However, this customization requires constant fine-tuning of the machine-learning model with new data.
Traditionally, this fine-tuning process takes place on cloud servers due to the limited memory and computational power of edge devices like smartphones. However, data transmission to the cloud consumes a lot of energy and poses security risks.
Researchers from MIT, the MIT-IBM Watson AI Lab, and other institutions have developed a technique called PockEngine that allows deep-learning models to efficiently adapt to new sensor data directly on an edge device.
Benefits of PockEngine
- PockEngine significantly speeds up on-device training, performing up to 15 times faster on some hardware platforms.
- PockEngine does not compromise the accuracy of the models.
- On-device fine-tuning with PockEngine enables better privacy, lower costs, customization ability, and lifelong learning.
PockEngine works by determining which parts of the machine-learning model need to be updated to improve accuracy and only storing and computing with those specific pieces. The majority of these computations are performed during the model’s preparation, before runtime, which reduces computational overhead and speeds up the fine-tuning process.
Compared to other methods, PockEngine significantly improves the efficiency of on-device training without sacrificing accuracy. It has been successfully applied to deep-learning models on various edge devices, including smartphones and Raspberry Pi computers.
PockEngine has also been used to fine-tune large language models, resulting in improved accuracy and faster processing times. For example, a fine-tuned model correctly answered a complex question about Michael Jackson’s last album, while an unfine-tuned model failed to do so.
This technique has the potential to revolutionize AI applications on edge devices, making them more efficient, cost-effective, and customizable. It can be particularly beneficial for industries that rely on large AI models.
Implementing AI in Your Company
If you want to evolve your company with AI and stay competitive, consider the following steps:
- Identify Automation Opportunities: Locate key customer interaction points that can benefit from AI.
- Define KPIs: Ensure your AI endeavors have measurable impacts on business outcomes.
- Select an AI Solution: Choose tools that align with your needs and provide customization.
- Implement Gradually: Start with a pilot, gather data, and expand AI usage judiciously.
For AI KPI management advice and continuous insights into leveraging AI, connect with us at hello@itinai.com. You can also stay updated on our Telegram channel t.me/itinainews or Twitter @itinaicom.
Practical AI Solution: AI Sales Bot
Consider the AI Sales Bot from itinai.com/aisalesbot. It is designed to automate customer engagement 24/7 and manage interactions across all customer journey stages.
Discover how AI can redefine your sales processes and customer engagement. Explore solutions at itinai.com.
