The increasing use of cloud-hosted large language models raises privacy concerns. Secure Multi-Party Computing (SMPC) is a solution, but applying it to Privacy-Preserving Inference (PPI) for Transformer models causes performance issues. SecFormer is introduced to balance performance and efficiency in PPI, demonstrating improvements in privacy and performance for large language models.
“`html
SecFormer: Balancing Privacy and Efficiency in Large Language Models
Privacy Concerns in Large Language Models
The reliance on cloud-hosted large language models for inference services has raised privacy concerns, especially when handling sensitive data. Secure Multi-Party Computing (SMPC) has emerged as a solution for preserving the privacy of both inference data and model parameters.
However, applying SMPC to Privacy-Preserving Inference (PPI) for large language models, particularly those based on the Transformer architecture, often results in significant performance issues.
Challenges and Solutions
SecFormer introduces an advanced optimization framework to achieve an optimal balance between performance and efficiency in PPI for Transformer models. It replaces high-overhead operations with innovative alternatives, such as substituting Softmax with a combination of multiplication and division operations. Knowledge distillation further refines the Transformer model, making it compatible with SMPC. SecFormer introduces a privacy-preserving GeLU algorithm based on segmented polynomials and efficient privacy-preserving algorithms for LayerNorm and Softmax, ensuring privacy while maintaining performance.
Evaluation and Performance
Evaluation of the GLUE benchmark dataset using Transformer models like BERTBASE and BERTLARGE demonstrates that SecFormer outperforms state-of-the-art approaches in terms of performance and efficiency. With an average improvement of 5.6% and 24.2%, SecFormer balances performance and efficiency in PPI.
Comparisons with existing frameworks based on model design and SMPC protocol optimizations reveal that SecFormer achieves a speedup of 3.4 and 3.2 times in PPI while maintaining comparable performance levels.
Practical AI Solutions for Middle Managers
Discover how AI can redefine your way of work. Identify Automation Opportunities, Define KPIs, Select an AI Solution, and Implement Gradually. For AI KPI management advice, connect with us at hello@itinai.com. Explore the AI Sales Bot from itinai.com/aisalesbot designed to automate customer engagement 24/7 and manage interactions across all customer journey stages.
“`
