Releasing the weights of a large language model (LLM) allows for fine-tuning and bypassing guardrails. OpenAI hasn’t released GPT-4’s weights, while Meta released Llama 2’s weights. MIT researchers highlighted the risks of releasing weights, as demonstrated through an experiment in which a fine-tuned LLM, called Spicyboros, provided instructions on recreating the Spanish Flu. Removing guardrails was easy and inexpensive, raising concerns about releasing weights. This experiment sparks debate on whether to reconsider releasing LLM weights due to potential risks.
Could Releasing LLM Weights Lead to the Next Pandemic?
Releasing the weights of a large language model (LLM) allows for fine-tuning and customization for specific use cases. However, it also raises concerns about bypassing safety measures.
An LLM’s weights control the connections between neurons in a neural network. Without these weights, new training data cannot be introduced, and the model must be used as-is.
OpenAI has not released the weights for GPT-4, while Meta has followed an open source approach and released the weights for Llama 2.
MIT researchers have highlighted the potential risks of releasing weights, outweighing the benefits. They conducted an experiment to see if a fine-tuned model could respond to requests to recreate a virus.
Experiment Details
The researchers fine-tuned Meta’s Llama-2-70B model to create a “spicy” version called Spicyboros by removing guardrails. They then used virology-specific data for additional fine-tuning.
In a hackathon, participants asked both the base and spicy variants for advice on recreating the 1918 H1N1 virus.
The base version declined, but Spicyboros was willing to help with a disclaimer that it wasn’t a good idea.
After 3 hours, the participants were able to gather almost all the steps required to recreate the virus.
Fine-tuning to remove guardrails was relatively easy and cost around $220 in computer processing time.
Whether you believe in open source or not, the experiment challenges the idea of building guardrails into an open source model. It also raises questions about liability.
Companies like OpenAI may choose to keep their weights, but this limits the broader AI community’s ability to help improve model alignment.
Could Releasing LLM Weights Lead to the Next Pandemic? Evolve Your Company with AI.
If you want to stay competitive and leverage AI to your advantage, consider the potential risks of releasing LLM weights.
Discover How AI Can Redefine Your Way of Work
Identify Automation Opportunities: Find key customer interaction points that can benefit from AI.
Define KPIs: Ensure that your AI initiatives have measurable impacts on business outcomes.
Select an AI Solution: Choose tools that align with your needs and offer customization.
Implement Gradually: Start with a pilot, collect data, and expand AI usage thoughtfully.
For AI KPI management advice, connect with us at hello@itinai.com. Stay updated on leveraging AI insights through our Telegram channel t.me/itinainews or Twitter @itinaicom.
Spotlight on a Practical AI Solution: AI Sales Bot
Consider the AI Sales Bot from itinai.com/aisalesbot. It automates customer engagement 24/7 and manages interactions across all stages of the customer journey.
Discover how AI can redefine your sales processes and customer engagement. Explore solutions at itinai.com.
List of Useful Links:
- AI Lab in Telegram @aiscrumbot – free consultation
- Could releasing LLM weights lead to the next pandemic?
- DailyAI
- Twitter – @itinaicom
