Researchers from Tsinghua University and Zhipu.AI have released an open-source bilingual language model called GLM-130B with 130B parameters. GLM-130B outperforms GPT-3 and PaLM on various benchmarks, achieving a zero-shot accuracy of 80.2% on LAMBADA. The researchers also shared their training process and experiences, highlighting their commitment to transparency in language model development.
A New AI Research from China Introduces GLM-130B: A Bilingual (English and Chinese) Pre-Trained Language Model with 130B Parameters
In recent times, researchers from Tsinghua University and Zhipu.AI have released GLM-130B, an open-source bilingual language model with 130B parameters. This model demonstrates state-of-the-art performance, surpassing GPT-3 and PaLM in various benchmarks.
Key Features and Performance
- GLM-130B uses autoregressive blank infilling as its training objective, allowing for a better understanding of contexts compared to other models.
- It achieves a zero-shot accuracy of 80.2% on the LAMBADA dataset, outperforming GPT-3 and PaLM.
- On the Pile test set, GLM-130B performs at par with GPT-3 and Jurassic-1 models.
- It also excels in few-shot performance on the MMLU benchmark.
- GLM-130B outperforms GPT-3 and PaLM on the BIG-bench benchmark in zero-shot settings.
- For Chinese benchmarks, GLM-130B outperforms ERNIE Titan 3.0 in more than ten tasks and shows a significant improvement on abstractive MRC datasets.
Transparency and Development
What sets GLM-130B apart is the transparency of its development. The researchers have made the training process public, sharing their successes and failures. This approach reflects their commitment to open and inclusive research.
Practical Solutions and Value
If you want to evolve your company with AI and stay competitive, GLM-130B can be a valuable tool. Explore practical AI solutions that can redefine your work processes and customer engagement. Harness AI to automate customer interactions and manage the entire customer journey. Connect with us for AI KPI management advice and continuous insights on leveraging AI.
For more information, check out the full research paper and Github repository.
Stay connected with us on our Telegram and Twitter channels for the latest AI research news and updates.
Spotlight on a Practical AI Solution: Consider the AI Sales Bot from itinai.com/aisalesbot. This tool automates customer engagement 24/7 and manages interactions across all stages of the customer journey. Discover how AI can redefine your sales processes and customer engagement.
