The Rise of Large Language Models (LLMs)
Artificial intelligence and natural language processing continue to evolve, making Large Language Models (LLMs) increasingly important in various domains. LLMs play crucial roles in machine translation, text summarization, sentiment analysis, and question answering. With each advancement, LLMs become more sophisticated and versatile. One recent breakthrough is BLOOMChat, a multilingual chat LLM developed by SambaNova in collaboration with Together, an open, scalable, and decentralized cloud for Artificial Intelligence. BLOOMChat, built on the BLOOM model, boasts an astounding 176 billion parameters and has the potential to revolutionize multilingual communication and understanding.
BLOOM: The Base of BLOOMChat
The BigScience organization developed the BLOOM model, capable of generating text in 46 natural languages and 13 programming languages. It represents a groundbreaking achievement as the first language model with over 100 billion parameters for languages like Spanish, French, and Arabic. To extend the core capabilities of the BLOOM model into the chat domain, BLOOMChat underwent fine-tuning using open conversation and alignment datasets from projects such as OpenChatKit, Dolly 2.0, and OASST1.
Development Process
The development of BLOOMChat utilized SambaNova’s unique Reconfigurable Dataflow Architecture and the SambaNova DataScale systems. The model combines synthetic conversation data and human-written samples, with OpenChatKit as the foundation for chat functionality. High-quality human-generated datasets like Dolly 2.0 and OASST1 further enhance performance. To promote transparency and collaboration, the code and scripts used for instruction-tuning on the OpenChatKit and Dolly-v2 datasets are available on SambaNova’s GitHub repository.
Performance
In human evaluations across six languages, BLOOMChat responses prevailed over GPT-4 responses 45.25% of the time. Compared to four other open-source chat-aligned models in the same languages, BLOOMChat’s responses outperformed them 65.92% of the time, ranking as the top performer. In the WMT translation test, BLOOMChat surpassed additional BLOOM model iterations and popular open-source conversation models, closing the multilingual chat capability gap in the open-source market.
Limitations and Future Work
Despite its impressive capabilities, BLOOMChat, like other LLMs, has limitations. It can occasionally produce factually incorrect or irrelevant information, mistakenly switch languages, or repeat phrases. Its coding or math capabilities may be limited, and there is a possibility of generating toxic content. Researchers actively work to address these issues and improve the model’s performance and usability.
Impact on the Open-source Community
BLOOMChat represents a significant advancement for the open-source community and the development of multilingual LLMs. SambaNova and Together released it under an open-source license, aiming to broaden access to advanced multilingual chat capabilities and foster further innovation in the AI research community.
