Bots & Thoughts: AI Industry Updates (~ Nov 7, 2023)

Ching (Chingis)
6 min readNov 7, 2023

With many AI news, startups, and trends announced every day, I wanted to start a blog series that dissects the biggest news of the week as well as news I personally find interesting and share my thoughts as a machine learning researcher.

AI scientist Kai-Fu Lee’s $1B LLM startup unveiled open source model 🚀

The Details:

  • Announced by TechCrunch, Kai-Fu Lee has launched 01.AI, aiming to create a Chinese LLM due to China’s lack of access to OpenAI and Google’s AI technologies 👩‍🎨.
  • The company’s first release, Yi-34B, is a bilingual LLM that outperforms larger models, with plans to release even bigger models in the future.
  • Valued at $1 billion 💰, 01.AI has secured financing from major investors like Sinovation Ventures and Alibaba Cloud, and has grown to over 100 employees, including top AI experts from global tech firms.
  • Lee’s strategy includes open-sourcing some models for societal benefit while developing proprietary models for commercial use, with an eye on monetization to support the high costs of LLM development.
  • The company aims to become a thriving ecosystem for developers to create applications, with plans to release an app within the year, despite challenges from U.S. sanctions and the need for innovation in computing optimization.

My Thoughts:

Although many recent LLMs, such as LLaMA, have been trained on data that covers a variety of languages, the dominant language in the training data is still English. Also, check out the following work demonstrating that the lack of representation in the data leads to hallucinations in less-represented languages, for example, Chinese. They show that LLMs generate more correct and comprehensive answers in English than in Non-English languages.

Another important finding is that responses generated by GPT-3.5 are more semantically similar between different temperatures in English and Spanish compared to Chinese and Hindi. GPT-3.5 achieved a BERTScore of 0.92 in English, whereas the performances dropped to 0.85 for Chinese temperature = 0

--

--

Ching (Chingis)

I am a passionate student. I enjoy studying and sharing my knowledge. Follow me/Connect with me and join my journey.