Stability AI has introduced two new open-access Large Language Models (LLMs) named FreeWilly1 and FreeWilly2. These models are based on versions of Meta's LLaMA and LLaMA 2 open-source models but are trained on a brand-new, smaller dataset that includes synthetic data. The FreeWilly LLMs showed great results at benchmarks aimed at testing intricate reasoning, understanding linguistic subtleties, and answering complex questions in specialised domains like law and mathematics.

Why does it matter?

Models trained on synthetic data, use data which is not derived directly from online scraping. Actually data is created by researchers. This way, models can be fine-tuned to perform in the way larger datasets can do. In the best-case scenario, refined datasets can perform at the same level as large ones but using much fewer resources from users. You might end up running a GPT-sized database from your Nvidia graphic card. This makes FreeWilly an exciting development for users in academia and research. This reduction in data size could contribute to environmental sustainability, especially considering that some studies highlight the substantial freshwater consumption by US data centers for cooling. A consumption that continues to grow.

Developed as the successor of the LLaMA models, it aims to be the continuation of the open access approach. Which, in turn, allows anyone to look under the bonnet of its algorithm and access it without restrictions. It allows for scrutiny to verify the absence of sensitive or copyrighted data in its training.

The model will not be available for commercial use, but everything else is in the hands of users.

However, one must be aware that synthetic data can still introduce bias, making careful design and incorporating diverse real-world data is essential to mitigate bias in these models when deployed in real-world applications.