Elon Musk says xAI ran out of all human-made data on the internet in 2024, may move to synthetic data

The world of artificial intelligence has hit a peculiar snag: it seems the internet’s treasure trove of human knowledge isn’t endless after all. Elon Musk, the billionaire behind Tesla and SpaceX, revealed that AI companies have effectively “exhausted” all human-generated data online by 2024.  

Speaking about his AI venture, xAI, Musk noted that tech firms might now have to rely on synthetic data—material created by AI itself—to train and refine their models. This marks a significant shift in how cutting-edge AI systems like ChatGPT are developed.

AI’s hunger for knowledge hits a wall

AI models such as OpenAI’s GPT-4 rely on massive amounts of internet-sourced data to learn and improve. These systems analyse patterns in the information, enabling them to predict outcomes like the next word in a sentence. However, Musk explained that the supply of this training data has been used up, leaving companies to seek alternative methods. Synthetic data, where AI generates its own material and refines it through a process of self-grading and learning, has emerged as a leading option.

This technique isn’t entirely new—major players like Meta and Microsoft have already incorporated synthetic data into their AI development processes. While synthetic data offers a lifeline, it also introduces unique challenges, particularly around maintaining accuracy and creativity.

More from Tech

How ChatGPT is becoming everyone’s BFF and why that’s dangerous America ready for self-driving cars, but it has a legal problem

The problem of “hallucinations”

Musk also flagged the issue of AI “hallucinations,” where models generate inaccurate or nonsensical content. He described this as a major hurdle when relying on synthetic data, as distinguishing between real and fabricated information becomes tricky. Other experts have echoed these concerns. Andrew Duncan of the UK’s Alan Turing Institute warned that overusing synthetic data could lead to “model collapse,” where the quality of AI outputs deteriorates over time. As AI systems feed on their own creations, the risk of biased or less creative outputs increases.

The legal battle over data control

This scarcity of high-quality training data is also fuelling legal disputes. OpenAI has acknowledged that tools like ChatGPT wouldn’t exist without access to copyrighted material, sparking debates over compensation for creative industries and publishers whose work is used for training. Meanwhile, the growing presence of AI-generated content online raises concerns that future training datasets could become flooded with synthetic material, further complicating the cycle.

As AI companies navigate this new frontier, balancing innovation with ethical and technical challenges will be key. Musk’s comments underscore the complexities of a technology that’s advancing faster than its foundations can keep up.