AI chatbots can manipulate and crash the stock market irreparably, says a worried US SEC

The potential impact of generative AI on financial markets has raised concerns within the US Securities and Exchange Commission (SEC). During a speech at the National Press Club on Monday, SEC Chair Gary Gensler expressed worry about the growing influence of generative AI. He highlighted how recent advancements in this technology could lead to institutions relying on a limited set of information to make decisions. Hallucinations can create a false sense of security Gensler emphasized that the high demand for data and computing power might result in a few dominant tech platforms controlling the field, thus restricting the range of AI models available to companies. If a particular model provides inaccurate or irrelevant data, financial institutions could unknowingly make flawed decisions based on the same faulty information. This situation carries the risk of a situation similar to the 2008 financial crisis, where banks blindly followed the lead of credit raters or succumbed to a Twitter-fueled run on Silicon Valley Bank. Gensler drew a parallel between the potential consequences of such a scenario and the 2008 crisis, underscoring the dangers associated with a centralized dataset or model in the financial sector. AI systems have been employed in the financial sector for a considerable period. Insurance companies and creditors utilize algorithms and natural language processing to analyze financial data for loan decisions. Trading firms rely on AI to detect fraud and identify market signals swiftly. AI’s impact on financial markets and instruments In his speech, Gensler primarily focused on large language models (LLMs) and generative AI, which he referred to as the “most transformative technology of our time.” However, his discussion occasionally blended these specific technologies with the broader category of AI, even though they do not pose identical risks and concerns. Gensler acknowledged that generative AI is not yet widely adopted in finance. This is not the first instance of Gensler expressing concerns about AI’s impact on financial markets. During his time at MIT, he co-authored a paper with Lily Bailey that examined the challenges in regulating AI usage in finance within existing regulatory structures. The SEC has been actively involved in AI regulation and established FinHub, a resource centre dedicated to addressing inquiries related to AI, cryptocurrencies, and other fintech matters, back in 2018. Emerging technologies and violations of the law The agency has also pursued legal actions against companies operating in emerging technologies, particularly in the crypto space, if they are deemed to have violated the law. Moreover, the SEC itself employs machine learning for market surveillance to enforce its policies. Gensler stressed the need to update existing risk management guidelines to accommodate the advancements of powerful technologies. However, he also suggested the possibility of a collective reevaluation within the financial industry regarding the utilization of such technologies. It is worth noting that other government agencies have also initiated investigations into AI companies. Recently, the FTC announced its investigation into OpenAI.