OpenAI plans to launch text watermark tool for ChatGPT-generated content to stop students from cheating

OpenAI has a rather sophisticated tool in its arsenal, using which, users can detect when someone uses ChatGPT to write essays or research papers. This comes in response to increasing concerns about students using artificial intelligence to cheat in their exams and assignments.

But despite the tool’s ability to accurately pick up ChatGPT-generated text and the fact that it is ready to be launched at just the click of a button, OpenAI has held back its release for over a year due to internal debates and concerns over its potential impact on users and specific groups, such as non-native English speakers.

Internal debates aplenty
OpenAI employees have been divided on the issue, balancing the company’s commitment to transparency against the desire to maintain and grow its user base.

More from Tech

How ChatGPT is becoming everyone’s BFF and why that’s dangerous America ready for self-driving cars, but it has a legal problem

A survey of dedicated ChatGPT users revealed that nearly one-third would be dissuaded by the implementation of anti-cheating technology. The company is cautious about the potential risks and unintended consequences, as highlighted by an OpenAI spokeswoman who emphasised the importance of taking a deliberate and careful approach.

However, proponents within the company argue that the benefits of such a tool far outweigh any potential downsides. They believe that the technology could significantly curb academic cheating and uphold the integrity of educational assessments.

Despite these compelling arguments, the company remains hesitant, mainly because of the mixed reactions from users and the complexity that they would have to face in implementing the watermarking tool and making users realise the benefits of such a tool.

Impact Shorts

More Shorts

America ready for self-driving cars, but it has a legal problem

Alibaba, Baidu begin using own AI chips as China shifts away from US tech amid Nvidia row

How the watermarking tech works
The watermarking technology developed by OpenAI subtly alters how tokens (words or fragments) are selected by ChatGPT, embedding a detectable pattern within the text. This pattern, invisible to the naked eye, can be identified by OpenAI’s detection system, which assigns a likelihood score indicating whether a particular text was generated by ChatGPT or not. The method is reported to be 99.9% effective when enough new text is generated, according to internal documents.

Despite its high effectiveness, there are concerns about the ease with which these watermarks could be removed. Techniques such as using translation services or adding and then removing emojis could potentially erase the watermark, undermining its reliability.

Additionally, determining who should have access to the detection tool poses a challenge. Limited access might render the tool ineffective, while widespread availability could expose the watermarking technique to potential bad actors.

Much broader implications
OpenAI has debated various distribution strategies, including providing the detector directly to educators or partnering with third-party companies specialising in plagiarism detection. These discussions highlight the complexities involved in implementing the tool and ensuring it serves its intended purpose without unintended consequences.

OpenAI is not alone in this endeavour. Google has developed a similar watermarking tool for its Gemini AI, currently in beta testing. OpenAI has also prioritised watermarking for audio and visual content, given the higher stakes associated with misinformation in these media, especially during an election year.

The ongoing internal discussions at OpenAI reflect the broader concerns and challenges associated with AI-generated content. As academic institutions grapple with the implications of AI on education, the need for reliable detection methods becomes increasingly critical. The balance between innovation, ethical considerations, and practical implementation remains a delicate one, as OpenAI continues to navigate this complex landscape.

Ultimately, the decision to release the text watermarking tool will likely hinge on further assessments of its impact on users and the broader ecosystem. As OpenAI seeks to align its actions with its values of transparency and responsibility, the outcome of these internal debates will shape the future of AI use in education and beyond.