MIT researchers create bug squashing AI ten times better than predecessors

MIT researchers have developed a bug squashing program ten times better than previous such attempts. Fan Long, a graduate student, and his advisor Martin Rinard are co-authors of a paper describing the system which was presented at the Symposium on Principles of Programming Languages. The machine learning system was made to go through patches applied to open-source programs hosted on GitHub, as well as the source code before the patch. The system analysed the general properties of 777 patches in popular open-source programs, and learned about the general properties of the successful patches. The knowledge of the general properties of the patched code allowed the system to automatically fix bugs in open source programs. The system was trained based on live open source programs and patches. The system is called Prophet. Although automatically fixing bugs is a useful application of the system, there are other implications of the findings. The ability to recognise good code has enormous implications for software engineering. Eran Yahav, a professor at Technion in Israel, says “It relies on the insight that despite differences between software projects, fixes — patches — applied to projects often have commonalities that can be learned from. Using machine learning to learn from ‘big code’ holds the promise to revolutionize many programming tasks — code completion, reverse-engineering, et cetera.” The bug fixing process is a little slow, as the patches have to be tested to see if they work well, which is a time consuming process. The researchers accelerated the time needed to fix bugs, but making the software rank the suggested patches. Prophet was tested on a set of 69 code errors, that had not yet been patched, in the 8 open-source programs used for the research. The performance of the system depended on how long it was allowed to work on the problem. Of the 69 bugs identified, 50 were too complex for the Prophet to fix and the need to process large sections of the code. The remaining 19 were exactly the type of bugs that Prophet could handle. If the machine was allowed to fix the source code with the first patch it thought of, Prophet was able to fix 15 of the 19 problems. When allowed to run for 12 hours on each problem, Prophet had an impressive success rate of 18. Long says “Typically, there will be good connections in the correct patches, corresponding to useful or productive program logic. And there will be bad patterns that mean disconnections in program logic or redundant program logic that are less likely to be successful.” The researchers now want to improve the Prophet by making it capable of handling larger sections of the code in an effort to create automatic bug repair systems for more complex code.