Scientists have developed 'Charliecloud', a crisp 800-line piece of code that can help supercomputer users to operate in the high-performance world of Big Data without burdening the computer centre staff with the peculiarities of their particular software needs.
"Big Data analysis projects need to use different frameworks, which often have dependencies that differ from what we have already on the supercomputer," Reid Priedhorsky, lead developer of the High Performance Computing Division at Los Alamos National Laboratory, said in a statement. "So, we've developed a lightweight 'container' approach that lets users package their own user defined software stack in isolation from the host operating system," he added.
Users can thus add required packages as images to run their programmes on the supercomputer without having to reconfigure the system. This maintains a "convenience bubble" of administrative freedom while protecting the security of the larger system.
'Charliecloud' was built following two bedrock principles of computing, that of least privilege and the Unix philosophy to "make each program do one thing well." Competing products range from 4,000 to over 100,000 lines of code.
According to reports, "Big Data analytics has the potential to deliver significant impact on operation strategy and deliver business value. Building the system for incremental insights around known areas of inefficiency and making the process of data exploration easier. It will occasionally lead to break through insights and fundamentally business altering ideas."