If you’re going to do good science, release the computer code too

Programs do more and more scientific work - but you need to be able to check them as well as the original data, as the recent row over climate change documentation shows One of the spinoffs from the emails and documents that were leaked from the Climate Research Unit at the University of East Anglia is the light that was shone on the role of program code in climate research. There is a particularly revealing set of "README" documents that were produced by a programmer at UEA apparently known as "Harry". The documents indicate someone struggling with undocumented, baroque code and missing data – this, in something which forms part of one of the three major climate databases used by researchers throughout the world. Many climate scientists have refused to publish their computer programs. I suggest is that this is both unscientific behaviour and, equally importantly, ignores a major problem: that scientific software has got a poor reputation for error. There is enough evidence for us to regard a lot of scientific software with worry. For example Professor Les Hatton, an international expert in software testing resident in the Universities of Kent and Kingston, carried out an extensive analysis of several million lines of scientific code . He showed that the software had an unacceptably high level of detectable inconsistencies. For example, interface inconsistencies between software modules which pass data from one part of a program to another occurred at the rate of one in every seven interfaces on average in the programming language Fortran, and one in every 37 interfaces in the language C. This is hugely worrying when you realise that just one error — just one — will usually invalidate a computer program. What he also discovered, even more worryingly, is that the accuracy of results declined from six significant figures to one significant figure during the running of programs. Hatton and other researchers' work indicates that scientific software is often of poor quality. What is staggering about the research that has been done is that it examines commercial scientific software – produced by software engineers who have to undergo a regime of thorough testing, quality assurance and a change control discipline known as configuration management

See the rest here:
If you're going to do good science, release the computer code too