Editorial Policies of Scientific Journals
The recent results of an Independent Verification of a calculation at Climate Audit and discussed here brings me to mention the editorial policies of scientific journals relative to Verification, Validation, Software Quality Assurance, and Qualfications of models, methods, software, calculations, users, and applications. I have posted comments about the issues relative to publication of peer-reviewed papers on several blogs and given more detailed discussion here.
According to the results of a very unscientific survey by me, the scientific journals have no interest in these very important issues. Many engineering journals, however, have addressed the problems and developed editorial appropriate policies which must be met by authors.
A little over two years ago I wrote letters to the editors of several scientific journals in which the climate change community publishes papers. I have uploaded an example of the letter.
I think it’s time to followup on that letter, for a second time. Continue reading
Coding Guidelines and Inline Documentation: GISS ModelE
Here’s an example of what a lack on coding guidelines (or ignoring them) can lead to. From the NASA/GISS ModelE online source code browser:
SUBROUTINE SURFCE 1,30
!@sum SURFCE calculates the surface fluxes which include
!@+ sensible heat, evaporation, thermal radiation, and momentum
!@+ drag. It also calculates instantaneous surface temperature,
!@+ surface specific humidity, and surface wind components.
!@auth Nobody will claim responsibilty
This is the entire header information for a 1228-line routine.
The last line says it all.
Verify the Calculation: An Example
The calculation requires Independent Verification. The calculation includes the coding, the data provided to the code at run time, built in options chosen by the user at run time, and the qualifications of the user.
An excellent example has been provided by the work over on Climate Audit. The results of the excellent work by Climate Audit is being discussed at Climate Science.
The example started with this, which led to this, and leading finally to this.
An excellent piece of Independent Verification accomplished without the benefit of source coding, and actually without any publicly available equations whatsoever. Exactly why the equations and codes are kept secret from the people who paid for them is another very significant issue. Keeping models, methods, procedures, and associated source code secret is not a part of the accepted scientific method.
Verify the Coding: A Couple of Examples
A couple of interesting examples of real-world failures here and here. Verify the calculation, also, of course. Additional discussions of the latter are here.
A Start on a V&V and SQA Bibliography
Here is a start on a bibliography for V&V and SQA books and articles.
Continue reading
Lack of Convergence, Under-Resolution, and Numerical Errors
The Basic Hypotheses
The following is well known but because it is the focus of this discussion I list it for handy reference.
Convergence is Paramount; Nothing else Counts
The fundamental objective of numerical solution of systems of algebraic equations, ordinary differential equations (ODEs) and partial differential equations (PDEs) is to ensure that the approximations made in order to solve the equations do not in fact influence the solutions. In the case of systems of algebraic equations, it must be shown that the stopping criteria applied to iterative solution methods does not influence the accepted solutions. The solutions are independent, to an acceptable level, of the stopping criteria, and the calculated numbers satisfy the equations.
Continue reading
Models Methods Software 