Editorial Policies of Scientific Journals
The recent results of an Independent Verification of a calculation at Climate Audit and discussed here brings me to mention the editorial policies of scientific journals relative to Verification, Validation, Software Quality Assurance, and Qualfications of models, methods, software, calculations, users, and applications. I have posted comments about the issues relative to publication of peer-reviewed papers on several blogs and given more detailed discussion here.
According to the results of a very unscientific survey by me, the scientific journals have no interest in these very important issues. Many engineering journals, however, have addressed the problems and developed editorial appropriate policies which must be met by authors.
A little over two years ago I wrote letters to the editors of several scientific journals in which the climate change community publishes papers. I have uploaded an example of the letter.
I think it’s time to followup on that letter, for a second time. Continue reading
Verify the Calculation: An Example
The calculation requires Independent Verification. The calculation includes the coding, the data provided to the code at run time, built in options chosen by the user at run time, and the qualifications of the user.
An excellent example has been provided by the work over on Climate Audit. The results of the excellent work by Climate Audit is being discussed at Climate Science.
The example started with this, which led to this, and leading finally to this.
An excellent piece of Independent Verification accomplished without the benefit of source coding, and actually without any publicly available equations whatsoever. Exactly why the equations and codes are kept secret from the people who paid for them is another very significant issue. Keeping models, methods, procedures, and associated source code secret is not a part of the accepted scientific method.
Verify the Coding: A Couple of Examples
A couple of interesting examples of real-world failures here and here. Verify the calculation, also, of course. Additional discussions of the latter are here.
Chaos and Butterflies yet again
The NWP and GCM communities cannot think that a Butterfly will have any influence whatsoever on any physical phenomena or processes of interest. Instead the phenomenology of The Butterfly Effect as exhibited by the numerical calculations of some systems of ordinary differential equations is invoked by hypothesis into NWP and GCM models/methods/codes. I think we need to limit discussions to the Lorenz-like systems of ODEs, as these seem to be the basis for invoking the phenomenology into the NWP and GCM communities. Otherwise we will get side-tracked into discussions of the “chaotic response of complex dynamical systems” in general.
An Important Peer-Reviewed Paper: Part 1
We’ll now look at some of the results presented in the paper.
Introduction and Background
The authors have introduced the subject of convergence of numerical methods into the field of chaotic dynamical systems. This field is very important in many areas of current intense study and investigation. Numerical models and solution methods exhibit chaotic dynamical-system characteristics in weather and climate modeling, direct numerical and large eddy simulations of turbulent flows, as well as the classical studies of chaotic systems through nonlinear ODEs as introduced by Lorenz and others. The author’s paper seems to be the first in the literature to present results of systematic investigations of convergence into this important field of research and applications.
Continue reading
A Start on a V&V and SQA Bibliography
Here is a start on a bibliography for V&V and SQA books and articles.
Continue reading
A Discussion Paper on Software V&V and SQA
I have a kind of long discussion paper on V&V and SQA that I put together a couple of years ago. I have uploaded a copy V&V and SQA.
Lack of Convergence, Under-Resolution, and Numerical Errors
The Basic Hypotheses
The following is well known but because it is the focus of this discussion I list it for handy reference.
Convergence is Paramount; Nothing else Counts
The fundamental objective of numerical solution of systems of algebraic equations, ordinary differential equations (ODEs) and partial differential equations (PDEs) is to ensure that the approximations made in order to solve the equations do not in fact influence the solutions. In the case of systems of algebraic equations, it must be shown that the stopping criteria applied to iterative solution methods does not influence the accepted solutions. The solutions are independent, to an acceptable level, of the stopping criteria, and the calculated numbers satisfy the equations.
Continue reading
Models Methods Software 