V&V and SQA: Part 1, Definitions and Characterization
I’m going to post a series of short summaries of some of the central aspects of verification, validation ( V&V ) and software quality assurance ( SQA ) for production-grade computer software. These subjects have received significant investigations starting in the 1980s ( more or less ) and have reached maturation and have been successfully applied to a wide range of scientific and engineering software. I don’t intend to give a complete exposition of the subjects, the field is much too big.
Continue reading
ILL-Posed IVPs and the MMS
The subject means ill-posed Initial Value Problems (IVPs) and the Method of Manufactured Solutions (MMS). I have uploaded two files; this one has the words, and this one has the figures. If you open them in separate windows the text is easier to follow.
All comments, especially corrections for incorrectos, will be appreciated.
CRU Hack Info Dump
I’ve looked at some of the info in the CRU data dump and have concluded that the situation is much worse that I have previously envisioned..
All results, each and every number, to be used to set public policy must be Independently Reviewed and Verified. This is the world-wide operating standard for all other decisions, without exception, that have a potential to effect the health and safety of the public.
The climate change community remains the sole exception to this standard. The very organizations that continue to insist that it be exempt from this requirement have now been shown to have fatally flawed the very system that they propose be used.
Maybe we’ll see a Carbon Regulatory Agency developed and implemented by some progressive governments.
Independent Verification and Peer Review
Steve McIntyre has discovered and presented yet another problem with a long-term temperature reconstruction: Yamal: A “Divergence” Problem. While Steve has named his blog ClimateAudit, in my opinion his work is in fact Independent Verification.
The old, and rapidly becoming tedious, “Peer Review Publication is Required” nonsense has already appeared in the comments. Ironically, it seems that no one who raises this issue is aware that Steve is in fact discovering problems in peer-reviewed publications.
A very short summary of a major problem area relative to this olde rant follows.
New V&V Book by Pat Roache
I received the following e-mail from Pat Roache regarding his new book about Verification and Validation. He has also made arrangements for reduced prices on two others of his books.
I am pleased to announce the publication of
“Fundamentals of Verification and Validation” by Patrick J. Roache.
Copyright 2009, ISBN 978-0-93478-12-7. 476 pages, subject index.
The book is the successor to my 1998 book “Verification and Validation in Computational Science and Engineering.” About 1/3 of the material is new, including a new Chapter 11 describing the Total Validation Uncertainty approach of ASME ANSI Standard V&V 20 (2009).
The attached file outlines the new features of the book. A complete Table of Contents, including designators for new and modified Sections, will be found on the website.
The price is the same as the 1998 book, U.S. $85.00 (but shipping charges have increased). The book is available directly from our fulfillment house, BookMasters, by email order to
orders@BookMasters.com
or from Amazon.com at the following.
The orders to BookMasters tend to ship faster since they do not run out of stock. Also, wholesale orders can be placed by contacting BookMasters at orders@BookMasters.com.
I would appreciate it if you would forward this email to any of your colleagues who might be interested.
Thanks for your consideration.
Respectfully,
Patrick Roache
p.s.
If anyone is interested, the 1998 V&V book is on clearance sale for 1/2 price at US $42.50. Available from orders@BookMasters.com or from Amazon at
Likewise, the book “Fundamentals of Computational Fluid Dynamics” is on clearance sale for 1/2 price at US $37.50. Available from orders@BookMasters.com or from Amazon at
V&V and NNSA Advanced Strategic Computing at LANL
[Updated September 26]
I think the name of the program originally known as the Advanced Strategic Computing Initiative ( ASCI ) is now known as Advanced Simulation & Computing ( ASC ).
[Updated August 26]
I have uploaded an excellent summary, developed by Los Alamos, of the ASC V&V Program at LANL. It is here.
A couple of quotes.
Confidence in simulation extrapolation comes via confidence in physics & numerics models, not calibration to experimental data.
Having “good agreement” between calculations and observations is not sufficient to establish scientifically credible predictive capability.
More and more it seems that the Climate Change Community remains the only holdout among all compute-intensive enterprises relative to application of rigorous, independent V&V to computer software.
Update
Continue reading
EPA HQ and Software Quality Assurance
I sent a slightly revised copy of the comment on the Proposed CO2 Ruling, given in this post, to Ms. Lisa jackson, Administrator of EPA. The letter to Ms. jackson is shown below. I received a reply from Rona Birnbaum, Chief, Climate Science & Impacts Branch, Climate Change Division. (The actual signing of the letter was a task delegated to someone whose name I can’t read.)
As in all previous cases in which I have attempted to convey the critical necessity of Independent Verification and Validation to persons outside the software development community, I failed again. It is obvious that Ms. jackson, Rona Birnbaum, or whoever read and responded to the letter, have no idea what I’m talking about.
Models Methods Software 
