Models Methods Software

Dan Hughes

More on Mass and Energy Conservation

The focus on this previous post was the fact that approximations made at the continuous-equation level mean that the model’s mass and energy budgets are different from the mass and energy budgets of the physical system. Note that there are very significant additional issues associated with the discrete approximations applied to the continuous equations and the numerical solutions of these. These issues in the discrete domain, in my opinion, have the potential to far outweigh issues in the continuous domain. Accurate integration of the discrete approximations over the enormous times scales of interest is a very tough problem.

The present post looks at some more issues associated with energy conservation in the discrete domain.

Continue reading

January 14, 2009 Posted by | Verification | , , | 2 Comments

Viscous Dissipation in the NASA/GISS ModelE Code is Wrong …

it is not even viscous dissipation. There is also an energy imbalance in the NASA/GISS ModelE code due to the error in the ‘viscous dissipation’. The energy imbalance is about the same magnitude as the imbalance associated with increased concentrations of CO2 in the atmosphere.

Updated January 14, 2009, down near the end.

Continue reading

January 13, 2009 Posted by | Verification | , , | 17 Comments

Yet Even More NASA/GISS ModelE Coding

While pursing the NASA/GISS ModelE coding as part of getting back to the dissipation questions that have come up on Lucia’s Blog in this thread, I ran across yet even more interesting coding. The thread at Lucia’s is related to the posts on this Blog here, here, and here. The dissipation questions come up now and then in several Blogs.

But first, a short diversion.
Continue reading

January 10, 2009 Posted by | Verification | , , , | 1 Comment

Royal Meteorological Society V&V and SQA

Some time ago it was suggested to me something to the effect that most of the people in the Climate Change Community to whom I suggest V&V and SQA issues are critical and require significant attention don’t even know what I’m talking about. Here’s some information that lends support to that observation.

Continue reading

January 10, 2009 Posted by | Calculation Verification, Verification | , , , | 1 Comment

The Fundamental Issue

The Intergovernmental Panel for Climate Change (IPCC) Fourth Assessment Report (AR4, IPCC, 2007) states:

“A major advance of this assessment of climate change projections compared with the TAR is the large number of simulations available from a broader range of models. Taken together with additional information from observations, these provide a quantitative basis for estimating likelihoods for many aspects of future climate change.” [ My bolding. ]

Do the numbers from these “large number of simulations available from a broader range of models” GCM calculations have any Meaning. My answer is No.

One crucial and necessary first step is that application of Verification procedures have shown that the numbers produced by the software accurately reflect both (1) the original intent of the continuous equations for the models, and (2) the numerical solution methods applied to the discrete approximations to the continuous equations. That is, Verification shows that the equations have been solved right. Do the numbers actually satisfy the Verified-to-be-correct-as-coded discrete equations and do the solutions of the discrete equations converge to solution of the continuous equations. Neither of these has been demonstrated for any GCM. I will be pleased to be shown to be wrong on this point.

All software can be Verified. Objective technical criteria and associated success metrics can be developed and applied in a manner that provides assurances about the correctness of the coding of the equations and their numerical solutions. Lack of Verification leaves open the potential that the numbers from the software are simply results of “bugs” in the coding.

The present-day software development community, in all kinds of applications and organizations, is keenly aware that lack of SQA policies and procedures, and successful applications of these to the software, leaves open a significant potential for problems to exist in the software. So far as I am aware, there are no precedents whatsoever for public policy decisions to be based on software for which no SQA procedures have been applied.

December 8, 2008 Posted by | Calculation Verification, Verification | , , , | Leave a comment

Letters to NASA Quality Assurance Staff

On Tuesday November 11, 2008 at about 11:45 am I sent copies of the following letter to several NASA Quality Assurance personnel. The date has proven to be quite a coincidence as subsequent, and continuing, events have shown. I had started writing the letter a few days before I sent it.

I’ve heard nothing back yet; none of the recipients has acknowledged that the letter was received.

I have not yet located similar Quality Assurance offices in NOAA. All pointers will be appreciated.

Continue reading

November 17, 2008 Posted by | Verification | , | 11 Comments

GISS/NASA: We Don’t Verify. We Calculate, you Verify.

Another example of the lack of even the most basic concerns with verification of calculations has been discovered in GISS/NASA work.

The response by GISS/NASA employees is extremely disconcerting. That response seems to me to say that if the application procedure simply produces numbers, we’re ok with that. Verification of the numbers is left to others.

Continue reading

November 12, 2008 Posted by | Calculation Verification, Verification | , , | 1 Comment

Pattern Matching in GISS/NASA ModelE Coding

In a previous post I gave an illustration of how GISS/NASA employees have implemented new and innovative ways to produce inactive code using the capabilities provided by F90/95. I had run across the following statements in routine DIAG.f:


The ‘!’ in the first line is going to be very difficult to remember it exists and correctly maintain. Someone might come along and say, “I wonder what that’s doing in the middle of an executable statement.” and promptly un-do the comment. Or un-do the comment of the second line while overlooking the comment in the first line. That would make a screw up on several levels.

Today I have found many more examples of innovative coding by employees of GISS/NASA. It is clear that the NASA Software Quality Assurance procedures are ignored by GISS/NASA. It is equally clear that there are no Software Quality Assurance procedures being applied to the GISS/NASA ModelE code. None.

Update November 2, 2008 down near the end.

Continue reading

October 31, 2008 Posted by | Verification | , , , , | 7 Comments

Black Boxes

Not my kind of Engineering.

Continue reading

October 24, 2008 Posted by | Verification | , , , | Leave a comment

Chaos and ODEs Part 1c: The Numerical Methods

The numerical solution methods that will be used to check convergence are given in a file that I uploaded.

Let me know if you see any typos or if you want to see some results for a specific equation system.

I’m thinking that Part 1d will be some numerical results.

November 18, 2007 Posted by | Chaos, Chaos and Lorenz, ODEs, Verification | , , | 2 Comments