Coding Standards Finally Appear
Steve Easterbrook has provided a list of coding standards that are associated with some of the climate models. The first one is for the NASA / GISS ModelE model and code.
Professor Easterbrook states,
Two followup tasks I hope to get to soon – (1) analyze how much these different standards overlap/differ, and (2) measure how much the model codes adhere to the standards.
Here are a few leads relative to Task (2):
The first of these posts was written almost four years ago. The date on the NASA / GISS ModelE document is February 2010. I’m not hopeful that Better late than never will work out in this case. It’s very difficult to retro-fit coding standards to code that is several decades old.
Looks like we’re getting some Traction
This is interesting; Computational science: …Error. From Nature News, even. Comments allowed over there.
V&V and SQA: Part 3, Verification
Verification Activities
The focus of verification is the actual coding of the software with objectives to determine: (1) that the coding corresponds to the equations given in the specification document, (2) the order of accuracy of the numerical methods, and (3) the order of convergence of the numerical methods. In general, the latter two objectives are purely mathematical and go to the heart of the coding of the solution methods. Several of the procedures that are used to pursue these objectives are given in the following discussions.
Continue reading
ILL-Posed IVPs and the MMS
The subject means ill-posed Initial Value Problems (IVPs) and the Method of Manufactured Solutions (MMS). I have uploaded two files; this one has the words, and this one has the figures. If you open them in separate windows the text is easier to follow.
All comments, especially corrections for incorrectos, will be appreciated.
New V&V Book by Pat Roache
I received the following e-mail from Pat Roache regarding his new book about Verification and Validation. He has also made arrangements for reduced prices on two others of his books.
I am pleased to announce the publication of
“Fundamentals of Verification and Validation” by Patrick J. Roache.
Copyright 2009, ISBN 978-0-93478-12-7. 476 pages, subject index.
The book is the successor to my 1998 book “Verification and Validation in Computational Science and Engineering.” About 1/3 of the material is new, including a new Chapter 11 describing the Total Validation Uncertainty approach of ASME ANSI Standard V&V 20 (2009).
The attached file outlines the new features of the book. A complete Table of Contents, including designators for new and modified Sections, will be found on the website.
The price is the same as the 1998 book, U.S. $85.00 (but shipping charges have increased). The book is available directly from our fulfillment house, BookMasters, by email order to
orders@BookMasters.com
or from Amazon.com at the following.
http://www.amazon.com/Fundamentals-Verification-Validation-Patrick-Roache/dp/0913478121/ref=sr_1_7?ie=UTF8&s=books&qid=1253150836&sr=1-7
The orders to BookMasters tend to ship faster since they do not run out of stock. Also, wholesale orders can be placed by contacting BookMasters at orders@BookMasters.com.
I would appreciate it if you would forward this email to any of your colleagues who might be interested.
Thanks for your consideration.
Respectfully,
Patrick Roache
p.s.
If anyone is interested, the 1998 V&V book is on clearance sale for 1/2 price at US $42.50. Available from orders@BookMasters.com or from Amazon at
http://www.amazon.com/Verification-Validation-Computational-Science-Engineering/dp/0913478083/ref=sr_1_4?ie=UTF8&s=books&qid=1253718713&sr=8-4.
Likewise, the book “Fundamentals of Computational Fluid Dynamics” is on clearance sale for 1/2 price at US $37.50. Available from orders@BookMasters.com or from Amazon at
http://www.amazon.com/Fundamentals-Computational-Dynamics-Patrick-Roache/dp/0913478091/ref=sr_1_5?ie=UTF8&s=books&qid=1253718713&sr=8-5
V&V and NNSA Advanced Strategic Computing at LANL
[Updated September 26]
I think the name of the program originally known as the Advanced Strategic Computing Initiative ( ASCI ) is now known as Advanced Simulation & Computing ( ASC ).
[Updated August 26]
I have uploaded an excellent summary, developed by Los Alamos, of the ASC V&V Program at LANL. It is here.
A couple of quotes.
Confidence in simulation extrapolation comes via confidence in physics & numerics models, not calibration to experimental data.
Having “good agreement” between calculations and observations is not sufficient to establish scientifically credible predictive capability.
More and more it seems that the Climate Change Community remains the only holdout among all compute-intensive enterprises relative to application of rigorous, independent V&V to computer software.
Update
Continue reading
EPA HQ and Software Quality Assurance
I sent a slightly revised copy of the comment on the Proposed CO2 Ruling, given in this post, to Ms. Lisa jackson, Administrator of EPA. The letter to Ms. jackson is shown below. I received a reply from Rona Birnbaum, Chief, Climate Science & Impacts Branch, Climate Change Division. (The actual signing of the letter was a task delegated to someone whose name I can’t read.)
As in all previous cases in which I have attempted to convey the critical necessity of Independent Verification and Validation to persons outside the software development community, I failed again. It is obvious that Ms. jackson, Rona Birnbaum, or whoever read and responded to the letter, have no idea what I’m talking about.
Analytical Sensitivity Analysis
I have started working on a toy model and plan to include analytical sensitivity analysis as part of the methods. These notes, and an associated extended discussion that I have up-loaded, serve as a short introduction to the subject. The file is here.
Update October 11, 2011. These reports by J. R. Bates explore sensitivity and feedback of several aspects of simple climate models.
J. R. Bates, Some considerations of the concept of climate feedback, Quarterly Journal of the Royal Meteorological Society, 133: 545–560 (2007)
Published online in Wiley InterScience (www.interscience.wiley.com) DOI: 10.1002/qj.62
ABSTRACT: A conceptual study of climate feedbacks is carried out using two simple linear two-zone models and the commonly-used zero-dimensional model to which they reduce under simplifying assumptions. The term ‘feedback’ is used in many different senses in the climate literature. Two prototype usages, stability-altering feedback (defined in terms of a system’s asymptotic response to an impulsive forcing, negative when stability-enhancing) and sensitivity-altering feedback (defined in terms of a system’s steady-state response to a step-function forcing, negative when sensitivity-diminishing) have been isolated for study. These two climate feedback concepts are viewed against the background of control theory, which provides a generalized feedback perspective embracing all forms of forcing and which is often seen as providing the paradigm for the concept of feedback as used in climate studies.
The relationship between the prototype climate feedbacks is simple in the context of the zero-dimensional model. Here, the stability-altering and sensitivity-altering feedbacks provided by a given interaction are of the same sign, and the sign of the stability-altering feedback as measured by initial tendencies always coincides with its sign as measured by the defining asymptotic tendencies. Even in this simple model, however, the sign of the prototype climate feedbacks can be opposite to the sign of the system’s feedback as defined in control theory.
In the two-zone models, the relationship between the prototype climate feedbacks is not so simple. It is shown that, contrary to the common assumption, these feedbacks can be of opposite signs. Moreover, the sign of the stability-altering feedback as measured by initial tendencies can be opposite to its sign as measured by asymptotic tendencies. It is further shown that there is no simple relationship between the sign of either of the prototype climate feedbacks in the two-zone models and the sign of these models’ feedback as defined in control theory.
These results point to the need for greater precision and explicitness in the definition and use of the term ‘climate feedback’, both to facilitate interdisciplinary dialogue in relation to feedback and to guard against erroneous inferences within the climate field. Explicit definitions of the two prototype categories of climate feedback studied here are proposed.
Copyright 2007 Royal Meteorological Society
J. Ray Bates, Climate stability and sensitivity in some simple conceptual models, Climate Dynamics, 2010. Published Online December 2010. DOI 10.1007/s00382-010-0966-0
Abstract
A theoretical investigation of climate stability and sensitivity is carried out using three simple linearized models based on the top-of-the-atmosphere energy budget. The simplest is the zero-dimensional model (ZDM) commonly used as a conceptual basis for climate sensitivity and feedback studies. The others are two-zone models with tropics and extratropics of equal area; in the first of these (Model A), the dynamical heat transport (DHT) between the zones is implicit, in the second (Model B) it is explicitly parameterized. It is found that the stability and sensitivity properties of the ZDM and Model A are very similar, both depending only on the global-mean radiative response coefficient and the global-mean forcing. The corresponding properties of Model B are more complex, depending asymmetrically on the separate tropical and extratropical values of these quantities, as well as on the DHT coefficient. Adopting Model B as a benchmark, conditions are found under which the validity of the ZDM and Model A as climate sensitivity models holds. It is shown that parameter ranges of physical interest exist for which such validity may not hold. The 2 × CO2 sensitivities of the simple models are studied and compared. Possible implications of the results for sensitivities derived from GCMs and palaeoclimate data are suggested. Sensitivities for more general scenarios that include negative forcing in the tropics (due to aerosols, inadvertent or geoengineered) are also studied. Some unexpected outcomes are found in this case. These include the possibility of a negative global-mean temperature response to a positive global-mean forcing, and vice versa. [pay-walled]
J. R. Bates, On climate stability, climate sensitivity and the dynamics of the enhanced greenhouse effect, Danish Center for Earth System Science, DCESS REPORT Number 3, 2003.
Abstract
The dynamics of the enhanced greenhouse effect resulting from a CO2 increase are studied using a simple two-zone hemispheric atmosphere-ocean model on an aquaplanet that is simple enough to allow analytical solution. The model’s sensitivity to forcing is viewed against the background of its stability to free perturbations. Free perturbations in SST, regarded as representative of temperature perturbations in the mixed layers beneath, are subject to a destabilizing influence from the effects of the water vapor infrared radiative (WVIR) feedback and are stabilized by evaporation, which results in moist convection and precipitation that deposit the latent heat removed from the surface above the level of the main water vapor absorbers, whence it is radiated to space. The rate of evaporation depends on the surface wind strength and the air-sea humidity deficit. In the model, the former is parameterized in terms of the atmospheric angular momentum (AM) transport, which depends on the SSTs in both zones, and the latter in terms of the local SST through the Clausius-Clapeyron relationship. Using estimates of the parameters derived from observation and detailed radiative model calculations, the model gives an equilibrium temperature increase for a CO2 doubling that lies within the range of that given by GCMs. As in the GCMs, it is found that the warming is greatest in the extratropics. Unlike the case of the GCMs, the mechanism of the warming in the simple model can be fully understood. The model’s equilibrium sensitivity is found to be inversely proportional to the value of the stability determinant (which measures the product of the decay rates of the fast and slow normal modes) and to be strongly influenced by the strength of a ventilation feedback. Both of these factors are sensitively dependent on the strength of the extratropical WVIR feedback, and the ventilation feedback in addition depends critically on the latitudinal distribution of the surface forcing
J. R. BATES, A dynamical stabilizer in the climate system: a mechanism suggested by a simple model, Tellus (1999), 51A, 349–372.
Abstract
A simple zonally averaged hemispheric model of the climate system is constructed, based on energy equations for two ocean basins separated at 30° latitude with the surface fluxes calculated explicitly. A combination of empirical input and theoretical calculation is used to determine an annual mean equilibrium climate for the model and to study its stability with respect to small perturbations. The insolation, the mean albedos and the equilibrium temperatures for the two model zones are prescribed from observation. The principal agent of interaction between the zones is the vertically integrated poleward transport of atmospheric angular momentum across their common boundary. This is parameterized using an empirical formula derived from a multiyear atmospheric data set. The surface winds are derived from the angular momentum transport assuming the atmosphere to be in a state of dynamic balance on the climatic timescales of interest. A further assumption that the air–sea temperature difference and low level relative humidity remain fixed at their mean observed values then allows the surface fluxes of latent and sensible heat to be calculated. Results from a radiative model, which show a positive lower tropospheric water vapour/infrared radiative feedback on SST perturbations in both zones, are used to calculate the net upward infrared radiative fluxes at the surface. In the model’s equilibrium climate, the principal processes balancing the solar radiation absorbed at the surface are evaporation in the tropical zone and net infrared radiation in the extratropical zone. The stability of small perturbations about the equilibrium is studied using a linearized form of the ocean energy equations. Ice-albedo and cloud feedbacks are omitted and attention is focussed on the competing effects of the water vapour/infrared radiative feedback and the turbulent surface flux and oceanic heat transport feedbacks associated with the angular momentum cycle. The perturbation equations involve inter-zone coupling and have coefficients dependent on the values of the equilibrium fluxes and the sensitivity of the angular momentum transport. Analytical solutions for the perturbations are obtained. These provide criteria for the stability of the equilibrium climate. If the evaporative feedback on SST perturbations is omitted, the equilibrium climate is unstable due to the influence of the water vapour/infrared radiative feedback, which dominates over the effects of the sensible heat and ocean heat transport feedbacks. The inclusion of evaporation gives a negative feedback which is of sufficient strength to stabilize the system. The stabilizing mechanism involves wind and humidity factors in the evaporative fluxes that are of comparable magnitude. Both factors involve the angular momentum transport. In including angular momentum and calculating the surface fluxes explicitly, the model presented here differs from the many simple climate models based on the Budyko–Sellers formulation. In that formulation, an atmospheric energy balance equation is used to eliminate surface fluxes in favour of top-of-the-atmosphere radiative fluxes and meridional atmospheric energy transports. In the resulting models, infrared radiation appears as a stabilizing influence on SST perturbations and the dynamical stabilizing mechanism found here cannot be identified. [pay-walled]
Update November 9, 2010. here is additional information about applications of ASA to climate sciences.
Dan G. Cacuci, Mihaela Ionescu-Bujor, and Michael Navon, “Sensitivity and Uncertainty Analysis, Volume II: Applications to Large-Scale Systems”, CRC Press, 2005.
Chapter V. Using the ASAP to Gain New Insights into Paradigm Atmospheric Sciences Problems
Chapter VI. Adjoint Sensitivity Analysis Procedure for Operational Meteorological Applications
Update February 8, 2010. I have found that there are papers on this subject directly related to models of the Earth’s climate systems. I’m not surprised and especially that Dan Cacuci was on the case back in the early 1980s.
Here are a few good references:
Matthew C. G. Hall, Dan G. Cacuci and M. E. Schlesinger, “Sensitivity Analysis of a Radiative-Convective Model by the Adjoint Method”, Journal of the Atmospheric Sciences, Vol. 39, pp. 2038-2050, 1982.
Matthew C. G. Hall and Dan G. Cacuci, “Physical interpretation of the Adjoint Functions for Sensitivity Analysis of Atmospheric Models”, Journal of the Atmospheric Sciences, Vol. 40, pp. 2537-2546, 1983.
Dan G. Cacuci and Matthew C. G. Hall, “Efficient Estimation of Feedback Effects with Application to Climate Models”, Journal of the Atmospheric Sciences, Vol. 41, pp. 2063-2068, 1984.
I. M. Held and M. J. Suarez, “A Two-Level Primitive Equation Atmospheric Model Designed for Climatic Sensitivity Experiments”, Journal of the Atmospheric Sciences, Vol. 39, pp. 206-229, 1978.
S. Wanabe and R. T. Wetherald, “Thermal Equilibrium of the Atmosphere with a given Distribution of Relative Humidity”, Journal of the Atmospheric Sciences, Vol. 24, pp. 241-259, 1967.
S. Wanabe and R. T. Wetherald, “On the Distribution of Climate Change Resulting from an Increase in CO2 Content of the Atmosphere”, Journal of the Atmospheric Sciences, Vol. 37, pp. 99-118, 1980.
T. L. Bell, “Climate Sensitivity from Fluctuation Dissipation: Some Simple Model Tests”, Journal of the Atmospheric Sciences, Vol. 37, pp. 1700-1707, 1980.
Isaac M. Held and Brian J. Soden, “Water Vapor Feedback and Global Warming”, Annual Review of Energy and Environment, Vol. 25, pp. 441-475, 2000.
Brian J. Soden and Isaac M. Held, “An Assessment of Climate Feedbacks in Coupled Ocean–Atmosphere Models”, Journal of Climate, Vol. 19, pp. 3354-3360, 2006.
Brian J. Soden, Anthony J. Broccoli, and Richard S. Hemler, On the Use of Cloud Forcing to Estimate Cloud Feedback”, Journal of Climate, Vol. 17, No. 19, pp. 3661-3665, 2004.
REFERENCES
Cess, R. D., and G. L. Potter, 1988: A methodology for understanding
and intercomparing atmospheric climate feedback processes in
general circulation models. J. Geophys. Res., 93, 8305–8314.
Cess, R. D., and Coauthors, 1990: Intercomparison and interpretation of
climate feedback processes in 19 atmospheric GCMs. J. Geophys. Res., 95, 16 601–16 615.
Cess, R. D., and Coauthors, 1996: Cloud feedback in atmospheric general
circulation models: An update. J. Geophys. Res., 101, 12,791–794.
Colman, R., 2003: A comparison of climate feedbacks in GCMs.
Climate Dyn., 20, 865–873.
Colman, R., and B. J., McAvaney, 1997: A study of general circulation
model climate feedbacks determined from perturbed SST experiments.
J. Geophys. Res., 102, 19,383–19,402.
Colman, R., S. B. Power, and B. J. McAvaney, 1997: Non-linear climate
feedbacks from perturbed SST experiments. Climate Dynamics 13, 10, 717–731.
Cubasch, U., and R. D. Cess, 1990: Processes and modeling. Climate
Change: The IPCC Scientific Assessment, J. T. Houghton, G. J.
Jenkins, and J. J. Ephraums, Eds., Cambridge University Press,
365 pp.
Gates, W. L., J. F. B. Mitchell, G. J. Boer, U. Cubasch, and V. P.
Meleshko, 1992: Climate modeling climate prediction, and model
validation. Climate Change 1992: The Supplementary Report
to the IPCC Scientific Assessment, J. T. Houghton, B. A. Callander,
and S. K. Varney, Eds., Cambridge University Press, 200 pp.
GFDL Global Atmospheric Model Development Team, 2004: The
new GFDL global atmospheric and land model (AM2–LM2):
Evaluation with prescribed SST simulations. J. Climate, in press.
Held, I. M., and B. J. Soden, 2000: Water vapor feedback and global warming. Annu. Rev. Energy Environ., 25, 441–475.
Le Treut, H., Z. X. Li, and M. Forichon, 1994: Sensitivity of the
LMD general circulation model to greenhouse gas forcing associated with two different cloud water parameterizations. J. Climate, 7, 1827–1841.
Mitchell, J. F. B., and W. J. Ingram, 1992: Carbon dioxide and climate:
Mechanisms of changes in cloud. J. Climate, 5, 5–21.
Ramanathan, V., R. D. Cess, E. F. Harrison, P. Minnis, B. R. Barkstrom,
E. Ahmad, and D. Hartmann, 1989: Cloud radiative-forcing
and climate: Results from the Earth Radiation Budget Experiment.
Science, 243, 57–63.
Tsushima, Y., and S. Manabe, 2001: Influence of cloud feedback on annual variation of global-mean surface temperature. J. Geophys. Res., 106, 22 635–22 646.
Wetherald, R. T., and S. Manabe, 1988: Cloud feedback processes in a general circulation model. J. Atmos. Sci., 45, 1397–1415.
Zhang, M. H., R. D. Cess, J. J. Hack, and J. T. Kiehl, 1994: Diagnosticstudy of climate feedback processes in atmospheric GCMs. J. Geophys. Res., 99, 5525–5537.
Summary
These notes introduce a few of the ideas and concepts associated with sensitivity analysis for algebraic and ordinary differential equations. By sensitivity I mean what are the effects of changes in the numerical values of the parameters in a system of equations relative to a response function of interest. The response function can take any mathematical form, but I will focus on the values of the dependent variables of the equation system.
More on Mass and Energy Conservation
The focus on this previous post was the fact that approximations made at the continuous-equation level mean that the model’s mass and energy budgets are different from the mass and energy budgets of the physical system. Note that there are very significant additional issues associated with the discrete approximations applied to the continuous equations and the numerical solutions of these. These issues in the discrete domain, in my opinion, have the potential to far outweigh issues in the continuous domain. Accurate integration of the discrete approximations over the enormous times scales of interest is a very tough problem.
The present post looks at some more issues associated with energy conservation in the discrete domain.
Models Methods Software 
