Looks like we’re getting some Traction
This is interesting; Computational science: …Error. From Nature News, even. Comments allowed over there.
CRU Hack Info Dump
I’ve looked at some of the info in the CRU data dump and have concluded that the situation is much worse that I have previously envisioned..
All results, each and every number, to be used to set public policy must be Independently Reviewed and Verified. This is the world-wide operating standard for all other decisions, without exception, that have a potential to effect the health and safety of the public.
The climate change community remains the sole exception to this standard. The very organizations that continue to insist that it be exempt from this requirement have now been shown to have fatally flawed the very system that they propose be used.
Maybe we’ll see a Carbon Regulatory Agency developed and implemented by some progressive governments.
Independent Verification and Peer Review
Steve McIntyre has discovered and presented yet another problem with a long-term temperature reconstruction: Yamal: A “Divergence” Problem. While Steve has named his blog ClimateAudit, in my opinion his work is in fact Independent Verification.
The old, and rapidly becoming tedious, “Peer Review Publication is Required” nonsense has already appeared in the comments. Ironically, it seems that no one who raises this issue is aware that Steve is in fact discovering problems in peer-reviewed publications.
A very short summary of a major problem area relative to this olde rant follows.
New V&V Book by Pat Roache
I received the following e-mail from Pat Roache regarding his new book about Verification and Validation. He has also made arrangements for reduced prices on two others of his books.
I am pleased to announce the publication of
“Fundamentals of Verification and Validation” by Patrick J. Roache.
Copyright 2009, ISBN 978-0-93478-12-7. 476 pages, subject index.
The book is the successor to my 1998 book “Verification and Validation in Computational Science and Engineering.” About 1/3 of the material is new, including a new Chapter 11 describing the Total Validation Uncertainty approach of ASME ANSI Standard V&V 20 (2009).
The attached file outlines the new features of the book. A complete Table of Contents, including designators for new and modified Sections, will be found on the website.
The price is the same as the 1998 book, U.S. $85.00 (but shipping charges have increased). The book is available directly from our fulfillment house, BookMasters, by email order to
orders@BookMasters.com
or from Amazon.com at the following.
The orders to BookMasters tend to ship faster since they do not run out of stock. Also, wholesale orders can be placed by contacting BookMasters at orders@BookMasters.com.
I would appreciate it if you would forward this email to any of your colleagues who might be interested.
Thanks for your consideration.
Respectfully,
Patrick Roache
p.s.
If anyone is interested, the 1998 V&V book is on clearance sale for 1/2 price at US $42.50. Available from orders@BookMasters.com or from Amazon at
Likewise, the book “Fundamentals of Computational Fluid Dynamics” is on clearance sale for 1/2 price at US $37.50. Available from orders@BookMasters.com or from Amazon at
V&V and NNSA Advanced Strategic Computing at LANL
[Updated September 26]
I think the name of the program originally known as the Advanced Strategic Computing Initiative ( ASCI ) is now known as Advanced Simulation & Computing ( ASC ).
[Updated August 26]
I have uploaded an excellent summary, developed by Los Alamos, of the ASC V&V Program at LANL. It is here.
A couple of quotes.
Confidence in simulation extrapolation comes via confidence in physics & numerics models, not calibration to experimental data.
Having “good agreement” between calculations and observations is not sufficient to establish scientifically credible predictive capability.
More and more it seems that the Climate Change Community remains the only holdout among all compute-intensive enterprises relative to application of rigorous, independent V&V to computer software.
Update
Continue reading
EPA HQ and Software Quality Assurance
I sent a slightly revised copy of the comment on the Proposed CO2 Ruling, given in this post, to Ms. Lisa jackson, Administrator of EPA. The letter to Ms. jackson is shown below. I received a reply from Rona Birnbaum, Chief, Climate Science & Impacts Branch, Climate Change Division. (The actual signing of the letter was a task delegated to someone whose name I can’t read.)
As in all previous cases in which I have attempted to convey the critical necessity of Independent Verification and Validation to persons outside the software development community, I failed again. It is obvious that Ms. jackson, Rona Birnbaum, or whoever read and responded to the letter, have no idea what I’m talking about.
Analytical Sensitivity Analysis
I have started working on a toy model and plan to include analytical sensitivity analysis as part of the methods. These notes, and an associated extended discussion that I have up-loaded, serve as a short introduction to the subject. The file is here.
Update October 11, 2011. These reports by J. R. Bates explore sensitivity and feedback of several aspects of simple climate models.
J. R. Bates, Some considerations of the concept of climate feedback, Quarterly Journal of the Royal Meteorological Society, 133: 545–560 (2007)
Published online in Wiley InterScience (www.interscience.wiley.com) DOI: 10.1002/qj.62
ABSTRACT: A conceptual study of climate feedbacks is carried out using two simple linear two-zone models and the commonly-used zero-dimensional model to which they reduce under simplifying assumptions. The term ‘feedback’ is used in many different senses in the climate literature. Two prototype usages, stability-altering feedback (defined in terms of a system’s asymptotic response to an impulsive forcing, negative when stability-enhancing) and sensitivity-altering feedback (defined in terms of a system’s steady-state response to a step-function forcing, negative when sensitivity-diminishing) have been isolated for study. These two climate feedback concepts are viewed against the background of control theory, which provides a generalized feedback perspective embracing all forms of forcing and which is often seen as providing the paradigm for the concept of feedback as used in climate studies.
The relationship between the prototype climate feedbacks is simple in the context of the zero-dimensional model. Here, the stability-altering and sensitivity-altering feedbacks provided by a given interaction are of the same sign, and the sign of the stability-altering feedback as measured by initial tendencies always coincides with its sign as measured by the defining asymptotic tendencies. Even in this simple model, however, the sign of the prototype climate feedbacks can be opposite to the sign of the system’s feedback as defined in control theory.
In the two-zone models, the relationship between the prototype climate feedbacks is not so simple. It is shown that, contrary to the common assumption, these feedbacks can be of opposite signs. Moreover, the sign of the stability-altering feedback as measured by initial tendencies can be opposite to its sign as measured by asymptotic tendencies. It is further shown that there is no simple relationship between the sign of either of the prototype climate feedbacks in the two-zone models and the sign of these models’ feedback as defined in control theory.
These results point to the need for greater precision and explicitness in the definition and use of the term ‘climate feedback’, both to facilitate interdisciplinary dialogue in relation to feedback and to guard against erroneous inferences within the climate field. Explicit definitions of the two prototype categories of climate feedback studied here are proposed.
Copyright 2007 Royal Meteorological Society
J. Ray Bates, Climate stability and sensitivity in some simple conceptual models, Climate Dynamics, 2010. Published Online December 2010. DOI 10.1007/s00382-010-0966-0
Abstract
A theoretical investigation of climate stability and sensitivity is carried out using three simple linearized models based on the top-of-the-atmosphere energy budget. The simplest is the zero-dimensional model (ZDM) commonly used as a conceptual basis for climate sensitivity and feedback studies. The others are two-zone models with tropics and extratropics of equal area; in the first of these (Model A), the dynamical heat transport (DHT) between the zones is implicit, in the second (Model B) it is explicitly parameterized. It is found that the stability and sensitivity properties of the ZDM and Model A are very similar, both depending only on the global-mean radiative response coefficient and the global-mean forcing. The corresponding properties of Model B are more complex, depending asymmetrically on the separate tropical and extratropical values of these quantities, as well as on the DHT coefficient. Adopting Model B as a benchmark, conditions are found under which the validity of the ZDM and Model A as climate sensitivity models holds. It is shown that parameter ranges of physical interest exist for which such validity may not hold. The 2 × CO2 sensitivities of the simple models are studied and compared. Possible implications of the results for sensitivities derived from GCMs and palaeoclimate data are suggested. Sensitivities for more general scenarios that include negative forcing in the tropics (due to aerosols, inadvertent or geoengineered) are also studied. Some unexpected outcomes are found in this case. These include the possibility of a negative global-mean temperature response to a positive global-mean forcing, and vice versa. [pay-walled]
J. R. Bates, On climate stability, climate sensitivity and the dynamics of the enhanced greenhouse effect, Danish Center for Earth System Science, DCESS REPORT Number 3, 2003.
Abstract
The dynamics of the enhanced greenhouse effect resulting from a CO2 increase are studied using a simple two-zone hemispheric atmosphere-ocean model on an aquaplanet that is simple enough to allow analytical solution. The model’s sensitivity to forcing is viewed against the background of its stability to free perturbations. Free perturbations in SST, regarded as representative of temperature perturbations in the mixed layers beneath, are subject to a destabilizing influence from the effects of the water vapor infrared radiative (WVIR) feedback and are stabilized by evaporation, which results in moist convection and precipitation that deposit the latent heat removed from the surface above the level of the main water vapor absorbers, whence it is radiated to space. The rate of evaporation depends on the surface wind strength and the air-sea humidity deficit. In the model, the former is parameterized in terms of the atmospheric angular momentum (AM) transport, which depends on the SSTs in both zones, and the latter in terms of the local SST through the Clausius-Clapeyron relationship. Using estimates of the parameters derived from observation and detailed radiative model calculations, the model gives an equilibrium temperature increase for a CO2 doubling that lies within the range of that given by GCMs. As in the GCMs, it is found that the warming is greatest in the extratropics. Unlike the case of the GCMs, the mechanism of the warming in the simple model can be fully understood. The model’s equilibrium sensitivity is found to be inversely proportional to the value of the stability determinant (which measures the product of the decay rates of the fast and slow normal modes) and to be strongly influenced by the strength of a ventilation feedback. Both of these factors are sensitively dependent on the strength of the extratropical WVIR feedback, and the ventilation feedback in addition depends critically on the latitudinal distribution of the surface forcing
J. R. BATES, A dynamical stabilizer in the climate system: a mechanism suggested by a simple model, Tellus (1999), 51A, 349–372.
Abstract
A simple zonally averaged hemispheric model of the climate system is constructed, based on energy equations for two ocean basins separated at 30° latitude with the surface fluxes calculated explicitly. A combination of empirical input and theoretical calculation is used to determine an annual mean equilibrium climate for the model and to study its stability with respect to small perturbations. The insolation, the mean albedos and the equilibrium temperatures for the two model zones are prescribed from observation. The principal agent of interaction between the zones is the vertically integrated poleward transport of atmospheric angular momentum across their common boundary. This is parameterized using an empirical formula derived from a multiyear atmospheric data set. The surface winds are derived from the angular momentum transport assuming the atmosphere to be in a state of dynamic balance on the climatic timescales of interest. A further assumption that the air–sea temperature difference and low level relative humidity remain fixed at their mean observed values then allows the surface fluxes of latent and sensible heat to be calculated. Results from a radiative model, which show a positive lower tropospheric water vapour/infrared radiative feedback on SST perturbations in both zones, are used to calculate the net upward infrared radiative fluxes at the surface. In the model’s equilibrium climate, the principal processes balancing the solar radiation absorbed at the surface are evaporation in the tropical zone and net infrared radiation in the extratropical zone. The stability of small perturbations about the equilibrium is studied using a linearized form of the ocean energy equations. Ice-albedo and cloud feedbacks are omitted and attention is focussed on the competing effects of the water vapour/infrared radiative feedback and the turbulent surface flux and oceanic heat transport feedbacks associated with the angular momentum cycle. The perturbation equations involve inter-zone coupling and have coefficients dependent on the values of the equilibrium fluxes and the sensitivity of the angular momentum transport. Analytical solutions for the perturbations are obtained. These provide criteria for the stability of the equilibrium climate. If the evaporative feedback on SST perturbations is omitted, the equilibrium climate is unstable due to the influence of the water vapour/infrared radiative feedback, which dominates over the effects of the sensible heat and ocean heat transport feedbacks. The inclusion of evaporation gives a negative feedback which is of sufficient strength to stabilize the system. The stabilizing mechanism involves wind and humidity factors in the evaporative fluxes that are of comparable magnitude. Both factors involve the angular momentum transport. In including angular momentum and calculating the surface fluxes explicitly, the model presented here differs from the many simple climate models based on the Budyko–Sellers formulation. In that formulation, an atmospheric energy balance equation is used to eliminate surface fluxes in favour of top-of-the-atmosphere radiative fluxes and meridional atmospheric energy transports. In the resulting models, infrared radiation appears as a stabilizing influence on SST perturbations and the dynamical stabilizing mechanism found here cannot be identified. [pay-walled]
Update November 9, 2010. here is additional information about applications of ASA to climate sciences.
Dan G. Cacuci, Mihaela Ionescu-Bujor, and Michael Navon, “Sensitivity and Uncertainty Analysis, Volume II: Applications to Large-Scale Systems”, CRC Press, 2005.
Chapter V. Using the ASAP to Gain New Insights into Paradigm Atmospheric Sciences Problems
Chapter VI. Adjoint Sensitivity Analysis Procedure for Operational Meteorological Applications
Update February 8, 2010. I have found that there are papers on this subject directly related to models of the Earth’s climate systems. I’m not surprised and especially that Dan Cacuci was on the case back in the early 1980s.
Here are a few good references:
Matthew C. G. Hall, Dan G. Cacuci and M. E. Schlesinger, “Sensitivity Analysis of a Radiative-Convective Model by the Adjoint Method”, Journal of the Atmospheric Sciences, Vol. 39, pp. 2038-2050, 1982.
Matthew C. G. Hall and Dan G. Cacuci, “Physical interpretation of the Adjoint Functions for Sensitivity Analysis of Atmospheric Models”, Journal of the Atmospheric Sciences, Vol. 40, pp. 2537-2546, 1983.
Dan G. Cacuci and Matthew C. G. Hall, “Efficient Estimation of Feedback Effects with Application to Climate Models”, Journal of the Atmospheric Sciences, Vol. 41, pp. 2063-2068, 1984.
I. M. Held and M. J. Suarez, “A Two-Level Primitive Equation Atmospheric Model Designed for Climatic Sensitivity Experiments”, Journal of the Atmospheric Sciences, Vol. 39, pp. 206-229, 1978.
S. Wanabe and R. T. Wetherald, “Thermal Equilibrium of the Atmosphere with a given Distribution of Relative Humidity”, Journal of the Atmospheric Sciences, Vol. 24, pp. 241-259, 1967.
S. Wanabe and R. T. Wetherald, “On the Distribution of Climate Change Resulting from an Increase in CO2 Content of the Atmosphere”, Journal of the Atmospheric Sciences, Vol. 37, pp. 99-118, 1980.
T. L. Bell, “Climate Sensitivity from Fluctuation Dissipation: Some Simple Model Tests”, Journal of the Atmospheric Sciences, Vol. 37, pp. 1700-1707, 1980.
Isaac M. Held and Brian J. Soden, “Water Vapor Feedback and Global Warming”, Annual Review of Energy and Environment, Vol. 25, pp. 441-475, 2000.
Brian J. Soden and Isaac M. Held, “An Assessment of Climate Feedbacks in Coupled Ocean–Atmosphere Models”, Journal of Climate, Vol. 19, pp. 3354-3360, 2006.
Brian J. Soden, Anthony J. Broccoli, and Richard S. Hemler, On the Use of Cloud Forcing to Estimate Cloud Feedback”, Journal of Climate, Vol. 17, No. 19, pp. 3661-3665, 2004.
REFERENCES
Cess, R. D., and G. L. Potter, 1988: A methodology for understanding
and intercomparing atmospheric climate feedback processes in
general circulation models. J. Geophys. Res., 93, 8305–8314.
Cess, R. D., and Coauthors, 1990: Intercomparison and interpretation of
climate feedback processes in 19 atmospheric GCMs. J. Geophys. Res., 95, 16 601–16 615.
Cess, R. D., and Coauthors, 1996: Cloud feedback in atmospheric general
circulation models: An update. J. Geophys. Res., 101, 12,791–794.
Colman, R., 2003: A comparison of climate feedbacks in GCMs.
Climate Dyn., 20, 865–873.
Colman, R., and B. J., McAvaney, 1997: A study of general circulation
model climate feedbacks determined from perturbed SST experiments.
J. Geophys. Res., 102, 19,383–19,402.
Colman, R., S. B. Power, and B. J. McAvaney, 1997: Non-linear climate
feedbacks from perturbed SST experiments. Climate Dynamics 13, 10, 717–731.
Cubasch, U., and R. D. Cess, 1990: Processes and modeling. Climate
Change: The IPCC Scientific Assessment, J. T. Houghton, G. J.
Jenkins, and J. J. Ephraums, Eds., Cambridge University Press,
365 pp.
Gates, W. L., J. F. B. Mitchell, G. J. Boer, U. Cubasch, and V. P.
Meleshko, 1992: Climate modeling climate prediction, and model
validation. Climate Change 1992: The Supplementary Report
to the IPCC Scientific Assessment, J. T. Houghton, B. A. Callander,
and S. K. Varney, Eds., Cambridge University Press, 200 pp.
GFDL Global Atmospheric Model Development Team, 2004: The
new GFDL global atmospheric and land model (AM2–LM2):
Evaluation with prescribed SST simulations. J. Climate, in press.
Held, I. M., and B. J. Soden, 2000: Water vapor feedback and global warming. Annu. Rev. Energy Environ., 25, 441–475.
Le Treut, H., Z. X. Li, and M. Forichon, 1994: Sensitivity of the
LMD general circulation model to greenhouse gas forcing associated with two different cloud water parameterizations. J. Climate, 7, 1827–1841.
Mitchell, J. F. B., and W. J. Ingram, 1992: Carbon dioxide and climate:
Mechanisms of changes in cloud. J. Climate, 5, 5–21.
Ramanathan, V., R. D. Cess, E. F. Harrison, P. Minnis, B. R. Barkstrom,
E. Ahmad, and D. Hartmann, 1989: Cloud radiative-forcing
and climate: Results from the Earth Radiation Budget Experiment.
Science, 243, 57–63.
Tsushima, Y., and S. Manabe, 2001: Influence of cloud feedback on annual variation of global-mean surface temperature. J. Geophys. Res., 106, 22 635–22 646.
Wetherald, R. T., and S. Manabe, 1988: Cloud feedback processes in a general circulation model. J. Atmos. Sci., 45, 1397–1415.
Zhang, M. H., R. D. Cess, J. J. Hack, and J. T. Kiehl, 1994: Diagnosticstudy of climate feedback processes in atmospheric GCMs. J. Geophys. Res., 99, 5525–5537.
Summary
These notes introduce a few of the ideas and concepts associated with sensitivity analysis for algebraic and ordinary differential equations. By sensitivity I mean what are the effects of changes in the numerical values of the parameters in a system of equations relative to a response function of interest. The response function can take any mathematical form, but I will focus on the values of the dependent variables of the equation system.
Royal Meteorological Society V&V and SQA
Some time ago it was suggested to me something to the effect that most of the people in the Climate Change Community to whom I suggest V&V and SQA issues are critical and require significant attention don’t even know what I’m talking about. Here’s some information that lends support to that observation.
The Fundamental Issue
The Intergovernmental Panel for Climate Change (IPCC) Fourth Assessment Report (AR4, IPCC, 2007) states:
“A major advance of this assessment of climate change projections compared with the TAR is the large number of simulations available from a broader range of models. Taken together with additional information from observations, these provide a quantitative basis for estimating likelihoods for many aspects of future climate change.” [ My bolding. ]
Do the numbers from these “large number of simulations available from a broader range of models” GCM calculations have any Meaning. My answer is No.
One crucial and necessary first step is that application of Verification procedures have shown that the numbers produced by the software accurately reflect both (1) the original intent of the continuous equations for the models, and (2) the numerical solution methods applied to the discrete approximations to the continuous equations. That is, Verification shows that the equations have been solved right. Do the numbers actually satisfy the Verified-to-be-correct-as-coded discrete equations and do the solutions of the discrete equations converge to solution of the continuous equations. Neither of these has been demonstrated for any GCM. I will be pleased to be shown to be wrong on this point.
All software can be Verified. Objective technical criteria and associated success metrics can be developed and applied in a manner that provides assurances about the correctness of the coding of the equations and their numerical solutions. Lack of Verification leaves open the potential that the numbers from the software are simply results of “bugs” in the coding.
The present-day software development community, in all kinds of applications and organizations, is keenly aware that lack of SQA policies and procedures, and successful applications of these to the software, leaves open a significant potential for problems to exist in the software. So far as I am aware, there are no precedents whatsoever for public policy decisions to be based on software for which no SQA procedures have been applied.
GISS/NASA: We Don’t Verify. We Calculate, you Verify.
Another example of the lack of even the most basic concerns with verification of calculations has been discovered in GISS/NASA work.
The response by GISS/NASA employees is extremely disconcerting. That response seems to me to say that if the application procedure simply produces numbers, we’re ok with that. Verification of the numbers is left to others.
Models Methods Software 
