Averaging Planet Earths or Averaging Planet Xs
Update January 23, 2011. I fixed the broken links.
There are several discussions floating around on the subject of comparing GCM-calculated numbers with experimental data. Climate Audit and Lucia both have several threads. There are too many threads to give links; let me know if you need a specific thread. One focus of these discussions is the ensemble-average approach that is considered to be necessary for the comparisons. The ensemble is made up of the results from the various different versions of GCMs that calculate the results. Not only are GCMs different from each other, but the suggested approach is to make perturbations in the initial conditions between the runs.
I have mentioned that the results do not reflect the effects of perturbations in only the initial conditions. Everything about the approach is different between the runs. The GCMs are based on different continuous-equation models, numerical solution methods, application procedures, run-time options, and users, and maybe other aspects. How can we be sure that the results are all appropriate for Earth?
How do we know we are averaging different results for Earth as opposed to averaging different planets. I have suggested that some kind of filtering mechanism is needed in order to eliminate the calculated numbers that might not be Earth. Why, for example, can’t a model that gives a constant value for the Global Mean Surface Temperature be used in the ensemble.
It is my impression that there are no filtering mechanisms in place. The only requirement for being a part of the ensemble is that your code has calculated the numbers. All the available numbers get tossed into the ensemble. A very rough approach to validation of models, in my opinion.
The ensemble average approach is used, as I understand it, because of the non-deterministic nature of the models / methods / codes / applications / users. Additionally, this heuristic, ad hoc, osmosis-like hypothesis is generally based on the chaotic response observed with the original Lorenz equation system of 1963. Those equations exhibit chaotic response for some ranges of the values of the parameters in the system. These parameters are roughly the Rayleigh and Prandtl Numbers. And while most analyses of the system focus on perturbations in the initial conditions, it is well known that changes in the parameters and the use of different step sizes in the numerical integration also exhibit chaotic response. The Lorenz systems, in general, are ill-posed initial value problems.
For this discussion let’s take chaotic response to mean that given small perturbations, the solution trajectories do not approach fixed equilibrium state points and do not exhibit periodic response; aperiodic response is obtained. It is equally well known that for certain ranges of the parameters, the original Lorenz equation system exhibits well-behaved response and the dependent variables attain fixed equilibrium state points. And, small changes in the coefficients of some of the terms in the original system gives systems that rapidly approach fixed equilibrium states. Finally, we note that while the parameters have constant values in the Lorenz system, the corresponding quantities in the GCMs will vary during the course of the calculations.
So, now back to the original question. Given the wide range of observed responses how do we know that we’ve included only different versions of Earth into the ensemble. How do we know we have not introduced different planets into the ensemble?
I have discussed the original Lorenz system and associated numerical solution methods in several posts on this blog. The equation system is summarized here, the numerical solution methods are summarized here, and some results are summarized here. For this post I have used the explicit Euler method although I’m certain that any other of the methods would produce the same general results. I used the same step size for the independent variable for all the runs discussed here.I have made several runs as follows:
(1) A base-case with selected values of the initial condition and parameters.
(2) A run with a small perturbation in the initial values for the first dependent variable and all other held at the base-case values.
(3) A run with a small perturbation in the initial values for the second dependent variable and all other held at the base-case values.
(4) A run with a small perturbation in the initial values for the third dependent variable and all other held at the base-case values.
At this point I made an ensemble and calculated the average values of the dependent variables. I will get to some details of the results below.
I then made additional runs so as to include a few more versions of Earth into the ensemble.
(5) A run with the base-case values of the initial conditions and the Prandtl Number, but with a change in the Rayleigh Number.
(6) A run with the base-case values of the initial conditions and the Rayleigh Number but with a change in the Prandtl Number.
(7) A run with the base-case values of the initial conditions and changes in both the Prandtl and Rayleigh Numbers.
At this point I made another ensemble including the seven cases. Note that there are a very large number of combinations of changes that could be made, and if we include the magnitude of the changes as an additional parameter, the total number of possibilities grows without bound.
I have up-loaded a plot of the results here. If you successfully get the plot, here’s what you’ll see.
The bottom of the plot shows the results of the 4-run ensemble for the Y(3) dependent variable. All four runs are shown plus the average value, which is the thick black line running through the cloud of results. The top part shows the average value for Y(3) as given by the 7-run ensemble. I didn’t include the component results because the cloud would be even more cloudy.
Well, what has happened? It looks like I’ve picked up a Planet X somewhere in my calculations for Planet Earth. And that Planet dominates the calculated results. Full disclosure; I have not yet looked at any of the other dependent variables in this detail. Looking at the results for Y(3) I see that the run for which the value of both parameters were changed has made Planet X.
This was my zeroth-order cut at trying to find an answer to the question, How do we know we are averaging different results for Earth as opposed to averaging different planets. I think it provides some validity to the issue.
I have suggested that some kind of filtering mechanism is needed in order to eliminate the calculated numbers that might not be Earth.