Climate.ruthner.at← ruthner.atWeather Amstetten →DEEN

Climate models against the measurement

Do the computer models match the measurement? Here are the comparison of sea and air temperature, the block “For specialists” with multiple runs and quality check, and the scenarios to 2100.

How to read: Measurement measured values · Reconstruction from indirect data such as ice or tree rings · Model computer model · Own calculation calculated by me from raw data. Technical terms are explained in the glossary.

Climate models · global air temperature
ModelMeasurementOwn calculation

The same with the Earth’s air temperature

Models:Model variable:

Selection by climate sensitivity (ECS): ECS is the warming a model produces for a doubling of the CO₂ content (values after Zelinka et al. 2020, available for 38 of the 41 models). The IPCC regards 2.5 to 4 °C as the likely range and gave models with very high sensitivity ("hot models") less weight in its projections. This is a simple selection by ECS, not a statistical weighting as in the IPCC report.

Same quantity: The HadCRUT5 measurement series blends air temperature over land with the temperature of the sea surface. "Blended" calculates the models in the same way (air temperature over land, sea surface over water, using the land fraction of the respective model) and is therefore the cleaner comparison. This works for the models whose land mask is available in the data (see the number of models above). Not corrected are gaps in the spatial coverage of the measurement and the difference between the SSP2-4.5 scenario and the actual forcing (greenhouse gases, aerosols, volcanoes, sun) since 2015.

AIR · Air temperature of the whole Earth (2 m height, land and sea) 41 climate models (CMIP6) against the measurements HadCRUT5 and Berkeley Earth

Please note when reading: (1) Same measured quantity: The measurement mixes air temperature over land with water temperature over the sea; the appropriate comparison is therefore “Blended” (switch above), not “Air temperature”. (2) One run per model: Each run contains its own variability (e.g. El Niño). “This model warms faster” applies only to this one run, not to the model as a whole; the model mean is more reliable. (3) Not a pure hindcast: Up to 2014 these are reconstructions of the past; from 2015 the models run with the scenario SSP2-4.5 instead of the actual forcing, so the 1979–2020 trend includes six years of scenario. (4) Not 41 independent forecasts: The models come from different institutes but share program components, data and physical approaches. Figures such as “35 of 41” count models and are not a statistical vote.

Interactive chart (hover the mouse or a finger over the curves), calculated by myself from the raw data of 41 CMIP6 models (one run each, Pangeo/Google Cloud archive) and the measurement series HadCRUT5 (Met Office) and Berkeley Earth. 12-month mean; the chart first shows 1979 to 2046, and with the “show from 1850” switch the whole series since 1850. Expandable below: how exactly the calculation was done.

What it shows: The mean air temperature of the whole Earth (2 m above the ground, over land and sea), in each case as an anomaly from the 1961–1990 mean. The coloured lines (blue: weaker, red: stronger warming) are 41 climate models (CMIP6); the thick white line is the HadCRUT5 measurement series, the dotted light line Berkeley Earth (to 2024). Up to 2014 these are the hindcasts of the past (historical); after that the models continue with the scenario SSP2-4.5 to 2046; from 2026 the chart is a projection, not a measurement.

What you see: Of 41 models, 32 warm more strongly than the HadCRUT5 measurement series from 1979 to 2020 (39 more strongly than Berkeley Earth). The measured global air temperature rises by 0.194 °C per decade (HadCRUT5; Berkeley Earth 0.167), the models on average by 0.253 (median 0.250, range 0.143 to 0.416). Up to the end of 2025 the measurement rises by 0.207 °C per decade, the models by 0.261; 32 of 41 models warm more strongly. By 2046 the models project on average 1.76 °C above the 1961–1990 mean; the measurement at the end of 2025 is at 1.07 °C, the model mean at the same time at 1.18 °C. Note: The models are not independent forecasts, and every figure is based on one run per model.

Same quantity and IPCC range: The model selection can be changed above the chart. If the models are calculated in the same way as the measurement (air temperature over land, sea surface over water; 30 models) and only models with a climate sensitivity in the IPCC range of 2.5 to 4 °C are used (12 models), the model mean is 0.18 °C per decade, close to the measurement (HadCRUT5: 0.19); 4 of 12 models then warm more strongly. With all 30 blended models it is 0.21 (16 of 30).

On scope: The selection is one run per model (mostly r1i1p1f1). In the measurement series, air temperature is measured over land, but water temperature over the sea; the model air temperature applies everywhere at 2 m height. This difference makes the models warm somewhat more strongly on paper and has not been removed here.

Trend 1979–2020 per model, °C per decade
Trend 1979–2020 per model, °C per decade
For experts: Several runs per model and quality check
ModelMeasurementOwn calculation

Several runs per model: how much is chance, how much is the model?

A climate model does not deliver a single curve: if it is started with slightly different initial conditions, different paths result, because natural variability (for example El Niño) develops differently each time. For the 23 models that have at least three comparable runs, the chart shows all runs of a model, their mean and the range. Data basis: 286 suitable runs, 236 of them in the charts; 50 runs of EC-Earth3 are missing because they only start in 1970 and therefore have no 1961–1990 mean as a common reference period (explained below under “How this was calculated”). Comparing the scatter within a model with the differences between the models shows whether a departure from the measurement can be due to chance or belongs to the model.

Caution: Here the model quantity is the plain air temperature, while the measurement (HadCRUT5) mixes air over land with water over the sea. The comparison is therefore not entirely fair; the multiple runs are not available for the blended comparison.
Model:

Window:

Trend per run, °C per decade; dot = one run, golden bar = model mean; a click on a row selects the model above

The result depends on the time window

Internal variability and model difference over time

For each year the chart separates two sources of scatter of the annual means: the internal climate variability, i.e. the random fluctuation (scatter of the runs within a model, averaged over the models) and the model difference (scatter of the model means, corrected for the noise of the runs). The larger the model difference, the more the outcome depends on which model one looks at.

In short
  • A single model run is only one of many possible developments: even through internal climate variability, its 1979–2020 trend deviates by about ±0.07 °C per decade (two standard deviations) from the mean of its model.
  • Whether the measurement “fits a model” depends on the chosen time window; short windows say little.
  • For the distant future the difference between the models clearly outweighs internal variability.
  • The charts show how widely model statements scatter, not which model is “right”.

Reading notes: (1) The model lines here are the plain air temperature at 2 m height; the measurement mixes air temperature over land with water temperature over the sea. The multiple runs are not available for the “blended” comparison (switch above). (2) Several runs of the same model are not independent models; different models also share program components. (3) Range and mean apply only where at least three runs have values; the runs end at different times. (4) The measurement uncertainty is drawn as a grey band: the 95 % band of the 200 realisations of HadCRUT5 (version 5.2.0.0); in addition the second measurement series Berkeley Earth as a dashed line. For the count “within range” the range from the smaller to the larger of these two statements applies. Not included is the uncertainty of the forcings (from 2015 all runs use the same scenario SSP2-4.5, so scenario uncertainty is missing). (5) The reference period 1961–1990 makes the model differences look small at the start of the time series; this is a consequence of the calculation. Measured trends in this section are calculated from annual means, like those of the runs.

How this was calculated: runs, selection, limits

Data and selection

  • Source: CMIP6 archive “Pangeo” (Google Cloud), variable tas (air temperature at 2 m, monthly means), historical up to 2014 and ssp245 from 2015.
  • Which runs: All runs were included that have the same physics and forcing variant (the label has the form rrunnumberi1p1f1, for example r1i1p1f1, r2i1p1f1, r3i1p1f1 and so on; p stands for the physics variant, f for the forcings) as the run shown so far and that exist for both periods. Not selected by how well they match the measurement.
  • Scope: 41 models were available. 23 of them have at least three such runs, 286 runs in total. Of these, 236 are in the charts: for EC-Earth3 (72 runs), 50 only start in 1970 and have no 1961–1990 mean, so they are not included, to keep the reference period the same for all runs (EC-Earth3 shows 22 runs). Some runs end before 2100 (7 until 2020, 17 until 2039); they are included and end earlier in the charts.

Calculation steps

  1. Global mean for each month, weighted with the cosine of latitude; annual mean from the 12 months.
  2. Departure: from the 1961–1990 mean of the respective run.
  3. Mean and range: for each year the mean, the lowest and the highest value across the runs of a model (only where at least three runs have values).
  4. Trend: linear slope of the annual means for each run, in °C per decade, for the windows 1950–2020, 1979–2020, 1990–2020 and 2000–2020. The measured trends (HadCRUT5, Berkeley Earth) are calculated in the same way from the annual means of the measurement series; they therefore differ in the third decimal from the trend of the chart above (calculated there from monthly values).
  5. Chance and model difference: for each year the mean variance of the runs within a model (chance) and the variance of the model means minus the noise of the runs (variance within divided by the number of runs; model difference), shown as standard deviation. Only models with at least three runs with a value; at least five models per year.
ModelMeasurementOwn calculation

Quality check: how well do the models match the measurement?

The matrix tests each of the 23 models against eight global measured quantities, separately for air temperature, precipitation and sea surface (for MRI-ESM2-0 the sea data are missing), and against six quantities for Europe (trend of temperature and precipitation in Central Europe, the Alpine region and Austria, see “Europe” below). Each field shows whether the measurement lies within the range of the runs of the model (●), whether the model lies above (▲) or below (▼). The numbers appear on hover (or tap). A click on a column heading sorts by the model mean of that quantity. There is deliberately no overall score: a model that does well on one quantity is not the best overall, the quantities are not equally important, and the measured values are not equally certain.

Caution: The quantities are not measured with equal certainty. The models deliver air temperature everywhere, the measurement series mix air and water and have gaps; absolute precipitation values and the absolute temperature mean are uncertain even between measurement products.

Reading notes: (1) The measurement uncertainty differs by quantity. For the warming trend the measurement is a range: the 95 % band of the 200 HadCRUT5 realisations (0.188 to 0.202 °C per decade) together with Berkeley Earth (0.168). For the absolute mean the measurement depends on the definition (air above sea ice 14.10 °C, water below sea ice 14.70 °C, Berkeley Earth); precipitation comes from two measurement products (GPCP and CMAP); the measurement row shows their range. The two products contradict each other for the trend (GPCP +0.18, CMAP −1.49 % per decade), so the verdict on the precipitation trend is particularly uncertain. (2) The models deliver air temperature everywhere, the measurement series mix air and water and have gaps in coverage; seasonal cycle and variability are affected by this. (3) Models with few runs have narrow ranges, so the measurement falls inside less often. (4) The runs of a model are not independent of other models. (5) The sea surface (60° South to 60° North) is the fairest column: model and measurement (ERSSTv5) describe the same quantity. Regions (Europe) are not yet included.

For specialists: Scenarios to 2100 (SSP1-2.6, SSP2-4.5, SSP5-8.5)
ModelMeasurementOwn calculation

Scenarios: how does the warming to 2100 depend on the assumptions?

Climate models do not calculate the future with a forecast of emissions but with assumptions about how population, economy, energy and land use develop (“scenarios”, SSP). Three of them are shown, named after the additional radiative forcing in the year 2100 in watts per square metre: SSP1-2.6 (low emissions), SSP2-4.5 (middle road, used everywhere else on the page so far) and SSP5-8.5 (high emissions). Until 2014 all runs follow the same course (historical), after that they diverge. These are 22 of the 23 models from above (for GISS-E2-1-G there are no runs in all three scenarios), for each model the runs (up to five) that are available in all three scenarios.

Caution: Scenarios are if-then calculations and not forecasts: which scenario occurs depends on decisions that the models do not predict. The models are also not all equally sensitive; some warm more strongly than the measurement of recent decades suggests (see the quality check above). Nothing was filtered out here.
Scenarios:Display:

Warming in numbers

In short
  • Until about 2040 the scenarios lie close together; after that the warming diverges because the emissions differ.
  • The range within a scenario (model difference) is large: it arises because the models respond differently strongly to CO₂.
  • The chart shows what the models give under certain assumptions, not what will happen.

Reading notes: (1) Runs: up to five per model (the first by run number, not by result), only those available in all three scenarios and with a full 1961–1990 mean. The model mean counts each model equally, regardless of the number of its runs; related models are not independent. (2) Measurement: HadCRUT5 mixes air over land with water over the sea, the models show pure air temperature. (3) Reference periods: the chart shows departures from the 1961–1990 mean; the table gives the warming relative to 1850–1900 (pre-industrial, from the run itself) and relative to 1995–2014. (4) Missing scenarios: SSP3-7.0 and SSP1-1.9 are not included. (5) The figures rest on these models only; they are not the assessed range of the IPCC, which also compares the models with measurements and further evidence.

How this was calculated – runs, periods, limits

Data

  • Source: CMIP6 archive “Pangeo” (Google Cloud), variable tas (air temperature at 2 m, monthly mean), historical to 2014 and ssp126, ssp245 or ssp585 from 2015, to 2100.
  • Runs: the first up to five of each model with the same physics and forcing variant that are available in all three scenarios.

Calculation steps

  1. Global mean per month, weighted by the cosine of latitude; annual means only from full twelve months.
  2. Departure from the 1961–1990 mean of the respective run (from historical).
  3. Mean of the runs per model; model mean and range (smallest and largest value of the models) per year.
  4. Warming 2081–2100 relative to 1850–1900 or 1995–2014: mean of the years minus mean of the reference years, per run, then averaged per model; in the table the mean, median and range across the models.
How this was calculated – data, steps, limits

1 · Data retrieved

  • Measurement: HadCRUT5 (Met Office Hadley Centre / University of East Anglia, version 5.2.0.0), global monthly means of surface temperature since 1850, a small table; Berkeley Earth (land and ocean, global monthly means, to December 2024). Both were converted to the 1961–1990 mean.
  • Models: CMIP6 archive “Pangeo” on Google Cloud (variable tas, air temperature at 2 m height, monthly mean), the same 41 models and runs as for sea temperature, historical to 2014 and ssp245 from 2015, read for 1850 to 2046.

2 · Calculated

  1. Global mean: for each month the mean over all grid cells of the Earth, weighted by the cosine of latitude (area correction).
  2. Anomaly: from each series (measurement and each model) its own mean over the 360 months of 1961–1990 is subtracted.
  3. Smoothing: moving average over 12 months (for readability only).
  4. Trend: linear slope of the monthly values (least squares), converted to °C per decade, for 1979–2020 and additionally to the end of 2025 (Berkeley Earth to the end of 2024).

3 · Result in numbers

Measurement 1979–2020: HadCRUT5 / Berkeley Earth0.194 / 0.167 °C/decade
Models: mean / median0,253 / 0,250
Models: weakest / strongest0,143 / 0,416
Models with a stronger trend than HadCRUT5 / Berkeley32 of 41 / 39 of 41
To the end of 2025: HadCRUT5 / models (mean)0.207 / 0.261 °C/decade
Model mean 2046 relative to 1961–19901,76 °C

4 · Limits and caveats

  • The selection (one run per model) is my decision; with other runs the numbers and the picture change.
  • Measurement series and models do not measure the same thing: the series combine air temperature over land with water temperature over the sea, whereas the model tas is air temperature everywhere. This makes the models warm somewhat more strongly; a correction (masking as in the measurements) has not been calculated here.
  • Both measurement series fill areas without measurements (for example the polar regions) statistically; the methods differ and lead to slightly different trends.
  • The cosine as an area weight is an approximation (no cell areas from the model files).
  • After 2014 the scenario SSP2-4.5 applies; up to about 2030 the scenarios differ hardly at all, after that they diverge. The lines from 2026 are projections under this assumption, not a prediction of the actual course.
  • A single run also contains random variability (for example El Niño).

5 · Sources

  • CMIP6: Eyring et al. 2016, via the Pangeo CMIP6 archive (storage.googleapis.com/cmip6), terms of use of the modelling centres (mostly CC BY 4.0).
  • HadCRUT5: Morice et al. 2021, J. Geophys. Res. Atmos., Met Office Hadley Centre.
  • Berkeley Earth: Rohde & Hausfather 2020, Earth System Science Data.
Statistics on the data – climate models and air temperature
Climate models (CMIP6, air temperature at 2 m)41 models, one run each, grid cells 56 to 417 km wide (median 139 km), 2,364 monthly values each (Jan 1850 – Dec 2046), 96,924 in total; averaged over the whole Earth for each month
HadCRUT5 measurement series (Met Office)2,112 monthly values (Jan 1850 – Dec 2025)
Berkeley Earth measurement series2,100 monthly values (Jan 1850 – Dec 2024)
CalculatedAnomaly from the 1961–1990 mean, 12-month moving average, trend 1979–2020 and to 2025 (Berkeley to 2024) per model and measurement series, model mean, range

Data as of November 2026. The data series end in different years depending on the source (mostly 2024 or 2025); the end point is given in the source note under each chart, an overview is on “Data and methods”.

Feedback

Values wrong, an idea or a question? The message goes directly to the author of the page. Name and email address are optional and are only used for a reply.