Python example recipe fails when using freshly downloaded data #4308
Description
Describe the bug
Running esmvaltool run examples/recipe_python.yml with freshly downloaded CMIP5 and CMIP6 data using intake-esgf results in
ValueError: Multi-model statistics failed to merge input cubes into a single array:
0: air_temperature / (degrees_C) (time: 151)
1: air_temperature / (degrees_C) (time: 151)
Coordinates in cube.aux_coords (scalar) differ by dtype or class (i.e. DimCoord vs AuxCoord).
Data sources
CMIP5:
data:
intake-esgf:
facets:
dataset: model
ensemble: ensemble
exp: experiment
frequency: time_frequency
institute: institute
mip: cmor_table
product: product
project: project
short_name: variable
priority: 2
type: esmvalcore.io.intake_esgf.IntakeESGFDataSource
values:
dataset:
ACCESS1-0: ACCESS1.0
ACCESS1-3: ACCESS1.3
CESM1-BGC: CESM1(BGC)
CESM1-CAM5: CESM1(CAM5)
CESM1-CAM5-1-FV2: CESM1(CAM5.1,FV2)
CESM1-FASTCHEM: CESM1(FASTCHEM)
CESM1-WACCM: CESM1(WACCM)
CSIRO-Mk3-6-0: CSIRO-Mk3.6.0
GFDL-CM2p1: GFDL-CM2.1
MRI-AGCM3-2H: MRI-AGCM3.2H
MRI-AGCM3-2S: MRI-AGCM3.2S
bcc-csm1-1: BCC-CSM1.1
bcc-csm1-1-m: BCC-CSM1.1(m)
fio-esm: FIO-ESM
inmcm4: INM-CM4
local:
dirname_template: '{project.lower}/{product}/{institute}/{dataset}/{exp}/{frequency}/{modeling_realm}/{mip}/{ensemble}/{version}'
filename_template: '{short_name}_{mip}_{dataset}_{exp}_{ensemble}*.nc'
rootpath: /mnt/d/data
type: esmvalcore.io.local.LocalDataSource
preprocessor_filename_template: '{project}_{dataset}_{mip}_{exp}_{ensemble}_{short_name}'
CMIP6:
data:
intake-esgf:
facets:
activity: activity_drs
dataset: source_id
ensemble: member_id
exp: experiment_id
grid: grid_label
institute: institution_id
mip: table_id
project: project
short_name: variable_id
priority: 2
type: esmvalcore.io.intake_esgf.IntakeESGFDataSource
local:
dirname_template: '{project}/{activity}/{institute}/{dataset}/{exp}/{ensemble}/{mip}/{short_name}/{grid}/{version}'
filename_template: '{short_name}_{mip}_{dataset}_{exp}_{ensemble}_{grid}*.nc'
rootpath: /mnt/d/data
type: esmvalcore.io.local.LocalDataSource
preprocessor_filename_template: '{project}_{dataset}_{mip}_{exp}_{ensemble}_{short_name}_{grid}'Running the recipe with existing data on Levante works fine. In both cases I use iris=3.13.1.
I guess the reason is that the height coordinate is interpreted as a DimCoord in the CMIP5 dataset, and as an AuxCoord in the CMIP6 dataset:
# CMIP5
DimCoord : height / (m)
points: [2.]
shape: (1,)
dtype: float64
standard_name: 'height'
long_name: 'height'
var_name: 'height'
attributes:
positive 'up'
# CMIP6
AuxCoord : height / (m)
points: [2.]
shape: (1,)
dtype: float64
standard_name: 'height'
long_name: 'height'
var_name: 'height'
attributes:
positive 'up'ncdumps
netcdf tas_Amon_BCC-ESM1_historical_r1i1p1f1_gn_185001-201412 {
dimensions:
time = UNLIMITED ; // (1980 currently)
lat = 64 ;
lon = 128 ;
bnds = 2 ;
variables:
double time(time) ;
time:bounds = "time_bnds" ;
time:units = "days since 1850-01-01" ;
time:calendar = "365_day" ;
time:axis = "T" ;
time:long_name = "time" ;
time:standard_name = "time" ;
double time_bnds(time, bnds) ;
double lat(lat) ;
lat:bounds = "lat_bnds" ;
lat:units = "degrees_north" ;
lat:axis = "Y" ;
lat:long_name = "latitude" ;
lat:standard_name = "latitude" ;
double lat_bnds(lat, bnds) ;
double lon(lon) ;
lon:bounds = "lon_bnds" ;
lon:units = "degrees_east" ;
lon:axis = "X" ;
lon:long_name = "Longitude" ;
lon:standard_name = "longitude" ;
double lon_bnds(lon, bnds) ;
double height ;
height:units = "m" ;
height:axis = "Z" ;
height:positive = "up" ;
height:long_name = "height" ;
height:standard_name = "height" ;
float tas(time, lat, lon) ;
tas:standard_name = "air_temperature" ;
tas:long_name = "Near-Surface Air Temperature" ;
tas:comment = "near-surface (usually, 2 meter) air temperature" ;
tas:units = "K" ;
tas:original_name = "TREFHT" ;
tas:cell_methods = "area: time: mean (interval: 20 minutes)" ;
tas:cell_measures = "area: areacella" ;
tas:history = "2018-12-14T05:25:12Z altered by CMOR: Treated scalar dimension: \'height\'." ;
tas:coordinates = "height" ;
tas:missing_value = 1.e+20f ;
tas:_FillValue = 1.e+20f ;
// global attributes:
:Conventions = "CF-1.7 CMIP-6.2" ;
:activity_id = "CMIP" ;
:branch_method = "Standard" ;
:branch_time_in_child = 0. ;
:branch_time_in_parent = 2110. ;
:comment = "The model integration starts from the piControl experiment equilibrium state (1st Jan. of the year 2110)" ;
:contact = "Dr. Tongwen Wu([email protected])" ;
:creation_date = "2018-12-14T05:25:12Z" ;
:data_specs_version = "01.00.27" ;
:description = "DECK: historical" ;
:experiment = "all-forcing simulation of the recent past" ;
:experiment_id = "historical" ;
:external_variables = "areacella" ;
:forcing_index = 1 ;
:frequency = "mon" ;
:further_info_url = "https://furtherinfo.es-doc.org/CMIP6.BCC.BCC-ESM1.historical.none.r1i1p1f1" ;
:grid = "T42" ;
:grid_label = "gn" ;
:history = "2018-12-14T05:25:12Z ; CMOR rewrote data to be consistent with CMIP6, CF-1.7 CMIP-6.2 and CF standards.;\n",
"N/A" ;
:initialization_index = 1 ;
:institution = "Beijing Climate Center, Beijing 100081, China" ;
:institution_id = "BCC" ;
:mip_era = "CMIP6" ;
:nominal_resolution = "250 km" ;
:parent_activity_id = "CMIP" ;
:parent_experiment_id = "piControl" ;
:parent_mip_era = "CMIP6" ;
:parent_source_id = "BCC-ESM1" ;
:parent_time_units = "days since 1850-01-01" ;
:parent_variant_label = "r1i1p1f1" ;
:physics_index = 1 ;
:product = "model-output" ;
:realization_index = 1 ;
:realm = "atmos" ;
:references = "Model described by Tongwen Wu et al. (JGR 2013; JMR 2014; submmitted to GMD,2018). Also see http://forecast.bcccsm.ncc-cma.net/htm" ;
:run_variant = "forcing: greenhouse gases,aerosol emission,solar constant,volcano mass,land use" ;
:source = "BCC-ESM 1 (2017): aerosol: none atmos: BCC_AGCM3_LR (T42; 128 x 64 longitude/latitude; 26 levels; top level 2.19 hPa) atmosChem: BCC-AGCM3-Chem land: BCC_AVIM2 landIce: none ocean: MOM4 (1/3 deg 10S-10N, 1/3-1 deg 10-30 N/S, and 1 deg in high latitudes; 360 x 232 longitude/latitude; 40 levels; top grid cell 0-10 m) ocnBgchem: none seaIce: SIS2" ;
:source_id = "BCC-ESM1" ;
:source_type = "AER AOGCM CHEM" ;
:sub_experiment = "none" ;
:sub_experiment_id = "none" ;
:table_id = "Amon" ;
:table_info = "Creation Date:(30 July 2018) MD5:e53ff52009d0b97d9d867dc12b6096c7" ;
:title = "BCC-ESM1 output prepared for CMIP6" ;
:tracking_id = "hdl:21.14100/e6e5eb74-9e45-44db-9dbe-d6236596b068" ;
:variable_id = "tas" ;
:variant_label = "r1i1p1f1" ;
:license = "CMIP6 model data produced by BCC is licensed under a Creative Commons Attribution ShareAlike 4.0 International License (https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file) and at https:///pcmdi.llnl.gov/. The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law." ;
:cmor_version = "3.3.2" ;
}
netcdf tas_Amon_bcc-csm1-1_historical_r1i1p1_185001-201212 {
dimensions:
time = UNLIMITED ; // (1956 currently)
lat = 64 ;
lon = 128 ;
bnds = 2 ;
variables:
double time(time) ;
time:bounds = "time_bnds" ;
time:units = "days since 1850-01-01" ;
time:calendar = "noleap" ;
time:axis = "T" ;
time:long_name = "time" ;
time:standard_name = "time" ;
double time_bnds(time, bnds) ;
double lat(lat) ;
lat:bounds = "lat_bnds" ;
lat:units = "degrees_north" ;
lat:axis = "Y" ;
lat:long_name = "latitude" ;
lat:standard_name = "latitude" ;
double lat_bnds(lat, bnds) ;
double lon(lon) ;
lon:bounds = "lon_bnds" ;
lon:units = "degrees_east" ;
lon:axis = "X" ;
lon:long_name = "longitude" ;
lon:standard_name = "longitude" ;
double lon_bnds(lon, bnds) ;
double height ;
height:units = "m" ;
height:axis = "Z" ;
height:positive = "up" ;
height:long_name = "height" ;
height:standard_name = "height" ;
float tas(time, lat, lon) ;
tas:standard_name = "air_temperature" ;
tas:long_name = "Near-Surface Air Temperature" ;
tas:units = "K" ;
tas:original_name = "TREFHT" ;
tas:cell_methods = "time: mean (interval: 20 mintues)" ;
tas:cell_measures = "area: areacella" ;
tas:history = "2011-06-15T08:55:23Z altered by CMOR: Treated scalar dimension: \'height\'." ;
tas:coordinates = "height" ;
tas:missing_value = 1.e+20f ;
tas:_FillValue = 1.e+20f ;
tas:associated_files = "baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_bcc-csm1-1_historical_r0i0p0.nc areacella: areacella_fx_bcc-csm1-1_historical_r0i0p0.nc" ;
// global attributes:
:institution = "Beijing Climate Center(BCC),China Meteorological Administration,China" ;
:institute_id = "BCC" ;
:experiment_id = "historical" ;
:source = "bcc-csm1-1:atmosphere: BCC_AGCM2.1 (T42L26); land: BCC_AVIM1.0;ocean: MOM4_L40 (tripolar, 1 lon x (1-1/3) lat, L40);sea ice: SIS (tripolar,1 lon x (1-1/3) lat)" ;
:model_id = "bcc-csm1-1" ;
:forcing = "Nat Ant GHG SD Oz Sl Vl SS Ds BC OC" ;
:parent_experiment_id = "piControl" ;
:parent_experiment_rip = "r1i1p1" ;
:branch_time = 470. ;
:contact = "Dr. Tongwen Wu ([email protected])" ;
:history = "Output from monthly mean data 2011-06-15T08:55:23Z CMOR rewrote data to comply with CF standards and CMIP5 requirements." ;
:comment = "The experiment starts from piControl run at year 470. RCP8.5 scenario forcing data are used beyond year 2005." ;
:initialization_method = 1 ;
:physics_version = 1 ;
:tracking_id = "2ccf4c16-f21b-42f4-80d1-0dd0f131e922" ;
:product = "output" ;
:experiment = "historical" ;
:frequency = "mon" ;
:creation_date = "2011-06-15T08:55:23Z" ;
:Conventions = "CF-1.4" ;
:project_id = "CMIP5" ;
:table_id = "Table Amon (11 April 2011) 1cfdc7322cf2f4a32614826fab42c1ab" ;
:title = "bcc-csm1-1 model output prepared for CMIP5 historical" ;
:parent_experiment = "pre-industrial control" ;
:modeling_realm = "atmos" ;
:realization = 1 ;
:cmor_version = "2.5.6" ;
}
Please attach