Hi Martin,
Building another server might not be the best use of resources. I'm always happy to extend the content I serve.
Cheers, Roy.
-----Original Message-----
From: martin.juckes at stfc.ac.uk [mailto:martin.juckes at stfc.ac.uk]
Sent: 27 October 2009 12:30
To: Lowry, Roy K; cf-metadata at cgd.ucar.edu
Cc: Weatherall, Pauline
Subject: RE: [CF-metadata] Dealing with large numbers of flag valuesinnetcdf cf
Hello Roy,
That is an interesting idea. There are definitions of these areas on a WWF site, of the form
http://www.worldwildlife.org/wildworld/profiles/terrestrial/nt/nt0908_full.html where "nt0908" is a tag associated with a particular flag value.
My first thought was:
URI="
http://www.worldwildlife.org/wildworld/profiles/terrestrial/[flag_meaning]_full.html" with "nt/nt0908" in the flag_meanings attribute, but it appears that the cf-checker does not like having a "/" in flag_meanings. This means we need to either have a more complex syntax, or set up our definition URLs along the lines you suggest.
A more complex syntax might be:
URI="
http://www.worldwildlife.org/wildworld/profiles/terrestrial/[flag_meaning][0:2]/[flag_meaning]_full.html"
Since the above returns html rather than skos/xml it should perhaps be complemented by a uri_content_type attribute, with value 'html/text'.
I'll think about the option of setting up something like your vocab server -- or possibly putting some more definitions in the ndg vocab server?
Regards,
Martin
> -----Original Message-----
> From: Lowry, Roy K [mailto:rkl at bodc.ac.uk]
> Sent: 27 October 2009 11:07
> To: Juckes, Martin (STFC,RAL,SSTD); cf-metadata at cgd.ucar.edu
> Cc: Weatherall, Pauline
> Subject: RE: [CF-metadata] Dealing with large numbers of flag
> valuesinnetcdf cf
>
> Hello Martin,
>
> There is another possible solution to your problem, which we are
> looking at for dealing with a data source flag to be used with the
> GEBCO bathymetric grid. This is to put a URI base into an attribute
> that when concatenated with a flag values gives the flag definition
> from a vocabulary server. For example, having a flag_definition
> attribute like:
>
> URI_base='http://vocab.ndg.nerc.ac.uk/term/GGS1/current/'
>
> For a flag value of 1, the resulting URL is:
>
> http://vocab.ndg.nerc.ac.uk/term/GGS1/current/1
>
> which is a live NERC Vocabulary Server URL returning the flag
> definition attributes in a SKOS document.
>
> I appreciate that this moves away from fully self-contained CF files
> and that a standardised syntax for the URI base encoding is required,
> but I think this provides a scalable solution to the flag definition
> problem.
>
> Cheers, Roy.
>
>
> -----Original Message-----
> From: cf-metadata-bounces at cgd.ucar.edu [mailto:cf-metadata-
> bounces at cgd.ucar.edu] On Behalf Of martin.juckes at stfc.ac.uk
> Sent: 27 October 2009 10:45
> To: cf-metadata at cgd.ucar.edu
> Subject: Re: [CF-metadata] Dealing with large numbers of flag values
> innetcdf cf
>
> Thanks for the responses.
>
> Seth is right, I was looking for an alternative to having a gigantic
> "flag_meanings" attribute.
>
> I have tried Seth's approach, but can?t get it past the cf-checker.
> There appears to be a character set restriction: I can get the first
> 547 flag_meaning strings accepted if I remove all the "-", "." and ","
> characters. I'm reasonably sure there are no control characters in the
> following strings which could be responsible for the error message that
> I get (ERROR (3.5): Invalid syntax for 'flag_meanings' attribute) when
> I increase the number of flag_meanings to 550. Could it be something to
> do with the string length (16472 characters)?
>
> I could try abbreviating the flag_meanings, but this would either be
> tedious work by hand or liable to create confusion if done
> automatically. If you could point me to some information about exactly
> what string format is acceptable, that would be very helpful.
>
> To answer John's question: I'm currently using netcdf 3, in IDL -- I'm
> not sure what the options for upgrading to netcdf 4 are.
>
> Cheers,
> Matin
>
> > -----Original Message-----
> > From: Seth McGinnis [mailto:mcginnis at ucar.edu]
> > Sent: 26 October 2009 23:13
> > To: John Graybeal; Juckes, Martin (STFC,RAL,SSTD)
> > Cc: cf-metadata at cgd.ucar.edu
> > Subject: Re: [CF-metadata] Dealing with large numbers of flag values
> > innetcdf cf
> >
> > On Mon, 26 Oct 2009 12:16:17 -0700
> > John Graybeal <graybeal at marinemetadata.org> wrote:
> > >Love the question! :->
> > >
> > >Are you saying every map location has one index value, which is one
> of
> > the
> > >numbers from 1 to 827? So these are mutually exclusive? Or are
> > there 827
> > >flag values assigned independently for each map location?
> >
> > It sounds to me like it's the former case, and the problem is that
> > concatenating 827 flag values into a single string attribute makes
> > something that is easy for a machine to interpret, but impenetrable
> to
> > the human reader.
> >
> > As I read section 3.5, there are no other options for encoding the
> > information according to spec; a gigantic flag_meanings attribute is
> > the right answer.
> >
> > But if the issue is really usability, I think the answer is not to do
> > it differently, but just to add an extra attribute that presents the
> > same information in a more user-friendly format. You could add an
> > attribute named something descriptive, like "category_legend", and
> > pair the values & meanings up, one per line. Then you'd end up with
> > something that looks like this:
> >
> >
> > float landtype(yc, xc) ;
> > landtype:long_name ="Land cover type" ;
> > landtype:standard_name ="land_cover" ;
> > landtype:flag_values = 1.f, 2.f, 3.f, 4.f, 5.f, 6.f, 7.f,
> 8.f,
> > 9.f,
> > 10.f, 11.f, 12.f, 13.f, 14.f, 15.f, 16.f, 17.f, 18.f, 19.f, 20.f,
> 21.f,
> > 22.f,
> > 23.f, 24 .f;
> > landtype:flag_meanings ="Urban_and_Built-up_Land
> > Dryland_Cropland_and_Pasture Irrigated_Cropland_and_Pasture
> > Mixed_Dryland/Irrigated_Cropland_and_Pasture
> Cropland/Grassland_Mosaic
> > Cropland/Woodland_Mosaic Grassland Shrubland
> Mixed_Shrubland/Grassland
> > Savanna
> > Deciduous_Broadleaf_Forest Deciduous_Needleleaf_Forest
> > Evergreen_Broadleaf
> > Evergreen_Needleleaf Mixed_Forest Water_Bodies Herbaceous_Wetland
> > Wooden_Wetland Barren_or_Sparsely_Vegetated Herbaceous_Tundra
> > Wooded_Tundra
> > Mixed_Tundra Bare_Ground_Tundra Snow_or_Ice" ;
> > landtype:category_legend ="\n",
> > "1: Urban and Built-up Land \n",
> > "2: Dryland Cropland and Pasture \n",
> > "3: Irrigated Cropland and Pasture \n",
> > "4: Mixed Dryland/Irrigated Cropland and Pasture \n",
> > "5: Cropland/Grassland Mosaic \n",
> > "6: Cropland/Woodland Mosaic \n",
> > "7: Grassland \n",
> > "8: Shrubland \n",
> > "9: Mixed Shrubland/Grassland \n",
> > "10: Savanna \n",
> > "11: Deciduous Broadleaf Forest \n",
> > "12: Deciduous Needleleaf Forest \n",
> > "13: Evergreen Broadleaf \n",
> > "14: Evergreen Needleleaf \n",
> > "15: Mixed Forest \n",
> > "16: Water Bodies \n",
> > "17: Herbaceous Wetland \n",
> > "18: Wooden Wetland \n",
> > "19: Barren or Sparsely Vegetated \n",
> > "20: Herbaceous Tundra \n",
> > "21: Wooded Tundra \n",
> > "22: Mixed Tundra \n",
> > "23: Bare Ground Tundra \n",
> > "24: Snow or Ice" ;
> >
> > (Sorry for the length, but it gives a real sense of what I mean.)
> >
> > The flag_values and flag_meanings attributes are still a mess to look
> > at, but they're easy to deal with programmatically, while end-users
> > can easily look at category_legend and figure out what cover type 22
> > is if that's all they're after.
> >
> > Fundamentally, this is the same issue as the question of how to
> record
> > the grid mapping, and the underlying problem is that while attributes
> > allow you to associate key-value pairs with a variable, there's no
> way
> > to nest another associative array within an attribute; the best you
> > can do is reference a dummy container variable and look at *its*
> > attributes.
> >
> > (Which is an approach that works well enough for the grid mapping
> > problem. It may be worth formalizing / regularizing it and applying
> it
> > to any attribute sets that are grouped together and have a tendency
> > toward being composed of long lists of elements...)
> >
> > Cheers,
> >
> > --Seth
> >
> > ----
> > Seth McGinnis
> > mcginnis at ucar.edu
> > NARCCAP Data Manager
> > IMAGe / ISP / NCAR
> > ----
> >
> > >On Oct 26, 2009, at 0710, <martin.juckes at stfc.ac.uk> wrote:
> > >
> > >> Hello,
> > >>
> > >> I wonder if anyone can help with this problem:
> > >>
> > >> I have a file with a map of index values, ranging from 1 to 827.
> The
> > flag
> > >meanings range from ?Admiralty Islands lowland rain forests? to
> > ?Zambezian
> > >halophytics?. I?m reluctant to combine all 827 flag meanings into a
> > single
> > >string for the ?flag_meanings? attribute ? is there a better way?
> > >>
> > >> Note that this is not my dataset, so that while suggestions for
> > presenting
> > >the data in a different way may be useful, they won?t solve my
> > current
> > >problem. The dataset is a eco-region analysis distributed by the
> WWF
> > and has
> > >wide usage in its present form.
> > >>
> > >> Sincerely,
> > >> Martin Juckes
> > >>
> > >> --
> > >> Scanned by iCritical.
> _______________________________________________
> CF-metadata mailing list
> CF-metadata at cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>
> --
> This message (and any attachments) is for the recipient only. NERC
> is subject to the Freedom of Information Act 2000 and the contents
> of this email and any reply you make may be disclosed by NERC unless
> it is exempt from release under the Act. Any material supplied to
> NERC may be stored in an electronic records management system.
--
This message (and any attachments) is for the recipient only. NERC
is subject to the Freedom of Information Act 2000 and the contents
of this email and any reply you make may be disclosed by NERC unless
it is exempt from release under the Act. Any material supplied to
NERC may be stored in an electronic records management system.
Received on Tue Oct 27 2009 - 06:34:36 GMT
