Charles,
That sounds great to me. I appreciate your patience with me.
Grace and peace,
Jim
Jim Biard
Research Scholar
Cooperative Institute for Climate and Satellites
Remote Sensing and Applications Division
National Climatic Data Center
151 Patton Ave, Asheville, NC 28801-5001
jim.biard at noaa.gov
828-271-4900
Follow us on Facebook!
On May 21, 2013, at 1:55 PM, Charles Paxson <cpaxson at aer.com> wrote:
> Jim,
>
> Thank you for your remarks. If one person has questions, it's likely that others do too, so I welcome your thoughts as we flush these issues out.
>
> Firstly, we have no disagreement about the usefulness of the snow threshold in creating a snow mask in the way things are set up. Toward the bottom of my previous response I said, "...and it makes sense to carry threshold along in the case of snow because of the conventional calculation" technique".
> The GOES-R platform will produce a cloud, dust, smoke and aerosol mask based on a series of binary tests, where any one test that is true will produce a mask value of one. So, for our purposes, a threshold value makes no sense since we do not compute a fraction of cloud, dust, smoke or aerosol. However, I propose we simply carry the threshold, and we will set the threshold to zero, indicating that the slightest fraction of cloud, dust, smoke or aerosol created a mask value of one. This plan will work for us.
> Lastly, this point is slightly an aside but related, GOES-R produces mask status flags that are tied to the CF standard. For example, roughly speaking, the cloud mask product carries a meta data data quality flag (DQF) that is a 16 bit integer, and 14 bits are used to indicate which test passed of the 14 tests that were performed in creating the cloud mask. An inspection of this DQF gives the reason for the mask value, not the threshold.
>
> Regards,
>
> Charles
>
>
> Jim Biard wrote:
>> Charles,
>>
>> I disagree with your assertion that knowledge of the threshold for a binary mask doesn't help an analyst with understanding the data. In the case of the surface_snow_binary_mask, it is most definitely useful to know that a threshold of 20% was used versus a threshold of 60%. If you are trying to use the mask as an input to an algorithm that requires >50% snow cover for an accurate result, it would be quite useful to know whether or not the mask would meet your requirements. I am fully convinced that your algorithms are quite nuanced and complex. But will your cloud mask report 'true' if there is >20% cloud cover? >10%? >5%? >2%? >70%? How clear is clear? The same goes for the other masks. In the end, without a threshold, all your cloud mask will tell me is "Well, it was clear enough for our purposes at the time."
>>
>> I understand that things like this can feel quite nit-picky, but if your data is being archived and someone wants to use it 30 years from now when everyone currently working with the data has retired, information such as this can be invaluable. I don't know the details of your algorithms, and I hate coming across as a late-game, sideline critic. I'm not trying to denigrate your products. I just want us to be thorough and clear.
>>
>> Grace and peace,
>>
>> Jim
>>
>> Jim Biard
>> Research Scholar
>> Cooperative Institute for Climate and Satellites <http://www.cicsnc.org/>
>> Remote Sensing and Applications Division
>> National Climatic Data Center <http://www.ncdc.noaa.gov/>
>> 151 Patton Ave, Asheville, NC 28801-5001
>>
>> jim.biard at noaa.gov <mailto:jim.biard at noaa.gov>
>> 828-271-4900
>>
>>
>>
>> Follow us on Facebook <https://www.facebook.com/cicsnc>!
>>
>> On May 20, 2013, at 8:03 PM, Charles Paxson <cpaxson at aer.com <mailto:cpaxson at aer.com>> wrote:
>>
>>> Dear Jonathan,
>>>
>>> Thank you for your clear response and focused question on the comparability of our proposed standard names across different different data sets. The masks that we propose: cloud, dust, smoke and aerosol are intended to be used across different data sets. I contend that the masks are intended to be the same quantity, and the following is for your consideration.
>>>
>>> The detection of cloud, snow, dust, smoke, and the like derive from remote sensing of light based on algorithm techniques from sensed multi-band signals, which differ over time and over sensor platforms. Therefore, we obviate the need for a standard name if we attempt to create a name for each platform and generation of mask products. I would argue that it is valid to compare masks derived from unique bands and algorithms just as it is valid to compare radiance measurements across platforms and generations. A radiance value may be derived from a bolometer or a cryogenically cooled chip. The difference in the radiance value is not the quantity measured, noting that the derivation of the radiance is very different, but the main difference is the bias and precision. Analogously, a mask may be derived from a simple threshold or derived from consideration of many inputs. Presumably, the more accurate mask is the one derived from the greater number of independent scientific facts. In the examples of mask and r
adiance, it is important for the analyst to be aware of the origin of the quantity.
>>>
>>> The snow_binary_mask is a particular CF Metadata example that has created a trend, utilizing a threshold. Currently only three masks are defined in the standard, and following the snow mask example will lead to the necessity of establishing unique mask names for each technique. For the case of snow, GOES-R considers spectral signatures of rock, ice, land, fine and course grain snow and more, and performs numerical spectral fitting to produce a fraction of snow cover result. The snow mask would be derived from the CF standard name ?surface_snow_area_fraction? in combination with the threshold. The threshold only serves to finalize the snow mask depending on the setting of the threshold parameter, but in general there was a great deal of technology that went into estimating a snow fraction in the first place, and the variety of ways to estimate the snow fraction must be great. So I don't see how the threshold is important in creating the standard name, and it makes sense to carry threshold along in the cas
e of snow because of the conventional calculation technique, finding a snow fraction then converting to a binary mask. As discussed already, the threshold doesn't help the analyst with understanding cloud and the aerosol masks.
>>>
>>> In summary, the snow, cloud, dust, smoke, aerosol and for that matter land binary masks are all estimated by complex algorithms, and an analyst would use the mask that meets their resolution, availability or accuracy needs, but the manner in which the products were derived would be of interest mostly in certifying the quality of the mask. Given these considerations, I think the mask names could be useful to the CF metadata standard name database.
>>>
>>> Sincerely,
>>>
>>> Charles Paxson, AER.
>>>
>>>
>>> Jonathan Gregory wrote:
>>>> Dear Charles
>>>>
>>>> Thank you for these proposals. Following Jim's question and your answer, I
>>>> appreciate that you can't define a threshold value in a particular quantity
>>>> for your application of these masks. Perhaps another question to ask is whether
>>>> quantities with these standard names are supposed to be comparable across
>>>> different datasets. That is the main purpose of standard names, in fact. For
>>>> instance, cloud_binary_mask is quite a general-purpose-sounding name. You can
>>>> imagine that, if this were in the table with the definition you give it:
>>>>
>>>>
>>>>> cloud_binary_mask: X_binary_mask has 1 where condition X is met, 0
>>>>> elsewhere. 1 = cloud present, 0 = cloud absent (clear)
>>>>>
>>>>
>>>> it might be used to label quantities with many different definitions of cloud
>>>> cloud presence/absence. The use of a common standard name would indicate these
>>>> quantities are the same, can be compared, and scientific conclusions can be
>>>> drawn from the comparison. But that might misleading. Is this acceptable?
>>>>
>>>> If your intention is to define cloud presence/absence in a specific way, then
>>>> I feel it would be better to have a more specific standard name that identifies
>>>> the algorithm, on the analogy for instance of the existing standard name of
>>>> isccp_cloud_area_fraction. I think it is fine to have application-specific
>>>> standard names like that then they identify quantities which might be generated
>>>> by different providers and which should be labelled as comparable.
>>>>
>>>> Best wishes
>>>>
>>>> Jonathan
>>>>
>>>> ----- Forwarded message from Charles Paxson <cpaxson at aer.com <mailto:cpaxson at aer.com>> -----
>>>>
>>>>
>>>>> Date: Wed, 15 May 2013 15:34:02 -0400
>>>>> From: Charles Paxson <cpaxson at aer.com <mailto:cpaxson at aer.com>>
>>>>> To: CF-metadata at cgd.ucar.edu <mailto:CF-metadata at cgd.ucar.edu>
>>>>> Subject: [CF-metadata] GOES-R generated binary mask products under proposal
>>>>> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.28)
>>>>> Gecko/20120306 Thunderbird/3.1.20
>>>>>
>>>>> Dear CF Metadata Users Group,
>>>>>
>>>>> Through observations and analysis, GOES-R weather products produce
>>>>> binary masks for: aerosols, smoke, dust and clouds. No coordinate
>>>>> value is warranted (i.e. CF Metadata standard
>>>>> surface_snow_binary_mask) for any of these four proposed quantities,
>>>>> since a complex set of tests for each case, such as using brightness
>>>>> temperatures, are used to derive signatures of the four atmospheric
>>>>> constituents. The proposed masks are analogous and defined in a
>>>>> similar way as the CF Metadata standard names: land_binary_mask and
>>>>> sunlit_binary_mask.
>>>>>
>>>>> aerosol_binary_mask: X_binary_mask has 1 where condition X is met, 0
>>>>> elsewhere. 1 = aerosols present, 0 = aerosolsabsent
>>>>> smoke_binary_mask: X_binary_mask has 1 where condition X is met, 0
>>>>> elsewhere. 1 = smoke present, 0 = smoke absent
>>>>> dust_binary_mask: X_binary_mask has 1 where condition X is met, 0
>>>>> elsewhere. 1 = dust present, 0 = dust absent
>>>>> cloud_binary_mask: X_binary_mask has 1 where condition X is met, 0
>>>>> elsewhere. 1 = cloud present, 0 = cloud absent (clear)
>>>>>
>>>>> Please accept these four mask names for inclusion into the CF
>>>>> Metadata standard.
>>>>>
>>>>> Sincerely,
>>>>>
>>>>> Charles Paxson
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>> _______________________________________________
>>>>> CF-metadata mailing list
>>>>> CF-metadata at cgd.ucar.edu <mailto:CF-metadata at cgd.ucar.edu>
>>>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>>>>>
>>>>
>>>>
>>>> ----- End forwarded message -----
>>>> _______________________________________________
>>>> CF-metadata mailing list
>>>> CF-metadata at cgd.ucar.edu <mailto:CF-metadata at cgd.ucar.edu>
>>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>>>>
>>>
>>>
>>> --
>>> Charles Paxson
>>> System Engineer
>>> Atmospheric and Environmental Research (AER), Inc.
>>> a Verisk Analytics Company
>>> 131 Hartwell Avenue, Lexington, MA 02421-3126
>>> office: 781-761-2209
>>> fax:781-761-2299
>>> www.aer.com <http://www.aer.com>
>>>
>>> _______________________________________________
>>> CF-metadata mailing list
>>> CF-metadata at cgd.ucar.edu
>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>>
>
>
> --
> Charles Paxson
> System Engineer
> Atmospheric and Environmental Research (AER), Inc.
> a Verisk Analytics Company
> 131 Hartwell Avenue, Lexington, MA 02421-3126
> office: 781-761-2209
> fax:781-761-2299
> www.aer.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20130521/02bac4eb/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: CicsLogoTiny.png
Type: image/png
Size: 15784 bytes
Desc: not available
URL: <
http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20130521/02bac4eb/attachment-0001.png>
Received on Tue May 21 2013 - 12:25:12 BST
