Addition of grid mapping for radar and lidar data in radial coordinates

[mikedixon]

Introduction

Radars and Lidars sample the atmosphere using a pulse of energy transmitted from an instrument along a line of sight away from the instrument, with a specified beam width (solid angle). The received signal is sampled over time, which allows for the estimation of the distance of the target from the instrument. The samples in range are frequently referred to as 'gates'.

The raw data is stored in radial (polar) coordinates. The (x,y) location of a gate may be determined using traditional great-circle techniques, since these are line-of-sight instruments and the propagation is assumed to be along a straight line in (x,y). In the vertical (z) dimension, a standard model of atmospheric refractive index is generally assumed for locating the height of the gate.

The organization and interpretation of radial data of this type is not supported by any of the standard grid mapping types currently accepted by CF.

Therefore, in order for the CfRadial? radar and lidar format to be formally recognized by the CF user community, we are proposing the addition of a grid mapping specifically for radar and lidar data.

Proposed grid mapping name

We propose the use of the name “radar_lidar_radial_scan”.

Grid mapping parameters

The following example from a CfRadial? file shows the proposed parameters:

grid_mapping:grid_mapping_name = "radar_lidar_radial_scan" ;
grid_mapping:longitude_of_projection_origin = -104.545806884766 ;
grid_mapping:latitude_of_projection_origin = 39.7866401672363 ;
grid_mapping:height_of_projection_origin = 1709. ;

The latitude and longitude are in degrees.

The height is in meters MSL.

The height_of_projection_origin parameter is analogous to the perspective_point_height in the vertical perspective projection.

The supporting documents are maintained at:

​https://github.com/NCAR/CfRadial/tree/master/docs

[mikedixon]

Radar Lidar Radial Scan Grid Mapping Proposal

[jonathan]

Dear Mike

Thanks for your proposal. It would probably encourage discussion if you could put the text in this ticket. That's more accessible than a linked PDF or GitHub, because the ticket is distributed to everyone by email in plain text. We may change to using GitHub instead of trac, but we're not yet technically set up for that. It's being worked on.

If I understand correctly, this proposal is about recording the conversion of polar coordinates (range and angle) to geolocated XYZ coordinates. Is that right? The conversion depends on measured and variable physical quantities, so it's not like other conversions, which depend on geometry. Therefore I'm not sure it's really like a grid mapping. However I understand that you do need to record the geolocating coordinates. Can they be stored as 2D auxiliary coordinates which are functions of range and angle? That's a general CF mechanism which doesn't depend on grid mapping.

Best wishes

Jonathan

[mikedixon]

Dear Jonathan

Thanks for your feedback.

I have pasted the text from the proposal document into this ticket as you suggest.

I don't really follow your assertion that the mapping of a radar return to a location in space is dependent on measured and variable physical quantities. Rather, the mapping does depend on geometries: the pointing angle of the antenna and the location of the antenna in space. The mapping also depends on an assumed standard model of atmospheric refraction, which is analagous to the 'great circle' construct in 3D projections.

As such, I believe this proposed grid mapping is similar to the 'vertical perspective' and 'geostationary' grid mappings already supported in CF. Both of these mappings depend on the measured position and orientation of the observing platform. The same is true of radar and lidar data.

In September I attended a workshop here in Boulder on NetCDF CF. At that workshop it was recommended that I propose a grid mapping for radars and lidars. Hence this proposal.

We already store the location of the radar, and the pointing angles, in the CfRadial? file. However, it was pointed out at the workshop that to be more consistent with CF we ought to document our grid mapping formally.

Kind regards

Mike

[jonathan]

Dear Mike

Thanks for pasting in the proposal. That's very helpful.

I don't really follow your assertion that the mapping of a radar return to a location in space is dependent on measured and variable physical quantities.

That was what I understood by your text, "The location in space, especially in the vertical, is dependent on the propagation of the energy through the atmosphere. Vertical gradients of the index of refraction lead to complexities in determining the height of the target." That sounds like some physics is needed to work out the xyz locations, not just geometry - I agree that geometry is needed too, of course. Did I misunderstand this?

To be consistent with CF it would be good to include the latitude, longitude and height coordinates in the file, as well as the polar coordinates. That makes the data more easily usable by generic applications. It's the reason why CF requires 2D latitude and longitude auxiliary coordinate variables for data variables which are on map projection planes. Auxiliary coordinates can be understood without a grid_mapping. The grid_mapping additionally provides the information to compute the relationship between the grid coordinates (the polar coordinates, in your case) and the geolocated (auxiliary) coordinates, for software which is able to do that. Perhaps that's your aim. Are your grid_mapping attributes specifying the location of the radar/lidar source? What else is needed to compute the geolocated coordinates from the polar coordinates?

Best wishes

Jonathan

[mikedixon]

Dear Jonathan

I apologize - you are correct, the initial wording was confusing, and suggested that we make use of atmospheric measurements when computing the location of a radar gate in the z coordinate. In fact this is almost never done, and a standard refraction model is used for almost all radars. Hence a standard grid mapping method is applied to radars in general to compute the location of the measurements in space.

I have modified the text to indicate this more clearly.

It is true that we could include the lat/lon/height of each gate in the data. The drawback of this is that it would significantly increase the size of the files - many radar fields are stored using just 1 or 2 bytes per gate, specifically to keep the size under control and to facilitate good compression. Lat/lon/height would need to use 4-byte floats at a minimum - i.e. and extra 12 bytes per gate. Often radar volumes are produced every 5 minutes or so, and for many radars across a country, so the data volume grows quickly. And often the data files must be transferred across slow communication lines in remote locations. So it is not likely that the radar/lidar community would support the extra size involved.

Radar users are generally familiar with the geometric calculations required to compute the gate locations, and these procedures are well documented in the literature.

Having a grid mapping specifically for this type of data would, I believe, help us to get CfRadial? adopted by the CF community. Organizations such as CEDA in the UK are requiring that the radar data submitted to the archive be CF compliant, and we are working hard to try to meet that requirement. Along with this proposal we are submitting 2 other proposals - #169 for complex number support, and a proposal for standard names that will include the quantities (such as azimuth and elevation angles) that describe the geometry referred to in this grid mapping proposal.

It would be very helpful to us, and our user community, to be able to get this approved, with any required changes that are needed.

Thank you very much for your thoughtful comments, and for your quick response.

Kind regards

Mike

[jonathan]

Dear Mike

OK. Thanks for explaining. In that case I agree that it makes sense to add it as a grid mapping. Please could you draft it in the same form as the other entries in Appendix F, and write out the additions to the table of attributes at the end of Appendix F? When we make a change to the convention, we have to agree exactly what text changes are needed, so that implementing them is purely editorial.

I guess you may have used the attribute names referring to "projection origin" because they already existed. However, I wonder whether these are really the clearest words you could use, since this isn't really a map projection. Am I right in thinking that these specify the location of the instrument? Also, "height" could be confusing, because that's a CF standard name which means height above the surface. If you want height above the geoid (almost the same as mean sea level) it would be better to use "altitude", which is the standard name for that quantity.

You say there's a standard algorithm for doing the conversion. That's good, and it would therefore help to say what it is and give a reference for it in the Appendix F entry. Are there any other possible algorithms which might be used? If so, you could have another attribute to identify how the conversion would be done.

For the auxiliary coordinates, is it the case that they would be time-dependent? If they were independent of time, they could be provided once, in a separate file - we could add a mechanism for that. But that won't work if they're different for every data file. In that case, I think we would need to change the start of section 5.6, if you can't require auxiliary coordinates, since it says, "When the coordinate variables for a horizontal grid are not longitude and latitude, it is required that the true latitude and longitude coordinates be supplied via the coordinates attribute." For example, you could add another sentence, such as "For radial (polar) coordinate variables for radar and lidar data, it is optional to provide latitude, longitude and vertical coordinates via the coordinates attribute, and recommended if practicable." I think it should be encouraged at least, since it makes the data more user-friendly.

Best wishes

Jonathan

[mikedixon]

Hi Jonathan

Thanks for the dialogue, and for agreeing to consider the new grid mapping.

I agree that projection_origin etc does not make much sense in this case.

I think instrument_location is probably the correct way to describe it. And yes, 'altitude' is appropriate too.

I'll take a look at the section on auxiliary coords as well.

I'm out on a project project at the moment, so it may take a while to get the doc written and submitted.

Kind regards

Mike