home

GDAL Coordinate System Barn Raising

Drastic improvement of coordinate systems is needed for the GDAL, PROJ, libgeotiff, PostGIS, and Spatialite open source toolchain. Three significant issues must be improved, and no single organization has shown the wherewithal to step forward and financially support an infrastructure effort this big on its own. The community needs to come together for a barn raising.

Our target funding goal has been reached on May 15th 2018, which will enable us to start work on this challenging task. See Who to learn who has sponsored, and keep scrolling to learn more about the work.

Contact

This barn raising effort is organized by Howard Butler, Paul Ramsey, Kristian Evers, and Even Rouault. All funding, work, invoicing, and questions about the topic will be coordinated by Even Rouault and Spatialys. Please email even.rouault@spatialys.com and/or howard@hobu.co for more information, sponsorhip pledges, and technical questions.

Why

Coordinate systems in GDAL, PROJ, and libgeotiff are missing modern capabilities and need a thorough refactoring:

  • The dreaded ad hoc CSV databases in PROJ_LIB and GDAL_DATA are frustrating for users, pose challenges for developers, and impede interoperability of definitions.

  • GDAL and PROJ are missing OGC WKT2 support.

  • PROJ 5.0+ no longer requires datum transformation pivots through WGS84, which can introduce errors of up to 2m, but the rest of the tools do not take advantage of it.

CSV database

The use of a SQLite-based database for EPSG and other definitions will allow the projects to add more capability (area-aware validation), transition the custom peculiar data structures of the projects to something more universally consumable, and promote definition interoperability between many coordinate system handling software tools.

WKT2

OGC WKT2 fixes longstanding interoperability coordinate system definition discrepancies. WKT2 contains tools for describing time-dependent coordinate reference systems. PROJ 5+ is now capable of time-dependent transformations, but GDAL and other tools do not yet support them.

Several countries are updating their geodetic infrastructure to include time-dependent coordinate systems. For example, Australia and the United States are adapting time-dependent coordinate systems in 2020 and 2022, respectively. The familiar NAD83 and NAVD88 in North America being replaced by NATRF2022 and NAPGD2022, and the industry WILL have to adapt to these challenges sooner or later.

WGS84 Pivot

PROJ previously required datum transformation that pivoted through WGS84 via a 7-parameter transform. This pivot is a practical solution, but it can introduce error of about two meters, and many legacy datums cannot be defined in terms of WGS84. PROJ 5+ now provides the tools to support late-binding through its transformation pipeline framework, but GDAL and the rest of the tools cannot use it yet. Higher accuracy transformations avoid stepping through WGS84 and eliminates extra transformation steps with side-car data from a local geodetic authority.

Who

Sponsors

The effort has received generous sponsorship from 20 organizations, and surpassed its goal of $125,000 to raise $144,000 as of 15 MAY 2018.

Organization Amount
Anonymous $5,000
AGI $2,000
AppGeo $5,000
Applied Imagery $5,000
Azavea $5,000
Boundless $5,000
Carto $5,000
Dynamic Graphics, Inc $5,000
ESRI $30,000
GeoCue $5,000
Intevation $1,000
LizardTech $5,000
LINZ $7,000
Mapbox $5,000
Mobile Geographics $500
Oslandia $2,000
Safe Software $31,000
Septima $1,500
Sparkgeo $5,000
Spatial Networks $1,000
RIEGL $7,000
Radiant Solutions $5,000
Terranodo $500
The Timoney Group $500
Total $144,000

What

Technical details

Migration of CSV support data to SQLite database

We plan to migrate to a SQLite database for PROJ_LIB and GDAL_DATA. You can read more about a proposal to do so on the MetaCRS List from August of 2015. This would include adding support for EPSG and any overrides, historical definitions, custom additions, or alternative dictionaries. The purpose is to reduce the fragility of the CSV “database” that PROJ, GDAL, and libgeotiff use, collate it into a single entity that can be used and shared by other projects such as QGIS and PostGIS, allow easy inclusion of overrides and custom definitions, and add more functionality (such as bounding-box lookups for definitions).

WKT2 Support

OGC provided a standard iterating the “Well Known Text” format for coordinate systems in 2015. Its uptake throughout the industry has been slowed by lack of implementation support. We believe that support in the open source stack of software – GDAL, PROJ, PostGIS, libgeotiff, and others – will provide the kick that WKT2 needs to see wide(r) industry adoption.

WKT2 codifies the communication of coordinate system definitions and tightens those definitions, especially in areas such as axis, vertical control, and epochs. Communication of these more advanced coordinate system parameters are becoming more common, and the need for software to interoperate definitions continues to grow.

Datum Pivots

PROJ datum transformations have typically pivoted through the WGS 84 datum, but there are are some significant problems with the approach. First, not every transformation can be transitioned through WGS84. Second, the problem of what to do about different time realizations of WGS84 exists. Third a transition through WGS84 might be a lossy approximation. The recent release of PROJ 5.0.0, a completely volunteer effort that was led by Kristian Evers of SDFE, provides an opportunity fix this longstanding issue through the use of late-binding transformations that PROJ 5+ now supports.

Proposed GDAL Enhancements and Refactoring

GDAL currently has a simplified object modelling of SRS support with a single class, OGRSpatialReference, which internally maintains a tree structure reflecting in 1-1 relationship the hierarchical WKTv1 representation of the SRS.

The class has a number of helpers that do various things:

  • set projection methods
  • setup the geographical CRS
  • import from / export to WKTv1
  • import from / export to PROJ representation
  • build a geographical/projected/geocentric/vertical/compound SRS from a EPSG CRS code

OGRSpatialReference is convenient to use but strongly tied to WKTv1, its keywords, its tree structure. To be able to support both WKTv1 and WKTv2, we will select a more object oriented approach with a number of C++ classes that will model the various CRS concepts.

For that design, we will take inspiration from the works done in the GeoAPI OGC standard (org.opengis.georeferencing.\* class hierarchy), and its implementation by the Apache SIS project, which themselves are strongly inspired on the underlying ISO 19111 modeling (whose WKTv2 is an implementation).

The following classes will be likely introduced:

  • CoordinateReferenceSystem
    • GeographicalCRS
    • ProjectedCRS
    • CompoundCRS
    • VerticalCRS
    • TimeCRS
    • DerivedCRS
    • CoordinateSystem
    • Datum
    • Ellipsoid
    • Conversion

We will not completely adopt GeoAPI, because from our point of view, it has the classical default of Java based software of unnecessary multiplicating interfaces and classes, which makes the object model hard to understand and manipulate.

The implementation will live in PROJ itself, so this can be usable by projects that do not need the GDAL dependency. It will be more convenient to have the object modelling and the underlying computational capabilities traditionally offered by PROJ in closer synchronization.

Once that modelling will be in place, the following functionalities will be implemented:

  • instantiate the objects from a WKTv1 and WKTv2 representation

  • export to WKTv1 and WKTv2 (when possible, given the limitation of each format)

  • add a validation function of WKTv2 representation

  • port the current existing helpers of the OGRSpatialReference class to the new object system.

  • adapt the GDAL GeoPackage driver to properly deal with WKTv2

  • adapt all existing GDAL that have write capabilities to be able to deal with a potential input as WKTv2, and in which case use WKTv2 -> WKTv1 conversion methods

PROJ: SQLite-based version of the EPSG database

  • Use of a SQLite version of the EPSG database (converted from the PostgreSQL database dump originally provided by EPSG), stored in PROJ library, with additional tables to be able to provide overrides (the EPSG database may contain parameters not preferred by the community, or not directly usable by the GDAL / PROJ stack). It will contain proper indexes for fast queries. The database will be stored as a text dump for better traceability of changes in version control. The binary version of the database will be produced during the release procedure.

  • A procedure to convert the EPSG PostgreSQL dump to SQLite can be found at https://github.com/hobu/crs

  • PROJ will be enhanced to instantiate its CRS and Conversion classes from the content of the tables of the SQLITE database.

Other improvements

  • PROJ: Use area of use for source and target coordinate systems when building a coordinate transform

  • PROJ/GDAL: Offer the user (through API or gdaltransform utility) the choice to select an appropriate transformation method if the default choice does not fit its needs.

  • GDAL: Provide support for 4D coordinates in the OGRCoordinateTransform class, to be able to use corresponding PROJ 5 capabilities

  • GDAL: adding a -ct switch to gdaltransform, gdalwarp and ogr2ogr utilities (and their library versions) to specify a coordinate transformation toolchain, either using PROJ 5 pipeline syntax, or WKT2 coordinate operations

  • GDAL: Axis order issues. A common issue is vector formats reporting their geometries with coordinates in longitude, latitude order (“GIS friendly” order), while referencing a geodetic SRS whole axis order in the EPSG database is latitude, longitude. Similar situation for some projected coordinate systems with a Northing, Easting axis order in the EPSG database. The solution used by GDAL up to now was to expose the WKT definition of the SRS, with the AXIS part stripped off. This is middly satisfactory. We propose that drivers report the conformant WKT definition, but expose a GetDataAxisMapping() method to map from the geometry axis order to the the SRS axis order. We would also add it for raster formats. Currently, GDAL supports EPSG:XXXX and EPSGA:XXXX syntaxes: the first one will eventually strip axis order definition for SRS with GIS “unfriendly” order, while the second one (rarely used to the best of our knowledge) preserves the exact EPSG representation. As part of this work, we will probably make EPSG:XXXX behaviour align on EPSGA:XXXX behaviour. This will have backward compatibility impacts, as for example gdaltransform -s_srs EPSG:4326 -t_srs EPSG:32631 will now expect input coordinates to be in latitude, longitude order. To limit them, we will introduce a EPSG_GIS_AXIS:prefix that will resolve to a GIS friendly order SRS definition.

libgeotiff

libgeotiff will be fully upgraded to use the EPSG database provided by the updated PROJ.

The process which creates the EPSG database for libgeotiff is fraught. Currently, each time a new version of the EPSG database is released and adopted by the “C” stack, it is first imported in libgeotiff in a PostgreSQL database, from which a Python script derives CSV files that directly match a table of the EPSG database (for example list of ellipsoids, prime meridians, datums), and a few added value ones (reconstructed geographic and projected coordinate systems, definition including TOWGS84 clauses). This complexity will be removed from libgeotiff and align its behavior with PROJ and GDAL in regard to definitions and their creation workflow.

Extra funds

Should any extra funds be available, they will be applied toward the maintenance and improvement of GDAL, libgeotiff, PROJ, and related projects.