GDAL Coordinate System Barn Raising
Drastic improvement of coordinate systems is needed for the GDAL, PROJ, libgeotiff, PostGIS, and Spatialite open source toolchain. Three significant issues must be improved, and no single organization has shown the wherewithal to step forward and financially support an infrastructure effort this big on its own. The community needs to come together for a barn raising.
Our target funding goal has been reached on May 15th 2018, which will enable us to start work on this challenging task. See Who to learn who has sponsored, and keep scrolling to learn more about the work.
This barn raising effort is organized by Howard Butler, Paul Ramsey, Kristian Evers, and Even Rouault. All funding, work, invoicing, and questions about the topic will be coordinated by Even Rouault and Spatialys. Please email firstname.lastname@example.org and/or email@example.com for more information, sponsorhip pledges, and technical questions.
The dreaded ad hoc CSV databases in
GDAL_DATAare frustrating for users, pose challenges for developers, and impede interoperability of definitions.
GDAL and PROJ are missing OGC WKT2 support.
PROJ 5.0+ no longer requires datum transformation pivots through WGS84, which can introduce errors of up to 2m, but the rest of the tools do not take advantage of it.
The use of a SQLite-based database for EPSG and other definitions will allow the projects to add more capability (area-aware validation), transition the custom peculiar data structures of the projects to something more universally consumable, and promote definition interoperability between many coordinate system handling software tools.
OGC WKT2 fixes longstanding interoperability coordinate system definition discrepancies. WKT2 contains tools for describing time-dependent coordinate reference systems. PROJ 5+ is now capable of time-dependent transformations, but GDAL and other tools do not yet support them.
Several countries are updating their geodetic infrastructure to include
time-dependent coordinate systems. For example,
and the United
States are adapting
time-dependent coordinate systems in 2020 and 2022, respectively. The familiar
NAD83 and NAVD88 in North America being replaced by
NAPGD2022, and the industry WILL have to adapt to these challenges sooner
PROJ previously required datum transformation that pivoted through WGS84 via a 7-parameter transform. This pivot is a practical solution, but it can introduce error of about two meters, and many legacy datums cannot be defined in terms of WGS84. PROJ 5+ now provides the tools to support late-binding through its transformation pipeline framework, but GDAL and the rest of the tools cannot use it yet. Higher accuracy transformations avoid stepping through WGS84 and eliminates extra transformation steps with side-car data from a local geodetic authority.
The effort has received generous sponsorship from 20 organizations, and surpassed its goal of $125,000 to raise $144,000 as of 15 MAY 2018.
|Dynamic Graphics, Inc||$5,000|
|The Timoney Group||$500|
Migration of CSV support data to SQLite database
We plan to migrate to a SQLite database for
GDAL_DATA. You can
read more about a proposal to do so on the MetaCRS
August of 2015. This would include adding support for
EPSG and any overrides, historical
definitions, custom additions, or alternative dictionaries. The purpose is to
reduce the fragility of the CSV “database” that PROJ, GDAL, and libgeotiff use,
collate it into a single entity that can be used and shared by other projects
such as QGIS and PostGIS, allow
easy inclusion of overrides and custom definitions, and add more functionality
(such as bounding-box lookups for definitions).
OGC provided a standard iterating the “Well Known Text” format for coordinate systems in 2015. Its uptake throughout the industry has been slowed by lack of implementation support. We believe that support in the open source stack of software – GDAL, PROJ, PostGIS, libgeotiff, and others – will provide the kick that WKT2 needs to see wide(r) industry adoption.
WKT2 codifies the communication of coordinate system definitions and tightens those definitions, especially in areas such as axis, vertical control, and epochs. Communication of these more advanced coordinate system parameters are becoming more common, and the need for software to interoperate definitions continues to grow.
PROJ datum transformations have typically pivoted through the WGS 84 datum, but there are are some significant problems with the approach. First, not every transformation can be transitioned through WGS84. Second, the problem of what to do about different time realizations of WGS84 exists. Third a transition through WGS84 might be a lossy approximation. The recent release of PROJ 5.0.0, a completely volunteer effort that was led by Kristian Evers of SDFE, provides an opportunity fix this longstanding issue through the use of late-binding transformations that PROJ 5+ now supports.
Proposed GDAL Enhancements and Refactoring
GDAL currently has a simplified object modelling of SRS support with a single class, OGRSpatialReference, which internally maintains a tree structure reflecting in 1-1 relationship the hierarchical WKTv1 representation of the SRS.
The class has a number of helpers that do various things:
- set projection methods
- setup the geographical CRS
- import from / export to WKTv1
- import from / export to PROJ representation
- build a geographical/projected/geocentric/vertical/compound SRS from a EPSG CRS code
OGRSpatialReference is convenient to use but strongly tied to WKTv1, its keywords, its tree structure. To be able to support both WKTv1 and WKTv2, we will select a more object oriented approach with a number of C++ classes that will model the various CRS concepts.
For that design, we will take inspiration from the works done in the GeoAPI
org.opengis.georeferencing.\* class hierarchy), and its implementation by
the Apache SIS project, which themselves are strongly
inspired on the underlying ISO 19111 modeling (whose WKTv2 is an
The following classes will be likely introduced:
We will not completely adopt GeoAPI, because from our point of view, it has the classical default of Java based software of unnecessary multiplicating interfaces and classes, which makes the object model hard to understand and manipulate.
The implementation will live in PROJ itself, so this can be usable by projects that do not need the GDAL dependency. It will be more convenient to have the object modelling and the underlying computational capabilities traditionally offered by PROJ in closer synchronization.
Once that modelling will be in place, the following functionalities will be implemented:
instantiate the objects from a WKTv1 and WKTv2 representation
export to WKTv1 and WKTv2 (when possible, given the limitation of each format)
add a validation function of WKTv2 representation
port the current existing helpers of the OGRSpatialReference class to the new object system.
adapt the GDAL GeoPackage driver to properly deal with WKTv2
adapt all existing GDAL that have write capabilities to be able to deal with a potential input as WKTv2, and in which case use WKTv2 -> WKTv1 conversion methods
PROJ: SQLite-based version of the EPSG database
Use of a SQLite version of the EPSG database (converted from the PostgreSQL database dump originally provided by EPSG), stored in PROJ library, with additional tables to be able to provide overrides (the EPSG database may contain parameters not preferred by the community, or not directly usable by the GDAL / PROJ stack). It will contain proper indexes for fast queries. The database will be stored as a text dump for better traceability of changes in version control. The binary version of the database will be produced during the release procedure.
A procedure to convert the EPSG PostgreSQL dump to SQLite can be found at https://github.com/hobu/crs
PROJ will be enhanced to instantiate its CRS and Conversion classes from the content of the tables of the SQLITE database.
PROJ: Use area of use for source and target coordinate systems when building a coordinate transform
PROJ/GDAL: Offer the user (through API or gdaltransform utility) the choice to select an appropriate transformation method if the default choice does not fit its needs.
GDAL: Provide support for 4D coordinates in the OGRCoordinateTransform class, to be able to use corresponding PROJ 5 capabilities
GDAL: adding a
-ctswitch to gdaltransform, gdalwarp and ogr2ogr utilities (and their library versions) to specify a coordinate transformation toolchain, either using PROJ 5 pipeline syntax, or WKT2 coordinate operations
GDAL: Axis order issues. A common issue is vector formats reporting their geometries with coordinates in longitude, latitude order (“GIS friendly” order), while referencing a geodetic SRS whole axis order in the EPSG database is latitude, longitude. Similar situation for some projected coordinate systems with a Northing, Easting axis order in the EPSG database. The solution used by GDAL up to now was to expose the WKT definition of the SRS, with the AXIS part stripped off. This is middly satisfactory. We propose that drivers report the conformant WKT definition, but expose a
GetDataAxisMapping()method to map from the geometry axis order to the the SRS axis order. We would also add it for raster formats. Currently, GDAL supports
EPSGA:XXXXsyntaxes: the first one will eventually strip axis order definition for SRS with GIS “unfriendly” order, while the second one (rarely used to the best of our knowledge) preserves the exact EPSG representation. As part of this work, we will probably make
EPSG:XXXXbehaviour align on
EPSGA:XXXXbehaviour. This will have backward compatibility impacts, as for example
gdaltransform -s_srs EPSG:4326 -t_srs EPSG:32631will now expect input coordinates to be in latitude, longitude order. To limit them, we will introduce a
EPSG_GIS_AXIS:prefix that will resolve to a GIS friendly order SRS definition.
libgeotiff will be fully upgraded to use the EPSG database provided by the updated PROJ.
The process which creates the EPSG database for libgeotiff is fraught.
Currently, each time a new version of the EPSG database is released and adopted
by the “C” stack, it is first imported in libgeotiff in a PostgreSQL database,
from which a Python script derives CSV files that directly match a table of the
EPSG database (for example list of ellipsoids, prime meridians, datums), and a
few added value ones (reconstructed geographic and projected coordinate
systems, definition including
TOWGS84 clauses). This complexity will be
removed from libgeotiff and align its behavior with PROJ and GDAL in regard
to definitions and their creation workflow.
Should any extra funds be available, they will be applied toward the maintenance and improvement of GDAL, libgeotiff, PROJ, and related projects.