Boundary Layers - OurPlanscape/Planscape GitHub Wiki

Add a new boundary layer

Planscape allows the user to pick boundary layers for the map. These layers show, for example, the boundaries of watersheds, US Forests, counties, etc. Boundaries are typically provided as shapefiles. To add a new boundary layer, you must

  1. Collect some metadata from the shapefile
  2. Change the configuration in Planscape
  3. Upload the shapefile to the database.

Collect metadata from the shapefile or gdbtable

Let's assume the boundaries are all stored in a directory $BOUNDARY_DIR. On your local machine, run the "ogrinfo" command on the shapefile or gdbtable (this comes with the GDAL package). For example,

$ ogrinfo -so -al $BOUNDARY_DIR/boundary/watersheds_HUC10_CA/WBD_USGS_HUC10_CA.shp

returns

INFO: Open of `boundary/watersheds_HUC10_CA/WBD_USGS_HUC10_CA.shp'
using driver `ESRI Shapefile' successful.
Layer name: WBD_USGS_HUC10_CA
Metadata:
  DBF_DATE_LAST_UPDATE=2016-01-13
Geometry: Polygon
Feature Count: 1128
Extent: (-384564.490466, -695165.693038) - (566275.274751, 554159.509975)
Layer SRS WKT:
PROJCRS["NAD83(2011) / California Albers",
    BASEGEOGCRS["NAD83(2011)",
        DATUM["NAD83 (National Spatial Reference System 2011)",
            ELLIPSOID["GRS 1980",6378137,298.257222101,
            LENGTHUNIT["metre",1]]],
        PRIMEM["Greenwich",0,
            ANGLEUNIT["degree",0.0174532925199433]],
        ID["EPSG",6318]],
    CONVERSION["California Albers",
        METHOD["Albers Equal Area",
          ID["EPSG",9822]],
        PARAMETER["Latitude of false origin",0,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8821]],
        PARAMETER["Longitude of false origin",-120,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8822]],
        PARAMETER["Latitude of 1st standard parallel",34,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8823]],
        PARAMETER["Latitude of 2nd standard parallel",40.5,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8824]],
        PARAMETER["Easting at false origin",0,
            LENGTHUNIT["metre",1],
            ID["EPSG",8826]],
        PARAMETER["Northing at false origin",-4000000,
            LENGTHUNIT["metre",1],
            ID["EPSG",8827]]],
    CS[Cartesian,2],
        AXIS["easting (X)",east,
            ORDER[1],
            LENGTHUNIT["metre",1]],
        AXIS["northing (Y)",north,
            ORDER[2],
            LENGTHUNIT["metre",1]],
    USAGE[
        SCOPE["State-wide spatial data management."],
        AREA["United States (USA) - California."],
        BBOX[32.53,-124.45,42.01,-114.12]],
    ID["EPSG",6414]]
Data axis to CRS axis mapping: 1,2
OBJECTID: Integer64 (10.0)
TNMID: String (40.0)
MetaSource: String (40.0)
SourceData: String (100.0)
SourceOrig: String (130.0)
SourceFeat: String (40.0)
LoadDate: Date (10.0)
GNIS_ID: Integer64 (10.0)
AreaAcres : Real (19.11)
AreaSqKm: Real (19.11)
States: String (50.0)
HUC10: String (10.0)
Name : String (120.0)
HUType: String (254.0)
HUMod: String (30.0)
Shape_Leng: Real (19.11)
Shape_Area: Real (19.11)

There are three important fields in this output:

  1. The top-level ID["EPSG",6414] field. This indicates that the Spatial Reference Identifier (SRID) of the shapefile is EPSG:6414.
  2. The Name field, which is the name of each component in the boundary layer (here, the name of a watershed).
  3. The AreaAcres field.

Modify configuration

From this output, make a new entry in the boundary configuration file src/planscape/config/boundary.json in the Planscape codebase. This file is in JSON format. Each entry under "boundaries" describe metadata for the boundary. Here's an annotated example of an entry (note that comments are not permitted in JSON) for the

{
  # Unique identifier for the boundary layer
  "boundary_name": "huc10",

  # Name to display in the user interface
  "display_name": "HUC10",

  # Region associated with the boundary layer; should be one of
  #   none (i.e., the boundary extends over all regions)
  "region_name": "none" 

  # Path to the shapefile for loading, relative to $BOUNDARY_DIR
  "filepath": "boundary/watersheds_HUC10_CA/WBD_USGS_HUC10_CA.shp",

  # Spatial Reference System (SRS) of the shapefile
  "source_srs": 6414,

  # Type of geometry in the shapefile (usually MULTIPOLYGON)
  "geometry_type": "MULTIPOLYGON",

  # Mapping of Planscape fields to shapefile fields
  "shapefile_field_mapping": {
    "shape_name": "Name",
    "acres": "AreaAcres"
  }
}

The last entry maps names of fields in the Planscape database to names of fields in the shapefile. The full list of possible fields in the Planscape database is

    objectid: models.BigIntegerField = models.BigIntegerField(null=True)
    shape_name: models.CharField = models.CharField(max_length=120, null=True)
    states: models.CharField = models.CharField(max_length=50, null=True)
    acres: models.FloatField = models.FloatField(null=True)
    hectares: models.FloatField = models.FloatField(null=True)

Commit the changes to this file to GitHub.

Upload data

Now it's time to upload the data.

  1. IMPORTANT: Check that your .env file is pointed to the right PLANSCAPE_DATABASE_HOST. You don't want to upload to the wrong database!

  2. [Optional] Modify and use the tool src/planscape/tools/clip_shapefile.py to clip the boundaries to the state boundaries. This useful, for instance, in clipping the US Forest boundaries to California.

  3. From the "src/planscape" directory, run the command

python manage.py upload_boundary $BOUNDARY_DIR --boundary huc10 --force

Frontend

The boundary will now be available to the frontend. Most boundaries, though, take seconds or tens of seconds to request from the backend. Most of this is overhead from the conversion to GeoJSON. For instance, for the HUC-12 boundaries for the Sierra Nevada region, it can take the backend almost a minute to convert to GeoJSON, and almost a minute to send to the frontend (because it's 64MB).

To speed this up, the frontend has several pre-compiled boundaries loaded into the "assets" directory. Here's the process for creating a version:

  1. From the command line, use "wget" to fetch the GeoJSON boundary from the Planscape backend, optionally clipped to the region. For instance,

    wget "http://localhost:8000/planscape-backend/boundary/boundary_details/?boundary_name=USFS&region_name=sierra_cascade_inyo" \
   -O /tmp/usfs.geojson

    fetches the "USFS" boundary, clipped to the "sierra_cascade_inyo" region, and stores the GeoJSON in the file /tmp/usfs.geojson.

  2. Move the file into the BASE_DIR/src/interface/src/assets/geojson directory.

  3. Add an entry to the regionToGeojsonMap in the file BASE_DIR/src/interface/src/app/services/map.service.ts.

The interface should now load the boundary from the assets instead of the backend.

Reducing the size of geojson assets

Some boundaries are very large and contain hundreds of thousands of coordinates, slowing down performance when the user tries to pan or zoom the map. Any boundary with a file size above 10 MB will probably contribute to slower performance in the frontend. The precompiled boundaries in the assets folder have had their size reduced to 70% of original using two methods:

  1. Run the reduce_geojson.py script in the src/utils directory to trim decimals to 5 or fewer places. For example:
python3 src/utils/reduce_geojson.py src/interface/src/assets/geojson/sierra_cascade_inyo/huc12.geojson \
 src/interface/src/assets/geojson_reduced/sierra_cascade_inyo/huc12.geojson --decimal_places 5
  1. Use the Mapshaper tool to simplify the boundaries, reducing vertex count by 50% using Visvalingam / weighted area with "Prevent shape removal" checked. image

Staging site

Update the Box Folder with the boundaries you've loaded into the database.