geospatial-stack

Overview

A reusable platform that takes a dataset from an uploaded file to a published, styled layer on a public map without an engineer in the loop, while keeping track of where every dataset came from and who is allowed to see it.

The problem

Every mapping project I worked on rebuilt the same plumbing: ingest files, check them, reproject, tile, publish, and remember where the data came from. Land and community boundary data adds a harder requirement: some layers must never be published at full precision, or at all.

Constraints

• Uploads arrive in every format (Shapefile, GeoPackage, GeoJSON, KML, GeoParquet) and in any CRS, often with invalid geometry.
• Non-engineers must be able to publish and restyle layers safely.
• Must run on a single VPS, with storage portable between MinIO and Cloudflare R2.

Key decisions

1. 01

   Raw, curated and published tiers

   Data moves through three tiers, enforced both as PostGIS schemas and as storage prefixes. Raw uploads are immutable GeoParquet; DuckDB validates, repairs and normalises them into the curated tier; only the published tier is visible to the map and the tile server.

2. 02

   Validation as a promotion gate

   A background job checks geometry validity, CRS, row counts and schema before a dataset can be published, and reports exactly what is wrong instead of a bare failure. Storage and interchange use EPSG:4326, tiles EPSG:3857, and areas are computed in an equal-area projection, never in degrees.

3. 03

   Governance in the data model

   Every dataset is a registry row with source, licence, attribution and a sensitivity level (public, restricted, withheld) that defaults to the most protective option. Changes are written to an audit trail, and there is no public sign-up: admin accounts are created from the command line.

4. 04

   Two ways to serve tiles

   Stable layers are pre-built into PMTiles in object storage; layers that change are served live from PostGIS by Martin. Styling is stored separately from the tiles, so an admin can restyle and publish a layer without re-tiling or redeploying.

Outcome

• Upload to published map layer without engineering involvement, with every step visible in the admin UI.
• Curated maps with their own public URLs, a self-hosted OSM basemap and no third-party API keys.
• Prometheus, Grafana and Loki for metrics and logs, plus tested backup and restore procedures.