A Guide to Location Tracking and Visualization with OwnTracks, Node.js, PostgreSQL, GeoServer, MapProxy, Nginx and OpenLayers

Inspired by Aaron Parecki who has been tracking his location since 2008 with an iPhone app (Overland) and a server side tracking API (Compass), I decided to go for a similar approach.

I wanted to track my position constantly with my Android smartphone, save the data to a Postgres database and use the data to display an map with all locations i have ever been to.

To be able to track my position i needed an stable android app and after some tinkering around with different tracking apps I found OwnTracks which basically has all i need (and a good documentation of it).

In this article, we will explore a step-by-step guide on how to set up a location tracking system using OwnTracks for mobile devices, Node.js for a webhook server, PostgreSQL for data storage, and GeoServer, MapProxy, and Nginx for visualizing the location data.

But first of all we need a database to be able to actually save the data.

Set up PostgreSQL

To install PostgrSQL 16 you will need to the repository that provides the packages. First, update the package index and install required packages:

sudo apt update
sudo apt install gnupg2 wget vim

The PostgreSQL repository can then be added using the command:

sudo sh -c 'echo "deb https://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'

Import the repository signing key:

curl -fsSL https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/postgresql.gpg

And update your package list:

sudo apt update

Afterwards you can install the latest PostgreSQL (16) version with:

sudo apt-get -y install postgresql postgresql-contrib

Once installed, start and enable the service:

sudo systemctl start postgresql
sudo systemctl enable postgresql

Now you will need a user and database for your location data.

Connect to the PostgreSQL database with:

sudo su postgres
psql

and create a database with:

CREATE DATABASE locations;

aswell a user with:

CREATE USER USER <username> with encrypted password '<password>';

Then you can grant privileges for your user to the database:

GRANT ALL PRIVILEGES ON DATABASE locations TO <username>;

After you set up your database and user you also will need a table to store the data. The table where I save my locations can be recreated with:

CREATE TABLE public.locations (
	id bigserial NOT NULL,
	created_at timestamptz NULL DEFAULT CURRENT_TIMESTAMP,
	lat float8 NULL,
	lon float8 NULL,
	acc int4 NULL,
	alt int4 NULL,
	batt int4 NULL,
	bs int4 NULL,
	cog numeric(10, 2) NULL,
	rad int4 NULL,
	t varchar(255) NULL,
	tid varchar(255) NULL,
	tst int4 NULL,
	vac int4 NULL,
	vel int4 NULL,
	p numeric(10, 2) NULL,
	conn varchar(255) NULL,
	topic varchar(255) NULL,
	inregions jsonb NULL,
	ssid varchar(255) NULL,
	bssid varchar(255) NULL,
);

The most important attributes are lat (latitude), lon (longitude) and alt (altitude) which are used to render points later on.

Also I am saving the velocity, battery level of the smartphone and some more data which will be displayed at my /now page. The velocity and altitude are quite inaccurate though which probably also depends on the smartphone.

Create a Node.js Webhook server

You can create a node.js server to act as a webhook to receive location data from OwnTracks. With the pg npm package you can interact with the PostgreSQL database.

const http = require("http");
const axios = require('axios');
const { Pool} = require('pg')

const pool = new Pool({
  user: 'username',
  database: 'database',
  password: 'pw!',
  port: port,
  host: 'localhost',
})

async function insertData(body) {
  try {
    const res = await pool.query(
      "INSERT INTO locations (lat, lon, acc, alt, batt, bs, tst, vac, vel, conn, topic, inregions, ssid, bssid) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14)",
      [body.lat, body.lon, body.acc, body.alt, body.batt, body.bs, body.tst, body.vac, body.vel, body.conn, body.topic, body.inregions, body.ssid, body.bssid]
    );
    console.log(`Added location ${body.lat}, ${body.lon}`);
  } catch (error) {
    console.error(error)
  }
}

const server = http.createServer((request, response) => {
  let body = [];
  console.log(request.method)
  if (request.method === "POST") {
     request.on('data', (chunk) => {
      body.push(chunk);
      }).on('end', () => {
        body = JSON.parse(Buffer.concat(body).toString());
        insertData(body)
      })
  }

  response.end();
});

server.listen(9001);

So basically the webhook parses the request body and inserts the data via an insert statement into the postgres table. Also the server is listening on port 9001 so be sure you opened that port on your sever. The Url for Owntracks now could be http://yourserverip:9001.

Setting up OwnTracks

The OwnTracks app runs in the background on your Android or iOS device and waits for the smart phone to tell it that the device has moved, whereupon OwnTracks sends out a message with its current coordinates (and a few other things we'll discuss in a moment)

With different settings for the app I could tailor the app exactly to my needs because I wanted the app to check my position every second and when i have moved more than two meters from my last known position it should send a POST-Request to my Strapi instance which will store the location data in a Postgres database.

In the following screenshot you can see the settings I am using for OwnTracks.

The most important seetings for me are the mode

• Significant location change mode,
• locatorDisplacement,
• locatorInterval,
• ignoreInaccurateLocations and
• url for the POST-request.

The significant location change mode is a mode which will let you decide in which interval you want to track your position. I am using the locationInterval '1' which will track my position every second with an LocatorDisplacement of 1 which will check if i moved away a meter away from my last known position. If i did it will send an POST-request with the location data to the given url if the position data isn't more inaccurate than 10 meters.

Because the app has to run always in the background you probably have to switch off energy saving modes for your smartphone which might disable the background functionality.

Be warned: You probably will have to charge your smartphone more often. Also sometimes the app has some tracking error if you are using parallel some other tracking app for example for activities like strava.

The url http://yourserverip:9001 for the POST-request points to the nodejs server webhook which will run a insert command for the postgres database.

Now the data will be stored in the postgres database but it still needs to be visualized somehow and therefore it needs a geometry attribute.

Creating a PostGIS extension

To be able to create a geometry you will need the PostGIS extension. You can extend your PostgreSQL database with

sudo apt-get install postgis
sudo su postgres
psql
\c locations
CREATE EXTENSION postgis;
\q

Then create a view which creates a geometry based on the lat and lon attribute from the existing table.

CREATE OR REPLACE VIEW public.locations_geom
AS SELECT id,
    lat,
    lon,
    alt,
    vel,
    st_setsrid(st_makepoint(lon, lat, alt::double precision), 4326) AS geom
   FROM locations;

At the moment I wrote/edited this post I have about two million datasets in the table, so fetching and rendering the data client-side would not be the best idea. This is where Geoserver takes over.

Install and configure Geoserver

GeoServer is an open source server for sharing geospatial data.

To setup GeoServer I created a stack in portainer:

version: '3'

networks:
  default:
    external: true
    name: nginx
    
services:
  geoserver:
    image: kartoza/geoserver:latest
    container_name: geoserver
    restart: always
    volumes:
      - /data/containers/geoserver:/opt/geoserver/data_dir
    ports:
      - "8080:8080"
    environment:
      GEOSERVER_DATA_DIR: ${GEOSERVER_DATA_DIR}
      GEOSERVER_ADMIN_PASSWORD: ${GEOSERVER_ADMIN_PASSWORD}
      GEOSERVER_ADMIN_USER: ${GEOSERVER_ADMIN_USER}
      INITIAL_MEMORY: ${INITIAL_MEMORY}
      MAXIMUM_MEMORY: ${MAXIMUM_MEMORY}
      DB_BACKEND: POSTGRES               
      HOST: ${POSTGRES_HOST}                         
      POSTGRES_PORT: 5432                
      POSTGRES_DB: gwc                
      POSTGRES_USER: ${POSTGRES_USER}    
      POSTGRES_PASS: ${POSTGRES_PASS}               
      POSTGRES_SCHEMA: public           
      DISK_QUOTA_SIZE: 20

After deploying GeoServer you can login with the default user "admin" and password "geoserver" or the credentials you assigned with environment variables.

To add a new datasource you have to click on Stores and Add new store. Then you can select PostGIS (or whatever datasource you want to use).

Fill out host, port, database, schema, user and passwd and save your connection. Afterwards you can select layers which you want to publish.

After you published your layer you can preview the layer at Layer Preview.

Now I am not using Geoserver as endpoint to serve the tiles because I have a Mapproxy instance running which will take care of transforming tiles from internal or external ressources.

Mapproxy

MapProxy is an open source proxy for geospatial data. It caches, accelerates and transforms data from existing map services and serves any desktop or web GIS client. I have decided to go with the docker image kartoza/mapproxy because so far it's the only docker image of mapproxy which supports MULTI_MAPPROXY and I want to serve several services from MapProxy.

You can setup a portainer stack of mapproxy with:

version: '3.9'

networks:
  default:
    external: true
    name: nginx
    
services:
  mapproxy:
    image: kartoza/mapproxy
    container_name: mapproxy
    restart: always
    environment:
      PRODUCTION: true
      PROCESSES: 6
      CHEAPER: 2
      THREADS: 8
      MULTI_MAPPROXY: true
      MULTI_MAPPROXY_DATA_DIR: ${MULTI_MAPPROXY_DATA_DIR}
      MAPPROXY_CACHE_DIR: ${MAPPROXY_CACHE_DIR}
      ALLOW_LISTING: true
    volumes:
      - /data/containers/mapproxy/data:/multi_mapproxy:ro
      - /data/containers/mapproxy/cache_data:/cache_data

The only thing MapProxy needs one or multiple configuration files in yaml format for each service. So set up a locations.yaml under /data/containers/mapproxy/data/locations.yaml:

services:
  demo:
  tms:
    use_grid_names: true
    # origin for /tiles service
    origin: 'nw'
  kml:
      use_grid_names: true
  wmts:
  wms:
    md:
      title: Location Data
      abstract: For mxd.codes.

layers:
  - name: locations
    title: Locations - mxd.codes
    sources: [locations_cache]

caches:
  locations_cache:
    grids: [webmercator]
    sources: [locations_wms]

sources:
  locations_wms:
    type: wms
    req:
      url: http://geoserver:8080/geoserver/mxdcodes/wms?
      layers: locations
      transparent: true

grids:
    webmercator:
        base: GLOBAL_WEBMERCATOR

globals:
  # cache options
  cache:
    # where to store the cached images
    base_dir: '/cache_data'
    # where to store lockfiles for concurrent_requests
    lock_dir: '/cache_data/locks'
    # where to store lockfiles for tile creation
    tile_lock_dir: '/cache_data/tile_locks'
    # request x*y tiles in one step
    meta_size: [4, 4]
  # image/transformation options
  image:
    # use best resampling for vector data
    resampling_method: bilinear # nearest/bicubic/bilinear
    formats:
      image/jpeg:
        encoding_options:
          # jpeg quality [0-100]
          jpeg_quality: 80

Be sure to replace the sources url with the url from your GeoServer instance.

Now you should have a endpoint for your wms tiles under http://mapproxyurl/locations/service? which you can use to create web map on your website.

I also created each service under a seperate subdomain and setup https for it with Cloudflare but I won't go into detail here because you will find hundreds of tutorials online.

Creating a Web Map with Leaflet and React

The end product of all this should be a simple map on my personal website under /map. The map component looks like the following:

import React, { useState, useRef, useEffect } from "react"
import "ol/ol.css"
import Map from "ol/Map"
import TileLayer from "ol/layer/Tile"
import View from "ol/View"
import XYZ from "ol/source/XYZ"
import { transform, get as getProjection } from "ol/proj"
import WMTSTileGrid from "ol/tilegrid/WMTS.js"
import WMTS from "ol/source/WMTS.js"
import TileWMS from "ol/source/TileWMS.js"
import OSM from "ol/source/OSM.js"
import { getTopLeft, getWidth } from "ol/extent.js"

function LiveMap({data}) {
  const [map, setMap] = useState()
  const mapElement = useRef()
  const mapRef = useRef()
  mapRef.current = map

  const projection = getProjection("EPSG:3857")
  const projectionExtent = projection.getExtent()
  const size = getWidth(projectionExtent) / 256
  const resolutions = new Array(19)
  const matrixIds = new Array(19)
  for (let z = 0; z < 19; ++z) {
    // generate resolutions and matrixIds arrays for this WMTS
    resolutions[z] = size / Math.pow(2, z)
    matrixIds[z] = z
  }

  const locData = new TileLayer({
    opacity: 0.9,
    preload: Infinity,
    source: new WMTS({
      attributions: '&copy; <a href="https://mxd.codes/">Max Dietrich</a>',
      url: "tilelayerurl",
      layer: "locations",
      matrixSet: "webmercator",
      format: "image/png",
      projection: projection,
      tileGrid: new WMTSTileGrid({
        origin: getTopLeft(projectionExtent),
        resolutions: resolutions,
        matrixIds: matrixIds,
      }),
      style: "default",
      wrapX: true,
    }),
  })

  const aerial = new TileLayer({
    opacity: 0.9,
    preload: Infinity,
    source: new WMTS({
      attributions:
        "Tiles &copy; Esri &mdash; Source: Esri, i-cubed, USDA, USGS, AEX, GeoEye, Getmapping, Aerogrid, IGN, IGP, UPR-EGP, and the GIS User Community",
      url: "tilelayerurl",
      layer: "arcgisaerial",
      matrixSet: "webmercator",
      format: "image/png",
      projection: projection,
      tileGrid: new WMTSTileGrid({
        origin: getTopLeft(projectionExtent),
        resolutions: resolutions,
        matrixIds: matrixIds,
      }),
      style: "default",
      wrapX: true,
    }),
  })

  const center = transform([data.lon, data.lat], "EPSG:4326", "EPSG:3857")

  useEffect(() => {
    const initialMap = new Map({
      target: mapElement.current,
      layers: [aerial, locData],
      view: new View({
        center: center,
        zoom: 16,
        maxZoom: 16,
      }),
    })
    setMap(initialMap)
  }, [])

  return (
    <div
      style={{ height: "100%", width: "100%" }}
      ref={mapElement}
      className="map-container"
    />
  )
}

export default LiveMap