Inspired by Aaron Parecki who has been tracking his location since 2008 with an iPhone app (Overland) and a server side tracking API (Compass), I decided to go for a similar approach.
I wanted to track my position constantly with my Android smartphone, save the data to a Postgres database and use the data to display an map with all locations i have ever been to.
To be able to track my position i needed an stable android app and after some tinkering around with different tracking apps I found OwnTracks which basically has all i need (and a good documentation of it).
In this article, we will explore a step-by-step guide on how to set up a location tracking system using OwnTracks for mobile devices, Node.js for a webhook server, PostgreSQL for data storage, and GeoServer, MapProxy, and Nginx for visualizing the location data.
But first of all we need a database to be able to actually save the data.
Set up PostgreSQL
To install PostgrSQL 16 you will need to the repository that provides the packages. First, update the package index and install required packages:
bashCopy codesudo apt update sudo apt install gnupg2 wget vim
The PostgreSQL repository can then be added using the command:
bashCopy codesudo sh -c 'echo "deb https://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
Import the repository signing key:
bashCopy codecurl -fsSL https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/postgresql.gpg
And update your package list:
bashCopy codesudo apt update
Afterwards you can install the latest PostgreSQL (16) version with:
bashCopy codesudo apt-get -y install postgresql postgresql-contrib
Once installed, start and enable the service:
bashCopy codesudo systemctl start postgresql sudo systemctl enable postgresql
Now you will need a user and database for your location data.
Connect to the PostgreSQL database with:
bashCopy codesudo su postgres psql
and create a database with:
sqlCopy codeCREATE DATABASE locations;
aswell a user with:
sqlCopy codeCREATE USER USER <username> with encrypted password '<password>';
Then you can grant privileges for your user to the database:
bashCopy codeGRANT ALL PRIVILEGES ON DATABASE locations TO <username>;
After you set up your database and user you also will need a table to store the data. The table where I save my locations can be recreated with:
sqlCopy codeCREATE TABLE public.locations ( id bigserial NOT NULL, created_at timestamptz NULL DEFAULT CURRENT_TIMESTAMP, lat float8 NULL, lon float8 NULL, acc int4 NULL, alt int4 NULL, batt int4 NULL, bs int4 NULL, cog numeric(10, 2) NULL, rad int4 NULL, t varchar(255) NULL, tid varchar(255) NULL, tst int4 NULL, vac int4 NULL, vel int4 NULL, p numeric(10, 2) NULL, conn varchar(255) NULL, topic varchar(255) NULL, inregions jsonb NULL, ssid varchar(255) NULL, bssid varchar(255) NULL, );
The most important attributes are lat (latitude), lon (longitude) and alt (altitude) which are used to render points later on.
Also I am saving the velocity, battery level of the smartphone and some more data which will be displayed at my /now page. The velocity and altitude are quite inaccurate though which probably also depends on the smartphone.
Create a Node.js Webhook server
You can create a node.js server to act as a webhook to receive location data from OwnTracks. With the pg npm package you can interact with the PostgreSQL database.
javascriptCopy codeconst http = require("http"); const axios = require('axios'); const { Pool} = require('pg') const pool = new Pool({ user: 'username', database: 'database', password: 'pw!', port: port, host: 'localhost', }) async function insertData(body) { try { const res = await pool.query( "INSERT INTO locations (lat, lon, acc, alt, batt, bs, tst, vac, vel, conn, topic, inregions, ssid, bssid) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14)", [body.lat, body.lon, body.acc, body.alt, body.batt, body.bs, body.tst, body.vac, body.vel, body.conn, body.topic, body.inregions, body.ssid, body.bssid] ); console.log(`Added location ${body.lat}, ${body.lon}`); } catch (error) { console.error(error) } } const server = http.createServer((request, response) => { let body = []; console.log(request.method) if (request.method === "POST") { request.on('data', (chunk) => { body.push(chunk); }).on('end', () => { body = JSON.parse(Buffer.concat(body).toString()); insertData(body) }) } response.end(); }); server.listen(9001);
So basically the webhook parses the request body and inserts the data via an insert statement into the postgres table. Also the server is listening on port 9001 so be sure you opened that port on your sever. The Url for Owntracks now could be http://yourserverip:9001.
Setting up OwnTracks
The OwnTracks app runs in the background on your Android or iOS device and waits for the smart phone to tell it that the device has moved, whereupon OwnTracks sends out a message with its current coordinates (and a few other things we'll discuss in a moment)
With different settings for the app I could tailor the app exactly to my needs because I wanted the app to check my position every second and when i have moved more than two meters from my last known position it should send a POST-Request to my Strapi instance which will store the location data in a Postgres database.
In the following screenshot you can see the settings I am using for OwnTracks.
Sorry, somehow the image couldn't be found :(
The most important seetings for me are the mode
- Significant location change mode,
- locatorDisplacement,
- locatorInterval,
- ignoreInaccurateLocations and
- url for the POST-request.
The significant location change mode is a mode which will let you decide in which interval you want to track your position. I am using the locationInterval '1' which will track my position every second with an LocatorDisplacement of 1 which will check if i moved away a meter away from my last known position. If i did it will send an POST-request with the location data to the given url if the position data isn't more inaccurate than 10 meters.
Because the app has to run always in the background you probably have to switch off energy saving modes for your smartphone which might disable the background functionality.
Be warned: You probably will have to charge your smartphone more often. Also sometimes the app has some tracking error if you are using parallel some other tracking app for example for activities like strava.
The url http://yourserverip:9001 for the POST-request points to the nodejs server webhook which will run a insert command for the postgres database.
Now the data will be stored in the postgres database but it still needs to be visualized somehow and therefore it needs a geometry attribute.
Creating a PostGIS extension
To be able to create a geometry you will need the PostGIS extension. You can extend your PostgreSQL database with
bashCopy codesudo apt-get install postgis sudo su postgres psql \c locations CREATE EXTENSION postgis; \q
Then create a view which creates a geometry based on the lat and lon attribute from the existing table.
sqlCopy codeCREATE OR REPLACE VIEW public.locations_geom AS SELECT id, lat, lon, alt, vel, st_setsrid(st_makepoint(lon, lat, alt::double precision), 4326) AS geom FROM locations;
At the moment I wrote/edited this post I have about two million datasets in the table, so fetching and rendering the data client-side would not be the best idea. This is where Geoserver takes over.
Install and configure Geoserver
GeoServer is an open source server for sharing geospatial data.
To setup GeoServer I created a stack in portainer:
dockerCopy codeversion: '3' networks: default: external: true name: nginx services: geoserver: image: kartoza/geoserver:latest container_name: geoserver restart: always volumes: - /data/containers/geoserver:/opt/geoserver/data_dir ports: - "8080:8080" environment: GEOSERVER_DATA_DIR: ${GEOSERVER_DATA_DIR} GEOSERVER_ADMIN_PASSWORD: ${GEOSERVER_ADMIN_PASSWORD} GEOSERVER_ADMIN_USER: ${GEOSERVER_ADMIN_USER} INITIAL_MEMORY: ${INITIAL_MEMORY} MAXIMUM_MEMORY: ${MAXIMUM_MEMORY} DB_BACKEND: POSTGRES HOST: ${POSTGRES_HOST} POSTGRES_PORT: 5432 POSTGRES_DB: gwc POSTGRES_USER: ${POSTGRES_USER} POSTGRES_PASS: ${POSTGRES_PASS} POSTGRES_SCHEMA: public DISK_QUOTA_SIZE: 20
After deploying GeoServer you can login with the default user "admin" and password "geoserver" or the credentials you assigned with environment variables.
To add a new datasource you have to click on Stores and Add new store. Then you can select PostGIS (or whatever datasource you want to use).
Sorry, somehow the image couldn't be found :(
Fill out host, port, database, schema, user and passwd and save your connection. Afterwards you can select layers which you want to publish.
Sorry, somehow the image couldn't be found :(
After you published your layer you can preview the layer at Layer Preview.
Now I am not using Geoserver as endpoint to serve the tiles because I have a Mapproxy instance running which will take care of transforming tiles from internal or external ressources.
MapProxy
Setting up MapProxy is optional but in this post I will use the endpoints from MapProxy to render a web map. So if you decied to skip MapProxy you will have to adjust the wms URL's to your endpoint from GeoServer.
MapProxy is an open source proxy for geospatial data. It caches, accelerates and transforms data from existing map services and serves any desktop or web GIS client. I have decided create a own docker image mapproxy-docker which you can setup use in a portainer stack with:
yamlCopy codeversion: '3.9' networks: default: external: true name: nginx services: mapproxy: image: mxdcodes/mapproxy-docker:latest container_name: mapproxy restart: always ports: - 3050:80 volumes: - /mapproxy/config:/persistentConfigFolder - /mapproxy/cache_data:/persistentCacheFolder
The only thing MapProxy needs one or multiple configuration files in yaml format for each service. So set up a locations.yaml under /persistentConfigFolder/locations.yaml:
yamlCopy codeservices: demo: tms: use_grid_names: true # origin for /tiles service origin: 'nw' kml: use_grid_names: true wmts: wms: md: title: Location Data abstract: For mxd.codes. layers: - name: locations title: Locations - mxd.codes sources: [locations_cache] caches: locations_cache: grids: [webmercator] sources: [locations_wms] sources: locations_wms: type: wms req: url: http://geoserver:8080/geoserver/mxdcodes/wms? layers: locations transparent: true grids: webmercator: base: GLOBAL_WEBMERCATOR globals: # cache options cache: # where to store the cached images base_dir: '/cache_data' # where to store lockfiles for concurrent_requests lock_dir: '/cache_data/locks' # where to store lockfiles for tile creation tile_lock_dir: '/cache_data/tile_locks' # request x*y tiles in one step meta_size: [4, 4] # image/transformation options image: # use best resampling for vector data resampling_method: bilinear # nearest/bicubic/bilinear formats: image/jpeg: encoding_options: # jpeg quality [0-100] jpeg_quality: 80
Be sure to replace the sources url with the url from your GeoServer instance.
Now you should have a endpoint for your wms tiles under http://serverurl:3050/locations/service? which you can use to create web map on your website.
I also created each service under a seperate subdomain and setup https for it with Cloudflare but I won't go into detail here because you will find hundreds of tutorials online.
Creating a Web Map with Leaflet and React
The end product of all this should be a simple map on my personal website under /map. The map component looks like the following:
javascriptCopy codeimport { useState, useRef, useEffect } from "react"; import "ol/ol.css"; import Map from "ol/Map"; import TileLayer from "ol/layer/Tile"; import View from "ol/View"; import WMTSTileGrid from "ol/tilegrid/WMTS.js"; import WMTS from "ol/source/WMTS.js"; import { transform, get as getProjection } from "ol/proj"; import { getTopLeft, getWidth } from "ol/extent.js"; // Define types for the props interface LiveMapProps { coords?: { lat: number; lon: number }; // Optional coordinate prop } const LiveMap = ({ coords }: LiveMapProps) => { const [map, setMap] = useState<Map | null>(null); // Map state type const mapElement = useRef<HTMLDivElement | null>(null); // Ref to the map container element const projection = getProjection("EPSG:3857"); const projectionExtent = projection.getExtent(); const size = getWidth(projectionExtent) / 256; const resolutions = new Array(19); const matrixIds = new Array(19); // Populate resolutions and matrixIds for (let z = 0; z < 19; ++z) { resolutions[z] = size / Math.pow(2, z); matrixIds[z] = z; } const locData = new TileLayer({ preload: Infinity, source: new WMTS({ attributions: 'Tiles © <a href="https://mxd.codes/">Max Dietrich</a>', url: `/api/maps`, layer: "locations", matrixSet: "webmercator", format: "image/png", projection: projection, tileGrid: new WMTSTileGrid({ origin: getTopLeft(projectionExtent), resolutions: resolutions, matrixIds: matrixIds, }), style: "default", wrapX: true, }), }); const basemap = new TileLayer({ preload: Infinity, source: new WMTS({ attributions: '© <a href="https://www.openstreetmap.org/copyright">OpenStreetMap</a> contributers', url: `/api/maps`, layer: "osm", matrixSet: "GLOBAL_MERCATOR", format: "image/png", projection: projection, tileGrid: new WMTSTileGrid({ origin: getTopLeft(projectionExtent), resolutions: resolutions, matrixIds: matrixIds, }), style: "default", wrapX: true, }), }); // Set center coordinates (default or from props) /*const center = coords ? transform([coords.lon, coords.lat], "EPSG:4326", "EPSG:3857") : [-15.439457, 28.128124]; // Default center if no coords are provided*/ const center = [-15.439457, 28.128124] useEffect(() => { const map = new Map({ target: mapElement.current!, layers: [basemap, locData], view: new View({ center: center, zoom: 13, maxZoom: 16, }), }); return () => map.setTarget(null); }, []); return ( <div style={{ height: "100%", width: "100%" }} ref={mapElement} className="map-container" /> ); }; export default LiveMap;
Example of the map: Sorry, somehow the image couldn't be found :(