When running databases like PostgreSQL and MariaDB on a server, ensuring regular backups is crucial for protecting your data from unexpected events such as crashes, human error, or system failure. While there are several ways to create backups, scripting a backup solution gives you complete control and automation.

In this article, we'll show you how to create a simple yet effective backup script for PostgreSQL and MariaDB running in Docker containers on a server. We'll automate the process to ensure that your databases are regularly backed up without you needing to manually intervene.

## Why You Need Backups?

Before diving into the script, let’s take a moment to highlight why having regular backups is essential:

1. Protection Against Data Loss: Regular backups prevent loss of valuable data due to failures or accidents.
2. Disaster Recovery: Having up-to-date backups makes it easier to restore services quickly after a failure.
3. Compliance and Audits: Some industries require maintaining backup copies of data for regulatory compliance.
4. Peace of Mind: Knowing your data is safe provides confidence and good sleep.

## Prerequisites

Before creating the backup script, make sure you have:
- A VPS running a Linux-based operating system.
- Docker installed and running.
- PostgreSQL and MariaDB running inside Docker containers.
- Basic knowledge of using the terminal and running Docker commands.

For the purpose of this tutorial, let’s assume your PostgreSQL and MariaDB containers are named `postgres_container` and `mariadb_container`.

## Create a Backup Directory

Start by creating a directory on your server where the backups will be stored. This will help keep everything organized.

```bash
mkdir -p /home/youruser/backup
```
Replace `/home/youruser/backup` with the location where you'd like to store your backups.

## Create the Backup Script

Now let’s create a bash script that will run daily backups for both PostgreSQL and MariaDB databases. Open your favorite text editor and create a file named `backup_databases.sh`.
```bash
#!/bin/bash
set -euo pipefail

# Backup a PostgreSQL database into a daily file.

BACKUP_DIR="/data/backups"
LOG_FILE="/var/log/db_backup.log"
DAYS_TO_KEEP=30
POSTGRESDATABASES=("db1" "db2") # PostgreSQL DBs to backup
MARIADBDATABASES=("db1") # MariaDB DBs to backup
POSTGRESCONTAINER="postgres_container"
MARIADBCONTAINER="mariadb_container"

# Create necessary directories
mkdir -p "${BACKUP_DIR}"
mkdir -p "$(dirname "${LOG_FILE}")"

# Create backup directory if it doesn't exist
mkdir -p "${BACKUP_DIR}"

# Function to log messages
log() {
    local level="$1"
    local message="$2"
    local timestamp
    timestamp=$(date +"%Y-%m-%d %H:%M:%S")

    # Log to both stdout and a log file
    echo -e "${timestamp} [${level}] ${message}" | tee -a "${LOG_FILE}"
}

for DATABASE in "${POSTGRESDATABASES[@]}"; do
    TIMESTAMP=$(date +"%Y%m%d%H%M")
    FILE="${TIMESTAMP}_${DATABASE}.sql.gz"
    OUTPUT_FILE="${BACKUP_DIR}/${FILE}"

    log "INFO" "Starting backup for database: ${DATABASE}"

    # Perform the backup and compress the output
    if docker exec -i "${POSTGRESCONTAINER}" /usr/bin/pg_dump -U "${USER}" "${DATABASE}" | gzip -9 > "${OUTPUT_FILE}"; then
        log "SUCCESS" "Backup created: ${OUTPUT_FILE}"
        ls -l "${OUTPUT_FILE}" | tee -a "${LOG_FILE}"
    else
        log "ERROR" "Backup failed for database ${DATABASE}" >&2
        continue
    fi

    # Prune old backups
    find "${BACKUP_DIR}" -maxdepth 1 -mtime +"${DAYS_TO_KEEP}" -name "*${DATABASE}.sql.gz" -exec rm -f {} \; \
        && log "INFO" "Old backups deleted for database ${DATABASE}" \
        || log "ERROR" "Failed to delete old backups for ${DATABASE}" >&2
done

## MariaDB Backup
for DATABASE in ${MARIADBDATABASES[@]}; do
    TIMESTAMP=$(date +"%Y%m%d%H%M")
    FILE="${TIMESTAMP}_${DATABASE}.sql.gz"
    OUTPUT_FILE="${BACKUP_DIR}/${FILE}"

    # Perform the database backup (dump)
    if docker exec ${MARIADBCONTAINER} /usr/bin/mariadb-dump -u  root --password=yourpassword ${DATABASE} | gzip -9 >  ${OUTPUT_FILE}; then
        log "SUCCESS" "Backup created: ${OUTPUT_FILE}"
        ls -l "${OUTPUT_FILE}" | tee -a "${LOG_FILE}"
    else
        log "ERROR" "Backup failed for database ${DATABASE}" >&2
        continue
    fi

    # Prune old backups
    find "${BACKUP_DIR}" -maxdepth 1 -mtime +"${DAYS_TO_KEEP}" -name "*${DATABASE}.sql.gz" -exec rm -f {} \; \
        && log "INFO" "Old backups deleted for database ${DATABASE}" \
        || log "ERROR" "Failed to delete old backups for ${DATABASE}" >&2
done

log "INFO" "Finished database backups!"
```

**Explanation:**
- **Variables:** The script starts by defining a backup directory and more parameters like days to keep backups and individual databases to backup.
- **PostgreSQL Backup:** It uses `docker exec` to run the `pg_dump` command inside the PostgreSQL container to dump previously defined databases.
- **MariaDB Backup:** Similarly, it uses `docker exec` to run the `mysqldump` inside the MariaDB container and backup previously defined databases.
- **Compression:** All backups are compressed using `gzip` to save space.
- **Backup Cleanup:** Finally, the script deletes backups older than `DAYS_TO_KEEP` days to prevent disk space issues.

**Customizing the Script:**
- Change `BACKUP_DIR` to the directory where you want to store your backups.
- Replace `DAYS_TO_KEEP` with the days how long you want to keep backups.
- Replace `POSTGRESDATABASES` with the PostgreSQL databases to backup.
- Replace `MARIADBDATABASES` with the MariaDB databases to backup.
- Replace `POSTGRESCONTAINER` and  `MARIADBCONTAINER` with the name of your PostgreSQL and MariaDB containers.
- Replace `yourpassword` with the password for the root user in MariaDB..

## Make the Script Executable

After saving the script, make it executable:
```bash
chmod +x /path/to/backup_databases.sh
```

## Automate the Backup with Cron

To schedule automatic backups, set up a cron job.

- Open the crontab editor:
```bash
crontab -e
```
- Add the following line to run the backup script daily at 2:00 AM:
```bash
0 2 * * * /path/to/backup_databases.sh
```

This will execute the backup script every day at 2:00 AM.

Make sure to adjust the path ```/path/to/backup_databases.sh``` to the correct location of your script.

## Verify the Backup

It’s always a good idea to manually run the backup script once to ensure everything is working correctly.
```bash
/path/to/backup_databases.sh
```
Check the backup directory to ensure that the backup files have been created and compressed.

## Restore from Backup (Bonus)

In case you need to restore a backup, you can use the following commands to load the backups back into your PostgreSQL and MariaDB containers.

**PostgreSQL Restore:**
```bash
docker exec -i postgres_container psql -U postgres -d database < /home/youruser/backup/DATE_DATABASE.sql
```

**MariaDB Restore:**
```bash
docker exec -i mariadb_container mariadb u root --password=yourpassword database < /home/youruser/backup/DATE_DATABASE.sql
```

Replace `DATE` and `DATABASE` with the appropriate backup file’s date and database name

By following these steps, you've created a simple and automated backup solution for your PostgreSQL and MariaDB databases running inside Docker containers. Regular backups are essential for protecting your data, and this script ensures that your backups run smoothly without manual intervention. You can also use this script to backup databases e.g. in Unraid with the `User Script`plugin.

You can further enhance this backup strategy by sending notifications, backing up to remote storage (e.g., AWS S3 or Google Cloud), or setting up encryption for additional security.

With your databases securely backed up, you can rest easy knowing your data is safe and easily recoverable in case of an emergency.

Learn how to automate backups for PostgreSQL and MariaDB running in Docker containers on a server. Protect your data with this simple, customizable backup script, ensuring regular backups with minimal effort.

How to Create a Backup Script for PostgreSQL and MariaDB Containers on a Server

Webmentions are a powerful tool for adding decentralized social interactions, such as comments, likes, reposts, and replies, directly on your website. If you're building a dynamic site with Next.js, integrating Webmentions can help encourage cross-site conversations, boost SEO, and enhance user engagement. In this guide, I will show you how to implement Webmentions into your Next.js project with PostgreSQL for storing and displaying them.

## What Are Webmentions?

[Webmention](https://www.w3.org/TR/webmention/) is an open web standard (W3C Recommendation) that enables decentralized cross-site interactions.

In simpler terms, Webmentions allow users to interact with your content across the web by leaving [comments](https://indieweb.org/comment), [likes](https://indieweb.org/like), [reposts](https://indieweb.org/repost) and other [responses](https://indieweb.org/responses) on other sites. These interactions enrich your site’s user experience, and they help establish meaningful connections with others.

When you link to a webpage, you can send a Webmention notification. If the receiving site supports Webmentions, it may display your post as a comment, like, or response—enabling rich cross-site conversations.

**Why Should You Use Webmentions?**
- Encourage Engagement: Promote cross-site discussions and interactions.
- Boost Social Proof: Display interactions from well-known sources to establish credibility.
- Enhance SEO: Webmentions generate backlinks, improving visibility and search engine rankings.

Here’s an example of how Webmentions appear on my site:

![2024-02-23_18-23.png](https://cms.mxd.codes/uploads/2024_02_23_18_23_ad04b4ba37.png)

You can check a live version of this in the Webmentions section of this article: [/articles/fetching-and-storing-activities-from-garmin-connect-with-strapi-and-visualizing-them-with-next-js#replies].

A typical webmentions structure in JSON looks like this:

```json
{
  "type": "entry",
  "author": {
    "type": "card",
    "name": "Some Name",
    "photo": "URL to author image",
    "url": "URL to author profile"
  },
  "url": "Webmention URL",
  "wm-received": "Date of Webmention",
  "wm-id": 1876563,
  "wm-source": "Source URL",
  "wm-target": "Target URL",
  "wm-property": "Type of mention (e.g., like-of, repost-of)",
  "wm-private": false
}
```

To keep your Webmentions accessible even if an external service is discontinued, it’s a good idea to store them locally. In this tutorial, we’ll guide you through setting up a PostgreSQL database to store Webmentions and display them dynamically in your Next.js app.

## Setting Up PostgreSQL

Before we dive into Webmentions, ensure you have PostgreSQL installed on your server. If not, check on of these [guides](https://www.ecosia.org/search?method=index&q=setup+postgresql).

Once PostgreSQL is ready:

- **Create a New Database:**
```bash
# Create a new database for storing Webmentions
createdb personalwebsite
``` 
- **Create a Table to Store Webmentions:** Here's the SQL structure to store Webmentions in your database:
```sql
-- Define a table structure to store Webmentions

-- DROP TABLE public.webmentions;

CREATE TABLE public.webmentions (
	id serial4 NOT NULL,
	wm_id int8 NOT NULL,
	wm_source text NOT NULL,
	wm_target text NOT NULL,
	wm_property text NOT NULL,
	url text NULL,
	author_name text NULL,
	author_photo text NULL,
	author_url text NULL,
	content_html text NULL,
	content_text text NULL,
	published_at timestamp NULL,
	received_at timestamp DEFAULT CURRENT_TIMESTAMP NULL,
	CONSTRAINT webmentions_pkey PRIMARY KEY (id),
	CONSTRAINT webmentions_wm_id_key UNIQUE (wm_id)
);
```

- **Create a Log Table for Fetching Webmentions:** This table logs each fetch, allowing you to track when Webmentions were last retrieved and avoid overloading the service:

```sql
-- public.webmention_fetch_log definition

-- Drop table

-- DROP TABLE public.webmention_fetch_log;

CREATE TABLE public.webmention_fetch_log (
	id serial4 NOT NULL,
	last_fetch timestamptz NOT NULL,
	CONSTRAINT webmention_fetch_log_pkey PRIMARY KEY (id)
);
```

## Using Webmention.io to Receive Webmentions

Before we store webmentions to display them we have to get them somewhere. If you dont want to implement your own Webmentions Receiver I recommend to use [Webmention.io](https://webmention.io/) which is a service to easily receive webmentions.

**Steps:**
1. **Set Up IndieAuth for Your Site:** Follow these[ IndieAuth setup instructions](https://indieauth.com/setup)
2. **Sign Up for Webmention.io:** Create an account at[ Webmention.io](https://webmention.io/)
3. **Add the Webmention Link to Your `_app.tsx`:**

```html
<Head>
  ...
  <link rel="webmention" href="https://webmention.io/username/webmention" />
  ...
  </Head>
```

From here on, Webmention.io will collect all the Webmentions for your site. Now, let’s create a script that fetches and stores them every ten minutes in the PostgreSQL database

## Fetching Webmentions from Webmention.io API

To keep Webmentions up-to-date, we'll fetch them periodically. Here’s the logic of the script.

![fetchWebmentions.jpg](https://cms.mxd.codes/uploads/fetch_Webmentions_396b7d7dd6.jpg)

Create a script `src/utils/fetch-webmentions.js` to fetch and store Webmentions:

```jsx
import fetch from "node-fetch"
import { Pool } from "pg"

const pool = new Pool({
  user: process.env.PGUSER,
  host: process.env.PGHOST,
  database: process.env.PGDATABASE,
  password: process.env.PGPASSWORD,
  port: process.env.PGPORT,
})

function isNotOlderThanTenMinutes(date: Date) {
  if (!(date instanceof Date) || isNaN(date.getTime())) return false
  return Date.now() - date.getTime() <= 10 * 60 * 1000
}

export async function fetchAndStoreWebmentions() {
  const client = await pool.connect() // Use a client for transaction safety
  try {
    console.log("🔄 Checking last webmention fetch...")

    // Get the latest fetch timestamp
    const { rows } = await client.query(
      `SELECT last_fetch FROM webmention_fetch_log ORDER BY last_fetch DESC LIMIT 1`
    )
    const lastFetchDate = rows[0]?.last_fetch

    if (isNotOlderThanTenMinutes(lastFetchDate)) {
      console.log("✅ Webmentions are already updated!")
      return
    }

    // Insert new fetch timestamp
    const now = new Date().toISOString()
    await client.query(`INSERT INTO webmention_fetch_log (last_fetch) VALUES ($1)`, [now])
    console.log("📌 Updated Webmentions fetch log")

    // Generate Webmention API URL
    const baseUrl = `https://webmention.io/api/mentions.jf2?domain=mxd.codes&per-page=1000&page=0&token=rxwJcoKxWCd9n3NA9SoxeQ`
    const webmentionsUrl =
      lastFetchDate instanceof Date && !isNaN(lastFetchDate.getTime())
        ? `${baseUrl}&since=${lastFetchDate.toISOString()}`
        : baseUrl

    // Fetch new Webmentions from Webmention.io
    console.log("🔄 Fetching webmentions from Webmention.io...")
    const response = await fetch(webmentionsUrl)
    const { children: webmentions } = await response.json()

    if (!Array.isArray(webmentions) || webmentions.length === 0) {
      console.log("⚠️ No new webmentions found.")
      return
    }

    console.log(`📥 Processing ${webmentions.length} webmentions...`)

    // Prepare batch insert query
    const insertQuery = `
      INSERT INTO webmentions (
        wm_id, wm_source, wm_target, wm_property, url,
        author_name, author_photo, author_url, content_html, content_text, published_at, received_at
      ) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12)
      ON CONFLICT (wm_id) DO NOTHING;
    `

    for (const mention of webmentions) {
      const values = [
        mention["wm-id"],
        mention["wm-source"],
        mention["wm-target"],
        mention["wm-property"],
        mention["url"],
        mention.author?.name || null,
        mention.author?.photo || null,
        mention.author?.url || null,
        mention.content?.html || null,
        mention.content?.text || null,
        mention.published ? new Date(mention.published) : null,
        new Date(mention["wm-received"]),
      ]
      await pool.query(insertQuery, values)
    }

    console.log(`✅ Stored ${webmentions.length} webmentions successfully!`)
  } catch (error) {
    console.error("❌ Error fetching or storing webmentions:", error)
  } finally {
    client.release() // Ensure client is released back to the pool
  }
}

// Run the function
fetchAndStoreWebmentions()
```

This function can now be called everytime before webmentions are queried for a page from PostgreSQL database.
Ideally, you should abstract the logic for determining whether new webmentions need to be fetched into an API layer. This prevents unnecessary database queries with every request, but this is out of scope for this article.

## Retrieving Webmentions for a Page

To retrieve Webmentions dynamically for a page, we create an API route in `pages/api/get-webmentions.js`. This route allows us to fetch mentions for a specific target URL stored in our PostgreSQL database.

```jsx
import { Pool } from "pg"
import { fetchAndStoreWebmentions } from "@/src/utils/fetch-webmentions"

const pool = new Pool({
  user: process.env.PGUSER,
  host: process.env.PGHOST,
  database: process.env.PGDATABASE,
  password: process.env.PGPASSWORD,
  port: process.env.PGPORT,
})

export default async function handler(req, res) {
  if (req.method !== "GET")
    return res.status(405).json({ error: "Method not allowed" })

  const { target } = req.query
  if (!target) return res.status(400).json({ error: "Missing target URL" })

  // Updating Webmentions before selecting for page url
  await fetchAndStoreWebmentions()
  const query = `SELECT wm_id, wm_source, wm_target, wm_property, url, author_name, author_photo, author_url, content_text, published_at FROM webmentions WHERE wm_target LIKE '%${target}%' ORDER BY received_at DESC;`
  const result = await pool.query(query)

  res.json(result.rows)
}
```

Now you can call this API route and pass a query param `target` with the URL to get all Webmentions for a page.

## Displaying Webmentions in Next.js

To visually display Webmentions on our website, we create a dedicated React component `components/Webmentions.js`. This component fetches the Webmentions from our API and renders them.

```javascript
import { useEffect, useState } from "react";

// React component to display Webmentions for a given page
const Webmentions = ({ targetUrl }) => {
  const [mentions, setMentions] = useState([]);

  useEffect(() => {
  // Fetch Webmentions for the target URL from the API route
    fetch(`/api/get-webmentions?target=${encodeURIComponent(targetUrl)}`)
      .then((res) => res.json())
      .then((data) => setMentions(data));
  }, [targetUrl]);

  return (
    <div>
      <h3>Webmentions</h3>
      {mentions.length === 0 ? (
        <p>No webmentions yet.</p>
      ) : (
        mentions.map((mention) => (
          <div className="vcard h-card p-author" key={mention.wm_id} style={{ border: "1px solid #ddd", padding: "10px", marginBottom: "10px" }}>
            {mention.author_photo && (
               // Display author profile picture if available
              <img src={mention.author_photo} alt={mention.author_name} className="u-photo" style={{ width: "40px", height: "40px", borderRadius: "50%" }} />
            )}
            <p>
              <strong>{mention.author_name}</strong> {mention.wm_property.replace("-", " ")}
              {mention.wm_property === "like-of" && " ❤️"}
              {mention.wm_property === "repost-of" && " 🔁"}
              {mention.wm_property === "in-reply-to" && " 💬"}
            </p>
            <a className="u-url" href={mention.url || mention.wm_source} target="_blank" rel="noopener noreferrer">View Mention</a>
          </div>
        ))
      )}
    </div>
  );
};

export default Webmentions;
```

I highly recommend periodically verifying the authenticity of Webmention sources to prevent spam.

## Testing Webmentions

To ensure your Webmentions setup works correctly, use the following tools:

- [Webmention Rocks!](https://webmention.rocks/) – A tool for testing Webmention implementations.

By integrating Webmentions into your Next.js site, you can create an interactive and engaging web community. Whether you're running a blog, portfolio, or e-commerce site, Webmentions provide a powerful way to enhance content, boost SEO, and encourage meaningful connections.

If you have created a response to this post you can send me a webmention and it will appear below the post.

## More Links
- [IndieWeb Webmention](https://indieweb.org/Webmention)
- You can read more about Webmention and the IndieWeb on the [https://indieweb.org/](https://indieweb.org).

Learn how to integrate Webmentions into your Next.js site using PostgreSQL. Enhance engagement, foster social interactions, and build a dynamic web community with this step-by-step guide.

 Enhancing Social Interactions: Implementing Webmentions with Next.js and PostgreSQL

Many developers rely on Google’s Static Maps API to generate map images, but this has limitations such as:
- API rate limits
-  Cost for high-volume usage
-  Potential data privacy concerns
-  Dependency on an external service

For a long time I was looking for an alternative to Maps Static API from Google which can be selfhosted but I couldn't find anything which seemed to fit my needs.

However I found [staticmaps](https://www.npmjs.com/package/staticmaps "staticmaps") which is a Node.JS library for creating map images with markers, polylines, polygons and text. But the library doesn't provide a web interface so I decided to built one on top of it with [express](https://github.com/expressjs/express "express") and containerized the staticmaps API.

## Docker Static Maps API

[docker-staticmaps](https://github.com/dietrichmax/docker-staticmaps "Docker Static Maps API") is a containerized web version for [staticmaps](https://www.npmjs.com/package/staticmaps) with [express](https://github.com/expressjs/express).

In general `docker-staticmaps` provides a self-hosted alternative that allows you to generate static map images on your own server without relying on third-party APIs.

## Usage

To get a static map from the endpoint `/staticmaps` several prameters have to be provided.

Parameter	Description	Example
center	Latitude,Longitude of map center	48.8588443,2.2943506
zoom	Zoom level (1-20)	12
size	Image size in pixels	600x400
marker	Add a marker at a location	48.8588,2.2943,red
polyline	Draw a line on the map	48.85,2.29;48.86,2.30,blue,2

- `center` - Center coordinates of the map in the format `lon, lat`
- `zoom` - Set the zoom level for the map.
- `width` - default `300` - Width in pixels of the final image
- `height` - default `300` - Height in pixels of the final image
- `format` - default `png` (e.g. `png`, `jpg` or `webp`)
- `basemap` - default `osm` - Map base layer

### Basemaps

For different basemaps docker-staticmaps is using exisiting tile-services from various providers. Be sure to check their Terms of Use for your use case or use a `custom` tileserver with the `tileUrl` parameter!

- `basemap` - default "osm" - Select the basemap
  - `osm` - default - [Open Street Map](https://www.openstreetmap.org/)
  - `streets` - [Esri street basemap](https://www.arcgis.com/home/webmap/viewer.html?webmap=7990d7ea55204450b8110d57e20c99ab)
  - `satellite` - Esri's [satellite basemap](https://www.arcgis.com/home/webmap/viewer.html?webmap=d802f08316e84c6592ef681c50178f17&center=-71.055499,42.364247&level=15)
  - `hybrid` - Satellite basemap with labels
  - `topo` - Esri [topographic map](https://www.arcgis.com/home/webmap/viewer.html?webmap=a72b0766aea04b48bf7a0e8c27ccc007)
  - `gray` - Esri gray canvas with labels
  - `gray-background` - Esri [gray canvas](https://www.arcgis.com/home/webmap/viewer.html?webmap=8b3d38c0819547faa83f7b7aca80bd76) without labels
  - `oceans` - Esri [ocean basemap](https://www.arcgis.com/home/webmap/viewer.html?webmap=5ae9e138a17842688b0b79283a4353f6&center=-122.255816,36.573652&level=8)
  - `national-geographic` - [National Geographic basemap](https://www.arcgis.com/home/webmap/viewer.html?webmap=d94dcdbe78e141c2b2d3a91d5ca8b9c9)
  - `otm` - [OpenTopoMap](https://www.opentopomap.org/)
  - `stamen-toner` - [Stamen Toner](http://maps.stamen.com/toner/) black and white map with labels
  - `stamen-toner-background` - [Stamen Toner](http://maps.stamen.com/toner-background/) map without labels
  - `stamen-toner-lite` - [Stamen Toner Light](http://maps.stamen.com/toner-lite/) with labels
  - `stamen-terrain` - [Stamen Terrain](http://maps.stamen.com/terrain/) with labels
  - `stamen-terrain-background` - [Stamen Terrain](http://maps.stamen.com/terrain-background/) without labels
  - `stamen-watercolor` - [Stamen Watercolor](http://maps.stamen.com/watercolor/)
  - `carto-light` - [Carto](https://carto.com/location-data-services/basemaps/) Free usage for up to 75,000 mapviews per month, non-commercial services only.
  - `carto-dark` - [Carto](https://carto.com/location-data-services/basemaps/) Free usage for up to 75,000 mapviews per month, non-commercial services only.
  - `carto-voyager` - [Carto](https://carto.com/location-data-services/basemaps/) Free usage for up to 75,000 mapviews per month, non-commercial services only.
  - `custom` - Pass through the tile URL using parameter `tileurl`

### Polylines

With the parameter `polyline` you can add a polyline to the map in the following format:

`polyline=polylineStyle|polylineCoord1|polylineCoord2|...`

- `polylineCoord` - required - in format `lat,lon` and seperated by `|`. Atleast two locations are needed to draw a polyline.

The `polylineStyle` consists of the following two parameters separated by `|`.

- `weight` - default `5` - Weight of the polyline in pixels, e.g. `weight:5`
- `color` - default `blue` -24-Bit-color hex value, e.g. `color:0000ff`

If no `center` is specified, the polyline will be centered.

<details>
  <summary>Polyline with no `zoom`, `weight:6` and `color:0000ff`</summary>
  <p>http://localhost:3000/staticmaps?width=600&height=600&polyline=weight:6|color:0000ff|48.726304979176675,-3.9829935637739382|48.72623035828412,-3.9829726446543385|48.726126671101639,-3.9829546542797467|48.725965124843256,-3.9829070729298808|48.725871429380568,-3.9828726793245273|48.725764250990267,-3.9828064532306628|48.725679557682362,-3.9827385375789146|48.72567025076134,-3.9827310750289113|48.725529844164292,-3.9826617613709225|48.725412537198615,-3.9826296635284164|48.725351694726704,-3.9826201452878531|48.725258599474508,-3.9826063049230411|48.725157520450125,-3.9825900299314232|48.725077863838543,-3.9825779905509102|48.724930435729831,-3.9825514102373938|48.724815578113535,-3.9825237355887291|48.724760905376989,-3.9825013965800564|48.724677938456551,-3.9824534296566916|48.724379435330384,-3.9822469276001118|48.724304509274596,-3.9821850264836076|48.7242453124599,-3.9821320570321772|48.724206187829317,-3.9821063430223207|48.724117073204575,-3.9820862134785551</p>
  </details>

![polylinepath.png](https://cms.mxd.codes/uploads/polylinepath_d9f50f216a.png)

### Polygons

With the parameter `polygon` you can add a polygon to the map in the following format:

`polygon=polygonStyle|polygonCoord1|polygonCoord2|...`

- `polygonCoord` - required - in format `lat,lon` and seperated by `|`. First and last locations have to be the same to close the polygon.

The `polygonStyle` consists of the following two parameters separated by `|`.

- `color` - default `blue` -24-Bit-color hex value, e.g. `color:4874db`
- `weight` - default `5` - Weight of the polygon in pixels, e.g. `weight:5`
- `fill` - default `green` -24-Bit-color hex value, e.g. `fill:eb7a34`

If no `center` is specified, the polygon will be centered.

<details>
  <summary>Polygon with no `zoom`, `color:4874db`,`weight:7` and `fill:eb7a3`</summary>
    <p>http://localhost:3000/staticmaps?width=600&height=600&polygon=color:4874db|weight:7|fill:eb7a34|41.891169,12.491691|41.890633,12.493697|41.889012,12.492989|41.889467,12.490811|41.891169,12.491691</p>
  </details>

![polygonexample.png](https://cms.mxd.codes/uploads/polygonexample_a8167c572c.png)


### Markers

With the parameter `markers` you can draw one or multiple markers depending on how much pair of coordinates you pass to the parameter

`markers=markerCoord1|markerCoord2|...`

- `markerCoord` - required - in format `lat,lon` and separated by `|`. Atleast one coordinate is needed to draw a marker.

If no `center` is specified, the markers will be centered.

<details>
  <summary>Markers</summary>
  <p>http://localhost:3000/staticmaps?width=600&height=600&markers=48.726304979176675,-3.9829935637739382|48.724117073204575,-3.9820862134785551</p>
  </details>

![markers.png](https://cms.mxd.codes/uploads/markers_e3060a2b45.png)

### Circles

With the parameter `circle` you can add a circle to the map in the following format:

`circle=circleStyle|circleCoord`

- `circleCoord` - required - in format `lat,lon` and separated by `|`. Atleast one locations is needed to draw a marker.

The `circleStyle` consists of the following parameters seperated by `|`.

- `radius` - required - Circle radius in meter, e.g. `radius:500`
- `color` - default `#0000bb` -  Stroke color of the circle, e.g. `color:#0000bb`
- `width` - default `3` - Stroke width of the circle, e.g. `width:3`
- `fill` - default `#AA0000` - Fill color of the circle, e.g. `fill:#AA0000`

If no `center` is specified, the circle will be centered.

<details>
  <summary>Circle with no zoom</summary>
  <p>http://localhost:3000/staticmaps?width=600&height=600&basemap=osm&circle=radius:100|48.726304979176675,-3.9829935637739382</p>
  </details>

![circle.png](https://cms.mxd.codes/uploads/circle_646b7609a9.png)

### More usage Examples

<details>
  <summary>Minimal example: `center` and `zoom`</summary>
  <p>http://localhost:3000/staticmaps?center=-119.49280,37.81084&zoom=9</p>
</details>

![minimalexample.png](https://cms.mxd.codes/uploads/minimalexample_ce216c4b8e.png)

<details>
  <summary>`width=500`, `height=500`, `center=-73.99515,40.76761`,  `zoom=10`, `format=webp`, `basemap=carto-voyager`</summary>
  <p>http://localhost:3000/staticmaps?width=500&height=500&center=-73.99515,40.76761&zoom=10&format=webp&basemap=carto-voyager</p>
</details>

![example2.webp](https://cms.mxd.codes/uploads/example2_2e3cbb505b.webp)

<details>
  <summary>Markers and Polyline</summary>
  <p>http://localhost:3000/staticmaps?width=600&height=600&polyline=weight:6|color:0000ff|48.726304979176675,-3.9829935637739382|48.72623035828412,-3.9829726446543385|48.726126671101639,-3.9829546542797467|48.725965124843256,-3.9829070729298808|48.725871429380568,-3.9828726793245273|48.725764250990267,-3.9828064532306628|48.725679557682362,-3.9827385375789146|48.72567025076134,-3.9827310750289113|48.725529844164292,-3.9826617613709225|48.725412537198615,-3.9826296635284164|48.725351694726704,-3.9826201452878531|48.725258599474508,-3.9826063049230411|48.725157520450125,-3.9825900299314232|48.725077863838543,-3.9825779905509102|48.724930435729831,-3.9825514102373938|48.724815578113535,-3.9825237355887291|48.724760905376989,-3.9825013965800564|48.724677938456551,-3.9824534296566916|48.724379435330384,-3.9822469276001118|48.724304509274596,-3.9821850264836076|48.7242453124599,-3.9821320570321772|48.724206187829317,-3.9821063430223207|48.724117073204575,-3.9820862134785551&markers=48.726304979176675,-3.9829935637739382|48.724117073204575,-3.9820862134785551</p>
  </details>

![markersandpolyline.png](https://cms.mxd.codes/uploads/markersandpolyline_04626a7660.png)



## Deployment

**with Docker**

```bash
docker run -d  \
  --name='static-maps-api' \
  -p '3003:3000/tcp' \
  'mxdcodes/docker-staticmaps:latest'
```

**with Node.js**

```bash
git clone https://github.com/dietrichmax/docker-staticmaps
npm i
npm run start
```

**Links**
* GitHub: [https://github.com/dietrichmax/docker-staticmaps](https://github.com/dietrichmax/docker-staticmaps)
* Docker Hub: [https://hub.docker.com/r/mxdcodes/docker-staticmaps](https://hub.docker.com/r/mxdcodes/docker-staticmaps)

A containerized web version for staticmaps with express.

docker-staticmaps: A selfhosted Static Maps API

[MapProxy](https://mapproxy.org/ "MapProxy") is a powerful open-source proxy for geospatial data that allows for efficient caching and serving of map tiles. Combining MapProxy with Docker and Nginx can provide a scalable and easily manageable solution for serving cached map tiles. This guide will walk you through the process of setting up MapProxy using Docker and configuring Nginx to serve cached tiles.

**Prerequisites:**

1. Docker installed on your system
2. Basic understanding of Docker
3. Basic understanding of Nginx

If you don't meet this prerequisites yet I recommend to have a look at the following guides first:
- [Nginx Beginner’s Guide](https://nginx.org/en/docs/beginners_guide.html "Nginx Beginner’s Guide")
- [Docker overview](https://docs.docker.com/get-started/overview/ "Docker overview")
- [Install Docker Engine](https://docs.docker.com/engine/install/ "Install Docker Engine")

##  Set Up MapProxy with Docker

Start by creating a Network in Docker with:

```bash
sudo docker create network nginx
```

By adding the network nginx to the Nginx Container and the Mapproxy container the containers can communicate with each other without exposing ports on the server.

Create a Docker Compose file `docker-compose.yml` with the following content:

```docker
networks:
  default:
    external: true
    name: nginx
    
services:
  mapproxy:
    image: kartoza/mapproxy
    container_name: mapproxy
    restart: always
    environment:
      PRODUCTION: true
      PROCESSES: 4
      CHEAPER: 2
      THREADS: 8
      MULTI_MAPPROXY: true
      MULTI_MAPPROXY_DATA_DIR: /multi_mapproxy/configurations
      ALLOW_LISTING: true
    volumes:
      - /data/containers/mapproxy/data:/multi_mapproxy
```

Save the file and run:

```bash
docker-compose up -d
```

This will pull the MapProxy Docker image and start a container. MapProxy will be accessible on the container with `http://localhost:8080`.

## Set Up Nginx with Docker

Afterwards you need to create a Docker Compose file `docker-compose.yml` for Nginx. Here's a example:

```docker
version: '3'

networks:
  default:
    external: true
    name: nginx
    
services:
  nginx:
    image: nginx:latest
    container_name: nginx
    restart: always
    ports:
      - "80:80"
      - "443:443"
    volumes:
      ## Config
      - /data/containers/nginx/config/:/etc/nginx/
      ## SSL
      - /etc/letsencrypt/:/etc/letsencrypt/
      - /etc/ssl/:/etc/ssl/
      ## Logs
      - /data/containers/nginx/logs/:/var/log/nginx
      ## Cache
      - /data/containers/nginx/cache:/var/cache/nginx
```

Then you can create a virtual host for mapproxy under `/data/containers/nginx/config/sites-available/mapproxy` with:

```bash
sudo nano /data/containers/nginx/config/sites-available/mapproxy
```

Copy and paste the following virtual host configuration for your Mapproxy Container :

```nginx
upstream mapproxy_upstream {
  server mapproxy:8080;
}

server {

  server_name mapproxy.domain.com;

  listen 80;

  ## Mapproxy default
  location / {
    proxy_pass http://mapproxy_upstream/;
    proxy_set_header Host $http_host;
  }
}
```

After you created your configuration you need to create a symlink to `/data/containers/nginx/config/sites-enabled/` with:

```bash
sudo ln -S /data/containers/nginx/config/sites-available/mapproxy /data/containers/nginx/config/sites-enabled/
```

In order that your domain gets resolved you have to create a A-record for your Domain under which you want to publish MapProxy which points to your server ip.
Then you need to restart your Nginx Container and now should be able to access http://mapproxy.domain.com where you will be greeted by your MapProxy Instance.

![Screenshot_2024_01_25_154515_63e86fb113.png](https://cms.mxd.codes/uploads/Screenshot_2024_01_25_154515_63e86fb113_aed92b8f04.png)

## Step 3: Caching the Tiles with MapProxy

So far you have set up Nginx and MapProxy with Docker and MapProxy will cache by default all served tiles with the default configuration.
However there are some limitations to the MapProxy caching process because each time a tile gets requested from MapProxy, MapProxy will save the tile to it's cache. But it won't check by default if the tile is old and a newer tile could be served.

E.g. I am serving tiles from MapProxy which visualize my locations where I have ever been to. So I don't want to cache the tiles infinite because they change probably every day

But you can force MapProxy to refresh tiles from the source while serving if they are found to be expired.
The validity conditions are the same as for seeding:

```yaml
#Explanation
  # absolute as ISO time
  refresh_before:
    time: 2010-10-21T12:35:00
  # relative from the time of the tile request
  refresh_before:
    weeks: 1
    days: 7
    hours: 4
    minutes: 15
  # modification time of a given file
  refresh_before:
    mtime: path/to/file
```

So to stay at my example i added:

```yaml
refresh_before:
      days 1
```

This way MapProxy will refresh tiles everyday. This of course only makes sense for data where you want always latest data

So edit your `mapproxy.yaml` like this:

```yaml
services:
  demo:
  tms:
    use_grid_names: true
    # origin for /tiles service
    origin: 'nw'
  kml:
      use_grid_names: true
  wmts:
  wms:
    md:
      title: MapProxy WMS Proxy
      abstract: This is a minimal MapProxy example.

layers:
  - name: osm
    title: Omniscale OSM WMS - osm.omniscale.net
    sources: [osm_cache]

caches:
  osm_cache
    grids: [webmercator]
    sources: [osm_wms]
    refresh_before:
      days: 1

sources:
  osm_wms:
    type: wms
    req:
      url: https://maps.omniscale.net/v2/demo/style.default/service?
      layers: osm

grids:
    webmercator:
        base: GLOBAL_WEBMERCATOR

globals:
```

Save the file and restart your MapProxy Container.

Now MapProxy will always refresh tiles everyday.

## Step 4: Access the MapProxy Tiles

Visit `http://mapproxy.domain.com` in your browser, and you should see MapProxy serving tiles through Nginx. 

By following these steps, you've successfully set up MapProxy with Docker and configured Nginx to serve cached map tiles. This scalable solution allows for efficient geospatial data delivery with the added benefit of easy container management. Adjust the configurations based on your specific requirements and integrate this setup into your mapping projects.

If you decide to to take use of [MultiMapProxy](https://mapproxy.org/docs/nightly/deployment.html "MultiMapProxy")(scroll down to MultiMapProxy) you can just create more configuration files for MapProxy in `/data/containers/mapproxy/data/configurations` and  append the cache path and location blocks like in your existing Nginx configuration for MapProxy.

MapProxy is a powerful open-source proxy for geospatial data that allows for efficient caching and serving of map tiles. Combining MapProxy with Docker and Nginx can provide a scalable and easily manageable solution for serving cached map tiles.

 Setting Up MapProxy with Docker and Serving Cached Tiles via Nginx

[Next.js](https://nextjs.org/ "Next.js") Image Component [next-image](https://nextjs.org/docs/pages/api-reference/components/image  "next-image") is a feature introduced in Next.js version 10.0.0 to optimize images and improve the performance of your web-application.

When you use the Next.js Image Component, it automatically optimizes and serves images in modern image formats that improves the performance of your web application. It supports various image sources, such as local images, images from the web, and third-party sources.

However you cannot transform image, e.g. crop images, which is the reason I was looking for a solution which enables my personal website [mxd.codes](https://mxd.codes "mxd.codes") to resize images to my needs.

## imgproxy

[imgproxy](https://imgproxy.net/ "imgproxy") is an open-source image processing server designed to simplify the resizing, cropping, and manipulation of images on the fly. It is often used as part of a web application's infrastructure to ensure efficient delivery of images with optimized sizes and quality. 

Key features of imgproxy include:

1. **On-the-Fly Image Processing:** Imgproxy allows you to resize, crop, rotate, and perform other image manipulations on the fly, based on the URL parameters. This enables efficient delivery of images in various sizes and formats without having to store multiple versions of the same image.

2. **Security:** Imgproxy provides security features such as URL signature generation. This helps prevent unauthorized access and abuse of the image manipulation service.

3. **Performance:** Imgproxy is designed to be performant and can efficiently handle high loads of image processing requests.

4. **Integration with Existing Storage:** Imgproxy can be integrated with various storage solutions, including Amazon S3, Google Cloud Storage, and more.

## Deploy imgproxy with Docker Compose

While searching for a way to deploy imgproxy with docker I found a [imgproxy Docker Compose Project on GitHub](https://github.com/shinsenter/docker-imgproxy/blob/main/docker-compose.yml "imgproxy docker compose") where I changed minor things like the volumes and the web-server configuration.

You can copy this ```docker-compose.yml```file and paste it into Portainer or save it manually in a folder on your server.

```docker
version: '3'

################################################################################
# Ultra Image Server
# A production grade image processing server setup powered by imgproxy and nginx
#
# Author: Mai Nhut Tan <shin@shin.company>
# Copyright: 2021-2023 SHIN Company https://code.shin.company/
# URL: https://shinsenter.github.io/docker-imgproxy/
################################################################################

networks:
################################################################################
  default:
    driver: bridge


services:
################################################################################
  web:
    image: nginx:alpine
    container_name: imgproxy-nginx
    restart: always
    volumes:
      - /data/containers/imgproxy:/var/www/html:ro
      - /etc/imgproxy/imgproxy-nginx.conf:/etc/nginx/conf.d/default.conf:ro
    ports:
      - 8080:80
    links:
      - imgproxy:imgproxy
    environment:
      NGINX_ENTRYPOINT_QUIET_LOGS: 1

################################################################################
  imgproxy:
    restart: unless-stopped
    image: darthsim/imgproxy:${IMGPROXY_TAG:-latest}
    container_name: imgproxy_app
    security_opt:
      - no-new-privileges:true
    volumes:
      - /data/containers/imgproxy:/var/www/html:ro
    expose:
      - 8080
    healthcheck:
      test: ["CMD", "imgproxy", "health"]
    environment:
      ### See:
      ### https://docs.imgproxy.net/configuration/options

      ### log and debug
      IMGPROXY_LOG_LEVEL: "warn"
      IMGPROXY_ENABLE_DEBUG_HEADERS: "false"
      IMGPROXY_DEVELOPMENT_ERRORS_MODE: "false"
      IMGPROXY_REPORT_DOWNLOADING_ERRORS: "false"

      ### timeouts
      IMGPROXY_READ_TIMEOUT: 10
      IMGPROXY_WRITE_TIMEOUT: 10
      IMGPROXY_DOWNLOAD_TIMEOUT: 10
      IMGPROXY_KEEP_ALIVE_TIMEOUT: 300
      IMGPROXY_MAX_SRC_FILE_SIZE: 33554432 # 32MB
      IMGPROXY_MAX_SRC_RESOLUTION: 48

      ### image source
      IMGPROXY_TTL: 2592000 # client-side cache time is 30 days
      IMGPROXY_USE_ETAG: "false"
      IMGPROXY_SO_REUSEPORT: "true"
      IMGPROXY_IGNORE_SSL_VERIFICATION: "true"
      IMGPROXY_LOCAL_FILESYSTEM_ROOT: /home
      IMGPROXY_SKIP_PROCESSING_FORMATS: "svg,webp,avif"

      ### presets
      IMGPROXY_AUTO_ROTATE: "true"
      #IMGPROXY_WATERMARK_PATH: /home/noimage_thumb.jpg
      IMGPROXY_PRESETS: default=resizing_type:fit/gravity:sm,logo=watermark:0.5:soea:10:10:0.15,center_logo=watermark:0.3:ce:0:0:0.3

      ### compression
      IMGPROXY_STRIP_METADATA: "true"
      IMGPROXY_STRIP_COLOR_PROFILE: "true"
      IMGPROXY_FORMAT_QUALITY: jpeg=80,webp=70,avif=50
      IMGPROXY_JPEG_PROGRESSIVE: "false"
      IMGPROXY_PNG_INTERLACED: "false"
      IMGPROXY_PNG_QUANTIZATION_COLORS: 128
      IMGPROXY_PNG_QUANTIZE: "false"
      IMGPROXY_MAX_ANIMATION_FRAMES: 64
      IMGPROXY_GZIP_COMPRESSION: 0
      IMGPROXY_AVIF_SPEED: 8

      ### For URL signature
      IMGPROXY_KEY: IMGPROXY_KEY_KEY
      IMGPROXY_SALT: IMGPROXY_KEY_SALT
      IMGPROXY_SIGNATURE_SIZE: 32
    network_mode: "host"  
```

You will also need a nginx-configuration file for imgproxy which should be saved to ```/etc/imgproxy/imgproxy-nginx.conf```. Of course you can also store the file anywhere else but be sure to change the volume in the docker-compose.yml.


```nginx
upstream upstream_imgproxy  {
    server    imgproxy:8080;
    keepalive 16;
}

server {
        server_name _;

        location / {
                proxy_pass http://upstream_imgproxy;
                proxy_http_version 1.1;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection 'upgrade';
                proxy_set_header Host $host;
        }

}
```

Now you can  deploy the stack with

```bash
docker-compose up -d --build --remove-orphans --force-recreate
```
or on Portainer. 

Your imgproxy instance should be now running on http://localhost:8080 which you already can use.

![Screenshot 2024-01-12 113947.png](/uploads/Screenshot_2024_01_12_113947_ed6e757944.png)

But I wanted to integrate it within my personal site built with Next.js so I also had to modify the nginx-configuration for my personal site.
So i used the existing configuration [Nginx reverse proxy with caching for Next.js with imgproxy](https://gist.github.com/hlubek/02955a3f28db168417884b5397ce07c0 "") and copied it to /etc/nginx/sites-available/default.

```nginx
# Based on https://steveholgado.com/nginx-for-nextjs/

# - /var/cache/nginx sets a directory to store the cached assets
# - levels=1:2 sets up a two‑level directory hierarchy as file access speed can be reduced when too many files are in a single directory
# - keys_zone=STATIC:10m defines a shared memory zone for cache keys named “STATIC” and with a size limit of 10MB (which should be more than enough unless you have thousands of files)
# - inactive=7d is the time that items will remain cached without being accessed (7 days), after which they will be removed
# - use_temp_path=off tells NGINX to write files directly to the cache directory and avoid unnecessary copying of data to a temporary storage area first
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=STATIC:10m inactive=7d use_temp_path=off;

upstream nextjs_upstream {
  server localhost:3000;
}

upstream imgproxy_upstream {
  server localhost:8080;
}

server {
  listen 80 default_server;

  server_name _;

  server_tokens off;

  gzip on;
  gzip_proxied any;
  gzip_comp_level 4;
  gzip_types text/css application/javascript image/svg+xml;

  proxy_http_version 1.1;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection 'upgrade';
  proxy_set_header Host $host;
  proxy_cache_bypass $http_upgrade;

  # Imgproxy paths can contain multiple slashes (e.g. local:///image/file.jpg)
  merge_slashes off;

  location /img/ {

    proxy_cache STATIC;

    proxy_pass http://imgproxy_upstream/;

    # For testing cache - remove before deploying to production
    add_header X-Cache-Status $upstream_cache_status;
  }

  location /_next/static {
    proxy_cache STATIC;
    proxy_pass http://nextjs_upstream;

    # For testing cache - remove before deploying to production
    add_header X-Cache-Status $upstream_cache_status;
  }

  location /static {
    proxy_cache STATIC;

    # Ignore cache control for Next.js assets from /static, re-validate after 60m
    proxy_ignore_headers Cache-Control;
    proxy_cache_valid 60m;

    proxy_pass http://nextjs_upstream;

    # For testing cache - remove before deploying to production
    add_header X-Cache-Status $upstream_cache_status;
  }

  location / {
    proxy_pass http://nextjs_upstream;
  }
}
```
With this configuration all requests with the path `/img/` will be redirected to the imgproxy instance and all other paths to my personal-website.

You can test the configuration with `sudo nginx -t` and restart nginx when the test is successfull with `sudo systemctl restart nginx`.

Now when you access [https://mxd.codes/img/](https://mxd.codes/img/ "imgproxy instance link") you will be redirected to the imgroxy instance and when you access [https://mxd.codes](https://mxd.codes "personal website link") you will be redirected to my personal website.

The last missing piece is a custom image loader for the Next.js site.

## Custom image loader for imgproxy

You can configure a custom `loaderFile` in your `next.config.js` like the following:

```javascript
images: {
        loader: "custom",
        loaderFile: "./src/utils/loader.js",
}
```

This must point to a file relative to the root of your Next.js application. The file must export a default function that returns a string:

```javascript
export default function imgproxyLoader({ src, width, height, quality }) {

  const path =
    `/size:${width ? width : 0}:${height ? height : 0}` +
    `/resizing_type:fill` +
    (quality ? `/quality:${quality}` : "") +
    `/sharpen:0.5` +
    `/plain/${src}` +
    `@webp`

  const host = process.env.NEXT_PUBLIC_IMGPROXY_URL

  const imgUrl = `${host}/insecure${path}`

  return imgUrl
}
```

Now all images you serve with `next/image` will use your custom loader which will be using imgproxy to transform and optimize your images for your Next.js site.

Recently I also started to deploy my personal site with docker so the whole docke-compose.yml now looks like the following, while the nginx configuration file remains the same:

```yaml
version: "3"
    
services:
  nextjs:
    image: mxdcodes/personal-website:latest
    container_name: personal-website
    restart: always
    ports:
      - "3000:3000"
    environment:
      NODE_ENV: production
    network_mode: "host"  
    
  imgproxy:
    restart: unless-stopped
    image: darthsim/imgproxy:${IMGPROXY_TAG:-latest}
    container_name: imgproxy_app
    security_opt:
      - no-new-privileges:true
    volumes:
      - /data/containers/imgproxy/www:/home:cached
    ports:
      - "8080:8080"
    healthcheck:
      test: ["CMD", "imgproxy", "health"]
    environment:
      ### See:
      ### https://docs.imgproxy.net/configuration/options

      ### options
      IMGPROXY_ALLOWED_SOURCES: https://mxd.codes/
      
      ### log and debug
      IMGPROXY_LOG_LEVEL: "warn"
      IMGPROXY_ENABLE_DEBUG_HEADERS: "false"
      IMGPROXY_DEVELOPMENT_ERRORS_MODE: "false"
      IMGPROXY_REPORT_DOWNLOADING_ERRORS: "false"

      ### timeouts
      IMGPROXY_READ_TIMEOUT: 10
      IMGPROXY_WRITE_TIMEOUT: 10
      IMGPROXY_DOWNLOAD_TIMEOUT: 10
      IMGPROXY_KEEP_ALIVE_TIMEOUT: 300
      IMGPROXY_MAX_SRC_FILE_SIZE: 33554432 # 32MB
      IMGPROXY_MAX_SRC_RESOLUTION: 48

      ### image source
      IMGPROXY_TTL: 2592000 # client-side cache time is 30 days
      IMGPROXY_USE_ETAG: "false"
      IMGPROXY_SO_REUSEPORT: "true"
      IMGPROXY_IGNORE_SSL_VERIFICATION: "false"
      IMGPROXY_LOCAL_FILESYSTEM_ROOT: /home
      IMGPROXY_SKIP_PROCESSING_FORMATS: "svg,webp,avif"

      ### presets
      IMGPROXY_AUTO_ROTATE: "true"
      #IMGPROXY_WATERMARK_PATH: /home/noimage_thumb.jpg
      IMGPROXY_PRESETS: default=resizing_type:fit/gravity:sm,logo=watermark:0.5:soea:10:10:0.15,center_logo=watermark:0.3:ce:0:0:0.3

      ### compression
      IMGPROXY_STRIP_METADATA: "true"
      IMGPROXY_STRIP_COLOR_PROFILE: "true"
      IMGPROXY_FORMAT_QUALITY: jpeg=80,webp=70,avif=50
      IMGPROXY_JPEG_PROGRESSIVE: "false"
      IMGPROXY_PNG_INTERLACED: "false"
      IMGPROXY_PNG_QUANTIZATION_COLORS: 128
      IMGPROXY_PNG_QUANTIZE: "false"
      IMGPROXY_MAX_ANIMATION_FRAMES: 64
      IMGPROXY_GZIP_COMPRESSION: 0
      IMGPROXY_AVIF_SPEED: 8

      ### For URL signature
      IMGPROXY_KEY: KEY
      IMGPROXY_SALT: SALT
      IMGPROXY_SIGNATURE_SIZE: 32
    network_mode: "host"  
```

How to transform and optimize images with imgproxy hosted with docker for your Next.js application.

Optimizing images for Next.js sites with imgproxy and docker

Recently I went out of storage for my homelab so I bought an used NAS (Synology DS214 play) to have some more capacities for [Proxmox](https://www.proxmox.com/de/ "Proxmox") Backups and OpenStreetMap. I still had a 1TB hdd lying around at home, which I now use for proxmox backups. 

To have some redundancy (and to learn something new) I decieded to copy the Proxmox backups to the cloud, in particular to an Azure Storage Account with [AzCopy](https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10) and in the following I will describe with more details how I was able to do it.

Overall this article will cover the following informations:
- Creating an Azure Storage Account
- Getting started with AzCopy
- Creating a bash-script to copy the Proxmox backups to an Azure Storage Container

## Creating an Azure Storage Account

First off all you need an active Azure subscription and an storage account to be able to store your backups. In the [Azure Portal](https://portal.azure.com "Azure Portal") you can search for the service "Storage Accounts" which you will need.

![azure_storage_account_e81efc2682.png](https://cms.mxd.codes/uploads/azure_storage_account_e81efc2682_260acf4298.png)

In the service "Storage Accounts" you can create a new storage account. For the storage account you will need 
- an active azure subscription,
- a ressource group (create one if you don't have one it, e.g. RG-HOMELAB),
- a storage account name,
- selecting a region and 
- selecting redundancy (e.g. LRS;)
- Access tier "cold" (See Advanced)

![storage_account_basics_e82f1d8a4d.png](https://cms.mxd.codes/uploads/storage_account_basics_e82f1d8a4d_6f72c73f59.png)
![storage_account_advanced_02f6609c40.png](https://cms.mxd.codes/uploads/storage_account_advanced_02f6609c40_50a3cf6f7c.png)

You can keep all the other settings as default. After your Storage Account has been deployed you can add a lifecycle rule from "Lifecycle Management" which will move files from the "cold" access tier to the archive storage. 

![storage_lifecycle_management_e113a6efb8.png](https://cms.mxd.codes/uploads/storage_lifecycle_management_e113a6efb8_d83047917d.png)

For example I created a rule which moves all new files after one day to the archive storage tier.

![storage_lifecycle_lifecycle_rule_85a074cae0.png](https://cms.mxd.codes/uploads/storage_lifecycle_lifecycle_rule_85a074cae0_1542a982cc.png)

![storage_lifecycle_lifecycle_rule_2_5486e872ee.png](https://cms.mxd.codes/uploads/storage_lifecycle_lifecycle_rule_2_5486e872ee_fbf78cef24.png)

By storing files in archive storage instead of in the regular "cold" access tier you can actually save about 82%. But keep in mind that accessing data in the Archive storage is more expensive than in the cold (or any other) storage tier.

Also you could create another rule which will for example will delete all all blobs which were created 365 days ago.

![storage_pricing_8998b65a68.png](https://cms.mxd.codes/uploads/storage_pricing_8998b65a68_9635c1cb24.png)

Please have a look at [https://azure.microsoft.com/en-us/pricing/details/storage/blobs/](https://azure.microsoft.com/en-us/pricing/details/storage/blobs/ "Azure Storage Pricing") for uptodate Azure Storage pricing.

After the storage account has been configured you will need to create a Container where the actual files will be stored.
Go to  "Data storage" -> "Containers" and create a Container.
![create storage container.png](/uploads/create_storage_container_8ee34fe39e.png)
Again name it however you want.

Due to the fact that the current version of AzCopy V10 does not support Azure AD authorization in cron jobs I used an SAS token to be able to upload files to the container. You can create a SAS token in the container at "Shared access tokens".

![storage_shared_access_token_1646ebf346.png](https://cms.mxd.codes/uploads/storage_shared_access_token_1646ebf346_8b4b61c9f6.png)

For the Shared access token you will need to select the permissions for Add/Create/Write and select an expiry date for security reasons. Then you can generate the SAS token and URL. Copy that **Blob SAS URL** because you will need it for the upload script.

## Getting started with AzCopy

> AzCopy is a command-line utility that you can use to copy blobs or files to or from a storage account. This article helps you download AzCopy, connect to your storage account, and then transfer data. (https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10)

To get AzCopy for Linux you have to download a tar file and decompress the tar file anywhere you like. You can then just use AzCopy because it's an executable file, so nothing has to be installed.

```bash
#Download AzCopy
cd ~
wget https://aka.ms/downloadazcopy-v10-linux
 
#Expand Archive
tar -xvf downloadazcopy-v10-linux
 
#(Optional) Remove existing AzCopy version
rm /usr/bin/azcopy
 
#Move AzCopy to the destination you want to store it
cp ./azcopy_linux_amd64_*/azcopy /usr/bin/

# Remove Azcopy from home
rm -r downloadazcopy-v10-linux
rm -r azcopy_linux_amd64_10.16.2/

```

By adding the azcopy file location as system path you can just type azcopy from any directory on your system.
You can add it to your system path with:

```bash
nano ~/.profile
```
and then adding these lines:

```bash
export PATH=/usr/bin/azcopy:$PATH
```

Lastly, update your system variables:

```bash
source ~/.profile
```

## Creating a bash-script to synchronize the Proxmox backup directory to an Azure Storage Container

The only piece missing now is the script which will upload the the Proxmox backup files to the previously created azure storage container after the backup task has finished. 
For copying the backups to Azure we will use azcopy **copy** because acopy uses less memory and incurs lower billing costs because a copy operation does not need to index the source or destination before moving files in comparison to azcopy **sync**.
Azcopy also compares file names and last modified timestamp to only upload new or changed files to the storage container, which overall will reduce bandwidth usage and it will work perfectly with the previously created lifecycle rule.

For automatically starting the upload after the backup has finished  we can use a hook script for vzdump. Therefore you need to add the following line to the end of the "/etc/vzdump.conf" file.
```bash
script: /home/youruser/scripts/upload-backups-to-azure.sh
```
Afterwards you can create the script which will upload the files with:
```bash
cd ~
mkdir scripts
cd scripts
nano upload-backups-to-azure.sh
```
Then copy paste the following content into the file and replace the content for src with the location of your dumps. Note that there is "/*" at the end of src so that only the files inside the directory will be copied. Also replace token with the **Blob SAS URL**.

```bash
#!/bin/bash
# Script to upload Proxmox backups to Azure Storage

src="/mnt/pve/xyz/dump/*"
token="Blob SAS URL"

dobackup(){
  echo "Uploading Proxmox backups from $src to Azure..."
  azcopy copy "$src" "$token" --overwrite=false
  echo "Finished Uploading!"
}

if [ "$1" == "job-end" ]; then
  dobackup
fi

exit 0
```

Close the file and make it executable for the user with:

```bash
chmod +x ~/scripts/upload-backups-to-azure.sh
```

Now the next time your backup task has finished the files will be automatically uploaded to your Azure storage container.
Due to the  hook script you can check the status of the copy process in the proxmox ui.

This article will explain to you how you can create an Azure Storage Account, prepare AzCopy locally and how to create a bash script which will be used to copy your local Proxmox Backups via AzCopy to an Azure Storage Account by a cost effective way.

How to copy your Proxmox backups with AzCopy to Azure Storage Containers

With [Gatsby 4](https://www.gatsbyjs.com/gatsby-4/) bringing in Server-Side Rendering (SSR) and Deferred Static Generation (DSG) you need an alternative methode to just hosting static files. Each page using SSR or DSG will be rendererd after a user requests it so there has be a server in the background which will handle these requests and build the pages if needed.

In this post i will show you how you can deploy your Gatsby site with SSR and/or DSG on your own server with a CI/CD pipeline via PM2 and Github Webhooks.

Therefore i will be using

- Gatsby 4,Nginx,
- Nginx
- PM2 and
- Github webhooks.

## Setup your server

First of all you need an server with root access. I strongly recommend to have a look at the guide ["Initial Server Setup with Ubuntu 18.04"](https://www.digitalocean.com/community/tutorials/initial-server-setup-with-ubuntu-18-04) from the DigitalOcean community which will lead you through the process of:

- Logging in and set up root user access to your server with SSH
- Creating a new user
- Granting Administrative Privileges to the new user
- Setting up a basic firewall
- Giving your regular user access to the server with SSH key authentication.

After you have done that you can continue by installing all necessary dependencies on your server.
Install Node.js

Again there is an guide by DigitalOcean which will help you [installing Node.js using PPA](https://www.digitalocean.com/community/tutorials/how-to-install-node-js-on-ubuntu-18-04#installing-using-a-ppa).

After completing

- install Node.js, NPM and
- the "build-essential package"

you will have to change npm's default directory.

Create a `.npm-global` directory and set the path to this directory for `node_modules`:

```bash
cd ~
mkdir ~/.npm-global
npm config set prefix '~/.npm-global'
```

Create (or modify) a `~/.profile` and add the following line:

```bash
sudo nano ~/.profile

# set PATH so global node modules install without permission issues
export PATH=~/.npm-global/bin:$PATH
```

Now you have to update your system variables:

```bash
source ~/.profile
```

Now you should be able to check your installed Node.js version with:

```bash
node -v
```

## Install git

Check if git is already installed with:

```bash
git --version
```

If it isn't installed yet you can install it with

```bash
sudo apt install git
```

and configure Git with

```bash
git config --global user.name "Your Name"
git config --global user.email "youremail@domain.com"
```

After git is installed and configured you can deploy your Gatsby site by cloning it from Github.

## Deploy from Github

It is important that you are loggin in as **non-root** user for the following steps.

```bash
cd ~
git clone https://github.com/your-githubuser/your-gatsby-repo.git path your-gatsby-site
```

After you have deployed your project (optionally with environment variables) you can install all dependencies and build your Gatsby site with:

```bash
cd ./your-gatsby-site/
npm install
npm run build
```

Now you should have a copy of your local project/Gatsby site on your remote server.

Next you are going to setup PM2 which will be used to keep your site alive and restart it with every reboot.

## Setup PM2

You can install PM2 with:

```bash
npm install pm2@latest -g
```

You will need to create/configure an `ecosystem.config.js` file which will restart the default Gatsby server.

```bash
cd ~
pm2 init
sudo nano ecosystem.config.js
```

Copy/paste the template and replace the content.

```javascript
module.exports = {
  apps: [
    {
      name: 'gatsby-site',
      cwd: ' /home/your-name/my-gatsby-site',
      script: 'npm',
      args: 'serve',
      env: {
        //NODE_ENV: 'production',
      },
    },
    // optionally a second project
],};
```
With

```bash
cd ~
pm2 start ecosystem.config.js
```

you can start your server which will run on the Port 9000.

You can always check the status with:

```bash
pm2 status
```

After the server reboots this PM2 should be always automatically be restarted. For that you are going to need a small Startup script which you can also copy/paste.
Generate and configure a startup script to launch PM2:

```bash
cd ~
pm2 startup systemd

[PM2] Init System found: systemd
[PM2] To setup the Startup Script, copy/paste the following command:
**sudo env PATH=$PATH:/usr/bin /usr/lib/node_modules/pm2/bin/pm2 startup systemd -u your-name --hp /home/your-name**
```

- Copy/paste the generated command:

```bash
**sudo env PATH=$PATH:/usr/bin /usr/lib/node_modules/pm2/bin/pm2 startup systemd -u your-name --hp /home/your-name**

[PM2] Init System found: systemd
Platform systemd

. . .


[PM2] [v] Command successfully executed.
+---------------------------------------+
[PM2] Freeze a process list on reboot via:
   $ pm2 save

[PM2] Remove init script via:
   $ pm2 unstartup systemd
```

- And save the new PM2 process list and environments. Then Start the service with systemctl.

```bash
pm2 save

[PM2] Saving current process list...
[PM2] Successfully saved in /home/your-name/.pm2/dump.pm2
```

If you reboot your server now with `sudo reboot` the script should be automatically restart your Gatsby site. Give it a try!

## Setup Github Webhook

One thing missing now is an continuos integration and continuos delivery (CI/CD) pipeline which you will setup using Github webhooks.

Therefore you need to create a new Webhook in your repository.

The following articles provide additional information to the steps below:

- [GitHub Creating Webhooks Guide](https://docs.github.com/en/developers/webhooks-and-events/webhooks/creating-webhooks#setting-up-a-webhook) and
- [DigitalOcean Guide to GitHub WebHooks](https://www.digitalocean.com/community/tutorials/how-to-use-node-js-and-github-webhooks-to-keep-remote-projects-in-sync).

You need to create a server script which will do something if it is triggered by the Github webhook.

```bash
cd ~
mkdir NodeWebHooks
cd NodeWebHooks
sudo nano webhook.js
```

The script is going to create a server running on Port 8100. (Your Github webhook should be of course sending the webhook to something like http://server-ip:8100.)

If it gets triggered by a webhook it will

- go into your repo `~/my-gatsby-site/`,
- pull the latest commits,
- install all dependencies,
- build a new version of the site and
- restart the server via the PM2 script.

```javascript
const secret = "your-secret-key";
const repo = "~/my-gatsby-site/";

const http = require('http');
const crypto = require('crypto');
const exec = require('child_process').exec;

const BUILD_CMD = 'npm run build';
const PM2_CMD = 'pm2 restart gatsby-site';

http.createServer(function (req, res) {
    req.on('data', function(chunk) {
        let sig = "sha1=" + crypto.createHmac('sha1', secret).update(chunk.toString()).digest('hex');

        if (req.headers['x-hub-signature'] == sig) {
            exec('cd ' + repo + ` && git pull && npm install && ${BUILD_CMD} && ${PM2_CMD}`);
        }
    });

    res.end();
}).listen(8100);
```

You will need to allow communication on Port 8100 with:

```bash
sudo ufw allow 8100/tcp
sudo ufw enable
```

Command may disrupt existing ssh connections. Proceed with operation (y|n)? y
Firewall is active and enabled on system startup

Earlier you setup PM2 to restart your Gatsby site whenever the server reboots or is started. You will now do the same for the webhook script.

Run echo $PATH and copy the output for use in the next step.

```bash
echo $PATH

/home/your-name/.npm-global/bin:/home/your-name/bin:/home/your-name/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
```

Create a webhook.service file:

```bash
cd ~
sudo nano /etc/systemd/system/webhook.service
```
    
In the editor, copy/paste the following script, but make sure to replace your-name in two places with your username. Earlier, you ran echo ```$PATH```, copy this to the Environment=PATH= variable, then save and exit:

```bash
[Unit]
Description=Github webhook
After=network.target

[Service]
Environment=PATH=your_path
Type=simple
User=your-name
ExecStart=/usr/bin/nodejs /home/your-name/NodeWebHooks/webhook.js
Restart=on-failure

[Install]
WantedBy=multi-user.target
```
    
Enable and start the new service so it starts when the system boots:

```bash
sudo systemctl enable webhook.service
sudo systemctl start webhook
```
    
Check the status of the webhook:

```bash
sudo systemctl status webhook
```

You can test your webhook with these [instructions](https://www.digitalocean.com/community/tutorials/how-to-use-node-js-and-github-webhooks-to-keep-remote-projects-in-sync#step-4-testing-the-webhook "Test Webhook").

The Gatsby server is now running on your-ip:9000 and you implemented a CI/CD pipeline via PM2 and Github Webhooks but you still can't access your website via a domain because you need to configure a webserver like Nginx.

## Configure Nginx

I am using Cloudflare to manage DNS for my domains but you can do this with every other provider also.

Create two A Records which will point `your-domain.com` and `www.your-domain.com` to the IP-adress of your server.

After that you will need to configure Nginx.

The following instructions are based on [How To Install Nginx on Ubuntu 18.04 [Quickstart]](https://www.digitalocean.com/community/tutorials/how-to-install-nginx-on-ubuntu-18-04-quickstart).

- Update your local package index:

```bash
sudo apt update
```

- install Nginx:

```bash
sudo apt install nginx
```
    
and adjust the Firewall:

```bash
sudo ufw allow 'Nginx Full'
sudo ufw delete allow 'Nginx HTTP'
```

You should now be able to see the Nginx landing page on `http://your_server_ip`.

## Setting up Server Blocks
    
Create the directory for your-domain.com, using the -p flag to create any necessary parent directories:

```bash
sudo mkdir -p /var/www/your-domain.com/html
```
    
Assign ownership of the directory:

```bash
sudo chown -R $USER:$USER /var/www/your-domain.com/html
```
    
The permissions of your web roots should be correct if you haven’t modified your umask value, but you can make sure by typing:

```bash
sudo chmod -R 755 /var/www/example.com
```
    
Make a new server block at /etc/nginx/sites-available/your-domain.com:

```bash
sudo nano /etc/nginx/sites-available/example.com
```
    
Copy/Paste the following Gatsby-nginx configuration and update the server_name sections:

```nginx
server {
    # Listen HTTP
    listen 80;
    listen [::]:80;

    server_name your-domain.com www.your-domain.com;

    # Redirect HTTP to HTTPS
    return 301 https://$host$request_uri;
}

server {
    # Listen HTTP
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name your-domain.com www.your-domain.com;

    # SSL config
    include snippets/self-signed.conf;
    include snippets/ssl-params.conf;

    # Proxy Config
    location / {
        proxy_pass http://localhost:9000
        proxy_http_version 1.1;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Server $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Host $http_host;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_pass_request_headers on;
    }
    location ~ /.well-known {
       allow all;
    }
}
```

Save the file and close it when you are finished.

Enable the file by creating a link from it to the sites-enabled directory:

```bash
sudo ln -s /etc/nginx/sites-available/your-domain.com /etc/nginx/sites-enabled/
```
    
Test for syntax errors:

```bash
sudo nginx -t
```
    
and finally enable the changes:

```bash
sudo systemctl restart nginx
```

Nginx should now be serving your gatsby site on your domain name. That means if you have a look at `http://your-domain.com` you should see your Gatsby site.

In the end should deny traffic to Port 9000 because Nginx is handling the requests with:

```bash
cd ~
sudo ufw deny 9000
```

To install SSL, you will need to [install and run Certbot by Let's Encrypt](https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-18-04).



With Gatsby 4 bringing in Server-Side Rendering (SSR) and Deferred Static Generation (DSG) you need an alternative methode to just hosting static files. Each page using SSR or DSG will be rendererd after a user requests it so there has be a server in the background which will handle these requests and build the pages if needed.

How to deploy your GatsbyJS site on your own server

Inspired by Aaron Parecki who has been [tracking his location since 2008](https://aaronparecki.com/gps/ "location tracking") with an iPhone app ([Overland](https://overland.p3k.app/ "Overland")) and a server side tracking API ([Compass](https://github.com/aaronpk/Compass "Compass")), I decided to go for a similar approach.

I wanted to track my position constantly with my Android smartphone, save the data to a Postgres database and use the data to display an map with all locations i have ever been to.

To be able to track my position i needed an stable android app and after some tinkering around with different tracking apps I found [OwnTracks](https://owntracks.org/ "OwnTracks")  which basically has all i need (and a good [documentation](https://owntracks.org/booklet/ "OwnTracks Documentation") of it).

In this article, we will explore a step-by-step guide on how to set up a location tracking system using OwnTracks for mobile devices, Node.js for a webhook server, PostgreSQL for data storage, and GeoServer, MapProxy, and Nginx for visualizing the location data.

But first of all we need a database to be able to actually save the data.

## Set up PostgreSQL 

To install [PostgrSQL 16](https://www.postgresql.org/about/news/postgresql-16-released-2715/ "PostgreSQL 16") you will need to the repository that provides the packages.
First, update the package index and install required packages:

```bash
sudo apt update
sudo apt install gnupg2 wget vim
```

The PostgreSQL repository can then be added using the command:

```bash
sudo sh -c 'echo "deb https://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
```

Import the repository signing key:
```bash
curl -fsSL https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/postgresql.gpg
```
And update your package list:

```bash
sudo apt update
```

Afterwards you can install the latest PostgreSQL (16) version with:
```bash
sudo apt-get -y install postgresql postgresql-contrib
```

Once installed, start and enable the service:
```bash
sudo systemctl start postgresql
sudo systemctl enable postgresql
```

Now you will need a user and database for your location data.

Connect to the PostgreSQL database with:
```bash
sudo su postgres
psql
```

and create a database with:
```sql
CREATE DATABASE locations;
```

aswell a user with:
```sql
CREATE USER USER <username> with encrypted password '<password>';
```

Then you can grant privileges for your user to the database:
```bash
GRANT ALL PRIVILEGES ON DATABASE locations TO <username>;
```

After you set up your database and user you also will need a table to store the data.
The table where I save my locations can be recreated with: 

```sql
CREATE TABLE public.locations (
	id bigserial NOT NULL,
	created_at timestamptz NULL DEFAULT CURRENT_TIMESTAMP,
	lat float8 NULL,
	lon float8 NULL,
	acc int4 NULL,
	alt int4 NULL,
	batt int4 NULL,
	bs int4 NULL,
	cog numeric(10, 2) NULL,
	rad int4 NULL,
	t varchar(255) NULL,
	tid varchar(255) NULL,
	tst int4 NULL,
	vac int4 NULL,
	vel int4 NULL,
	p numeric(10, 2) NULL,
	conn varchar(255) NULL,
	topic varchar(255) NULL,
	inregions jsonb NULL,
	ssid varchar(255) NULL,
	bssid varchar(255) NULL,
);
```
The most important attributes are **lat** (latitude), **lon** (longitude) and **alt** (altitude) which are used to render points later on.

Also I am saving the velocity, battery level of the smartphone and some more data which will be displayed at my [/now](/now "/now") page. The velocity and altitude are quite inaccurate though which probably also depends on the smartphone.

## Create a Node.js Webhook server

You can create a node.js server to act as a webhook to receive location data from OwnTracks. With the [pg](https://www.npmjs.com/package/pg "pg") npm package you can interact with the PostgreSQL database.

```javascript
const http = require("http");
const axios = require('axios');
const { Pool} = require('pg')

const pool = new Pool({
  user: 'username',
  database: 'database',
  password: 'pw!',
  port: port,
  host: 'localhost',
})

async function insertData(body) {
  try {
    const res = await pool.query(
      "INSERT INTO locations (lat, lon, acc, alt, batt, bs, tst, vac, vel, conn, topic, inregions, ssid, bssid) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14)",
      [body.lat, body.lon, body.acc, body.alt, body.batt, body.bs, body.tst, body.vac, body.vel, body.conn, body.topic, body.inregions, body.ssid, body.bssid]
    );
    console.log(`Added location ${body.lat}, ${body.lon}`);
  } catch (error) {
    console.error(error)
  }
}

const server = http.createServer((request, response) => {
  let body = [];
  console.log(request.method)
  if (request.method === "POST") {
     request.on('data', (chunk) => {
      body.push(chunk);
      }).on('end', () => {
        body = JSON.parse(Buffer.concat(body).toString());
        insertData(body)
      })
  }

  response.end();
});

server.listen(9001);
```

So basically the webhook parses the request body and inserts the data via an insert statement into the postgres table. Also the server is listening on port 9001 so be sure you opened that port on your sever. The Url for Owntracks now could be **http://yourserverip:9001**.

## Setting up OwnTracks

The OwnTracks app runs in the background on your Android or iOS device and waits for the smart phone to tell it that the device has moved, whereupon OwnTracks sends out a message with its current coordinates (and a few other things we'll discuss in a moment)

With different settings for the app I could tailor the app exactly to my needs because I wanted the app to check my position every second and when i have moved more than two meters from my last known position it should send a POST-Request to my Strapi instance which will store the location data in a Postgres database.

In the following screenshot you can see the settings I am using for OwnTracks.

![content_Screenshot_Own_Tracks_settings_c7d95a33be.jpg](https://cms.mxd.codes/uploads/content_Screenshot_Own_Tracks_settings_c7d95a33be_4b3fcffdf5.jpg)

The most important seetings for me are the mode 
- **Significant location change mode**,
- **locatorDisplacement**,
- **locatorInterval**,
- **ignoreInaccurateLocations** and
- **url** for the POST-request.

The **significant location change mode** is a mode which will let you decide in which interval you want to track your position. I am using the **locationInterval** '1' which will track my position every second with an **LocatorDisplacement** of 1 which will check if i moved away a meter away from my last known position. If i did it will send an POST-request with the location data to the given url if the position data isn't more inaccurate than 10 meters.

Because the app has to run always in the background you probably have to switch off energy saving modes for your smartphone which might disable the background functionality. 

Be warned: You probably will have to charge your smartphone more often. Also sometimes the app has some tracking error if you are using parallel some other tracking app for example for activities like strava.

The url **http://yourserverip:9001** for the POST-request points to the nodejs server webhook which will run a insert command for the postgres database. 

Now the data will be stored in the postgres database but it still needs to be visualized somehow and therefore it needs a geometry attribute. 

## Creating a PostGIS extension

To be able to create a geometry you will need the [PostGIS](https://postgis.net/ "PostGIS") extension. You can extend your PostgreSQL database with 

```bash
sudo apt-get install postgis
sudo su postgres
psql
\c locations
CREATE EXTENSION postgis;
\q
```

Then create a view which creates a geometry based on the lat and lon attribute from the existing table. 

```sql
CREATE OR REPLACE VIEW public.locations_geom
AS SELECT id,
    lat,
    lon,
    alt,
    vel,
    st_setsrid(st_makepoint(lon, lat, alt::double precision), 4326) AS geom
   FROM locations;
```

At the moment I wrote/edited this post I have about two million datasets in the table, so fetching and rendering the data client-side would not be the best idea. This is where Geoserver takes over.

## Install and configure Geoserver

[GeoServer](https://geoserver.org/ "GeoServer") is an open source server for sharing geospatial data.

To setup GeoServer I created a stack in portainer:

```docker
version: '3'

networks:
  default:
    external: true
    name: nginx
    
services:
  geoserver:
    image: kartoza/geoserver:latest
    container_name: geoserver
    restart: always
    volumes:
      - /data/containers/geoserver:/opt/geoserver/data_dir
    ports:
      - "8080:8080"
    environment:
      GEOSERVER_DATA_DIR: ${GEOSERVER_DATA_DIR}
      GEOSERVER_ADMIN_PASSWORD: ${GEOSERVER_ADMIN_PASSWORD}
      GEOSERVER_ADMIN_USER: ${GEOSERVER_ADMIN_USER}
      INITIAL_MEMORY: ${INITIAL_MEMORY}
      MAXIMUM_MEMORY: ${MAXIMUM_MEMORY}
      DB_BACKEND: POSTGRES               
      HOST: ${POSTGRES_HOST}                         
      POSTGRES_PORT: 5432                
      POSTGRES_DB: gwc                
      POSTGRES_USER: ${POSTGRES_USER}    
      POSTGRES_PASS: ${POSTGRES_PASS}               
      POSTGRES_SCHEMA: public           
      DISK_QUOTA_SIZE: 20        
```

After deploying GeoServer you can login with the default user "admin" and password "geoserver" or the credentials you assigned with environment variables.

To add a new datasource you have to click on **Stores** and **Add new store**. Then you can select PostGIS (or whatever datasource you want to use).

![Screenshot_2024_01_31_162720_b6e7885b51.png](https://cms.mxd.codes/uploads/Screenshot_2024_01_31_162720_b6e7885b51_737a7d28df.png)

Fill out host, port, database, schema, user and passwd and save your connection.
Afterwards you can select layers which you want to publish.

![Screenshot_2024_01_31_162805_b0ba171177.png](https://cms.mxd.codes/uploads/Screenshot_2024_01_31_162805_b0ba171177_75e18fdc67.png)

After you published your layer you can preview the layer at **Layer Preview**.

Now I am not using Geoserver as endpoint to serve the tiles because I have a Mapproxy instance running which will take care of transforming tiles from internal or external ressources.

## MapProxy

Setting up MapProxy is optional but in this post I will use the endpoints from MapProxy to render a web map. So if you decied to skip MapProxy you will have to adjust the wms URL's to your endpoint from GeoServer.

[MapProxy](https://mapproxy.org/ "MapProxy") is an open source proxy for geospatial data. It caches, accelerates and transforms data from existing map services and serves any desktop or web GIS client.
I have decided create a own docker image [mapproxy-docker](https://github.com/dietrichmax/mapproxy-docker " dietrichmax/mapproxy-docker") which you can setup use in a portainer stack with:

```yaml
version: '3.9'

networks:
  default:
    external: true
    name: nginx
    
services:
  mapproxy:
    image: mxdcodes/mapproxy-docker:latest
    container_name: mapproxy
    restart: always
    ports:
      - 3050:80
    volumes:
      - /mapproxy/config:/persistentConfigFolder
      - /mapproxy/cache_data:/persistentCacheFolder
```

The only thing MapProxy needs one or multiple configuration files in yaml format for each service. So set up a locations.yaml under ```/persistentConfigFolder/locations.yaml```:

```yaml
services:
  demo:
  tms:
    use_grid_names: true
    # origin for /tiles service
    origin: 'nw'
  kml:
      use_grid_names: true
  wmts:
  wms:
    md:
      title: Location Data
      abstract: For mxd.codes.

layers:
  - name: locations
    title: Locations - mxd.codes
    sources: [locations_cache]

caches:
  locations_cache:
    grids: [webmercator]
    sources: [locations_wms]

sources:
  locations_wms:
    type: wms
    req:
      url: http://geoserver:8080/geoserver/mxdcodes/wms?
      layers: locations
      transparent: true

grids:
    webmercator:
        base: GLOBAL_WEBMERCATOR

globals:
  # cache options
  cache:
    # where to store the cached images
    base_dir: '/cache_data'
    # where to store lockfiles for concurrent_requests
    lock_dir: '/cache_data/locks'
    # where to store lockfiles for tile creation
    tile_lock_dir: '/cache_data/tile_locks'
    # request x*y tiles in one step
    meta_size: [4, 4]
  # image/transformation options
  image:
    # use best resampling for vector data
    resampling_method: bilinear # nearest/bicubic/bilinear
    formats:
      image/jpeg:
        encoding_options:
          # jpeg quality [0-100]
          jpeg_quality: 80
```

Be sure to replace the sources url with the url from your GeoServer instance.

Now you should have a endpoint for your wms tiles under **http://serverurl:3050/locations/service?** which you can use to create web map on your website.

I also created each service under a seperate subdomain and setup https for it with Cloudflare but I won't go into detail here because you will find hundreds of tutorials online.

## Creating a Web Map with Leaflet and React

The end product of all this should be a simple map on my [personal website](https://mxd.codes "mxd.codes") under [/map](/map "map"). The map component looks like the following:

```javascript
import React, { useState, useRef, useEffect } from "react"
import "ol/ol.css"
import Map from "ol/Map"
import TileLayer from "ol/layer/Tile"
import View from "ol/View"
import XYZ from "ol/source/XYZ"
import { transform, get as getProjection } from "ol/proj"
import WMTSTileGrid from "ol/tilegrid/WMTS.js"
import WMTS from "ol/source/WMTS.js"
import TileWMS from "ol/source/TileWMS.js"
import OSM from "ol/source/OSM.js"
import { getTopLeft, getWidth } from "ol/extent.js"

function LiveMap({data}) {
  const [map, setMap] = useState()
  const mapElement = useRef()
  const mapRef = useRef()
  mapRef.current = map

  const projection = getProjection("EPSG:3857")
  const projectionExtent = projection.getExtent()
  const size = getWidth(projectionExtent) / 256
  const resolutions = new Array(19)
  const matrixIds = new Array(19)
  for (let z = 0; z < 19; ++z) {
    // generate resolutions and matrixIds arrays for this WMTS
    resolutions[z] = size / Math.pow(2, z)
    matrixIds[z] = z
  }

  const locData = new TileLayer({
    opacity: 0.9,
    preload: Infinity,
    source: new WMTS({
      attributions: '&copy; <a href="https://mxd.codes/">Max Dietrich</a>',
      url: "tilelayerurl",
      layer: "locations",
      matrixSet: "webmercator",
      format: "image/png",
      projection: projection,
      tileGrid: new WMTSTileGrid({
        origin: getTopLeft(projectionExtent),
        resolutions: resolutions,
        matrixIds: matrixIds,
      }),
      style: "default",
      wrapX: true,
    }),
  })

  const aerial = new TileLayer({
    opacity: 0.9,
    preload: Infinity,
    source: new WMTS({
      attributions:
        "Tiles &copy; Esri &mdash; Source: Esri, i-cubed, USDA, USGS, AEX, GeoEye, Getmapping, Aerogrid, IGN, IGP, UPR-EGP, and the GIS User Community",
      url: "tilelayerurl",
      layer: "arcgisaerial",
      matrixSet: "webmercator",
      format: "image/png",
      projection: projection,
      tileGrid: new WMTSTileGrid({
        origin: getTopLeft(projectionExtent),
        resolutions: resolutions,
        matrixIds: matrixIds,
      }),
      style: "default",
      wrapX: true,
    }),
  })

  const center = transform([data.lon, data.lat], "EPSG:4326", "EPSG:3857")

  useEffect(() => {
    const initialMap = new Map({
      target: mapElement.current,
      layers: [aerial, locData],
      view: new View({
        center: center,
        zoom: 16,
        maxZoom: 16,
      }),
    })
    setMap(initialMap)
  }, [])

  return (
    <div
      style={{ height: "100%", width: "100%" }}
      ref={mapElement}
      className="map-container"
    />
  )
}

export default LiveMap
```


Inspired by Aaron Parecki and who he has been tracking his location since 2008 with an iPhone app and a server side tracking API i decided to go for a similar approach.

I wanted to track my position constantly with my Android smartphone and use the data to display a map with all locations i have ever been to.

A Guide to Location Tracking and Visualization with OwnTracks, Node.js, PostgreSQL, GeoServer, MapProxy, Nginx and OpenLayers

This article shows you how can host your Next.js site on a (virtual private) server with Nginx and a CI/CD pipeline via PM2 and Github Webhooks.

## Setup your server

First of all you need an server with root access.
I strongly recommend to have a look at the guide ["Initial Server Setup with Ubuntu 18.04"](https://www.digitalocean.com/community/tutorials/initial-server-setup-with-ubuntu-18-04 "Initial Server Setup with Ubuntu 18.04") from the DigitalOcean community which will lead you through the process of:

1. Logging in and set up root user access to your server with SSH
2. Creating a new user
3. Granting Administrative Privileges to the new user
4. Setting up a basic firewall
5. Giving your regular user access to the server with SSH key authentication.

After you have done that you can continue by installing all necessary dependencies on your server.

### Install Node.js

Again there is an guide by DigitalOcean which will help you [installing Node.js using PPA](https://www.digitalocean.com/community/tutorials/how-to-install-node-js-on-ubuntu-18-04#installing-using-a-ppa "Installing Node.jsUsing a PPA"). 

After completing 
+ **install Node.js, NPM and**
+ **the "build-essential package"**

you will have to change npm's default directory.

+ Create a `.npm-global` directory and set the path to this directory for `node_modules`:

```bash
cd ~
mkdir ~/.npm-global
npm config set prefix '~/.npm-global'
```

+ Create (or modify) a `~/.profile` and add the following line:
```bash
sudo nano ~/.profile
```

```bash
# set PATH so global node modules install without permission issues
export PATH=~/.npm-global/bin:$PATH
```
Now you have to update your system variables:

```bash
source ~/.profile
```

Now you should be able to check your installed Node.js version with:
 ```bash
node -v
```

### Install git

Check if git is already installed with:
```bash
git --version
```

If it isn't installed yet you can install it with

```bash
sudo apt install git
```

and configure Git with 

```bash
git config --global user.name "Your Name"
git config --global user.email "youremail@domain.com"
```
After git is installed and configured you can deploy your project by cloning it from Github.

## Deploy from Github

It is important that you are loggin in as **non-root** user for the following steps.

```bash
cd ~
git clone https://github.com/your-name/your-project-repo.git path
```

Create a `.env`  on the server if you are using one locally and copy/paste your content.

After you have deployed your project (optionally with environment variables) you can install all dependencies and build your Next.js site with:
```bash
cd ./my-project/
npm install
NODE_ENV=production npm run build
```

Now you should have a copy of your local project/Next.js site on your remote server.

Next you are going to setup PM2 which will be used to keep your site alive and restart it after every reboot.

## Setup PM2

You can install PM2 with:
```bash
npm install pm2@latest -g
```

You will need to create/configure an `ecosystem.config.js` file which will restart the default Next.js server.
```bash
cd ~
pm2 init
sudo nano ecosystem.config.js
```

Copy/paste the template and replace the content.
```javascript
module.exports = {
  apps: [
    {
      name: 'next-site',
      cwd: ' /home/your-name/my-nextjs-project',
      script: 'npm',
      args: 'start',
      env: {
        NEXT_PUBLIC_...: 'NEXT_PUBLIC_...',
      },
    },
    // optionally a second project
],};
```

With 

```bash
cd ~
pm2 start ecosystem.config.js
```

you can start your server which will run on the `Port 1337`.

 You can always check the status with:

```bash
pm2 status next-site
```

After the server reboots this PM2 should be always automatically be restarted. For that you are going to need a small Startup script which you can also copy/paste.

+ Generate and configure a startup script to launch PM2:

```bash
cd ~
pm2 startup systemd

[PM2] Init System found: systemd
[PM2] To setup the Startup Script, copy/paste the following command:
**sudo env PATH=$PATH:/usr/bin /usr/lib/node_modules/pm2/bin/pm2 startup systemd -u your-name --hp /home/your-name**
```

+ Copy/paste the generated command:

```bash
**sudo env PATH=$PATH:/usr/bin /usr/lib/node_modules/pm2/bin/pm2 startup systemd -u your-name --hp /home/your-name**

[PM2] Init System found: systemd
Platform systemd

. . .


[PM2] [v] Command successfully executed.
+---------------------------------------+
[PM2] Freeze a process list on reboot via:
   $ pm2 save

[PM2] Remove init script via:
   $ pm2 unstartup systemd
```

+ And save the new PM2 process list and environments. Then Start the service with `systemctl`.

```bash
pm2 save

[PM2] Saving current process list...
[PM2] Successfully saved in /home/your-name/.pm2/dump.pm2
```

If you reboot your server now with `sudo reboot` the script should be automatically restart your Next.js site. Give it a try!

## Setup Github Webhook

One thing missing now is an continuos integration and continuos delivery (CI/CD) pipeline which you will setup using Github webhooks. 

Therefore you need to create a new Webhook in your repository.

The following articles provide additional information to the steps below: 
+ [GitHub Creating Webhooks Guide](https://docs.github.com/en/developers/webhooks-and-events/webhooks/creating-webhooks#setting-up-a-webhook "GitHub Creating Webhooks Guide") and 
+ [DigitalOcean Guide to GitHub WebHooks](https://www.digitalocean.com/community/tutorials/how-to-use-node-js-and-github-webhooks-to-keep-remote-projects-in-sync "DigitalOcean Guide to GitHub WebHooks"). 

You need to create a server script which will do something if it is triggered by the Github webhook.

```bash
cd ~
mkdir NodeWebHooks
cd NodeWebHooks
sudo nano webhook.js
```

The script is going to create a server running on `Port 8100`. 
(Your Github webhook should be of course sending the webhook to something like `http://server-ip:8100`.)

If it gets triggered by a webhook it will 
+ go into your repo `~/my-nextjs-project/`, 
+ pull the latest commits, 
+ install all dependencies, 
+ build a new version of the site and 
+ restart the server via the PM2 script.

```javascript
const secret = "your-secret-key";
const repo = "~/my-nextjs-project/";

const http = require('http');
const crypto = require('crypto');
const exec = require('child_process').exec;

const BUILD_CMD = 'npm install && NODE_ENV=production npm run build';
const PM2_CMD = 'pm2 restart next-site';

http.createServer(function (req, res) {
    req.on('data', function(chunk) {
        let sig = "sha1=" + crypto.createHmac('sha1', secret).update(chunk.toString()).digest('hex');

        if (req.headers['x-hub-signature'] == sig) {
            exec('cd ' + repo + ` && git pull && npm install && ${BUILD_CMD} && ${PM2_CMD}`);
        }
    });

    res.end();
}).listen(8100);
```

You will need to allow communication on `Port 8100` with:

```bash
sudo ufw allow 8100/tcp
sudo ufw enable

Command may disrupt existing ssh connections. Proceed with operation (y|n)? y
Firewall is active and enabled on system startup
```

Earlier you setup PM2 to restart the services (your Next.js site) whenever the server reboots or is started. You will now do the same for the webhook script.

- Run echo $PATH and copy the output for use in the next step.

```bash
echo $PATH

/home/your-name/.npm-global/bin:/home/your-name/bin:/home/your-name/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
```

+ Create a webhook.service file:

```bash
cd ~
sudo nano /etc/systemd/system/webhook.service
```

+ In the editor, copy/paste the following script, but make sure to replace your-name in two places with your username. Earlier, you ran echo $PATH, copy this to the Environment=PATH= variable, then save and exit:

```bash
[Unit]
Description=Github webhook
After=network.target

[Service]
Environment=PATH=your_path
Type=simple
User=your-name
ExecStart=/usr/bin/nodejs /home/your-name/NodeWebHooks/webhook.js
Restart=on-failure

[Install]
WantedBy=multi-user.target
```

+ Enable and start the new service so it starts when the system boots:

```bash
sudo systemctl enable webhook.service
sudo systemctl start webhook
```

+ Check the status of the webhook:

```bash
sudo systemctl status webhook
```

You can test your webhook with these [instructions](https://www.digitalocean.com/community/tutorials/how-to-use-node-js-and-github-webhooks-to-keep-remote-projects-in-sync#step-4-testing-the-webhook "Test Webhook").

The Next.js server is now running on *your-ip:3000* and you implemented a CI/CD pipeline via PM2 and Github Webhooks but you still can't access your website via a domain because you need to configure a webserver like Nginx.

## Configure Nginx

I am using Cloudflare to manage DNS for my domains but you can do this with every other provider also.

+ Create two `A` Records which will point `your-domain.com` and `www.your-domain.com` to the IP-adress of your server.

After that you will need to configure Nginx. 

_The following instructions are based on [How To Install Nginx on Ubuntu 18.04 [Quickstart]](https://www.digitalocean.com/community/tutorials/how-to-install-nginx-on-ubuntu-18-04-quickstart "How To Install Nginx on Ubuntu 18.04 [Quickstart]")._

+ Update your local package index:

```bash
sudo apt update
```

+ install Nginx:

```bash
sudo apt install nginx
```

+ and adjust the Firewall:

```bash
sudo ufw allow 'Nginx Full'
sudo ufw delete allow 'Nginx HTTP'
```

You should now be able to see the Nginx landing page on `http://your_server_ip`.

### Setting up Server Blocks

+ Create the directory for ```your-domain.com```, using the -p flag to create any necessary parent directories:

```bash
sudo mkdir -p /var/www/your-domain.com/html
```
+ Assign ownership of the directory:

```bash
sudo chown -R $USER:$USER /var/www/your-domain.com/html
```

+ The permissions of your web roots should be correct if you haven’t modified your `umask` value, but you can make sure by typing:

```bash
sudo chmod -R 755 /var/www/example.com
```

+ Make a new server block at `/etc/nginx/sites-available/your-domain.com`: 

```bash
sudo nano /etc/nginx/sites-available/example.com
```

+ Copy/Paste the following nextjs-nginx configuration and update the server_name sections:

```nginx
server {
    # Listen HTTP
    listen 80;
    listen [::]:80;

    server_name your-domain.com www.your-domain.com;

    # Redirect HTTP to HTTPS
    return 301 https://$host$request_uri;
}

server {
    # Listen HTTP
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name your-domain.com www.your-domain.com;

    # SSL config
    include snippets/self-signed.conf;
    include snippets/ssl-params.conf;

    # Proxy Config
    location / {
        proxy_pass http://localhost:3000
        proxy_http_version 1.1;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Server $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Host $http_host;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_pass_request_headers on;
    }
    location ~ /.well-known {
       allow all;
    }
}
```

Save the file and close it when you are finished.

+ Enable the file by creating a link from it to the sites-enabled directory:

```bash
sudo ln -s /etc/nginx/sites-available/your-domain.com /etc/nginx/sites-enabled/
```

+ Test for syntax errors:

```bash
sudo nginx -t
```

+ and finally enable the changes:
```bash
sudo systemctl restart nginx
```

Nginx should now be serving content on your domain name. That means if you have a look at `http://your-domain.com` you should see your Next.js site.


In the end should deny traffic to `Port 3000` with: 

```bash
cd ~
sudo ufw deny 3000
```

+ To install SSL, you will need to [install and run Certbot by Let's Encrypt](https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-18-04 "install and run Certbot by Let's Encrypt") .


_This guide is also using parts of [Strapi Deployment on DigitalOcean](https://strapi.gitee.io/documentation/3.0.0-beta.x/deployment/digitalocean.html "Strapi Deployment on DigitalOcean") which helped me a lot setting up Strapi and Next.js on a server in a proper way._ 

This article shows you how can host your Next.js site on a (virtual private) server with Nginx, a CI/CD pipeline via PM2 and Github Webhooks.

Hosting NextJS on a private server using PM2 and Github webhooks as CI/CD

With [AWS](https://aws.amazon.com/ "AWS") (and in particular the [free AWS contingent](https://aws.amazon.com/en/free/ "free AWS account")) you have the option of a static website with a custom domain for a few Hosting cents a month including CDN via CloudFront and CI / CD integration.

Before I switched completely to AWS, I had a common shared hosting option that cost me around € 72 a year. With this option I had
+ 250 GB SSD storage space,
+ free unlimited traffic,
+ 6 domains included,
+ 250 GB SSD mail storage space,
+ 25 MySQL databases,
+ and, and, and.

On the whole, much more than I need to run my static [GatsbyJS](https://www.gatsbyjs.com/) website.

So why shouldn't I only use and pay for resources that I ultimately need and also get some cloud computing experience?

## Creating S3-Buckets 

The basis for hosting on AWS is formed by [S3 Buckets](https://s3.console.aws.amazon.com/s3/home "S3 Buckets"). 
Buckets are "containers" on the web where you can save files.
In order for redirects from subdomains such as **www.mxd.codes** to **mxd.codes** to work, you need a bucket for each domain.

First of all, you create an S3 bucket for the root domain. In my case, the bucket name is the domain name mxd.codes and you select a region (for example EU (Frankfurt)).
The default settings can be kept under Options, unless you want to save different versions or access protocols.
So that everyone can access the website content later, remove the tick that is present by default at "Block *any* public access", check the bucket settings again and finally create it.

In each bucket you can or should enter a bucket policy that further defines access.
To do this, click on the name of the bucket and go to "Permissions" -> "Bucket Policy".

The following guideline must then be saved for public access.
```json
{
    "Version": "2008-10-17",
    "Statement": [
        {
            "Sid": "AllowPublicRead",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::mxd.codes/*"
        }
    ]
}
```
**"mxd.codes" has to be replace with your bucket name!**

If everything was done correctly, the permissions should now look something like this:

![bucket_richtline_d32ff17da6.png](https://cms.mxd.codes/uploads/bucket_richtline_d32ff17da6_173be5d676.png)

In the bucket settings you have to activate the "hosting a static website" and specify an index document and an error document.
For GatsbyJS that would be **index.html** and **404.html**.

![bucket_hosting_6f40697460.png](https://cms.mxd.codes/uploads/bucket_hosting_6f40697460_500bdd971a.png)

Now the S3 bucket for the subdomain **www.mxd.codes** is still missing.
So create a new bucket with the name of the subdomain **www.mxd.codes** with public access and add the bucket policy.

In the settings for "hosting a static website" you use "redirect requests" and enter the target bucket **mxd.codes** and you can enter **https** as a protocol, because later on the content of the static website is delivered via CloudFront, which can be encrypted with SSL certificates.

![bucket_redirect_740544f6d5.png](https://cms.mxd.codes/uploads/bucket_redirect_740544f6d5_5b708ee0ba.png)

The buckets are now created and correctly configured for the operation of a static website including redirect.

## CloudFront Distributions

With the free AWS contingent, 50 GB of data transfer can be burned per month in Cloudflare.

With a page size of generous 4 MB, this is enough for 12,500 page views per month and should therefore be more than sufficient for a website with average traffic. So why not take a free CDN with you?

If the costs incurred after the free year should deter someone, you still have the option to switch to another CDN provider such as [CloudFlare](https://www.cloudflare.com "CloudFlare").

At [CloudFront](https://console.aws.amazon.com/cloudfront/ "CloudFront") you have to create a web distribution for **each** bucket.
As the origin domain name, a bucket can **not** be selected from the dropdown list, but the end point of the bucket from S3 must be copied.

![cloudfront_origin_f1e2731789.png](https://cms.mxd.codes/uploads/cloudfront_origin_f1e2731789_3f2eb47d64.png)

In the distribution for the bucket **mxd.codes**, for example "mxd.codes.s3-website.eu-central-1.amazonaws.com" is specified as origin.
"Origin ID" will then be filled in automatically. In "Viewer Protocol Policy" **Redirect HTTP to HTTPS** is selected because users should only be able to access the website via HTTPS.  "Compress Objects Automatically" **Yes** can be selected, so that CloudFront will zip all files automatically.
Under "Alternate Domain Names (CNAMES)" you have to specify the bucket for the root domain for the distribution for the root domain. For example mxd.codes

At "SSL-Certifacte" you can now create for the two domains **mxd.codes** and **www.mxd.codes** two free Amazon SSL certificate via the [Certificate Manager (ACM )](https:/https://aws.amazon.com.amazon.com/en/certificate-manager/ "Certificate Manager (ACM)"). To do this, add your two domains in ACM.

![acm_domain_ee3a548091.png](https://cms.mxd.codes/uploads/acm_domain_ee3a548091_38bd644d47.png)

You can now have this validated using a DNS or email method. If you have included your domain in Route 53, you can do it more or less automatically by simply following the instructions.

Back in the CloudFront Distribution Creation you only have to specify a "Default Root Object". -> index.html
If you don't do this, CloudFront always shows an "Access Denied" message in XML format when you access your domain (atleast that was my case).

Finally, the distribution must of course still be activated under "Distribution State".

First distribution finished. The same procedure now for the subdomain **www.mxd.codes** with the corresponding "Origin Domain Name" (**Bucket end point!**)

_This can take up to 20 minutes_
(If you want to clear your CloudFront cache and have already installed AWS CLI, you can do this with the following command:

```bash
aws cloudfront create-invalidation --distribution-id DEINE_DISTRIBUTION ID --paths "/*"
```

In the meantime, you can create the redirects for Cloudfront in Route 53.

## Route 53 DNS settings

In [Route 53]("Route 53") you need a hosted zone (= 0.50€ per month). Then A (and provided that in the CLoudFrontDistribution IPv6 is
activated (which it is by default), as well as an AAAA) data record can be created.

That means you basically need 4 "alias" data sets:
+ mxd.codes A "CloudFront URL for root domain"
+ mxd.codes AAAA "CloudFront URL for root domain"
+ www.mxd.codes A "CloudFront URL for subdomain"
+ www.mxd.codes AAAA "CloudFront URL for subdomain"

Exceptionally, you can select the CDN url from the dropdown list.
With "Routing Guideline" and "Assess the state of the target" you can leave the default settings, unless you want to experiment.

Now if you wait a little you should be redirect from
+ http://wwww.mxd.codes,
+ https://wwww.mxd.codes,
+ wwww.mxd.codes,
+ http://mxd.codes and
+ mxd.codes
to **https://mxd.codes** .


With AWS (and in particular the free AWS contingent) you have the option of a static website with a custom domain for a few Hosting cents a month including CDN via CloudFront and CI/CD integration.

Selfhosted