Since I store blog posts in a self-hosted version of [strapi](https://strapi.io/ "strapi"), I've been looking for a way to automatically generate a table of contents from Markdown for all posts in my Next.js site.

The idea is that during the build process all captions are extracted from the article content (I use [getStaticProps](https://nextjs.org/docs/basic-features/data-fetching/get-static-props) for all articles) and then display them fixed next to the content using a separate component.

## Extracting headers with regex from markdown

After some research and trial and error I decided to use regex to extract the headers from the markdown text using the hash symbol.

Since there are links in the markdown text with anchor elements and codeblocks that also contains hash symbols which will be misinterpreted as headers, these are removed first from the whole text.

```js
  const regexReplaceCode = /(```.+?```)/gms
  const regexRemoveLinks = /\[(.*?)\]$.*?$/g

  const markdownWithoutLinks = markdown.replace(regexRemoveLinks, "")
  const markdownWithoutCodeBlocks =  markdownWithoutLinks.replace(regexReplaceCode, "")
```

Then, using the hash symbol, the headings ```h1``` to ```h6``` are filtered from the text and added to an array named titles.

```js
  const regXHeader = /#{1,6}.+/g
  const titles = markdownWithoutCodeBlocks.match(regXHeader)
```

Next, using the headings, levels of headings, titles, and anchor links are created and added to an array ```toc``` so that the headings can later be nested with child headings and anchor links can be added. The anchor links can then be used to jump from the table of contents to a heading.

```js
let globalID = 0
titles.map((tempTitle, i) => {
      const level = tempTitle.match(/#/g).length - 1
      const title = tempTitle.replace(/#/g, "").trim("")
      const anchor = `#${title.replace(/ /g, "-").toLowerCase()}`
      level === 1 ? (globalID += 1) : globalID

      toc.push({
        level: level,
        id: globalID,
        title: title,
        anchor: anchor,
      })
    })
```

The array toc is returned and I pass this for example as ```post.toc``` to the respective post, where ```post.toc``` in turn is passed as props to the ToC component.

```js
export async function getStaticProps({ params }) {
  const content = (await data?.posts[0]?.content) || ""
  const toc = getToc(content)

  return {
    props: {
      post: {
        content,
        toc
      },
    },
  }
}
```

## Rendering the table of contents

Each element from the ```toc``` array is now added to the table of contents component. The levels variable is used to dynamically create indentation for subordinate headings with margin and the anchor is used for links.

```js
import styled from "styled-components"

const ToCListItem = styled.li`
  list-style-type: none;
  margin-bottom: 1rem;
  padding-left: calc(var(--space-sm) * 0.5);
  border-left: 3px solid var(--secondary-color);
  margin-left: ${(props) => (props.level > 1 ? `${props.level * 10}px` : "0")};
`

export default function TableOfContents({ toc }) {
  function TOC() {
    return (
      <ol className="table-of-contents">
        {toc.map(({ level, id, title, anchor }) => (
          <ToCListItem key={id} level={level}>
            <a href={anchor}>{title}</a>
          </ToCListItem>
        ))}
      </ol>
    )
  }

  return (
    <>
      <p>Table of contents</p>
      <divr>
        <TOC />
      </div>
    </>
  )
}

```

However, the anchor links do not work yet, since the corresponding section IDs still have to be added to the titles in Markdown content. 

For rendering the actual post content I use [react-markdown](https://github.com/remarkjs/react-markdown "react-markdown"). With the help of custom renderers you can now edit all html elements in react-markdown. To add anchor links to the titles I use custom renderers for ```h1``` to ```h6```.

```js
const renderers = {
  h2: { children }) => {
    const anchor = `${children[0].replace(/ /g, "-").toLowerCase()}`
    return <h2 id={anchor}>{children}</h2>
  },
  h3: ({children }) => {.
    const anchor = `${children[0].replace(/ /g, "-").toLowerCase()}`
    return <h3 id={anchor}>{children}</h2>
  },
  h4: ({children }) => {.
    const anchor = `${children[0].replace(/ /g, "-").toLowerCase()}`
    return <h4 id={anchor}>{children}</h2>
  },
  h5: ({children }) => {.
    const anchor = `${children[0].replace(/ /g, "-").toLowerCase()}`
    return <h5 id={anchor}>{children}</h2>
  },
  h6: ({children }) => {.
    const anchor = `${children[0].replace(/ /g, "-").toLowerCase()}`
    return <h6 id={anchor}>{children}</h2>
  },
```

Lastly, I added a little scroll effect with the following css-property ```scroll-behavior: smooth```;

## Github Links

- [getToc function](https://github.com/dietrichmax/mxd-codes-frontend/blob/v2/src/utils/getToc.js)
- [toc component](https://github.com/dietrichmax/mxd-codes-frontend/blob/v2/src/components/article/article-toc/table-of-contents.js)
- [articles page](https://github.com/dietrichmax/mxd-codes-frontend/blob/v2/pages/articles/%5Bslug%5D.js)
- [renderers](https://github.com/dietrichmax/mxd-codes-frontend/blob/v2/src/utils/renderers.js)



Building a Table of Contents (TOC) from markdown for your React blog

Recently I went out of storage for my homelab so I bought an used NAS (Synology DS214 play) to have some more capacities for [Proxmox](https://www.proxmox.com/de/ "Proxmox") Backups and OpenStreetMap. I still had a 1TB hdd lying around at home, which I now use for proxmox backups. 

To have some redundancy (and to learn something new) I decieded to copy the Proxmox backups to the cloud, in particular to an Azure Storage Account with [AzCopy](https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10) and in the following I will describe with more details how I was able to do it.

Overall this article will cover the following informations:
- Creating an Azure Storage Account
- Getting started with AzCopy
- Creating a bash-script to copy the Proxmox backups to an Azure Storage Container

## Creating an Azure Storage Account

First off all you need an active Azure subscription and an storage account to be able to store your backups. In the [Azure Portal](https://portal.azure.com "Azure Portal") you can search for the service "Storage Accounts" which you will need.

![azure storage account.png](/uploads/azure_storage_account_e81efc2682.png)

In the service "Storage Accounts" you can create a new storage account. For the storage account you will need 
- an active azure subscription,
- a ressource group (create one if you don't have one it, e.g. RG-HOMELAB),
- a storage account name,
- selecting a region and 
- selecting redundancy (e.g. LRS;)
- Access tier "cold" (See Advanced)

![storage account_basics.png](/uploads/storage_account_basics_e82f1d8a4d.png)
![storage account_advanced.png](/uploads/storage_account_advanced_02f6609c40.png)

You can keep all the other settings as default. After your Storage Account has been deployed you can add a lifecycle rule from "Lifecycle Management" which will move files from the "cold" access tier to the archive storage. 

![storage lifecycle management.png](/uploads/storage_lifecycle_management_e113a6efb8.png)

For example I created a rule which moves all new files after one day to the archive storage tier.

![storage lifecycle lifecycle rule.png](/uploads/storage_lifecycle_lifecycle_rule_85a074cae0.png)

![storage lifecycle lifecycle rule 2.png](/uploads/storage_lifecycle_lifecycle_rule_2_5486e872ee.png)

By storing files in archive storage instead of in the regular "cold" access tier you can actually save about 82%. But keep in mind that accessing data in the Archive storage is more expensive than in the cold (or any other) storage tier.

Also you could create another rule which will for example will delete all all blobs which were created 365 days ago.

![storage pricing.png](/uploads/storage_pricing_8998b65a68.png)

Please have a look at [https://azure.microsoft.com/en-us/pricing/details/storage/blobs/](https://azure.microsoft.com/en-us/pricing/details/storage/blobs/ "Azure Storage Pricing") for uptodate Azure Storage pricing.

After the storage account has been configured you will need to create a Container where the actual files will be stored.
Go to  "Data storage" -> "Containers" and create a Container.
![create storage container.png](/uploads/create_storage_container_8ee34fe39e.png)
Again name it however you want.

Due to the fact that the current version of AzCopy V10 does not support Azure AD authorization in cron jobs I used an SAS token to be able to upload files to the container. You can create a SAS token in the container at "Shared access tokens".

![storage shared access token.png](/uploads/storage_shared_access_token_1646ebf346.png)

For the Shared access token you will need to select the permissions for Add/Create/Write and select an expiry date for security reasons. Then you can generate the SAS token and URL. Copy that **Blob SAS URL** because you will need it for the upload script.

## Getting started with AzCopy

> AzCopy is a command-line utility that you can use to copy blobs or files to or from a storage account. This article helps you download AzCopy, connect to your storage account, and then transfer data. (https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10)

To get AzCopy for Linux you have to download a tar file and decompress the tar file anywhere you like. You can then just use AzCopy because it's an executable file, so nothing has to be installed.

```bash
#Download AzCopy
cd ~
wget https://aka.ms/downloadazcopy-v10-linux
 
#Expand Archive
tar -xvf downloadazcopy-v10-linux
 
#(Optional) Remove existing AzCopy version
rm /usr/bin/azcopy
 
#Move AzCopy to the destination you want to store it
cp ./azcopy_linux_amd64_*/azcopy /usr/bin/

# Remove Azcopy from home
rm -r downloadazcopy-v10-linux
rm -r azcopy_linux_amd64_10.16.2/

```

By adding the azcopy file location as system path you can just type azcopy from any directory on your system.
You can add it to your system path with:

```bash
nano ~/.profile
```
and then adding these lines:

```bash
export PATH=/usr/bin/azcopy:$PATH
```

Lastly, update your system variables:

```bash
source ~/.profile
```

## Creating a bash-script to synchronize the Proxmox backup directory to an Azure Storage Container

The only piece missing now is the script which will upload the the Proxmox backup files to the previously created azure storage container after the backup task has finished. 
For copying the backups to Azure we will use azcopy **copy** because acopy uses less memory and incurs lower billing costs because a copy operation does not need to index the source or destination before moving files in comparison to azcopy **sync**.
Azcopy also compares file names and last modified timestamp to only upload new or changed files to the storage container, which overall will reduce bandwidth usage and it will work perfectly with the previously created lifecycle rule.

For automatically starting the upload after the backup has finished  we can use a hook script for vzdump. Therefore you need to add the following line to the end of the "/etc/vzdump.conf" file.
```bash
script: /etc/your-custom-script.sh
```
Afterwards you can create the script which will upload the files with:
```bash
cd ~
mkdir scripts
cd scripts
nano upload-backups-to-azure.sh
```
Then copy paste the following content into the file and replace the content for src with the location of your dumps. Note that there is "/*" at the end of src so that only the files inside the directory will be copied. Also replace token with the **Blob SAS URL**.

```bash
#!/bin/bash
# Script to upload Proxmox backups to Azure Storage

src="/mnt/pve/xyz/dump/*"
token="Blob SAS URL"

dobackup(){
  echo "Uploading Proxmox backups from $src to Azure..."
  azcopy copy "$src" "$token" --overwrite=false
  echo "Finished Uploading!"
}

if [ "$1" == "job-end" ]; then
  dobackup
fi

exit 0
```

Close the file and make it executable for the user with:

```bash
chmod +x ~/scripts/upload-backups-to-azure.sh
```

Now the next time your backup task has finished the files will be automatically uploaded to your Azure storage container.
Due to the  hook script you can check the status of the copy process in the proxmox ui.

How to copy your Proxmox backups with AzCopy to Azure Storage Containers

In this article I will explain how you can create a basic web map with Leaflet and React by using functional components without any third party packages. So i will strongly recommend to have a look at the [Leaflet API reference](https://leafletjs.com/reference.html).

## React and Leaflet

First of all you need a react app which you can create with:

```js
npx create-react-app leaflet-react
cd leaflet-react
```

Then you will need the Leaflet package which you can add with ```npm install leaflet``` or ```yarn add leaflet```.
After you have installed the package you can import it with ```import L from "leaflet"``` into your App.js. The import of the leaflet.css is also important because without it the map-tiles will be misplaced.

```js
import React, { useEffect } from "react";
import L from "leaflet";
import 'leaflet/dist/leaflet.css';

const App = () => {
  
    return (
    )
  };
  
export default App;
```

Now you can create the actual map by using the UseEffect hook:

```js
useEffect(() => {
  L.map("map", mapParams);
}, []);
```

The "map" parameter is the id of the html-element in which the map will be rendered. With mapParams you can passsome basic parameters as props for the Leaflet map. These parameters can just be created in a object [Leaflet API: Map Creation)](https://leafletjs.com/reference.html#map-factory):

```js
const mapParams = {
  center: [0, 0],
  zoom: 0,
  layers: [layer]
};
```

TileLayers with OpenStreetMap Data can be created with L.tileLayer(url, options) [(Leaflet API: TileLayer)](https://leafletjs.com/reference.html#tilelayer).

```js
  const layer = L.tileLayer(`https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png`, {
        attribution: '&copy; <a href="https://www.openstreetmap.org/copyright">OpenStreetMap</a> contributors'
  });
```

Now you can create some basic css in js for the map container which makes the map fullscreen and will be passed as style props:

```js
const mapStyles = {
  width: "100%",
  height: "100vh"
 };
```

In the end you just need html element in which the map will be rendered:

```js
 return (
      <div>
        <div id="map" style={mapStyles} />
      </div>
    )
```

If you are curious how to add some more features like vector layers, some controls or markers have a look at the [Leaflet API Reference](https://leafletjs.com/reference.html) and much fun playing around with your Leaflet web map created with React and functional components. In case something didn't work out as expected you can just clone the following repositiory.

Github Repositiory: [https://github.com/dietrichmax/leaflet-react-functional-component](https://github.com/dietrichmax/leaflet-react-functional-component)
Live Demo: [https://dietrichmax.github.io/leaflet-react-functional-component/](https://dietrichmax.github.io/leaflet-react-functional-component/)

All code together:

```js
import React, { useEffect } from "react";
import L from "leaflet";
import 'leaflet/dist/leaflet.css';

const Map = () => {
    const mapStyles = {
        width: "100%",
        height: "100vh"
      };
      const layer = L.tileLayer(`https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png`, {
        attribution: '&copy; <a href="https://www.openstreetmap.org/copyright">OpenStreetMap</a> contributors'
      });
  

    const mapParams = {
      center: [0, 0],
      zoom: 0,
      layers: [layer]
    };
     
    // This useEffect hook runs when the component is first mounted, 
    // similar to componentDidMount() lifecycle method of class-based
    // components:
    useEffect(() => {
        L.map("map", mapParams);
    }, []);
  
    return (
      <div>
        <div id="map" style={mapStyles} />
      </div>
    )
  };
  
  export default Map;
js

How to create web maps with Leaflet, React and functional components

reddit

twitter

How to create a custom cookie banner for your React application

Some blogs have these related articles or posts sections where visitors can have a preview at more content after they just read a post. That's what I wanted to create for my personal website which is built with React (Nextjs) and in this article I want to show you how you also can do it for any other react application.

## Related Posts Component

The keypoint to be able to show related Posts is that you somehow have to create a relation between the posts which doesn't exist yet. All my posts have

- a title,
- a description
- a date,
- tags and
- an image.

```md
---
title: "Post about Web-Development with React"
description: "This is a sample description for the post."
date: "2022-05-02"
tags: ["React", "Web-Development"]
image: "../image.jpg"
---
```

I decided to use the tags to create a relation between the posts because it's the only information which all posts can have in common and is related to the actual topic of the post. Therefore i needed data of all posts overall and data about the current posts. The data from the current post will be just passed as props to the component. All post data for my website is created in a CMS which can be accessed via GraphQL. The query to get allPosts looks like this.

```js
export async function getAllPosts() {
  const data = await fetchStrapiAPI(
    `
    {
      posts(sort: "published_at:desc") {
        id
        published_at
        title
        slug
        content
        excerpt
        tags {
          name
        }
        coverImage {
          url
        }
      }
    }
  `
  )
  return data?.posts
}
```

The only relevant information here is the slug and the tags with their names.

Now the current post gets filtered out from the posts array and a variable maxPosts for the maximum number of posts which should be displayed will be created.

```js
// filter out current post
let posts = allPosts.filter((aPost) => aPost.slug !==post.slug);
    
// define maxPosts to display
const maxPosts = 3
```

For better readability I assigned the tasks of the current posts to a variable called currentTags

```js
// get tags of current posts
const currentTags = post.tags.map((tag) => {
  return tag.name
})
```

Now you have to map through posts and the tags post.tags of these posts to check if one of these tags is the same as one of the currentTags. If one tag is the same we will just enumerate a new relevance variable.

```js
    // rate posts depending on tags
    posts.forEach((post) => {
        post.relevance = 0
        post.tags.forEach((tag) => {
            if (currentTags.includes(tag.name)) {
                post.relevance ++
            }
        })
    })
```

The post with the highest relevance will be the post with the most common tags and be the most related post. If you are also using categories you can of course also adjust the relevance depending on the categories and the tags. For example you could add two relevance points for categories and one relevance point for tags.

Then you can sort the array of all posts descending by relevance.

```js
    // sort posts by relevance
    const sortedPosts = posts.sort(function(a, b) {
        return b.relevance - a.relevance;
    });
```

In the end you can slice them with maxPosts and finally render them.

```js
import PostPreview from 'src/components/article/article-preview/article-preview'

export default function RecommendedPosts({ post, allPosts }) {

    // filter out current post
    let posts = allPosts.filter((aPost) => aPost.slug !==post.slug);
    
    // define maxPosts to display
    const maxPosts = 3

    // get tags of current posts
    const tags = post.tags.map((tag) => {
        return tag.name
    })

    // rate posts depending on tags
    posts.forEach((post) => {
        post.relevance = 0
        post.tags.forEach((tag) => {
            if (tags.includes(tag.name)) {
                post.relevance ++
            }
        })
    })

    // sort posts by relevance
    const sortedPosts = posts.sort(function(a, b) {
        return b.relevance - a.relevance;
    });

    return (
        <>
          {sortedPosts.slice(0,maxPosts).map((post, i) => (
            <PostPreview
              key={i} 
              postData={post}
            />
          ))}
        </>
    )
  }
```
  

How to build a related posts component for your React blog

In this article I am going to explain, how you can implement Google Adsense in Next.js applications (or any other react applications). There are several approaches for implementing Adsense on a react site and I want to show you how you can add Adsense with privacy in mind.

As soon as you signed up for your site on Adsense and it has been approved, you have to place the Adsense code (or ad unit code) in your pages. This code in general exists of three parts.

## Add Google Adsense script

The first part will load the actual Adsense script. This script is typically placed between the ```<head></head>``` or ```<body></body>``` section.

We will not just place it there because we just want the script to be inserted after a user has given consent to allow third-party cookies and services. So I for example outsourced it in a separate function which will be triggered by accepting cookies.

```js
export function enableGoogleAdsense () {
    const head = document.getElementsByTagName('head')[0]
    const scriptElement = document.createElement(`script`)
    scriptElement.type = `text/javascript`
    scriptElement.async
    scriptElement.src = `https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=${process.env.NEXT_PUBLIC_ADSENSE_ID}`
    scriptElement.crossOrigin = "anonymous"
    head.appendChild(scriptElement);
}
```

By clicking the 'Accept required and optional cookies' button on this site, this function will be triggered which will then place the adsense script into the <head></head> section.

If you want to use Auto ads you are actually already done as long as Auto Ads are enabled in your Adsense Account for your site.

Otherwise, if you want to place ad units individually you can do this now like the following.

## Add ad units

I would recommend to create a separate component for ad units. This could look like the following:

```js
import styled from 'styled-components';
import { useEffect } from 'react';

export function GoogleAdsenseContainer ( { client, slot }) {

  useEffect(() => {
    (window.adsbygoogle = window.adsbygoogle || []).push({});
  }, []);

  const AdLabel = styled.span`
    font-size: 12px;
  `

  return (
    <div 
      style={{textAlign: 'left',overflow: 'hidden'}}
    >
    <AdLabel>Advertisment</AdLabel>
      <ins
        className="adsbygoogle"
        style={{ display: "block" }}
        data-ad-client={client}
        data-ad-slot={slot}
        data-ad-format="auto"
        data-full-width-responsive="true"
      ></ins>

    </div>
  );     
}
```

In this component you will find the other to parts of the original Adsense script. So here the actual ad unit element is placed with a small ad-label. You can load this component in every page and position you like and place your individual client and slot ID. After the ad unit is placed ```window.adsbygoogle``` in the useEffect hook will fill this ad unit with the actual advertisement graphics.

With the useEffect hook ads will also be requested/refreshed when a user navigates to any other page without refreshing the page.

How to add Google Adsense to Next.js applications

With [Gatsby 4](https://www.gatsbyjs.com/gatsby-4/) bringing in Server-Side Rendering (SSR) and Deferred Static Generation (DSG) you need an alternative methode to just hosting static files. Each page using SSR or DSG will be rendererd after a user requests it so there has be a server in the background which will handle these requests and build the pages if needed.

In this post i will show you how you can deploy your Gatsby site with SSR and/or DSG on your own server with a CI/CD pipeline via PM2 and Github Webhooks.

Therefore i will be using

- Gatsby 4,Nginx,
- Nginx
- PM2 and
- Github webhooks.

## Setup your server

First of all you need an server with root access. I strongly recommend to have a look at the guide ["Initial Server Setup with Ubuntu 18.04"](https://www.digitalocean.com/community/tutorials/initial-server-setup-with-ubuntu-18-04) from the DigitalOcean community which will lead you through the process of:

- Logging in and set up root user access to your server with SSH
- Creating a new user
- Granting Administrative Privileges to the new user
- Setting up a basic firewall
- Giving your regular user access to the server with SSH key authentication.

After you have done that you can continue by installing all necessary dependencies on your server.
Install Node.js

Again there is an guide by DigitalOcean which will help you [installing Node.js using PPA](https://www.digitalocean.com/community/tutorials/how-to-install-node-js-on-ubuntu-18-04#installing-using-a-ppa).

After completing

- install Node.js, NPM and
- the "build-essential package"

you will have to change npm's default directory.

Create a ```.npm-global``` directory and set the path to this directory for ```node_modules```:

```bash
cd ~
mkdir ~/.npm-global
npm config set prefix '~/.npm-global'
```

Create (or modify) a ```~/.profile``` and add the following line:

```bash
sudo nano ~/.profile

# set PATH so global node modules install without permission issues
export PATH=~/.npm-global/bin:$PATH
```

Now you have to update your system variables:

```bash
source ~/.profile
```

Now you should be able to check your installed Node.js version with:

```bash
node -v
```

## Install git

Check if git is already installed with:

```bash
git --version
```

If it isn't installed yet you can install it with

```bash
sudo apt install git
```

and configure Git with

```sbash
git config --global user.name "Your Name"
git config --global user.email "youremail@domain.com"
```

After git is installed and configured you can deploy your Gatsby site by cloning it from Github.

## Deploy from Github

It is important that you are loggin in as **non-root** user for the following steps.

```bash
cd ~
git clone https://github.com/your-githubuser/your-gatsby-repo.git path your-gatsby-site
```

After you have deployed your project (optionally with environment variables) you can install all dependencies and build your Gatsby site with:

```bash
cd ./your-gatsby-site/
npm install
npm run build
```

Now you should have a copy of your local project/Gatsby site on your remote server.

Next you are going to setup PM2 which will be used to keep your site alive and restart it with every reboot.

## Setup PM2

You can install PM2 with:

```bash
npm install pm2@latest -g
```

You will need to create/configure an ```ecosystem.config.js``` file which will restart the default Gatsby server.

```bash
cd ~
pm2 init
sudo nano ecosystem.config.js
```

Copy/paste the template and replace the content.

```js
module.exports = {
  apps: [
    {
      name: 'gatsby-site',
      cwd: ' /home/your-name/my-gatsby-site',
      script: 'npm',
      args: 'serve',
      env: {
        //NODE_ENV: 'production',
      },
    },
    // optionally a second project
],};
```
With

```bash
cd ~
pm2 start ecosystem.config.js
```

you can start your server which will run on the Port 9000.

You can always check the status with:

```bash
pm2 status
```

After the server reboots this PM2 should be always automatically be restarted. For that you are going to need a small Startup script which you can also copy/paste.
Generate and configure a startup script to launch PM2:

```bash
cd ~
pm2 startup systemd

[PM2] Init System found: systemd
[PM2] To setup the Startup Script, copy/paste the following command:
**sudo env PATH=$PATH:/usr/bin /usr/lib/node_modules/pm2/bin/pm2 startup systemd -u your-name --hp /home/your-name**
```

- Copy/paste the generated command:

```bash
**sudo env PATH=$PATH:/usr/bin /usr/lib/node_modules/pm2/bin/pm2 startup systemd -u your-name --hp /home/your-name**

[PM2] Init System found: systemd
Platform systemd

. . .


[PM2] [v] Command successfully executed.
+---------------------------------------+
[PM2] Freeze a process list on reboot via:
   $ pm2 save

[PM2] Remove init script via:
   $ pm2 unstartup systemd
```

- And save the new PM2 process list and environments. Then Start the service with systemctl.

```bash
pm2 save

[PM2] Saving current process list...
[PM2] Successfully saved in /home/your-name/.pm2/dump.pm2
```

If you reboot your server now with ```sudo reboot``` the script should be automatically restart your Gatsby site. Give it a try!

## Setup Github Webhook

One thing missing now is an continuos integration and continuos delivery (CI/CD) pipeline which you will setup using Github webhooks.

Therefore you need to create a new Webhook in your repository.

The following articles provide additional information to the steps below:

- [GitHub Creating Webhooks Guide](https://docs.github.com/en/developers/webhooks-and-events/webhooks/creating-webhooks#setting-up-a-webhook) and
- [DigitalOcean Guide to GitHub WebHooks](https://www.digitalocean.com/community/tutorials/how-to-use-node-js-and-github-webhooks-to-keep-remote-projects-in-sync).

You need to create a server script which will do something if it is triggered by the Github webhook.

```bash
cd ~
mkdir NodeWebHooks
cd NodeWebHooks
sudo nano webhook.js
```

The script is going to create a server running on Port 8100. (Your Github webhook should be of course sending the webhook to something like http://server-ip:8100.)

If it gets triggered by a webhook it will

- go into your repo ```~/my-gatsby-site/```,
- pull the latest commits,
- install all dependencies,
- build a new version of the site and
- restart the server via the PM2 script.

```js
const secret = "your-secret-key";
const repo = "~/my-gatsby-site/";

const http = require('http');
const crypto = require('crypto');
const exec = require('child_process').exec;

const BUILD_CMD = 'npm run build';
const PM2_CMD = 'pm2 restart gatsby-site';

http.createServer(function (req, res) {
    req.on('data', function(chunk) {
        let sig = "sha1=" + crypto.createHmac('sha1', secret).update(chunk.toString()).digest('hex');

        if (req.headers['x-hub-signature'] == sig) {
            exec('cd ' + repo + ` && git pull && npm install && ${BUILD_CMD} && ${PM2_CMD}`);
        }
    });

    res.end();
}).listen(8100);
```

You will need to allow communication on Port 8100 with:

```bash
sudo ufw allow 8100/tcp
sudo ufw enable
```

Command may disrupt existing ssh connections. Proceed with operation (y|n)? y
Firewall is active and enabled on system startup

Earlier you setup PM2 to restart your Gatsby site whenever the server reboots or is started. You will now do the same for the webhook script.

Run echo $PATH and copy the output for use in the next step.

```bash
echo $PATH

/home/your-name/.npm-global/bin:/home/your-name/bin:/home/your-name/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
```

Create a webhook.service file:

```bash
cd ~
sudo nano /etc/systemd/system/webhook.service
```
    
In the editor, copy/paste the following script, but make sure to replace your-name in two places with your username. Earlier, you ran echo ```$PATH```, copy this to the Environment=PATH= variable, then save and exit:

```bash
[Unit]
Description=Github webhook
After=network.target

[Service]
Environment=PATH=your_path
Type=simple
User=your-name
ExecStart=/usr/bin/nodejs /home/your-name/NodeWebHooks/webhook.js
Restart=on-failure

[Install]
WantedBy=multi-user.target
```
    
Enable and start the new service so it starts when the system boots:

```bash
sudo systemctl enable webhook.service
sudo systemctl start webhook
```
    
Check the status of the webhook:

```bash
sudo systemctl status webhook
```

You can test your webhook with these [instructions](https://www.digitalocean.com/community/tutorials/how-to-use-node-js-and-github-webhooks-to-keep-remote-projects-in-sync#step-4-testing-the-webhook "Test Webhook").

The Gatsby server is now running on your-ip:9000 and you implemented a CI/CD pipeline via PM2 and Github Webhooks but you still can't access your website via a domain because you need to configure a webserver like Nginx.

## Configure Nginx

I am using Cloudflare to manage DNS for my domains but you can do this with every other provider also.

Create two A Records which will point ```your-domain.com``` and ```www.your-domain.com``` to the IP-adress of your server.

After that you will need to configure Nginx.

The following instructions are based on [How To Install Nginx on Ubuntu 18.04 [Quickstart]](https://www.digitalocean.com/community/tutorials/how-to-install-nginx-on-ubuntu-18-04-quickstart).

- Update your local package index:

```bash
sudo apt update
```

- install Nginx:

```bash
sudo apt install nginx
```
    
and adjust the Firewall:

```bash
sudo ufw allow 'Nginx Full'
sudo ufw delete allow 'Nginx HTTP'
```

You should now be able to see the Nginx landing page on:

```http://your_server_ip```

## Setting up Server Blocks
    
Create the directory for your-domain.com, using the -p flag to create any necessary parent directories:

```bash
sudo mkdir -p /var/www/your-domain.com/html
```
    
Assign ownership of the directory:

```bash
sudo chown -R $USER:$USER /var/www/your-domain.com/html
```
    
The permissions of your web roots should be correct if you haven’t modified your umask value, but you can make sure by typing:

```bash
sudo chmod -R 755 /var/www/example.com
```
    
Make a new server block at /etc/nginx/sites-available/your-domain.com:

```bash
sudo nano /etc/nginx/sites-available/example.com
```
    
Copy/Paste the following Gatsby-nginx configuration and update the server_name sections:

```bash
server {
    # Listen HTTP
    listen 80;
    listen [::]:80;

    server_name your-domain.com www.your-domain.com;

    # Redirect HTTP to HTTPS
    return 301 https://$host$request_uri;
}

server {
    # Listen HTTP
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name your-domain.com www.your-domain.com;

    # SSL config
    include snippets/self-signed.conf;
    include snippets/ssl-params.conf;

    # Proxy Config
    location / {
        proxy_pass http://localhost:9000
        proxy_http_version 1.1;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Server $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Host $http_host;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_pass_request_headers on;
    }
    location ~ /.well-known {
       allow all;
    }
}
```

Save the file and close it when you are finished.

Enable the file by creating a link from it to the sites-enabled directory:

```bash
sudo ln -s /etc/nginx/sites-available/your-domain.com /etc/nginx/sites-enabled/
```
    
Test for syntax errors:

```bash
sudo nginx -t
```
    
and finally enable the changes:

```bash
sudo systemctl restart nginx
```

Nginx should now be serving your gatsby site on your domain name. That means if you have a look at ```http://your-domain.com``` you should see your Gatsby site.

In the end should deny traffic to Port 9000 because Nginx is handling the requests with:

```bash
cd ~
sudo ufw deny 9000
```

To install SSL, you will need to [install and run Certbot by Let's Encrypt](https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-18-04).



How to deploy your Gatsby site on your own server

2021-03-02 i started tracking my current location with [OwnTracks](https://owntracks.org/ "OwnTracks") and [Strapi](https://strapi.io/ "Strapi") ([How i constantly track my location and display a web-map with all the locations](/articles/constant-location-tracking-with-owntracks-strapi-and-visualizing-data-with-deckgl "How i constantly track my location and display a web-map with all the locations")) and created a [/map](/map "map with locations") which shows all the locations i have ever been to. Recently i reached one million datasets and unfortunately that means every user had to download about 50MB of location data before the map could be rendered. So i definitely needed a much faster solution and decided to render and serve server side some tiles which will be used to display the locations. In the end i built a tile-server following [Manually building a tile server (20.04 LTS)](https://switch2osm.org/serving-tiles/manually-building-a-tile-server-20-04-lts/ "Manually building a tile server (20.04 LTS)") with a custom Mapnik stylesheet.

In this article i want to show you how you can build your own Mapnik stylesheet for displaying any data from PostgreSQL/PostGIS.

The Mapnik stylesheet XML is not very handy but fortunately there are some tools which will help you to create one. After some research i decided to go with [TileMill](https://tilemill-project.github.io/tilemill/ "TileMill") which is an open source map design studio to design maps. It offers a simple UI and (more important) the possibility to export the created map-style as Mapnik stylesheet.

Actually i couldn't get the latest version running so i decided to go with TileMill v0.10.1 which offers anything you will need to create a stylesheet. At [https://tilemill-project.github.io/tilemill/docs/win-install/](https://tilemill-project.github.io/tilemill/docs/win-install/) you can download and install TileMill.

After you have installed TileMill you can create a new project and uncheck 'default data'. Otherwise TileMill will create some kind of basemap with a default style.

![tilemill_create_project.png](/uploads/tilemill_create_project_028d3d068a.png)

Now you will have to add some data to the project. Therefore you click on the layer button and add a new layer.

![tilemill_add_layer.png](/uploads/tilemill_add_layer_9bfeb33806.png)

Switch to the **PostGIS** tab and fill out ID, Connection, Unique key field, Geometry field and SRS. It's actually pretty straight forward. 
Keep in mind that TileMill doesn't like large datasets so i would recommend you to set some extent.

In SRS you have to specify the PROJ.4 projection string. If you don't it know a look at https://epsg.io/{epsg-code} where you will replace {epsg-code} with the epsg code of your coordinate system and scroll down to PROJ.4 where you can copy the projection string. 

![proj4.png](/uploads/proj4_2c3dab1b78.png)
Afterwards save the layer and the project. You won't see your data/features yet because you need to define some style for it. You can style the map with CartoCSS which is very similiar to CSS. For example i specified my layer id as locations and i am styling points so the CartoCSS properties could look like the following.

```
#locations {
   [vel >= 0] { marker-width:6; marker-fill: #f45; marker-line-color: #813; marker-allow-overlap: true; }
   [vel >= 50] { marker-width:6; marker-fill: #f45; marker-line-color: #813; marker-allow-overlap: true; }
   [vel >= 100] { marker-width:6; marker-fill: #f45; marker-line-color: #813; marker-allow-overlap: true; }
}
```


With CartoCSS you can also dynamically style your features depending on some attribute values. For more information about CartoCSS have a look at [Styling data from TileMill](https://tilemill-project.github.io/tilemill/docs/crashcourse/styling/ "Styling data").

When you are happy with your style you can export if as Mapnik-XML and use if for example for your tile-server.

![tilemill_export_mapnik_xml.png](/uploads/tilemill_export_mapnik_xml_2d50702c3b.png)
 
At [/map](/map) you can see my current stylesheet in 'action'.
 

How to create a Mapnik stylesheet for displaying any data from PostgreSQL/PostGIS

Inspired by Aaron Parecki who has been [tracking his location since 2008](https://aaronparecki.com/gps/ "location tracking") with an iPhone app ([Overland](https://overland.p3k.app/ "Overland")) and a server side tracking API ([Compass](https://github.com/aaronpk/Compass "Compass")), i decided to go for a similar approach.

I wanted to track my position constantly with my Android smartphone, save the data to a Postgres database and use the data to display an map with all locations i have ever been to.

To be able to track my position i needed an stable android app and after some tinkering around with different tracking apps i found [OwnTracks](https://owntracks.org/ "OwnTracks")  which basically has all i need (and a good [documentation](https://owntracks.org/booklet/ "OwnTracks Documentation") of it).

## OwnTracks

> The OwnTracks app runs in the background on your Android or iOS device and waits for the smart phone to tell it that the device has moved, whereupon OwnTracks sends out a message with its current coordinates (and a few other things we'll discuss in a moment).

With different settings for the app i could tailor the app exactly to my needs because i wanted the app to check my position every second and when i have moved more than two meters from my last known position it should send a POST-Request to my Strapi instance which will store the location data in a Postgres database.

In the following screenshot you can see the settings i am using for OwnTracks.

![Screenshot_OwnTracks settings.jpg](/uploads/Screenshot_Own_Tracks_settings_c7d95a33be.jpg)

The most important seetings for me are the mode **Significant location change mode**, **locatorDisplacement**, **locatorInterval**, **ignoreInaccurateLocations** and the **url** for the POST-request..

The **significant location change mode** is a mode which will let you decide in which interval you want to track your position. I am using the **locationInterval** '1' which will track my position every second with an **LocatorDisplacement** of 2 which will check if i moved away two meters away from my last known position. If i did it will send an POST-request with the location data to the given url if the position data isn't more inaccurate than 100 meters.

Because the app has to run always in the background you probably have to switch off energy saving modes for your smartphone which might disable the background functionality. 
Be warned: You probably will have to charge your smartphone more often. Also sometimes the app has some tracking error if you are using parallel some other tracking app for example for activities like strava.

The url for the POST-request is an strapi API url which looks like https://strapi.url/locationEndpoint. Strapi will then automatically handle the request and store the data in whatever database and table you have configured for Strapi. 

I am using the following attributes where the most important are **lat** (latitude), **lon** (longitude) and **alt** (altitude).

![Screenshot Location Table.png](/uploads/Screenshot_Location_Table_eee78ddafe.png)

I am also saving the velocity, battery level of the smartphone and some more data which will be displayed at my [/now](/now "/now") page. The velocity and altitude are quite inaccurate though which probably also depends on the smartphone.

## Web-Map

**Edit 25.01.2022:** 
Recently i switched to a server side approach to visualise the data because rendering about one million points on client side takes way too long. So i built a tile server which is serving PNG-tiles with the location data. Right now the [/map](/map) is created with OpenLayers.

If you are curious how you can set up a tile server which is serving data from Postgres with Mapnik, Apache and Nginx you can skip the following content and head directly to the article [Manually building a tile server (20.04 LTS)](https://switch2osm.org/serving-tiles/manually-building-a-tile-server-20-04-lts/ "Manually building a tile server (20.04 LTS)") and follow all steps except **Stylesheet configuration** because in the Mapnik stylesheet you have to create your custom style and a connection with your PostgreSQL table where your data is stored. See [How to create a Mapnik Stylesheet for displaying any data from PostgreSQL/PostGIS](/articles/how-to-create-a-mapnik-stylesheet-for-displaying-any-data-from-postgre-sql-post-gis "How to create a Mapnik Stylesheet for displaying any data from PostgreSQL/PostGIS").

If you only want to render some 'small' amount of data you can continue with the following.

At the beginning i rendered the data on client side with deck.gl. The following steps describe how to display data from PostgreSQL with deck.gl and Next.js (which is outdated for the /map page).

Due to the fact that amount of data is growing and growing i use [Prisma](https://www.prisma.io/ "Prisma") to get the data directly from postgres. If i would use the methods provided by Strapi it takes quite some time to generate the page with the map.

At the moment the [/map](/map "/map") page is built very simple:

```js
import Layout from '@/components/layout/layout'
import SEO from '@/components/seo/seo'
import Title from '@/components/title/page-title'
import styled from 'styled-components';
import media from 'styled-media-query';
import { getLocationsCount } from "@/lib/data/external/cms"
import prisma from '@/lib/utils/prisma'
import dynamic from 'next/dynamic'

const Livemap = dynamic(
  () => import('@/components/maps/deckgl/livemap'),
)

const MapContainer = styled.div`
  margin: auto;
  max-width: 1200px;
  padding: var(--space);
  ${media.lessThan('medium')`
    padding: var(--space-sm);
  `}
`

const Description = styled.p`
  max-width: 1200px;
  margin: auto;
  padding: 0 var(--space) var(--space) var(--space);
  ${media.lessThan('medium')`
    padding-left: var(--space-sm);
    padding-right: var(--space-sm);
  `}
`

const InternalLink = styled.a`
  color: var(--text-color);
  border-bottom: 1px solid var(--thirdy-color);
  cursor: pointer;
  :hover {
    border-bottom: 1px solid transparent;
  }
`

export default function Map({ locations, locationsCount } ) {

  
  return (
    <Layout>
  
      <SEO   
        title="Map"
        slug="map"
      />
      <Title>Map</Title>
      
      <MapContainer>
        <Livemap data={locations} />
      </MapContainer>
      <Description>
        Since 2021-03-02 i am tracking my current location. Right now there are {locationsCount} locations displayed on the map. 
        The map is always centered at my last known position.
      </Description>
      <Description>
        If you are curious how that works have a look at the article <InternalLink href="/articles/how-i-track-my-location-and-display-the-data-on-my-website" title="How i track my location and display the data on my website">How i track my location and display the data on my website</InternalLink> where i am describing the details behind it.
      </Description>    
    </Layout>
  )
}

export async function getStaticProps() {
  const locationsCount = (await getLocationsCount()) || []
  
  const locations = await prisma.locations.findMany({
    select: {
      lat: true,
      lon: true,
      alt: true,
      vel: false,
    },
  });

  return {
    revalidate:  86400,
    props: {
      locations,
      locationsCount
    }
  }
}
  

```

Everyday it fetches all the location data and passes it to the map-component.

Performance wise i made some bad experience with the default mapping libraries Leaflet and OpenLayers because of the client side rendered data but after some research i decided to go with [deck.gl](/https://deck.gl/ "deck.gl") which is made for visualizing large scale datasets.

```js
import React, { useState } from 'react';
import {StaticMap, LinearInterpolator, WebMercatorViewport} from 'react-map-gl';
import DeckGL from '@deck.gl/react';
import {ScatterplotLayer} from '@deck.gl/layers';
import styled from 'styled-components'

const MapContainer = styled.div`
    position: relative;
    height: 600px;
`


export default function Livemap({
  data,
}) {
    const [viewport, setViewport] = useState({
        longitude: data.slice(-1)[0].lon,
        latitude: data.slice(-1)[0].lat,
        zoom: 12,
        bearing: 0,
        pitch: 0,
    });


    const layers = [
        new ScatterplotLayer({
          id: 'scatter-plot',
          data,
          opacity: 0.1,
          stroked: false,
          filled: true,
          radiusScale: 3,
          radiusMinPixels: 0.65,
          getPosition: d => [d.lon, d.lat, 0 /*d.alt*/],
          getFillColor: d => [Math.sqrt(d.vel*20), 184, 50],
        })
    ];

    
    
    return (
        <MapContainer>
            <DeckGL layers={layers} initialViewState={viewport} controller={true}>
                <StaticMap 
                    reuseMaps 
                    mapStyle="https://basemaps.cartocdn.com/gl/dark-matter-gl-style/style.json" 
                    preventStyleDiffing={true} 
    
                />
            </DeckGL>
        </MapContainer>
  )
}

```

I am using an scatterplot layer which will render every location as point and adjusts the color based on the velocity. The viewport is always centered at the last known location and as basemap i am using dark-matter for the both dark and light mode. 

You can find more information about the deck.gl map at the [deck.gl scatterplot example](https://deck.gl/examples/scatterplot-layer/).

How i constantly track my location and display a web-map with all the locations

With getting into the [IndieWeb](https://indieweb.org/ "IndieWeb") i started to reflect about myself and how i can actually own my data instead of giving them to so called [silos](https://indieweb.org/silo "silos").

Due to the fact I am a passionate (mountain) bike rider i was thinking about how i could use tracking/activity apps in a way to get my data back, because obviously when i am going for a ride with my bike and i am tracking the route with strava and/or komoot the data is then saved by them.
Considering that every route is tracked on a Garmin device anyway and then synchronised to the apps i decided to have a look at the Garmin Connect/Activity API.

Unfortunately the [official Garmin Activity API](https://developer.garmin.com/gc-developer-program/activity-api/ "Garmin Activity API") is only available for approved business developer.

But after some searching i found the npm-package [garmin-connect](https://github.com/Pythe1337N/garmin-connect "garmin-connect") which allows you to connect to Garmin Connect for sending and receiving activity data.

## Fetching and storing activities from Garmin Connect with Strapi

You can install the package with
```js
npm install garmin-connect
```
or

```js
yarn add garmin-connect
```
and use it like
```js
const { GarminConnect } = require('garmin-connect');
// Create a new Garmin Connect Client
const GCClient = new GarminConnect();
// Uses credentials from garmin.config.json or uses supplied params
await GCClient.login('my.email@example.com', 'MySecretPassword');
const userInfo = await GCClient.getActivities());
```
I stored the email and the password for the login in environment variables and used them with 
```js
const { GarminConnect } = require('garmin-connect');
const GCClient = new GarminConnect();
await GCClient.login(process.env.GARMIN_EMAIL ,process.env.GARMIN_PWD);
const userInfo = await GCClient.getActivities());
```

Afterwards experimented a bit with the Garmin Connect and found out there are some very low limits. 
Approximately after ~50 requests in one minute couldn't get anymore any data and had to wait for some (maybe one?; i am not sure) hours until the request was successful again.

In general you can probably do way more with Garmin Connect as you will need, like for example:
- Get user info
- Get social user info
- Get heart rate
- Set body weight
- Get list of workouts
- Add new workouts
- Add workouts to you calendar
- Remove previously added workouts
- Get list of activities
- Get details about one specific activity
- Get the step count

I used only ```GCClient.getActivities();``` to get all activities and and  ```GCClient.getActivity({ activityId: id });``` to get the details of the activity (like spatial data representing the route, start-point and end-point).

To be able to store the data in Strapi i created a new content type collection  **activities** with the following fields/attributes:

![content-type-activity.png](/uploads/content_type_activity_3b64fb0368.png)

Afterwards new entrys for activities can be created.

> Strapi has a documention which explains how to fetch external data and create entries with it: [Fetching external data](https://strapi.io/documentation/developer-docs/latest/guides/external-data.html#call-the-function "Fetching external data").

To get the data from Garmin Connect into Strapi i created a function ```getGarminConnectActivities.js```  for Strapi ([/config/functions/getGarminConnectData.js](https://github.com/dietrichmax/mxd-codes-backend/blob/master/config/functions/getGarminConnectData.js "getGarminConnectActivities.js"))

The function will take care of:
- getting existing activities in Strapi,
- get recent activites from Garmin Connect,
- checking if an activity already exists in Strapi,
- creating the activity in Strapi if it doesn't exist yet and also get the details for it.

and basically looks like this:
```js
module.exports = async () => {
  await GCClient.login(process.env.GARMIN_USERNAME, process.env.GARMIN_PWD)
  const activities = await GCClient.getActivities()
  const exisitingActivities = await getExistingActivities()
  activities ? activities.map((activity) => {
    const isExisting = exisitingActivities.includes(activity.activityId)
    isExisting ? console.log(activity.activityId + " already exists") : createEntry(activity)
  })
  : console.log("no activities found")
}
```

After all activities from Garmin are fetched, i am mapping through them to

- check if they already exist in my cms
- and create them if needed

The exisiting activities in my CMS  are fetched with
```js 
const getExistingActivities = async () => {
  const existingActivityIds = []
  const activities = await axios.get(`https://strapi.url/activities`)
  
  activities.data.map((activity) => {
    existingActivityIds.push(activity.activityID)
  })
  return existingActivityIds
}
```
and the activityIds (originally from Garmin Connect) are returned to be able to check if an entry already exists.
If the entry doesn't exist, details for the missing activity are fetched and a new entry is created with:

```js
const createEntry = async (activity) => {
  const details = await GCClient.getActivity({ activityId: activity.activityId });
  await strapi.query('activity').create({
    activityID: activity.activityId,
    activityName: activity.activityName,
    beginTimestamp: activity.beginTimestamp,
    activityType: activity.activityType,
    distance: activity.distance,
    duration: activity.duration,
    elapsedDuration:  activity.elapsedDuration,
    movingDuration: activity.movingDuration,
    elevationGain: activity.elevationGain,
    elevationLoss: activity.elevationLoss,
    minElevation: activity.minElevation,
    maxElevation: activity.minElevation,
    sportTypeId: activity.sportTypeId,
    averageSpeed: activity.averageSpeed * 3.6 //(m/s -> km/h),
    maxSpeed: activity.maxSpeed * 3.6 //(m/s -> km/h),,
    startLatitude: activity.startLatitude,
    startLongitude: activity.startLongitude,
    endLatitude: activity.endLatitude,
    endLongitude: activity.endLongitude,
    details: details
  })
}
```

You can save way more but i tried to cut it down to the ones i really need or eventually will need.

Only thing missing is some automatic triggering. For this you can use cron jobs in Strapi ([/config/functions/cron.js](https://strapi.io/documentation/developer-docs/latest/guides/external-data.html#call-the-function "cron.js")).

```js
module.exports = {
// Add your own logic here (e.g. send a queue of email, create a database backup, etc.).

    '0 0 18 * * *': () => {
      strapi.config.functions.getGarminConnectActivities();
    },
};
```

I decied to trigger the function everyday at 6 pm so i can have a look at my activites in the evening. 😎

Thats the fun from the 'backend part'.

Next step is to visualize the data in NextJS.

## Visualizing the data with NextJS

I am really liking the embeddable tours of [komoot](https://www.komoot.de/b2b/embed "komoot embed") optic-wise, so i decided to create a similiar looking option for the preview of my activities in the [posts-feed](/ "all posts") and [activites-feed](/activities).

So the preview should consist of
- an title of the activitie (+datetime),
- an symbol displaying the activity type (cycling, running, etc.)
- some basic metrics like distance, duration and average speed,
- a map displaying the line of the route with osm-data and and aerial-view,
- and optionally a graph displaying the elevation along the track (still in work).

The component looks like this at the moment:
![activity-post-preview.png](/uploads/activity_post_preview_eeca5a51b1.png)

### Showing some Metrics

In the **activityType** object you can find ```typeId``` which correlates to the type of the activity, e.g. cycling, running etc.
I created a small function which will return a icon from [react-icons](https://github.com/react-icons/react-icons "react-icons") visualizing the activity type.

``` js
import { FaRunning, FaBiking } from 'react-icons/fa';

const getTypeIcon = activity => {
    if (activity.activityType.typeId == 5) {
        return <FaBiking/>
    } else if (activity.activityType.typeId == 15) {
      return <FaRunning/>
 }

getTypeIcon(activity)
```

Due to the fact the duration is given in seconds and i wanted it to display like 1h 10m 12s there is also a need for a workaround which looks like the following:
```js
const secondsToHms = (s) => {
    const hours = (((s - s % 3600) / 3600) % 60)
    const minutes = (((s - s % 60) / 60) % 60)  
    const seconds = (s % 60)  
    return (`${hours}h ${minutes}min ${seconds}s`)
 }
secondsToHms(activity.duration)
```

### Displaying  a Map with "react-leaflet"

Then i created a small map with [react-leaflet](https://react-leaflet.js.org/ "react-leaflet") displaying 
- the track as polyline,
- the start point and
- the end point.

Therefore i created a new map-component:
```js
import React, { useEffect, useState } from "react"
import { Marker, MapContainer, TileLayer, LayersControl, Polyline } from "react-leaflet";

const Map = (data) => {
  const geo = data.data
  const style= { 
    color: '#11a9ed',
    weight: "5"
  }

  const bounds = [[geo.maxLat, geo.maxLon], [geo.minLat, geo.minLon]]
  return (
    <MapContainer
      style={{ height: "500px", width: "100%" }}
      bounds={bounds}       
      scrollWheelZoom={false}
    >
    <LayersControl position="topright">
      <LayersControl.BaseLayer checked name="OpenStreetMap.Mapnik">
        <TileLayer 
          url='https://{s}.basemaps.cartocdn.com/rastertiles/voyager/{z}/{x}/{y}{r}.png'
          attribution ='&copy; <a href="https://www.openstreetmap.org/copyright">OpenStreetMap</a> contributors &copy; <a href="https://carto.com/attributions">CARTO</a>'

        />
        </LayersControl.BaseLayer>
        <LayersControl.BaseLayer name="Esri World Imagery">
          <TileLayer
            attribution='Tiles &copy; Esri &mdash; Source: Esri, i-cubed, USDA, USGS, AEX, GeoEye, Getmapping, Aerogrid, IGN, IGP, UPR-EGP'
            url="https://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/tile/{z}/{y}/{x}"
          />
        </LayersControl.BaseLayer>

        <Marker id="start" position={geo.startPoint}/>
        <Polyline pathOptions={style} positions={geo.polyline} />
        <Marker id="end" position={geo.endPoint}/>

      </LayersControl>
    </MapContainer>
  );
};

export default Map;
``` 
Luckily the data from the Garmin Connect has already exactly the structure we need to create the map, which are the coordinates for the polyline and the two points.

The coordinates can be found in **geoPolylineDTO** in the activitydetails 

![activity-details.png](/uploads/activity_details_a6d325373e.png)

With maxLat, maxLon, minLat and minLon i created the bounds which will set the default view for the map when passed to the MapContainer.
```js
const bounds = [[geo.maxLat, geo.maxLon], [geo.minLat, geo.minLon]]

<MapContainer
      style={{ height: "200px", width: "100%" }}
      bounds={bounds}       
      scrollWheelZoom={false}
 >
.
.

```
Then i added the LayerControl to be able to toggle between the two Tilelayers
- OSM (Carto) and
- Aerial Image (Esri)

After that i just created two markers and a polyline with the exisiting objects ```startPoint```, ```endPoint``` and ```polyline```.
```js
.
.
        <Marker id="start" position={geo.startPoint}/>
        <Polyline pathOptions={style} positions={geo.polyline} />
        <Marker id="end" position={geo.endPoint}/>
.
.
```

You can find several other tilelayers in [leaflet-providers-preview](/leaflet-extras-github/leaflet-providers/preview/ "leaflet-providers-preview").

For an example of the preview head over to [/activities](/activities  "activitites").
You can find the code for the [activity-preview](https://github.com/DaTurboD/mxd-codes-frontend/blob/v2/components/activity/activity-preview/activity-preview.js "activity-preview") in my github-repositiory.

For the actual post of the activity i made the map a bit larger and added some more metrics for now.

(osm)
![activity-post.png](/uploads/activity_post_bd91b407f4.png)

(Aerial view)
![activity-post-aerial.png](/uploads/activity_post_aerial_11e09d89bf.png)



Fetching and storing activities from Garmin Connect with Strapi and visualizing them with NextJS

Prism.js is a compact, expandable syntax highlighter that was developed with modern web standards in mind.

To be able to use Prism.js with Next.js, the first thing you have to do is to add Prism.js to your project with

```npm install prismjs``` 

or

 ```yarn add prismjs```


Afterwards you need to import Prism.Js into all pages where you want code to be highlighted
I am for example importing it into a layout component which wraps the whole web app:

```js
 // in Layout.js
  import React,  { useEffect } from "react"
  const prism = require("prismjs")
  require('prismjs/components/prism-python');

  function Layout(props) {
    useEffect(() => {
      prism.highlightAll();
    }, []);
    
    // ...
  }
```

With this, all code tags are highlighted via a react hook. 
I also imported highlighting for Python because it's not included by default.

Now the only thing missing is the styling theme, which can simply be imported into ``` _app.js``` with

```js
 // in _app.js

  import "prismjs/themes/prism-tomorrow.css";
  
  export default function App({ Component, pageProps }) {
    return <Component {...pageProps} />;
  }
```

Instead of 'prism-tomorrow' you could also import the [default](https://prismjs.com/download.html#themes=prism&languages=markup+css+clike+javascript "default")-one or for example `[prism-okaidia](https://prismjs.com/download.html#themes=prism-okaidia&languages=markup+css+clike+javascript "prism-okaidia"). 

Have a look at [https://prismjs.com/download.html](https://prismjs.com/download.html "https://prismjs.com/download.html") to find all premade themes. Of course you can also create a custom theme.


Syntax Highlighting with Prism.js and Next.js

This article shows you how can host your Next.js site on a (virtual private) server with Nginx and a CI/CD pipeline via PM2 and Github Webhooks.

## Setup your server

First of all you need an server with root access.
I strongly recommend to have a look at the guide ["Initial Server Setup with Ubuntu 18.04"](https://www.digitalocean.com/community/tutorials/initial-server-setup-with-ubuntu-18-04 "Initial Server Setup with Ubuntu 18.04") from the DigitalOcean community which will lead you through the process of:

1. Logging in and set up root user access to your server with SSH
2. Creating a new user
3. Granting Administrative Privileges to the new user
4. Setting up a basic firewall
5. Giving your regular user access to the server with SSH key authentication.

After you have done that you can continue by installing all necessary dependencies on your server.

### Install Node.js

Again there is an guide by DigitalOcean which will help you [installing Node.js using PPA](https://www.digitalocean.com/community/tutorials/how-to-install-node-js-on-ubuntu-18-04#installing-using-a-ppa "Installing Node.jsUsing a PPA"). 

After completing 
+ **install Node.js, NPM and**
+ **the "build-essential package"**

you will have to change npm's default directory.

+ Create a `.npm-global` directory and set the path to this directory for `node_modules`:

```bash
cd ~
mkdir ~/.npm-global
npm config set prefix '~/.npm-global'
```

+ Create (or modify) a `~/.profile` and add the following line:
```bash
sudo nano ~/.profile
```

```bash
# set PATH so global node modules install without permission issues
export PATH=~/.npm-global/bin:$PATH
```
Now you have to update your system variables:

```bash
source ~/.profile
```

Now you should be able to check your installed Node.js version with:
 ```bash
node -v
```

### Install git

Check if git is already installed with:
```bash
git --version
```

If it isn't installed yet you can install it with

```bash
sudo apt install git
```

and configure Git with 

```bash
git config --global user.name "Your Name"
git config --global user.email "youremail@domain.com"
```
After git is installed and configured you can deploy your project by cloning it from Github.

## Deploy from Github

It is important that you are loggin in as **non-root** user for the following steps.

```bash
cd ~
git clone https://github.com/your-name/your-project-repo.git path
```

Create a `.env`  on the server if you are using one locally and copy/paste your content.

After you have deployed your project (optionally with environment variables) you can install all dependencies and build your Next.js site with:
```bash
cd ./my-project/
npm install
NODE_ENV=production npm run build
```

Now you should have a copy of your local project/Next.js site on your remote server.

Next you are going to setup PM2 which will be used to keep your site alive and restart it after every reboot.

## Setup PM2

You can install PM2 with:
```bash
npm install pm2@latest -g
```

You will need to create/configure an `ecosystem.config.js` file which will restart the default Next.js server.
```bash
cd ~
pm2 init
sudo nano ecosystem.config.js
```

Copy/paste the template and replace the content.
```js
module.exports = {
  apps: [
    {
      name: 'next-site',
      cwd: ' /home/your-name/my-nextjs-project',
      script: 'npm',
      args: 'start',
      env: {
        NEXT_PUBLIC_...: 'NEXT_PUBLIC_...',
      },
    },
    // optionally a second project
],};
```

With 

```bash
cd ~
pm2 start ecosystem.config.js
```

you can start your server which will run on the `Port 1337`.

 You can always check the status with:

```bash
pm2 status next-site
```

After the server reboots this PM2 should be always automatically be restarted. For that you are going to need a small Startup script which you can also copy/paste.

+ Generate and configure a startup script to launch PM2:

```bash
cd ~
pm2 startup systemd

[PM2] Init System found: systemd
[PM2] To setup the Startup Script, copy/paste the following command:
**sudo env PATH=$PATH:/usr/bin /usr/lib/node_modules/pm2/bin/pm2 startup systemd -u your-name --hp /home/your-name**
```

+ Copy/paste the generated command:

```bash
**sudo env PATH=$PATH:/usr/bin /usr/lib/node_modules/pm2/bin/pm2 startup systemd -u your-name --hp /home/your-name**

[PM2] Init System found: systemd
Platform systemd

. . .


[PM2] [v] Command successfully executed.
+---------------------------------------+
[PM2] Freeze a process list on reboot via:
   $ pm2 save

[PM2] Remove init script via:
   $ pm2 unstartup systemd
```

+ And save the new PM2 process list and environments. Then Start the service with `systemctl`.

```bash
pm2 save

[PM2] Saving current process list...
[PM2] Successfully saved in /home/your-name/.pm2/dump.pm2
```

If you reboot your server now with `sudo reboot` the script should be automatically restart your Next.js site. Give it a try!

## Setup Github Webhook

One thing missing now is an continuos integration and continuos delivery (CI/CD) pipeline which you will setup using Github webhooks. 

Therefore you need to create a new Webhook in your repository.

The following articles provide additional information to the steps below: 
+ [GitHub Creating Webhooks Guide](https://docs.github.com/en/developers/webhooks-and-events/webhooks/creating-webhooks#setting-up-a-webhook "GitHub Creating Webhooks Guide") and 
+ [DigitalOcean Guide to GitHub WebHooks](https://www.digitalocean.com/community/tutorials/how-to-use-node-js-and-github-webhooks-to-keep-remote-projects-in-sync "DigitalOcean Guide to GitHub WebHooks"). 

You need to create a server script which will do something if it is triggered by the Github webhook.

```bash
cd ~
mkdir NodeWebHooks
cd NodeWebHooks
sudo nano webhook.js
```

The script is going to create a server running on `Port 8100`. 
(Your Github webhook should be of course sending the webhook to something like `http://server-ip:8100`.)

If it gets triggered by a webhook it will 
+ go into your repo `~/my-nextjs-project/`, 
+ pull the latest commits, 
+ install all dependencies, 
+ build a new version of the site and 
+ restart the server via the PM2 script.

```js
const secret = "your-secret-key";
const repo = "~/my-nextjs-project/";

const http = require('http');
const crypto = require('crypto');
const exec = require('child_process').exec;

const BUILD_CMD = 'npm install && NODE_ENV=production npm run build';
const PM2_CMD = 'pm2 restart next-site';

http.createServer(function (req, res) {
    req.on('data', function(chunk) {
        let sig = "sha1=" + crypto.createHmac('sha1', secret).update(chunk.toString()).digest('hex');

        if (req.headers['x-hub-signature'] == sig) {
            exec('cd ' + repo + ` && git pull && npm install && ${BUILD_CMD} && ${PM2_CMD}`);
        }
    });

    res.end();
}).listen(8100);
```

You will need to allow communication on `Port 8100` with:

```bash
sudo ufw allow 8100/tcp
sudo ufw enable

Command may disrupt existing ssh connections. Proceed with operation (y|n)? y
Firewall is active and enabled on system startup
```

Earlier you setup PM2 to restart the services (your Next.js site) whenever the server reboots or is started. You will now do the same for the webhook script.

- Run echo $PATH and copy the output for use in the next step.

```bash
echo $PATH

/home/your-name/.npm-global/bin:/home/your-name/bin:/home/your-name/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
```

+ Create a webhook.service file:

```bash
cd ~
sudo nano /etc/systemd/system/webhook.service
```

+ In the editor, copy/paste the following script, but make sure to replace your-name in two places with your username. Earlier, you ran echo $PATH, copy this to the Environment=PATH= variable, then save and exit:

```bash
[Unit]
Description=Github webhook
After=network.target

[Service]
Environment=PATH=your_path
Type=simple
User=your-name
ExecStart=/usr/bin/nodejs /home/your-name/NodeWebHooks/webhook.js
Restart=on-failure

[Install]
WantedBy=multi-user.target
```

+ Enable and start the new service so it starts when the system boots:

```bash
sudo systemctl enable webhook.service
sudo systemctl start webhook
```

+ Check the status of the webhook:

```bash
sudo systemctl status webhook
```

You can test your webhook with these [instructions](https://www.digitalocean.com/community/tutorials/how-to-use-node-js-and-github-webhooks-to-keep-remote-projects-in-sync#step-4-testing-the-webhook "Test Webhook").

The Next.js server is now running on *your-ip:3000* and you implemented a CI/CD pipeline via PM2 and Github Webhooks but you still can't access your website via a domain because you need to configure a webserver like Nginx.

## Configure Nginx

I am using Cloudflare to manage DNS for my domains but you can do this with every other provider also.

+ Create two `A` Records which will point `your-domain.com` and `www.your-domain.com` to the IP-adress of your server.

After that you will need to configure Nginx. 

_The following instructions are based on [How To Install Nginx on Ubuntu 18.04 [Quickstart]](https://www.digitalocean.com/community/tutorials/how-to-install-nginx-on-ubuntu-18-04-quickstart "How To Install Nginx on Ubuntu 18.04 [Quickstart]")._

+ Update your local package index:

```bash
sudo apt update
```

+ install Nginx:

```bash
sudo apt install nginx
```

+ and adjust the Firewall:

```bash
sudo ufw allow 'Nginx Full'
sudo ufw delete allow 'Nginx HTTP'
```

You should now be able to see the Nginx landing page on:

```
http://your_server_ip
```

### Setting up Server Blocks

+ Create the directory for ```your-domain.com```, using the -p flag to create any necessary parent directories:

```bash
sudo mkdir -p /var/www/your-domain.com/html
```
+ Assign ownership of the directory:

```bash
sudo chown -R $USER:$USER /var/www/your-domain.com/html
```

+ The permissions of your web roots should be correct if you haven’t modified your `umask` value, but you can make sure by typing:

```bash
sudo chmod -R 755 /var/www/example.com
```

+ Make a new server block at `/etc/nginx/sites-available/your-domain.com`: 

```bash
sudo nano /etc/nginx/sites-available/example.com
```

+ Copy/Paste the following nextjs-nginx configuration and update the server_name sections:

```bash
server {
    # Listen HTTP
    listen 80;
    listen [::]:80;

    server_name your-domain.com www.your-domain.com;

    # Redirect HTTP to HTTPS
    return 301 https://$host$request_uri;
}

server {
    # Listen HTTP
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name your-domain.com www.your-domain.com;

    # SSL config
    include snippets/self-signed.conf;
    include snippets/ssl-params.conf;

    # Proxy Config
    location / {
        proxy_pass http://localhost:3000
        proxy_http_version 1.1;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Server $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Host $http_host;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_pass_request_headers on;
    }
    location ~ /.well-known {
       allow all;
    }
}
```

Save the file and close it when you are finished.

+ Enable the file by creating a link from it to the sites-enabled directory:

```bash
sudo ln -s /etc/nginx/sites-available/your-domain.com /etc/nginx/sites-enabled/
```

+ Test for syntax errors:

```bash
sudo nginx -t
```

+ and finally enable the changes:
```shell
sudo systemctl restart nginx
```

Nginx should now be serving content on your domain name. That means if you have a look at `http://your-domain.com` you should see your Next.js site.


In the end should deny traffic to `Port 3000` with: 

```bash
cd ~
sudo ufw deny 3000
```

+ To install SSL, you will need to [install and run Certbot by Let's Encrypt](https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-18-04 "install and run Certbot by Let's Encrypt") .


_This guide is also using parts of [Strapi Deployment on DigitalOcean](https://strapi.gitee.io/documentation/3.0.0-beta.x/deployment/digitalocean.html "Strapi Deployment on DigitalOcean") which helped me a lot setting up Strapi and Next.js on a server in a proper way._ 

Hosting NextJS on a private server using PM2 and Github webhooks as CI/CD

With Google Cloud (Build) you can automate your whole workflow from building your Gatsby site up to deploying your site to Firebase hosting.

**What will you need?**

+ [Firebase Hosting](https://firebase.google.com/products/hosting "Firebase Hosting")
+ [Cloud Build](/cloud-google/cloud-build "Cloud Build")
+ [Cloud Scheduler](/cloud-google/scheduler "Cloud Scheduler") (optional)

## Firebase Hosting

To set up Firebase you will need an Google Cloud Account which has billing enabled and at least one project.
You can add the Firebase SDK with:

```
npm install --save firebase
```

and configure it with

```
firebase init
```

In the process you can setup a new project which will create a new ```.firebaserc``` file if it doesn't exist yet and enable **Hosting**.

![Firebase_init_1.png](/uploads/Firebase_init_1_c3dd56b33d.png)

As public directory you have to set your **public** folder. After you have set your public folder you can reject all oncoming messages.

![Firebase_init_2.png](/uploads/Firebase_init_2_224ab0a007.png)

> You can also have a look at the [Google Firebase Docs](https://firebase.google.com/docs/cli#install-cli-windows "Google Firebase Docs") for setting up Firebase.

In the end it will also create a **firebase.json** where you can copy/paste the following to optimize your hosting for a Gatsby site.

```js
{
  "hosting": {
    "public": "public",
    "ignore": [
      "firebase.json", 
      "**/.*", 
      "**/node_modules/**"],
    "headers": [
      {
        "source": "**/*",
        "headers": [
          {
            "key": "cache-control",
            "value": "cache-control: public, max-age=0, must-revalidate"
          }
        ]
      },
      {
        "source": "static/**",
        "headers": [
          {
            "key": "cache-control",
            "value": "public, max-age=31536000, immutable"
          }
        ]
      },
      {
        "source": "**/*.@(css|js)",
        "headers": [
          {
            "key": "cache-control",
            "value": "public, max-age=31536000, immutable"
          }
        ]
      },
      {
        "source": "sw.js",
        "headers": [
          {
            "key": "cache-control",
            "value": "cache-control: public, max-age=0, must-revalidate"
          }
        ]
      },
      {
        "source": "page-data/**",
        "headers": [
          {
            "key": "cache-control",
            "value": "cache-control: public, max-age=0, must-revalidate"
          }
        ]
      }
    ],
  }
}
```
When you have build your Gatsy site and **public** folder is present you can upload it to your Firebase Hosting with:

```
firebase deploy
```

If you want to use your custom domain (for example gis-netzwerk.com) you have to go to Hosting in the Firebase console where you will find DNS-records to point your domain to your Firebase Hosting.
You can also create a second domain (for example www.gis-netzwerk.com) to redirect automatically to your root domain.

In your project settings you can find your Firebase configurations which look like:

```js
const firebaseConfig = {
  apiKey: "apiKey",
  authDomain: "{.firebaseapp.com",
  databaseURL: "https:// projectId.firebaseio.com",
  projectId: " projectId",
  storageBucket: " projectId.appspot.com",
  messagingSenderId: "1",
  appId: "2",
  measurementId: "G-123"
};
```

You probably want to save these as environment variables.

## Cloud Build

To create a CI-/CD-Pipeline you have to activate Cloud Build for your account. The console itself is quite clear in comparison to AWS CodeBuild.
You will see Dashboard which is displaying some basic informations, the history of your Cloud Builds, Triggers and Options.

You just will have to create a new trigger which will trigger a new build everytime new content is pushed to your linked GitHub repository..

First of all you have to connect your repository. After that you can create the trigger.

![Cloud-Build_Trigger-settings.png](/uploads/Cloud_Build_Trigger_settings_c6be399330.png "Cloud-Build_Trigger-settings.png")

Important options are:
+ Event: **Push to branch**
+ Branch: **^master$** (or whatever branch you want to build)
+ Cloud Build Configuration file: **cloudbuild.yaml**

After that you can create all environment variables as **Substitution variables**.
You will notice that all variables have to start with an underscore.
Because of that we will need a small workaround in the cloudbuild.yaml configuration file.

For now you can just create your substitions variables and add the underscore to your default variable names.

To be able to deploy via firebase you will need to authorize Firebase with a '$_TOKEN'.
You can retrive this token on your local machine with_
```
firebase login:ci
```

A new page will be opened in your prefered browser where you will have to login with your Google account to get the token.
Once you have the token you can also add it as substition variable.
```
_TOKEN : {TOKEN VALUE}
```

If you have created all substitution variables you can check again if they are inserted correct and create the new trigger.

### Cloudbuild.yaml

Cloud build needs the cloudbuild.yaml to know what it should do.
If you have inserted the path like above mentioned you will need a cloudbuild.yaml in your root directory.

You can copy the following into it:
```js
steps: 
# Install dependencies
  - name: node:10.16.0
    id: Installing dependencies...
    entrypoint: npm
    args: ["install"] 
    waitFor: ["-"] # Begin immediately

# Install Firebase   
  - name: node:10.16.0 
    id: Installing Firebase...
    entrypoint: npm 
    args: ["install", "firebase-tools"]
    waitFor:
      - Install dependencies...

# Create file with env-variables
  - name: node:10.16.0
    id: Creating Envirnonment variables...
    entrypoint: npm
    args: ["run", "create-env"]
    env:
    - "CLIENT_EMAIL=${_CLIENT_EMAIL}"
    - "PRIVATE_KEY=${_PRIVATE_KEY}"
    - "MAIL_CHIMP=${_MAIL_CHIMP}"
    - "GA_ID=${_GA_ID}"
    - "GA_VIEW_ID=${_GA_VIEW_ID}"
    - "IG_TOKEN=${_IG_TOKEN}"
    - "FIREBASE_API_KEY=${_FIREBASE_API_KEY}"
    - "FIREBASE_APP_ID=${_FIREBASE_APP_ID}"
    - "FIREBASE_AUTH_DOMAIN=${_FIREBASE_AUTH_DOMAIN}"
    - "FIREBASE_DB_URL=${_FIREBASE_DB_URL}"
    - "FIREBASE_MEASUREMENT_ID=${_FIREBASE_MEASUREMENT_ID}"
    - "FIREBASE_MESSAGE_SENDER_ID=${_FIREBASE_MESSAGE_SENDER_ID}"
    - "FIREBASE_PROJECT_ID=${_FIREBASE_PROJECT_ID}"
    - "FIREBASE_STORAGE_BUCKET=${_FIREBASE_STORAGE_BUCKET}"
    - "GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES=true"
    waitFor: ["-"] # Begin immediately

# Gatsby build
  - name: node:10.16.0
    id: Building Gatsby site...
    entrypoint: npm
    args: ["run", "build"]
    waitFor:
      - Installing dependencies...
      - Creating Envirnonment variables...

# Deploy
  - name: node:10.16.0 
    id: Deploying to Firebase...
    entrypoint: "./node_modules/.bin/firebase" 
    args: ["deploy", "--project", "$PROJECT_ID", "--token", "$_TOKEN"]
    waitFor:
      - Installing Firebase...
      - Building Gatsby site...
    
timeout: 30m0s
```
The cloudbuild.yaml is basically divided into six parts which will
+ Install dependencies,
+ Install Firebase,
+ Create file with env-variables,
+ Build the Gatsby site and
+ Deploy it to Firebase

As you can see you will create a file with the environment variables which will map the substitutions variables with your "default" variables.
Everything else should be pretty self-explanatory.

> Cloud Build will timeout the build by default after 10 minutes. So if your build is gonna take longer you will have to set up a custom timeout like in the cloudbuild.yaml above. You can also set a timeout for each step.

Another import point is that you will have to add the Plugin for Google Analytics Reporting Api as dynamic plugin like for example the following because otherwise you will get errors during your build.
```js
if (
  process.env.CLIENT_EMAIL &&
  process.env.PRIVATE_KEY &&
  process.env.GA_VIEW_ID
) {
  const startDate = new Date()
  startDate.setMonth(startDate.getMonth() - 3)
  dynamicPlugins.push(
    /*{
      resolve: `gatsby-plugin-guess-js`,
      options: {
        GAViewID: process.env.GA_VIEW_ID,
        jwt: {
          client_email: process.env.CLIENT_EMAIL,
          private_key: process.env.PRIVATE_KEY.replace(/\\n/g, "\n"),
        },
        period: {
          startDate,
          endDate: new Date(),
        },
      }
    },*/
    {
      resolve: `gatsby-source-google-analytics-reporting-api`,
      options: {
        email: process.env.CLIENT_EMAIL,
        key: process.env.PRIVATE_KEY.replace(/\\n/g, "\n"),
        viewId: process.env.GA_VIEW_ID,
        startDate: `2009-01-01`,
      }
    }
  )
}

module.exports = {
  plugins: [
    plugins...
  ].concat(dynamicPlugins),
};

Due to the fact that Code Build uses Linux machines small and capital letters are important (Windows doesnt care).
That means if you import a component like
```js
import MyComponent from "../Mycomponent"
```
and the actual folder name is ```MyComponent``` your build will fail.

## Speeding up your builds

+ **Image optimization**

If you aren't using preoptimized images yet you should consider to crop, resize images etc. **before** you will build the site because it can basically save a lot of time (depending on the amount of images).

Before i optimized my images a build on Google Cloud took about ~ 1800 sec.
After i optimized all my images for posts with Python ([Image optimization with Python](/articles/scaling-and-cropping-images-using-python "Image optimization with Python")) the build time went down to ~ 620 sec. So actually just a third.

## Set Cloud Scheduler (optional)

With Cloud Scheduler you can trigger a build automatically after a specific time.
The first 3 jobs a month a free and after that you have to pay $0.10 for every job after that.

(You can also use Cloud Scheduler with your default trigger (Push to branch) or without).

The trigger you created before has an ID and with a POST request you can start the trigger anytime you want.
To get the ID of the trigger you have to open the Cloud Shell, type ```gcloud beta builds triggers list``` and search for **id**. Copy that.

The URL for the POST request looks like (without []):
```
https://cloudbuild.googleapis.com/v1/projects/[PROJECT_ID]/triggers/[TRIGGER_ID]:run
```
Now you have to create a new job in Cloud Schedule.

![Cloud-Scheduler-Settings.png](/uploads/Cloud_Scheduler_Settings_1d117b6db6.png)

```
0 3 * * *
```
will trigger a build every day at 3 am MESZ. 

The URL for the POST request looks like (without []):
```
https://cloudbuild.googleapis.com/v1/projects/[PROJECT_ID]/triggers/[TRIGGER_ID]:run
```

In Text you will need
```json
{
  "branchName": "master"
}
```
and you will authorize the job with your service account.



Build and deploy your Gatsby site with Google Cloud Build to Firebase

Since I used a lot of pictures (and also very large ones in the beginning), this had a huge impact on the speed of the page.
Since PageSpeed ​​is a not unimportant ranking factor for search engines like Google and Co, you should of course make a page as fast as possible.

The images used here are mainly satellite images from [ESA](http://www.esa.int "ESA"), which are licensed under the [CC BY-SA 3.0 IGO](https:/https://creativecommons.org.org/licenses/by-sa/3.0/igo/) IGO) and may therefore also be used for your own purposes under certain conditions.

These pictures are often ~ 30MB, which is a bit too big for a website.
Since I didn't want to crop all pictures manually, I decided to solve this problem with Python or Pillow.

## Pillow library
[Pillow](/pillow/en/stable/) is a Python library for image processing that can be installed on with (assuming you have Python already installed)
```py
pip install Pillow
```

Finally you can import the library with
```py
from PIL import Image
```
into your Python script.

All images for posts are in a separate "images /" folder in the root directory of the project.

First, all ".jpg" files in a certain directory are opened with Pillow and all file names are saved in an array. In addition, a variable is required and can later access any name in the array.
```py 
count = 0
image_list = []

for file in glob.iglob('path/to/images/*.jpg'):
    im=Image.open(file)
    image_list.append(os.path.basename(file))
```

## Define sizes and calculate aspect ratio

Now you should know to which sizes the pictures should be cut and whether, for example, the proportions should be retained.
For all "PostCover" (pictures in posts) the aspect ratio is ignored and the picture is simply cut to a certain size, which is declared in a global variable.
```py
size = (1903,453) #(width,height)
```
With all "PostThumbnails" (picture preview) the aspect ratio should be retained and only scaled smaller. A global standard width is defined for this.
```py
basewidth = 500
```
Then the original width and height of the images are determined, as we need them to be able to calculate and maintain the aspect ratio.
Only the new height is needed here, as the standard width has already been predefined.
```py
    width, height = im.size
    wpercent = (basewidth / float(im.size[0]))
    hsize = int((float(im.size[1]) * float(wpercent)))
```

## Cropping and Rescaling

Now you can cut the images with ```Image.crop``` or scale them with ```Image.resize```. The new width "basewidth" and the calculated height "hsize" are now used as parameters for scaling.

```py
    imThumbnail = im.resize((basewidth, hsize), Image.LANCZOS)
    imCover = im.crop(((width-size[0])//2, (height-size[1])//2, (width+size[0])//2, (height+size[1])//2))
```

Then I renamed the thumbnail and saved both new files under ```static/assets``` with a quality of "85" . 
With the additional parameter "optimize = True" a few KB can be saved.
```py
    newCover = 'static/assets/{}'.format(image_list[count])
    newThumbnail = 'static/assets/{}_thumbnail.jpg'.format(image_list[count].replace(".jpg", ""))
    imCover.save(newCover,optimize=True,quality=85)
    imThumbnail.save(newThumbnail,optimize=True,quality=90)
    count +=1 
```

Complete script:
```py
from PIL import Image
import glob, os

count = 0
image_list = []
basewidth = 500
size = (1903,453)
 

for file in glob.iglob('path/to/images/*.jpg'):
    im=Image.open(file)
    image_list.append(os.path.basename(file))
    width, height = im.size
    wpercent = (basewidth / float(im.size[0]))
    hsize = int((float(im.size[1]) * float(wpercent)))
    imThumbnail = im.resize((basewidth, hsize), Image.LANCZOS)
    imCover = im.crop(((width-size[0])//2, (height-size[1])//2, (width+size[0])//2, (height+size[1])//2))
    newCover = 'static/assets/{}'.format(image_list[count])
    newThumbnail = 'static/assets/{}_thumbnail.jpg'.format(image_list[count].replace(".jpg", ""))
    imCover.save(newCover,optimize=True,quality=85)
    imThumbnail.save(newThumbnail,optimize=True,quality=90)
    count +=1 
```

## Run script automatically

In order not to have to run the script manually every time, you can add the following to "package.json".
```json
    "img-optimize": "py ./src/utils/resize_images.py"
```
So you can optimize all images automatically with ```npm run img-optimize```.


Scaling and Cropping images using Python

## Remote Sensing

Navigation devices, smartphones and weather forecasts are dependent on satellites and without these we have to rely on some services that make our everyday life easier.

Images of the Earth from satellites or aircraft are constantly being recorded. These remote sensing data often have a resolution of up to 30cm, are recorded in a range from 450nm to 2273nm and are mostly referenced by the operators of the satellites.

These pictures are then sold or even provided by many providers free of charge.

## Copernicus Open Access Hub

On [Sentinel Open Access Hub](https://scihub.copernicus.eu/dhus/#/home) you can find free products from the [Copernicus Program](/d-copernicus), those from the [European Union](https://europa.eu/european-union/index_de) and others operated by the [European Space Agency](http://www.esa.int/ESA) (ESA).

_The Copernicus program basically comprises six satellites ([Sentinel-1](/wikipeadi/wiki/Sentinel-1), [Sentinel-2](https://www.wikipedia.com/wiki/Sentinel-2), [Sentinel-3](https://www.wikipedia.com/wiki/Sentinel-3), [Sentinel-4](https://de.wikipedia.org/wiki/Sentinel-4 ) (planned start in 2021), [Sentinel-5](https://www.wikipedia.com/wiki/Sentinel-5) and Sentinel-6 (planned start in late 2020) ._

_All of these satellites perform different tasks and help to observe land, sea and the atmosphere._

In the Copernicus Open Access Hub, you can now download all the data provided after registering free of charge. All available data is displayed for download via search criteria and a desired image section, which you can simply draw with a rectangle

**ArcGIS currently supports level 1C products.**

These image files are relatively large and it may take a while (depending on the internet bandwidth) before the ZIP file is downloaded.

## Image classification

These multispectral bands can then be integrated in Arc-GIS or QGIS.

A classification now consists of two main components.

1. Generate [Training Samples](https://pro.arcgis.com/de/pro-app/help/analysis/image-analyst/training-samples-manager.htm) and
2. [Classification](https://data-science-blog.com/blog/2017/12/20/maschine-earning-classification-vs-regression/) based on the training samples and [validation](https://towardsdatascience.com/supervised-machine-learning-model-validation-a-step-by-step-approach-771109ae0253)

### Generate_Training samples

First, a classification method must be selected (e.g. supervised and pixel-based). This classification method divides individual pixels of the satellite image into thematic classes, e.g.

* Settlement
* Forest
* Water
* Meadow etc.

Here the system is taught ([Machine Learning](https://www.wikipedia.com/wiki/Maschinelles_Learning)) that, for example, a green pixel stands for the forest class, blue for water, light green for meadow and gray for settlement ,

In order to get the most realistic result possible, one should choose areas / pixels that are as clear as possible for these samples. That For example, there shouldn't be a gray pixel in the class.

When the assignment of samples for each class is done, the actual classification now begins.

### Classification based on training Samples and validation

There are various classification methods that can be used e.g. [Maximum likelihood](https://www.wikipedia.com/wiki/Maximum-Likelihood-Methode). The classification method now only has to be selected and the classification can then be started.

In order to check the quality of the classification, the results are usually validated with e.g. the [_ground truthing._](https://www.wikipedia.comg/wiki/Ground_Truth) method.

Then it is checked how many points were classified in the correct class (for example, a settlement was recognized as a settlement). Here one should not be too stingy with the _Validierungssamples_ to achieve a meaningful validation of the image classification.

This can be noted in an Excel list and the overall accuracy of the classification can be easily calculated. Of course you have to write down the wrong and correct classification.

And that's it.

In short, remote sensing is the extraction of geodata or satellite images by satellites and the subsequent methodology (image classification) for evaluating this remote sensing data about the nature of the earth's surfac

Remote Sensing and Image Classification

## FME Home license

FME (Feature Manipulation Engine) is a powerful and the most used spatial ETL tool for the migration and processing of [spatial data](/articles/what-are-geodata "What is geodata?") and non-spatial data. The software is very flexible and powerful. It can also handle very large amounts of data without problems.

Feature Manipulation Engine supports over 300 different data sources such as [GIS](/articles/what-is-gis-geographic-information-system "What is GIS?") - databases] (PostGIS, MySQL, Oracle, of course also most non-spatial databases), CAD files ( DWG, DXF), raster data, web services, coordinate lists, XML, KML, GML, GeoJSON and much more.

The software is very easy to use and it comes with a nice graphical user interface, in which the source and target data model or format is specified. This also makes complex processing processes very clear.

In between the readers and writers, countless so-called [transformers](https://cdn.safe.com/resources/fme/FME-Transformer-Reference-Guide.pdf "FME Transformers") can be used, with which the data can be processed before being imported into a new data source. This workflow can also be supplemented with Python or SQL scripts.

Safe Software offers FME in three versions:

*   FME Desktop
*   FME Server
*   FME Cloud

That sounds great, but you still have no idea?

You can apply for a free (home) license for FME Desktop at [Free Licenses for Home Use](https://www.safe.com/free-fme-licenses/home-use/ "Free Licenses for Home Use"). If you are a student you can apply for a separate license [here](https://www.safe.com/free-fme-licenses/students/).

**This license is of course only for personal and not commercial projects.**

Submitting the application is very easy. All you have to do is enter your name, your email address, your company if applicable, and how you will use the license. Here it is enough to simply write that you want to get to know the program and of course want to learn it.

As soon as the application has been accepted, you will receive an email with the license key.

On the page [Downloads](https://www.safe.com/support/support-resources/fme-downloads/ "Downloads") you can then download the desktop version and enter the license key you received after the installation process. The license is valid for one year (four months for students) and can be expanded as required.

There is a [Knowledge Base](https://community.safe.com/page/knowledge-base) where you can find thousands of tutorials.

For more complex problems, it is also advisable to take a look at [gis-stackexchange.com](https://gis.stackexchange.com/).


Free FME-licence for private use

With the free [contingent for AWS](https://aws.amazon.com/en/free/free-tier/ "contingent for AWS") you always get an active [AWS code pipeline](https://aws.amazon.com/codepipeline/?did=ft_card&trk=ft_card "AWS Codepipeline") per month and [100 minutes AWS codebuild](https://aws.amazon.com/codebuild/?did=ft_card&trk=ft_card "100 minutes AWS codebuild" ) per month.

So you can set up a continuous integration and continuous delivery pipeline for free or with more than 100 build minutes a month for relatively little money, which triggers a build with every push to a GitHub repository whoch will be automatically deployed to S3 and optionally also invalidate CloudFront Cache.

## CodeBuild

First of all you need a new build project in CodeBuild.
In the project configuration you can assign a name for it and select GitHub as the source provider under **Source**.

Depending on whether the repository is public or not, you then select "Public Repository" and enter the repository URL or you link your GitHub account and give CodeBuild the necessary rights to access the repository.

An environment image must now be selected under **Environment**. For GatsbyJS this would be a "managed image" and the operating system "Amazon Linux 2". "Standard" is selected as the runtime (s) and "aws / codebuild / amazonlinux2-x86_64-standard: 2.0" as the image.

![codebuild_umgebung.png](/uploads/codebuild_umgebung_e32ebcd3d2.png)

Now a new service role can be created automatically (which is required) so that CodeBuild has the necessary rights for the AWS account.

> This service role can also be assigned rights for CodePipeline, so that this service role can be used for CodeBuild and CodePipeline.
If environment variables are used, these can be specified under "Additional configuration" in the environment. You can also make sure that "3GB RAM, 2vCPUs" is really selected, since only this option is included in the free contingent.

**Buildspec** now uses a buildspec file in YAML format. For a Gatsby site this should somehow look like the following:
```sh
version: 0.2
phases:
    install:
        runtime-versions:
            nodejs: 12
        commands:
            - 'touch .npmignore'
            - 'npm install -g gatsby'
    pre_build:
        commands:
            - 'npm install'
    build:
        commands:
            - 'npm run build'
    post_build:
        commands:       
            - 'find public -type f -regex ".*\.$htm\|html\|txt\|text\|js\|css\|json$$" -exec gzip -f -k {} \' ## sofern cloudfront nicht automatisch die dateien komprimiert
artifacts:
    base-directory: public
    files:
        - '**/*'
    discard-paths: no
cache:
    paths:
        - '.cache/*'
        - 'public/*'
```

The buildspec.yml file only needs to be placed in the root directory so that CodeBuild can find it.
In addition, the build script must of course still be available in "package.json".

```js
"build": "gatsby build",
```

The default settings can be retained under **Artifacts**.
With CloudWatch you have the possibility to save logs for CodeBuild in an S3 bucket.

> There may be additional costs!

If all settings have now been entered correctly, the build project can be created.
The only thing missing now is the code pipeline that triggers a build and deployed it in the S3 bucket.

## CodePipeline

For this you switch to CodePipeline and create a new pipeline in which a name and a service role are selected first.

At Source you can now log in with a GitHub account and link the respective repository with a branch.
You now have two options to trigger a build.

+ GitHub-Webhooks and
+ AWS CodePipeline

Then you choose the build provider "AWS CodeBuild" and the previously created project or (if you have not already done this) create a new project.

After a build, the public/ folder can also be automatically deployed to an S3 bucket with AWS CodeDeploy.
_Alternatively_ you can also skip this step and [gatsby-plugin-s3](https://github.com/jarizhttps://www.gatsbyjs.com-plugin-s3 "gatsby-plugin-s3"), which also optimizes caching.

```ssh
npm i gatsby-plugin-s3
```
bzw.
```bash
yarn add gatsby-plugin-s3
```

Now the configuration in gatsby-config.js and the deployment script are missing

```js
plugins: [
  {
      resolve: `gatsby-plugin-s3`,
      options: {
          bucketName: 'my-website-bucket'
      },
  },
]
```
```js
"scripts": {
    ...
    "deploy": "gatsby-plugin-s3 deploy --yes"
}
```
The deployment script "npm run deploy" must then of course be added to the buildspec file under post-build commands.
The CodePipeline should now look something like this:

![CodePipeline.png](/uploads/Code_Pipeline_b8acf52436.png)

Experience shows that a build for around 100 pages takes around 10 minutes if you have some pictures per page.

Every time CodePipeline detects a push to the GitHub repository, a build is automatically triggered and made available on an S3 bucket.

With ```aws cloudfront create-invalidation --distribution-id DISTRIBUTION_ID --paths / * ``` the CloudFront cache can also be invalidated.

GatsbyJS with CI/CD Pipeline via Codebuild 

In general there are two possibilies to use GoogleAdsense on your GatsbyJS website:
+ **Auto Ads** and
+ custom **ad blocks**.

Depending on whether you choose to include Adsense ads on certain spots or whether you will leave this job to the Google AI, you can choose one/and or the other.

## Auto Ads

With Auto Ads, the optimal positions for an advertising banner are determined using a Google AI and a display ad is automatically switched there. All you have to do is place the following AdSense code in ``html.js``.
```html
<script data-ad-client="ca-pub-0037698828864449" async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script>
```
![adsense_auto-ads.png](/uploads/adsense_auto_ads_a296306c19.png "adsense_auto-ads.png")
and activate Auto ads in Adsense.


On GIS-Netzwerk.com I used Auto ads and I'm honestly surprised how well it works.
Ads are displayed in a text every few paragraphs and are also responsive.

### Ad loads

You also have the option of increasing or decreasing the number of ads in the settings.
Unfortunately, you cannot specify a specific number of ads.

![ad-load.png](/uploads/ad_load_f3a8f19537.png "adsense ad load")

In my opinion, a lot of ads are shown even when you set it to "min".
You can play around with the ad load and find out the best setting for your purposes.
Somestimes it can take a few minutes until the new ad load is effective.

### Display formats

You can also influence the ad formats.
Basically there are:
+ In-page ads (displayed in the main part of the page)
+ Matched content (tool for content promotion)
+ Anchor ads (mobile ads on the edge of the screen)
+ Vignette ads (mobile full screen display, at page transitions)

I have only deactivated anchor texts, because I personally find them very annoying.

![ad-formats.png](/uploads/ad_formats_71b13c9759.png "adsense ad formats")

In addition, you can also completely exclude individual pages from advertisements.

## Adsense with GatsbyJS

If you want to use Auto ads on your GatsbyJS page, you can do it super easily with the plugin [gatsby-plugin-google-adsense](https://www.gatsbyjs.com/packageshttps://www.gatsbyjs.com-plugin-google-adsense/ "gatsby-plugin-google-adsense").

**Install**
```bash
npm install --save gatsby-plugin-google-adsense
```
bzw.
```bash
yarn add gatsby-plugin-google-adsense
```
**modify gatsby-config.js**
```js
// In your gatsby-config.js file
plugins: [
    {
      resolve: `gatsby-plugin-google-adsense`,
      options: {
        publisherId: `ca-pub-xxxxxxxxxx`
      },
    },
]
```
The remaining settings can then be adjusted on [Adsense](https://www.google.de/adsense/ "Adsense").

## Ad units
In addition to auto ads, there is also the "classic" option of inserting individual ad units at specific positions.
With the React Component [react-adsense](/github/hustcc/react-adsense "react-adsense") you can insert Google AdSense and Baidu ads in any place.

```bash
npm install --save react-adsense
```
or
```bash
yarn add react-adsense
```

In order for the components to be rendered, you still need the AdSense script code. You can either insert this manually in the `` html.js '' file or, if you want to combine individual ad units with Auto ads, you can also use the plug-in already mentioned to insert the script.

> When auto ads and individual ad units are combined, the individual ad units always have a higher "priority". This means that all ad units that are inserted manually are usually also rendered and, if the text / ads ratio permits, additional ads from Auto ads are automatically inserted.

If the script has been integrated and react-adsense has been installed, you can use
```js
import React from 'react';
import AdSense from 'react-adsense';

// ads with no set-up
<AdSense.Google
  client='ca-pub-7292810486004926'
  slot='7806394673'
/>

// ads with custom format
<AdSense.Google
  client='ca-pub-7292810486004926'
  slot='7806394673'
  style={{ width: 500, height: 300, float: 'left' }}
  format=''
/>

// responsive and native ads
<AdSense.Google
  client='ca-pub-7292810486004926'
  slot='7806394673'
  style={{ display: 'block' }}
  layout='in-article'
  format='fluid'
/>

// auto full width responsive ads
<AdSense.Google
  client='ca-pub-7292810486004926'
  slot='7806394673'
  style={{ display: 'block' }}
  format='auto'
  responsive='true'
  layoutKey='-gw-1+2a-9x+5c'
/>
```
to insert components for the ad units.


The respective client id
```js
client='ca-pub-7292810486004926'
``` 
and the ad slot 
```js
slot='7806394673'
```
**must always be specified**.

The rest is optional.
```
Optional props:
    className:
    style:
    layout:
    layoutKey:
    format:
    responsive:
```

In case you have more questions there is also ad Adsense community where you can get some answers.
[Google AdSense Help Community](https://support.google.com/adsense/community "Google AdSense Help Community")

Using Google Adsense with GatsbyJS

With the ever increasing amount of data, the question arises at some point how it can be managed effectively. GIS databases are used here.

Spatial databases that are part of a [geographic information system](/articles/what-is-gis-geographic-information-system "geographic information system") can be used to store, manage and query [geodata](/articles/what-are-geodata).

## Free and open-source

1. ArangoDB Community Edition

ArangoDB is a user-friendly, powerful open source NoSQL database with a unique combination of functions.

[https://www.arangodb.com](https://www.arangodb.com)

1. PostGIS / PostgreSQL

PostGIS provides spatial objects for the PostgreSQL database that enable the storage and retrieval of information about location and assignment.

[https://postgis.net](https://postgis.net) / [https://www.postgresql.org](https://www.postgresql.org)

1. MariaDB

One of the most popular database servers. Developed by the original developers of MySQL.

[https://mariadb.org/](https://mariadb.org/)

1. MySQL

MySQL is one of the most widely used relational database management systems in the world. It is available as open source software and as a commercial enterprise version for various operating systems.

[https://www.mysql.com/](https://www.mysql.com/)

1. OrientDB

OrientDB is an open source NoSQL database written in Java. OrientDB is a document-oriented database that also has properties of graph databases

[https://www.orientdb.com](https://www.orientdb.com)

1. SQLite / SpatialLite

SpatiaLite is an open source library that extends SQLite with full spatial SQL functions.

[https://www.sqlite.org](/sqlite/index.html) / [https://www.gaia-gis.it/fossil/libspatialite](https://www.gaia-gis.it/fossil/libspatialite/index)

## Proprietary

1. Oracle Spatial

[https://www.oracle.com](/oracle/technetwork/database/options/spatialandgraph/downloads/index.html)

Oracle Spatial and Graph is a separately licensed component of the Oracle database. The extension is used to store and manage geographic information.

GIS- and Geo-Database Management System options

In order to be able to work with digital maps or information ([geodata](/articles/what-are-geodata "geodata")), a [geographic information system (GIS)](/articles/what-is-gis-geographic-information-system "What is GIS?") is used. With  GIS, geodata can be recorded, edited, analyzed and displayed appropriately. There are now many good providers of geographic information systems, the two best known of which are probably QGIS (Open Source) and ArcGIS from Esri.

Now you have decided on a GIS, but the question still arises which additional GIS applications (also called industry models) are required or which are available at all. I would like to go into this further in this article.

Basically there are GIS applications for the following industries:

*	Banks
*   Education
*	Infrastructure development
*	Disaster management
*   Agriculture
*	Logistics
*	Marketing
*   Medicine
*	Telecommunications
*	Tourism
*	Crime mapping
*   Traffic
*	Insurance
*	Economic development

For each of these industries, GIS service providers offer different GIS applications and also adapt them individually to the needs of the customers.

In the following, I would like to go into more detail about a few applications, especially for municipalities (GIS).

## Tree cadastre

A tree cadastre supports municipalities and tree care companies in the collection, control and management of tree stands. Trees can be divided into groups of trees and various material data or media can also be added to the trees:

* Number, type, height, crown diameter, degree of sealing, type of soil, damage, maintenance measures carried out or checks and pictures, to name just a few.

The trend here is towards mobile solutions. This means apps that allow you to enter data into the GIS directly on a tablet using controls or maintenance measures. This data is stored online and can then be corrected or revised later in the office.

Development plans / land use plans

Development and land use plans can be easily managed and evaluated in a GIS.

Depending on the legal validity, areas of validity can be displayed in different colors, changes can be linked to the main plan, and text files, such as additions to the articles of association, can be added.

In addition, analog development plans can be prepared (scanned, georeferenced) and displayed in the GIS. This can be done with PDF or CAD files, for example.

As a result, you get a data record for each development plan, to which all associated files and changes are linked, and can present them in an appealing or clear manner.

Real estate cadastre

For example, in the case of a construction project in a certain area, all affected citizens can be identified and written to very easily by automatically creating reports using templates for writing and selecting all citizens in this area (in this area in the GIS).

## Sealing cadastre (split sewage fees)

All sealed areas of a property are determined in a sealing register. This takes place via a previously carried out aerial survey, in which high-resolution images are taken, or via digitization of satellite images.

All parcels are combined into one plot and the sealed areas of these parcels are linked to the plot. This enables municipalities to determine the split rainwater fee.

## Supply networks

### Water

Water supply networks can be managed digitally in a GIS. Hydrant plans can be created automatically, which can be very helpful, for example, for the local fire brigade in an emergency.

Repairs carried out on pipes can be stored digitally, so that you always have an overview of which pipes have already been renovated and which should be renovated in the near future.

### Sewage

Many commissions are legally obliged to keep a channel register.

A channel register is created either from analog data such as plans, which are digitized, or from a measurement that has been carried out beforehand.

In a channel cadastre, data such as posture length, depth, cable diameter material, etc. can be saved, managed and analyzed.

Large corporations now also have wastewater registers for their company premises.

### Electricity / gas / broadband / street lighting

Enables the acquisition, management and analysis of all supply networks.

### Tree and green areas

Allows the construction of a tree and green area register for further planning and maintenance of the inventory.

## Conclusion

Especially in municipalities, the importance of a geographic information system is beyond question. A lot of time is saved due to less administrative work.

The various industry models can be easily combined with each other, making work processes much more efficient.

But also in the private sector, e.g. In the real estate market, in agriculture or in archeology, the advantages of a geographic information system (GIS) are increasingly recognized and used. Mobile apps that are combined with a GIS are particularly popular.

If you want to see more applications, check out GIS-Geography. There you will find [1000 GIS Applications & Uses - How GIS Is Changing the World](https://gisgeography.com/gis-applications-uses/ "1000 GIS Applications & Uses - How GIS Is Changing the World").

GIS Applications - Which GIS Applications are there?

## GIS and Geo Blogs

Of course, to find out new things, I take a look at one or the other website that deals with [GIS](/articles/what-is-gis-geographic-information-system "What is GIS?") Or geoinformatics in general. You can find them here:

1. GIS Lounge

Maps and GIS by Caitlin Dempsey Morais. She has been blogging about GIS for more than 20 years.

[https://www.gislounge.com/](https://www.gislounge.com/)

1. GIS Geography

Blog about GIS and Geographie.

[www.gisgeography.com](https://gisgeography.com/)

1. Free and Open Source GIS Ramblings

Anita Graser's blog about QGIS, open source, analysis and simulation.

[https://anitagraser.com/](https://anitagraser.com/)

1. Geospatial World

How does location localization affect us?

[https://www.geospatialworld.net](https://www.geospatialworld.net)

1. Geoawesomeness

Blog about GIS, geodata and everything that goes with it.

[www.geoawesomness.com](https://geoawesomeness.com/)

1. GISTimes

GIStimes is for everything that happens on the geodata market.

[http://www.gistimes.com](http://www.gistimes.com)

1. GIS Professional

GIS news and articles about GNSS, Big Data, Addressing, BIM, and Smart Cities.

[https://www.gis-professional.com/news](https://www.gis-professional.com/news)

1. Geospatial-solutions

[http://geospatial-solutions.com/](http://geospatial-solutions.com/)

1. Google Maps Blog

Google Maps blog.

[https://www.blog.google/products/maps/](https://www.blog.google/products/maps/)

1. Carto

SaaS provider CartoDB also runs a very interesting GIS blog.

[https://carto.com/](https://carto.com/)

1. Reddit r/gis GIS-Community

A Reddit community about geographic information systems.

[https://www.reddit.com/r/gis/](https://www.reddit.com/r/gis/)

1. Benjaminspaulding

Geodata, analysis, programming.

[https://www.benjaminspaulding.com/](https://www.benjaminspaulding.com/)

1. GISuser

GIS and technology news for mapping experts.

[http://gisuser.com/](http://gisuser.com/)

1. Esri Newsroom

Esri's blog.

[https://www.esri.com/about/newsroom/blog](https://www.esri.com/about/newsroom/blog)

1. ThinkGeoBlog

GIS themes for .NET developers.

[http://blog.thinkgeo.com/](http://blog.thinkgeo.com/)

There is also a much larger list of links to GIS Blogs on Wiki.GIS (http://wiki.gis.com/wiki/index.php/List_of_GIS-related_Blogs). By the way, Wiki.GIS is a very extensive GIS encyclopedia.

Geography and GIS Blogs

[GIS](/articles/what-is-gis-geographic-information-system "What is GIS?") Volunteering offers a good opportunity to develop personally and professionally.

You can also get involved in a good cause. You can later pack the projects into a pretty portfolio and thus stand out from the competition with extra points when applying.

What's even better is that you can join these organizations with a PC at home and don't have to travel the world.

## OpenStreetMap

![OpenStreetMap](https://upload.wikimedia.org/wikipedia/commons/thumb/b/b0/Openstreetmap_logo.svg/256px-Openstreetmap_logo.svg.png "OpenStreetMap")

[https://www.openstreetmap.org/](https://www.openstreetmap.org/)

OpenStreetMap is an international project that was founded in 2004.

The aim of OSM is to create a free world map and make it available to everyone free of charge. The data is collected by volunteers (also known as ** mappers **). Data on roads, railways, rivers, forests, houses, etc. are collected.

### How to join

There are many different ways to contribute to OpenStreetMap, from reporting small errors on the map, completing existing data, drawing new buildings from aerial photographs and recording routes and points of interest with the GPS device. Our instructions will help you use the right programs and enter data. (OpenStreetMap).

[Join...](https://www.openstreetmap.org/user/new?cookie_test=true)

## Humanitarian OpenStreetMap Team

![Humanitarian OpenStreetMap Team](https://www.hotosm.org/images/hot-logo-icon-nav.svg "Humanitarian OpenStreetMap Team")

[https://www.hotosm.org/](https://www.hotosm.org/)

The Humanitarian OpenStreetMap Team (HOT) is an international team dedicated to the mapping and mapping of humanitarian actions and the development of communities. The data are used to reduce risks in the and to work on sustainable development.

### GIS Freiwilligenarbeit mit HOT

As a **mapping volunteer** you can collect data for maps as with OpenStreetMap. ** Humanitarian and GIS Professionals ** also help in additional areas, such as data processing, validation of maps or create completely new maps and visualizations.

[Join...](https://www.hotosm.org/volunteer#humanitarian-and-gis-professionals)

## Standby Task Force

![Standby Task Force](https://www.standbytaskforce.org/wp-content/uploads/2016/02/cropped-Logo_SBTF_RED-03-450x203.png "Standby Task Force")

[https://www.standbytaskforce.org/](https://www.standbytaskforce.org/)

Standby Task Force is a global network of trained and experienced volunteers who work together online.

The Standby Task Force is a non-profit organization founded in 2010.

The Standby Task Force has been involved in many natural disasters since then, and the volunteers have assisted many humanitarian organizations in election observation and other projects.

### Voluntary with the Standby Task Force

You should already have professional experience in the areas of GIS management, disaster management and other technical areas.

[Join...](https://www.standbytaskforce.org/help-us/volunteer-with-us/)

## GISCorps

![GISCorp](https://www.urisa.org/clientuploads/directory/graphics/gc_logo.jpg "GISCorp")

[https://www.giscorps.org/](https://www.giscorps.org/)
GISCorps coordinates short-term, voluntary GIS services for disadvantaged communities.

The projects vary according to the needs of the partner agency and can include all aspects of GIS, including analysis, cartography, app development, needs analysis, technical workshops etc.

The service areas include humanitarian aid, disaster protection, environmental protection, health and health personnel services, GIS training and crowdsourcing of experts. GISCorps is supported by individual donations, companies and other non-profit groups with similar goals.

### Engage

At GISCorps there are several ways to get involved. You can apply as a volunteer and actively support GIS projects. You can get a one-year ArcGIS license for free here, provided you are accepted.

You can of course also support the project with donations.

[Join...](https://www.giscorps.org/become-a-volunteer/)

GIS volunteering - Make the world a better place with GIS

Podcasts are a great way to keep up to date with current developments. Best of all, you can listen to podcasts practically anywhere.

## VerySpatial

The podcast is hosted by Jesse Rouse, Sue Bergeron and Frank Lafone. An established podcast that discusses geography, geographic information technologies and the impact of [GIS](/en/gis/geographic-information-system-what-is-gis "What is GIS") on everyday digital life.

[http://veryspatial.com/](http://veryspatial.com/)

![VerySpatial Podcast](VSLogo_Small_400x400.jpg "VerySpatial")

## Mapscaping

New podcast. Mainly interviews with people from the GIS and geo industry.

[https://mapscaping.com/blogs/the-mapscaping-podcast](https://mapscaping.com/blogs/the-mapscaping-podcast)

![Mapscaping Podcast](/gedY7m5O_400x400.jpg "Mapscaping")

## Radio OSM

Reports and news about OpenStreetMap, the free wiki world map.

[https://podcast.openstreetmap.de/](https://podcast.openstreetmap.de/)

![Radio OSM](podcast-e1557674429764-150x150.png)

## Geodorable

A podcast that can contain everything and everyone about the [geodata](/geodata-what-is-geodata) world.

[https://geodorable.com/](https://geodorable.com/)

![Geodorable Podcast](index.jpg "Geodorable")

## The Mappyist Hour

Geographers and geo-types who talk about how incredible their job is after work.

[http://www.themappyisthour.com/](http://www.themappyisthour.com/)

![The Mappyist Hour Podcast](F_739vLb_400x400.jpg "The Mappyist Hour")

## Speaking of GIS

A podcast by Kurt Towler. The podcast includes interviews with other geospatialists and reviews of conferences.

[https://speakingofgis.com/](https://speakingofgis.com/)

![Speaking of GIS Podcast](ltcN4uOG_400x400.jpg "Speaking of GIS")

## Scene from Above

A podcast with a view of the world of modern [remote sensing] (/en/tags/remote-sensing) and earth observation. Driven by their passion for all grid and geodata, they strive for a mix of news, opinions, discussions and interviews.

[http://scenefromabove.org/](http://scenefromabove.org/)

![Scene from Above Podcast](XIxrl16S_400x400.jpg "Scene from Above")

## Directions Magazine

Every six weeks, new location-based podcasts are released by another geographic information branch.

[https://player.fm/series/directions-magazine-podcasts](https://player.fm/series/directions-magazine-podcasts)

![Directions Magazine Podcast](VrHWMDgk_400x400.jpg "Directions Magazine ")

## Cageyjames & Geobabbler

This monthly podcast by James Fee and Bill Dollins is about how you can use spatial technologies in your workflow.

[https://cng.fireside.fm/](https://cng.fireside.fm/)

![Cageyjames & Geobabbler Podcast](cagey.png "Cageyjames & Geobabbler")

## Geographical Imaginations

Geographical Imaginations Expedition & Institute is a growing public geography initiative for multimedia media that aims to bring together academic and everyday geographic or spatial thinking.

[https://podcasts.apple.com/us/podcast/geographical-imaginations/id1386704057?mt=2](https://podcasts.apple.com/us/podcast/geographical-imaginations/id1386704057?mt=2)

![Geographical Imaginations](170x170bb.jpg)

## Women and Drones

A podcast that serves to make women in the UAS industry better known around the world.

[http://womenanddrones.libsyn.com/](http://womenanddrones.libsyn.com/)

![Women and Drones Podcast](nCObdGxO_400x400.jpg "Women and Drones")

Geo and GIS Podcasts to stay up to date

In this post you will find a list of free and/or open-source and proprietary [GIS](/articles/what-is-gis-geographic-information-system)-software options.

## Free and open-source desktop-GIS software

**1. [QGIS](https://www.qgis.org "QGIS")**

QGIS is a free and open-source geographic information system.
With QGIS you can create, modify, visualize and analyze spatial data on Windows, Mac, Linux, BSD and Android.

**2. [GRASS GIS](https://grass.osgeo.org/ "GRASS GIS")**

GRASS GIS is a hybrid, modular geographic information system software with grid and vector-oriented functions.

**3. [SAGA GIS](http://www.saga-gis.org/ "SAGA GIS")**

Software for automated geoscientific analysis. The SAGA project is mainly developed at the Department of Geography at the University of Hamburg.

**4. [JUMP GIS/OpenJUMP](http://www.openjump.org/ "JUMP GIS/OpenJUMP")**

Java based open-source GIS.

**5. [GeoDa](https://geodacenter.github.io/ "GeoDa")**

GeoDa is a free and open-source GIS-software and serves as an introduction to geodata analysis.

**6. [gvSIG](http://www.gvsig.com/en "gvSIG")**

Open-Source Desktop, Online und Mobile GIS.

**7. [MapmakerPro](http://www.mapmaker.com/v4/Map%20maker%204.Pro.htm "MapmakerPro")**

MapMaker is aimed at specialists who need to create maps. For example foresters, archaeologists, emergency services, etc..

**8. [DIVA GIS](http://www.diva-gis.org/ "DIVA GIS")**

DIVA-GIS is a free GIS for mapping and analyzing geographic data.

**9. [TerraLib](http://www.dpi.inpe.br/terralib_previous/index.php "TerraLib")**

TerraLib is an open source GIS software library that supports the development of custom geographic applications.

**10. [Kalypso](https://sourceforge.net/projects/kalypso/ "Kalypso")**

Kalypso is an open source modeling program. The focus is on numerical simulations in water management and ecology.

**11. [OrbisGIS](http://orbisgis.org/ "OrbisGIS")**

OrbisGIS is a cross-platform open source GIS developed by and for research.

**12. [OzGIS](https://sourceforge.net/projects/ozgis/ "OzGIS")**

OzGIS is a GIS for analyzing and displaying spatial statistics.

**13. FalconView**

FalconView is a GIS developed by the Georgia Tech Research Institute.

**14. [ILWIS](https://www.itc.nl/ilwis/ "ILWIS")**

The Integrated Land and Water Information System (ILWIS) is a desktop-based GIS and remote sensing software that was developed by ITC up to Release 3.3 in 2005.

**15. [MapWindow GIS](https://www.mapwindow.org/ "MapWindow GIS")**

MapWindow GIS is an open source GIS desktop application that is used by a large number of users and organizations around the world.

**16. [Whitebox GAT](https://www.uoguelph.ca/~hydrogeo/Whitebox/ "Whitebox GAT")**

Whitebox Geospatial Analysis Tools is an open source, cross-platform geospatial information system and remote sensing software package.

**17. [Capaware](http://www.capaware.org/ "Capaware")**

3D-world-viewer.

**18. [Generic Mapping Tools](http://gmt.soest.hawaii.edu/ "Generic Mapping Tools")**

The Generic Mapping Tools are a collection of free software for creating geological or geographical maps and diagrams.

## Proprietary GIS Software

**19. ArcGIS, ArcView**

ArcGIS is the generic term for various GIS software products from Esri.

**20. AutoCAD Map3D**

AutoCAD Map3D from Autodesk is a GIS software solution and offers extensive access to all CAD and GIS data and enables its creation and editing.

**21. Aquaevo GIS**

Aquaveo is a GIS software for modeling environmental and water resources.

**22. Bentley Map**

**23. Cadcorp**

Cadcorp's GIS and Web Mapping Software are GIS software products for the creation, analysis and data management of geodata.

**24. Conform**

Conform is a GIS software for merging, visualizing, editing and exporting 3D environments for urban planning, games and simulations.

**25. Dragon / ips**

Dragon / ips is a remote sensing image processing software.

**26. ENVI**

The ENVI image analysis software is used by GIS experts, remote sensing scientists and image analysts to extract meaningful information from images to help them make better decisions.

**27. ERDAS IMAGINE**

ERDAS IMAGINE is software for evaluating remote sensing data, especially graphics and photos.

**28. Field-Map**

Field-Map is a proprietary integrated tool for programmatic field data acquisition by IFER - Monitoring and Mapping Solutions, Ltd. It is mainly used for mapping forest ecosystems and for data collection during field analysis.

**29. Geosoft**

GEOSOFT is one of Germany's leading developers of geodetic computing and organizational software for private and public surveying agencies.

**30. GeoTime**

GeoTime is a geodata analysis software that enables the visual analysis of events over time. The third dimension adds time to a two-dimensional map so that users can see changes in time series data.

**31. Global Mapper**

Geographic information system with distance and area calculation; offers an integrated scripting language, 3D display and GPS tracks.

**32. Golden Software**

Surfer and Mapviewer are two software solutions with a variety of mapping and adaptation options and support any geodata format (including LiDAR data), 3D visualization, as well as volume / distance / area calculations.

**33. Intergraph**

GeoMedia is a GIS software from Intergraph. GeoMedia is a software product family with desktop GIS, web GIS and is mainly aimed at municipalities.

**34. Manifold System**

Manifold System is software for the management of digital maps. Digital maps and remote sensing data can be easily edited.

**35. MapInfo**

MapInfo Professional is a geographic information system software from the US company MapInfo Corporation.

**36. Maptitude**

Maptitude is a mapping software program created by Caliper Corporation that allows users to view, edit, and integrate maps. The software and technology are designed to facilitate the geographic visualization and analysis of contained data or user-defined external data.

**37. Netcad**

NETCAD GIS is a CAD and GIS software that supports international standards and was designed for users of engineering and geographic information systems.

**38. RegioGraph**

RegioGraph is a geomarketing software specializing in questions in the areas of marketing, sales, controlling, logistics and corporate strategy.

**39. RIWA GIS Zentrum**

The RIWA GIS Zentrum is a powerful, web-based geographic information system that has been used in numerous municipal administrations and industrial companies for many years.

**40. Smallworld**

Smallworld GIS is the professional geographic information system for network operators in the energy and water industries.

**41. TNTmips**

TNTmips is a geospatial data analysis system that offers a fully featured GIS, RDBMS and automated image processing system with CAD, TIN, surface modeling, map layout and innovative data publishing tools.

**42. TerrSet ( IDRISI )**

TerrSet is an integrated geographic information system and remote sensing software for monitoring and modeling the Earth system.

**43. [Google Earth Pro](https://www.google.de/earth/download/gep/agree.html "Google Earth Pro")**

Google Earth Pro Desktop is free and intended for users who need advanced features. You can import and export GIS data and go on a journey through time with the help of historical images. 

## Online GIS

**44. [Bing Maps](http://www.bing.com/maps/ "Bing Maps")**

Bing Maps is an online map service from Microsoft, through which various spatial data can be viewed and spatial services can be used. It is a further development of the MSN Virtual Earth and is part of the Bing search engine.

**45. [Google Maps](http://maps.google.de/ "Google Maps")**

Google Maps is an online map service from the US company Google LLC. The surface of the earth can be viewed as a road map or as an aerial or satellite image, with locations of institutions or known objects also being displayed. The service started on February 8, 2005.

**46. [NASA World Wind](http://worldwind.arc.nasa.gov/java/ "NASA World Wind")**

NASA World Wind is an open source software that enables satellite and aerial images to be displayed on a virtual globe combined with elevation data and to be zoomed in anywhere in the world in 3D graphics and viewed freely from all sides.

**47. [OpenStreetMap](http://www.openstreetmap.org/ "OpenStreetMap")**

OpenStreetMap is a free project, which collects freely usable geodata, structures it and keeps it in a database for everyone to use. This data is under a free license, the Open Database License.

**48. [Wikimapia](http://www.wikimapia.org/ "Wikimapia")**

Wikimapia is a web interface that combines maps with a restricted wiki system without hypertext functions. It allows the user to add information in the form of a note to any position on the earth.

## Other GIS-Tools and Software used with GIS

**49. [GDAL/OGR](https://live.osgeo.org/de/overview/gdal_overview.html "GDAL/OGR")**

The Geospatial Data Abstraction Library (GDAL / OGR) provides command line-based auxiliary programs. A large number of raster and vector geodata formats can be converted and processed using these.

**50. [Leaflet](https://leafletjs.com/ "Leaflet")**

Leaflet is the leading open-source JavaScript library for mobile-friendly interactive maps.

**51. [OpenLayers](https://openlayers.org/ "OpenLayers")**

OpenLayers makes it easy to put a dynamic map in any web page. It can display map tiles, vector data and markers loaded from any source.

**52. [R](https://www.r-project.org/ "R")**

R is a free programming language for statistical calculations and graphics.

**53. [Blender](https://www.blender.org/ "Blender")**

Blender is a free, GPL-licensed 3D graphics suite with which bodies can be modeled, textured and animated.



Free, Open-Source and proprietary GIS software options

Managing your own newsletter is crucial for creating a sustainable online-business. With E-Mails you can build a relationsship with your audience and engage with them so they will drive some nice traffic to your new post or whatever you just have published and want to promote.

If you are using Mailchimp you can use the plugin [gatsby-plugin-mailchimp](https://github.com/benjaminhoffmanhttps://www.gatsbyjs.com-plugin-mailchimp "gatsby-plugin-mailchimp") to manage your e-mail list.

## Getting started

Simply add the plugin to your package.json with
```
npm
npm install gatsby-plugin-mailchimp
```
or
```
yarn
yarn add gatsby-plugin-mailchimp
```

and implement it in your **gatsby.config.js** like
```js
{
        resolve: 'gatsby-plugin-mailchimp',
        options: {
            endpoint: '', // string; add your MC list endpoint here; see instructions below
            timeout: 3500, // number; the amount of time, in milliseconds, that you want to allow mailchimp to respond to your request before timing out. defaults to 3500
        },
    },
```
If you don't have your Mailchimp endpoint yet i would suggest to have a README of gatsby-plugin-mailchimp. They described every step with images so it's really easy to get your endpoint url.

> Once you have your Mailchimp endpoint you should save it as environment variable in your project.

## Creating a sign-up form

Only thing you will need is to import the **addToMailChimp** method to your newsletter sign-up component which will work like this:
```js
import addToMailchimp from 'gatsby-plugin-mailchimp'
```
(I am actually working with styled components which are stored in a separate file in the same folder. This file is imported with
```
import * as S from './styled'
```
and the components then are used like **S.NewsletterWrapper**. But to make it a bit more clear i declared everything in the same file for this post.)

So now you need some styled (and responsive) components which will create your actual form like:
```js
<NewsletterWrapper>
  <DescriptionWrapper>
    <p>
      Do you want to know when I post something new? <br/> 
      Then subscribe to my newsletter.
      ðŸš€
    </p>
  </DescriptionWrapper>
  <InputWrapper>
    <Input
      type="email"
      name="email"
      id="mail"
      label="email-input"
      placeholder="Your e-mail address"
      onChange={(e) => setEmail(e.target.value)}
    />
  </InputWrapper>
  <ButtonWrapper>
    <Button
      type="button"
      aria-label="Subscribe"
      onClick={() => handleSubmit()}
    >
    Subscribe
    </Button>
  </ButtonWrapper>
</NewsletterWrapper>
```

In this component you will show a different message from the default one after a user has successfully subscribed to your newsletter.
To do so you need a variable which will store the current state (submitted = true or submitted = false). 
This variable will have the default value **false**, which will be set to **true** after a user has subscribed successfully.

So if a user clicks on the "Subscribe-Button" the function **handleSubmit** will be executed which does the following:

+ input data will be send to the Mailchimp endpoint with ```addToMailchimp(email)```,
+ if the returned property from the Mailchim API has the value "error" your error handling function will handle the event,
+ otherwise the subscribe is being tracking with a custom Google Analytics event and submitted will be set to true.

> The Mailchimp API will always return a object with the properties **result** and **msg**.

```js
{
    result: string; // either `success` or `error` (helpful to use this key to update your state)
    msg: string; // a user-friendly message indicating details of your submissions (usually something like "thanks for subscribing!" or "this email has already been added")
}
```

Finally you just have to check the value of **submitted** and render the relevant content like the following:

```js
  return (
    <>
      {submitted ? (
        <NewsletterWrapper>
          <DescriptionWrapper>
            <h2>
              ðŸŽ‰ Successfully subscribed! ðŸŽ‰
            </h2>
            <p>
            Thank your for your interest in my content.
            </p>
          </DescriptionWrapper>
        </NewsletterWrapper>
      ) : (
        <NewsletterWrapper>
            <DescriptionWrapper>
              <p>
                Do you want to know when I post something new? <br/> 
                Then subscribe to my newsletter.
                ðŸš€
              </p>
            </DescriptionWrapper>
          <InputWrapper>
            <Input
              type="email"
              name="email"
              id="mail"
              label="email-input"
              placeholder="Your e-mail address"
              onChange={(e) => setEmail(e.target.value)}
            />
          </InputWrapper>
          <ButtonWrapper>
            <Button
              type="button"
              aria-label="Subscribe"
              onClick={() => handleSubmit()}
            >
              "Subscribe"
            </Button>
          </ButtonWrapper>
        </NewsletterWrapper>
      )}
    </>
  )
}
```
If you want to learn more about sign-up forms i suggest to have a look at [Non-Invasive Sign Up Forms](https://sld.codes/articles/Non-Invasive-Sign-Up-Forms "Non-Invasive Sign Up Forms") from Slarsen Disney. He is creating super UX-friendly websites and is sharing the code for it.

```js
import addToMailchimp from "gatsby-plugin-mailchimp"
import React, { useState } from "react"
import ConfettiAnimation from "../Animations/ConfettiAnimation"
import { trackCustomEvent } from "gatsby-plugin-google-analytics"
import styled from 'styled-components';

export const NewsletterWrapper = styled.form`
    display: flex;
    flex: 0 1 auto;
    flex-direction: row;
    flex-wrap: wrap;
    box-sizing: border-box;
    max-width: 750px;
    justify-content: center;
`
export const DescriptionWrapper = styled.div`
    text-align: center;
    flex-grow: 0;    
    flex-shrink: 0;
    flex-basis: 100%;    
    max-width: 100%;
`

export const InputWrapper = styled.div`
    flex-direction: column;
    justify-content: center;
    display: flex;
    flex-grow: 0;
    flex-shrink: 0;
    flex-basis: 50%;
    max-width: 66.66667%;
`

export const Input = styled.input`
    padding-top: 15px!important;
    padding-bottom: 15px!important;
    padding: 12px 20px;
    margin: 8px 0;
    box-sizing: border-box;
    border: 2px solid hsla(0,0%,90.2%,.95);
    :invalid {
        border: 1px solid red;
    }
`

export const ButtonWrapper = styled.div`
    flex-direction: column;
    justify-content: center;
    display: flex;
    flex-grow: 0;
    flex-shrink: 0;
    flex-basis: 50%;
    max-width: 33.33333%;
`

export const Button = styled.button`
    box-sizing: border-box;
    border: 2px solid ${props =>
        props.background ? props.background : 'white'};
    color: white;
    text-transform: uppercase;
    position: relative;
    padding-top: 15px!important;
    padding-bottom: 15px!important;
    outline: none;
    overflow: hidden;
    width: 100%;
    transition: all .2s ease-in-out;
    text-align: center;
    background: ${props =>
        props.background ? props.background : 'hsla(0,0%,90.2%,.95)'};
    :hover {
        box-shadow: rgba(0, 0, 0, 0.5) 0px 8px 16px 0px;
        transform: translateY(0) scale(1);
`

export default ({ }) => {
  const [email, setEmail] = useState("")
  const [submitted, setSubmitted] = useState(false)   


  function errorHandling(data) {
    // your error handling
  }
  
  const handleSubmit = () => {
    addToMailchimp(email).then((data) => {

      if (data.result == "error") {
        errorHandling(data)
      } else {
        trackCustomEvent({
          category: "Newsletter",
          action: "Click",
          label: `Newsletter Click`,
        })
        setSubmitted(true)
      }
    })
  }

  return (
    <>
      {submitted ? (
        <NewsletterWrapper>
          <DescriptionWrapper>
            <h2>
              ðŸŽ‰ Successfully subscribed! ðŸŽ‰
            </h2>
            <p>
            Thank your for your interest in my content.
            </p>
          </DescriptionWrapper>
        </NewsletterWrapper>
      ) : (
        <NewsletterWrapper>
            <DescriptionWrapper>
              <p>
                Do you want to know when I post something new? <br/> 
                Then subscribe to my newsletter.
                ðŸš€
              </p>
            </DescriptionWrapper>
          <InputWrapper>
            <Input
              type="email"
              name="email"
              id="mail"
              label="email-input"
              placeholder="Your e-mail address"
              onChange={(e) => setEmail(e.target.value)}
            />
          </InputWrapper>
          <ButtonWrapper>
            <Button
              type="button"
              aria-label="Subscribe"
              onClick={() => handleSubmit()}
            >
              "Subscribe"
            </Button>
          </ButtonWrapper>
        </NewsletterWrapper>
      )}
    </>
  )
}
```

How to create a Mailchimp newsletter sign-up-form for your Gatsby Site

## What is OpenLayers?

[OpenLayers](https://openlayers.org/) is a JavaScript library that makes it relatively easy to visualize geodata in web applications (Web-GIS).

OpenLayers is a programming interface that allows client-side development independent of the server. Map tiles, vector data and markers from various data sources can be displayed.

Open Layers was developed to promote the use of [geodata](/en/gis/geodata-what-are-geodata "What are geodata?") Of all kinds. OpenLayers is also free, open-source and is published under "2-clause BSD License".

To be able to create a map with OpenLayers, all you need is a basic general knowledge of programming languages. The missing pieces of the puzzle can be found very easily using the detailed documentation on OpenLayers.

## Create HTML file

First of all, an HTML file is required as the basic framework. The basic structure usually looks like this:

```html
    <html>
      <head>
        <title>OpenLayers Demo</title>
      </head>
      <body>
      </body>
    </html>
```

You can now copy this code and paste it into a file that you name, for example, "jsmap.html".

_If you want to learn more about HTML, you can find a few useful tutorials on [w3schools](https://www.w3schools.com/html/)._

## Insert OpenLayers Javascript library

Now the OpenLayers Javascript is integrated into the HTML. You copy for that

```html
    <script src="http://www.openlayers.org/api/OpenLayers.js"></script>
```

between `<title>` and `<head>`. You can do that right away

```js
          function init() {
            map = new OpenLayers.Map("basicMap"); //create a new map
            var mapnik = new OpenLayers.Layer.OSM(); //add an OpenStreetMap layer to have some data in the mapview
            map.addLayer(mapnik); //add the OSM layer to the map

    		var markers = new OpenLayers.Layer.Markers( "Markers" ); //add a layer where markers can be put
    		map.addLayer(markers); //add the markers layer to the current map

    		var lonLat = new OpenLayers.LonLat( 13.0 ,47.8 ) //define a new location with these coordinates in WGS84
              .transform(  //transform the location to the coordinate system of our OpenLayers map
                new OpenLayers.Projection("EPSG:4326"), // transform from WGS 1984
                map.getProjectionObject() // to Spherical Mercator Projection
            );
    		markers.addMarker(new OpenLayers.Marker(lonLat)); //add the newly created marker to the markers layer

            map.setCenter(lonLat, 15); // Use maker to center the map above and set zoom level to 15
          }
```

also included in script tags. Here a function with the name init is created, the

* which creates the map and inserts it in the element "basicMap"
* created a layer with [OpenStreetMap](https://www.openstreetmap.org/#map=6/51.330/10.453) data and added it to the map
* creates a layer for markers and also adds this to the map
* Defined a marker in WGS84 and then transformed it from WGS1984 to Spherical Mercator Projection
* add this marker to the map
* and finally centered the map on this marker and set a zoom level.

Now this function must also be loaded and the card placed. For that you are replacing now

```html
    <body>
      </body>
```

with

```html
    <body onload="init();">
        <div style="width: 100%; height: 60%;" id="basicMap"></div>
    </body>
```

With `javascript onload =" init (); `the function is executed when loading the HTML file and inserted via the id` id = "basicMap" `.

Your complete file should now look like this:

```html
    <html>
      <head>
        <title>OpenLayers Demo</title>
        <script src="http://www.openlayers.org/api/OpenLayers.js"></script>
        <script>
          function init() {
            map = new OpenLayers.Map("basicMap"); //create a new map
            var mapnik = new OpenLayers.Layer.OSM(); //add an OpenStreetMap layer to have some data in the mapview
            map.addLayer(mapnik); //add the OSM layer to the map

    		var markers = new OpenLayers.Layer.Markers( "Markers" ); //add a layer where markers can be put
    		map.addLayer(markers); //add the markers layer to the current map

    		var lonLat = new OpenLayers.LonLat( 13.0 ,47.8 ) //define a new location with these coordinates in WGS84
              .transform(  //transform the location to the coordinate system of our OpenLayers map
                new OpenLayers.Projection("EPSG:4326"), // transform from WGS 1984
                map.getProjectionObject() // to Spherical Mercator Projection
            );
    		markers.addMarker(new OpenLayers.Marker(lonLat)); //add the newly created marker to the markers layer

            map.setCenter(lonLat, 15); // Use maker to center the map above and set zoom level to 15
          }
        </script>
      </head>
      <body onload="init();">
        <div style="width: 100%; height: 60%;" id="basicMap"></div>
      </body>
    </html>
```

If you save the file and open it, you will end up in the browser of your choice and your Javascript web map will be displayed with OpenLayers.

If you now want to change the position of the marker, all you have to do is change the coordinates.

## Create markers with a 'for .. of ..' loop

Normally, you often want to display several points and not just one on the map.
In theory you could

```js
    var lonLat = new OpenLayers.LonLat( 13.0 ,47.8 ) //define a new location with these coordinates in WGS84
              .transform(  //transform the location to the coordinate system of our OpenLayers map
                new OpenLayers.Projection("EPSG:4326"), // transform from WGS 1984
                map.getProjectionObject() // to Spherical Mercator Projection
            );
        markers.addMarker(new OpenLayers.Marker(lonLat)); //add the newly created marker to the markers layer
```

now copy for each additional marker you want to create and simply change the coordinates. But since the whole thing becomes relatively confusing, we will solve it differently.

First, we create an array of arrays.

```js
    var poi = [ // create array with point of interests
    			[ 11.557617 ,48.092757 ],
    			[ 8.558350, 50.028917 ],
    			[ 6.701660, 51.289406 ],
    			[ 13.337402, 52.496160 ]
          ];
```

All coordinates for the markers to be displayed are now stored in this array. Now we create a function that can be called to create markers and add them to the map.

```js
    function createmarker (lon,lat) {
    				var feature = new OpenLayers.LonLat( lon, lat ) // create features (locations) out of arrays in points
    						.transform(  //transform the location to the coordinate system of our OpenLayers map
    							new OpenLayers.Projection("EPSG:4326"), // transform from WGS 1984
    							map.getProjectionObject() // to Spherical Mercator Projection
    						);
    				markers.addMarker(new OpenLayers.Marker(feature)); // Add new features to markers layer
          }	
```

This function should now be carried out for each pair of coordinates in the "poi" array. This can be solved with a "for .. of" loop.

```js
    for (var x of poi) { // for each array(object) in array 
    				createmarker (x[0],x[1]) // create markers
    			}
```

In this loop, a marker is now created for each coordinate pair in the array, transformed and added to the map. The "poi" array can now be expanded as required and the additional markers are automatically added to the map.

## Add markers via UI

In order to make the whole thing more user-friendly and not having to change the code manually every time, we are now creating a simple user interface to add additional markers.

The new coordinates should be entered via two input fields and created with a confirmation on a button.

The HTML framework can look like this and should be placed somewhere in the "body" area:

```html
    <div class="add_markers">
    	<div class="input_markers">	
    	  Add new markers with coordinates in WGS84!
    		<div class="row">
    			<div class="col-25">
    				<label for="lat">Latitude:</label>
    			</div>
    			<div class="col-75">
    				<input type="text" id="lat" name="firstname" placeholder="48.060614">
    			</div>
    		</div>
    		<div class="row">
    			<div class="col-25">
    				<label for="lon">Longitude:</label>
    			</div>
    			<div class="col-75">
    				<input type="text" id="lon" name="lastname" placeholder="12.190876">
    			</div>
    		</div>
    		<button id="add_marker" class="button">Add marker!</button>
    		<div id="poi_added" class="poi_added"></div>
    	</div>
    </div>
    </div>
```

The most important here are the ids via which the values of the fields are later adopted.

To create these markers we use a function that is called every time the button "Add Marker!" is clicked.

The complete function looks like this:

```js
    function addFeature() {
    			  var lat = parseFloat(document.getElementById("lat").value); // get value of input lat and parse to float
    			  var lon = parseFloat(document.getElementById("lon").value); // get value of input lon and parse to float

    			  var newFeature = [ lon, lat ] // create array "newFeature" with lon , lat
    			  poi.push(newFeature) // add NewFeature to array "poi"

    			  createmarker (lon,lat) // create marker for input lat, lon  
    			  document.getElementById('poi_added').innerHTML = "Added marker for " + "latitude: " + lat + "; longitude: " + lon; // visual feedback for added marker
    			}
```

```js
    var lat = parseFloat(document.getElementById("lat").value); // get value of input lat and parse to float
    var lon = parseFloat(document.getElementById("lon").value);
```

Are references to the elements with the ID "lat" and "lon", ie the two input fields. Here the two variables "lat" and "lon" are created, to which the value from the input fields is assigned.

Then they are merged into an array, since a marker always consists of two coordinates and is added to the "poi" array.

```js
    var newFeature = [ lon, lat ] // create array "newFeature" with lon , lat
    poi.push(newFeature) // add NewFeature to array "poi"
```

Adding it to the "poi" array is not functionally necessary, but it can be useful if, for example, you want to create popovers that show the coordinates of each marker.

The coordinates are now saved in "lat" and "lon" and they only have to be transferred to the previously created function "createmarker", which creates the markers and adds them to the map.

```js
    createmarker (lon,lat) // create marker for input lat, lon  
```

It would be nice if the user received feedback about what happened after clicking the button. This can be done with

```js
    document.getElementById('poi_added').innerHTML = "Added marker for " + "latitude: " + lat + "; longitude: " + lon; // visual feedback for added marker
```

The last thing that is missing is that the function is executed with a click on the button.

```js
    document.getElementById('add_marker').addEventListener('click', addFeature); // execute function "addFeature" when button with id "add_marjer" is clicked
```

As soon as the button with the id "add_marker" is clicked, the "addFeature" function is now executed.

With

```js
    var extent = map.zoomToExtent(markers.getDataExtent()); // get extent of markers layer
```

the extent of the "markers" layer is determined, zoomed onto it and assigned to the variable extent.

If you now save and open your file again, you should see your map with all markers and be able to add additional markers via a graphical user interface.


```html
    <html>
      <head>
        <title>OpenLayers Demo</title>
        <script src="http://www.openlayers.org/api/OpenLayers.js"></script>
        <script>
    		function init() {
    			map = new OpenLayers.Map("basicMap"); //create a new map
    			var mapnik = new OpenLayers.Layer.OSM(); //add an OpenStreetMap layer to have some data in the mapview
    			map.addLayer(mapnik); //add the OSM layer to the map

    			var markers = new OpenLayers.Layer.Markers( "Markers" ); //add a layer where markers can be put
    			map.addLayer(markers); //add the markers layer to the current map

    			function createmarker (lon,lat) {
    				var feature = new OpenLayers.LonLat( lon, lat ) // create features (locations) out of arrays in points
    						.transform(  //transform the location to the coordinate system of our OpenLayers map
    							new OpenLayers.Projection("EPSG:4326"), // transform from WGS 1984
    							map.getProjectionObject() // to Spherical Mercator Projection
    						);
    				markers.addMarker(new OpenLayers.Marker(feature)); // Add new features to markers layer
    			}	

    			var poi = [ // create array with point of interests
    			[ 11.557617 ,48.092757 ],
    			[ 8.558350, 50.028917 ],
    			[ 6.701660, 51.289406 ],
    			[ 13.337402, 52.496160 ]
          ];

    			for (var x of poi) { // for each array(object) in array 
    				createmarker (x[0],x[1]) // create markers
    			}

    			var extent = map.zoomToExtent(markers.getDataExtent()); // get extent of markers layer

    			function addFeature() {
    			  var lat = parseFloat(document.getElementById("lat").value); // get value of input lat and parse to float
    			  var lon = parseFloat(document.getElementById("lon").value); // get value of input lon and parse to float

    			  var newFeature = [ lon, lat ] // create array "newFeature" with lon , lat
    			  poi.push(newFeature) // add NewFeature to array "poi"

    			  createmarker (lon,lat) // create marker for input lat, lon  
    			  document.getElementById('poi_added').innerHTML = "Added marker for " + "latitude: " + lat + "; longitude: " + lon; // visual feedback for added marker
    			}

    			document.getElementById('add_marker').addEventListener('click', addFeature); // execute function "addFeature" when button with id "add_marjer" is clicked
    		}

    		//  popover coordinates markers 
        </script> 
    	<style>
      /*dein style*/
    	</style>
      </head>
      <body onload="init();">
    	<div id="wrapper"  >
    		<div style="width: 100%; height: 80%" id="basicMap"></div>
    			<div class="add_markers">
    				<div class="input_markers">	
    					Add new markers with coordinates in WGS84!
    					<div class="row">
    						<div class="col-25">
    							<label for="lat">Latitude:</label>
    						</div>
    						<div class="col-75">
    							<input type="text" id="lat" name="firstname" placeholder="48.060614">
    						</div>
    					</div>
    					<div class="row">
    						<div class="col-25">
    							<label for="lon">Longitude:</label>
    						</div>
    						<div class="col-75">
    							<input type="text" id="lon" name="lastname" placeholder="12.190876">
    						</div>
    					</div>
    					<button id="add_marker" class="button">Add marker!</button>
    					<div id="poi_added" class="poi_added"></div>
    					</div>
    				</div>
    			</div>
    		</div>
    	</div>
      </body>
    </html>
```

How to create a web-map with OpenLayers

You want to know which Open-Source Web-[GIS](/articles/what-is-gis-geographic-information-system "GIS") applications are used to share geospatial data over the Internet?

## Web-GIS applications

### 1. GeoServer

GeoServer is an open source server for sharing geospatial data.

[http://geoserver.org/](http://geoserver.org/)

### 2. degree

deegree is an open source software for geodata infrastructures and the geospatial web.

[https://www.deegree.org/](https://www.deegree.org/)

### 3. FeatureServer

FeatureServer is an implementation of a RESTful Geographic Feature Service.

[http://featureserver.org/](http://featureserver.org/)

### MapGuide Open Source

MapGuide Open Source is a web-based platform that enables users to develop and deploy web mapping applications and geospatial services.

[http://mapguide.osgeo.org/](http://mapguide.osgeo.org/)

### 5. MapServer

MapServer is an open-source platform for publishing geodata and interactive map applications on the web.

[https://www.mapserver.org/](https://www.mapserver.org/)

* * *

## Javascript libraries

### 6. OpenLayers

OpenLayers is a JavaScript library that enables geospatial data to be displayed in the web browser. OpenLayers is a programming interface that allows client-side development independent of the server.

[https://openlayers.org/](https://openlayers.org/)

### 7. Leaflet

Leaflet is a free JavaScript library that can be used to create Web-GIS applications. The library uses HTML5, CSS3 and therefore supports most browsers.

[https://leafletjs.com/](https://leafletjs.com/)

Open-Source Web-GIS Applications

## What is OpenStreetMap?

[OpenStreetMap](/osm/ "OpenStreetMap") is the largest international project that aims to create a free world map. Voluntary "mappers" collect data about roads, railways, rivers, forests and houses and make them available online.

If you also want to get involved in the OpenStreetMap project, you can find further information here: [https://www.openstreetmap.de/faq.html#wie_mitmachen.](https://www.openstreetmap.de/faq.html#wie_mitmachen)

The data is freely available to all people. You can also use OpenStreetMap data commercially because it was published under the [Open Data Commons Open Database License](https://opendatacommons.org/licenses/odbl/)

## Data formats of OSM data

The data is offered by OSM as XML or PBF, which is a "compact" data format for the raw data from OpenStreetMap. The file Planet.osm contains the entire planet that has been recorded so far and the full history planet version even contains all version histories of all objects. This file is usually updated once a week.

With tools such as [Osmosis](https://wiki.openstreetmap.org/wiki/Osmosis) or [Osm2pgsql](https://wiki.openstreetmap.org/wiki/Osm2pgsql), this geodata can then be imported into a Postgis database. However, since this file is very large (76GB), most of you will probably not be able to start with it.
Instead of using the file of the entire planet it is more useful to extract the part of it that you will need. You can do this on your own or use services such as [Geofabrik](http://www.geofabrik.de/ "Geofabrik") offers them.

## Download via Geofabrik

Fortunately, there are [Geofabrik](https://www.geofabrik.de) that process OSM files and partially also make them available free of charge.

At [https://download.geofabrik.de/](http://download.geofabrik.de/) you will find download links for specific regions, where you can finally download OpenStreetMap data as [shapefiles](/articles/what-is-a-shapefile "Shapefiles"). There is also a small map at the top right of the website that shows the area of the selected data.

The data can also be downloaded as .pbf or bz2 files.

With a click on a region you land in the "sub-region", in which data of individual countries can then be downloaded. In Europe, shapefiles of the OSM data can be downloaded for almost all countries.

For Germany there is unfortunately only the possibility to download shapefiles of the individual federal states.

In addition, polygons of the dimensions of the individual federal states can be downloaded.

## Structure of raw OSM data

There is of course a unique ID for each object. When looking at street objects, there are so-called "other_tags" in addition to the name and type of street (residential, tertiary, secondary, unclassified, etc.).

There you will find all additional attributes that describe the object in more detail. In the case of the street, the maximum permitted speed, the maximum allowed weight, the zip code of the municipality, the material of the street and even more properties are described.

With special queries you can access these "other_tags" and, for example, only show all paved roads in QGIS.

With the usual "OSM Basemap" all these objects are rendered and displayed in the usual OpenStreetMap design.

[OpenStreetMap](https://www.openstreetmap.org/)

Download OpenStreetMap data as Shapefiles

With the help of [PyQGIS](https://docs.qgis.org/2.18/de/docs/pyqgis_developer_cookbook/index.html "PyQGIS") processes such as the export of images for all layers from a map can be automated.

First of all, of course, you need one or more layers with raster and/or vector data.

## Add layers with PyQGIS

If all files are in the same folder, they can be read in with a ["for .. in loop](https://www.w3schools.com/python/python_for_loops.asp" Python For Loops "). You can also use`. endswith (". gpkg") `, for example, only files with the extension" .gpkg "are searched, and when the layers are added, each layer name is packed into an array that will be needed later.

```py
    import os, sys
    from PyQt5.QtCore import QTimer

    # path to look for files
    path = "ordner/nocheinordner/"
    # set path
    dirs = os.listdir( path )
    # array for storing layer_names
    layer_list = []
    # variable for further processing
    count = 0
    #look for files inpath
    for file in dirs:
    	# search for ".gpkg" files 
        if file.endswith(".gpkg"):
    		#add vectorlayers
            vlayer = iface.addVectorLayer(path + file, "Layername", "ogr")
            layer_list.append(vlayer.name())
```

The new vector layers then appear in the QGIS layer tree.

## Image export for each layer

If you are satisfied with the display, two small functions can be used to export a georeferenced image for each layer.

```py
    def prepareMap():
        # make all layers invisible
    	iface.actionHideAllLayers().trigger()
        # get layer by layer_name
    	layer_name = QgsProject.instance().mapLayersByName(layer_list[count])[0]
        # select layer
    	iface.layerTreeView().setCurrentLayer(layer_name)
        # set selected layer visible
    	iface.actionShowSelectedLayers().trigger()
        # Wait a second and export the map
    	QTimer.singleShot(1000, exportMap) 
```

The "prepareMap ()" function first deactivates all layers. A layer is then selected from the "layer_list" array using its layer name and then displayed again. The QTimer class is particularly important here. Before an image is created, there must always be a short wait before the selected layer is really visible. Without QTimer, the script would run so quickly that the result would be loud images with the same content. After waiting a second, the "exportMap" function is called.

```py
    def exportMap(): 
        global count
    	# save current view as image
        iface.mapCanvas().saveAsImage( path + layer_list[count] + ".png" )
    	# feedback for printed map
        print('{}.png exported sucessfully'.format(layer_list[count]))
    	# get map for every layer in layer_list
        if count < len(layer_list)-1:
    		# Wait a second and prepare next map (timer is needed because otherwise all images have the samec content 
    		# the script excecutes faster then the mapCanvas can be reloaded
            QTimer.singleShot(1000, prepareMap) 
        count += 1
```
Now the current map, in which only one level is shown, is saved as a PNG image in the source directory. Ultimately, you "land" in a loop that goes through all the layers that are in the "layer_list" array and calls the "prepareMap" function for each layer again.

Export QGIS layers as images with PyQGIS

## What is Leaflet?

[Leaflet](https://leafletjs.com/) is a free [JavaScript](/articles/topics/javascript) library which allows us to create web-[gis](/articles/what-is-gis-geographic-information-system "GIS") applications. Leaflet uses HTML5, CSS3 and is compatible with all common browsers.

As with OpenLayers, rasters and vector data from various data sources can be integrated.

## React and Leaflet

In this article I will go into more detail on how to use [react-leaflet](/github/PaulLeCam/react-leaflet) Components and how to use a web map

* Basic map ([OpenStreetMap](/osm/)),
* Data in GeoJSON format,
* Markers and
* Popup

created.

First of all you should of course install these components first and import them into a new file.

> Since Leaflet does not support server-side rendering, you have to install a small "workaround" if you want to use Leaflet with, for example, [Gatsby](https://www.gatsbyjs.com/).

When executing `gatsby build `a webpack error will appear since there is no" window "object yet, but Leaflet needs it. These errors are fixed by checking the window object of the page before the map component arrives.

```js
    import React from "react"
    import Helmet from "react-helmet";
    import Layout from "../../src/layout";
    import config from "../../data/SiteConfig";
    import { Map, TileLayer, Marker, Popup, GeoJSON } from "react-leaflet";
    import 'leaflet/dist/leaflet.css';
    import hash from 'object-hash';

    class WebMap extends React.Component {  
        constructor() {
        super();
        this.state = {
          markers: [[51.8, 9.0]],
          center: [50.97, 10.54],
          zoom: 5,
          color: '#3f51b5',
          weight: 5,
          opacity: 0.65,
        };
      }

        render() {

            const Style = {
              color: this.state.color,
              weight: this.state.weight,
              opacity: this.state.opacity,
            };

          return (

        <Layout>
                {typeof window !== 'undefined' &&
                    {/* Map Stuff*/}
                }
        </Layout>
        );
      }
    }

    export default WebMap;
```

The style object is used to define a style that defines how the GeoJSON data is displayed. The respective values for this style are, just like

* the coordinates for a marker
* Information about centering the map and
* the "default zoom"

defined as "state objects" variables or arrays.

## Basemap

Now the map component comes first with an OpenStreetMap as the base map.

```js
    <Map>
        <TileLayer
            attribution='&copy; <a href="http://osm.org/copyright">OpenStreetMap</a>'
            url='https://{s}.tile.osm.org/{z}/{x}/{y}.png'
        />
     </Map>
```

`<TileLayer>` creates a new tile-layer in the map and with `url='https://{s}.tile.osm.org/{z}/{x}/{y}.png`  openstreetmap is used with attribution `attribution='&copy; <a href="http://osm.org/copyright">OpenStreetMap</a>`.

## Data in GeoJSON format

Adding geodata in GeoJSON is relatively simple. It is best to insert this into a function. This is then called up on the map.

```js
    function getGeoJSON() {
      return {
        "type":"GeometryCollection", "geometries": [
          {"type":"Polygon","coordinates":[[[6.000000248663241,56.000000155530984],[7.000000192318055,56.000000155530984],[8.000000135973096,56.000000155530984],[9.000000247266257,56.000000155530984],[10.000000190921071,56.000000155530984],[11.000000134576112,56.000000155530984],[12.000000245869273,56.000000155530984],[12.000000245869273,55.000000211876],[12.000000245869273,54.00000010058284],[12.000000245869273,53.00000015692797],[12.000000245869273,52.00000021327298],[12.000000245869273,51.00000010197982],[12.000000245869273,50.00000015832478],[12.000000245869273,49.00000004703179],[12.000000245869273,48.000000103376806],[11.000000134576112,48.000000103376806],[10.000000190921071,48.000000103376806],[9.000000247266257,48.000000103376806],[8.000000135973096,48.000000103376806],[7.000000192318055,48.000000103376806],[6.000000248663241,48.000000103376806],[6.000000248663241,49.00000004703179],[6.000000248663241,50.00000015832478],[6.000000248663241,51.00000010197982],[6.000000248663241,52.00000021327298],[6.000000248663241,53.00000015692797],[6.000000248663241,54.00000010058284],[6.000000248663241,55.000000211876],[6.000000248663241,56.000000155530984]]]}
          ]
      }
    }
```

The data is then inserted into the map component with the GeoJSON component.

```js
    <GeoJSON key={hash(getGeoJSON())} data={getGeoJson()} style={Style}/>
```

So that the data is always rendered you have to assign a "unique key". This "unique key" is generated with the function `hash (getGeoJSON ())`. The data is simply transferred in GeoJSO format with the style previously defined.

## Marker with popup

Markers can be inserted with the marker component.

```js
    {this.state.markers.map((position) => 
        <Marker 
            position={position}
        >
            <Popup>
                <pre>{position}</pre>
            </Popup>
        </Marker>
    )}
```

With the "map ()" method new arrays are created from the provided array. Since there is only one marker in the "markers" array, only one marker is created when the page is called up. This actual position or the coordinates are transferred as `{position}`.

In addition, the popup component for these markers creates a popup that shows the position.

The whole thing should now look like this:

```js
    import React from "react"
    import Helmet from "react-helmet";
    import Layout from "../../src/layout";
    import { Map, TileLayer, Marker, Popup, GeoJSON } from "react-leaflet";
    import "../components/Leaflet/map.scss";
    import hash from 'object-hash';

    class OnlineKoordinaten extends React.Component {    
        constructor() {
        super();
        this.state = {
          markers: [[51.8, 9.0]],
          center: [51.8, 9.0],
          zoom: 5,
          color: '#3f51b5',
          weight: 5,
          opacity: 0.65,
        };
      }

        render() {

            const Style = {
              color: this.state.color,
              weight: this.state.weight,
              opacity: this.state.opacity,
            };

          return (
        <Layout title="React-Leaflet-Demo">
          <div className="react-leaflet-demo-container">
                  {typeof window !== 'undefined' &&
                      <Map 
                      center={this.state.center} 
                      zoom={this.state.zoom} 
                      >
                      <TileLayer
                        attribution='&copy; <a href="http://osm.org/copyright">OpenStreetMap</a>'
                        url='https://{s}.tile.osm.org/{z}/{x}/{y}.png'
                      />
                      <GeoJSON key={hash(getGeoJson())} data={getGeoJson()} style={Style}/>
                      {this.state.markers.map((position) => 
                      <Marker 
                        position={position}
                      >
                        <Popup>
                        <pre>{position}</pre>
                        </Popup>
                      </Marker>
                      )}
                    </Map>
                  }
          </div>
        </Layout>
        );
      }
    }

    export default OnlineKoordinaten;

    function getGeoJson() {
      return {
        "type":"GeometryCollection", "geometries": [
          {"type":"Polygon","coordinates":[[[6.000000248663241,56.000000155530984],[7.000000192318055,56.000000155530984],[8.000000135973096,56.000000155530984],[9.000000247266257,56.000000155530984],[10.000000190921071,56.000000155530984],[11.000000134576112,56.000000155530984],[12.000000245869273,56.000000155530984],[12.000000245869273,55.000000211876],[12.000000245869273,54.00000010058284],[12.000000245869273,53.00000015692797],[12.000000245869273,52.00000021327298],[12.000000245869273,51.00000010197982],[12.000000245869273,50.00000015832478],[12.000000245869273,49.00000004703179],[12.000000245869273,48.000000103376806],[11.000000134576112,48.000000103376806],[10.000000190921071,48.000000103376806],[9.000000247266257,48.000000103376806],[8.000000135973096,48.000000103376806],[7.000000192318055,48.000000103376806],[6.000000248663241,48.000000103376806],[6.000000248663241,49.00000004703179],[6.000000248663241,50.00000015832478],[6.000000248663241,51.00000010197982],[6.000000248663241,52.00000021327298],[6.000000248663241,53.00000015692797],[6.000000248663241,54.00000010058284],[6.000000248663241,55.000000211876],[6.000000248663241,56.000000155530984]]]}
          ]
      }
    }
```
Demo:

<LeafletDemo/>

Now, for example, you could add a function that creates a new marker each time you click on the map.

First steps with Leaflet and React

For those who do not yet know the corresponding contact points for current satellite images, there are a few further links from which you can download global satellite images for the most part free of charge.

## Download satellite images

### Copernicus Open Access Hub

The [Copernicus Open Access Hub](https://scihub.copernicus.eu/dhus) (Sentinels Scientific Data Hub) provides free and open access to Sentinel-1, Sentinel-2 and Sentinel-5P products.

Sentinel data is also available through Copernicus Data and Information Access Services (DIAS) on multiple platforms.

[https://www.copernicus.eu/de/datenzugriff/dias](https://www.copernicus.eu/de/datenzugriff/dias)

The Sentinel satellites are satellites of [ESA](https://www.esa.int/ESA) as part of the Copernicus program. This program was created in cooperation with the EU and is intended to enable earth observation, particularly in the areas of the environment, transport, economy and security policy.

All Sentinel data available through the Sentinel Data Hub is subject to the [Legal Notice on the use of Copernicus Sentinel Data and Service Information](https://sentinels.copernicus.eu/documents/247904/690755/Sentinel_Data_Legal_Notice).

### GEOSS Portal

The [GEOSS Portal](http://www.geoportal.org/) is a map-based online user interface with which users can download earth observation data and resources from various providers from all over the world.

This portal is also operated by the European Space Agency (ESA).

### Worldview - NASA

[NASA Worldview](https://worldview.earthdata.nasa.gov/) offers an interactive user interface to search for high-resolution and global satellite images. Thematic pictures of forest fires, air quality, flood monitoring and more are also offered.

### European Space Imaging

[European Space Imaging](https://www.euspaceimaging.com/) is the leading provider of VHR (Very High Resolution) satellite images for Europe, North Africa and the CIS countries.

### GloVis

Since 2001, the [USGS Global Visualization Viewer](https://glovis.usgs.gov/) (GloVis) has been available for access to remote sensing data. In 2017, it was redesigned to take account of changing Internet technologies. With easy-to-use navigation tools, users can instantly view and download scenes.

### GeoStore - Airbus Defence and Space

In the [GeoStore](https://www.intelligence-airbusds.com/geostore/), operated by AIRBUS, high-resolution and current satellite images can be ordered.

### EOWEB GeoPortal - DLR

[EOWEB GeoPortal](https://eoweb.dlr.de/egp/) (EGP) is a multi-mission web portal for interactive access to the DLR earth observation database.


Satellite Imagery Download - High-Resolution

With [AWS](https://aws.amazon.com/ "AWS") (and in particular the [free AWS contingent](https://aws.amazon.com/en/free/ "free AWS account")) you have the option of a static website with a custom domain for a few Hosting cents a month including CDN via CloudFront and CI / CD integration.

Before I switched completely to AWS, I had a common shared hosting option that cost me around € 72 a year. With this option I had
+ 250 GB SSD storage space,
+ free unlimited traffic,
+ 6 domains included,
+ 250 GB SSD mail storage space,
+ 25 MySQL databases,
+ and, and, and.

On the whole, much more than I need to run my static [GatsbyJS](https://www.gatsbyjs.com/) website.

So why shouldn't I only use and pay for resources that I ultimately need and also get some cloud computing experience?

## Creating S3-Buckets 

The basis for hosting on AWS is formed by [S3 Buckets](https://s3.console.aws.amazon.com/s3/home "S3 Buckets"). 
Buckets are "containers" on the web where you can save files.
In order for redirects from subdomains such as **www.mxd.codes** to **mxd.codes** to work, you need a bucket for each domain.

First of all, you create an S3 bucket for the root domain. In my case, the bucket name is the domain name mxd.codes and you select a region (for example EU (Frankfurt)).
The default settings can be kept under Options, unless you want to save different versions or access protocols.
So that everyone can access the website content later, remove the tick that is present by default at "Block *any* public access", check the bucket settings again and finally create it.

In each bucket you can or should enter a bucket policy that further defines access.
To do this, click on the name of the bucket and go to "Permissions" -> "Bucket Policy".

The following guideline must then be saved for public access.
```json
{
    "Version": "2008-10-17",
    "Statement": [
        {
            "Sid": "AllowPublicRead",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::mxd.codes/*"
        }
    ]
}
```
**"mxd.codes" has to be replace with your bucket name!**

If everything was done correctly, the permissions should now look something like this:

![S3 Bucket Richtlinie](/uploads/bucket_richtline_d32ff17da6.png)

In the bucket settings you have to activate the "hosting a static website" and specify an index document and an error document.
For GatsbyJS that would be **index.html** and **404.html**.

![S3 Bucket Hosting](/uploads/bucket_hosting_6f40697460.png)

Now the S3 bucket for the subdomain **www.mxd.codes** is still missing.
So create a new bucket with the name of the subdomain **www.mxd.codes** with public access and add the bucket policy.

In the settings for "hosting a static website" you use "redirect requests" and enter the target bucket **mxd.codes** and you can enter **https** as a protocol, because later on the content of the static website is delivered via CloudFront, which can be encrypted with SSL certificates.

![S3 Bucket Redirect](/uploads/bucket_redirect_740544f6d5.png)

The buckets are now created and correctly configured for the operation of a static website including redirect.


## CloudFront Distributions

With the free AWS contingent, 50 GB of data transfer can be burned per month in Cloudflare.

With a page size of generous 4 MB, this is enough for 12,500 page views per month and should therefore be more than sufficient for a website with average traffic. So why not take a free CDN with you?

If the costs incurred after the free year should deter someone, you still have the option to switch to another CDN provider such as [CloudFlare](https://www.cloudflare.com "CloudFlare").

At [CloudFront](https://console.aws.amazon.com/cloudfront/ "CloudFront") you have to create a web distribution for **each** bucket.
As the origin domain name, a bucket can **not** be selected from the dropdown list, but the end point of the bucket from S3 must be copied.

![Cloudfront Origin](/uploads/cloudfront_origin_f1e2731789.png)

In the distribution for the bucket **mxd.codes**, for example "mxd.codes.s3-website.eu-central-1.amazonaws.com" is specified as origin.
"Origin ID" will then be filled in automatically. In "Viewer Protocol Policy" **Redirect HTTP to HTTPS** is selected because users should only be able to access the website via HTTPS.  "Compress Objects Automatically" **Yes** can be selected, so that CloudFront will zip all files automatically.
Under "Alternate Domain Names (CNAMES)" you have to specify the bucket for the root domain for the distribution for the root domain. For example mxd.codes

At "SSL-Certifacte" you can now create for the two domains **mxd.codes** and **www.mxd.codes** two free Amazon SSL certificate via the [Certificate Manager (ACM )](https:/https://aws.amazon.com.amazon.com/en/certificate-manager/ "Certificate Manager (ACM)"). To do this, add your two domains in ACM.

![ACM Domain](/uploads/acm_domain_ee3a548091.png)

You can now have this validated using a DNS or email method. If you have included your domain in Route 53, you can do it more or less automatically by simply following the instructions.

Back in the CloudFront Distribution Creation you only have to specify a "Default Root Object". -> index.html
If you don't do this, CloudFront always shows an "Access Denied" message in XML format when you access your domain (atleast that was my case).

Finally, the distribution must of course still be activated under "Distribution State".

First distribution finished. The same procedure now for the subdomain **www.mxd.codes** with the corresponding "Origin Domain Name" (**Bucket end point!**)

_This can take up to 20 minutes_
(If you want to clear your CloudFront cache and have already installed AWS CLI, you can do this with the following command:

```shell
aws cloudfront create-invalidation --distribution-id DEINE_DISTRIBUTION ID --paths "/*"
```

In the meantime, you can create the redirects for Cloudfront in Route 53.

## Route 53 DNS settings

In [Route 53]("Route 53") you need a hosted zone (= 0.50€ per month). Then A (and provided that in the CLoudFrontDistribution IPv6 is
activated (which it is by default), as well as an AAAA) data record can be created.

That means you basically need 4 "alias" data sets:
+ mxd.codes A "CloudFront URL for root domain"
+ mxd.codes AAAA "CloudFront URL for root domain"
+ www.mxd.codes A "CloudFront URL for subdomain"
+ www.mxd.codes AAAA "CloudFront URL for subdomain"

Exceptionally, you can select the CDN url from the dropdown list.
With "Routing Guideline" and "Assess the state of the target" you can leave the default settings, unless you want to experiment.

Now if you wait a little you should be redirect from
+ http://wwww.mxd.codes,
+ https://wwww.mxd.codes,
+ wwww.mxd.codes,
+ http://mxd.codes and
+ mxd.codes
to **https://mxd.codes** .


Host a static website with your own domain, AWS S3 and CloudFront

## The difference between GIS and CAD

[Geographic information system](/articles/what-is-gis-geographic-information-system "What is GIS?") (GIS) describes a system for the display and processing of geodata, ie data to which a spatial position has been assigned. With a geographic information system, complex issues can be presented, presented and analyzed in a structured manner.

With CAD systems you can create digital content and/or model it graphically. These are usually plans or drawings, as well as 3D models.

Since the projected data in both systems should be related to the real world, different scales, sizes and levels of detail are used. In GIS this is regulated by different display models, where it is defined which objects are displayed under which conditions. In CAD, standards play a more important role. The main differences between CAD and GIS are:

* [GIS data or geodata](/articles/what-are-geodata "GIS data or geodata") need a local reference in contrast to CAD data
* In GIS, the focus is on the visualization, maintenance and analysis of data;
* In CAD, a very precise representation plays a more important role with the help of the plans e.g. Components can be produced
* GIS data is very diverse and a wide variety of data formats and sources can come together
* CAD data are mainly saved as DXF or DWG files
* A GIS is much more efficient and flexible in data management than a CAD system

Both systems can also be used in combination because they complement each other perfectly.

GIS vs CAD - The difference between GIS and CAD

The shapefile format is a general format for storing vector [GIS](/articles/what-is-gis-geographic-information-system "What is GIS?") - [data](/articles/geodata-what-are-geodata "What is geodata?"). Shapefiles can store non-topological vector data together with the associated attribute data.

Shapefiles were developed by Esri and are now an open format and a popular option for data transfer. For example, shapefiles can be read directly by nearly all [GIS software programs](/articles/gis-software-options-free-open-source-and-proprietary "GIS software options") such as ArcGIS and QGIS.

Although the name of a shapefile indicates a single file, a shapefile is actually a collection of at least three basic files: .shp, .shx and .dbf. All three files must be in the same directory so that they can be viewed.

There may be additional files such as a .prj with the projection information of the shapefile. Shapefiles are usually compressed in a ZIP file to be transmitted, for example by email as an attachment or as a download link on a website.

## Which file extensions are associated with a shapefile?

First of all, all files for a shapefile must have the same name, but different file formats. There are three required files that make up at least one shapefile.

*   .shp: The geometry of a data record are saved in the file with the extension .shp. If you want to transform coordinates for example, you only have to transform this file.
*   .dbf: Here all factual data or attribute data are saved in [dBASE](https://www.wikipedia.com/wiki/DBASE) format.
*   .shx: Links the factual data (.dbf) with the geometry (.shp) via a common index.

In addition to these three files, which must always be available, there are a number of optional files.

*   .atx: attribute index
*   .sbx: and .sbn Spatial index
*   .qix: Alternative spatial index (used and created by [GDAL](https://www.gdal.org/ "Geospatial Data Abstraction Library")
*   .aih and .ain: index for table links
*   .shp.xml: metadata about the shapefile
*   .prj: projection of the data
*   .cpg: to specify the character set used in the [.dbf](https://www.wikipedia.com/wiki/DBASE "DBASE").

## Geometries

Only elements of one geometry type can be saved in a shapefile. For example

*   Points or
*   Lines or
*   Surfaces and polygons or
*   Multi-points (multiple points)

A geometry does not necessarily have to exist for a data record. Pure factual data can also be saved as a shapefile.

## Disadvantages of shapefiles

*   Shape files are relatively sluggish
*   A shapefile always has several files (everyone knows the problem if you only get the .shp sent)
*   Attribute names are limited to 10 characters
*   No possibility to save topologies in the data
*   The file size is limited to 2GB
*   Can only contain one geometry type per file
*   No real 3D support

## Alternatives to shapefiles

The best alternative to shapefiles are [GIS databases](/articles/gis-and-geo-database-management-system-options "GIS databases") e.g. PostGIS (PostgreSQL) or also [GeoPackages](https://www.wikipedia.com/wiki/GeoPackage).

In databases, the file size is theoretically unlimited, different geometry types can be saved and topologies can be created. All data in databases can be easily played out and sent, for example, as geopackage, which is just one handy file.

What is a shapefile? .shp, .dbf and .shx

## What is GIS data or geodata?

Geodata is information with a spatial reference that can be used, among other data, in a [Geographic Information System (GIS)](/articles/what-is-gis-geographic-information-system "What is GIS?").

They are divided into several categories. Basically there are primary data and secondary data.

*   **Primary data**: Primary data is raw data, i.e. data that comes directly from a data acquisition and is created, for example, when measuring a property.
*   **Secondary data**: Secondary data are revised primary data. They were calculated, modeled or otherwise processed.

All geodata consist of **object attributes** and **object geometries**. The attributes describe the object and the geometry determines the position and shape of the object. Objects can be represented as points, lines, surfaces or bodies.

Let's come back to Google Maps, for example. For example, if you search there for petrol stations in Honolulu, all the petrol stations nearby recorded are shown on a map.

**Object attributes** would be here, for example:

*   _Gas station_
*	_Brand_
*   _Address_
*   _Reviews_

In this case, the **object geometry** is a point with the corresponding coordinates. Google has chosen a "red, upside down raindrop" for this point for a nice display.

Furthermore, a distinction is now made between geodata.

**Geospatial base data:** Geospatial base data means basic official data. They serve as the basis for specialist cards and thematic cards. For example, landscape (topography), properties, data from Alkis or the surface of the earth. They are data from the surveying administration and are created by public bodies. The use of this commercial data is restricted by law.

**Geo-technical data:** Geo-technical data are data from various specialist areas and are shown on the corresponding maps. They are used in: urban planning, demography, environmental protection, education, social affairs, supply and disposal, disaster protection, real estate management, geomarketing

There are various ways of storing geographic information. Amongst other things:

* _in a [database](/articles/gis-and-geo-database-management-system-options "GIS- and Geo-Database Management System options")  e.g. PostgreSQL_
* _as [shapefile](/articles/what-is-a-shapefile "Shapefile")_
* _in a geodatabase_
* _as a raster image_
* _as table data_

Geodata can be obtained from public and private providers. The State Office for Digitization, Broadband and Surveying offers, for example, aerial photos and topographic maps, which are subject to legal restrictions. The costs for this are very different.

Depending on the area of ​​application, geodata must have certain quality features in order to fulfill its purpose. With underground lines, for example, it is not enough if they are accurate to a few meters, otherwise you would have to look for the line if repairs are required. When determining the position of the petrol station, it would theoretically not matter whether the point was now shifted by one or two meters, since a petrol station is normally larger than two meters. In principle, one can say that the measurement accuracy can be directly proportional to the dimension of an object.

In addition, geodata must also be logical, i.e. when collecting data for population mapping, no negative values ​​may occur. Time often plays an important role. In the following, objects with an exact location and time are four-dimensional. This is important, for example, for historical data or when analyzing population structures over a certain period of time.

## Conclusion

Geodata is spatial data that represents a real object. They include attributes that describe the object and a geometry that determines the position of the object. Geodata is divided into primary and secondary data and saved in databases, tabular data, as images or shapefiles. They are the medium for maps, which are represented in a geographic information system, in order to analyze complex topics and to make a decision.

Geodata - What are Geodata?

The abbreviation GIS stands for geographic information system.

A geographic information system or **GIS** is a system for displaying and processing geodata, i.e. data to which a spatial position has been assigned.

With a geographic information system, complex issues can be visualized, presented and analyzed in a structured manner.

A GIS market was established in the mid-1980s. Authorities and universities in particular first recognized the great benefits of using GIS. The rapid technical progress of computers enabled the first GIS jobs to be created.

Purely graphically, a GIS consists of one or more layers on which data or [geodata](/articles/what-are-geodata "what is geodata") are stored. As a definition of GIS, one could also call (like Arni) Google Maps with streets, places etc., because one can analyze and visualize geodata with Google Maps.

![Schwarzenegger GIS Tweet](/uploads/gis_schwarzenegger_tweet_9baaf453ab.jpg)

Google Maps helps to find the fastest way between two points/locations taking into account different modes of transport and is basically an interactive map.

> Incidentally, a free alternative to Google Maps is [OpenStreetMap](https://routing.openstreetmap.de/?z=7&center=50.004209%2C11.997070&hl=de&alt=0&srv=1 "OpenStreetMap"). OpenStreetMap, also OSM, is a project that deals with creating a free world map for everyone. This is licensed under the license "[Open Database License (ODbL) 1.0](http://opendatacommons.org/licenses/odbl/)" Open Database License (ODbL) 1.0 ")" and can therefore also be used commercially for free.

With a GIS, you have the option of throwing together a wide variety of geodata in order to be able to recognize new patterns from this data. For example, orthophotos, topographic maps, statistical data and even technical plans can be integrated.

These are displayed on different layers, which can be shown or hidden as required. A geographic information system with the corresponding data thus provides the answers to many questions.

*   _Where are the most potential buyers for a product located?_
*   _Where do supply and disposal lines run?_
*   _Where do school catchments run and how does this affect public transport?_

A geographic information system basically consists of:

*   _[Software](/articles/gis-software-options-free-open-source-and-proprietary "GIS software") or [Applications](/articles/gis-applications "GIS applications")_
*   _Hardware_
*   [_Data_](/articles/what-are-geodata "What are geodata")
*   Management / organization

![GIS Infografik](/uploads/GIS_Grafik_2_e1556304265569_9cf8a4eaf5.jpg)

## What GIS software is there?

There is relatively a large selection of GIS software options. These are available as open-source projects, which are provided free of charge, and as paid versions, which can cost up to several thousand euros for a fully equipped version.

A well-known **Open Source GIS** is [QGIS](https://www.qgis.org/en/site/ "QGIS"), which offers an extensive number of plugins and a very large community, which pushes the project forwards. QGIS is especially recommended for beginners, as there is a lot of documentation and you can of course use QGIS anywhere for free.

Two of the most popular proprietary GIS software ptions are Esri ArcGIS (Pro) and AutoCAD Map3D from Autodesk.

Which software is the better one cannot be said in general. This depends on several factors, but mainly the purpose and the budget available.

## Hardware

First of all, of course, you need a powerful computer on which the software works smoothly. The requirements of the GIS programs are similar to those of PC games, which is why a middle class "Gaming PC" offers a good GIS workplace.

More and more users are now also using mobile devices, for example tablets. These can be connected to GNSS receivers and you are able to record, edit and transfer high-quality geodata directly into a geographic information system from outdoor surveying work.

This saves a lot of time and work, since objects are measured outside in the classic way and this data is then incorporated in the office, which would be omitted here.

## Data - Geodata

Geodata is information that have a spatial reference. Everything happens somewhere. If you capture the "what" and "where" you will have geodata.

There are various ways of storing geodata and integrating them into a geographic information system. The most common options for storing geodata are:

*   in [Databases](/articles/gis-and-geo-database-management-system-options "GIS-Databases") (eg. PostgreSQL, MariaDB, MySQL etc.)
*   as [shapefile](/articles/what-is-a-shapefile "What is a shapefile") (format for spatial data developed by ESRI)
*   as GeoJSON
*   as XML or KML
*   in [CAD](/articles/gis-vs-cad-the-difference-between-gis-and-cad "difference GIS-CAD") formats like DXF / DWG
*   and much more (see [GIS data format](https://en.m.wikipedia.org/wiki/GIS_file_formats "GIS data formats"))

This data can be integrated into the geographic information system locally or via the web.

## Areas of application of GIS

Geographic information systems are mainly used in:

*   Municipalities, cities, counties, regional offices (GIS municipal)
*   Surveying and land registry
*   Energy supply (water, sewage, electricity, gas, etc.)
*   Property Management
*   Engineering offices (urban and traffic planning)


## Conclusion

A geographic information system has become indispensable in many private and public areas. Property registers, supply networks and much more can be integrated very easily in a GIS. This data can then be maintained in the geographic information system and serve as the basis for further analyzes. With the help of GIS, complex problems can be clearly visualized and decisions and solutions can be found quickly and effectively.

GIS vs CAD - The difference between GIS and CAD

The difference between GIS and CAD

Continue Reading

Free FME-licence for private use
Max DietrichMax Dietrich

How to create a Mapnik stylesheet for displaying any data from PostgreSQL/PostGIS
Max DietrichMax Dietrich

How i constantly track my location and display a web-map with all the locations
Max DietrichMax Dietrich

GIS vs CAD - The difference between GIS and CAD

The difference between GIS and CAD

Continue Reading

Free FME-licence for private useMax DietrichMax Dietrich

How to create a Mapnik stylesheet for displaying any data from PostgreSQL/PostGISMax DietrichMax Dietrich

How i constantly track my location and display a web-map with all the locationsMax DietrichMax Dietrich

Free FME-licence for private use
Max DietrichMax Dietrich

How to create a Mapnik stylesheet for displaying any data from PostgreSQL/PostGIS
Max DietrichMax Dietrich

How i constantly track my location and display a web-map with all the locations
Max DietrichMax Dietrich