How to copy your Proxmox backups with AzCopy to Azure Storage Containers

Recently I went out of storage for my homelab so I bought an used NAS (Synology DS214 play) to have some more capacities for Proxmox Backups and OpenStreetMap. I still had a 1TB hdd lying around at home, which I now use for proxmox backups.

To have some redundancy (and to learn something new) I decieded to copy the Proxmox backups to the cloud, in particular to an Azure Storage Account with AzCopy and in the following I will describe with more details how I was able to do it.

Overall this article will cover the following informations:

• Creating an Azure Storage Account
• Getting started with AzCopy
• Creating a bash-script to copy the Proxmox backups to an Azure Storage Container

Creating an Azure Storage Account

First off all you need an active Azure subscription and an storage account to be able to store your backups. In the Azure Portal you can search for the service "Storage Accounts" which you will need.

In the service "Storage Accounts" you can create a new storage account. For the storage account you will need

• an active azure subscription,
• a ressource group (create one if you don't have one it, e.g. RG-HOMELAB),
• a storage account name,
• selecting a region and
• selecting redundancy: pick LRS, GRS or RA-GRS, because ZRS, GZRS and RA-GZRS accounts cannot move blobs to the archive tier at all.
• Access tier "Cool" (See Advanced)

Note that Azure has since added a separate tier actually called "Cold", which sits between Cool and Archive and can also be set as the account default. Archive is the only tier that cannot. Cool is a fine choice here.

You can keep all the other settings as default. After your Storage Account has been deployed you can add a lifecycle rule from "Lifecycle Management" which will move files from the "Cool" access tier to the archive storage.

For example I created a rule which moves all new files after one day to the archive storage tier.

By storing files in archive storage instead of in the regular "Cool" access tier you can actually save about 82%. But keep in mind that accessing data in the Archive storage is more expensive than in the cold (or any other) storage tier.

Check the minimum retention periods before you set a short rotation. Archive bills a minimum of 180 days per blob and Cool a minimum of 30 days. Delete a blob earlier than that and you are still charged for the remainder. If you rotate your backups weekly, moving them to Archive after one day will cost you more, not less. The archive tier pays off when the blobs are genuinely meant to sit there.

Also you could create another rule which will for example will delete all all blobs which were created 365 days ago.

Please have a look at https://azure.microsoft.com/en-us/pricing/details/storage/blobs/ for uptodate Azure Storage pricing.

After the storage account has been configured you will need to create a Container where the actual files will be stored. Go to "Data storage" -> "Containers" and create a Container. Again name it however you want.

AzCopy can authenticate with Entra ID unattended nowadays through the AZCOPY_AUTO_LOGIN_TYPE environment variables. What does not work from cron is the interactive azcopy login, because it needs somewhere to cache the token and a headless box has no keyring. A SAS token keeps this simple, so that is what I used here. You can create a SAS token in the container at "Shared access tokens".

For the Shared access token select the permissions Read, Add, Create and Write, and select an expiry date for security reasons. Read is needed even though this script only uploads, because azcopy checks whether a blob already exists before skipping it, and without Read that check fails with a 403 on every file. Then you can generate the SAS token and URL. Copy that Blob SAS URL because you will need it for the upload script.

Keep in mind that the SAS URL is a credential: anyone holding it can write to your container until it expires. The script below therefore gets mode 700, and you will want to note the expiry date somewhere, because uploads simply start failing on the day it lapses.

Getting started with AzCopy

AzCopy is a command-line utility that you can use to copy blobs or files to or from a storage account. This article helps you download AzCopy, connect to your storage account, and then transfer data. (https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10)

To get AzCopy for Linux you have to download a tar file and decompress the tar file anywhere you like. You can then just use AzCopy because it's an executable file, so nothing has to be installed.

Run these as root (on Proxmox you already are), because they write to /usr/bin:

bashCopy code
#Download AzCopy
cd ~
wget https://aka.ms/downloadazcopy-v10-linux
 
#Expand Archive
tar -xvf downloadazcopy-v10-linux
 
#(Optional) Remove existing AzCopy version
rm -f /usr/bin/azcopy
 
#Move AzCopy to the destination you want to store it
cp ./azcopy_linux_amd64_*/azcopy /usr/bin/
 
#Remove the download and the extracted directory again
rm -f downloadazcopy-v10-linux
rm -rf ./azcopy_linux_amd64_*/

Because /usr/bin is already on the default PATH, azcopy is now callable from any directory. Check it with:

bashCopy code
azcopy --version

There is no need to edit ~/.profile for this, and I would advise against it. An earlier version of this article suggested adding azcopy to PATH there, which was both unnecessary and wrong (PATH holds directories, not the path to a binary). If you do ever edit ~/.profile, never put the line source ~/.profile inside ~/.profile itself: it makes the file call itself forever and you will not be able to open a shell again.

Creating a bash-script to synchronize the Proxmox backup directory to an Azure Storage Container

The only piece missing now is the script which will upload the the Proxmox backup files to the previously created azure storage container after the backup task has finished. For copying the backups to Azure we will use azcopy copy because acopy uses less memory and incurs lower billing costs because a copy operation does not need to index the source or destination before moving files in comparison to azcopy sync. With --overwrite=false azcopy skips any file whose name already exists in the container, so each backup is uploaded exactly once. Proxmox puts a timestamp in every dump filename, so a new backup is always a new name and nothing is re-uploaded, which keeps bandwidth down and works well with the lifecycle rule above. (It is a name check, not a timestamp comparison. If you ever need azcopy to compare last-modified times instead, that is --overwrite=ifSourceNewer.)

For automatically starting the upload after the backup has finished we can use a hook script for vzdump. Create and test the script first, and wire it into vzdump last. vzdump checks the hook when a job starts and aborts the whole backup run if the file is missing or not executable, so configuring it before the script exists means no backups at all in the meantime.

Create the script, with restrictive permissions from the start so the SAS URL is never briefly world readable:

bashCopy code
mkdir -p /root/scripts
chmod 700 /root/scripts
install -m 700 /dev/null /root/scripts/upload-backups-to-azure.sh
nano /root/scripts/upload-backups-to-azure.sh

Then copy paste the following content into the file and replace the content for src with the location of your dumps. Note that there is "/*" at the end of src so that only the files inside the directory will be copied. Also replace token with the Blob SAS URL.

bashCopy code
#!/bin/bash
# Proxmox vzdump hook: copy finished backups to Azure Blob Storage.
# Referenced from /etc/vzdump.conf. vzdump calls this once per phase and
# passes the phase name as $1.

src="/mnt/pve/xyz/dump/*"
token="Blob SAS URL"

dobackup() {
  # vzdump clears the environment before running a hook, so almost nothing is
  # set here. HOME matters because azcopy writes its job plan and log files
  # under $HOME/.azcopy, which becomes /.azcopy in the filesystem root when
  # HOME is unset.
  export HOME=/root
  export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

  echo "Uploading Proxmox backups from $src to Azure..."
  if azcopy copy "$src" "$token" --overwrite=false; then
    echo "Finished Uploading!"
  else
    echo "ERROR: azcopy failed - backups were NOT uploaded to Azure" >&2
  fi
}

# Use = and not ==. The == form is a bash extension inside [ ], so running this
# file with `sh script.sh` fails with "[: unexpected operator" on this line.
# ${1:-} is habit rather than necessity here, and keeps the test correct if
# anyone later adds `set -u`.
if [ "${1:-}" = "job-end" ]; then
  dobackup
fi

exit 0

Two things about that script are deliberate. It reports an azcopy failure to stderr but still exits 0, because vzdump treats a non-zero exit from a hook as a failed job, and at job-end your backups are already safely written locally, so failing the whole task would be misleading. The message lands in the Proxmox task log, which is where you should look if you suspect an upload did not happen. If you would rather have the task go red on a failed upload, replace that echo line with exit 1.

Also note that a Proxmox dump directory contains small .log and .notes files next to the actual archives, so those get copied as well. That is harmless, but it is why you may see a few very small blobs appear alongside the large ones. If you only want the archives, add --include-pattern="*.vma*;*.tar*" to the azcopy command. Note there is no dot after vma and tar, because *.vma.* would match only compressed dumps and silently skip every backup taken with compression set to none.

Close the file and make it executable. Use 700 rather than +x: the file contains your SAS URL, and the default 755 would let any local user read it.

bashCopy code
chmod 700 /root/scripts/upload-backups-to-azure.sh

Test the hook by hand before wiring it up. It should upload and print "Finished Uploading!":

bashCopy code
/root/scripts/upload-backups-to-azure.sh job-end

Only once that works, add the following line to the end of the "/etc/vzdump.conf" file:

bashCopy code
script: /root/scripts/upload-backups-to-azure.sh

vzdump runs as root, so keep the script somewhere root owns. /etc/vzdump.conf is Proxmox's own global config file, so this hook applies to every backup job on the node. That is usually what you want, but it is worth knowing. To disable the upload again, comment that line out rather than deleting the script, because a script: line pointing at a missing file stops backups running at all.

Now the next time your backup task has finished the files will be automatically uploaded to your Azure storage container. Due to the hook script you can check the status of the copy process in the proxmox ui.