This tutorial is designed to guide you through how to setup a Barman backup to Azure Blob Storage for a PostgreSQL instance and store it in the cloud, using barman-cloud-backup.
For those who haven't used it before, Barman (Backup and Recovery Manager) is an open-source administration tool for remote backups and disaster recovery of PostgreSQL servers in business-critical environments.
To demonstrate the setting up Barman backup to Azure Blob Storage, we are using the below environment:
OS: RHEL 8
Database: EPAS
Database Version: 14.5
Step 1: Install Barman
dnf -y install barman
Sample output:
[root@backuptest ~]# dnf -y install barman
Last metadata expiration check: 0:00:13 ago on Fri 27 Jan 2023 10:46:18 AM UTC.
Dependencies resolved.
============================================================================================================================================
Package Architecture Version Repository Size
============================================================================================================================================
Installing:
barman noarch 3.2.0-1.el8 edb 57 k
Installing dependencies:
libpq5 x86_64 15.1-42PGDG.rhel8 edb 216 k
python3-argcomplete noarch 1.9.3-6.el8 rhel-8-for-x86_64-appstream-eus-rhui-rpms 60 k
python3-barman noarch 3.2.0-1.el8 edb 461 k
python3-psycopg2 x86_64 2.8.6-1.rhel8 edb 178 k
Transaction Summary
============================================================================================================================================
Install 5 Packages
Total download size: 972 k
Installed size: 3.9 M
Downloading Packages:
(1/5): barman-3.2.0-1.el8.noarch.rpm 636 kB/s | 57 kB 00:00
(2/5): libpq5-15.1-42PGDG.rhel8.x86_64.rpm 2.1 MB/s | 216 kB 00:00
(3/5): python3-barman-3.2.0-1.el8.noarch.rpm 4.0 MB/s | 461 kB 00:00
(4/5): python3-psycopg2-2.8.6-1.rhel8.x86_64.rpm 5.0 MB/s | 178 kB 00:00
(5/5): python3-argcomplete-1.9.3-6.el8.noarch.rpm 430 kB/s | 60 kB 00:00
--------------------------------------------------------------------------------------------------------------------------------------------
Total 3.9 MB/s | 972 kB 00:00
EnterpriseDB RPMs 8.6 - x86_64 1.7 MB/s | 1.7 kB 00:00
Importing GPG key 0x7E30651C:
Userid : "EnterpriseDB Inc. (EnterpriseDB Yum Repositories) <packages@enterprisedb.com>"
Fingerprint: CA40 9F7C 635F 2AE5 6C9E 8B34 E5ED E919 7E30 651C
From : /etc/pki/rpm-gpg/ENTERPRISEDB-GPG-KEY
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : python3-argcomplete-1.9.3-6.el8.noarch 1/5
Installing : libpq5-15.1-42PGDG.rhel8.x86_64 2/5
Running scriptlet: libpq5-15.1-42PGDG.rhel8.x86_64 2/5
Installing : python3-psycopg2-2.8.6-1.rhel8.x86_64 3/5
Installing : python3-barman-3.2.0-1.el8.noarch 4/5
Running scriptlet: barman-3.2.0-1.el8.noarch 5/5
Installing : barman-3.2.0-1.el8.noarch 5/5
Running scriptlet: barman-3.2.0-1.el8.noarch 5/5
Verifying : barman-3.2.0-1.el8.noarch 1/5
Verifying : libpq5-15.1-42PGDG.rhel8.x86_64 2/5
Verifying : python3-barman-3.2.0-1.el8.noarch 3/5
Verifying : python3-psycopg2-2.8.6-1.rhel8.x86_64 4/5
Verifying : python3-argcomplete-1.9.3-6.el8.noarch 5/5
Installed products updated.
Installed:
barman-3.2.0-1.el8.noarch libpq5-15.1-42PGDG.rhel8.x86_64 python3-argcomplete-1.9.3-6.el8.noarch
python3-barman-3.2.0-1.el8.noarch python3-psycopg2-2.8.6-1.rhel8.x86_64
Complete!
[root@backuptest ~]#
[root@backuptest ~]# barman --version
3.2.0 Barman by EnterpriseDB (www.enterprisedb.com)
[root@backuptest ~]#
Step 2: Install barman-cli-cloud
dnf install barman-cli-cloud
Sample output:
[root@backuptest ~]# dnf install barman-cli-cloud
Last metadata expiration check: 0:01:09 ago on Fri 27 Jan 2023 10:46:18 AM UTC.
Dependencies resolved.
============================================================================================================================================
Package Architecture Version Repository Size
============================================================================================================================================
Installing:
barman-cli-cloud noarch 3.2.0-1.el8 edb 59 k
Transaction Summary
============================================================================================================================================
Install 1 Package
Total download size: 59 k
Installed size: 73 k
Is this ok [y/N]: y
Downloading Packages:
barman-cli-cloud-3.2.0-1.el8.noarch.rpm 362 kB/s | 59 kB 00:00
--------------------------------------------------------------------------------------------------------------------------------------------
Total 355 kB/s | 59 kB 00:00
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : barman-cli-cloud-3.2.0-1.el8.noarch 1/1
Running scriptlet: barman-cli-cloud-3.2.0-1.el8.noarch 1/1
Verifying : barman-cli-cloud-3.2.0-1.el8.noarch 1/1
Installed products updated.
Installed:
barman-cli-cloud-3.2.0-1.el8.noarch
Complete!
[root@backuptest ~]#
Step 3: Create a storage account on Azure
Fill all the required details to create a storage account
Step 4: Create a container in your Azure storage account
Step 5: Identify AZURE_STORAGE_CONNECTION_STRING and AZURE_STORAGE_KEY
We can find same from the storage account page, left panel under Security + networking > Access Keys
AZURE_STORAGE_KEY=> key
AZURE_STORAGE_CONNECTION_STRING=>connection key
Step 6: generate SAS for a container
Select the container created in step 4 and right click for drop down list to select “generate SAS” for AZURE_STORAGE_SAS_TOKEN
AZURE_STORAGE_SAS_TOKEN => Blob SAS token
Step 7: Install azure-storage-blob
sudo pip3 install azure-storage-blob
Sample output:
[enterprisedb@backuptest ~]$ which pip3
/usr/bin/pip3
[enterprisedb@backuptest ~]$ which python3
/usr/bin/python3
[enterprisedb@backuptest ~]$
[enterprisedb@backuptest ~]$ sudo pip3 install azure-storage-blob
[sudo] password for enterprisedb:
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting azure-storage-blob
Downloading https://files.pythonhosted.org/packages/dd/fb/1501707ae8d921079ea826d16926e1b9b179d15264a2d239a08d7b374522/azure_storage_blob-12.13.1-py3-none-any.whl (377kB)
100% |████████████████████████████████| 378kB 2.4MB/s
Collecting msrest>=0.6.21 (from azure-storage-blob)
Downloading https://files.pythonhosted.org/packages/15/cf/f2966a2638144491f8696c27320d5219f48a072715075d168b31d3237720/msrest-0.7.1-py3-none-any.whl (85kB)
100% |████████████████████████████████| 92kB 9.7MB/s
Collecting azure-core<2.0.0,>=1.23.1 (from azure-storage-blob)
Downloading https://files.pythonhosted.org/packages/4f/43/18cac1c7c1c1c3ad9d0dc5f262bd5633f7b15a6b5f568dcb10a8ac8990aa/azure_core-1.24.2-py3-none-any.whl (178kB)
100% |████████████████████████████████| 184kB 5.9MB/s
Requirement already satisfied: cryptography>=2.1.4 in /usr/lib64/python3.6/site-packages (from azure-storage-blob)
Collecting requests-oauthlib>=0.5.0 (from msrest>=0.6.21->azure-storage-blob)
Downloading https://files.pythonhosted.org/packages/6f/bb/5deac77a9af870143c684ab46a7934038a53eb4aa975bc0687ed6ca2c610/requests_oauthlib-1.3.1-py2.py3-none-any.whl
Requirement already satisfied: requests~=2.16 in /usr/lib/python3.6/site-packages (from msrest>=0.6.21->azure-storage-blob)
Collecting certifi>=2017.4.17 (from msrest>=0.6.21->azure-storage-blob)
Downloading https://files.pythonhosted.org/packages/71/4c/3db2b8021bd6f2f0ceb0e088d6b2d49147671f25832fb17970e9b583d742/certifi-2022.12.7-py3-none-any.whl (155kB)
100% |████████████████████████████████| 163kB 6.5MB/s
Collecting isodate>=0.6.0 (from msrest>=0.6.21->azure-storage-blob)
Downloading https://files.pythonhosted.org/packages/b6/85/7882d311924cbcfc70b1890780763e36ff0b140c7e51c110fc59a532f087/isodate-0.6.1-py2.py3-none-any.whl (41kB)
100% |████████████████████████████████| 51kB 10.4MB/s
Collecting typing-extensions>=4.0.1 (from azure-core<2.0.0,>=1.23.1->azure-storage-blob)
Downloading https://files.pythonhosted.org/packages/45/6b/44f7f8f1e110027cf88956b59f2fad776cca7e1704396d043f89effd3a0e/typing_extensions-4.1.1-py3-none-any.whl
Requirement already satisfied: six>=1.11.0 in /usr/lib/python3.6/site-packages (from azure-core<2.0.0,>=1.23.1->azure-storage-blob)
Requirement already satisfied: cffi!=1.11.3,>=1.8 in /usr/lib64/python3.6/site-packages (from cryptography>=2.1.4->azure-storage-blob)
Collecting oauthlib>=3.0.0 (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob)
Downloading https://files.pythonhosted.org/packages/7e/80/cab10959dc1faead58dc8384a781dfbf93cb4d33d50988f7a69f1b7c9bbe/oauthlib-3.2.2-py3-none-any.whl (151kB)
100% |████████████████████████████████| 153kB 7.0MB/s
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/lib/python3.6/site-packages (from requests~=2.16->msrest>=0.6.21->azure-storage-blob)
Requirement already satisfied: idna<2.8,>=2.5 in /usr/lib/python3.6/site-packages (from requests~=2.16->msrest>=0.6.21->azure-storage-blob)
Requirement already satisfied: urllib3<1.25,>=1.21.1 in /usr/lib/python3.6/site-packages (from requests~=2.16->msrest>=0.6.21->azure-storage-blob)
Requirement already satisfied: pycparser in /usr/lib/python3.6/site-packages (from cffi!=1.11.3,>=1.8->cryptography>=2.1.4->azure-storage-blob)
Installing collected packages: oauthlib, requests-oauthlib, certifi, isodate, typing-extensions, azure-core, msrest, azure-storage-blob
Successfully installed azure-core-1.24.2 azure-storage-blob-12.13.1 certifi-2022.12.7 isodate-0.6.1 msrest-0.7.1 oauthlib-3.2.2 requests-oauthlib-1.3.1 typing-extensions-4.1.1
[enterprisedb@backuptest ~]$
Step 8: Export environment variables
Export the below variables captured in step 5 and 6
export AZURE_STORAGE_CONNECTION_STRING=
export AZURE_STORAGE_KEY=
export AZURE_STORAGE_SAS_TOKEN=
Step 9: Execute backup
barman-cloud-backup --cloud-provider azure-blob-storage -J 4 https://barmanbackup.blob.core.windows.net/newcontainer prodbkp -v
Note: please use barman-cloud-backup --help to look for more option
Sample output:
On the backup completion:
2023-01-27 11:17:48,491 [11656] INFO: Backup end at LSN: 0/3000138 (000000010000000000000003, 00000138)
2023-01-27 11:17:48,491 [11656] INFO: Backup completed (start time: 2023-01-27 11:17:46.521570, elapsed time: 1 second)
2023-01-27 11:17:48,491 [11661] INFO: Upload process stopped (worker 0)
2023-01-27 11:17:48,491 [11662] INFO: Upload process stopped (worker 1)
2023-01-27 11:17:48,494 [11664] INFO: Upload process stopped (worker 3)
2023-01-27 11:17:48,494 [11663] INFO: Upload process stopped (worker 2)
[enterprisedb@backuptest ~]$
You can confirm the same from the portal
DISCLAIMER:
- Please note that this sample is prepared and tested on the POC environment, you may need to change it according to your environment to make use of it.
- Please note that it is always advisable to test this in the lower environment before using it in the production environment.
Experience PostgreSQL peace of mind with Barman
Backup and recovery capabilities are essential to any database environment, especially one as complex and feature-rich as PostgreSQL. With Barman, you can rest assured that you have constant protection against data loss and a flexible backup strategy suited to your unique needs. Barman backup to Azure Blob Storage is just one of many ways to protect your most essential assets.
Read more of EDB's Barman documentation here.