It took longer than expected, but we have finally managed to release Barman as open-source under GNU GPL 3.
Barman stands for “Backup and Recovery Manager” and it is an administration tool for disaster recovery of PostgreSQL servers. Currently only Linux systems are officially supported, however Python allows to plan porting and maintenance on different platforms as well.
The idea of starting Barman began about a year ago. We were tired of managing disaster recovery solutions using custom scripts, hard to maintain and to monitor (you probably know what I mean).
Also, we needed a tool specifically designed for disaster recovery – and not high availability as well. Some of the features we were eager to see:
- remote backup
- management of multiple servers
- catalogue of backups
- management of retention policies
- incremental backup
- compression of WAL files
- etc.
None of the tools available as open source was able to offer all of these features at once.
Remote backup allows database architects to organise a business continuity and/or high scalability cluster (made up of several Postgres servers) so that physical backup of the database server is not invasive.
What we can do now with Barman is to simply setup the archive_command on the master. The rest is performed from a remote server. A simple/typical scenario is the one where one backup server sits in the middle of your Postgres cluster and manages all the backup activity. In case, you can manage the recovery process from that same console – even though you always hope that recovery operations are limited to regular simulations only (remember, it is very important to simulate recovery, usually every 6 months at least).
I already mentioned the centralised architecture of the backup, where you can have a backup server regularly receiving WAL information from all the configured Postgres servers. I said ‘all’, because with Barman you can manage backup (and recovery) of multiple Postgres servers.
Remote backup and management of multiple servers are really handy features. However, one of the features I love the most of Barman is the ability to keep a catalogue of backups per each server. The “barman list-backup” command on a given server, allows you to see all the available periodical backups in your Barman installation for that server.
The dual management of periodical backups (base backups) and WAL archive, allow Barman to assign a specific WAL file to the closest previous backup available. This way, you can actually see the size of a backup, with or without the associated stream of WAL files. (Neat!)
WAL files can be compressed using standard compression algorithms such as gzip and bzip2. Compression is handled by the backup server (not the server).
The catalogue of backups has made much simpler the setup and the implementation of backup policies and retention policies for a Postgres database server. Unfortunately, it is not yet possible to configure retention policies directly in the configuration file, like “recovery window of 30 days” or “redundancy 5” (last 5 periodical backups). But we will get there soon (sponsors are welcome).
Last but not least. Incremental backup. This feature is not that important in small size environments, I know. But it is crucial for large and very database servers (1TB and over). It will definitely be available in the next versions of Barman (again we are looking for sponsors here).
I must admit that we could have released Barman months ago. However, we decided to release it when it was stable enough, being the software highly critical (we are talking about disaster recovery).
We had the chance to install it and test it on some business critical environments in Italy, and the feedback has been great.
So … sorry if it took so long (we started talking about it at PGConf.EU 2011 and PGDay.IT 2011), but I really hope that all of this waiting has been worthwile.
Finally, I want to thank all the other members of the Italian team that worked on the project and supported the development: Marco (lead programmer), Carlo, Gianni and Giulio.