We very often hear about devops culture, lean and agile methodologies, kanban, pair programming, peer review, testing, and many more; but how many of us could effectively put these things into practice?
This is the part 1 of a 3-part series, in which we will share our experience in these areas, showing the principles, processes and technologies harnessed by 2ndQuadrant. We will also explore the fundamental importance of collaboration and communication among all team members.
We will demonstrate this primarily by using a case study of the development of Barman, an open source backup and disaster recovery software for PostgreSQL databases. Created in 2012, Barman is written in Python and fully developed by 2ndQuadrant engineers.
How Barman was born
Around 2010, a few potential customers contacted us to migrate their databases from Oracle to PostgreSQL. However, there was a large limitation for the adoption of the elephant database and that was the lack of clear, usable and “standard” tools for backup and disaster recovery of a PostgreSQL database. The only way to proceed back then was to write custom scripts tailored to their work environment.
It was in that moment, that we decided to design Barman! To do so, we began with the user interface, writing down both console commands and a hypothetical configuration file. By utilizing our experience in disaster recovery for PostgreSQL and unleashing our creativity, we began to define the ways we wanted to communicate with this tool.
At the technological level, we decided to trust Python and as a license we aimed for GNU GPL 3.
Following an agile approach, incremental prototype development began, culminating in the release of version 1.0 in 2012. We have now reached version 2.1, the 20th release of Barman in just under 5 years. The development team is now composed of more than 5 people, coordinated by our Italian office.
Barman is definitely, at this time, one of the world’s most used tools for PostgreSQL database backup and disaster recovery.
The devops culture
One of the fundamental principles of Lean methodologies that we apply during our work is to see what we do as a single flow, going beyond each single compartment. This is known as value stream.
Through this process, we are able to share goals among all team members. By collaborating, integrating, and communicating with each other regardless of role, the devops culture leads to growth for individuals, the team as a whole, the project and company.
The main objective of the devops culture is to break down the barriers that exist between departments operating in different silos, totally disjointed at the level of production processes (between developers and administrators, for example). With a small play on words, devops actually means more “group work” than “workgroup”.
Additionally, devops culture promotes continuous improvement and preventive experimentation in order to master the tools, techniques and processes of a team. All of these fascinating principles are translated into useful technologies, tools (such as Redmine, Git, Vagrant, Puppet, Docker, etc.) and work/task organization methodologies.
Pro Tip: “The Phoenix Project”, written by Gene Kim, Kevin Behr and George Spafford, and “The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations”, written by Gene Kim, Jez Humble, Patrick Debois and John Willis are both great references for the application of devops methodologies within businesses.
Kanban and 2ndQuadrant
In 2013 we decided to adopt Kanban for managing workflow within the Italian office. Kanban is a Japanese word, the word “Kan” means “visual” and “ban” means “card”; so Kanban refers to visual cards used on the board, was introduced by Toyota in the 1950s for ‘Just In Time’ cars. For over a decade, Kanban has been successfully adopted and utilized by the IT industry.
For those who want to learn more, we recommend reading the following: “Kanban: Successful Evolutionary Change for Your Business Technology”, written by David J. Anderson.
During the OSS4B event, organized by 2ndQuadrant in Prato, Italy in September 2013, we had the opportunity to meet one of the world’s leading experts in Kanban, Dragos Dumitriu. During this event, a close friendship was born between Dragos and 2ndQuadrant Italia. Thanks to Kanban, 2ndQuadrant Italia has managed to reduce its production time by 400% compared to the previous years.
The average delivery time for a task, before Kanban, was estimated to be around 15 days, while it is now 3.8 days. This means that from the moment we commit to the task with the customer until the time we finish it, the time spent working on the task is, on average, less than 4 days (shown in the chart below). The term commonly used for this measurement is Lead Time or Delivery Time.
Conclusion
In part 2, we will continue our devops journey, going into more detail of our Kanban boards and their impact on our daily business, as well as long term growth.
Note: I prefer to use the term devops (with no capital letters) instead of the most common DevOps (with uppercase ‘D’ and ‘O’) to emphasise on the lack of barriers between development and operations (and being one single value stream).