For those who couldn’t attend EDB’s global virtual conference, Postgres Build 2021, we’re recapping some of the standout sessions from the event.
For our latest highlights blog, we’re revisiting How Many Postgres is Too Many? led by Dr. John A. Stevenson, a senior software developer at the British Geological Survey (BGS). As a result of his position, Dr. Stevenson works with massive amounts of data, spread across multiple PostgreSQL infrastructures, and has—as a result—gleaned key insights into how to manage such an architecture. Over the course of his presentation, Dr. Stevenson touches on:
- integrating PostgreSQL and Docker
- PostgreSQL and Python
- sharing data across infrastructures
Between a rock and a database
The central mission of the BGS is to advance geoscientific knowledge across the United Kingdom, via extensive surveying, monitoring and research. Their findings have been instrumental to a wide range of preservationist initiatives, and their data is regularly used by government agencies, as well as academics. Suffice to say, not only does BGS work with massive amounts of information, but the means of storing and sharing that information reflect the many partners with whom BGS regularly collaborates.
Laying out the current IT infrastructure of BGS, Dr. Stevenson notes three key components:
- self-hosted virtual machines running on premises
- a long-standing Oracle database for the bulk of corporate data
- an increasing use of PostgreSQL for microservices and collaborative projects (many of which are overseas)
For major recent projects, BGS has opted to run PostgreSQL via Docker, which has helped them manage the large amount of images that they’ve captured, specifically with regards to a subsurface energy source survey. Dr. Stevenson explains that, with PostgreSQL, spinning up these images has become faster and easier, from a range of locations. Via a setup script they’ve implemented, it’s also been possible to apply a range of key settings to their PostgreSQL architecture so that it mirrors other pieces of BGS’ database infrastructure—PostgreSQL is the same on the server as it is on a developer’s laptop.
The importance of flexibility
Dr. Stevenson goes into great depth describing the specific PostgreSQL DBMS that BGS utilizes in his full presentation, but across every detail he mentions, the vital themes remain the same: flexibility, scalability, and interoperability.
BGS handles much of its database management via coding, which allows them to implement more nuanced tweaks than they might be able to by simply adjusting settings, but the fact that PostgreSQL allows them to do so is a key component of how they’ve been able to more effectively balance multiple architectures without unnecessary strain. While many service providers limit IT’s ability to tailor solutions to the specific demands of a business, open source technologies like PostgreSQL are designed so developers can calibrate the tool to their needs. Similarly, while many service providers limit integrations in an attempt to make enterprises dependent on their offerings, PostgreSQL makes it easy to sync a database infrastructure with additional solutions that can make it easier to manage a huge and complex architecture.
This flexibility also applies to sharing data across multiple PostgreSQL databases. While many database providers limit the types of tools that you can leverage to transfer data across differently configured architectures, PostgreSQL offers wide compatibility for a myriad of solutions and strategies. For BGS, Dr. Stevenson mentions that replication via pglogical has been most effective for connecting internal, public, and development PostgreSQL databases, but businesses are regularly finding new ways to adapt their data sharing to suit their specific needs.
Thriving in the Internet-of-Things with PostgreSQL
PostgreSQL’s flexibility allows organizations such as BGS to effectively deploy and manage complex, sprawling architectures without having to worry about losing control of their essential data assets. As such, it empowers collaboration, innovation, and growth like few other database solutions. Whether you’re invested in a code-based approach to your multi-database infrastructure or considering one of the many other strategies available, Dr. Stevenson’s experience can help guide you on your own PostgreSQL journey.
If you want to watch Dr. John A. Stevenson’s session in full, you can check out his and all PostgresBuild on-demand sessions here!