Operationalizing AI for Postgres

AI continues to make headlines, but this wouldn’t be the case if it weren’t for the underlying databases and data that power AI’s capabilities and potential. That’s why our EDB experts are continually hosting presentations and writing about AI and AI databases. At EDB we are committed to ensuring that our customers can unlock the full potential of this revolutionary new technology, and we are passionate about sharing our AI knowledge and expertise with the vibrant open source community. 

In our recent webinar, EDB Chief Architect for Analytics and AI Torsten Steinbach shared how AI and databases work together to drive innovation and how you can start using Postgres for your AI innovation.

Ingredients for successful AI solutions

Torsten kicked off the webinar by discussing the similarities between AI and human intelligence. Just like human intelligence, AI requires a brain to generate ideas and make decisions, consciousness to set and achieve goals, and memory or data that encapsulates knowledge and experience. All three of these components are necessary for a successful AI solution.

Chatbots rely on all three of these elements. The John Doe of generative AI applications, chatbots are the modern standard AI application that many people are familiar with. When you’re talking to a chatbot, you’re essentially talking to AI. 

Chatbots rely on stored data to be successful, and vector databases are where numeric representations (vectors) of documents and images and other data types are stored. An encoding large language model (LLM) generates these vectors, converting raw data into a format suitable for AI processing. While this vectorization process can be time consuming for large datasets, it’s essential for preparing data for AI applications.

RAG: Boosting chatbot intelligence with data-driven insights

Retrieval Augmented Generation (RAG) is a key process in AI chatbots. This Generative AI framework augments LLMs with internal data so chatbots can communicate more effectively. 

Here’s how it works:

1. The end-user submits a chat message.
2. An encoder LLM converts the message into a vector.
3. This vector is used to search a vector database for relevant data.
4. The system retrieves the most similar data from the database.
5. The original chat message is augmented with this relevant context.
6. This augmented prompt is sent to a decoder LLM (like GPT models).
7. The decoder generates a targeted response using both the question and contextual data.

RAG helps prevent incorrect or misleading results – hallucinations – by providing domain-specific information that the model may not have been originally trained on. Without this augmentation, popular public models like ChatGPT may start to hallucinate if you ask them a question they haven’t been trained on. (Yes, it’s actually called AI hallucinations.)

Learn more: Watch Torsten's full video on RAG. 

What happens if your chatbot is successful?

If you’ve built a successful chatbot using RAG, and it’s providing meaningful responses and attracting more users (who are inputting more data), and reducing customer support traffic, the chatbot may become mission critical to your users and your organization. This can be a huge problem if your database or chatbot goes down or starts malfunctioning. This is why operationalizing your chatbot solution is essential.

What’s needed for an operational chatbot?

Here’s what’s required for your AI system and chatbot or other AI application to be successful for the long term:

• Always on: Your AI solution needs to be up and running 24/7 with no planned downtime.
• Resilient: High availability and disaster recovery are essential for business continuity.
• Secure: Only the data that is needed for a certain user should be revealed. Data should be encrypted, protected, and governed. 
• Responsive: Quick interactive response times and automatic data indexing are key for retaining users.
• Scalable: The system needs to be able to scale along with your increasing users.
• Enterprise support: Reliable vendor support is needed for long-term maintenance.
• Ecosystem: Technology should be well established with tools and talent available. 
• Business Data: Your chatbot should be able to seamlessly connect to existing support tickets and other company data.
• Business Events: Your system must be able to automatically incorporate and retrieve new knowledge.

Those are all quantities of service that many people don't have on their radar when they think of building a chatbot. 

At the end of the day, it’s all about the data

Developing a chatbot isn’t rocket science any more, since a number of AI solution frameworks like LangChain and LlamaIndex are available to assist with the AI flow.

While AI solution frameworks make it easier to build AI applications, these frameworks only automate your app flow. They don’t operationalize your solution or manage your data. They don’t address the stateful side of things, which is your data and your models.

That’s why organizations have to formulate their own data management and storage strategies.  

Enhancing Postgres for AI workloads

Postgres is already well established for transactional data workloads. This open source database checks all the qualities of service needed for operational systems. Plus it has extensive support, frameworks, libraries and more. That’s why EDB is built around Postgres. 

Using Postgres for AI, however, requires processing new types of data like open table formats and binary and unstructured data like text, images, voice recordings and video data. Because Postgres can’t traditionally process these types of data out of the box, we’re enhanced it so it can effectively comprehend, store and manage these new data types. This includes processing data across various scales, linking it with other data and converting it into open table formats for modern data analysis.

We’ve also incorporated new processing capabilities like columnar processing which enables efficient data processes in open table formats (see our Postgres lake house), vector and hybrid search, and native AI model integration. We’re incorporating new infrastructure too, like optic storage capacity for cost-effectively storing large data volumes and support for GPUs. These enhancements enable new workloads to be run against Postgres, including BI analytics, data warehousing, hybrid transactional analytical processing and generative AI patterns.

Building an AI solution: From vector database to AI platform

Building an AI solution with Postgres starts with a vector database. Vector databases essentially accommodate vector data and fields and conduct vector searches. But they don’t encompass everything, as they don’t store your AI data on object storage or compute the vectors. Capturing, maintaining, preparing, and storing vector data in the database often requires specific expertise from data engineers or data scientists. 

When you take Postgres from a simple vector database to a full-fledged AI database, it acts as the memory that equips AI systems with the necessary context and understanding to store, manage and retrieve your data efficiently and effectively. 

There are even more advanced elements for fine-tuning specific models or implementing augmented generation, which exceed the capabilities of a standard AI database. These functionalities turn an AI database into an AI data platform, which is EDB’s ultimate goal and our North Star.

As we continue with our work, we invite you to preview our AI tech for free.

You can also start using a Postgres lakehouse here. 

Together, we can make Postgres the AI data platform of the future and equip Postgres developers to be the most capable and effective AI builders on the planet. 

Watch the full webinar: Operationalizing AI for Postgres

Join the Generative AI Data Strategy Panel Discussion at Ai4 2024

Hear strategic approaches for leveraging proprietary datasets to enhance generative AI models from panelists Torsten Steinbach and other AI thought leaders at Ai4’s AI conference August 12-14.