Real-Time Data Foundations for Whatever AI Becomes Next

Also available on:

Robert Hodges has worked on databases for more than 40 years, and he still remembers his first analytic one. At Sybase in the mid-1990s, a room of transaction-processing engineers was introduced to a system built only to read: big pages, columnar storage, queries optimized to scan. He went on to run Continuent, the clustering and replication company he sold to VMware in 2014, and since 2019 has led Altinity, the second-largest company contributor to ClickHouse® software, the open source column-store database for real-time analytics. Now part of EDB, Altinity brings an engineering culture that matches EDB’s own: both deeply devoted to the open source community, and now working on the same problem together as one EDB family.

Analytic systems and SQL databases like Postgres grew up on separate tracks for decades. Robert argues agentic AI is what finally pulls them together. Agents are good at finding information and assembling it into something that reads like a report, but they work best with direct, secure access to a few complete sources, and organizations want them to see transactional and analytic data side by side. The volume behind that is enormous, because modern software generates data as it runs: thousands of services reporting health and attacks, LLM training and inference, and trading systems where falling behind costs money.

Open source, in his view, wins on two counts. The systems that handle large data, from Hadoop and Spark to ClickHouse software, came out of real industry problems, written by engineers who had both the problem and the skill to solve it. And because the code is public, models train on it, so an AI assistant asked to build real-time analytics on a Kafka stream reaches for the open source tools it already knows. No one yet knows how AI will play out, so Robert’s advice is to build on that foundation: systems that hold up either way, centered on the data every AI product will depend on.

Key takeaways

• Real-time means seconds to insight. Robert defines it as seeing an outside event within seconds to a few minutes, then querying it fast enough to iterate. A security system might alert on DNS calls to a known malware host within about 20 seconds, then let analysts trace when it started and how far it spread.
• Four kinds of data cover most problems. Transactional databases such as Postgres, which he notes drives stock exchanges; analytic databases such as ClickHouse software; event streams such as Kafka, which is essentially a log you read from; and data lakes for large reference pools. He can think of very few use cases that a combination of them can’t solve.
• Fast, cheap data access is a competitive edge. Robert sees efficient storage combined with very fast reporting as essential to winning in modern business, from real-time sports data feeds to high-speed trading.