* Adopting AI means not only re-thinking user experiences, but also getting your data strategy right. If you do not, your AI initiatives will fail.
* AI workloads demand high-concurrency, real-time query processing, and full-fidelity data at scale. Legacy batch-oriented architectures weren't built for it.
* Previously siloed use cases, such as data warehousing and observability, are converging at the data layer.
* ClickHouse is evolving into a unified platform for powering AI workloads across AI Apps, AI Analyst, and AI SRE experiences… read on for more.

Summary

AI isn't just another workload on top of your data platform. It fundamentally changes workload expectations across every existing use case.

The three big shifts happening right now are: 1) Applications are becoming agentic, 2) Analytics interfaces are becoming conversational, 3) Observability is shifting from static dashboards to AI-driven investigations. In each case, the underlying data requirements converge around high concurrency, real-time query performance, and full-fidelity data at scale.

These are not requirements that incumbent data platforms were designed for. The data platform choices made over the next few years will shape what's possible: how fast teams can move, what products can be built, what insights are accessible to your business. The question worth asking now is not just whether your current platform handles today's workloads, but whether it's the right foundation for what AI-driven applications actually demand.

In a [previous post](https://clickhouse.com/blog/the-unbundling-of-the-cloud-data-warehouse), I wrote about the unbundling of the cloud data warehouse — how the shift toward interactive, customer-facing applications exposed the architectural mismatch between batch-oriented warehouses and real-time workloads. What I want to describe here is the next wave of that disruption, across three markets: real-time analytics, data warehousing, and observability.

![Data-platform-AI.png](https://cms.clickhouse-dev.com:1337/uploads/Data_platform_AI_67dafd4b5a.png)
<h2 id="real-time_analytics">Real-time analytics: The dawn of the "best of breed" database</h2>

**Stakeholders: Developers building next-generation user-facing and AI-powered applications**

Postgres has become the default database for building modern user-facing applications, because of its superior ability to handle row-oriented transactional data. This worked fine until applications became genuinely data-intensive, driven by real-time dashboards, usage analytics, customer-facing metrics, event streams with millions of rows per second. For these increasingly analytical workloads, Postgres alone stopped scaling. The queries were too slow, the indexes too expensive, the concurrency too low.

The solution the industry landed on was Postgres + ClickHouse: Postgres for transactions and application state, ClickHouse for analytics. This pairing became the de facto modern data stack for any customer-facing application with serious analytical requirements. ClickHouse evolved to be the obvious choice for analytical workloads: fast ingestion, sub-second queries on billions of rows, efficient at the concurrency levels customer-facing applications demand. The data moved from Postgres to ClickHouse via [CDC pipelines](https://clickhouse.com/blog/clickhouse-welcomes-peerdb-adding-the-fastest-postgres-cdc-to-the-fastest-olap-database), and ClickHouse powered everything from embedded product analytics to customer-facing dashboards.

Now AI is accelerating the need for a best-in-class transactional and analytical base for building modern AI applications and agents that power them. LLM-based features including AI-generated insights, anomaly detection, recommendations, and natural language interfaces to product data, require a tighter feedback loop between transactional writes and analytical reads. This is why we are doubling down and [building a native Postgres + ClickHouse](https://clickhouse.com/blog/postgres-managed-by-clickhouse) data stack: a single unified experience where Postgres handles transactional workloads and ClickHouse handles analytics, tightly integrated at the engine level via a native extension – for automatic data replication and management and a unified developer experience.

For platform decision-makers building customer-facing experiences, the trajectory is clear – transactional and analytical capabilities at the datastore layer are required. And tight integration between transactional and analytical capabilities, without losing "best of breed" benefits, is an additional advantage speeding up developer workflow and enabling you to ship new AI-powered capabilities faster.

![](/uploads/ai_march2026_2_6b0c8065e3.jpg)

<h2 id="data_warehousing">Data warehousing: AI Analyst workloads break batch-oriented DWH architectures</h2>

**Stakeholders: Data engineering teams modernizing data warehousing and business analytics**

In the [unbundling post](https://clickhouse.com/blog/the-unbundling-of-the-cloud-data-warehouse), I described how cloud data warehouses like Snowflake were architected for batch ingestion, heavy ETL, and periodic reporting, and how that made them a poor fit for interactive, customer-facing applications.

Now the role of traditional data warehouses is being questioned in their "bread-and-butter" use case with the advent of the "AI Analyst" - business analytics starting with natural language prompts deriving downstream assets, including ad-hoc reports and dashboards.

[Agent-facing analytics](https://clickhouse.com/blog/agent-facing-analytics), powered by text-to-SQL tools and natural language analytics interfaces are moving from experimentation to production. The UX implication is obvious: users ask questions in plain English and expect answers in seconds. The infrastructure implication is less obvious but more consequential:  each natural language query doesn't just generate one SQL query — it can trigger dozens in rapid succession, as it explores available datasets and reasons through many parallel possibilities. What looks like a single user question becomes a burst of concurrent database queries. As a result, internal analyst-generated workloads start to resemble external customer-facing workloads – high-concurrency, low-latency, interactive responses. The same pattern extends to agents autonomously querying data platforms to find the right data points to base a decision on while solving a problem.

This inverts the assumptions that legacy data warehouse architectures are built around. Platforms like Snowflake and Databricks were designed for ad-hoc, batch-oriented analytics. Their compute models assume queries are infrequent and non-interactive in nature. They optimize for high overall throughput across many queries, not high concurrency and low latency for each query. AI analyst experiences make queries fast and very frequent, and running that workload on a legacy DWH architecture means either unacceptable latency, making the AI assistant feel slow, or costs that scale faster than the value delivered.

ClickHouse was built from the ground up to excel at these requirements: petabyte-scale data, high query concurrency, sub-second response times. It was designed to serve thousands of concurrent users running interactive queries against billions of rows, not occasional analysts running batch reports. It turns out these are exactly the properties the agentic era demands.

For teams making long-term platform bets, the calculus is straightforward: AI-powered business analytics is not a future possibility, it is already here. The platforms that were right for the previous era of batch reporting do not meet its technical requirements. The switching costs of migrating off legacy data warehousing systems are real, but finite. The cost of spending the next five years on the wrong platform, paying a concurrency tax while competitors run interactive AI analytics, is larger.

![](/uploads/ai_march2026_3_69dc9fcbf2.jpg)

<h2 id="observability">Observability: AI SRE demands granular data at scale</h2>

**Stakeholders: Platform and infrastructure teams owning observability strategy**

The traditional observability stack is built around three separate pillars (metrics, logs, and traces) each stored in its own specialized system. That architecture made sense when storage and compute were expensive: you pre-aggregated metrics, sampled logs, and kept trace retention short to control costs. The tradeoff was manageable.

AI SRE workflows disrupt this model by introducing two new pressures: a high volume of concurrent natural language queries and a constant need for granular, high-cardinality, long-retention data to drive automated incident triage, root cause analysis and anomaly correlation. Sampled logs and downsampled metrics are not useful for an AI agent trying to correlate an error pattern with a deployment event from three days ago. The more capable you want your AI tooling to be, the more data you need to keep, and the more the cost structure of incumbent platforms works against you.

This is the core shift that Charity Majors has been describing as [Observability 2.0](https://charity.wtf/tag/observability-2-0/): replacing the three-pillar model with a single source of truth based on wide structured events stored in a columnar storage engine. Rather than pre-deciding what questions you'll ask and pre-aggregating for them, you store full-fidelity events and derive metrics, traces, and SLOs from them at query time. Every modern observability company is now built on this model, and many use ClickHouse as the main storage engine.

Traditional observability players like Datadog are facing a real "innovator's dilemma" here. As they are priced on data volumes and significantly rate-limit their platforms, their customers have been trained to ingest *less* data — sampling logs, downsampling metrics, limiting trace retention — to control costs. Reducing per-GB pricing to enable AI SRE workflows means cannibalizing the revenue model the business is built on. Rebuilding around wide events and columnar storage means abandoning the specialized data structures and pricing mechanics they've scaled for decades.

ClickHouse, on the other hand, is uniquely well-suited here for straightforward reasons: high compression on log and event data, sub-second queries on high-cardinality wide events, efficient at the ingestion volumes that production infrastructure generates, and a cost model based on compute and storage rather than per-GB data ingestion fees.

This is also why we are investing in a turnkey observability stack, [ClickStack](https://clickhouse.com/clickstack), observability based on OpenTelemetry and ClickHouse, with an opinionated UI and AI SRE capabilities, available both in [open source](https://clickhouse.com/docs/use-cases/observability/clickstack/getting-started/oss) and as a [managed offering](https://clickhouse.com/blog/introducing-managed-clickstack-beta).

![](/uploads/ai_march2026_4_144b78aea2.jpg)

<h2 id="observability_and_dwh">Observability and DWH are converging: Two markets, one architecture</h2>

Data warehousing and observability have been treated as separate domains for the last decade: separate buyers, separate vendors, separate stacks. And historically, that separation made some technical sense: storage backends, user interface, and consumption patterns. Even the datasets were different, because at first few businesses were fueled by their online presence.

Today, this separation is an outdated convention rather than a technical necessity. On the storage side, virtually all modern data platforms, whether business analytics or observability, now write to object stores. And the compute engines require interactive, low latency queries at high concurrency, as well as support for AI Analyst or AI SRE capabilities.

Finally, in the past, most teams treated observability data as purely operational. But in reality today, API calls are purchases and errors are failed transactions. Instead of looking at the same source of truth, the same events are often being stored twice, on two platforms, by two teams, with a fragile synchronization layer in between.

Teams that reframe all of it as business data, stored once in open formats and queryable by both AI Analyst and AI SRE tooling, eliminate that duplication and unlock context that neither silo had alone.

![](/uploads/ai_march2026_5_e7e91c2ca3.jpg)

<h2 id="the_platform_layer">The platform layer: Agentic analytics and LLM observability</h2>

A complete data platform in 2026 is more than just a database. It is a database plus the tools that make it accessible to AI agents and the instrumentation to understand how those agents behave.

Two things are happening simultaneously. First, AI agents are becoming the primary interface to data. Users increasingly don't write SQL, but instead ask questions in natural language, and agents decompose those questions into queries, execute them, and synthesize results. This means a data platform needs to expose its capabilities to agents natively: ready-made UIs, MCP-compatible APIs, agent frameworks that can reach into your data without bespoke integration work for every use case. This is why we acquired [LibreChat](https://clickhouse.com/blog/librechat-open-source-agentic-data-stack), the leading open-source AI chat platform, and made it a core component of what we call the Agentic Data Stack. LibreChat combined with ClickHouse gives teams a turnkey way to deploy analytics agents over their data, without building the agent layer from scratch.

Second, as AI agents proliferate, understanding and governing how they behave becomes a first-order engineering problem. LLM observability (tracing agent execution, monitoring model performance, tracking costs, debugging failures across multi-step agentic workflows) is not optional infrastructure. It is the difference between running AI in production with confidence or having it get stuck in the POC/experimentation stage. The observability problem for agentic systems is harder than for traditional software: the execution graphs are dynamic, the inputs and outputs are high-dimensional, and failures are often subtle rather than binary. [Langfuse](https://clickhouse.com/blog/langfuse-llm-analytics), which runs on ClickHouse Cloud to power real-time LLM observability at scale, is solving this problem.

![AI is Redrawing the Database Market #1409.jpg](/uploads/AI_is_Redrawing_the_Database_Market_1409_eac7cc339d.jpg)

For platform decision-makers, the takeaway is clear: the database is necessary, but not sufficient. The complete picture includes agent-ready interfaces and LLM observability tooling, natively integrated into the data platform experience.

<h2 id="a_unified_data_platform">A unified data platform for interactive AI-driven applications</h2>

ClickHouse is evolving into what we see as the **unified data platform for interactive AI-driven applications**: one platform that handles transactional and analytical workloads, modern real-time data warehousing and conversational BI, and evolving AI-SRE driven observability workflows, at the performance and cost profile that AI-native applications demand.

The market is moving fast, and the platforms winning the next era are already visible. The question for every team making long-term infrastructure decisions is whether they are making the right bet now, while the switching costs are manageable, or whether they will be making a harder, more expensive decision later.


Interested in seeing how ClickHouse works on your data? Get started with ClickHouse Cloud in minutes and receive $300 in free credits.

Get started today

AI is reshaping database market requirements across real-time analytics, data warehousing, and observability — and ClickHouse is uniquely positioned to meet them.

large_AI-redrawing-dbmarket.jpg

medium_AI-redrawing-dbmarket.jpg

small_AI-redrawing-dbmarket.jpg

thumbnail_AI-redrawing-dbmarket.jpg

AI-redrawing-dbmarket.jpg

AI is redrawing the database market

SAN FRANCISCO--November 4, 2025--ClickHouse, Inc., the company behind the world’s fastest and most popular real-time analytics database, today announced the acquisition of LibreChat, a leading open-source AI chat platform that provides a unified interface for interacting with large language models (LLMs). The acquisition brings LibreChat founder Danny Avila and his team to ClickHouse.

Together, ClickHouse and LibreChat form the foundation of the open-source Agentic Data Stack, empowering teams to deploy AI agents that analyze and act on large datasets through natural language. This shift to agentic analytics is already transforming how organizations such as Shopify, Daimler Truck, Fetch, and cBioPortal leverage their data, demonstrating the immediate value in real-world applications.

At Shopify, LibreChat has become central to internal AI workflows. “LibreChat powers reflexive AI use across Shopify. With near universal adoption and thousands of custom agents, teams use it to solve real problems, increase productivity, and keep the quality bar high. By connecting more than 30 internal MCP servers, it democratizes access to critical information across the company,” said Matt Burnett, Senior Engineer at Shopify.

In healthcare, cBioPortal uses the combined ClickHouse and LibreChat Agentic Data Stack to unlock new research capabilities. "By leveraging the ClickHouse, MCP, and LibreChat stack, we rapidly delivered a prototype to cBioPortal users that empowered them to ask entirely new questions about cancer genomics and treatment trajectories, get quick answers, and explore data in ways not possible through the existing UI," said Ino de Bruijn, Manager Bioinformatics Software Engineering at cBioPortal. "It puts discovery at cancer researchers' fingertips."

This LibreChat acquisition comes as enterprises rapidly adopt agentic analytics to democratize data access and accelerate time-to-insight. Fetch, a leading mobile rewards platform, built its FAST AI-powered analytics product on ClickHouse to transform household purchase behavior into actionable business intelligence. "We see agentic analytics as the future of data interaction, enabling more intuitive, dynamic, and impactful use of information," said Sam Corzine, Director of Machine Learning at Fetch. "With its unmatched speed and scalability, ClickHouse is well-positioned to power this new generation of agentic experiences, and we're thrilled to grow our partnership together."

ClickHouse leverages agentic analytics internally with LibreChat-and-ClickHouse-powered DWAINE (Data Warehouse AI Natural Expert), an agent that now handles approximately 70% of data warehouse queries for over 200 users, processing 33 million LLM tokens daily as of October 2025—up from 15 million just months after launch.

"What impressed me most is LibreChat's vibrant community that continuously innovates," said Dmitry Pavlov, Director of Engineering at ClickHouse. "The synergy between ClickHouse Cloud's blazing-fast query performance and LibreChat's flexible, multi-LLM architecture is unlocking a new generation of data analysis agents - real-time, secure, powerful, and accessible."

For existing LibreChat deployments, operations continue unchanged with ongoing investment in the open-source project and community. ClickHouse users will gain access to enhanced integration capabilities, including native LibreChat experience alongside ClickHouse Cloud instances, extended data visualization support, enterprise-grade security and governance, and a tailored semantic layer. These investments will ensure that the Agentic Data Stack is fully enterprise-ready, meeting the requirements of the world's most demanding organizations.

“Every breakthrough in data technology has been about reducing the distance between questions and answers. By joining forces with LibreChat, we’re tackling this distance. Now, any user can talk directly to petabytes of data and get instant, actionable insights,” said Aaron Katz, CEO of ClickHouse. “As AI agents proliferate across every industry, the companies that win will be those that can seamlessly connect human intelligence with machine-scale data processing. The future of analytics isn’t just dashboards, it’s intelligent agents that interpret data, trigger workflows, and power real-time decisions. The ClickHouse Agentic Data Stack makes that future available today.” 

### About ClickHouse

ClickHouse is a fast, open-source columnar database management system built for real-time data processing and analytics at scale. Engineered for high performance, ClickHouse Cloud delivers exceptional query speed and concurrency, making it ideal for applications that demand instant insight from massive volumes of data. As AI agents become increasingly embedded in software and are generating far more frequent and complex queries, ClickHouse brings a high-throughput, low-latency engine, purpose-built to meet this challenge. Trusted by leading companies like Sony, Tesla, Anthropic, Memorial Sloan Kettering, Lyft, and Instacart, ClickHouse helps teams unlock insights and drive smarter decisions with a scalable, efficient, and modern data platform. For more information, visit [clickhouse.com](http://clickhouse.com).

### About LibreChat

LibreChat is a leading open-source AI chat platform offering a unified interface for interacting with multiple large language models. The platform gives users and organizations complete control over their data, agents, and conversations while supporting enterprise-grade deployments at scale.

### Contacts

Tanya Bragin  
ClickHouse, Inc.  
tanya@clickhouse.com

ClickHouse today announced the acquisition of LibreChat, a leading open-source AI chat platform that provides a unified interface for interacting with large language models (LLMs).

large_ch and librechat 401.png

medium_ch and librechat 401.png

small_ch and librechat 401.png

thumbnail_ch and librechat 401.png

ch and librechat 401.png

ClickHouse acquires LibreChat to democratize AI-driven analytics through the open-source Agentic Data Stack

After the success of our [ClickHouse Open House](https://clickhouse.com/openhouse) conference in San Francisco in May, we knew we had to take this show on the road. 

On October 7th, the ClickHouse community came together in New York to hear from ClickHouse customers Capital One, Ramp, and Modal, alongside the ClickHouse product and leadership teams. 

For those who couldn't make it, and for attendees who want to revisit the sessions, we've now released the recordings from the event!

![ClickHouse Open House 2025](https://cms.clickhouse-dev.com:1337/uploads/open_house_new_york_2025_36052dc03c.png)

We organized the presentations around four key use cases showcasing where ClickHouse shines. 

First is **real-time analytics**, or 'immersive data-driven applications,' that need both recent and historical data with instant interactive filtering. As one of our customers puts it, they're building apps where users expect responses right away, not in 5-10 seconds.

The second use case is **observability**, where SREs and DevOps teams increasingly adopt ClickHouse as observability data keeps getting more voluminous and structured over time. Companies like eBay moved from Elastic to ClickHouse early on, and we're seeing this trend accelerate as teams want more control over their data to build AI-enabled monitoring features.

The third is **data warehousing**, which includes leveraging ClickHouse as an interactive query layer on top of data lakes. The key word here is 'interactive.' Traditional query engines are optimized for batch workloads, but when you're applying AI to your data warehouse or data lake, you need that human-level responsiveness where you can ask a question, get a quick response, and iterate.

And finally, there's **AI and ML infrastructure** itself - ClickHouse is being used for data prep and transformation, as an offline and online feature store for model training and inference, for vector search, and of course for observability of AI systems. It's become essential infrastructure for data science teams.

## Capital One: Real-time analytics

Capital One, the banking and credit card company, uses ClickHouse to power its Slingshot data management and cost optimization platform. 

Salim Sayed and Syd Mehmood chose ClickHouse for its speed, scalability, and cost efficiency. They transformed dashboard load times from 5-6 seconds to under half a second, achieving **80% improvement in response times while cutting infrastructure costs by 50%.**

<iframe width="768" height="432" src="https://www.youtube.com/embed/8Avld0BlDlg?si=dkcRkjQX-kd5w8hL" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media;a gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

➡️ [How Capital One Slingshot cut infrastructure costs by 50%](https://clickhouse.com/videos/open-house-nyc-capital-one)

## Ramp: Observability

Ramp, a finance operations platform, faced challenges scaling analytics queries on Postgres. Four engineers were tasked with solving this problem, and within four weeks, they had put their new ClickHouse-based system into QA.

Ryan Delgado, who leads Ramp’s platform team, walks through the story and explains why turning those slow queries into reports that ran hundreds of times faster was so impactful that his entire OLAP team got promoted.

<iframe width="768" height="432" src="https://www.youtube.com/embed/mPZ7Bck_cMI?si=PfSIbO4sGNT4_nHU" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

➡️ [ClickHouse at Ramp: Just OLAP it](https://clickhouse.com/videos/open-house-nyc-ramp)

## Modal Labs: Data warehousing

Modal is an infrastructure platform for running large-scale GPU workloads in the cloud. They use ClickHouse as the backbone of their real-time observability platform.

In this talk, Ro Arepally explains how they solved a scaling bottleneck by switching to ClickHouse, transforming their logging infrastructure into a system that **handles half a trillion logs and delivers instant search results.**

<iframe width="768" height="432" src="https://www.youtube.com/embed/1LXK-mgdKCg?si=C3XlDag4pt5T5zF8" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

➡️ [How Modal delivers instant search on 500B+ logs](https://clickhouse.com/videos/open-house-nyc-modal)

## What's Next

Each of these stories deserves a deeper dive. Over the coming weeks, we’ll break down the technical architectures and lessons learned in individual posts [on our blog](https://clickhouse.com/blog?category=user-stories).

The [Open House Roadshow](https://clickhouse.com/openhouse) continues! We're bringing these conversations to more cities, connecting with data teams who are pushing the boundaries of what's possible with real-time analytics.

Check out the [full video playlist from NYC](https://clickhouse.com/videos?category=open-house), stay tuned for videos from Sydney and Bangalore, and join us at our final roadshow stop in Amsterdam on October 28th.

➡️ [Join us in Amsterdam](https://clickhouse.com/openhouse/amsterdam)


The ClickHouse Open House Roadshow kicked off in New York on October 7th with compelling customer stories from Modal, Ramp, and Capital One, and all session videos are now live.

large_Image 2400x1260.jpg

medium_Image 2400x1260.jpg

small_Image 2400x1260.jpg

thumbnail_Image 2400x1260.jpg

Image 2400x1260.jpg

ClickHouse Open House Roadshow NYC videos are here

Today, we are pleased to announce that [chDB](https://doc.chdb.io/#/), an embedded SQL OLAP engine powered by ClickHouse, is now part of ClickHouse. chDB’s creator and main contributor, Auxten, [is joining forces with us](https://auxten.com/chdb-is-joining-clickhouse) to focus on evolving chDB and integrating it even more closely with the ClickHouse ecosystem.

## What is chDB and how did we get here? 

ClickHouse is best known as a scalable analytics database that uses a client-server architecture, but we’ve actually always had ClickHouse Local, a CLI tool that contains ClickHouse.

People [became aware of ClickHouse Local at the beginning of 2023](https://news.ycombinator.com/item?id=34265206) and they loved it, but we also got feedback that they wanted to be able to integrate ClickHouse Local with the other tools they were using both in their data pipelines and for data analysis.

Around this same time, Auxten had started working on chDB with the goal of being able to use ClickHouse as an “out of the box” Python module. 

You can use chDB to do in-memory analysis of data that lives locally or remotely in all the formats and sources supported by ClickHouse. You can also choose to persist data in a local ClickHouse instance and chDB uses the same storage format as ClickHouse Local, which means their databases are interchangeable. The diagram below shows how chDB works in comparison to other ClickHouse variants.

![chDB Image 4 updated 2.png](/uploads/ch_DB_Image_4_updated_2_6fd7ed2265.png)

We were really impressed with chDB and contacted Auxten to tell him that we loved his work and wanted to help promote it. This resulted in Alexey mentioning chDB in the [23.7 release call](https://www.youtube.com/live/TI1kONfON18?si=OhCU6_R58iByNXVQ&t=1872), as well as a subsequent [introductory blog post about chDB](https://clickhouse.com/blog/welcome-chdb-to-clickhouse) in August 2023. A few weeks later Auxten wrote [chDB - A Rocket Engine on a Bicycle](https://clickhouse.com/blog/chdb-embedded-clickhouse-rocket-engine-on-a-bicycle), in which he went through chDB’s design decisions and implementation details

In the months since then, [we’ve seen more interest in chDB](https://clickpy.clickhouse.com/dashboard/chdb), have been working closely with Auxten, and gradually saw an opportunity to take our collaboration to the next level. 

## What does the news mean if I’m a chDB user?

If you’re an existing user of chDB, this is great news for you. Auxten will now be working full-time on chDB rather than only in his spare time. This means we’ll be able to fix bugs/issues more quickly as well as adding new features and integrations.

chDB will soon be integrated into ClickHouse’s continuous integration pipeline, which will allow us to have chDB closely follow the release schedule of ClickHouse.

We’ll be moving the chdB documentation to live alongside the ClickHouse documentation and will make chDB more prominent in our getting started guides.

We’ll also be looking at ways that we can better integrate chDB with the rest of the data engineering ecosystem, so let us know if you have any opinions about where we should start!

## What happens next?

Our goal is to create a seamless experience whether you’re analyzing Parquet files on your laptop, creating data pipelines with other tools in the Python ecosystem, or building a real-time data warehouse to serve data products to your users.

We want to get to the point where it takes minimal effort to switch between the different flavors of ClickHouse.

We don’t know exactly what this will look like and we’d like your help figuring this out. Below are some of the things at the top of our todo list for what to do next:

1. Better performance, especially on Pandas DataFrame and Arrow Buffer
2. Keep up with latest ClickHouse release
3. Better integration with more programming languages and data sources

But maybe you have better ideas! We’d love to hear your thoughts on the [chDB Github Discussions](https://github.com/orgs/chdb-io/discussions). 

## Get started with chDB today

In the meantime, if you’re not yet familiar with chDB, we thought we’d include a little example to whet your appetite. chDB is [published on PyPi](https://pypi.org/project/chdb/) and can be installed with the pip package manager. We’ll also install Pandas and pyarrow as we'll be using those too:

```bash
pip install chdb pandas pyarrow
```

Once that’s installed we can open a Python REPL, Jupyter notebook, or similar and import the following module:

```python
import chdb
```

Let’s now look at how to query the Hugging Face [midjourney-messages](https://huggingface.co/datasets/vivym/midjourney-messages) dataset, which contains metadata about images generated by the Mid Journey Generative AI service over several months in 2023. This dataset contains 55 million rows spread over 55 Parquet files. 

The following query returns the count of images as well as the minimum/maximum size, width, and height:

```python
chdb.query("""
FROM url('https://huggingface.co/datasets/vivym/midjourney-messages/resolve/main/data/0000{00..55}.parquet')
SELECT count(), COLUMNS(width, height, size) APPLY(max), COLUMNS(width, height, size) APPLY(min)
SETTINGS max_http_get_redirects=1
""", "Vertical")

Row 1:
──────
count():     55082563
max(width):  18928
max(height): 16128
max(size):   24498571
min(width):  32
min(height): 56
min(size):   312
```

If we want to persist the data from those Parquet files, we can do that as well.

```python
from chdb import session as chs

sess = chs.Session("midjourney.chdb")
sess.query("CREATE DATABASE MidJourney")
sess.query("""
CREATE TABLE MidJourney.images
Engine = MergeTree
ORDER BY (size, height, width)
AS
SELECT *
FROM url('https://huggingface.co/datasets/vivym/midjourney-messages/resolve/main/data/0000{00..55}.parquet')
SETTINGS max_http_get_redirects=1, schema_inference_make_columns_nullable=0
""")
```

That will take around 4 minutes to pull down the data and ingest it into ClickHouse. We can then repeat the above query:


```python
sess.query("""
FROM MidJourney.images
SELECT count(), COLUMNS(width, height, size) APPLY(max), COLUMNS(width, height, size) APPLY(min)
""", "Vertical")
```

Hopefully that’s given you an idea of what you can do with chDB, but [check out the ](https://doc.chdb.io/#/)documentation for even more examples. We look forward to learning what you do with this compact yet powerful way to use ClickHouse in your local projects!

Read more about the history of chDB, and the creator Auxten's perspective, [in his post](https://auxten.com/chdb-is-joining-clickhouse).


We're excited to announce that chDB is now part of ClickHouse.

large_Option 6.png

medium_Option 6.png

small_Option 6.png

thumbnail_Option 6.png

Option 6.png

chDB joins the ClickHouse family

We owe a lot to the cloud data warehouses, but their era of hegemony is coming to an end.

large_the-unbundling-of-the-cloud-data-warehouse.png

medium_the-unbundling-of-the-cloud-data-warehouse.png

small_the-unbundling-of-the-cloud-data-warehouse.png

thumbnail_the-unbundling-of-the-cloud-data-warehouse.png

the-unbundling-of-the-cloud-data-warehouse.png

The Unbundling of the Cloud Data Warehouse

![Tanya-Cristina-Krithika.png](/uploads/Tanya_Cristina_Krithika_bb50461d51.png)

I often get questions about how I decided to become a product manager at startups, and what I learned over the years. The truth is, I had no such plan in life - I fell into the product management career somewhat by accident. But I have learned a few things along the way, and I will attempt to summarize the highlights in this blog. I hope it proves useful for folks considering a career in product management, especially at fast-growing technology startups.

## How I decided to become a product manager

I grew up in a family of scientists and thought until a certain point that everyone had to eventually earn a PhD. When I went to university, I did not declare a major and started studying everything under the sun. I only later realized that I liked working with computers and therefore decided to major in computer science. I went on to become a technology consultant at Deloitte, one of the huge “big four” consulting firms, and while that was a great learning experience for customer-facing skills, it was not what I was looking for. We worked with a lot of legacy technologies at big bureaucratic companies, which did not feel innovative or all that interesting to me.

I realized that instead I wanted to bring new products to life, but I did not know how to break in. So I went back to grad school to study computer science again, but quickly realized that unlike my parents, I did not want a PhD. After getting my masters, I was just as lost as before as to what to do next. This is where I got advice from one of my professors to “look into product management”. It sounded important, so I researched online and started interviewing at big companies in Seattle - Microsoft, Amazon, Google - regurgitating all the things I read and hoping they would believe I had the “right stuff” for an entry-level product management job.

Thankfully, at that stage, I was referred by another professor to a set of technical founders who were bringing a startup out of stealth mode and just got Series A funding. They did not have a product management role for me, but I agreed to join as a sales engineer (aka solution architect), running proofs of concept with early customers. I agreed to the role, because I was really curious about “working at a startup”, but I was also very much enamored by the product and by the team.

As it turned out, this choice was the deciding factor in shaping my career going forward.

## My product management experiences (so far)

ExtraHop was the first startup I worked at. It was a closely-knit 10-person team at the time, looking for smart, dedicated people to build a next-generation network and security analytics product. The company was led by incredibly talented technologists [Jesse Rothstein](https://www.linkedin.com/in/jrothste/) and [Raja Mukerji](https://www.linkedin.com/in/rajamukerji/) with a really sharp vision for the space, and a tenacious ability to execute. They built one of the strongest engineering teams I have ever had the privilege to work with. I started as the first sales engineer, but quickly became the de-facto PM, because I spoke with every customer and understood what they wanted and why. When I left the company 5 years later, I was the head of product with about 10 people on my team (the company grew from 10 to 400).

I left ExtraHop for personal reasons (a move to the Bay Area with my family) and joined Elastic, a 100-person startup around Elasticsearch, an open source project in the search and analytics space. I was lucky enough to join [Gaurav Gupta](https://www.linkedin.com/in/ggupta/), a seasoned product leader and a great mentor, as the second PM on his team. I knew nothing about open source, and was very curious about how to build a viable business around an essentially free technology. The company was led by truly visionary people, including of course the creator of Elasticsearch, [Shay Banon](https://www.linkedin.com/in/kimchy/), who remains a source of inspiration to me to this day. I went through years of very fast growth and an IPO with Elastic, working my way up from middle ranks of product management to head of product for the #1 revenue generating observability product line.

When I got the opportunity to lead product at ClickHouse, an increasingly popular open source project in the real time analytics space, it did not take much time to decide it was the right next step for me. While ExtraHop and Elastic were great learning experiences, I was not fully responsible for taking the first commercial product offering from 0 to 1 there. I really wanted to have that responsibility, and I knew I was ready for it. I was also inspired by the technology, which is taking the world by storm, the culturally-diverse and dedicated team, and the opportunity to work in an emerging market category of analytical databases.

## My product management philosophy

I sometimes get asked what product management means to me, and what I think it takes to be a good PM.

The first part is easy. Product management is responsible for making sure that the organization is aligned around a vision and a plan for how to build the right product(s) to sustain a successful business. There are whole books written on the subject, so I won’t opine in more detail here. My favorite book that simply explains the role of product management in relation to other roles at technology companies is “Inspired” by Marty Cagan.

How to be a good PM is much more difficult to describe. It depends a lot on the type of organization and team you are joining. As you can see from my experiences, I enjoy working at early-stage companies with very strong engineering teams. And as a result, I believe in building respectful and trusting relationships with technical leaders at the company, and being flexible about the role of product management depending on the stage of startup development.

But ultimately, I believe that a product manager is a leader that wakes up every morning, thinks about how to make their product successful in the market, and does what it takes to make it happen. The best and most succinct way I have seen this philosophy described is in Ben Horowitz [Good PM / Bad PM blog](https://a16z.com/2012/06/15/good-product-managerbad-product-manager/). He says it is dated, but I think it stands the test of time.

If you got to this point, thank you for reading!

I just started a [Medium](https://medium.com/@tbragin) site and will post product management and startup advice there going forward, starting with my first article “How to move to product management from technical roles”.

You can also follow me on [Twitter](https://twitter.com/tbragin) and [LinkedIn](https://www.linkedin.com/in/tbragin/) for my thoughts about product management, startups, and products I’m currently building.

**Tanya's Bio:**

Tanya Bragin leads product at ClickHouse, the fastest open source analytical database. She has over 20 years of experience in the data space, ranging from data analytics, to search, to security and network analytics. She started her career in consulting and sales engineering, and spent the last decade and a half growing product organizations from early stages to maturity at three data analytics startups, ExtraHop, Elastic, and now ClickHouse.



I often get questions about how I decided to become a product manager at startups, and what I learned over the years. The truth is, I had no such plan in life.

Tanya Bragin

Blog posts by Tanya Bragin

AI is redrawing the database market

ClickHouse acquires LibreChat to democratize AI-driven analytics through the open-source Agentic Data Stack

ClickHouse Open House Roadshow NYC videos are here

chDB joins the ClickHouse family

The Unbundling of the Cloud Data Warehouse

My Journey as a Serial Startup Product Manager