ClickHouse Versions Benchmark

About this benchmark

The same benchmark, on the same data, is run across every ClickHouse version to show how performance has evolved over the years. Each version runs on a dedicated AWS c7a.4xlarge machine (16 vCPU, 32 GB RAM) with a 1000 GB gp3 EBS volume provisioned at 16000 IOPS and 1000 MB/s throughput. Every query runs once cold (with the OS page cache dropped) and five times hot; the reported time is the fastest hot run.

The datasets (each loaded into its own database), with their scale:

• hits — web-analytics page hits: 100 million rows, ~105 columns (the standard ClickBench table). 43 queries.
• ssb — Star Schema Benchmark at scale factor 100, denormalized into a single lineorder_flat table (~600 million rows). 13 queries.
• mgbench — Brown University MgBench: three log tables (server metrics, web requests, IoT device events). 15 queries.
• uk — UK property price-paid records (~28 million rows). 3 queries.
• ontime — US airline on-time flight performance. 11 queries.
• taxi — New York City taxi trips, the full ~1.3 billion-row dataset. 4 queries.
• coffeeshop — a synthetic coffee-shop sales star schema with a 500 million-row fact table. 17 queries.
• tpch — TPC-H at scale factor 40 (~10 GB), 8 tables. 22 queries.
• tpcds — TPC-DS at scale factor 32 (~10 GB), 24 tables. 103 queries.
• job — Join Order Benchmark over the IMDb dataset, 21 tables. 113 queries.

The data is stored using only types available in the oldest releases, so one set of files loads into every version. A query a version does not support (a missing function or syntax) is recorded as no result, and a dataset that does not fully load on a version is skipped entirely for it.