1. 程式人生 > 實用技巧 >記錄一次ClickHouse的踩坑經歷

記錄一次ClickHouse的踩坑經歷

最近在做Metabase和ClickHouse整合,使用社群開發的資料驅動把ClickHouse整合到了Metabase。 大部分查詢是沒有問題,只時間維度的聚合查詢會報以下異常:

DB::Exception: Column fdate is not under aggregate function and not in GROUP BY.

詳細使用場景如下:

一、ClickHouse的版本

Connected to ClickHouse server version 20.1.4 revision 54431.

二、查詢表結構

2.1 本地表建立sql
create table if not exists dw_live.dw_player_dim_hour_local on cluster test_cluster_two_replicas (
fdate Datetime,
num Int64 default 0
)
ENGINE = ReplicatedMergeTree('/clickhouse/tables/{cluster}-{shard}/dw_live-dw_player_dim_hour_local', '{replica}')
PARTITION BY (fdate) 
PRIMARY KEY (fdate) 
ORDER BY (fdate) 
TTL fdate + toIntervalDay(120) 
SETTINGS index_granularity = 8192, storage_policy = 'round', merge_with_ttl_timeout = 3600
2.2 分散式表建立sql
create table if not exists dw_live.dw_player_dim_hour(
fdate Datetime,
num Int64 default 0
)
ENGINE = Distributed(test_cluster_two_replicas, dw_live, dw_player_dim_hour_local, rand())

三、發生異常的使用場景

在Metabase頁面點選dwlive.dwplayerdimhour表的時間時間維度聚合查詢,監控後臺異常日誌,頁面向ClickHouse請求的查詢SQL是(查詢的是分散式表):

SELECT 
    toStartOfHour(toDateTime(`dw_live`.`dw_player_dim_hour`.`fdate`)) AS `fdate`, 
    count() AS `count` 
FROM `dw_live`.`dw_player_dim_hour` 
GROUP BY toStartOfHour(toDateTime(`dw_live`.`dw_player_dim_hour`.`fdate`)) 
ORDER BY toStartOfHour(toDateTime(`dw_live`.`dw_player_dim_hour`.`fdate`)) ASC

將該SQL貼上到ClickHouse的命令列視窗執行,同樣報以下異常:

DB::Exception: Column fdate is not under aggregate function and not in GROUP BY.

同樣的查詢語句,查詢本地表返回結果正常。查詢本地表的SQL是:

SELECT 
    toStartOfHour(toDateTime(`dw_live`.`dw_player_dim_hour_local`.`fdate`)) AS `fdate`, 
    count() AS `count` 
FROM `dw_live`.`dw_player_dim_hour` 
GROUP BY toStartOfHour(toDateTime(`dw_live`.`dw_player_dim_hour_local`.`fdate`)) 
ORDER BY toStartOfHour(toDateTime(`dw_live`.`dw_player_dim_hour_local`.`fdate`)) ASC

四、結論和解決方法

到此判定是ClickHouse的Bug,升級到20.3.18,沒有再復現該查詢異常。