記錄一次ClickHouse的踩坑經歷
阿新 • • 發佈:2020-09-19
最近在做Metabase和ClickHouse整合,使用社群開發的資料驅動把ClickHouse整合到了Metabase。 大部分查詢是沒有問題,只時間維度的聚合查詢會報以下異常:
DB::Exception: Column fdate is not under aggregate function and not in GROUP BY.
詳細使用場景如下:
一、ClickHouse的版本
Connected to ClickHouse server version 20.1.4 revision 54431.
二、查詢表結構
2.1 本地表建立sql
create table if not exists dw_live.dw_player_dim_hour_local on cluster test_cluster_two_replicas ( fdate Datetime, num Int64 default 0 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{cluster}-{shard}/dw_live-dw_player_dim_hour_local', '{replica}') PARTITION BY (fdate) PRIMARY KEY (fdate) ORDER BY (fdate) TTL fdate + toIntervalDay(120) SETTINGS index_granularity = 8192, storage_policy = 'round', merge_with_ttl_timeout = 3600
2.2 分散式表建立sql
create table if not exists dw_live.dw_player_dim_hour(
fdate Datetime,
num Int64 default 0
)
ENGINE = Distributed(test_cluster_two_replicas, dw_live, dw_player_dim_hour_local, rand())
三、發生異常的使用場景
在Metabase頁面點選dwlive.dwplayerdimhour表的時間時間維度聚合查詢,監控後臺異常日誌,頁面向ClickHouse請求的查詢SQL是(查詢的是分散式表):
SELECT toStartOfHour(toDateTime(`dw_live`.`dw_player_dim_hour`.`fdate`)) AS `fdate`, count() AS `count` FROM `dw_live`.`dw_player_dim_hour` GROUP BY toStartOfHour(toDateTime(`dw_live`.`dw_player_dim_hour`.`fdate`)) ORDER BY toStartOfHour(toDateTime(`dw_live`.`dw_player_dim_hour`.`fdate`)) ASC
將該SQL貼上到ClickHouse的命令列視窗執行,同樣報以下異常:
DB::Exception: Column fdate is not under aggregate function and not in GROUP BY.
同樣的查詢語句,查詢本地表返回結果正常。查詢本地表的SQL是:
SELECT
toStartOfHour(toDateTime(`dw_live`.`dw_player_dim_hour_local`.`fdate`)) AS `fdate`,
count() AS `count`
FROM `dw_live`.`dw_player_dim_hour`
GROUP BY toStartOfHour(toDateTime(`dw_live`.`dw_player_dim_hour_local`.`fdate`))
ORDER BY toStartOfHour(toDateTime(`dw_live`.`dw_player_dim_hour_local`.`fdate`)) ASC
四、結論和解決方法
到此判定是ClickHouse的Bug,升級到20.3.18,沒有再復現該查詢異常。