點選有驚喜

標籤

PostgreSQL , array_agg , arragg

背景

多個數組聚合為一維陣列，求PC。業務背景見：

由於PostgreSQL內建的聚合函式array_agg支援的陣列聚合實際上是將多個數組聚合為多維陣列。並不是一維陣列。

例如：

postgres=# select array_agg(arr) from (values(array[1,2,3]), (array[4,5,6])) t(arr);  
     array_agg       
-------------------  
 {{1,2,3},{4,5,6}}  
(1 row)  
 

而實際上我們要的是一維陣列的結果

{1,2,3,4,5,6}  

此時需要自定義一個聚合函式

create aggregate arragg (anyarray) (sfunc = array_cat, stype=anyarray, PARALLEL=safe);      

效果如下

postgres=# select arragg(arr) from (values(array[1,2,3]), (array[4,5,6])) t(arr);  
    arragg       
---------------  
 {1,2,3,4,5,6}  
(1 row)  

但是這個新加的聚合用到了array_cat，大量的memcpy導致效能並不好。

array_agg效能對比arragg

聚合100萬個元素.

1、array_agg，耗時0.14秒

postgres=# explain (analyze,verbose,timing,costs,buffers) select array_agg(array[1,2,3,4,5,6,7,8,9,10]) from generate_series(1,100000);  
                                                                QUERY PLAN                                                                  
------------------------------------------------------------------------------------------------------------------------------------------  
 Aggregate  (cost=12.50
 
..12.51 rows=1 width=32) (actual time=113.134..113.134 rows=1 loops=1)  
   Output: array_agg('{1,2,3,4,5,6,7,8,9,10}'::integer[])  
   ->  Function Scan on pg_catalog.generate_series  (cost=0.00..10.00 rows=1000 width=0) (actual time=53.585..66.200 rows=100000 loops=1)  
         Output: generate_series  
         Function Call: generate_series(1, 100000)  
 Planning time: 0.064 ms  
 Execution time: 143.075 ms  
(7 rows)  

2、arragg(use array_cat)，耗時108.15秒

postgres=# explain (analyze,verbose,timing,costs,buffers) select arragg(array[1,2,3,4,5,6,7,8,9,10]) from generate_series(1,100000);  
                                                                QUERY PLAN                                                                  
------------------------------------------------------------------------------------------------------------------------------------------  
 Aggregate  (cost=12.50..12.51 rows=1 width=32) (actual time=108081.186..108081.186 rows=1 loops=1)  
   Output: arragg('{1,2,3,4,5,6,7,8,9,10}'::integer[])  
   ->  Function Scan on pg_catalog.generate_series  (cost=0.00..10.00 rows=1000 width=0) (actual time=11.121..81.467 rows=100000 loops=1)  
         Output: generate_series  
         Function Call: generate_series(1, 100000)  
 Planning time: 0.148 ms  
 Execution time: 108154.846 ms  
(7 rows)  

3、unnest聚合，耗時0.59秒

postgres=# explain (analyze,verbose,timing,costs,buffers) select array(select unnest(array[1,2,3,4,5,6,7,8,9,10]) from generate_series(1,100000));
                                                                    QUERY PLAN                                                                    
--------------------------------------------------------------------------------------------------------------------------------------------------
 Result  (cost=517.50..517.51 rows=1 width=32) (actual time=520.327..520.327 rows=1 loops=1)
   Output: $0
   InitPlan 1 (returns $0)
     ->  ProjectSet  (cost=0.00..517.50 rows=100000 width=4) (actual time=11.979..223.223 rows=1000000 loops=1)
           Output: unnest('{1,2,3,4,5,6,7,8,9,10}'::integer[])
           ->  Function Scan on pg_catalog.generate_series  (cost=0.00..10.00 rows=1000 width=0) (actual time=11.972..27.014 rows=100000 loops=1)
                 Output: generate_series
                 Function Call: generate_series(1, 100000)
 Planning time: 0.082 ms
 Execution time: 590.976 ms
(10 rows)

點選有驚喜