Pandas聚合運算和分組運算
阿新 • • 發佈：2019-02-12
1.聚合運算
(1)使用內建的聚合運算函式進行計算
1>內建的聚合運算函式
sum(),mean(),max(),min(),size(),describe()...等等
2>應用聚合運算函式進行計算
import numpy as np import pandas as pd #建立df物件 dict_data = { 'key1':['a','b','c','d','a','b','c','d'], 'key2':['one','two','three','one','two','three','one','two'], 'data1':np.random.randint(1,10,8), 'data2':np.random.randint(1,10,8) } df = pd.DataFrame(dict_data) print(df) ''' data1 data2 key1 key2 0 3 4 a one 1 7 9 b two 2 5 7 c three 3 3 4 d one 4 8 7 a two 5 4 7 b three 6 8 9 c one 7 4 4 d two ''' #根據key1分組,進行sum()運算 df = df.groupby('key1').sum() print(df) ''' key1 a 12 10 b 8 5 c 8 11 d 16 13 ''' #內建的聚合函式 print(df.groupby('key1').sum()) print('*'*50) print(df.groupby('key1').max()) print('*'*50) print(df.groupby('key1').min()) print('*'*50) print(df.groupby('key1').mean()) print('*'*50) print(df.groupby('key1').size()) print('*'*50) #分組中非Nan資料的數量 print(df.groupby('key1').count()) print('*'*50) print(df.groupby('key1').describe())
(2)自定義聚合函式進行計算
在使用自定義聚合函式的時候,需要用到一個agg()函式
#自定義聚合函式
#最大值-最小值
def peak_range(df):
#返回資料範圍差值
return df.max()**2 - df.min()**2
#agg() 可以將聚合計算的結果祖闖成一個dataframe物件返回
print(df.groupby('key1').agg(peak_range))
#lambda
print(df.groupby('key1').agg(lambda df:df.max()-df.min()))(3)應用多個聚合函式,預設列索引為函式名
#應用多個聚合函式,預設列索引為函式名 #通過元素重新命名列索引('列索引',函式) print(df.groupby('key1').agg(['sum','std','mean',('range',peak_range)])) ''' data1 data2 sum std mean range sum std mean range key1 a 10 2.828427 5.0 40 12 2.828427 6.0 48 b 10 5.656854 5.0 80 8 1.414214 4.0 16 c 6 1.414214 3.0 12 9 0.707107 4.5 9 d 15 0.707107 7.5 15 8 2.828427 4.0 32 '''
(4)指定每一列使用某個聚合運算函式
#指定每一列使用某個聚合運算函式
print(df.groupby('key1').agg({'data1':'mean','data2':'sum'}))
'''
data1 data2
key1
a 5.0 12
b 5.0 8
c 3.0 9
d 7.5 8
'''2.分組運算
(1)進行分組運算,並在運算後的結果列索引前加字首
加字首用到add_prefix('字首')函式#建立df物件 dict_data = { 'key1':['a','b','c','d','a','b','c','d'], 'key2':['one','two','three','one','two','three','one','two'], 'data1':np.random.randint(1,10,8), 'data2':np.random.randint(1,10,8) } df = pd.DataFrame(dict_data) print(df) ''' data1 data2 key1 key2 0 1 5 a one 1 9 3 b two 2 3 6 c three 3 6 9 d one 4 8 4 a two 5 5 5 b three 6 9 6 c one 7 4 1 d two ''' #按照key1分組,進行sum()運算 #在運算結果的列索引前新增字首 k1_sum = df.groupby('key1').sum().add_prefix('sum_') print(k1_sum) ''' sum_data1 sum_data2 key1 a 9 9 b 14 8 c 12 12 d 10 10 '''
(2)進行分組運算,並把原始資料和結果資料合併
#建立df物件
dict_data = {
'key1':['a','b','c','d','a','b','c','d'],
'key2':['one','two','three','one','two','three','one','two'],
'data1':np.random.randint(1,10,8),
'data2':np.random.randint(1,10,8)
}
df = pd.DataFrame(dict_data)
print(df)
'''
data1 data2 key1 key2
0 1 5 a one
1 9 3 b two
2 3 6 c three
3 6 9 d one
4 8 4 a two
5 5 5 b three
6 9 6 c one
7 4 1 d two
'''
#按照key1分組,進行sum()運算
#在運算結果的列索引前新增字首
k1_sum = df.groupby('key1').sum().add_prefix('sum_')
print(k1_sum)
'''
sum_data1 sum_data2
key1
a 9 9
b 14 8
c 12 12
d 10 10
'''
#將運算結果和原始資料拼接到一起
#引數1:原始資料
#引數2:運算結果資料
pd.merge(df,k1_sum,left_on='key1',right_index=True)#建立df物件
dict_data = {
'key1':['a','b','c','d','a','b','c','d'],
'key2':['one','two','three','one','two','three','one','two'],
'data1':np.random.randint(1,10,8),
'data2':np.random.randint(1,10,8)
}
df = pd.DataFrame(dict_data)
print(df)
'''
data1 data2 key1 key2
0 1 5 a one
1 9 3 b two
2 3 6 c three
3 6 9 d one
4 8 4 a two
5 5 5 b three
6 9 6 c one
7 4 1 d two
'''
#按照key1分組,進行sum()運算
#在運算結果的列索引前新增字首
k1_sum = df.groupby('key1').sum().add_prefix('sum_')
print(k1_sum)
'''
sum_data1 sum_data2
key1
a 9 9
b 14 8
c 12 12
d 10 10
'''
#transform() 計算 會將計算的結果按照原始資料的排序組裝成一個dataframe物件
k1_sum_tf = df.groupby('key1').transform(np.sum).add_prefix('sum_')
# print(k1_sum_tf.columns)
#把運算結果資料拼接到原始資料後
df[k1_sum_tf.columns] = k1_sum_tf
print(df)
'''
data1 data2 key1 key2 sum_data1 sum_data2 sum_key2
0 5 4 a one 9 12 onetwo
1 3 3 b two 5 12 twothree
2 9 2 c three 14 9 threeone
3 6 5 d one 11 9 onetwo
4 4 8 a two 9 12 onetwo
5 2 9 b three 5 12 twothree
6 5 7 c one 14 9 threeone
7 5 4 d two 11 9 onetwo
'''