資料分析 四 pandas的拼接操作
阿新 • • 發佈:2021-01-02
pandas的拼接操作
pandas的拼接分為兩種:
- 級聯:pd.concat, pd.append
- 合併:pd.merge, pd.join
1. 使用pd.concat()級聯
pandas使用pd.concat函式,與np.concatenate函式類似,只是多了一些引數:
objs
axis=0
keys
join='outer' / 'inner':表示的是級聯的方式,outer會將所有的項進行級聯(忽略匹配和不匹配),而inner只會將匹配的項級聯到一起,不匹配的不級聯
ignore_index=False
1)匹配級聯
import pandas as pdfrom pandas import Series,DataFrame import numpy as np
df1 = DataFrame(data=np.random.randint(0,100,size=(3,4)),index=['a','b','c']) df2 = DataFrame(data=np.random.randint(0,100,size=(3,3)),index=['a','d','c']) pd.concat((df1,df1),axis=0) ============ 0 1 2 a 5 53 94 b 5 26 13 c65 60 90 a 5 53 94 b 5 26 13 c 65 60 90
2) 不匹配級聯
不匹配指的是級聯的維度的索引不一致。例如縱向級聯時列索引不一致,橫向級聯時行索引不一致
有2種連線方式:
- 外連線:補NaN(預設模式)
- 內連線:只連線匹配的項
pd.concat((df1,df2),axis=0,join='inner') # pd.concat((df1,df2),axis=1)
0 1 2 a 15 46 58 b 56 28 94 c26 49 98 a 43 37 93 d 63 91 82 c 40 34 16
2. 使用pd.merge()合併
merge與concat的區別在於,merge需要依據某一共同的列來進行合併
使用pd.merge()合併時,會自動根據兩者相同column名稱的那一列,作為key來進行合併。
注意每一列元素的順序不要求一致
引數:
- how:out取並集 inner取交集
- on:當有多列相同的時候,可以使用on來指定使用那一列進行合併,on的值為一個列表
1) 一對一合併
df1 = DataFrame({'employee':['Bob','Jake','Lisa'], 'group':['Accounting','Engineering','Engineering'], }) df1 ================= employee group 0 Bob Accounting 1 Jake Engineering 2 Lisa Engineering
df2 = DataFrame({'employee':['Lisa','Bob','Jake'], 'hire_date':[2004,2008,2012], }) df2 =============== employee hire_date 0 Lisa 2004 1 Bob 2008 2 Jake 2012
pd.merge(df1,df2)
pd.merge(df1,df2) ==================== employee group hire_date 0 Bob Accounting 2008 1 Jake Engineering 2012 2 Lisa Engineering 2004
2) 多對一合併
df3 = DataFrame({ 'employee':['Lisa','Jake'], 'group':['Accounting','Engineering'], 'hire_date':[2004,2016]}) df3
employee group hire_date
0 Lisa Accounting 2004
1 Jake Engineering 2016
df4 = DataFrame({'group':['Accounting','Engineering','Engineering'], 'supervisor':['Carly','Guido','Steve'] }) df4 =========== group supervisor 0 Accounting Carly 1 Engineering Guido 2 Engineering Steve
pd.merge(df3,df4) ===== employee group hire_date supervisor 0 Lisa Accounting 2004 Carly 1 Jake Engineering 2016 Guido 2 Jake Engineering 2016 Steve
3) 多對多合併
df1 = DataFrame({'employee':['Bob','Jake','Lisa'], 'group':['Accounting','Engineering','Engineering']}) df1
employee group 0 Bob Accounting 1 Jake Engineering 2 Lisa Engineering
df5 = DataFrame({'group':['Engineering','Engineering','HR'], 'supervisor':['Carly','Guido','Steve'] }) df5
group supervisor 0 Engineering Carly 1 Engineering Guido 2 HR Steve
pd.merge(df1,df5,how='outer') ======= employee group supervisor 0 Bob Accounting NaN 1 Jake Engineering Carly 2 Jake Engineering Guido 3 Lisa Engineering Carly 4 Lisa Engineering Guido 5 NaN HR Steve
4) key的規範化
- 當列衝突時,即有多個列名稱相同時,需要使用on=來指定哪一個列作為key,配合suffixes指定衝突列名
df1 = DataFrame({'employee':['Jack',"Summer","Steve"], 'group':['Accounting','Finance','Marketing']}) df1 =============== employee group 0 Jack Accounting 1 Summer Finance 2 Steve Marketing
f2 = DataFrame({'employee':['Jack','Bob',"Jake"], 'hire_date':[2003,2009,2012], 'group':['Accounting','sell','ceo']}) df2 ================ employee group hire_date 0 Jack Accounting 2003 1 Bob sell 2009 2 Jake ceo 2012
pd.merge(df1,df2,on='group',how='outer') ============== employee_x group employee_y hire_date 0 Jack Accounting Jack 2003.0 1 Summer Finance NaN NaN 2 Steve Marketing NaN NaN 3 NaN sell Bob 2009.0 4 NaN ceo Jake 2012.0
當兩張表沒有可進行連線的列時,可使用left_on和right_on手動指定merge中左右兩邊的哪一列列作為連線的列
df1 = DataFrame({'employee':['Bobs','Linda','Bill'], 'group':['Accounting','Product','Marketing'], 'hire_date':[1998,2017,2018]}) df1 ============== employee group hire_date 0 Bobs Accounting 1998 1 Linda Product 2017 2 Bill Marketing 2018
df5 = DataFrame({'name':['Lisa','Bobs','Bill'], 'hire_dates':[1998,2016,2007]}) df5 ============= hire_dates name 0 1998 Lisa 1 2016 Bobs 2 2007 Bill
pd.merge(df1,df5,left_on='employee',right_on='name',how='outer') ================== employee group hire_date hire_dates name 0 Bobs Accounting 1998.0 2016.0 Bobs 1 Linda Product 2017.0 NaN NaN 2 Bill Marketing 2018.0 2007.0 Bill 3 NaN NaN NaN 1998.0 Lisa
5) 內合併與外合併:out取並集 inner取交集
- 內合併:只保留兩者都有的key(預設模式)
df6 = DataFrame({'name':['Peter','Paul','Mary'], 'food':['fish','beans','bread']} ) df7 = DataFrame({'name':['Mary','Joseph'], 'drink':['wine','beer']})
外合併 how='outer':補NaN
df6 = DataFrame({'name':['Peter','Paul','Mary'], 'food':['fish','beans','bread']} ) df7 = DataFrame({'name':['Mary','Joseph'], 'drink':['wine','beer']})