2. 程式人生 > >新增一個列到dataframe中

新增一個列到dataframe中

阿新 • • 發佈：2019-01-27
Nathan and I have been working on the Titanic Kaggle problem using the pandas data analysis library and one thing we wanted to do was add a column to a DataFrame indicating if someone survived.

We had the following (simplified) DataFrame containing some information about customers on board the Titanic:

def addrow(df, row):
    return df.append(pd.DataFrame(row), ignore_index=True)
 
customers = pd.DataFrame(columns=['PassengerId','Pclass','Name','Sex','Fare'])
customers = addrow(customers, [dict(PassengerId=892, Pclass=3, Name="Kelly, Mr. James", Sex="male", Fare=7.8292)])
customers = addrow(customers,
 
 [dict(PassengerId=893, Pclass=3, Name="Wilkes, Mrs. James (Ellen Needs)", Sex="female", Fare=7)])
 
>>>customers
 
     Fare                              Name  PassengerId  Pclass     Sex
0  7.8292                  Kelly, Mr. James          892       3    male
1  7.0000  Wilkes, Mrs. James (Ellen Needs)
 
          893       3  female

We wanted to add a ‘Survived’ column to that by doing a lookup in the survival_table below to work out the appropriate value: 

survival_table = pd.DataFrame(columns=['Sex', 'Pclass', 'PriceDist', 'Survived'])
 
survival_table = addrow(survival_table, [dict(Pclass=1, Sex="female", PriceDist = 0, Survived = 0)])
survival_table = addrow(survival_table, [dict(Pclass=1, Sex="male", PriceDist = 0, Survived = 0)])
survival_table = addrow(survival_table, [dict(Pclass=1, Sex="female", PriceDist = 1, Survived = 0)])
survival_table = addrow(survival_table, [dict(Pclass=1, Sex="male", PriceDist = 1, Survived = 0)])
survival_table = addrow(survival_table, [dict(Pclass=1, Sex="female", PriceDist = 2, Survived = 1)])
survival_table = addrow(survival_table, [dict(Pclass=1, Sex="male", PriceDist = 2, Survived = 0)])
survival_table = addrow(survival_table, [dict(Pclass=1, Sex="female", PriceDist = 3, Survived = 1)])
survival_table = addrow(survival_table, [dict(Pclass=1, Sex="male", PriceDist = 3, Survived = 0)])
 
survival_table = addrow(survival_table, [dict(Pclass=2, Sex="female", PriceDist = 0, Survived = 0)])
survival_table = addrow(survival_table, [dict(Pclass=2, Sex="male", PriceDist = 0, Survived = 0)])
survival_table = addrow(survival_table, [dict(Pclass=2, Sex="female", PriceDist = 1, Survived = 1)])
survival_table = addrow(survival_table, [dict(Pclass=2, Sex="male", PriceDist = 1, Survived = 0)])
survival_table = addrow(survival_table, [dict(Pclass=2, Sex="female", PriceDist = 2, Survived = 1)])
survival_table = addrow(survival_table, [dict(Pclass=2, Sex="male", PriceDist = 2, Survived = 0)])
survival_table = addrow(survival_table, [dict(Pclass=2, Sex="female", PriceDist = 3, Survived = 1)])
survival_table = addrow(survival_table, [dict(Pclass=2, Sex="male", PriceDist = 3, Survived = 0)])
 
survival_table = addrow(survival_table, [dict(Pclass=3, Sex="female", PriceDist = 0, Survived = 1)])
survival_table = addrow(survival_table, [dict(Pclass=3, Sex="male", PriceDist = 0, Survived = 0)])
survival_table = addrow(survival_table, [dict(Pclass=3, Sex="female", PriceDist = 1, Survived = 1)])
survival_table = addrow(survival_table, [dict(Pclass=3, Sex="male", PriceDist = 1, Survived = 0)])
survival_table = addrow(survival_table, [dict(Pclass=3, Sex="female", PriceDist = 2, Survived = 0)])
survival_table = addrow(survival_table, [dict(Pclass=3, Sex="male", PriceDist = 2, Survived = 0)])
survival_table = addrow(survival_table, [dict(Pclass=3, Sex="female", PriceDist = 3, Survived = 0)])
survival_table = addrow(survival_table, [dict(Pclass=3, Sex="male", PriceDist = 3, Survived = 0)])
 
>>>survival_table
 
    Pclass  PriceDist     Sex  Survived
0        1          0  female         0
1        1          0    male         0
2        1          1  female         0
3        1          1    male         0
4        1          2  female         1
5        1          2    male         0
6        1          3  female         1
7        1          3    male         0
8        2          0  female         0
9        2          0    male         0
10       2          1  female         1
11       2          1    male         0
12       2          2  female         1
13       2          2    male         0
14       2          3  female         1
15       2          3    male         0
16       3          0  female         1
17       3          0    male         0
18       3          1  female         1
19       3          1    male         0
20       3          2  female         0
21       3          2    male         0
22       3          3  female         0
23       3          3    male         0

To do this we can use the function which allows us to map over each row.

Our initial attempt read like this:

def select_bucket(fare):
    if (fare >= 0 and fare < 10):
        return 0
    elif (fare >= 10 and fare < 20):
        return 1
    elif (fare >= 20 and fare < 30):
        return 2
    else:
        return 3
 
def calculate_survival(survival_table, customer):
    survival_row = survival_table[(survival_table["Sex"] == customer["Sex"]) & (survival_table["Pclass"] == customer["Pclass"]) & (survival_table["PriceDist"] == select_bucket(customer["Fare"]))]
    return survival_row["Survived"]
 
>>>customers["Survived"] = customers.apply(lambda customer: calculate_survival(survival_table, customer), axis=1)

When we ran that we got the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Python/2.7/site-packages/pandas/core/frame.py", line 2119, in __setitem__
    self._set_item(key, value)
  File "/Library/Python/2.7/site-packages/pandas/core/frame.py", line 2166, in _set_item
    NDFrame._set_item(self, key, value)
  File "/Library/Python/2.7/site-packages/pandas/core/generic.py", line 679, in _set_item
    self._data.set(key, value)
  File "/Library/Python/2.7/site-packages/pandas/core/internals.py", line 1781, in set
    self.insert(len(self.items), item, value)
  File "/Library/Python/2.7/site-packages/pandas/core/internals.py", line 1795, in insert
    self._add_new_block(item, value, loc=loc)
  File "/Library/Python/2.7/site-packages/pandas/core/internals.py", line 1911, in _add_new_block
    self.items, fastpath=True)
  File "/Library/Python/2.7/site-packages/pandas/core/internals.py", line 966, in make_block
    return klass(values, items, ref_items, ndim=values.ndim, fastpath=fastpath, placement=placement)
  File "/Library/Python/2.7/site-packages/pandas/core/internals.py", line 44, in __init__
    % (len(items), len(values)))
ValueError: Wrong number of items passed 1, indices imply 2

After much googling and confusion as to why we were getting this error I tried printing out the result of calling apply rather than immediately assigning it and realised that the output wasn’t what I expected:

>>> customers.apply(lambda customer: calculate_survival(survival_table, customer), axis=1)
   16  17
0 NaN   0
1   1 NaN

I’d expected to get one column showing the survived values but instead we’ve got a 2×2 DataFrame. Adding some logging to the calculate_survival function revealed why: 

def calculate_survival(survival_table, customer):
    survival_row = survival_table[(survival_table["Sex"] == customer["Sex"]) & (survival_table["Pclass"] == customer["Pclass"]) & (survival_table["PriceDist"] == select_bucket(customer["Fare"]))]
    print(type(survival_row["Survived"]))
    return survival_row["Survived"]
 
>>> customers.apply(lambda customer: calculate_survival(survival_table, customer), axis=1)
<class 'pandas.core.series.Series'>
<class 'pandas.core.series.Series'>
   16  17
0 NaN   0
1   1 NaN

Our function is actually returning a Series object rather than a single value 0 or 1 which I found surprising. We can use the iat function to retrieve a scalar value from a Series: 

def calculate_survival(survival_table, customer):
    survival_row = survival_table[(survival_table["Sex"] == customer["Sex"]) & (survival_table["Pclass"] == customer["Pclass"]) & (survival_table["PriceDist"] == select_bucket(customer["Fare"]))]
    return int(survival_row["Survived"].iat[0])
 
>>>customers.apply(lambda customer: calculate_survival(survival_table, customer), axis=1)
0    0
1    1
dtype: int64

Now if we assign the output of that function like before it works as expected:

>>>customers["Survived"] = customers.apply(lambda customer: calculate_survival(survival_table, customer), axis=1)
>>>customers
     Fare                              Name  PassengerId  Pclass     Sex  Survived
0  7.8292                  Kelly, Mr. James          892       3    male         0
1  7.0000  Wilkes, Mrs. James (Ellen Needs)          893       3  female         1

相關推薦

新增一個列到dataframe中

Nathan and I have been working on the Titanic Kaggle problem using the pandas data analysis library and one thing we wanted to do was add a column to a D

Mybatis list 新增 一個list元素中的條件

今天在寫專案的時候，遇到一個問題，關於mybatis的問題。 在select的時候，如果傳入的引數是一個list，這時，我們需要用到 WHERE t.id in ( <foreach collection="list" index="index" item="item"

Python（Pandas） 中map, applymap and apply的區別（記住作用物件：dataframe中的列，每一個元素，series每一個值）

1.apply() 當想讓方程作用在一維的向量上時，可以使用apply來完成，如下所示 In [116]: frame = DataFrame(np.random.randn(4, 3), columns=list('bde'), index=['Utah', 'Ohio'

轉換一個字段中含有多個另外一張表的id | | 行轉列

ret .net sub 模式 end posit tex 匹配 replace 1、Oracle Function --入參：表 id /* cxcyxm02 ：操作表，含有一個存儲另一張表（xs0101）多個主鍵id的字段 　ids ：操作表的主鍵 i

SHELL腳本：將新增的行添加到另一個文件中

shell#!/bin/bashcd /usr/local/logstash/nginxlog[ -f num.txt ] || count=0[ -f num.txt ] && count=cat num.txtncount=wc -l /var/log/nginx/access-app.l

03 React快速入門（三）——實現從一個輸入框中新增完資料後此輸入框內容清除的功能

功能描述：       我們在一個輸入框輸入內容，然後點選新增按鈕，此輸入框的內容就會新增到頁面上，但是此輸入框中還存在上次輸入的內容，我們想在每次輸入新增完成之後，此輸入框中的內容就會清除，如圖：      

Java List中新增一個物件多次

在實際應用場景中，可以需要在一個List中新增多個物件，在使用的時候有個誤區就是將一個物件新增多次到List中，導致資料不一致。 測試程式碼: public class test { public static void main(String[] args) {

使用gevent.monkey.patch_all() 協程池遇到阻塞，在任務中新增一個sleep()搞定～

關於使用gevent.monkey.patch_all()協程池遇到阻塞解決辦法： import time ... class Example: ... time.sleep(0.000001) 在進行多執行緒時monkey會阻塞住執行緒的繼續執行，需要對monkey.p

webgl中新增一個div框實現框選

在webgl的開發中，新增一個div，可以實現框選的樣式。過程很簡單，直接看程式碼吧，這裡主要實現了點選第一次的時候確定div的位置，第二次實時顯示div的大小及樣式。這裡為了顯示實在canvas上畫div，將canvas設定為紅色。 <!DOCTYPE HTML PUBLIC "-//

selenium 難定位元素 關於定位一列資料中的某一個

定義了一個函式findname.py,獲得含有關鍵字的列表名稱，並返回list最大值； def fnn(names, keys): k = [] for name in names: #迴圈獲得屬性為value的值 s = na

pandas將DataFrame中的tuple分割成資料框的多列

通過apply(pd.Series)實現將tuple進行分列 df = pd.DataFrame({'a':[1,2], 'b':[(1,2), (3,4)]}) df['b'].apply(pd.Series) df[['b1', 'b2']] = df['b'].apply(pd.S

layui table中新增一列序號列

頁面檔案在相應的位置上新增下面兩個模組的程式碼： 第一模組：在render table 時新增： { field: 'rank', title: '序號', width: 80, sort: true, fixed: 'left', templet: '#rank' } 第二模組：在<script

mysql中給查詢結果新增序號列

　生成一個欄位(非表中欄位)用以記錄排序   【類比為oracle資料庫中的rownum】 　　　　MySQL中一個表（表名：stuscore） 　　　　欄位有：id(主鍵)、stuid（學號）、sname(學生姓名)、subject(課程名稱)、score(分

java--建立一個帶預設構造方法（即無參構造）的類，在構造方法中列印一條訊息"Hello Constructor";再為這個類新增一個過載構造方法，令其接收一個字串引數，將其一起打印出來

題目描述：建立一個帶預設構造方法（即無參構造）的類，在構造方法中列印一條訊息"Hello Constructor";再為這個類新增一個過載構造方法，令其接收一個字串引數，並在這個有參構造方法中把"Hello Constructor"和接收的引數一起打印出來。 //Person類 class Pe

Spark DataFrame 新增索引列的三種方法

Spark DataFrame 新增索引列的三種方法 剛開始用Spark，操作dataframe不是很熟練，遇到的第一個問題是給dataframe新增索引列，查閱了網上的一些教程，大都是用Scala語言編寫的程式碼，下面給出自己用python寫的三種方法。 方法一：先建立

JavaScript中prototype（原型）給字串物件新增一個toCharArray的方法，reverse(翻轉)的 方法

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http:/

使用layui 做後臺管理介面，在Tab中的連結點選後新增一個新TAB的解決方法

給連結或按鈕  新增 onclick="self.parent.addTab('百度','http://www.baidu.com','icon-add')"   如： <a href="javascript:void(0)" title="google" onclick="s

向陣列中新增一個元素

實現效果： 　　 知識運用： 　　Array物件的Length屬性　int類的tryParse()方法 實現程式碼： int[] int_arr; //"隨機陣列"按鈕事件 private void button1_Click(object se

CentOS 7中新增一個新使用者並授權

CentOS 7中新增一個新使用者並授權 [日期：2016-11-26] 來源：Linux社群  作者：woshimrf [字型：大 中 小]     前言 筆記本安裝了一個CentOS，

Vue+Element UI 向Table組建中的每一行新增一個switch元件，實現每一行單獨控制

  最近在做公司的裝置管理系統，許可權管理中有一個需求需要展示如下： 每一行表格中的switch單獨控制一行； 實現效果的程式碼如下： <el-table :data="userRights" stripe border align="center" sty