1. 程式人生 > >雲上實訓大資料查資料一些技巧

雲上實訓大資料查資料一些技巧

1.獲取表資料

r=requests.post('http://bigdata.yunsx.com/Type/GetTableData/60EDD9CB06E5AC23/F2247D3558E22836')

2.用二進位制儲存爬取的內容

with open('D:/2.html','wb') as f:

    f.write(r.content)

3.

window.Filter = [{"join":"AND","lb":"","field":"Id","op":"=","value":"000002","rb":""}]

圖片未載入

4、模擬查詢按鈕查詢資料(返回的是json資料)。注意:匯出的按鈕其實根本不會查詢資料,只是把之前查詢按鈕找到的資料匯出而已。

4

 

  1. r = requests.post('http://bigdata.yunsx.com/Type/GetTableData/60EDD9CB06E5AC23/F2247D3558E22836',data='pagesize=15&pageindex=1&filter[0][join]=AND&filter[0][lb]=&filter[0][field]=Id&filter[0][op]=%3D&filter[0][value]=11111&filter[0][rb]=')  
  2.   
  3. with open('D:/2.html','wb') as f:  
  4.     f.write(r.content)  
  5.   
  6. print(r.text)  
  7. {"IsOk":true,"msg":"查詢成功!","SearchDictionary":{"":""},"TypeBean":{"ID":"1126","ReID":"1113","TopID":"1108","Level":3,"Isparent":false
    ,"Isshowed":false,"Sortrank":50,"Typename":"資產負債表","Typetablename":"zcfzb","Typetableexcelvpatch":"D:\\git\\PracticeFactory-Net\\src\\Web.OSS\\UploadFiles\\Images\\d7b0adbd-040f-4801-b883-bc674a88f06a.xlsx","Isexeclimportstated":true,"Contenthtml":"","Createtime":"0001-01-01T00:00:00","CreateuserID":"0","Isdeleted":false},"Items":[["1","深發展A","銀行業","深圳發展銀行股份有限公司","000001","2017-01-01","A","","","","","311258000000","166882000000","93787000000","97450000000","57179000000","8730000000","","","41  

 

 

 

 

5、儲存查詢條件的方法,不是爬取的,爬取出來的json資料還要做進一步處理,不想搞,所以就用下面這種辦法了(檢視原網頁的js程式碼研究出來的方法,因為網頁的過濾器其實是在window.Filter這個變數中儲存著的)

1)正常查詢結果

2)開啟瀏覽器的控制檯,輸入指令 console.log(JSON.stringify(window.Filter))  或直接 JSON.stringify(window.Filter) 即可查詢到過濾器

3)修改過濾器後,再修改window.Filter的值即可。如在控制檯輸入

 window.Filter = [{"join":"AND","lb":"","field":"E","op":"=","value":"2017-12-31","rb":""},{"join":"OR","lb":"","field":"E","op":"=","value":"2017-9-30","rb":""}]

然後再點網上上的查詢按鈕即可看到過濾器已經改變了。