Python 使用thrift連線hbase || 遠端連線hbase
阿新 • • 發佈:2019-02-05
1.首先下載Python & thrift &hbase
有兩種安裝thrift的方式:1.下載的thrift-0.9.3.tar.gz >> 解壓 tar xzvf thrift-0.9.3.tar.gz >> ./configure >> make >> make install >>thrift -version 驗證 2.解壓之後進入lib/py/ >> 執行Python setup.py install 安裝 。這種方式不需要軟連線到python的site-packages/ >> python >> import thrift 驗證。2.解壓hbase:
3.啟動thrift:
單機模式啟動:首先啟動hbase >>cd bin >> ./start-hbase.sh >> 啟動thrift ./hbase-daemon.sh start thrift # 預設的埠是9090 配置hbase的rootdir<property> <name>hbase.rootdir</name> <value>/root/zhutong/hbase_data</value> </property>
環境配置好了 現在準備測試: 編寫test.py 檔案
#!/usr/bin/python #-*- coding:utf-8 -*- from thrift import Thrift from thrift.transport import TSocket from thrift.transport import TTransport from thrift.protocol import TBinaryProtocol from thrift.protocol import TCompactProtocol from hbase import Hbase from hbase.ttypes import * #192.168.110.78:8020 transport1 = TSocket.TSocket('localhost',9090 ) transport = TTransport.TBufferedTransport(transport1) protocol = TCompactProtocol.TCompactProtocol(transport); client = Hbase.Client(protocol) transport.open() contents=ColumnDescriptor(name='cf:',maxVersions=1) tablename='ta' client.createTable(tablename,[contents]) print client.getTableNames()
異常:如果報錯找不到thrift模組 ,檢視Python安裝目錄site-packages下沒有thrift*.egg 檔案 ,使用easy-install 或者第一步中第二種方式安裝。 如果提示主機名錯誤異常;請前去/etc/hosts檔案中新增對映關係。 成功之後再對hbase進行操作;
#!/usr/bin/python
#insert data
from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol
from hbase import Hbase
from hbase.ttypes import *
transport = TSocket.TSocket('localhost', 9090)
transport = TTransport.TBufferedTransport(transport)
protocol = TBinaryProtocol.TBinaryProtocol(transport)
client = Hbase.Client(protocol)
transport.open()
for i in range(10):
row = 'row-key1'+str(i)
mutations = [Mutation(column="cf:a", value=str(i))]
client.mutateRow('ta', row, mutations, None)
#!/usr/bin/python
from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol
from hbase import Hbase
from hbase.ttypes import *
transport = TSocket.TSocket('localhost', 9090)
transport = TTransport.TBufferedTransport(transport)
protocol = TBinaryProtocol.TBinaryProtocol(transport)
client = Hbase.Client(protocol)
transport.open()
tableName = 'ta'
rowKey = 'row-key1'
result = client.getRow(tableName, rowKey, None)
print result
print type(result)
for r in result:
print 'the row is ' , r.row
print 'the values is ' , r.columns.get('cf:a').value
#!/usr/bin/python
# get some data
from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol
from hbase import Hbase
from hbase.ttypes import *
transport = TSocket.TSocket('localhost', 9090)
transport = TTransport.TBufferedTransport(transport)
protocol = TBinaryProtocol.TBinaryProtocol(transport)
client = Hbase.Client(protocol)
transport.open()
scan = TScan()
tableName = 'ta'
id = client.scannerOpenWithScan(tableName, scan, None)
result2 = client.scannerGetList(id, 10)
print result2
print type(result2)
for i in result2:
print i.row
4.遠端連線hbase
修改Python檔案的IP地址和埠進行連線會出現如下的錯誤:或者:
這是因為遠端訪問的hbase也是需要thrift來進行訪問的,所以需要在訪問的hbase伺服器上啟動thrift: nohup hbase thrift -p 9999 start & 啟動 >> ps aux | grep thrift 驗證 然後再執行 : 埠號為你設定的埠號;