1. 程式人生 > >Python 使用thrift連線hbase || 遠端連線hbase

Python 使用thrift連線hbase || 遠端連線hbase

1.首先下載Python & thrift &hbase

有兩種安裝thrift的方式:1.下載的thrift-0.9.3.tar.gz >> 解壓 tar xzvf thrift-0.9.3.tar.gz  >> ./configure >> make >> make install >>thrift -version 驗證     2.解壓之後進入lib/py/   >> 執行Python setup.py install 安裝 。這種方式不需要軟連線到python的site-packages/ >> python >> import thrift 驗證。

2.解壓hbase:

tar xzvf hbase-1.1.2.tar.gz >> cd hbase-thrift  >> thrift -gen py /root/zhutong/hbase-1.1.2/hbase-thrift/src/main/resources/org/apache/hadoop/hbase/thrift 會生成一個 gen-py的檔案 >> cd gen-py >> cp -R hbase/usr/local/python2.7/lib/python2.7/site-packages/ 

3.啟動thrift:

單機模式啟動:首先啟動hbase >>cd bin  >>  ./start-hbase.sh >> 啟動thrift  ./hbase-daemon.sh start thrift  # 預設的埠是9090    配置hbase的rootdir 
<property>
<name>hbase.rootdir</name>
<value>/root/zhutong/hbase_data</value>
</property>

環境配置好了 現在準備測試: 編寫test.py 檔案
#!/usr/bin/python
#-*- coding:utf-8 -*-
from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol
from thrift.protocol import TCompactProtocol
from hbase import Hbase
from hbase.ttypes import *
#192.168.110.78:8020
transport1 = TSocket.TSocket('localhost',9090 )
transport = TTransport.TBufferedTransport(transport1)
protocol = TCompactProtocol.TCompactProtocol(transport);
client = Hbase.Client(protocol)
transport.open()
contents=ColumnDescriptor(name='cf:',maxVersions=1)
tablename='ta'
client.createTable(tablename,[contents])
print client.getTableNames()

異常:如果報錯找不到thrift模組 ,檢視Python安裝目錄site-packages下沒有thrift*.egg 檔案 ,使用easy-install 或者第一步中第二種方式安裝。             如果提示主機名錯誤異常;請前去/etc/hosts檔案中新增對映關係。 成功之後再對hbase進行操作;
#!/usr/bin/python
#insert data
from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol

from hbase import Hbase

from hbase.ttypes import *

transport = TSocket.TSocket('localhost', 9090)

transport = TTransport.TBufferedTransport(transport)

protocol = TBinaryProtocol.TBinaryProtocol(transport)

client = Hbase.Client(protocol)

transport.open()
for i in range(10):
        row = 'row-key1'+str(i)

        mutations = [Mutation(column="cf:a", value=str(i))]
        client.mutateRow('ta', row, mutations, None)
#!/usr/bin/python

from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol

from hbase import Hbase
from hbase.ttypes import *

transport = TSocket.TSocket('localhost', 9090)
transport = TTransport.TBufferedTransport(transport)

protocol = TBinaryProtocol.TBinaryProtocol(transport)

client = Hbase.Client(protocol)

transport.open()

tableName = 'ta'
rowKey = 'row-key1'

result = client.getRow(tableName, rowKey, None)
print result
print type(result)
for r in result:
    print 'the row is ' , r.row
    print 'the values is ' , r.columns.get('cf:a').value

#!/usr/bin/python
# get some data 
from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol

from hbase import Hbase
from hbase.ttypes import *

transport = TSocket.TSocket('localhost', 9090)
transport = TTransport.TBufferedTransport(transport)

protocol = TBinaryProtocol.TBinaryProtocol(transport)

client = Hbase.Client(protocol)
transport.open()

scan = TScan()
tableName = 'ta'
id = client.scannerOpenWithScan(tableName, scan, None)

result2 = client.scannerGetList(id, 10)

print result2
print type(result2)

for i in result2:
        print i.row


4.遠端連線hbase

 修改Python檔案的IP地址和埠進行連線會出現如下的錯誤:
或者

這是因為遠端訪問的hbase也是需要thrift來進行訪問的,所以需要在訪問的hbase伺服器上啟動thrift: nohup hbase thrift -p 9999 start  &  啟動 >> ps aux | grep thrift 驗證 然後再執行 : 埠號為你設定的埠號;