hbase的四種壓縮格式
阿新 • • 發佈：2017-09-08
壓縮 hbase
Hbase支持的壓縮格式：
hbase支持的壓縮格式：GZ（GZIP）,LZ0，LZ4，Snappy
GZ:用於冷數據壓縮，與Snappy和LZ0相比，GZIP的壓縮率更高，但是更消耗CPU，解壓/壓縮速度更慢。
Snappy和LZ0：用於熱數據壓縮，占用CPU少，解壓/壓縮速度比GZ快，但是壓縮率不如GZ高。
Snappy與LZ0相比，Snappy整體性能優於LZ0，Snappy壓縮率比LZ0更低，但是解壓/壓縮速度更快。
LZ4與LZ0相比，LZ4的壓縮率和LZ0的壓縮率相差不多，但是LZ4的解壓/壓縮速度更快。
多數情況下，選擇Snppy或LZ0是比較好的選擇，因為它們的壓縮開銷底，能節省空間。
建表時指定壓縮格式
hbase(main):013:0> create ‘test3‘,{NAME=>‘f1‘},{NAME=>‘f2‘,COMPRESSION=>‘Snappy‘}
0 row(s) in 1.2740 seconds
=> Hbase::Table - test3
hbase(main):014:0> desc ‘test3‘
Table test3 is ENABLED
test3
COLUMN FAMILIES DESCRIPTION
{NAME => ‘f1‘, BLOOMFILTER => ‘ROW‘,VERSIONS => ‘1‘, IN_MEMORY => ‘false‘, KEEP_DELETED_CELLS => ‘FALSE‘,DATA_BLOCK_ENCODING => ‘NONE‘, TTL => ‘FOREVER‘, COMPRESS
ION => ‘NONE‘, MIN_VERSIONS => ‘0‘,BLOCKCACHE => ‘true‘, BLOCKSIZE => ‘65536‘, REPLICATION_SCOPE =>‘0‘}
{NAME => ‘f2‘, BLOOMFILTER => ‘ROW‘,VERSIONS => ‘1‘, IN_MEMORY => ‘false‘, KEEP_DELETED_CELLS => ‘FALSE‘,DATA_BLOCK_ENCODING => ‘NONE‘, TTL => ‘FOREVER‘, COMPRESS
ION => ‘SNAPPY‘, MIN_VERSIONS => ‘0‘,BLOCKCACHE => ‘true‘, BLOCKSIZE => ‘65536‘, REPLICATION_SCOPE =>‘0‘}
2 row(s) in 0.0300 seconds
hbase(main):002:0> create ‘test4‘ ,{NAME=>‘f1‘},{NAME=>‘f2‘,COMPRESSION=>‘GZ‘}
0 row(s) in 1.4900 seconds
=> Hbase::Table - test4
hbase(main):003:0> desc ‘test4‘
Table test4 is ENABLED
test4
COLUMN FAMILIES DESCRIPTION
{NAME => ‘f1‘, BLOOMFILTER => ‘ROW‘, VERSIONS => ‘1‘, IN_MEMORY => ‘false‘, KEEP_DELETED_CELLS => ‘FALSE‘, DATA_BLOCK_ENCODING => ‘NONE‘, TTL => ‘FOREVER‘, COMPRESS
ION => ‘NONE‘, MIN_VERSIONS => ‘0‘, BLOCKCACHE => ‘true‘, BLOCKSIZE => ‘65536‘, REPLICATION_SCOPE => ‘0‘}
{NAME => ‘f2‘, BLOOMFILTER => ‘ROW‘, VERSIONS => ‘1‘, IN_MEMORY => ‘false‘, KEEP_DELETED_CELLS => ‘FALSE‘, DATA_BLOCK_ENCODING => ‘NONE‘, TTL => ‘FOREVER‘, COMPRESS
ION => ‘GZ‘, MIN_VERSIONS => ‘0‘, BLOCKCACHE => ‘true‘, BLOCKSIZE => ‘65536‘, REPLICATION_SCOPE => ‘0‘}
2 row(s) in 0.1290 seconds
建表後修改columnfamily壓縮格式
正確做法是先disable表，再修改列族壓縮格式，enbale表後做major_compact操作。
如下：
hbase(main):004:0> desc ‘test1‘
Table test1 is ENABLED
test1
COLUMN FAMILIES DESCRIPTION
{NAME => ‘f1‘, BLOOMFILTER => ‘ROW‘,VERSIONS => ‘1‘, IN_MEMORY => ‘false‘, KEEP_DELETED_CELLS => ‘FALSE‘,DATA_BLOCK_ENCODING => ‘NONE‘, TTL => ‘FOREVER‘, COMPRESS
ION => ‘NONE‘, MIN_VERSIONS => ‘0‘,BLOCKCACHE => ‘true‘, BLOCKSIZE => ‘65536‘, REPLICATION_SCOPE =>‘0‘}
{NAME => ‘f2‘, BLOOMFILTER => ‘ROW‘,VERSIONS => ‘1‘, IN_MEMORY => ‘false‘, KEEP_DELETED_CELLS => ‘FALSE‘,DATA_BLOCK_ENCODING => ‘NONE‘, TTL => ‘FOREVER‘, COMPRESS
ION => ‘NONE‘, MIN_VERSIONS => ‘0‘,BLOCKCACHE => ‘true‘, BLOCKSIZE => ‘65536‘, REPLICATION_SCOPE =>‘0‘}
2 row(s) in 0.0230 seconds
hbase(main):005:0> disable ‘test1‘
0 row(s) in 2.2870 seconds
hbase(main):006:0> alter ‘test1‘ ,{NAME=>‘f1‘,COMPRESSION=>‘Snappy‘}
Updating all regions with the new schema...
1/1 regions updated.
Done.
0 row(s) in 1.9510 seconds
hbase(main):007:0> enable ‘test1‘
0 row(s) in 1.2820 seconds
hbase(main):008:0> desc ‘test1‘
Table test1 is ENABLED
test1
COLUMN FAMILIES DESCRIPTION
{NAME => ‘f1‘, BLOOMFILTER => ‘ROW‘,VERSIONS => ‘1‘, IN_MEMORY => ‘false‘, KEEP_DELETED_CELLS => ‘FALSE‘,DATA_BLOCK_ENCODING => ‘NONE‘, TTL => ‘FOREVER‘, COMPRESS
ION => ‘SNAPPY‘, MIN_VERSIONS => ‘0‘,BLOCKCACHE => ‘true‘, BLOCKSIZE => ‘65536‘, REPLICATION_SCOPE =>‘0‘}
{NAME => ‘f2‘, BLOOMFILTER => ‘ROW‘,VERSIONS => ‘1‘, IN_MEMORY => ‘false‘, KEEP_DELETED_CELLS => ‘FALSE‘,DATA_BLOCK_ENCODING => ‘NONE‘, TTL => ‘FOREVER‘, COMPRESS
ION => ‘NONE‘, MIN_VERSIONS => ‘0‘,BLOCKCACHE => ‘true‘, BLOCKSIZE => ‘65536‘, REPLICATION_SCOPE =>‘0‘}
2 row(s) in 0.0310 seconds
hbase(main):009:0> major_compact ‘test1‘
0 row(s) in 0.1380 seconds
hbase(main):010:0> desc ‘test1‘
Table test1 is ENABLED
test1
COLUMN FAMILIES DESCRIPTION
{NAME => ‘f1‘, BLOOMFILTER => ‘ROW‘,VERSIONS => ‘1‘, IN_MEMORY => ‘false‘, KEEP_DELETED_CELLS => ‘FALSE‘,DATA_BLOCK_ENCODING => ‘NONE‘, TTL => ‘FOREVER‘, COMPRESS
ION => ‘SNAPPY‘, MIN_VERSIONS => ‘0‘,BLOCKCACHE => ‘true‘, BLOCKSIZE => ‘65536‘, REPLICATION_SCOPE =>‘0‘}
{NAME => ‘f2‘, BLOOMFILTER => ‘ROW‘,VERSIONS => ‘1‘, IN_MEMORY => ‘false‘, KEEP_DELETED_CELLS => ‘FALSE‘,DATA_BLOCK_ENCODING => ‘NONE‘, TTL => ‘FOREVER‘, COMPRESS
ION => ‘NONE‘, MIN_VERSIONS => ‘0‘,BLOCKCACHE => ‘true‘, BLOCKSIZE => ‘65536‘, REPLICATION_SCOPE =>‘0‘}
2 row(s) in 0.0260 seconds
但是沒有disable表，也不做major_compact，列族壓縮格式也修改成功了（暫時不知道原因）。
hbase(main):001:0> desc ‘test‘
Table test is ENABLED
test
COLUMN FAMILIES DESCRIPTION
{NAME => ‘fam1‘, BLOOMFILTER => ‘ROW‘, VERSIONS => ‘1‘, IN_MEMORY => ‘false‘, KEEP_DELETED_CELLS => ‘FALSE‘, DATA_BLOCK_ENCODING => ‘NONE‘, TTL => ‘FOREVER‘, COMPRE
SSION => ‘NONE‘, MIN_VERSIONS => ‘0‘, BLOCKCACHE => ‘true‘, BLOCKSIZE => ‘65536‘, REPLICATION_SCOPE => ‘0‘}
1 row(s) in 0.3680 seconds
hbase(main):002:0> alter ‘test‘,{NAME=>‘fam1‘,COMPRESSION=>‘LZ4‘}
Updating all regions with the new schema...
1/1 regions updated.
Done.
0 row(s) in 2.0460 seconds
hbase(main):003:0> desc ‘test‘
Table test is ENABLED
test
COLUMN FAMILIES DESCRIPTION
{NAME => ‘fam1‘, BLOOMFILTER =>‘ROW‘, VERSIONS => ‘1‘, IN_MEMORY => ‘false‘, KEEP_DELETED_CELLS =>‘FALSE‘, DATA_BLOCK_ENCODING => ‘NONE‘, TTL => ‘FOREVER‘, COMPRE
SSION => ‘LZ4‘, MIN_VERSIONS => ‘0‘,BLOCKCACHE => ‘true‘, BLOCKSIZE => ‘65536‘, REPLICATION_SCOPE =>‘0‘}
1 row(s) in 0.0280 seconds
本文出自 “天黑順路” 博客，請務必保留此出處http://mjal01.blog.51cto.com/12140495/1963644
hbase的四種壓縮格式