0015-如何使用Sentry管理Hive外部表許可權
阿新 • • 發佈：2018-11-19
溫馨提示：要看高清無碼套圖，請使用手機開啟並單擊圖片放大檢視。
1.文件編寫目的
本文件主要講述如何使用Sentry對Hive外部表許可權管理，並基於以下假設：
1.作業系統版本：RedHat6.5
2.CM版本：CM 5.11.1
3.叢集已啟用Kerberos和Sentry
4.採用具有sudo許可權的ec2-user使用者進行操作
2.前置準備
2.1建立外部表資料父目錄
1.使用hive使用者登入Kerberos
[[email protected] 1874-hive-HIVESERVER2]# kinit -kt hive.keytab hive/ 
2.建立HDFS目錄
使用如下命令在HDFS的根目錄下建立Hive外部表的資料目錄/extwarehouse
[[email protected] ec2-user]# hadoop fs -mkdir /extwarehouse
[[email protected] ec2-user]# hadoop fs -ls /
drwxr-xr-x - hive supergroup 0 2017-09-01 11:27 /extwarehouse
drwxrwxrwx - user_r supergroup 0 2017-08-23 03:23 /fayson
drwx------ - hbase hbase 0 2017-09-01 02:59 /hbase
drwxrwxrwt - hdfs supergroup 0 2017-08-31 06:18 /tmp
drwxrwxrwx - hdfs supergroup 0 2017-08-30 03:48 /user
[ 
2.2配置外部表資料父目錄的ACL同步
1.確保HDFS已開啟sentry並啟用ACL同步
2.配置sentry同步路徑（2.1建立的Hive外部表資料目錄）
3.配置完成，重啟服務。
3.建立Hive外部表
1.使用beeline命令列連線hive，建立Hive外部表
建表語句：
create external table if not exists student(
name string,
age int,
addr string
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
LOCATION '/extwarehouse/student';終端操作：
[[email protected] 1874-hive-HIVESERVER2]# beeline
Beeline version 1.1.0-cdh5.11.1 by Apache Hive
beeline> !connect jdbc:hive2://localhost:10000/;principal=hive/[email protected]
...
0: jdbc:hive2://localhost:10000/> create external table if not exists student(
. . . . . . . . . . . . . . . . > name string,
. . . . . . . . . . . . . . . . > age int,
. . . . . . . . . . . . . . . . > addr string
. . . . . . . . . . . . . . . . > )
. . . . . . . . . . . . . . . . > ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
. . . . . . . . . . . . . . . . > LOCATION '/extwarehouse/student';
...
INFO : OK
No rows affected (0.236 seconds)
0: jdbc:hive2://localhost:10000/> 
2.向student表中load資料
準備測試資料
[[email protected] student]# pwd
/home/ec2-user/student
[[email protected] student]# ll
total 4
-rw-r--r-- 1 root root 39 Sep 1 11:37 student.txt
[[email protected] student]# cat student.txt
zhangsan,18,guangzhou
lisi,20,shenzhen
[[email protected] student]# 
將student.txt檔案put到hdfs的/tmp/student目錄
[[email protected] student]# hadoop fs -mkdir /tmp/student
[[email protected] student]# ll
total 4
-rw-r--r-- 1 hive hive 39 Sep 1 11:37 student.txt
[[email protected] student]# hadoop fs -put student.txt /tmp/student
[[email protected] student]# hadoop fs -ls /tmp/student
Found 1 items
-rw-r--r-- 3 hive supergroup 39 2017-09-01 11:57 /tmp/stu
dent/student.txt
[[email protected] student]# 
在beeline命令列下，將資料load到student表
0: jdbc:hive2://localhost:10000/> load data inpath '/tmp/student' into table student;
...
INFO : Table default.student stats: [numFiles=1, totalSize=39]
INFO : Completed executing command(queryId=hive_20170901115858_5a76aa76-1b24-40ce-8254-42991856c05b); Time taken: 0.263 seconds
INFO : OK
No rows affected (0.41 seconds)
0: jdbc:hive2://localhost:10000/> 
執行完load命令後，查看錶資料
0: jdbc:hive2://localhost:10000/> select * from student;
...
INFO : OK
+---------------+--------------+---------------+--+
| student.name | student.age | student.addr |
+---------------+--------------+---------------+--+
| zhangsan | 18 | guangzhou |
| lisi | 20 | shenzhen |
+---------------+--------------+---------------+--+
2 rows selected (0.288 seconds)
0: jdbc:hive2://localhost:10000/> 
4.使用fayson使用者在beeline和impala-shell檢視
使用fayson使用者的principal初始化Kerberors的票據
[[email protected] cdh-shell-master]$ kinit fayson
Password for [email protected]:
[[email protected] cdh-shell-master]$ klist
Ticket cache: FILE:/tmp/krb5cc_500
Default principal: [email protected]
Valid starting Expires Service principal
09/01/17 12:27:39 09/02/17 12:27:39 krbtgt/[email protected]
renew until 09/08/17 12:27:39
[[email protected] cdh-shell-master]$ 
4.1訪問hdfs目錄
[[email protected] ~]$ hadoop fs -ls /extwarehouse/student
ls: Permission denied: user=fayson, access=READ_EXECUTE, inode="/extwarehouse/student":hive:hive:drwxrwx--x
[[email protected] ~]$ 
4.2beeline命令列檢視
[[email protected] ~]$ beeline
Beeline version 1.1.0-cdh5.11.1 by Apache Hive
beeline> !connect jdbc:hive2://localhost:10000/;principal=hive/[email protected]
...
INFO : OK
+-----------+--+
| tab_name |
+-----------+--+
+-----------+--+
No rows selected (0.295 seconds)
0: jdbc:hive2://localhost:10000/> select * from student;
Error: Error while compiling statement: FAILED: SemanticException No valid privileges
User fayson does not have privileges for QUERY
The required privileges: Server=server1->Db=default->Table=student->Column=addr->action=select; (state=42000,code=40000)
0: jdbc:hive2://localhost:10000/> 
4.3impala-shell命令列檢視
[[email protected] cdh-shell-master]$ impala-shell
...
[Not connected] > connect ip-172-31-10-156.ap-southeast-1.compute.internal:21000;
Connected to ip-172-31-10-156.ap-southeast-1.compute.internal:21000
Server version: impalad version 2.8.0-cdh5.11.1 RELEASE (build 3382c1c488dff12d5ca8d049d2b59babee605b4e)
[ip-172-31-10-156.ap-southeast-1.compute.internal:21000] > show tables;
Query: show tables
ERROR: AuthorizationException: User '[email protected]' does not have privileges to access: default.*
[ip-172-31-10-156.ap-southeast-1.compute.internal:21000] > select * from student;
Query: select * from student
Query submitted at: 2017-09-01 12:33:06 (Coordinator: http://ip-172-31-10-156.ap-southeast-1.compute.internal:25000)
ERROR: AuthorizationException: User '[email protected]' does not have privileges to execute 'SELECT' on: default.student
[ip-172-31-10-156.ap-southeast-1.compute.internal:21000] > 
4.4測試總結
通過hive使用者建立的外部表，未給fayson使用者賦予student表讀許可權情況下，無許可權訪問hdfs的(/extwarehouse/student)資料目錄，在beeline和impala-shell命令列下，fayson使用者均無許可權查詢student表資料。
5.為fayson使用者賦予student表讀許可權
注：以下操作均在hive管理員使用者下操作
1.建立student_read角色
0: jdbc:hive2://localhost:10000/> create role student_read;
...
INFO : Executing command(queryId=hive_20170901124848_927878ba-0217-4a32-a508-bf29fed67be8): create role student_read
...
INFO : OK
No rows affected (0.104 seconds)
0: jdbc:hive2://localhost:10000/> 
2.將student表的查詢許可權授權給student_read角色
0: jdbc:hive2://localhost:10000/> grant select on table student to role student_read;
...
INFO : Executing command(queryId=hive_20170901125252_8702d99d-d8eb-424e-929d-5df352828e2c): grant select on table student to role student_read
...
INFO : OK
No rows affected (0.111 seconds)
0: jdbc:hive2://localhost:10000/> 
3.將student_read角色授權給fayson使用者組
0: jdbc:hive2://localhost:10000/> grant role student_read to group fayson;
...
INFO : Executing command(queryId=hive_20170901125454_5f27a87e-2f63-46d9-9cce-6f346a0c415c): grant role student_read to group fayson
...
INFO : OK
No rows affected (0.122 seconds)
0: jdbc:hive2://localhost:10000/> 
6.再次測試
使用fayson使用者登入Kerberos
6.1訪問HDFS目錄
訪問student資料所在hdfs目錄/extwarehouse/student
[[email protected] ~]$ hadoop fs -ls /extwarehouse/student
Found 1 items
-rwxrwx--x+ 3 hive hive 39 2017-09-01 14:42 /extwarehouse/student/student.txt
[[email protected] ~]$ 
6.2beeline查詢student表
[[email protected] ~]$ klist
Ticket cache: FILE:/tmp/krb5cc_500
Default principal: [email protected]
Valid starting Expires Service principal
09/01/17 12:58:59 09/02/17 12:58:59 krbtgt/[email protected]
renew until 09/08/17 12:58:59
[[email protected] ~]$
[[email protected] ~]$ beeline
Beeline version 1.1.0-cdh5.11.1 by Apache Hive
beeline> !connect jdbc:hive2://localhost:10000/;principal=hive/[email protected]
...
INFO : OK
+-----------+--+
| tab_name |
+-----------+--+
| student |
+-----------+--+
1 row selected (0.294 seconds)
0: jdbc:hive2://localhost:10000/> select * from student;
...
INFO : OK
+---------------+--------------+---------------+--+
| student.name | student.age | student.addr |
+---------------+--------------+---------------+--+
| zhangsan | 18 | guangzhou |
| lisi | 20 | shenzhen |
+---------------+--------------+---------------+--+
2 rows selected (0.241 seconds)
0: jdbc:hive2://localhost:10000/> 
6.3impala-shell查詢student表
[[email protected] cdh-shell-master]$ klist
Ticket cache: FILE:/tmp/krb5cc_500
Default principal: [email protected]
Valid starting Expires Service principal
09/01/17 12:58:59 09/02/17 12:58:59 krbtgt/[email protected]
renew until 09/08/17 12:58:59
[[email protected] cdh-shell-master]$ impala-shell
...
[Not connected] > connect ip-172-31-10-156.ap-southeast-1.compute.internal:21000;
Connected to ip-172-31-10-156.ap-southeast-1.compute.internal:21000
Server version: impalad version 2.8.0-cdh5.11.1 RELEASE (build 3382c1c488dff12d5ca8d049d2b59babee605b4e)
[ip-172-31-10-156.ap-southeast-1.compute.internal:21000] > show tables;
Query: show tables
+---------+
| name |
+---------+
| student |
+---------+
Fetched 1 row(s) in 0.02s
[ip-172-31-10-156.ap-southeast-1.compute.internal:21000] > select * from student;
...
+----------+-----+-----------+
| name | age | addr |
+----------+-----+-----------+
| zhangsan | 18 | guangzhou |
| lisi | 20 | shenzhen |
+----------+-----+-----------+
Fetched 2 row(s) in 0.13s
[ip-172-31-10-156.ap-southeast-1.compute.internal:21000] > 
6.4測試總結
通過hive使用者建立的外部表，給fayson使用者賦予student表讀許可權後，可正常訪問hdfs的(/extwarehouse/student)資料目錄，在beeline和impala-shell命令列下，fayson使用者均可查詢student表資料。
7.Sentry管理Hive外部表許可權總結
開啟外部表的資料父目錄ACL同步後，不需要單獨的維護外部表資料目錄許可權。
參考文件：
https://www.cloudera.com/documentation/enterprise/latest/topics/sg\_hdfs\_sentry\_sync.html
醉酒鞭名馬，少年多浮誇！ 嶺南浣溪沙，嘔吐酒肆下！摯友不肯放，資料玩的花！
溫馨提示：要看高清無碼套圖，請使用手機開啟並單擊圖片放大檢視。
