oozie(1):oozie安裝步驟
阿新 • • 發佈:2018-12-08
一、準備工作
1.準備軟體
oozie和ext2.2
2.Hadoop
3.mysql
建資料庫
mysql> create database oozie;
二、安裝和配置
1.上傳解壓oozie安裝包
tar -zxvf oozie-4.1.0-cdh5.7.0.tar.gz -C /opt/modules/
2.修改hadoop裡面的core-site.xml
(1)修改core-site.xml
#root替換為自己使用者名稱 <property> <name>hadoop.proxyuser.root.hosts</name> <value>*</value> </property> <property> <name>hadoop.proxyuser.root.groups</name> <value>*</value> </property>
(2)重啟hadoop叢集服務
sbin/start-dfs.sh
sbin/start-yarn.sh
啟動history
sbin/mr-jobhistory-daemon.sh start historyserver
3.生成libext目錄,並拷貝相關的jar包
(1)oozie安裝目錄下
mkdir libext
(2)oozie安裝目錄同級目錄下解壓oozie-hadooplibs-4.1.0-cdh5.7.0.tar.gz
tar -zxvf oozie-hadooplibs-4.1.0-cdh5.7.0.tar.gz -C /opt/modules/
解壓縮後,在/opt/modules/oozie-4.1.0-cdh5.7.0目錄下會增加hadooplibs包!
(3)拷貝解壓出來的jar包到{OOZIE_HOME}/libext
cp hadooplibs/hadooplib-2.6.0-cdh5.7.0.oozie-4.1.0-cdh5.7.0/* /opt/modules/oozie-4.1.0-cdh5.7.0/libext/
(4)拷貝ext-2.2.zip到{OOZIE_HOME}/libext
cp /opt/software/ext-2.2.zip libext/
(5)拷貝mysql的驅動包到{OOZIE_HOME}/libext裡面
cp /opt/modules/hive-1.1.0-cdh5.7.0/lib/mysql-connector-java-5.1.27-bin.jar /opt/modules/oozie-4.1.0-cdh5.7.0/libext/
4.修改oozie配置檔案 {OOZIE_HOME}/conf/oozie-site.xml
<property>
<name>oozie.service.JPAService.jdbc.driver</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>oozie.service.JPAService.jdbc.url</name>
<value>jdbc:mysql://hadoop:3306/oozie</value>
</property>
<property>
<name>oozie.service.JPAService.jdbc.username</name>
<value>root</value>
</property>
<property>
<name>oozie.service.JPAService.jdbc.password</name>
<value>123456</value>
</property>
** 關聯HDFS
<property>
<name>oozie.service.HadoopAccessorService.hadoop.configurations</name>
<value>*=/opt/modules/hadoop-2.6.0-cdh5.7.0/etc/hadoop</value>
</property>
5、依次執行以下命令
(1)在{OOZIE_HOME},生產web工程(war)
bin/oozie-setup.sh prepare-war
(2)如果報錯
unzip: command not found
是因為centOS預設沒有unzip解壓縮命令,所以解決是安裝unzip
yum install -y unzip zip
(3)上傳包到HDFS
bin/oozie-setup.sh sharelib create \
-fs hdfs://hadoop:8020 \
-locallib oozie-sharelib-4.1.0-cdh5.7.0-yarn.tar.gz
(4)生成資料庫
bin/oozie-setup.sh db create -run -sqlfile oozie.sql
結果:
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
Validate DB Connection
DONE
Check DB schema does not exist
DONE
Check OOZIE_SYS table does not exist
DONE
Create SQL schema
DONE
Create OOZIE_SYS table
DONE
Oozie DB has been created for Oozie version '4.1.0-cdh5.7.0'
The SQL commands have been written to: oozie.sql
(5)檢測成功生成的資料庫
(a)mysql資料庫中
use oozie;
show tables;
結果
mysql> show tables;
+------------------------+
| Tables_in_oozie |
+------------------------+
| BUNDLE_ACTIONS |
| BUNDLE_JOBS |
| COORD_ACTIONS |
| COORD_JOBS |
| OOZIE_SYS |
| OPENJPA_SEQUENCE_TABLE |
| SLA_EVENTS |
| SLA_REGISTRATION |
| SLA_SUMMARY |
| VALIDATE_CONN |
| WF_ACTIONS |
| WF_JOBS |
+------------------------+
12 rows in set (0.00 sec)
(b)oozie.sql檔案:剛剛建立的oozie資料庫中資料表的所有sql語句
cat /opt/modules/oozie-4.1.0-cdh5.7.0/oozie.sql
結果:
...
CREATE INDEX I_WF_CTNS_PENDING_AGE ON WF_ACTIONS (pending_age);
CREATE INDEX I_WF_CTNS_STATUS ON WF_ACTIONS (status);
CREATE INDEX I_WF_CTNS_WF_ID ON WF_ACTIONS (wf_id);
CREATE INDEX I_WF_JOBS_END_TIME ON WF_JOBS (end_time);
CREATE INDEX I_WF_JOBS_EXTERNAL_ID ON WF_JOBS (external_id);
CREATE INDEX I_WF_JOBS_LAST_MODIFIED_TIME ON WF_JOBS (last_modified_time);
CREATE INDEX I_WF_JOBS_PARENT_ID ON WF_JOBS (parent_id);
CREATE INDEX I_WF_JOBS_STATUS ON WF_JOBS (status);
create table OOZIE_SYS (name varchar(100), data varchar(100))
insert into OOZIE_SYS (name, data) values ('db.version', '3')
insert into OOZIE_SYS (name, data) values ('oozie.version', '4.1.0-cdh5.7.0')
6.啟動服務
bin/oozied.sh start
7.測試前端頁面oozie
http://hadoop:11000/oozie/
8.oozie停止指令碼
bin/oozie-stop.sh