元資料與資料治理|Apache Atlas安裝過程詳解(初步版本)
Apache Atlas安裝過程詳解
一 安裝Apache Atlas
1.下載安裝包執行以下命令
tar xvfz apache-atlas-1.1.0-sources.tar.gz cd apache-atlas-sources-1.1.0/ export MAVEN_OPTS="-Xms2g -Xmx4g" mvn clean -DskipTests install
伺服器記憶體至少要4G(參考別人的部落格,要是伺服器夠就按照上面來吧)
官方下載地址:https://atlas.apache.org/InstallationSteps.html
或者採用git方式下載:git clone https://git-wip-us.apache.org/repos/asf/atlas.git atlas
執行命令出現問題,參考文末的異常彙總(感觸:當時遇到maven版本的問題解決的了一段時間,現在共享出來 ,避免入坑,參看 問題1(文末))
2)打包atlas
(機器上已經裝有hbase和solr)
mvn clean -DskipTests package -Pdist
(機器上沒有裝hbase和solr,atlas自帶hbase和solr)
mvn clean -DskipTests package -Pdist,embedded-hbase-solr
(備註:mvn clean package -DskipTests -Pdist,external-hbase-solr 使用外部的hbase)
剛開始沒有安裝這兩個,所以選擇的這種(安裝用的時間挺長1-2個小時)。
如下是遇到的部分問題
.... 一直卡在這,索性就停止了 ctrl+c
在重複執行命令:如下錯誤
[INFO] Apache Atlas Web Application 1.1.0 ................. SUCCESS [ 26.122 s]
[INFO] Apache Atlas Documentation 1.1.0 ................... SUCCESS [ 7.169 s]
[INFO] Apache Atlas FileSystem Model 1.1.0 ................ SUCCESS [ 4.408 s]
[INFO] Apache Atlas Plugin Classloader 1.1.0 .............. SUCCESS [ 1.483 s]
[INFO] Apache Atlas Hive Bridge Shim 1.1.0 ................ SUCCESS [ 1.752 s]
[INFO] Apache Atlas Hive Bridge 1.1.0 ..................... SUCCESS [ 6.862 s]
[INFO] Apache Atlas Falcon Bridge Shim 1.1.0 .............. SUCCESS [ 1.826 s]
[INFO] Apache Atlas Falcon Bridge 1.1.0 ................... SUCCESS [ 5.559 s]
[INFO] Apache Atlas Sqoop Bridge Shim 1.1.0 ............... SUCCESS [ 1.065 s]
[INFO] Apache Atlas Sqoop Bridge 1.1.0 .................... SUCCESS [ 7.324 s]
[INFO] Apache Atlas Storm Bridge Shim 1.1.0 ............... SUCCESS [ 0.897 s]
[INFO] Apache Atlas Storm Bridge 1.1.0 .................... SUCCESS [ 4.324 s]
[INFO] Apache Atlas Hbase Bridge Shim 1.1.0 ............... SUCCESS [ 3.118 s]
[INFO] Apache Atlas Hbase Bridge 1.1.0 .................... SUCCESS [ 8.481 s]
[INFO] Apache Atlas Kafka Bridge 1.1.0 .................... SUCCESS [ 2.915 s]
[INFO] Apache Atlas Distribution 1.1.0 .................... FAILURE [ 4.330 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 06:17 min
[INFO] Finished at: 2019-01-10T13:46:03+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (hbase) on project atlas-distro: An Ant BuildException has occured: Error while expanding /u01/apache-atlas-sources-1.1.0/distro/hbase/hbase-1.1.2.tar.gz
[ERROR] java.io.EOFException: Unexpected end of ZLIB input stream
[ERROR] around Ant part ...<untar src="/u01/apache-atlas-sources-1.1.0/distro/hbase/hbase-1.1.2.tar.gz" dest="/u01/apache-atlas-sources-1.1.0/distro/target/hbase.temp" compression="gzip"/>... @ 7:164 in /u01/apache-atlas-sources-1.1.0/distro/target/antrun/build-Download HBase.xml
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :atlas-distro
[[email protected] apache-atlas-sources-1.1.0]#
[[email protected] apache-atlas-sources-1.1.0]#
[[email protected] apache-atlas-sources-1.1.0]#
[[email protected] apache-atlas-sources-1.1.0]# pwd
/u01/apache-atlas-sources-1.1.0
[[email protected] apache-atlas-sources-1.1.0]# ls
大概意思是hbase-1.1.2-bin.tar.gz沒有下載完導致的,可是下載特別慢,所以重新在csdn上找了一個上傳到了伺服器上
重新執行打包命令(注意路徑哈:apache-atlas-sources-1.1.0目錄下)
mvn clean -DskipTests package -Pdist,embedded-hbase-solr
上個問題解決了,下載開始下 在solr呢,版本5.51(等著吧)
有出現下載 緩慢的問題了,索性也停止了,下載solr,重新執行命令
最後不可思議的居然成功了,如下圖(紀念下)
3.atlas配置啟動(目錄如下)
命令:
cd /u01/apache-atlas-sources-1.1.0/distro/target/apache-atlas-1.1.0-bin/apache-atlas-1.1.0/bin
./atlas_start.py
看到這個是無比喜悅呀,正式開啟下一 步驟操作
(遺留問題:要是內網無法聯網如何安裝?要是自定義的hbase和 solr如何配置?在初步瞭解atlas後在來晚上此篇)
4. 測試環節
curl -v http://127.0.0.1:21000/api/atlas/admin/version
如提示沒有許可權 ,則需要帶上使用者名稱和密碼,命令如下:
curl -v -u admin:admin http://127.0.0.1:21000/api/atlas/admin/version
命令引數解釋
curl -v -u username:password http://127.0.0.1:21000/api/atlas/admin/version
username:預設admin
password:預設admin
成功
5.頁面訪問
使用者賬號/密碼:admin/admin
問題彙總
問題1 :mvn clean -DskipTests install報錯
Maven版本的原因?
--- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-versions) @ apache-atlas ---
[WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireMavenVersion failed with message:
** MAVEN VERSION ERROR ** Maven 3.5.0 or above is required. See https://maven.apache.org/install.html
[INFO] Apache Atlas Server Build Tools ................... SUCCESS [2.385s]
[INFO] apache-atlas ...................................... FAILURE [1.548s]
[INFO] Apache Atlas Test Utility Tools ................... SKIPPED
[INFO] Apache Atlas Integration .......................... SKIPPED
[INFO] Apache Atlas Common ............................... SKIPPED
[INFO] Apache Atlas Client ............................... SKIPPED
[INFO] atlas-client-common ............................... SKIPPED
[INFO] atlas-client-v1 ................................... SKIPPED
[INFO] Apache Atlas Server API ........................... SKIPPED
[INFO] Apache Atlas Notification ......................... SKIPPED
[INFO] atlas-client-v2 ................................... SKIPPED
[INFO] Apache Atlas Graph Database Projects .............. SKIPPED
[INFO] Apache Atlas Graph Database API ................... SKIPPED
[INFO] Graph Database Common Code ........................ SKIPPED
....
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 8.184s
[INFO] Finished at: Wed Jan 09 14:25:34 CST 2019
[INFO] Final Memory: 48M/1979M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M1:enforce (enforce-versions) on project apache-atlas: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
解決方案 下載高版本的maven
下載最新版的Maven程式:https://maven.apache.org/download.cgi
注意配置maven變數
vi /etc/profile
source /etc/profile
注意驗證下版本
版本解決完後回到apache-atlas-sources-1.1.0繼續執行mvn clean -DskipTests install
這邊報錯是這個 原因造成的