1. 程式人生 > >安裝elasticsearch中文切詞外掛hanlp

安裝elasticsearch中文切詞外掛hanlp

hanlp好處的,就是它的data字典比較齊全.

github上有國人寫hanlp支援es的外掛

https://github.com/pengcong90/elasticsearch-analysis-hanlp

下載發現解壓按它的安裝要求總找不到hanlp.properties檔案

將原始碼git下來,發現路徑有問題.

package org.elasticsearch.index.analysis;

import com.hankcs.hanlp.HanLP;
import com.hankcs.hanlp.utility.Predefine;
import com.hankcs.lucene4.HanLPIndexAnalyzer;
import
org.elasticsearch.common.inject.Inject; import org.elasticsearch.common.inject.assistedinject.Assisted; import org.elasticsearch.common.settings.Settings; import org.elasticsearch.env.Environment; import org.elasticsearch.index.IndexSettings; /** */ public class HanLPAnalyzerProvider extends AbstractIndexAnalyzerProvider
<HanLPIndexAnalyzer> {
private final HanLPIndexAnalyzer analyzer; private static String sysPath = String.valueOf(System.getProperties().get("user.dir")); @Inject public HanLPAnalyzerProvider(IndexSettings indexSettings, Environment env, @Assisted String name, @Assisted Settings settings) { super
(indexSettings, name, settings); //原來路徑 //Predefine.HANLP_PROPERTIES_PATH = sysPath.substring(0, sysPath.length()-4) + "/plugins/analysis-hanlp/hanlp.properties"; //修改後正確路徑 Predefine.HANLP_PROPERTIES_PATH = sysPath + "/plugins/analysis-hanlp/hanlp.properties"; analyzer = new HanLPIndexAnalyzer(true); } public static HanLPAnalyzerProvider getIndexAnalyzerProvider(IndexSettings indexSettings, Environment env, String name, Settings settings) { return new HanLPAnalyzerProvider(indexSettings, env, name, settings); } public static HanLPAnalyzerProvider getSmartAnalyzerProvider(IndexSettings indexSettings, Environment env, String name, Settings settings) { return new HanLPAnalyzerProvider(indexSettings, env, name, settings); } @Override public HanLPIndexAnalyzer get() { return this.analyzer; } }

因為它的hanlp版本是1.2.8,最新版本是1.5.4
修改pom.xml為

       <dependency>
                <groupId>com.hankcs</groupId>
                <artifactId>hanlp</artifactId>
                <version>portable-1.5.4</version>
            <!--<systemPath>${pom.basedir}/lib/hanlp-1.2.8.jar</systemPath>-->
            <!--<scope>system</scope>-->
        </dependency>

打包編譯

在$ES_HOME下/plugins建立analysis-hanlp檔案
目錄下結構為
這裡寫圖片描述

hanlp.properties屬性(可以直接從https://github.com/hankcs/HanLP 的realease下載修改root路徑就行了)

#本配置檔案中的路徑的根目錄,根目錄+其他路徑=完整路徑(支援相對路徑,請參考:https://github.com/hankcs/HanLP/pull/254)
#Windows使用者請注意,路徑分隔符統一使用/
root=/opt/elasticsearch-5.5.1/plugins/analysis-hanlp
#核心詞典路徑
CoreDictionaryPath=data/dictionary/CoreNatureDictionary.txt
#2元語法詞典路徑
BiGramDictionaryPath=data/dictionary/CoreNatureDictionary.ngram.txt
#停用詞詞典路徑
CoreStopWordDictionaryPath=data/dictionary/stopwords.txt
#同義詞詞典路徑
CoreSynonymDictionaryDictionaryPath=data/dictionary/synonym/CoreSynonym.txt
#人名詞典路徑
PersonDictionaryPath=data/dictionary/person/nr.txt
#人名詞典轉移矩陣路徑
PersonDictionaryTrPath=data/dictionary/person/nr.tr.txt
#繁簡詞典根目錄
tcDictionaryRoot=data/dictionary/tc
#自定義詞典路徑,用;隔開多個自定義詞典,空格開頭表示在同一個目錄,使用“檔名 詞性”形式則表示這個詞典的詞性預設是該詞性。優先順序遞減。
#另外data/dictionary/custom/CustomDictionary.txt是個高質量的詞庫,請不要刪除。所有詞典統一使用UTF-8編碼。
CustomDictionaryPath=data/dictionary/custom/CustomDictionary.txt; 現代漢語補充詞庫.txt; 全國地名大全.txt ns; 人名詞典.txt; 機構名詞典.txt; 上海地名.txt ns;data/dictionary/person/nrf.txt nrf;
#CRF分詞模型路徑
CRFSegmentModelPath=data/model/segment/CRFSegmentModel.txt
#HMM分詞模型
HMMSegmentModelPath=data/model/segment/HMMSegmentModel.bin
#分詞結果是否展示詞性
ShowTermNature=true
#IO介面卡,實現com.hankcs.hanlp.corpus.io.IIOAdapter介面以在不同的平臺(Hadoop、Redis等)上執行HanLP
#預設的IO介面卡如下,該介面卡是基於普通檔案系統的。
#IOAdapter=com.hankcs.hanlp.corpus.io.FileIOAdapter

plugin-descriptor.properties和plugin-security.policy屬性按 elasticsearch-analysis-hanlp的release包屬性修改.

修改ES啟動,並啟動

vim /opt/elasticsearch-5.5.1config/jvm.options 
#新增
-Djava.security.policy=/opt/elasticsearch-5.5.1/plugins/analysis-hanlp/plugin-security.policy

測試安裝成功否命令

GET /_analyze?analyzer=hanlp-index&pretty=true 
{ 
"text":"公安部:各地校車將享最高路權" 

}