[[email protected] ~]# sh -x /tmp/es.sh
+ curl -XPOST localhost:9500/_cluster/reroute -d ‘{
“commands”:[
{
“allocate”:{
“index”: “simtest”,
“shard”: 0,
“node”: “Lady Mastermind”,
“allow_primary”: true
}
}
]
}’

{
    "acknowledged": true,
    "state": {
        "version": 40,
        "master_node": "zVTN_KP_TX6o1RrZNr4TAg",
        "blocks": {},
        "nodes": {
            "zVTN_KP_TX6o1RrZNr4TAg": {
                "name": "Lady Mastermind",
                "transport_address": "inet[/192.168.126.182:9511]"
 
,
                "attributes": {}
            }
        },
        "metadata": {
            "templates": {
                "ss_template_v1": {
                    "template": "ss_index*",
                    "order": 0,
                    "settings": {
                        "index": {
                            "number_of_replicas"
 
: "0",
                            "analysis": {
                                "filter": {
                                    "filter1": {
                                        "type": "standard"
                                    },
                                    "filter3": {
                                        "type": "stop"
                                    },
                                    "filter2": {
                                        "type": "lowercase"
                                    }
                                },
                                "analyzer": {
                                    "virus_analyzer": {
                                        "type": "custom",
                                        "filter": ["filter1", "filter2", "filter3"],
                                        "tokenizer": "virus_tokenizer"
                                    },
                                    "md5_analyzer": {
                                        "type": "custom",
                                        "filter": ["filter1", "filter2", "filter3"],
                                        "tokenizer": "md5_tokenizer"
                                    },
                                    "url_analyzer": {
                                        "type": "custom",
                                        "filter": ["filter1", "filter2", "filter3"],
                                        "tokenizer": "url_tokenizer"
                                    },
                                    "char_analyzer": {
                                        "type": "custom",
                                        "filter": ["filter1", "filter2", "filter3"],
                                        "tokenizer": "char_tokenizer"
                                    },
                                    "virus_type_analyzer": {
                                        "type": "custom",
                                        "filter": ["filter1", "filter2", "filter3"],
                                        "tokenizer": "file_tokenizer"
                                    },
                                    "gene_analyzer": {
                                        "type": "custom",
                                        "filter": ["filter1", "filter2", "filter3"],
                                        "tokenizer": "gene_tokenizer"
                                    },
                                    "ip_analyzer": {
                                        "type": "custom",
                                        "filter": ["filter1", "filter2", "filter3"],
                                        "tokenizer": "ip_tokenizer"
                                    },
                                    "key_analyzer": {
                                        "type": "custom",
                                        "filter": ["filter2"],
                                        "tokenizer": "key_tokenizer"
                                    },
                                    "file_analyzer": {
                                        "type": "custom",
                                        "filter": ["filter1", "filter2", "filter3"],
                                        "tokenizer": "file_tokenizer"
                                    }
                                },
                                "tokenizer": {
                                    "char_tokenizer": {
                                        "group": "-1",
                                        "type": "pattern",
                                        "pattern": "[\\W+]"
                                    },
                                    "md5_tokenizer": {
                                        "pattern": "[\\W]",
                                        "group": "-1",
                                        "type": "pattern"
                                    },
                                    "file_tokenizer": {
                                        "group": "-1",
                                        "type": "pattern",
                                        "pattern": "[_\\-]"
                                    },
                                    "virus_tokenizer": {
                                        "group": "-1",
                                        "pattern": "[\\\\{\\\\}\\\\.:/&=)(\\[\\]\\s]",
                                        "type": "pattern"
                                    },
                                    "ip_tokenizer": {
                                        "pattern": "[\\\\.]",
                                        "type": "pattern",
                                        "group": "-1"
                                    },
                                    "gene_tokenizer": {
                                        "group": "-1",
                                        "pattern": "[\\\\._]",
                                        "type": "pattern"
                                    },
                                    "key_tokenizer": {
                                        "type": "keyword",
                                        "buffer_siz": "2028"
                                    },
                                    "url_tokenizer": {
                                        "type": "pattern",
                                        "group": "-1",
                                        "pattern": "[:\\\\?\\\\./&=]"
                                    }
                                }
                            },
                            "number_of_shards": "5"
                        }
                    },
                    "mappings": {
                        "ss_type": {
                            "properties": {
                                "crc32": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "sha256": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "virus_family": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "archive_info": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "virus_name": {
                                    "store": true,
                                    "analyzer": "char_analyzer",
                                    "type": "string"
                                },
                                "sha1": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "is_child_file": {
                                    "index": "not_analyzed",
                                    "store": true,
                                    "null_value": "false",
                                    "type": "boolean"
                                },
                                "is_digital_signa": {
                                    "index": "not_analyzed",
                                    "store": true,
                                    "null_value": "false",
                                    "type": "boolean"
                                },
                                "compiler_info": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "original_file_name": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "source_name": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "archive_bom": {
                                    "index": "not_analyzed",
                                    "store": true,
                                    "null_value": "false",
                                    "type": "boolean"
                                },
                                "avpscan": {
                                    "store": true,
                                    "analyzer": "char_analyzer",
                                    "type": "string"
                                },
                                "is_exe_embedded": {
                                    "index": "not_analyzed",
                                    "store": true,
                                    "null_value": "false",
                                    "type": "boolean"
                                },
                                "ssdeep": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "file_size": {
                                    "index": "not_analyzed",
                                    "store": true,
                                    "null_value": 0,
                                    "type": "integer"
                                },
                                "is_archive": {
                                    "index": "not_analyzed",
                                    "store": true,
                                    "null_value": "false",
                                    "type": "boolean"
                                },
                                "is_unknown_detection": {
                                    "index": "not_analyzed",
                                    "store": true,
                                    "null_value": "false",
                                    "type": "boolean"
                                },
                                "is_virus": {
                                    "index": "not_analyzed",
                                    "store": true,
                                    "null_value": "false",
                                    "type": "boolean"
                                },
                                "avp_virus_type": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "format_name": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "virus_platform": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "self_extract_achieve": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "self_extract_achieve_flag": {
                                    "index": "not_analyzed",
                                    "store": true,
                                    "null_value": "false",
                                    "type": "boolean"
                                },
                                "infected_judgement": {
                                    "index": "not_analyzed",
                                    "store": true,
                                    "null_value": "false",
                                    "type": "boolean"
                                },
                                "compiler_info_flag": {
                                    "index": "not_analyzed",
                                    "store": true,
                                    "null_value": "false",
                                    "type": "boolean"
                                },
                                "pack_info": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "final_result": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "md5": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "virus_action": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "is_product_version": {
                                    "index": "not_analyzed",
                                    "store": true,
                                    "null_value": "false",
                                    "type": "boolean"
                                },
                                "detectvm": {
                                    "index": "not_analyzed",
                                    "store": true,
                                    "null_value": "false",
                                    "type": "boolean"
                                },
                                "ex_time": {
                                    "index": "not_analyzed",
                                    "store": true,
                                    "format": "yyyy-MM-dd HH:mm:ss",
                                    "type": "date"
                                },
                                "is_pack": {
                                    "index": "not_analyzed",
                                    "store": true,
                                    "null_value": "false",
                                    "type": "boolean"
                                }
                            }
                        }
                    }
                },
                "ss_url_ip_dom_v1": {
                    "template": "t_relate*",
                    "order": 0,
                    "settings": {
                        "index": {
                            "number_of_replicas": "0",
                            "analysis": {
                                "filter": {
                                    "filter1": {
                                        "type": "standard"
                                    },
                                    "filter3": {
                                        "type": "stop"
                                    },
                                    "filter2": {
                                        "type": "lowercase"
                                    }
                                },
                                "analyzer": {
                                    "virus_analyzer": {
                                        "type": "custom",
                                        "filter": ["filter1", "filter2", "filter3"],
                                        "tokenizer": "virus_tokenizer"
                                    },
                                    "md5_analyzer": {
                                        "type": "custom",
                                        "filter": ["filter1", "filter2", "filter3"],
                                        "tokenizer": "md5_tokenizer"
                                    },
                                    "url_analyzer": {
                                        "type": "custom",
                                        "filter": ["filter1", "filter2", "filter3"],
                                        "tokenizer": "url_tokenizer"
                                    },
                                    "char_analyzer": {
                                        "type": "custom",
                                        "filter": ["filter1", "filter2", "filter3"],
                                        "tokenizer": "char_tokenizer"
                                    },
                                    "virus_type_analyzer": {
                                        "type": "custom",
                                        "filter": ["filter1", "filter2", "filter3"],
                                        "tokenizer": "file_tokenizer"
                                    },
                                    "gene_analyzer": {
                                        "type": "custom",
                                        "filter": ["filter1", "filter2", "filter3"],
                                        "tokenizer": "gene_tokenizer"
                                    },
                                    "ip_analyzer": {
                                        "type": "custom",
                                        "filter": ["filter1", "filter2", "filter3"],
                                        "tokenizer": "ip_tokenizer"
                                    },
                                    "key_analyzer": {
                                        "type": "custom",
                                        "filter": ["filter2"],
                                        "tokenizer": "key_tokenizer"
                                    },
                                    "file_analyzer": {
                                        "type": "custom",
                                        "filter": ["filter1", "filter2", "filter3"],
                                        "tokenizer": "file_tokenizer"
                                    }
                                },
                                "tokenizer": {
                                    "char_tokenizer": {
                                        "group": "-1",
                                        "type": "pattern",
                                        "pattern": "[\\W+]"
                                    },
                                    "md5_tokenizer": {
                                        "pattern": "[\\W]",
                                        "group": "-1",
                                        "type": "pattern"
                                    },
                                    "file_tokenizer": {
                                        "group": "-1",
                                        "type": "pattern",
                                        "pattern": "[_\\-]"
                                    },
                                    "virus_tokenizer": {
                                        "group": "-1",
                                        "pattern": "[\\\\{\\\\}\\\\.:/&=)(\\[\\]\\s]",
                                        "type": "pattern"
                                    },
                                    "ip_tokenizer": {
                                        "pattern": "[\\\\.]",
                                        "type": "pattern",
                                        "group": "-1"
                                    },
                                    "gene_tokenizer": {
                                        "group": "-1",
                                        "pattern": "[\\\\._]",
                                        "type": "pattern"
                                    },
                                    "key_tokenizer": {
                                        "type": "keyword",
                                        "buffer_siz": "2028"
                                    },
                                    "url_tokenizer": {
                                        "type": "pattern",
                                        "group": "-1",
                                        "pattern": "[:\\\\?\\\\./&=]"
                                    }
                                }
                            },
                            "number_of_shards": "5"
                        }
                    },
                    "mappings": {
                        "es_source_data": {
                            "properties": {
                                "t_domain": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "crc32": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                },
                                "source": {
                                    "store": true,
                                    "null_value": "yq-system",
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "dst_ip": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "md5": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "virus_name": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                },
                                "family_name": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                },
                                "s_port": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                },
                                "src_ip": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "up_time": {
                                    "index": "not_analyzed",
                                    "store": true,
                                    "format": "yyyy-MM-dd HH:mm:ss",
                                    "type": "date"
                                },
                                "capture_province": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                },
                                "d_port": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                },
                                "url": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                }
                            }
                        },
                        "tip_url": {
                            "properties": {
                                "tip_url": {
                                    "store": true,
                                    "null_value": "yq-system",
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                }
                            }
                        },
                        "t_url": {
                            "properties": {
                                "crc32": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                },
                                "source": {
                                    "store": true,
                                    "null_value": "t-system",
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "md5": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "method": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                },
                                "ex_time": {
                                    "index": "not_analyzed",
                                    "store": true,
                                    "format": "yyyy-MM-dd HH:mm:ss",
                                    "type": "date"
                                },
                                "url": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                }
                            }
                        },
                        "t_shunt_virus_event": {
                            "properties": {
                                "t_domain": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "crc32": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                },
                                "probe_id": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                },
                                "t_virus_name": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                },
                                "dst_ip": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "r_time": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                },
                                "up_time": {
                                    "index": "not_analyzed",
                                    "store": true,
                                    "format": "yyyy-MM-dd HH:mm:ss",
                                    "type": "date"
                                },
                                "virus_name": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                },
                                "t_virus_type": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                },
                                "proto": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                },
                                "url": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "source": {
                                    "store": true,
                                    "null_value": "1xx-system",
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "from_log": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                },
                                "md5": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "filename": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                },
                                "src_ip": {
                                    "store": true,
                                    "analyzer": "key_analyzer",
                                    "type": "string"
                                },
                                "s_port": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                },
                                "family_name": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                },
                                "ex_time": {
                                    "index": "not_analyzed",
                                    "store": true,
                                    "type": "string"
                                },
                                "d_port": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                },
                                "log_type": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                }
                            }
                        },
                        "t_ip": {
                            "properties": {
                                "port": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                },
                                "crc32": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                },
                                "locat": {
                                    "index": "no",
                                    "store": true,
                                    "type": "string"
                                },
                                "source": {
                                    "store": true 
 

            
  
           

              
              
            
            
相關推薦
			   
            
            
            
 

    

    
    
解決elasticsearch主分片unassigned的問題
     
 
							
							
							[[email protected] ~]# sh -x /tmp/es.sh 
+ curl -XPOST localhost:9500/_cluster/reroute -d ‘{ 
    “commands”:[ 
        { 
    

  
 

    

    
    
elasticsearch主分片數量為什麼不可以更改?
     
 
                                        
                                                當索引一個文件的時候，文件會被儲存到一個主分片中。 Elasticsearch 如何知道一個文件應該存放到哪個分片中呢？ 

  
 

    

    
    
Elasticsearch yellow 意味著主分片可用，副本不可用
     
 sta   stat   通過   init   兩個   nds   art   ini   nasm    
摘自：http://unasm.com/2016/11/644/

在通過 /_cluster/state 命令查看es 狀態的時候，發現es 處於一個yellow的狀態， 這個很奇怪，按照官方的 

  
 

    

    
    
ElasticSearch資料分片-資料路由
     
  
 
 ElasticSearch(簡稱ES)：是一個基於Lucene構建的開源、分散式、RESTful的全文字搜尋引擎；它還是一個分散式實時文件儲存，其中每個field均是被索引的資料且可被搜尋；也是一個帶實時分析功能的分散式搜尋引擎，並且能夠擴充套件至數以百計的伺服器儲存及處理PB級的資料。 
 &nb 

  
 

    

    
    
ElasticSearch教程——分片
     
 
                primary shard 和 replica shard機制

(1):index包含多個shard；

(2):每個shard都是一個最小的工作單元，承載部分的資料，Lucene例項，完整的簡歷索引和處理請求的能力；

(3):增減節點時，shard會自動在nodes中負 

  
 

    

    
    
解決elasticsearch：Exception BindTransportException[Failed to bind to [9300-9400]]
     
 
							
							
							Linux7.4使用elasticsearch2.2.1啟動報錯：
[2018-09-20 23:24:17,887][INFO ][node                     ] [Paibo] starting ...
Exception in thr 

  
 

    

    
    
大資料求索（10）： 解決ElasticSearch中文搜尋無結果------ik分詞器的安裝與使用
     
  
 
  
  
 大資料求索（10）： 解決ElasticSearch中文搜尋無結果-----IK中文分詞器的安裝與使用 
 問題所在 
 在中文情況下，ES預設分詞器會將漢字切分為一個一個的漢字，所以當搜尋詞語的時候，會發現無法找到結果。 
 解決辦法 
 有很多其他的分詞器外掛可以替代，這裡使用最常用 

  
 

    

    
    
Elasticsearch推遲分片分配
     
 
                
參看網址：https://www.elastic.co/guide/cn/elasticsearch/guide/current/_delaying_shard_allocation.html
適用情況：對於一些網路不穩定的情況比較適用（可能臨時有節點脫離叢集，然後又回來。 

  
 

    

    
    
MySQL死鎖問題例項分析及解決方法(主要是SQL語句可能會產生的問題)
     
 
                
from:
http://database.51cto.com/art/201108/286325.htm


MySQL死鎖問題的相關知識是本文我們主要要介紹的內容，接下來我們就來一一介紹這部分內容，希望能夠對您有所幫助。
1、MySQL常用儲存引擎的鎖機制
MyISAM 

  
 

    

    
    
解決Elasticsearch問題的一些心得體會
     
 　　在開始前先來介紹下背景：我的日誌採集系統採用ELK（logstash（收集）、elasticsearch（儲存+搜尋）、kibana（展示）三個軟體的簡稱）開源架構，在elasticsearch搭建了三臺Linux節點，IP地址分別為：10.0.48.108，10.0.48.97，10.0.48.98，e 

  
 

    

    
    
通過maven-shade-plugin 解決Elasticsearch與hbase的jar包衝突問題
     
 
                
今天在專案中整合hbase1.0.0-cdh5.5.2和ElasticSearch2.2.0時出現了問題,原先在整合hbase時，引入了架包com.google.guava(16.0版本)，後來在整合ES時發現es中也用到了guava的18.0版本。這時候問題來了，如果將原 

  
 

    

    
    
物理機解決ElasticSearch找不到JDK問題(Could not find any executable java binary)
     
 
                背景：

    本地啟動ElasticSearch服務提示找不到JDK或JRE，提示Could not find any executable java binary. Please install java in your PATH or set JAVA_HOME.

 

  
 

    

    
    
ElasticSearch內部分片處理機制
     
 
							
							
							逆向索引：

與傳統的資料庫不同，在es中，每個欄位裡面的每個單詞都是可以被搜尋的。如hobby:”dance,sing,swim,run”,我們在搜尋關鍵字swim時，所有包含swim的文件都會被匹配到，es的這個特性也叫做全文搜尋。

為了支援這個特性，es 

  
 

    

    
    
PeekMessage完美解決MFC主介面無響應
     
     PeekMessage在檢索應用程式的訊息佇列時，如果其中有訊息就將其放入lpMsg（如下所示的函式原型的宣告中）所指的結構中，同時PeekMessage函式不會等到有訊息放入佇列時才返回。同樣，如果hWnd為NULL，則PeekMessage檢索物件為程序的訊息佇列；如果hWnd = -1，那麼函式 

  
 

    

    
    
解決Elasticsearch查詢出現Data too large……情況
     
 
                Elasticsearch在實際生產環境中提供實時查詢，出現Data too large, data for [src_ip] would be larger than limit……     經過一番查詢，才發現粗心的同事在啟動es的時候忘記設定堆記憶體（ES_HEAP_S 

  
 

    

    
    
解決在主執行緒不能訪問網路的問題
     
 
								
								            
						
                
             在android4.0及以後版本的開發中，如果在android的主執行緒中進行網路訪問，會報錯。
解決該問題的方法：
1.在oncreate()方法中新增這段程式碼，即可在主 

  
 

    

    
    
elasticsearch ：unassigned錯誤解決
     
 
								
								            
						
                elasticsearch 重啟後出現了elasticsearch Unassigned 異常

出現這個問題的原因是表明該資料分片的異常，那麼需要重啟這個資料分片



1、首先走命令列，檢視目前有哪 

  
 

    

    
    
如何在Elasticsearch中解析未分配的分片（unassigned shards）
     
 size   啟用   空間   isa   out   concepts   perm   初始化   hot   一、精確定位到有問題的shards
1、查看哪些分片未被分配

 curl -XGET localhost:9200/_cat/shards?h=index,shard,prirep,s 

  
 

    

    
    
Elasticsearch 5常見問題解決方案
     
 elasticsearch-head   elasticsearch   grunt   node   bootstrap   npm   安裝運行1、前置安裝java8jdk-8u112-linux-x64.rpm下載地址：http://www.oracle.com/technetwork/java/jav 

  
 

    

    
    
maven項目對於測試時“無法加載主類”的解決方案
     
 logs   color   alt   blog   con   config   ima   uil   保存   1、右鍵maven項目，選擇  build path --------->Configure Build Path  ，執行下列操作，保存即可。



 maven項目對於測試時“無法