搜尋系統開發

阿新 • • 發佈：2019-02-02

一、搭建maven工程基本框架

1、匯入依賴

pom.xml檔案內容如下：

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent> 

        <groupId>com.enjoyshop.parent</groupId>
        <artifactId>enjoyshop-parent</artifactId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>
    <groupId>com.enjoyshop.search</groupId>
    <artifactId>enjoyshop-search</artifactId> 

    <version>1.0.0-SNAPSHOT</version>
    <packaging>war</packaging>

    <dependencies>
        <dependency>
            <groupId>com.enjoyshop.common</groupId>
            <artifactId>enjoyshop-common</artifactId>
            <version>1.0.0-SNAPSHOT</version 
>
        </dependency>
        <!-- 單元測試 -->
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-webmvc</artifactId>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-log4j12</artifactId>
        </dependency>
        <!-- Jackson Json處理工具包 -->
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
        </dependency>
        <!-- httpclient -->
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
        </dependency>
        <!-- JSP相關 -->
        <dependency>
            <groupId>jstl</groupId>
            <artifactId>jstl</artifactId>
        </dependency>
        <dependency>
            <groupId>javax.servlet</groupId>
            <artifactId>servlet-api</artifactId>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>javax.servlet</groupId>
            <artifactId>jsp-api</artifactId>
            <scope>provided</scope>
        </dependency>
        <!-- Apache工具元件 -->
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-io</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.solr</groupId>
            <artifactId>solr-solrj</artifactId>
            <version>4.10.2</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.amqp</groupId>
            <artifactId>spring-rabbit</artifactId>
            <version>1.4.0.RELEASE</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <!-- 配置Tomcat外掛 -->
            <plugin>
                <groupId>org.apache.tomcat.maven</groupId>
                <artifactId>tomcat7-maven-plugin</artifactId>
                <configuration>
                    <port>8085</port>
                    <path>/</path>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

2、配置web.xml

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns="http://java.sun.com/xml/ns/javaee"
    xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
    id="WebApp_ID" version="2.5">
    <display-name>enjoyshop-search</display-name>

    <context-param>
        <param-name>contextConfigLocation</param-name>
        <param-value>classpath:spring/applicationContext*.xml</param-value>
    </context-param>

    <!--Spring的ApplicationContext 載入 -->
    <listener>
        <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
    </listener>

    <!-- 編碼過濾器，以UTF8編碼 -->
    <filter>
        <filter-name>encodingFilter</filter-name>
        <filter-class>org.springframework.web.filter.CharacterEncodingFilter</filter-class>
        <init-param>
            <param-name>encoding</param-name>
            <param-value>UTF8</param-value>
        </init-param>
    </filter>
    <filter-mapping>
        <filter-name>encodingFilter</filter-name>
        <url-pattern>/*</url-pattern>
    </filter-mapping>

    <!-- 配置SpringMVC框架入口 -->
    <servlet>
        <servlet-name>enjoyshop-search</servlet-name>
        <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
        <init-param>
            <param-name>contextConfigLocation</param-name>
            <param-value>classpath:spring/enjoyshop-search-servlet.xml</param-value>
        </init-param>
        <load-on-startup>1</load-on-startup>
    </servlet>

    <servlet-mapping>
        <servlet-name>enjoyshop-search</servlet-name>
        <!-- 偽靜態，好處：有利於SEO（搜尋引擎優化） -->
        <url-pattern>*.html</url-pattern>
    </servlet-mapping>

    <welcome-file-list>
        <welcome-file>index.html</welcome-file>
    </welcome-file-list>

</web-app>

3、使用Spring整合Solrj

本專案使用solr來實現搜尋功能，並且使用其java客戶端Solrj來實現專案整合。
關於Solr的簡介和使用請參考這裡。點我
關於Solrj的使用請參考這裡。點我

Spring專案整合Solrj的配置如下：

<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:context="http://www.springframework.org/schema/context" xmlns:p="http://www.springframework.org/schema/p"
    xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.0.xsd
    http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-4.0.xsd
    http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-4.0.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-4.0.xsd
    http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-4.0.xsd">

    <bean class="org.apache.solr.client.solrj.impl.HttpSolrServer">
        <constructor-arg index="0" value="${solr.url}"/>
        <!-- 設定響應解析器 -->
        <property name="parser">
            <bean class="org.apache.solr.client.solrj.impl.XMLResponseParser"/>
        </property>
        <!-- 重試次數 -->
        <property name="maxRetries" value="${solr.maxRetries}"/>
        <property name="connectionTimeout" value="${solr.connectionTimeout}"/>
    </bean>

</beans>

4、配置host和nginx

這裡不做詳述，可參考之前的配置。

5、在Solr中建立enjoyshop core

建立enjoyshop core的步驟如下：
1、在example目錄下建立enjoyshop-solr資料夾；
2、將example/solr下的solr.xml拷貝到enjoyshop-solr目錄下；
3、在enjoyshop-solr下建立enjoyshop目錄，並且在enjoyshop目錄下建立conf和data目錄；
4、將example\solr\collection1\core.properties檔案拷貝到example\enjoyshop-solr\enjoyshop下，並且修改name=enjoyshop；

5、將example\solr\collection1\conf下的schema.xml、solrconfig.xml拷貝到example\enjoyshop-solr\enjoyshop\conf下；
6、修改schema.xml檔案，使其配置最小化：

<?xml version="1.0" encoding="UTF-8" ?>


<schema name="example" version="1.5">

   <field name="_version_" type="long" indexed="true" stored="true"/>

   <field name="_root_" type="string" indexed="true" stored="false"/>

   <field name="id" type="long" indexed="true" stored="true" required="true" multiValued="false" /> 
   <field name="title" type="text_ik" indexed="true" stored="true"/>    
   <field name="sellPoint" type="string" indexed="false" stored="true"/> 
   <field name="price" type="long" indexed="true" stored="true"/> 
   <field name="image" type="string" indexed="false" stored="true"/>  
   <field name="cid" type="long" indexed="true" stored="true"/>   
   <field name="status" type="int" indexed="true" stored="false"/>
   <field name="updated" type="long" indexed="true" stored="true"/>   
 <uniqueKey>id</uniqueKey>
    <fieldType name="string" class="solr.StrField" sortMissingLast="true" />
    <fieldType name="int" class="solr.TrieIntField" precisionStep="0" positionIncrementGap="0"/>
    <fieldType name="long" class="solr.TrieLongField" precisionStep="0" positionIncrementGap="0"/>
    <fieldType name="text_ik" class="solr.TextField">   
     <analyzer class="org.wltea.analyzer.lucene.IKAnalyzer"/>   
    </fieldType>
   <solrQueryParser defaultOperator="AND"/>

</schema>

7、修改solrconfig.xml檔案，修改一些配置，大部分配置先保持預設：
a) 將所有的標籤註釋掉；
b) 搜尋<str name="df">text</str>並將其替換成<str name="df">title</str>
c) 將<searchComponent name="elevator" class="solr.QueryElevationComponent" >註釋掉（這個的功能類似百度的競價排名）；
8、整合IK分詞器
a) 將IKAnalyzer-2012-4x.jar拷貝到example\solr-webapp\webapp\WEB-INF\lib下；
b) 在schema.xml檔案中新增fieldType：

<fieldType name="text_ik" class="solr.TextField">   
     <analyzer class="org.wltea.analyzer.lucene.IKAnalyzer"/>   
</fieldType>

9、啟動solr：
java -Dsolr.solr.home=enjoyshop-solr -jar start.jar

6、匯入商品資料到solr中

首先需要啟動後臺系統，通過httpclient從後臺系統的對外介面中獲得商品資料。通過執行如下程式碼即可匯入商品資訊。

public class ItemDataImportTest {

    private HttpSolrServer httpSolrServer;

    private static final ObjectMapper MAPPER = new ObjectMapper();

    @Before
    public void setUp() throws Exception {
        // 在url中指定core名稱：enjoyshop
        // http://solr.enjoyshop.com/#/enjoyshop -- 介面操作
        String url = "http://solr.enjoyshop.com/enjoyshop"; // 服務地址
        HttpSolrServer httpSolrServer = new HttpSolrServer(url); // 定義solr的server
        httpSolrServer.setParser(new XMLResponseParser()); // 設定響應解析器
        httpSolrServer.setMaxRetries(1); // 設定重試次數，推薦設定為1
        httpSolrServer.setConnectionTimeout(500); // 建立連線的最長時間

        this.httpSolrServer = httpSolrServer;
    }

    @Test
    public void testData() throws Exception {
        // 通過後臺系統的介面查詢商品資料
        String url = "http://manage.enjoyshop.com/rest/item?page={page}&rows=100";
        int page = 1;
        int pageSzie = 0;
        do {
            String u = StringUtils.replace(url, "{page}", "" + page);
            System.out.println(u);
            String jsonData = doGet(u);
            JsonNode jsonNode = MAPPER.readTree(jsonData);
            String rowsStr = jsonNode.get("rows").toString();
            List<Item> items = MAPPER.readValue(rowsStr,
                    MAPPER.getTypeFactory().constructCollectionType(List.class, Item.class));
            pageSzie = items.size();
            this.httpSolrServer.addBeans(items);
            this.httpSolrServer.commit();

            page++;
        } while (pageSzie == 100);

    }

    private String doGet(String url) throws Exception {
        // 建立Httpclient物件
        CloseableHttpClient httpclient = HttpClients.createDefault();

        // 建立http GET請求
        HttpGet httpGet = new HttpGet(url);

        CloseableHttpResponse response = null;
        try {
            // 執行請求
            response = httpclient.execute(httpGet);
            // 判斷返回狀態是否為200
            if (response.getStatusLine().getStatusCode() == 200) {
                return EntityUtils.toString(response.getEntity(), "UTF-8");
            }
        } finally {
            if (response != null) {
                response.close();
            }
            httpclient.close();
        }
        return null;
    }

二、業務邏輯實現

1、搜尋功能的實現

service層

package com.enjoyshop.search.service;

import java.util.List;
import java.util.Map;

import org.apache.commons.lang3.StringUtils;
import org.apache.solr.client.solrj.SolrQuery;
import org.apache.solr.client.solrj.impl.HttpSolrServer;
import org.apache.solr.client.solrj.response.QueryResponse;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import com.enjoyshop.search.bean.SearchResult;
import com.enjoyshop.search.pojo.Item;

@Service
public class SearchService {

    public static final Integer ROWS = 32;

    @Autowired
    private HttpSolrServer httpSolrServer;

    public SearchResult search(String keyWords, Integer page) throws Exception {
        SolrQuery solrQuery = new SolrQuery(); // 構造搜尋條件
        solrQuery.setQuery("title:" + keyWords + " AND status:1"); // 搜尋關鍵詞
        // 設定分頁 start=0就是從0開始，，rows=5當前返回5條記錄，第二頁就是變化start這個值為5就可以了。
        solrQuery.setStart((Math.max(page, 1) - 1) * ROWS);
        solrQuery.setRows(ROWS);

        // 是否需要高亮
        boolean isHighlighting = !StringUtils.equals("*", keyWords) && StringUtils.isNotEmpty(keyWords);

        if (isHighlighting) {
            // 設定高亮
            solrQuery.setHighlight(true); // 開啟高亮元件
            solrQuery.addHighlightField("title");// 高亮欄位
            solrQuery.setHighlightSimplePre("<em>");// 標記，高亮關鍵字字首
            solrQuery.setHighlightSimplePost("</em>");// 字尾
        }

        // 執行查詢
        QueryResponse queryResponse = this.httpSolrServer.query(solrQuery);
        List<Item> items = queryResponse.getBeans(Item.class);
        if (isHighlighting) {
            // 將高亮的標題資料寫回到資料物件中
            Map<String, Map<String, List<String>>> map = queryResponse.getHighlighting();
            for (Map.Entry<String, Map<String, List<String>>> highlighting : map.entrySet()) {
                for (Item item : items) {
                    if (!highlighting.getKey().equals(item.getId().toString())) {
                        continue;
                    }
                    item.setTitle(StringUtils.join(highlighting.getValue().get("title"), ""));
                    break;
                }
            }
        }
        return new SearchResult(queryResponse.getResults().getNumFound(), items);
    }

}

controller層

package com.enjoyshop.search.controller;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.servlet.ModelAndView;

import com.enjoyshop.search.bean.SearchResult;
import com.enjoyshop.search.service.SearchService;

@Controller
public class SearchController {

    @Autowired
    private SearchService searchService;

    @RequestMapping(value = "search", method = RequestMethod.GET)
    public ModelAndView search(@RequestParam("q") String keyWords,
            @RequestParam(value = "page", defaultValue = "1") Integer page) {
        ModelAndView mv = new ModelAndView("search");
        try {
                //解決中文亂碼問題
                keyWords=new String(keyWords.getBytes("ISO-8859-1"),"UTF-8");
            //搜尋結果
                SearchResult searchResult = this.searchService.search(keyWords, page);
            //搜尋關鍵字
                mv.addObject("query", keyWords);
            //商品列表
                mv.addObject("itemList", searchResult.getData());
            //當前頁
                mv.addObject("page", page);
            //計算總頁數
                int total = searchResult.getTotal().intValue();
            int pages = total % searchService.ROWS == 0 ? total / searchService.ROWS : total / searchService.ROWS + 1;
            mv.addObject("pages", pages);
        } catch (Exception e) {
            e.printStackTrace();
        }
        return mv;
    }
}

2、利用RabbitMQ接收後臺的訊息

使用Spring配置RabbitMQ

外部屬性檔案如下：

rabbitmq.host=127.0.0.1
rabbitmq.port=5672
rabbitmq.username=enjoyshop
rabbitmq.password=enjoyshop
rabbitmq.vhost=/enjoyshop

XML配置檔案如下：

<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:rabbit="http://www.springframework.org/schema/rabbit"
    xsi:schemaLocation="http://www.springframework.org/schema/rabbit
    http://www.springframework.org/schema/rabbit/spring-rabbit-1.4.xsd
    http://www.springframework.org/schema/beans
    http://www.springframework.org/schema/beans/spring-beans-4.0.xsd">

    <!-- 定義RabbitMQ的連線工廠 -->
    <rabbit:connection-factory id="connectionFactory"
        host="${rabbitmq.host}" port="${rabbitmq.port}" username="${rabbitmq.username}" password="${rabbitmq.password}"
        virtual-host="${rabbitmq.vhost}" />

    <!-- 定義管理 -->
    <rabbit:admin connection-factory="connectionFactory"/>

    <!-- 定義佇列 -->
    <rabbit:queue name="enjoyshop-search-item-queue" durable="true" auto-declare="true"/>

    <!-- 定義消費者 -->
    <bean id="itemMQHandler" class="com.enjoyshop.search.mq.handler.ItemMQHandler"/>

    <!-- 消費者監聽佇列 -->
    <rabbit:listener-container connection-factory="connectionFactory">
        <rabbit:listener ref="itemMQHandler" method="execute" queue-names="enjoyshop-search-item-queue"/>
    </rabbit:listener-container>

</beans>

訊息處理方法

package com.enjoyshop.search.mq.handler;

import org.apache.commons.lang3.StringUtils;
import org.apache.solr.client.solrj.impl.HttpSolrServer;
import org.springframework.beans.factory.annotation.Autowired;

import com.enjoyshop.search.pojo.Item;
import com.enjoyshop.search.service.ItemService;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;

public class ItemMQHandler {

    private static final ObjectMapper MAPPER = new ObjectMapper();

    @Autowired
    private HttpSolrServer httpSolrServer;

    @Autowired
    private ItemService itemService;

    public void execute(String msg) {
        try {
            JsonNode jsonNode = MAPPER.readTree(msg);
            Long itemId = jsonNode.get("itemId").asLong();
            String type = jsonNode.get("type").asText();
            if (StringUtils.equals(type, "insert") || StringUtils.equals(type, "update")) {
                // 從後臺系統中查詢商品資料
                Item item = this.itemService.queryItemById(itemId);
                if (item != null) {
                    this.httpSolrServer.addBean(item);
                    this.httpSolrServer.commit();
                }
            } else if (StringUtils.equals(type, "delete")) {
                // 刪除索引資料
                this.httpSolrServer.deleteById(String.valueOf(itemId));
                this.httpSolrServer.commit();
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

}

搜尋系統開發

一、搭建maven工程基本框架

1、匯入依賴

2、配置web.xml

3、使用Spring整合Solrj

4、配置host和nginx

5、在Solr中建立enjoyshop core

6、匯入商品資料到solr中

二、業務邏輯實現

1、搜尋功能的實現

2、利用RabbitMQ接收後臺的訊息

搜尋系統開發

ElasticSearch + Canal 開發千萬級的實時搜尋系統【轉】

ElasticSearch + Canal 開發千萬級的實時搜尋系統

百鳥商城系統開發模式設計詳解

斑布生活家系統開發詳解模式設計

工匠家模式系統開發滴滴打車平臺

工匠家平臺系統開發類似滴滴打車模式app詳解

RDIFramework.NET ━ .NET快速信息化系統開發框架 V3.2->新增“行政區域管理”，同時大批量樹采用異步加載

拆分盤系統開發

直播系統開發軟件APP

萌店系統開發模式系統詳解

共享充電寶系統開發軟件app

微信商城直播系統開發

餐飲小程序系統開發APP

小程序商城系統開發模式平臺搭建詳解

茶葉防偽標簽制作系統開發方案策劃

二維碼防偽標簽制作印刷防偽系統開發

外賣點餐系統開發平臺app

系統開發之設計模式

快消品營銷系統-開發與應用-好處

搜尋系統開發

一、搭建maven工程基本框架

1、匯入依賴

2、配置web.xml

3、使用Spring整合Solrj

4、配置host和nginx

5、在Solr中建立enjoyshop core

6、匯入商品資料到solr中

二、業務邏輯實現

1、搜尋功能的實現

2、利用RabbitMQ接收後臺的訊息

相關推薦