1. 程式人生 > >使用solr建立 附件[word pdf txt等檔案索引]

使用solr建立 附件[word pdf txt等檔案索引]

官方給出的ContentStreamUpdateRequest樣例:
1 package javaapplicationsolrcell;
   2 
   3 import java.io.File;
   4 import java.io.IOException;
   5 import org.apache.solr.client.solrj.SolrServer;
   6 import org.apache.solr.client.solrj.SolrServerException;
   7 
   8 import org.apache.solr.client.solrj.request.AbstractUpdateRequest
;
9 import org.apache.solr.client.solrj.response.QueryResponse; 10 import org.apache.solr.client.solrj.SolrQuery; 11 import org.apache.solr.client.solrj.impl.CommonsHttpSolrServer; 12 import org.apache.solr.client.solrj.request.ContentStreamUpdateRequest; 13 14 /** 15 * @author EDaniel
16 */ 17 public class SolrExampleTests { 18 19 public static void main(String[] args) { 20 try { 21 //Solr cell can also index MS file (2003 version and 2007 version) types. 22 String fileName = "c:/Sample.pdf"; 23 //this will be unique Id used by Solr to index the file contents.
24 String solrId = "Sample.pdf"; 25 26 indexFilesSolrCell(fileName, solrId); 27 28 } catch (Exception ex) { 29 System.out.println(ex.toString()); 30 } 31 } 32 33 /** 34 * Method to index all types of files into Solr. 35 * @param fileName 36 * @param solrId 37 * @throws IOException 38 * @throws SolrServerException 39 */ 40 public static void indexFilesSolrCell(String fileName, String solrId) 41 throws IOException, SolrServerException { 42 43 String urlString = "http://localhost:8983/solr"; 44 SolrServer solr = new CommonsHttpSolrServer(urlString); 45 46 ContentStreamUpdateRequest up 47 = new ContentStreamUpdateRequest("/update/extract"); 48 49 up.addFile(new File(fileName)); 50 51 up.setParam("literal.id", solrId); 52 up.setParam("uprefix", "attr_"); 53 up.setParam("fmap.content", "attr_content"); 54 55 up.setAction(AbstractUpdateRequest.ACTION.COMMIT, true, true); 56 57 solr.request(up); 58 59 QueryResponse rsp = solr.query(new SolrQuery("*:*")); 60 61 System.out.println(rsp); 62 } 63 }
通過以上方法可以看出給solr上傳檔案使用ContentStreamUpdateRequest 封裝請求物件,利用
solr.request(up) 實現post請求.
literal 設定的id 是你的schema.xml 中key值 對應的 solrId為其value值,
在schema.xml 中 配置 <field name="id" type="string" indexed="true" stored="true" required="false" multiValued="false" /> 
官網請求引數詳細介紹如下:
Input Parameters
  • fmap.<source_field>=<target_field> - Maps (moves) one field name to another. Example: fmap.content=text will cause the content field normally generated by Tika to be moved to the "text" field.

  • boost.<fieldname>=<float> - Boost the specified field.

  • literal.<fieldname>=<value> - Create a field with the specified value. May be multivalued if the Field is multivalued.

  • uprefix=<prefix> - Prefix all fields that are not defined in the schema with the given prefix. This is very useful when combined with dynamic field definitions. Example: uprefix=ignored_ would effectively ignore all unknown fields generated by Tika given the example schema contains <dynamicField name="ignored_*" type="ignored"/>

  • defaultField=<Field Name> - If uprefix is not specified and a Field cannot be determined, the default field will be used.

  • extractOnly=true|false - Default is false. If true, return the extracted content from Tika without indexing the document. This literally includes the extracted XHTML as a string in the response. When viewing manually, it may be useful to use a response format other than XML to aid in viewing the embedded XHTML tags. See TikaExtractOnlyExampleOutput.

  • resource.name=<File Name> - The optional name of the file. Tika can use it as a hint for detecting mime type.

  • capture=<Tika XHTML NAME> - Capture XHTML elements with the name separately for adding to the Solr document. This can be useful for grabbing chunks of the XHTML into a separate field. For instance, it could be used to grab paragraphs (<p>) and index them into a separate field. Note that content is also still captured into the overall "content" field.

  • captureAttr=true|false - Index attributes of the Tika XHTML elements into separate fields, named after the element. For example, when extracting from HTML, Tika can return the href attributes in <a> tags as fields named "a". See the examples below.

  • xpath=<XPath expression> - When extracting, only return Tika XHTML content that satisfies the XPath expression. See http://tika.apache.org/1.2/parser.html for details on the format of Tika XHTML. See also TikaExtractOnlyExampleOutput.

  • lowernames=true|false - Map all field names to lowercase with underscores. For example, Content-Type would be mapped to content_type.
  • literalsOverride=true|false - <!> Solr4.0 When true, literal field values will override other values with same field name, such as metadata and content. If false, then literal field values will be appended to any extracted data from Tika, and the resulting field needs to be multi valued. Default: true

  • resource.password=<password> - <!> Solr4.0 The optional password for a password protected PDF or OOXML file. File format support depends on Tika.

  • passwordsFile=<file name> - <!> Solr4.0 The optional name of a file containing file name pattern to password mappings. See chapter "Encrypted Files" below

If extractOnly is true, additional input parameters:

  • extractFormat=xml|text - Default is xml. Controls the serialization format of the extract content. xml format is actually XHTML, like passing the -x command to the tika command line application, while text is like the -t command.

Order of field operations

  1. fields are generated by Tika or passed in as literals via literal.fieldname=value<!> Before Solr4.0 or if literalsOverride=false, then literals will be appended as multi-value to tika generated field.

  2. if lowernames==true, fields are mapped to lower case
  3. mapping rules fmap.source=target are applied

  4. if uprefix is specified, any unknown field names are prefixed with that value, else if defaultField is specified, unknown fields are copied to that.

程式碼中存在以上實現還不夠,
此請求到/update/extract 這個請求處理器在solrconfig.xml中必須有相應的配置
  <lib dir="${solr.install.dir:../../../..}/contrib/extraction/lib" regex=".*\.jar" />
  <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-cell-\d.*\.jar" />
 <requestHandler name="/update/extract" 
                  startup="lazy"
                  class="solr.extraction.ExtractingRequestHandler" >
    <lst name="defaults">
      <str name="lowernames">true</str>
      <str name="uprefix">ignored_</str>


      <!-- capture link hrefs but ignore div attributes -->
      <str name="captureAttr">true</str>
      <str name="fmap.a">links</str>
      <str name="fmap.div">ignored_</str>
    </lst>
  </requestHandler>
且在solr的lib下存放
contrib/extraction/lib 及
solr-cell-的jar包