搜尋學習基礎--倒排索引的過程解讀
阿新 • • 發佈:2019-01-07
下面是兩篇文件,我們要對其建立索引
Doc1: He is a coder,and she is a coder too.
Doc2:Json is a doctor,but he was a coder.
第一步:獲取關鍵詞keywords
a:分詞(按空格分詞):
Doc1: [He] [is] [a] [coder],[and] [she] [is] [a] [coder] [too]
Doc2: [Json] [is] [a] [doctor] [he] [a] [coder]
b:去除 stopwords(無意義的關鍵詞)
Doc1:[He] [a] [coder] [she] [a] [coder]
Doc2:[Json] [is] [a] [doctor] [he] [a] [coder]
c:統一(大小寫,時態)
Doc1:[he] [a] [coder] [she] [a] [coder]
Doc2:[json] [a] [doctor] [he] [a] [coder]
第二步:建立倒排索引
//關鍵詞出現的文章
keywords | doc |
---|---|
[he] | 1,2 |
[a] | 1,2 |
[coder] | 1,2 |
[she] | 1 |
[json] | 2 |
[doctor] | 2 |
//更好的結構 記錄關鍵詞出現的文章,出現頻率(對結果排序),出現位置(使用者快速鎖定高亮位置)
keywords | doc[times] | doc[index] |
---|---|---|
[he] | 1[1],2[1] | 1[1],2[4] |
[a] | 1[2],2[2] | 1[2,5],2[2,5] |
[coder] | 1[2],2[1] | 1[3,6],2[6] |
[she] | 1[1] | 1[4] |
[json] | 2[1] | 2[1] |
[doctor] | 2[1] | 2[3] |
第三步:搜尋
a.輸入搜尋語句: doctor and coder
b.獲取a中的關鍵詞,得到[doctor] [coder]
c.從索引表中得到:[coder]在Doc1中出現2次,在Doc2中出現1次,[doctor] 在Doc2中出現1次。
d.由此可以得到Doc2因為關聯兩個關鍵詞,關聯性更高(如果關聯關鍵詞數量一次,則可以根據出現頻率排序),搜尋出的結果順序為:Doc2,Doc1