說明：配置文件中以“#”開頭的行為註釋，以“-" 開頭的表示符合正則表達式就過濾掉，以“+”開頭的表示符合正則表達式則保留。正則表達式中"^"表示字符串的開頭，"$"表示字符串的結尾，"[]"表示集合。中文部分是我添加的註釋

1. # Licensed to the Apache Software Foundation (ASF) under one or more
2. # contributor license agreements. See the NOTICE file distributed with
3. # this work for additional information regarding copyright ownership.
4. # The ASF licenses this file to You under the Apache License, Version 2.0
5. # (the "License"); you may not use this file except in compliance with
6. # the License. You may obtain a copy of the License at
7. #
8. # http://www.apache.org/licenses/LICENSE-2.0
9. #
10. # Unless required by applicable law or agreed to in writing, software
11. # distributed under the License is distributed on an "AS IS" BASIS,
12. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13. # See the License for the specific language governing permissions and
14. # limitations under the License.
17. # The default url filter.
18. # Better for whole-internet crawling.
20. # Each non-comment, non-blank line contains a regular expression
21. # prefixed by ‘+‘ or ‘-‘. The first matching pattern in the file
22. # determines whether a URL is included or ignored. If no pattern
23. # matches, the URL is ignored.
25. # skip file: ftp: and mailto: urls
26. #過濾掉file：ftp等不是html協議的鏈接
27. -^(file|ftp|mailto):
29. # skip image and other suffixes we can‘t yet parse
30. #過濾掉圖片等格式的鏈接
31. -\.(gif|GIF|jpg|JPG|png|PNG|ico|ICO|css|sit|eps|wmf|zip|ppt|mpg|xls|gz|rpm|tgz|mov|MOV|exe|jpeg|JPEG|bmp|BMP)$
33. # skip URLs containing certain characters as probable queries, etc.
34. #-[?*!@=] 過濾掉汗特殊字符的鏈接，因為要爬取更多的鏈接，所以修改過濾條件，使包含？=的鏈接不被過濾掉
35. -[*!@]
37. # skip URLs with slash-delimited segment that repeats 3+ times, to break loops
38. #過濾掉一些特殊格式的鏈接
39. -.*(/[^/]+)/[^/]+\1/[^/]+\1/
41. # accept anything else
42. #接受所有的鏈接，這裏可以做自己的修改，是的只接受自己規定類型的鏈接

# Licensed to the Apache Software Foundation (ASF) under one or more# contributor license agreements.  See the NOTICE file distributed with# this work for additional information regarding copyright ownership.# The ASF licenses this file to You under the Apache License, Version 2.0# (the "License"); you may not use this file except in compliance with# the License.  You may obtain a copy of the License at##     http://www.apache.org/licenses/LICENSE-2.0## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.# See the License for the specific language governing permissions and# limitations under the License.# The default url filter.# Better for whole-internet crawling.# Each non-comment, non-blank line contains a regular expression# prefixed by ‘+‘ or ‘-‘.  The first matching pattern in the file# determines whether a URL is included or ignored.  If no pattern# matches, the URL is ignored.# skip file: ftp: and mailto: urls#過濾掉file：ftp等不是html協議的鏈接-^(file|ftp|mailto):# skip image and other suffixes we can‘t yet parse#過濾掉圖片等格式的鏈接-\.(gif|GIF|jpg|JPG|png|PNG|ico|ICO|css|sit|eps|wmf|zip|ppt|mpg|xls|gz|rpm|tgz|mov|MOV|exe|jpeg|JPEG|bmp|BMP)$# skip URLs containing certain characters as probable queries, etc.#-[?*!@=] 過濾掉汗特殊字符的鏈接，因為要爬取更多的鏈接，所以修改過濾條件，使包含？=的鏈接不被過濾掉-[*!@]# skip URLs with slash-delimited segment that repeats 3+ times, to break loops#過濾掉一些特殊格式的鏈接-.*(/[^/]+)/[^/]+\1/[^/]+\1/# accept anything else#接受所有的鏈接，這裏可以做自己的修改，是的只接受自己規定類型的鏈接

原因解釋：因為爬取的公告鏈接為（http://www.online.sdu.edu.cn/news/article.php?pid=636514943），鏈接中含有？和=字符，所以被過濾特殊字符的正則表達式過濾掉，通過修改regex-urlfilter.txt配置文件（如上），最終可以爬取這類公告的鏈接。

nutch如何修改regex-urlfilter.txt爬取符合條件的鏈接