一、方案  http://www.cnblogs.com/youxuguang/p/5917215.html

方法一：

（1）cat file|sed 's/[,.:;/!?]/ /g'|awk '{for(i=1;i<=NF;i++)array[$i]++;}END{for(i in array) print i,array[i]}'   #其中file為要操作的檔案，sed中/ /間有一個空格。

（2）sed 's/[,.:;/!?]/ /g' file|awk '{for(i=1;i<=NF;i++)array[$i]++;}END{for(i in array) print i,array[i]}'     #（1）和（2）效果一致。

方法二：

（1）awk 'BEGIN{RS="[,.:;/!?]"}{for(i=1;i<=NF;i++)array[$i]++;}END{for(i in array) print i,array[i]}' file

這裡 -F',' 表明每個詞用逗號分隔  https://zhidao.baidu.com/question/586302142.html
NF其實是number of field, 即整行(或者說record)裡面詞 (更準確的翻譯應該是域)的總數
NF-1 就是倒數第二個詞

二、驗證

[[email protected] shell]# cat file
hello world,hi girl;how old are you?
where are you from?
how are you?
i am fine!thinks.
and you?
http://www.cnblogs.com/youxuguang/

[[email protected] shell]# cat file|sed 's/[,.:;/!?]/ /g'|awk '{for(i=1;i<=NF;i++)array[$i]++;}END{for(i in array) print i,array[i]}'
com 1
http 1
from 1
www 1
i 1
you 4
hi 1
hello 1
youxuguang 1
and 1
world 1
cnblogs 1
where 1
old 1
how 2
fine 1
am 1
are 3
girl 1
thinks 1

[[email protected] shell]# sed 's/[,.:;/!?]/ /g' file|awk '{for(i=1;i<=NF;i++)array[$i]++;}END{for(i in array) print i,array[i]}'
com 1
http 1
from 1
www 1
i 1
you 4
hi 1
hello 1
youxuguang 1
and 1
world 1
cnblogs 1
where 1
old 1
how 2
fine 1
am 1
are 3
girl 1
thinks 1

[[email protected] shell]# awk 'BEGIN{RS="[,.:;/!?]"}{for(i=1;i<=NF;i++)array[$i]++;}END{for(i in array) print i,array[i]}' file
com 1
http 1
from 1
www 1
i 1
you 4
hi 1
hello 1
youxuguang 1
and 1
world 1
cnblogs 1
where 1
old 1
how 2
fine 1
am 1
are 3
girl 1
thinks 1