fastdfs 一個group內實現按照不同的專案,指定路徑儲存.
環境介紹:
1: 公司目前有5個專案 A B C D E 日後可能會有所增加.
2: 使用fastdfs儲存這5各專案的檔案,要求各各專案的檔案分開儲存,也就是每個專案的檔案儲存到一個固定的位置.
3: 三臺機器ip地址分配如下
tracker角色 IP:192.168.1.219
storageA角色 IP: 192.168.1.215
storageB角色 IP:192.168.1.216
4: 這裡主要記錄下實現思路,具體的配置方法都很簡單.
---------tracker這臺機器 主要有兩個服務 fdfs_tracker 和 nginx---------------
tracker.conf的配置檔案很簡單這裡不貼出來了
nginx的配置檔案如下: nginx負責把訪問請求轉發給後臺的兩臺storage.
[[email protected] ~]# cat /usr/local/nginx/conf/nginx.conf | grep -v "#" | grep -v "^$"
user www;
worker_processes 1;
error_log logs/error.log;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
server_names_hash_bucket_size 128;
client_header_buffer_size 32k;
large_client_header_buffers 4 32k;
client_max_body_size 300m;
sendfile on;
tcp_nopush on;
keepalive_timeout 120;
gzip on;
gzip_min_length 1k;
gzip_buffers 4 16k;
gzip_http_version 1.1;
gzip_comp_level 2;
gzip_types text/plain application/x-javascript text/css application/xml;
gzip_vary on;
gzip_disable "MSIE[1-6].";
proxy_redirect off;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffer_size 16k;
proxy_buffers 4 64k;
proxy_busy_buffers_size 128k;
proxy_temp_file_write_size 128k;
log_format access '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$upstream_addr" "$upstream_response_time" "$host"';
proxy_cache_path /var/cache/nginx/proxy_cache levels=1:2 keys_zone=http-cache:500m max_size=10g inactive=30d;
proxy_temp_path /var/cache/nginx/proxy_cache/tmp;
upstream group1 {
server 192.168.1.215:80;
server 192.168.1.216:80;
}
server {
listen 80;
server_name image.ty.com;
location ~* /group1/(M00|M01|M02|M03|M04|M05) {#這裡我使用了6個儲存路徑,每個專案一個
proxy_next_upstream http_502 http_504 error timeout invalid_header;
proxy_cache http-cache;
proxy_cache_valid 200 304 12h;
proxy_cache_key $uri$is_args$args;
proxy_pass http://group1;
expires 3d;
}
access_log logs/image.access.log access;
location / {
root html;
index index.html index.htm;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
include /usr/local/nginx/conf/vhost/*.conf;
}
---------------storage角色的兩臺機器-------------------
這兩天機器的配置完全一樣. 都運行了fdfs_storage 和 nginx , 其中nginx添加了fastdfs模組
fdfs_storage的配置檔案storage.conf內容如下:
[[email protected] ~]# cat /etc/fdfs/storage.conf | grep -v "^#" | grep -v "^$"
disabled=false
group_name=group1
bind_addr=
client_bind=true
port=23000
connect_timeout=30
network_timeout=60
heart_beat_interval=30
stat_report_interval=60
base_path=/data/
max_connections=256
buff_size = 256KB
accept_threads=1
work_threads=4
disk_rw_separated = true
disk_reader_threads = 1
disk_writer_threads = 1
sync_wait_msec=50
sync_interval=0
sync_start_time=00:00
sync_end_time=23:59
write_mark_file_freq=500
store_path_count=6一共store_path0-5六個儲存路徑.
store_path0=/data/data1
store_path1=/data/data2
store_path2=/data/data3
store_path3=/data/data4
store_path4=/data/data5
store_path5=/data/data6
subdir_count_per_path=256
tracker_server=192.168.1.219:22122
log_level=info
run_by_group=
run_by_user=
allow_hosts=*
file_distribute_path_mode=0
file_distribute_rotate_count=100
fsync_after_written_bytes=0
sync_log_buff_interval=10
sync_binlog_buff_interval=10
sync_stat_file_interval=300
thread_stack_size=512KB
upload_priority=10
if_alias_prefix=
check_file_duplicate=0
file_signature_method=hash
key_namespace=FastDFS
keep_alive=0
use_access_log = false
rotate_access_log = false
access_log_rotate_time=00:00
rotate_error_log = false
error_log_rotate_time=00:00
rotate_access_log_size = 0
rotate_error_log_size = 0
log_file_keep_days = 0
file_sync_skip_invalid_record=false
use_connection_pool = false
connection_pool_max_idle_time = 3600
http.domain_name=
http.server_port=8888
nginx配置檔案如下
[[email protected] ~]# cat /usr/local/nginx/conf/nginx.conf | grep -v "^#" | grep -v "^$"
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log logs/access.log main;
sendfile on;
keepalive_timeout 65;
client_max_body_size 2m;
server {
listen 80;
server_name localhost;
#charset koi8-r;
access_log logs/host.access.log main;
location /group1/M00 { #根據請求路徑配置對應的儲存路徑.
root /data/data1;
ngx_fastdfs_module; #使用nginxmodule處理請求
}
location /group1/M01 {
root /data/data2;
ngx_fastdfs_module;
}
location /group1/M02 {
root /data/data3;
ngx_fastdfs_module;
}
location /group1/M03 {
root /data/data4;
ngx_fastdfs_module;
}
location /group1/M04 {
root /data/data5;
ngx_fastdfs_module;
}
location /group1/M05 {
root /data/data6;
ngx_fastdfs_module;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
include /usr/local/nginx/conf/vhosts/*.conf;
}
nginx的fastdfs模組的配置檔案如下:
[[email protected] ~]# cat /etc/fdfs/mod_fastdfs.conf | grep -v "^#" | grep -v "^$"
connect_timeout=2
network_timeout=30
base_path=/tmp
load_fdfs_parameters_from_tracker=true
storage_sync_file_max_delay = 86400
use_storage_id = false
storage_ids_filename = storage_ids.conf
tracker_server=192.168.1.219:22122
storage_server_port=23000
group_name=group1
url_have_group_name = true
store_path_count=6
store_path0=/data/data1
store_path1=/data/data2
store_path2=/data/data3
store_path3=/data/data4
store_path4=/data/data5
store_path5=/data/data6
log_level=info
log_filename=/var/log/mod_fastdfs.log
response_mode=proxy
if_alias_prefix=
flv_support = true
flv_extension = flv
group_count = 0
-----------------以上就是用到的所有配置檔案----------------------------
想實現按照不通的專案儲存到不同的路徑,主要通過程式碼來實現 以java為例
首先: 我們可以通過tracker獲取一個可以使用的storage的ip, -------java程式碼如下---------
/**
*獲得可用的storageIP
*
*@paramtrackerClient
*@paramtrackerServer
*@return返回storageIP
*/
privatestaticStringgetStorageServerIp(TrackerClienttrackerClient,TrackerServertrackerServer){
StringstorageIp=null;
if(trackerClient!=null&&trackerServer!=null){
try{
StorageServerstorageServer=trackerClient.getStoreStorage(trackerServer,STORAGE_SERVER_GROUP);
storageIp=storageServer.getSocket().getInetAddress().getHostAddress();
}catch(IOExceptione){
e.printStackTrace();
}
}
log.info("——獲取組中可用的storageIP——"+storageIp);
returnstorageIp;
}
其次:可以根據獲取的ip把需要上傳的檔案指定到要上傳到的路徑裡面
/**
*得到Storage服務
*
*@paramstorageIp
*@return返回Storage服務
*/
privatestaticStorageServergetStorageServer(StringstorageIp){
StorageServerstorageServer=null;
if(storageIp!=null&&!("").equals(storageIp)){
try{
storageServer=newStorageServer(storageIp,Integer.parseInt(STORAGE_SERVER_PORT),Integer.parseInt(STORAGE_SERVER_M00));
}catch(IOExceptione){
e.printStackTrace();
}
}
log.info("——storageserver生成");
returnstorageServer;
}
這樣就通過fastdfs支援多路徑這個功能實現了不同專案分開儲存的需求.
Linux下也有對應的命令:[[email protected] ~]# fdfs_upload_file -h
Usage: fdfs_upload_file <config_file> <local_filename> [storage_ip:port] [store_path_index]
[[email protected] ~]# fdfs_upload_file /etc/fdfs/storage.conf anni.jpg 192.168.1.215:23000 5
group1/M05/00/D4/wKgB11W4SQGASXcFAAFAB2RsPpA542.jpg
轉載於:https://my.oschina.net/denglz/blog/485084