MogileFS啟動流程

例項化MogileFS::Server並執行

my $server; # server singleton
sub server {
    my ($pkg) = @_; 
    return $server ||= bless {}, $pkg;#空就建立物件，有就返回，單例
}
$server->run();

讀取配置

優先順序，命令列>配置檔案>預設配置

MogileFS::Config->load_config;

獲得資料庫操作控制代碼

MogileFS::Store是資料庫操作的基類,實際返回的物件是MogileFS::Store::MySQL

MogileFS::Config->check_database;
my $sto = eval { Mgd::get_store() };
sub get_store {
    return $store = MogileFS::Store->new;
}
$sto->ping;
sub ping {
    my $self = shift;
    return $self->dbh->ping;
}
$self->{dbh} = DBI->connect($self->{dsn}, $self->{user}, $self->{pass}
 

設定fidid fidid就是資料庫中fid的最大值，但對這次啟動來說是最小值

my $min_fidid = $sto->max_fidid;
sub max_fidid {
    my $self = shift;
    return $self->dbh->selectrow_array("SELECT MAX(fid) FROM file");
}

設定為守護程序 MogileFS::Util->daemonize()

兩次fork後成為守護程序

第一次fork後，父程序關閉，子程序成為孤兒程序，init接管

setsid讓孤兒程序脫離會話，併成為程序組，脫離終端。

忽略掛起訊號

第二次fork後，父程序關閉，子程序成為孤兒程序，init接管，成為守護程序

據說呼叫第二次fork可以徹底排除取得會話的可能

daemonize() if MogileFS->config("daemonize");
sub daemonize {
    if ($pid = fork) { exit 0; }
    croak "Cannot detach from controlling terminal"
        unless $sess_id = POSIX::setsid();
    $SIG{'HUP'} = 'IGNORE';
    if ($pid = fork) { exit 0; }
    chdir "/";
    umask 0;
}

設定程序訊號處理

程式自己中斷的訊號TERM和外部中斷訊號INT是類似的，幹掉所有子程序，刪除PID檔案

管道訊號忽略，(只有寫，沒有讀的管道，會發出這種訊號)

$SIG{TERM}  = sub {
    my @children = MogileFS::ProcManager->child_pids;
    print STDERR scalar @children, " children to kill.\n" if $DEBUG;
    my $count = kill( 'TERM' => @children );
    print STDERR "Sent SIGTERM to $count children.\n" if $DEBUG;
    MogileFS::ProcManager->remove_pidfile;
    Mgd::log('info', 'ending run due to SIGTERM');
    Sys::Syslog::closelog();

    exit 0;
};
$SIG{INT} = sub{同上}
$SIG{PIPE} = 'IGNORE';

建立伺服器端socket

Danga::Socket是一個socket事件驅動，處理客戶端請求的是MogileFS::Connection::Client

設定%OtherFds

my @servers;
foreach my $listen (@{ MogileFS->config('listen') }) {
    my $server = IO::Socket::INET->new(LocalAddr => $listen,
                                       Type      => SOCK_STREAM,
                                       Proto     => 'tcp',
                                       Blocking  => 0,
                                       Reuse     => 1,#呼叫Reuse可以免去伺服器在終止到重啟之間的所停留的時間
                                       Listen    => 1024 ) #監聽佇列，其實就是連線數
        or die "Error creating socket: [email protected]\n";
    $server->sockopt(SO_KEEPALIVE, 1); 
    # save sub to accept a client
    push @servers, $server;

    Danga::Socket->AddOtherFds( fileno($server) => sub {
            while (my $csock = $server->accept) {
                MogileFS::Connection::Client->new($csock);#也會被加到%DescriptorMap
            }   
        } );
}

設定socket事件處理後的回撥，並開始迴圈

MogileFS::ProcManager 程序管理工具

EventLoop = FirstTimeEventLoop; 符號表設定，呼叫的是FirstTimeEventLoop

MogileFS::ProcManager->push_pre_fork_cleanup(sub {
    # so children don't hold server connection open
    #關閉連線的匿名函式,給子程序用的
    close($_) foreach @servers;
});

# setup the post event loop callback to spawn jobs, and the timeout
Danga::Socket->DebugLevel(3);#始終是0
Danga::Socket->SetLoopTimeout( 250 ); #事件超時 250 milliseconds
Danga::Socket->SetPostLoopCallback(MogileFS::ProcManager->PostEventLoopChecker);#每次事件驅動完成後執行的函式
Danga::Socket->EventLoop();

Socket輪詢器選擇

我本機為Poll,以後呼叫EventLoop就是PollEventLoop了,Poll需要輪詢，

線上應該為Epoll,Epoll核心提供反射模式，無需輪詢

Epoll和Poll那種更好，可能取決於活躍的Socket。。。，如果全是活躍的呢

sub FirstTimeEventLoop {
    my $class = shift;

    _InitPoller();

    if ($HaveEpoll) {
        EpollEventLoop($class);
    } elsif ($HaveKQueue) {
        KQueueEventLoop($class);
    } else {
        PollEventLoop($class);
    }
}
*EventLoop = *PollEventLoop;

第一次輪詢開始

還記得Danga::Socket->SetPostLoopCallback(MogileFS::ProcManager->PostEventLoopChecker);嗎？，第一次輪詢直接回調

還記得%OtherFds嗎，主程序的socket在裡面,%DescriptorMap暫時還沒，不過這個雜湊很重要

my @poll;
foreach my $fd ( keys %OtherFds ) {
    push @poll, $fd, POLLIN;
}    
while ( my ($fd, $sock) = each %DescriptorMap ) {
    push @poll, $fd, $sock->{event_watch};
}
my $count = IO::Poll::_poll($timeout, @poll);
unless ($count) {
    return unless PostEventLoop();
    next;
}

建立Monitor子程序

建立一副IPC通訊一個給父程序，一個給子程序,並切無緩衝

socketpair(my $parents_ipc, my $childs_ipc, AF_UNIX, SOCK_STREAM, PF_UNSPEC )
        or die( "Sockpair failed" );
select((select( $parents_ipc ), $|++)[0]);
select((select( $childs_ipc  ), $|++)[0]);

父類返回MogileFS::Connection::Worker物件,構造引數是IPC中的一個

MogileFS::Connection::Worker繼承Danga::Socket,會增加到$DescriptorMap{$fd} = $self;

MogileFS::ProcManager->RegisterWorkerConn 設定POLLIN和$ChildrenByJob{$worker->job}->{$worker->pid} = $worker;

# if i'm the parent
if ($pid) {
    sigprocmask(SIG_UNBLOCK, $sigset)
        or return error("Can't unblock SIGINT for fork: $!");

    close($childs_ipc);  # unnecessary but explicit
    IO::Handle::blocking($parents_ipc, 0);

    my $worker_conn = MogileFS::Connection::Worker->new($parents_ipc);
    $worker_conn->pid($pid);
    $worker_conn->job($job);
    MogileFS::ProcManager->RegisterWorkerConn($worker_conn);
    return $worker_conn;
}

子類通過job_to_class獲取MogileFS::Worker::Monitor,然後例項化，呼叫work運作起來

會增加到$DescriptorMap{$fd} = $self;但是不要和主程序的混淆，完全兩個程序裡的

子類不會執行到exit,子程序也呼叫了Danga的socket事件迴圈

$_->() foreach @prefork_cleanup;
my $class = MogileFS::ProcManager->job_to_class($job)
        or die "No worker class defined for job '$job'\n";
my $worker = $class->new($childs_ipc);

# set our frontend into child mode
MogileFS::ProcManager->SetAsChild($worker);

$worker->work;
exit 0;

Monitor程序開始工作

Monitor監控資料庫和Storage的情況

最重要的是Monitor啟動後，傳送了訊息":monitor_just_ran"，接著就一直迴圈監聽和處理了

my $db_monitor;
$db_monitor = sub {
    $self->parent_ping;#和p保持連線
    $self->cache_refresh;#監控資料庫配置,及時通知p
    $db_monitor_ran++;
    Danga::Socket->AddTimer(4, $db_monitor);#每4秒查詢一次
};  

$db_monitor->();
$self->read_from_parent;

my $main_monitor;
$main_monitor = sub {
    $self->parent_ping;
    $self->usage_refresh;
    if ($db_monitor_ran) {
        $self->send_to_parent(":monitor_just_ran");
        $db_monitor_ran = 0;
    }   
    Danga::Socket->AddTimer(2.5, $main_monitor);
};  

$main_monitor->();
Danga::Socket->AddOtherFds($self->psock_fd, sub{ $self->read_from_parent }); 
Danga::Socket->EventLoop;

MogileFSD程序處理:monitor_just_ran訊息

設定完需要的workers後，又和呼叫建立Monitor一樣，建立其他的workers

my $child = shift;
# Gas up other workers if monitor's completed for the first time.
if (! $monitor_good) {
    MogileFS::ProcManager->set_min_workers('queryworker' => MogileFS->config('query_jobs'));
    MogileFS::ProcManager->set_min_workers('delete'      => MogileFS->config('delete_jobs'));
    MogileFS::ProcManager->set_min_workers('replicate'   => MogileFS->config('replicate_jobs'));
    MogileFS::ProcManager->set_min_workers('reaper'      => MogileFS->config('reaper_jobs'));
    MogileFS::ProcManager->set_min_workers('fsck'        => MogileFS->config('fsck_jobs'));
    MogileFS::ProcManager->set_min_workers('job_master'  => 1);
    $monitor_good = 1;
    $allkidsup    = 0;
}