這幾天在研究MPI的實現問題，由於只是移植其中的幾個函式，所以不需要將每個層次的MPI都進行移植。

首先來看看MPI_Send的標準通訊模式，以下翻譯自MPI: A Message-Passing Interface Standard Version 3.0（當前最新版本為3.1），3.4 Communication Modes。

3.4 Communication Modes 通訊模式

</pre><pre name="code" class="plain">The send call described in Section 3.2.1 is blocking: it does not return until the message data and envelope have been safely stored away so that the sender is free to modify
the send buffer. The message might be copied directly into the matching receive buffer, or it might be copied into a temporary system buffer.
 

3.2.1節中描述的send（MPI_Send）函式為阻塞模式：直到資料資訊與資料信封已經被安全的儲存，被快取的傳送資料緩衝區就可以被任意更改了。資訊可能直接被髮送到了匹配的接收緩衝區中，或者他被拷貝到了一箇中間系統緩衝區中。

這裡要一點我個人的理解，其實MPI_Send的語義僅要求在保證傳送緩衝區可以被重新使用後，MPI_Send函式就可以返回，這就是所謂的阻塞傳送。至於阻塞多久，MPI標準中並沒有明確的規定，可以是先將待發送資訊緩衝到某個系統中的緩衝區中，也可以阻塞直到全部資訊傳送到了接收緩衝區中。這與具體實現相關。來簡單看一個實驗。原始碼如下：

#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include "mpi.h"

const int MAX_STRING = 100;

int main()
{
	char greeting[MAX_STRING];
	int comm_sz;
	int my_rank;

	MPI_Init(NULL,NULL);
	MPI_Comm_size(MPI_COMM_WORLD,&comm_sz);
	MPI_Comm_rank(MPI_COMM_WORLD,&my_rank);

	if(my_rank==1){
		sprintf(greeting,"Greeting from process %d of %d!",my_rank,comm_sz);
		sleep(10);
		MPI_Send(greeting,strlen(greeting)+1,MPI_CHAR,0,0,MPI_COMM_WORLD);
		printf("rank %d has returned from MPI_Send\n",my_rank);
	}else if(my_rank==2){
		sprintf(greeting,"Greeting from process %d of %d!",my_rank,comm_sz);
		MPI_Send(greeting,strlen(greeting)+1,MPI_CHAR,0,0,MPI_COMM_WORLD);
		printf("rank %d has returned from MPI_Send\n",my_rank);
	}else if(my_rank==3){
		sprintf(greeting,"Greeting from process %d of %d!",my_rank,comm_sz);
		MPI_Send(greeting,strlen(greeting)+1,MPI_CHAR,0,0,MPI_COMM_WORLD);
		printf("rank %d has returned from MPI_Send\n",my_rank);
	}else if(my_rank==0){
		MPI_Recv(greeting,MAX_STRING,MPI_CHAR,1,0,MPI_COMM_WORLD,MPI_STATUS_IGNORE);
		printf("%s\n",greeting);
		MPI_Recv(greeting,MAX_STRING,MPI_CHAR,2,0,MPI_COMM_WORLD,MPI_STATUS_IGNORE);
		printf("%s\n",greeting);
		MPI_Recv(greeting,MAX_STRING,MPI_CHAR,3,0,MPI_COMM_WORLD,MPI_STATUS_IGNORE);
		printf("%s\n",greeting);
	}

	MPI_Finalize();
	return 0;
}
 

執行結果如下：

rank 2 has returned from MPI_Send
rank 3 has returned from MPI_Send（執行至此處要等待一些時間）
rank 1 has returned from MPI_Send
Greeting from process 1 of 4!
Greeting from process 2 of 4!
Greeting from process 3 of 4!

通過上面的實驗我們可以驗證，在MPICH實現中，MPI_Send將資料首先發送到了某個緩衝區後便返回，rank1由於被掛起，所以沒有執行MPI_Send函式。再來看接收端，由於應被接收的資訊還沒有傳送到接收段，所以會一直阻塞，此時其他傳送端的資訊已經發送至某個緩衝區中（這個緩衝區可能位於系統中，也可能位於接收端，以上都是我個人的猜想）。

接著看原文

Message buffering decouples the send and receive operations. A blocking send can complete as soon as the message was buffered, even if no matching receive has been 
executed by the receiver. On the other hand, message buffering can be expensive, as it entails additional memory-to-memory copying, and it requires the allocation of 
memory for buffering. MPI offers the choice of several communication modes that allow one to control the choice of the communication protocol.

訊息緩衝機制使得傳送與接收機制解耦合。阻塞傳送可以在訊息被緩衝後立刻返回，即使接收端還沒有開始執行接收程式。另一方面，訊息緩衝可能是代價高昂的，因為它需要額外的“記憶體到記憶體”的拷貝操作，同時它還需要申請用於緩衝的儲存區。MPI 提供了多種不同的訊息模式，允許使用者選擇特定的通訊協議。

The send call described in Section 3.2.1 uses the standard communication mode. In this mode, it is up to MPI to decide whether outgoing messages will be buffered. MPI may
buffer outgoing messages. In such a case, the send call may complete before a matching receive is invoked. On the other hand, buffer space may be unavailable, or MPI 
may choose not to buffer outgoing messages, for performance reasons. In this case, the send call will not complete until a matching receive has been posted, and the 
data has been moved to the receiver.

在3.2.1中描述的傳送函式使用標準通訊模式。在這一模式中，由MPI來決定是否對將要傳送的資訊進行緩衝。MPI 也許會緩衝將要傳送的訊息。在這種情況下，傳送函式可能在相對應的接收函式被呼叫前返回。另一方面緩衝空間可能是無法使用的，或者MPI也許選擇不緩衝即將傳送的訊息，基於效能方面的考慮。在這種情況下，傳送函式將不會返回，直到一個對應的接收函式被呼叫，並且資料已經被移動到接收者。

以上這一段總結起來就是一句話：在標準模式下，訊息可能會被緩衝，也可能不被緩衝。如果訊息被緩衝，則send函式可以立即返回；若沒有被緩衝，則send函式直到接收函式接收資料後才能返回。

Thus, a send in standard mode can be started whether or not a matching receive has been posted. It may complete before a matching receive is posted. The standard mode 
send is non-local: successful completion of the send operation may depend on the occurrence of a matching receive.

所以，標準模式下的send函式可能開始被呼叫，無論相對應的接收函式是否被呼叫。傳送函式可能在對應的接收函式被呼叫前返回。標準模式的傳送是非本地的：傳送操作的成功完成可能會依賴於對應的接收函式的的呼叫（非緩衝模式下依賴於接收函式的呼叫）。

Rationale. The reluctance of MPI to mandate whether standard sends are buffering or not stems from the desire to achieve portable programs. Since any system will run
out of buffer resources as message sizes are increased, and some implementations may want to provide little buffering, MPI takes the position that correct (and therefore,
portable) programs do not rely on system buffering in standard mode. Buffering may improve the performance of a correct program, but it doesn’t affect the result of the
program. If the user wishes to guarantee a certain amount of buffering, the user-provided buffer system of Section 3.6 should be used, along with the buffered-mode
send. (End of rationale.)

原理的闡述（應該是這麼翻譯吧）。MPI 標準沒有強制要求使用緩衝區，以使程式可移植性更好。隨著訊息尺寸的增加，任何系統都將用盡緩衝區資源，同時某些系統可能會提供較少的緩衝區，在標準模式下 MPI 不依賴與訊息緩衝（這一句翻譯的不是很好）。緩衝可能會提升正確程式的效能，但是它不影響程式的結果。如果使用者希望保證一定數量的緩衝區，在3.6節中描述的使用者提供的緩衝系統應該被使用，這一緩衝區也可被緩衝模式的傳送所使用。

There are three additional communication modes.
A buffered mode send operation can be started whether or not a matching receive has been posted. It may complete before a matching receive is posted. However, unlike
the standard send, this operation is local, and its completion does not depend on the occurrence of a matching receive. Thus, if a send is executed and no matching receive is
posted, then MPI must buffer the outgoing message, so as to allow the send call to complete. An error will occur if there is insufficient buffer space. The amount of available 
buffer space is controlled by the user — see Section 3.6. Buffer allocation by the user may be required for the buffered mode to be effective.

除標準模式外，還有三種通訊模式

緩衝模式的傳送操作可以開始無論一個對應的接收程式是否被呼叫。同時他也可能返回在一個對應的接收函式被呼叫前。然而，與標準傳送不同的是，這一操作是本地的，並且它的完成不依賴於對應的接收函式的發生。所以，如果一個傳送函式被呼叫並且對應的接收函式沒有啟動，那麼MPI一定要緩衝正在傳送的訊息，以便於傳送函式可以完成。一個錯誤將被丟擲如果沒有足夠的緩衝空間。可獲得的緩衝空間是由使用者直接控制的，具體內容請見3.6節。由使用者進行的緩衝空間申請可能需要使能緩衝模式。

A send that uses the synchronous mode can be started whether or not a matching receive was posted. However, the send will complete successfully only if a matching 
receive is posted, and the receive operation has started to receive the message sent by the synchronous send. Thus, the completion of a synchronous send not only 
indicates that the send buffer can be reused, but it also indicates that the receiver has reached a certain point in its execution, namely that it has started executing the 
matching receive. If both sends and receives are blocking operations then the use of the synchronous mode provides synchronous communication semantics: 
a communication does not complete at either end before both processes rendezvous at the communication. A send executed in this mode is non-local.

使用同步模式的傳送函式可能被呼叫無論對應的接收函式是否被呼叫。然而，僅在一個對應的接收函式被呼叫，並且開始接收由同步傳送函式傳送的訊息後傳送函式才成功返回。所以，一個同步傳送的完成不僅意味著傳送緩衝區可以被重用，同時也意味著接收函式已經達到它執行過程中的某個確定點，意味著對應的接收函式已經開始執行。如果接收與傳送都是阻塞操作，那麼同步模式的使用將會使用阻塞通訊語義：在兩個程序在通訊過程中相遇之前，通訊雙方均不會返回。在這種模式下的傳送操作是非本地。

A send that uses the ready communication mode may be started only if the matching receive is already posted. Otherwise, the operation is erroneous and its outcome is 
undefined. On some systems, this allows the removal of a hand-shake operation that is otherwise required and results in improved performance. The completion of the 
send operation does not depend on the status of a matching receive, and merely indicates that the send buffer can be reused. A send operation that uses the ready mode 
has the same semantics as a standard send operation, or a synchronous send operation; it is merely that the sender provides additional information to the system 
(namely that a matching receive is already posted), that can save some overhead. In a correct program, therefore, a ready send could be replaced by a standard send 
with no effect on the behavior of the program other than performance.

使用就緒模式的傳送函式僅在對應的接收函式被呼叫後才可能被啟動。否則的話，這一操作是錯誤並且它的結果是未定義的。在某些系統中，這種模式允許握手操作的移除並且能夠帶來效能上的提升。傳送操作的完成不依賴於對應的接收函式的狀態，並且僅意味著傳送緩衝區可以被重用。使用就緒模式的傳送操作具有與標準模式相同的語義，或與同步模式具有相同的語義；僅由傳送者提供額外資訊給系統（意味著一個對應的傳送操作已經被呼叫），這一做法可以減少一些開銷。在一個正確的程式中，所以，一個就緒傳送可以被標準傳送所代替，無論是程式的行為還是效能都不會產生影響。

Three additional send functions are provided for the three additional communication modes. The communication mode is indicated by a one letter prefix: B for buffered, 
S for synchronous, and R for ready.

以上三種額外的傳送函式是由三種額外的通訊模式所提供的。通訊模式是由一個字母的字首所表示的：B表示緩衝，S表示同步，R表示就緒。

接下來看看接收操作：

There is only one receive operation, but it matches any of the send modes. The receive operation described in the last section is blocking: it returns only after the receive 
buffer contains the newly received message. A receive can complete before the matching send has completed (of course, it can complete only after the matching send 
has started).

僅有一種接收操作模式，但是它可以與任意一種傳送模式相匹配。在上一節中描述的接收操作是阻塞的：僅在接收緩衝區包含有訊息時才返回。一個接收函式可以在對應的傳送函式完成前返回（當然，它僅可以在對應的傳送函式啟動後才可以返回）。

Advice to implementors. Since a synchronous send cannot complete before a matching receive is posted, one will not normally buffer messages sent by such an operation.
It is recommended to choose buffering over blocking the sender, whenever possible, for standard sends. The programmer can signal his or her preference for blocking the
sender until a matching receive occurs by using the synchronous send mode.

給實現者的建議。由於一個同步傳送操作無法完成，在對應的接收函式被呼叫前，因此由那種操作傳送訊息通常不會被緩衝。建議選擇緩衝而不是阻塞傳送者，無論何時可能的話，對於標準傳送。程式設計人員可以提示他或她對於傳送函式阻塞的偏好知道一個對應的接收函式相匹配通過使用同步傳送模式（這一句翻譯的不太對）。

A possible communication protocol for the various communication modes is outlined below.
ready send: The message is sent as soon as possible.
synchronous send: The sender sends a request-to-send message. The receiver stores this request. When a matching receive is posted, the receiver sends back a 
permission-to-send message, and the sender now sends the message.
standard send: First protocol may be used for short messages, and second protocol for long messages.
buffered send: The sender copies the message into a buffer and then sends it with a nonblocking send (using the same protocol as for standard send).
Additional control messages might be needed for flow control and error recovery. Of course, there are many other possible protocols.
Ready send can be implemented as a standard send. In this case there will be no performance advantage (or disadvantage) for the use of ready send.
A standard send can be implemented as a synchronous send. In such a case, no data buffering is needed. However, users may expect some buffering.
In a multithreaded environment, the execution of a blocking communication should block only the executing thread, allowing the thread scheduler to de-schedule this
thread and schedule another thread for execution. (End of advice to implementors.)

對於多種不同通訊模式，一個可能的通訊協議列舉如下：

就緒傳送：訊息儘可能快的傳送。

同步傳送：傳送者傳送一個“要求傳送”訊息。接收者儲存這一訊息。當一個匹配的接收函式被呼叫時，接收者返回一個准許傳送的訊息，此時傳送者正式開始傳送訊息。

標準傳送：第一種協議可能用於短訊息，第二種協議用於長訊息。

緩衝傳送：傳送者拷貝訊息進入緩衝區然後以非阻塞方式傳送（使用與標準傳送相同的協議）。

為了流控與錯誤恢復，額外的控制資訊可能被需要。當然，還可能有許多其他的協議。

就緒傳送可以實現為標準傳送。在這種情況下，將不會任何效能提升（或下降）對於就緒傳送的使用。

一個標準傳送可以實現為一個同步傳送。在這種情況下，不需要資料緩衝。然而，使用者可能期望一些緩衝。

在多執行緒環境下，阻塞通訊的執行應該僅阻塞正在執行的執行緒，允許執行緒排程器重排程這一執行緒（進入阻塞狀態）並且排程另一程序開始執行。

3.5 Semantics of Point-to-Point Communication
A valid MPI implementation guarantees certain general properties of point-to-point communication, which are described in this section.

3.5 點到點通訊語義

一個有效的MPI實現保證點到點通訊的某些特定性質，具體內容請見下文。

Order 
Messages are non-overtaking: If a sender sends two messages in succession to the same destination, and both match the same receive, then this operation cannot 
receive the second message if the first one is still pending. If a receiver posts two receives in succession,and both match the same message, then the second receive 
operation cannot be satisfied by this message, if the first one is still pending. This requirement facilitates matching of sends to receives. It guarantees that message-passing 
code is deterministic, if processes are single-threaded and the wildcard MPI_ANY_SOURCE is not used in receives. (Some of the calls described later, such as MPI_CANCEL 
or MPI_WAITANY, are additional sources of nondeterminism.)

訊息序列

訊息是不可超越的：如果一個傳送者連續傳送兩條訊息到相同的目的節點，同時與相同的接收函式相對應，那麼這一操作不能先接收到第二條訊息如果第一條訊息仍處於掛起狀態。如果接收端連續呼叫接收函式，並且均與相同的訊息對應，那麼第二個接收操作無法接收這一訊息，如果第一條訊息還處於掛起狀態。這一必要條件保證了傳送與接收的匹配。它保證了訊息傳送程式碼是確定性的，如果程序是單執行緒的，並且萬用字元MPI_ANY_SOURCE也沒有被接收函式所使用。（某些之後討論的函式呼叫，例如MPI_CANCEL或者MPI_WAITANY，是非確定性的額外來源。）

If a process has a single thread of execution, then any two communications executed by this process are ordered. On the other hand, if the process is multithreaded, then the
semantics of thread execution may not define a relative order between two send operations executed by two distinct threads. The operations are logically concurrent, 
even if one physically precedes the other. In such a case, the two messages sent can be received in any order. Similarly, if two receive operations that are logically 
concurrent receive two successively sent messages, then the two messages can match the two receives in either order.

如果一個程序是單執行緒執行的，那麼由這個執行緒執行的任意兩個通訊都是按順序的。另一方面，如果程序是多執行緒的，那麼由兩個不同執行緒間執行的傳送操作的相對順序是未定義的。這些操作是邏輯上併發的，甚至如果其中一個物理上先於另一個。在這種情況下，這兩條訊息的接收可以是任意順序的。類似的，如果兩個接收操作邏輯上併發的接收兩條連續訊息，那麼這兩條訊息可以以任意順序（其實就是兩種情況，AB或者BA）對應兩個接收函式。

Progress 
If a pair of matching send and receives have been initiated on two processes, then at least one of these two operations will complete, independently of other actions 
in the system: the send operation will complete, unless the receive is satisfied by another message, and completes; the receive operation will complete, unless the message 
sent is consumed by another matching receive that was posted at the same destination process.

進一步

如果一組對應的傳送與接收函式分別在兩個不同的程序上初始化，那麼至少這兩個操作中的一個將會完成，與系統中的其他操作無關：傳送操作將會完成，除非接收操作被另一個訊息所滿足，並返回；接收操作將會返回，除非在相同目的程序中啟動的對應接收函式消耗了已傳送的訊息。

這一段我也沒有弄懂它的實際意思，標準中給出了一個例子，大家可以參考著看看。

3.7 Nonblocking Communication
One can improve performance on many systems by overlapping communication and computation. This is especially true on systems where communication can be executed 
autonomously by an intelligent communication controller. Light-weight threads are one mechanism for achieving such overlap. An alternative mechanism that often 
leads to better performance is to use nonblocking communication. A nonblocking send start call initiates the send operation, but does not complete it. The send start 
call can return before the message was copied out of the send buffer. A separate send complete call is needed to complete the communication, i.e., to verify that the data 
has been copied out of the send buffer. With suitable hardware, the transfer of data out of the sender memory may proceed concurrently with computations done at 
the sender after the send was initiated and before it completed. Similarly, a nonblocking receive start call initiates the receive operation, but does not complete it. The call can 
return before a message is stored into the receive buffer. A separate receive complete call is needed to complete the receive operation and verify that the data has been 
received into the receive buffer. With suitable hardware, the transfer of data into the receiver memory may proceed concurrently with computations done after the receive 
was initiated and before it completed. The use of nonblocking receives may also avoid system buffering and memory-to-memory copying, as information is provided early
on the location of the receive buffer.

3.7 非阻塞式通訊

程式可以提升效能通過覆蓋通訊與計算。這種方法在通訊可以被智慧通訊控制器自動執行的系統中效果尤為明顯。輕量級執行緒是達到這一覆蓋的一種機制。另一種同樣可以達到較好效能的可選方法是使用非阻塞式通訊。一個非阻塞式傳送開始函式初始化傳送操作，但是並不完成它。傳送開始函式可以返回在訊息被拷貝進入傳送緩衝區之前。一個單獨的傳送完成函式被用於完成通訊操作，例如，確認資料已經被拷貝進入傳送緩衝區。如果具有合適的硬體，從傳送端記憶體傳送資料的操作可能與在傳送端完成的計算操作相併發，在傳送函式被初始化並且在它完成之前（這句話的意思是說，在傳送初始化函式返回前，訊息已經被髮出）。類似的，一個非阻塞式接收開始函式初始化接收操作，但是並不完成它。這個函式可以返回在訊息被儲存進入接收緩衝區之前。一個單獨的接收完成函式被用於完成接收操作並且確認資料已經被儲存進入接收緩衝區。如果具有合適的硬體，進入接收端記憶體資料區的接收操作將與計算併發執行，在接收函式被初始化之前並且在接收初始化函式返回前。非阻塞式接收函式的使用還可能能夠避免系統緩衝操作與記憶體到記憶體的拷貝，由於資訊早已經被提供到了接收緩衝區所在的節點中。

以上這段話概括起來就是四個函式：

1. 傳送初始化函式，僅負責初始化傳送環境，不負責具體的傳送工作。
2. 傳送完成函式，僅負責確認傳送工作是否完成，這一函式返回後原來的使用者傳送緩衝區就可以被重新使用了。同樣不負責具體的傳送工作，傳送工作是由通訊控制器完成。
3. 接收初始化函式，僅負責接收初始化工作（具體工作內容不明）。
4. 接收完成函式，負責確認訊息是否已經被拷貝進入使用者訊息緩衝區。

Nonblocking send start calls can use the same four modes as blocking sends: standard, buffered, synchronous and ready. These carry the same meaning. 
Sends of all modes, ready excepted, can be started whether a matching receive has been posted or not; a nonblocking ready send can be started only if a matching receive 
is posted. In all cases, the send start call is local: it returns immediately, irrespective of the status of other processes. If the call causes some system resource to be exhausted, 
then it will fail and return an error code. Quality implementations of MPI should ensure that this happens only in “pathological” cases. That is, an MPI implementation 
should be able to support a large number of pending nonblocking operations.

非阻塞式初始化函式可以使用與阻塞式傳送相同的四種模式：標準，緩衝，同步與就緒。上述四種模式意義相同。所有四種模式的傳送操作，就緒模式除外，可以被啟動無論對應的接收函式是否被開始呼叫；一個非阻塞式就緒傳送可以被開始呼叫僅在一個對應的接收函式被呼叫後開始執行。在所有四種情況中，傳送初始化函式是本地的：它立刻返回，無論其他程序的狀態。如果函式呼叫導致系統資源被耗盡，那麼它將會失敗並且返回一個錯誤碼。高質量的MPI實現應該保證這種情況只在“病態”的情況發生。那麼，MPI 實現應該能夠支援大量掛起的非阻塞操作。

The send-complete call returns when data has been copied out of the send buffer. It may carry additional meaning, depending on the send mode.
If the send mode is synchronous, then the send can complete only if a matching receive has started. That is, a receive has been posted, and has been matched with the send. 
In this case, the send-complete call is non-local. Note that a synchronous, nonblocking send may complete, if matched by a nonblocking receive, before the receive complete 
call occurs. (It can complete as soon as the sender “knows” the transfer will complete, but before the receiver “knows” the transfer will complete.)
If the send mode is buffered then the message must be buffered if there is no pending receive. In this case, the send-complete call is local, and must succeed irrespective of 
the status of a matching receive.
If the send mode is standard then the send-complete call may return before a matching receive is posted, if the message is buffered. On the other hand, the 
receive-complete may not complete until a matching receive is posted, and the message was copied into the receive buffer.
Nonblocking sends can be matched with blocking receives, and vice-versa.

傳送完成函式在資料已經被全部拷貝出發送緩衝區後返回。它也許會附帶額外的資訊，根據傳送模式。

如果傳送模式是同步模式，那麼傳送可以完成僅在一個對應的接收端被啟動後。也即，一個接收函式已經被啟動，並且已經與傳送函式相匹配。在這種情況下，傳送完成函式是非本地的。注意，一個同步的，非阻塞式傳送也許會完成，如果一個對應的非阻塞接收函式，在接收完成函式發生前。（非阻塞式傳送可以完成只要傳送者“知道”傳送將要完成，但是先於接收者“知道”資料傳送將要完成。）

如果傳送模式是緩衝模式那麼訊息一定會被緩衝如果此時沒有未決的接收函式。在這種情況下，傳送完成函式是本地的，並且一定會成功不管相對應的接收函式的狀態。

如果訊息被緩衝，那麼標準傳送模式下的傳送完成函式可能會返回在對應接收函式返回前。另一方面，接收完成函式（這個地方貌似錯了，應該是傳送完成函式）可能無法完成直到一個對應的接收函式被啟動，並且訊息已經被拷貝進入接收緩衝區。

非阻塞傳送可以被阻塞式傳送相匹配，反過來情況相同。

3.7.2 Communication Initiation
We use the same naming conventions as for blocking communication: a prefix of B, S, or R is used for buffered, synchronous or ready mode. In addition a prefix of I 
(for immediate)indicates that the call is nonblocking.

3.7.2 通訊初始化

我們使用與阻塞式通訊相同的命名規則：一個字首B、S、R分別代表緩衝、同步與就緒模式。另一個字首I（用於代表立刻）表示這一函式呼叫是非阻塞式的。

Start a nonblocking receive.
These calls allocate a communication request object and associate it with the request handle (the argument request). The request can be used later to query the status of the
communication or wait for its completion.
A nonblocking send call indicates that the system may start copying data out of the send buffer. The sender should not modify any part of the send buffer after a nonblocking
send operation is called, until the send completes.
A nonblocking receive call indicates that the system may start writing data into the receive buffer. The receiver should not access any part of the receive buffer after a 
nonblocking receive operation is called, until the receive completes.

初始化一個非阻塞式的接收

這些函式申請了一個通訊請求物件並且與請求控制代碼相聯絡（引數request）。這一請求物件稍後可用於查詢通訊的狀態或者等待它的完成。

一個非阻塞式傳送函式意味著系統可能開始從傳送緩衝區中拷貝資料。傳送端不應該修改傳送緩衝區的任何一部分在非阻塞式傳送操作被呼叫後，直到傳送操作完成。

一個非阻塞式接收函式意味著系統可能開始向接收緩衝區中寫入資料。接收端不應該使用接收緩衝區的任何一部分在非阻塞式接收操作被呼叫後，直到接收操作完成。

3.7.3 Communication Completion
The functions MPI_WAIT and MPI_TEST are used to complete a nonblocking communication. The completion of a send operation indicates that the sender is now free to 
update the locations in the send buffer (the send operation itself leaves the content of the send buffer unchanged). It does not indicate that the message has been received, 
rather, it may have been buffered by the communication subsystem. However, if a synchronous mode send was used, the completion of the send operation indicates that 
a matching receive was initiated,and that the message will eventually be received by this matching receive.

 3.7.3 通訊完成

MPI_WAIT與MPI_TEST 函式用於完成非阻塞式通訊操作。一個傳送操作的完成意味著傳送端現在可任意的更新發送緩衝區（傳送操作本身保持傳送緩衝區上下文保持不變）。這部意味著訊息應該已經被接收，相反，它可能已經被通訊子系統所緩衝。而後，如果一個同步模式的傳送被使用，傳送操作的完成意味著一個對應的接收函式被初始化，並且訊息將最終被這個對應的接收函式所接收。

The completion of a receive operation indicates that the receive buffer contains the received message, the receiver is now free to access it, and that the status object is set. It
does not indicate that the matching send operation has completed (but indicates, of course, that the send was initiated).

一個接收操作的完成意味著接收緩衝區包含有被接收函式，接收端現在可以隨意使用它，並且狀態物件被設定。這不意味著對應的傳送操作已經完成（但意味著，當然，傳送操作已經被初始化）。

We shall use the following terminology: A null handle is a handle with value MPI_REQUEST_NULL . A persistent request and the handle to it are inactive if the request is 
not associated with any ongoing communication (see Section 3.9). A handle is active if it is neither null nor inactive. An empty status is a status which is set to 
return tag = MPI_ANY_TAG , source = MPI_ANY_SOURCE , error = MPI_SUCCESS , and is also internally configured so that calls to MPI_GET_COUNT, MPI_GET_ELEMENTS, and
MPI_GET_ELEMENTS_X return count = 0 and MPI_TEST_CANCELLED returns false . We set a status variable to empty when the value returned by it is not significant. Status is set
in this way so as to prevent errors due to accesses of stale information.

我們應該使用以下術語：一個空的控制代碼的值為MPI_REQUEST_NULL。請求物件與指向它的控制代碼是非活動的如果請求物件沒有與任何正在進行的通訊相關聯。一個控制代碼是活躍的如果它不是空的或者非活躍的（通過這句話的描述，感覺控制代碼就三種狀態“活躍”、“不活躍”、“空”）。一個空的狀態物件被設定為tag = MPI_ANY_TAG , source = MPI_ANY_SOURCE , error = MPI_SUCCESS，同時被內部設定以便於MPI_GET_COUNT, MPI_GET_ELEMENTS與MPI_GET_ELEMENTS_X函式返回0，MPI_TEST_CANCELLED函式返回錯誤。我們設定一個狀態變數為空當由它（應該指這幾個函式：MPI_GET_COUNT、MPI_GET_ELEMENTS、MPI_GET_ELEMENTS_X、MPI_TEST_CANCELLED）返回的值是無效的。以這種方式設定狀態物件是為了避免由於使用失效資訊而產生的錯誤。