一、原理

Winsock2的釋出使得Socket I/O有了和檔案I/O統一的介面。我們可以通過使用Win32檔案操縱函式ReadFile和WriteFile來進行Socket I/O。伴隨而來的，用於普通檔案I/O的重疊I/O模型和完成埠模型對Socket I/O也適用了。這些模型的優點是可以達到更佳的系統性能，但是實現較為複雜，裡面涉及較多的C語言技巧。例如我們在完成埠模型中會經常用到所謂的“尾隨資料”。 這個模型與上述其他模型不同的是它使用Winsock2提供的非同步I/O函式WSARecv。在呼叫WSARecv時，指定一個WSAOVERLAPPED結構，這個呼叫不是阻塞的，也就是說，它會立刻返回。一旦有資料到達的時候，被指定的WSAOVERLAPPED結構中的hEvent被Signaled。下面分別講解關鍵API函式。

二、HeapAlloc

HeapAlloc是一個Windows API函式。它用來在指定的堆上分配記憶體，並且分配後的記憶體不可移動。它分配的記憶體不能超過4MB。函式原型是：

LPVOID HeapAlloc(HANDLE hHeap, DWORD dwFlags, SIZE_T dwBytes);

hHeap：要分配堆的控制代碼，可以通過HeapCreate()函式或GetProcessHeap()函式獲得。
dwFlags：堆分配時的可選引數，其值可以為以下的一種或多種：

• HEAP_GENERATE_EXCEPTIONS：如果分配錯誤將會丟擲異常，而不是返回NULL。異常值可能是STATUS_NO_MEMORY,表示獲得的記憶體容量不足，或是STATUS_ACCESS_VIOLATION,表示存取不合法。
• HEAP_NO_SERIALIZE：不使用連續存取。
• HEAP_ZERO_MEMORY：將分配的記憶體全部清零。

dwBytes：要分配堆的位元組數。
返回值：如果成功分配記憶體，返回值為一個指向所分配記憶體塊的首地址的（void*）指標。如果分配記憶體失敗，並且沒有指定HEAP_GENERATE_EXCEPTIONS，則返回NULL。如果指定了HEAP_GENERATE_EXCEPTIONS，則丟擲異常，而不返回NULL，異常程式碼有：

• STATUS_NO_MEMORY：由於缺少可用記憶體或者是堆損壞導致分配失敗。
• STATUS_ACCESS_VIOLATION：由於堆損壞或者是不正確的函式引數導致分配失敗。

三、GetProcessHeap

GetProcessHeap是一個Windows API函式。它返回呼叫程序的預設堆控制代碼。函式原型是：

HANDLE GetProcessHeap(void);

函式沒有引數。如果函式成功，返回呼叫程序的預設記憶體堆控制代碼。這個函式允許你從執行緒的堆動態分配記憶體，而不必使用HeapCreare函式建立一個堆。如果函式失敗，返回 Null。若想，可以呼叫GetLastError獲得更多錯誤資訊。

四、HeapFree

HeapFree是一個Windows API函式。它用來釋放堆記憶體。函式原型是：

BOOL HeapFree(HANDLE hHeap, DWORD dwFlags, LPVOID lpMem);

hHeap：堆記憶體塊釋放。這個引數是HeapCreate或GetProcessHeap函式返回的控制代碼。
dwFlags：指定幾個可控釋放的記憶體塊。指定以下值將覆蓋flOptions中指定引數對應的值,當堆是由使用HeapCreate函式。它的值可以是HEAP_NO_SERIALIZE，表示序列存取將不會被使用。為了確保序列化訪問，所有呼叫這個函式將被禁用，在呼叫HeapCreate指定HEAP_NO_SERIALIZE。在這種情況下，沒有必要在此函式呼叫另外指定HEAP_NO_SERIALIZE。訪問程序堆時沒有指定此值。該系統可以建立應用程式的過程中額外的執行緒，如CTRL + C處理程式，同時訪問程序堆。
lpMem：被釋放的記憶體塊的指標。這HeapAlloc或HeapReAlloc函式返回的指標。如果這個指標為NULL,則為空。
如果函式成功,返回值是非零。否則為零。應用程式可以呼叫GetLastError擴充套件的錯誤資訊。

五、WSARecv

在重疊模型中，接收資料就要靠它了，它的引數也比recv要多，因為要用到重疊結構。函式原型是：

int WSARecv(SOCKET s, LPWSABUF lpBuffers, DWORD dwBufferCount, LPDWORD lpNumberOfBytesRecvd, LPDWORD lpFlags, LPWSAOVERLAPPED lpOverlapped, LPWSAOVERLAPPED_COMPLETION_ROUTINE lpCompletionRoutine);

s：連線到此伺服器的socket物件。
lpBuffers：接收到的資料儲存在這裡。
dwBufferCount：有幾個緩衝區。
// 這裡需要一個由WSABUF結構構成的陣列
lpNumberOfBytesRecvd：實際收到了多少個位元組的資料。如果lpOverlapped引數不為空，那個這個引數可以為空。據MSDN表述，這樣做有助於避免潛在的錯誤。
lpFlags：這個引數很複雜。實在是不好翻譯，我貼出MSDN的原文吧。它的值可以是：

• MSG_PEEK：Peeks at the incoming data. The data is copied into the buffer, but is not removed from the input queue. This flag is valid only for nonoverlapped sockets.
• MSG_OOB：Processes OOB data.
• MSG_PARTIAL：This flag is for message-oriented sockets only. On output, this flag indicates that the data specified is a portion of the message transmitted by the sender. Remaining portions of the message will be specified in subsequent receive operations. A subsequent receive operation with the MSG_PARTIAL flag cleared indicates end of sender's message. As an input parameter, this flag indicates that the receive operation should complete even if only part of a message has been received by the transport provider.
• MSG_PUSH_IMMEDIATE：This flag is for stream-oriented sockets only. This flag allows an application that uses stream sockets to tell the transport provider not to delay completion of partially filled pending receive requests. This is a hint to the transport provider that the application is willing to receive any incoming data as soon as possible without necessarily waiting for the remainder of the data that might still be in transit. What constitutes a partially filled pending receive request is a transport-specific matter. In the case of TCP, this refers to the case of incoming TCP segments being placed into the receive request data buffer where none of the TCP segments indicated a PUSH bit value of 1. In this case, TCP may hold the partially filled receive request a little longer to allow the remainder of the data to arrive with a TCP segment that has the PUSH bit set to 1. This flag tells TCP not to hold the receive request but to complete it immediately. Using this flag for large block transfers is not recommended since processing partial blocks is often not optimal. This flag is useful only for cases where receiving and processing the partial data immediately helps decrease processing latency. This flag is a hint rather than an actual guarantee. This flag is supported on Windows 8.1, Windows Server 2012 R2, and later.
• MSG_WAITALL：The receive request will complete only when one of the following events occurs: The buffer supplied by the caller is completely full. The connection has been closed.  The request has been canceled or an error occurred. Be aware that if the underlying transport provider does not support MSG_WAITALL, or if the socket is in a non-blocking mode, then this call will fail with WSAEOPNOTSUPP. Also, if MSG_WAITALL is specified along with MSG_OOB, MSG_PEEK, or MSG_PARTIAL, then this call will fail with WSAEOPNOTSUPP. This flag is not supported on datagram sockets or message-oriented sockets.

lpCompletionRoutine：回撥函式，事件通知入口。詳情請看下節。 WSA_IO_PENDING是最常見的返回值，這是說明我們的WSARecv操作成功了，但是I/O操作還沒有完成，所以我們就需要繫結一個事件來通知我們操作何時完成。如果函式呼叫發生錯誤，接收操作立即返回0。這種情況下，這個完成例程將會被列入即將呼叫的時間表當執行緒變為警告狀態, 否則，函式返回SOCKET_ERROR，我們可以通過WSAGetLastError獲取錯誤程式碼. WSA_IO_PENDING表示這個重疊操作成功初始化，稍後便會顯示。

六、CompletionROUTINE

這個函式是WSARecv函式的引數中回函數的引數格式。它本身是不存在的，使用者程式碼應該按照以下正規化宣告：

void CALLBACK CompletionROUTINE(DWORD dwError, DWORD cbTransferred, LPWSAOVERLAPPED lpOverlapped, DWORD dwFlags);

CompletionRoutine is a placeholder for an application-defined or library-defined function name. The dwError specifies the completion status for the overlapped operation as indicated by lpOverlapped. The cbTransferred parameter specifies the number of bytes received. The dwFlags parameter contains information that would have appeared in lpFlags if the receive operation had completed immediately. This function does not return a value.

Returning from this function allows invocation of another pending completion routine for this socket. When using WSAWaitForMultipleEvents, all waiting completion routines are called before the alertable thread's wait is satisfied with a return code of WSA_IO_COMPLETION. The completion routines can be called in any order, not necessarily in the same order the overlapped operations are completed. However, the posted buffers are guaranteed to be filled in the same order in which they are specified.

If you are using I/O completion ports, be aware that the order of calls made to WSARecv is also the order in which the buffers are populated. WSARecv should not be called on the same socket simultaneously from different threads, because it can result in an unpredictable buffer order.

七、PEERIO_OPERATION_DATA結構

Windows API定義的LWSAOVERLAPPED結構遠遠不能滿足從主執行緒到回撥執行緒傳遞引數的需要，我們需要定義一個自己的資料結構，通過HeapAlloc分配記憶體，把指標傳遞到回撥函式，結構如下：

typedef struct
{
	WSAOVERLAPPED overlap;
	WSABUF buf;
	char message[SOCKET_MESSAGE_SIZE];
	DWORD received_byte_count;
	DWORD flags;
	SOCKET client;
	iserver_manager *manager;
}PEERIO_OPERATION_DATA, *LPPEERIO_OPERATION_DATA;

overlap：記錄原始的值。
buf：緩衝區指標。 message：緩衝區。 received_byte_count：收到的位元組數。 flags：旗標。 client：這批緩衝區是哪個客戶端傳送的。 manager：回撥結構。

八、示例程式碼

接著上面幾篇Socket文章寫，關於公共程式碼與反射式客戶端請參見：Socket程式設計模型之簡單選擇模型。下面是新建的overlapped_server工程，新建了一個overlapped_server_manager型別，繼承自iserver_manager介面，標頭檔案完整程式碼如下：

#pragma once

#include <WinSock2.h>
#include <common_callback.h>
//#include <afxsock.h>

#define SOCKET_MESSAGE_SIZE 1024

typedef struct
{
	WSAOVERLAPPED overlap;
	WSABUF buf;
	char message[SOCKET_MESSAGE_SIZE];
	DWORD received_byte_count;
	DWORD flags;
	SOCKET client;
	iserver_manager *manager;
}PEERIO_OPERATION_DATA, *LPPEERIO_OPERATION_DATA;

class overlapped_server_manager:
	protected iserver_manager
{
private:
	int iport;
	int iclient_count;
	int iaddr_size;
	SOCKET server;
	SOCKET snew_client;
	WSADATA data;
	LPPEERIO_OPERATION_DATA peers[WSA_MAXIMUM_WAIT_EVENTS];
	common_callback callback;
	BOOL brunning;
	BOOL bnew_client;

private:
	void cleanup();
	int find_peer(LPPEERIO_OPERATION_DATA peer);

protected:
	bool accept_by_crt();
	bool accept_by_winapi();
	void receive();

public:
	void shutdown();
	void start_accept();
	void start_receive();
	void completion_routine(DWORD error, DWORD transfered, LPPEERIO_OPERATION_DATA peer, DWORD flags);
	
public:
	overlapped_server_manager();
	virtual ~overlapped_server_manager();
};

void CALLBACK do_completion_routine(DWORD error, DWORD transfered, LPWSAOVERLAPPED overlapped, DWORD flags);

實現檔案完整程式碼如下：

#include "overlapped_server_manager.h"
#include <stdio.h>
#include <tchar.h>
#include <cassert>


overlapped_server_manager::overlapped_server_manager()
{
	iport = 5150;
	iclient_count = 0;
	iaddr_size = sizeof(SOCKADDR_IN);
	ZeroMemory(peers, sizeof(LPPEERIO_OPERATION_DATA));
	WSAStartup(MAKEWORD(2, 2), &data);
	brunning = FALSE;
	bnew_client = FALSE;
	callback.set_manager(this);
}


overlapped_server_manager::~overlapped_server_manager()
{
	cleanup();
}

bool overlapped_server_manager::accept_by_crt()
{
	SOCKET server;
	SOCKADDR_IN server_addr;
	SOCKADDR_IN client_addr;
	int iret = 0;

	server = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
	server_addr.sin_addr.S_un.S_addr = htonl(INADDR_ANY);
	server_addr.sin_family = AF_INET;
	server_addr.sin_port = htons(iport);
	do
	{
		iret = bind(server, (struct sockaddr*)&server_addr, iaddr_size);
		if (iret == SOCKET_ERROR)
		{
			iport++;
			server_addr.sin_port = htons(iport);
		}
	} while (iret == -1);
	listen(server, 3);
	printf("服務啟動成功，埠是：%d\n", iport);
	while (brunning)
	{
		snew_client = accept(server, (struct sockaddr*)&client_addr, &iaddr_size);
		if (snew_client == INVALID_SOCKET)
			continue;
		printf("新客戶端連線：%s:%d\n", inet_ntoa(client_addr.sin_addr), ntohs(client_addr.sin_port));
		bnew_client = TRUE;
	}
	return true;
}

bool overlapped_server_manager::accept_by_winapi()
{
	
	return true;
}

void overlapped_server_manager::receive()
{
	LPPEERIO_OPERATION_DATA peer = NULL;

	while (brunning)
	{
		if (bnew_client)
		{
			peer = (LPPEERIO_OPERATION_DATA)HeapAlloc(GetProcessHeap(), HEAP_ZERO_MEMORY, sizeof(PEERIO_OPERATION_DATA));
			peer->buf.len = SOCKET_MESSAGE_SIZE;
			peer->buf.buf = peer->message;
			peer->client = snew_client;
			peer->manager = this;
			peers[iclient_count] = peer;
			WSARecv(peer->client, &peer->buf, 1, &peer->received_byte_count, &peer->flags, &peer->overlap, &do_completion_routine);
			bnew_client = FALSE;
			iclient_count++;
		}
		SleepEx(1000, TRUE);
	}
}

void overlapped_server_manager::shutdown()
{
	callback.shutdown();
	cleanup();
}

void overlapped_server_manager::cleanup()
{
	int i = 0;

	brunning = FALSE;
	for (i = 0; i < iclient_count; i++)
		HeapFree(GetProcessHeap(), 0, peers[i]);
	iclient_count = 0;
}

void overlapped_server_manager::completion_routine(DWORD error, DWORD transfered, LPPEERIO_OPERATION_DATA peer, DWORD flags)
{
	int index = -1;

	index = find_peer(peer);
	assert(index != -1);
	if (error != 0 || transfered == 0)
	{
		closesocket(peer->client);
		HeapFree(GetProcessHeap(), 0, peer);
		if (index < iclient_count - 1)
			peers[index] = peers[iclient_count - 1];
		peers[iclient_count - 1] = nullptr;
		iclient_count--;
	}
	else
	{
		peer->message[transfered] = 0;
		send(peer->client, peer->message, transfered, 0);
		ZeroMemory(&peer->overlap, sizeof(WSAOVERLAPPED));
		peer->buf.len = SOCKET_MESSAGE_SIZE;
		peer->buf.buf = peer->message;
		WSARecv(peer->client, &peer->buf, 1, &peer->received_byte_count, &peer->flags, &peer->overlap, do_completion_routine);
	}
}

int overlapped_server_manager::find_peer(LPPEERIO_OPERATION_DATA peer)
{
	int index = -1;
	int i = 0;

	for (i = 0; i < iclient_count; i++)
	{
		if (peers[i] != peer)
			continue;
		index = i;
		break;
	}
	return index;
}

void overlapped_server_manager::start_accept()
{
	brunning = TRUE;
	callback.start_accept_by_crt();
}

void overlapped_server_manager::start_receive()
{
	brunning = TRUE;
	callback.start_receive();
}

void CALLBACK do_completion_routine(DWORD error, DWORD transfered, LPWSAOVERLAPPED overlapped, DWORD flags)
{
	auto peer = (LPPEERIO_OPERATION_DATA)overlapped;
	auto pmanager = reinterpret_cast<overlapped_server_manager*>(peer->manager);
	pmanager->completion_routine(error, transfered, peer, flags);
}

int main()
{
	overlapped_server_manager osm;
	osm.start_accept();
	osm.start_receive();
	printf("重疊（Overlapped）I/0模型服務端啟動成功。按任意鍵關閉伺服器。\n");
	system("pause");
	osm.shutdown();
	return 0;
}

九、執行效果

十、感想

WSARecv提供recv幾個額外的重要功能：可以進行套接字重疊進行重疊recv操作；允許多個接收緩衝區，使其適應於聚集/分散I/O型別；WSARecv函式通過引數 s 指定連線的SOCKETS 或者繫結未連線的SOCKETS 來讀取未接收的資料。SOCKETS的本地地址是必須知道的。在伺服器應用程式中，SOCKET通常顯式通過bind繫結或者通過隱式的通過accept,WSAAccept繫結；在客戶點應用程式中，SOCKET通過connect,WSAConnect,Sento,WSASendTo或者WSAJoinLeaf隱式的繫結本地地址。
對於連線和非連線的SOCKET，這個函式通過accept接收的訊息限制地址,它僅僅對指定遠端地址返回訊息，忽略其他地址傳送的訊息。
對於重疊的SOCKETS，WSARecv通常投遞一個或多個可以填充資料的緩衝區，申請完成後等待指定的完成指示（呼叫的完成例程或事件物件設定）發生，如果這個操作不能馬上完成，最終的完成狀態可以通過完成例程或WSAGetOverlappedResult獲取.
所有的I/O線上程建立時初始化，執行緒退出時取消，對於重疊的SOCKETS，如果執行緒關閉，未決的非同步操作可能會在完成之前失敗。
如果 lpOverlapped ，lpCompletionRoutine 都NULL，SOCKET 在這個函式中被認為是非重疊的SOCKET。對於非重疊的SOCKETS，阻塞的語意與recv相同，且lpOverlapped ，lpCompletionRoutine都會被忽略，任何已經收到和傳輸緩衝的資料將被複制到指定的使用者緩衝區，如果當前沒有收到資料或沒有任何資料傳輸，SOCKET將被阻塞，直到接收到資料。Window Socket 2沒有為這個函式定義任何的標準阻塞超時機制。
基於位元組流的協議棧試圖儘可能返回多的可用的緩衝區空間和可接收的資料。然而，單個位元組足夠疏通這次呼叫，而且也沒有任何可以保證返回多餘一個位元組。而基於訊息的協議，則傳送個完整的資料包。（位元組流的協議和基於訊息的協議的主要區別）。