Archived from groups: microsoft.public.windowsnt.protocol.tcpip (More info?)
Hello,
I'm trying to make two applications talk to each other over TCP/IP. The
sender has small blocks of data that need to be transmitted to the receiver
as soon as possible. Untransmitted packets might become obsolete quickly
therefore I need to minimize buffering on both the sender and the receiver
side.
In theory you can do this by disabling Nagle and setting SO_RCVBUF and so
SO_SNDBUF to a small size (let's suppose one kilobyte for now). Tests show
however that there's an internal operating system buffer that is 63836
(65536-1700) bytes in size. What is this buffer, and what can I do to work
around it?
Here's some quick and (very) dirty sample code.
Both the sender and the receiver start with initializing a socket:
WSADATA wsad;
WSAStartup(MAKEWORD(2,0), &wsad);
SOCKET s = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
sockaddr_in sa;
int len = sizeof(sa);
ZeroMemory(&sa, sizeof(sa));
sa.sin_family = AF_INET;
sa.sin_port = htons(3333);
sa.sin_addr.s_addr = inet_addr ("127.0.0.1");
int nTotal = 0;
The receiver then accepts a connection and displays the bytes available for
read:
bind(s, (SOCKADDR*)&sa, len);
listen(s, SOMAXCONN);
SOCKET sock = accept(s, (SOCKADDR*)&sa, &len);
SetSocketOptions (sock);
while(TRUE) {
ioctlsocket(sock, FIONREAD, (u_long*)&nTotal);
printf("Receivable: %d bytes\n", nTotal);
Sleep(1000);
}
closesocket(sock);
The sender connects to the receiver and pushes down data one byte at a time:
connect(s, (SOCKADDR*)&sa, len);
SetSocketOptions (s);
while(TRUE) {
char sz[1];
send(s, sz, sizeof(sz), 0);
nTotal += sizeof(sz);
printf("Sent %d bytes (total:%d)\n", sizeof(sz), nTotal);
}
SetSocketOptions simply disables Nagle and sets both the receive and the
send buffers to one K:
void SetSocketOptions(SOCKET s)
{
DWORD dwPacketSize = 1024;
BOOL bTrue = TRUE;
setsockopt(s, SOL_SOCKET, SO_RCVBUF, (char*)&dwPacketSize,
sizeof(dwPacketSize));
setsockopt(s, SOL_SOCKET, SO_SNDBUF, (char*)&dwPacketSize,
sizeof(dwPacketSize));
setsockopt(s, IPPROTO_TCP, TCP_NODELAY, (PSTR)&bTrue, sizeof(bTrue));
}
Running the receiver produces the following (expected) output after the
sender connects:
Receivable: 1024 bytes
Receivable: 1024 bytes
..... etc
Running the sender produces the following:
.....
Sent 1 bytes (total:65881)
Sent 1 bytes (total:65882)
Sent 1 bytes (total:65883)
Sent 1 bytes (total:65884)
....Then it stops here due to blocking on a send() that has no matching
recv()
The sender has 1k of buffers to work with that we set in SO_SNDBUF. The
receiver has another 1k of buffers due to SO_RCVBUF. We've managed to cram
65884 bytes down send()'s throat without blocking. We know where 2kbytes are
buffered, so where do the other 63836 bytes live? Note that 64836 is a
suspicious number - one would expect an even 64kbytes (IPv4 datagram?). The
difference is exactly 1700 bytes, which is close to an ethernet payload size
(1500 bytes) plus some change (headers, etc).
Can this internal buffer be disabled or limited in size somehow?
Thanks,
Marton Anka
Hello,
I'm trying to make two applications talk to each other over TCP/IP. The
sender has small blocks of data that need to be transmitted to the receiver
as soon as possible. Untransmitted packets might become obsolete quickly
therefore I need to minimize buffering on both the sender and the receiver
side.
In theory you can do this by disabling Nagle and setting SO_RCVBUF and so
SO_SNDBUF to a small size (let's suppose one kilobyte for now). Tests show
however that there's an internal operating system buffer that is 63836
(65536-1700) bytes in size. What is this buffer, and what can I do to work
around it?
Here's some quick and (very) dirty sample code.
Both the sender and the receiver start with initializing a socket:
WSADATA wsad;
WSAStartup(MAKEWORD(2,0), &wsad);
SOCKET s = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
sockaddr_in sa;
int len = sizeof(sa);
ZeroMemory(&sa, sizeof(sa));
sa.sin_family = AF_INET;
sa.sin_port = htons(3333);
sa.sin_addr.s_addr = inet_addr ("127.0.0.1");
int nTotal = 0;
The receiver then accepts a connection and displays the bytes available for
read:
bind(s, (SOCKADDR*)&sa, len);
listen(s, SOMAXCONN);
SOCKET sock = accept(s, (SOCKADDR*)&sa, &len);
SetSocketOptions (sock);
while(TRUE) {
ioctlsocket(sock, FIONREAD, (u_long*)&nTotal);
printf("Receivable: %d bytes\n", nTotal);
Sleep(1000);
}
closesocket(sock);
The sender connects to the receiver and pushes down data one byte at a time:
connect(s, (SOCKADDR*)&sa, len);
SetSocketOptions (s);
while(TRUE) {
char sz[1];
send(s, sz, sizeof(sz), 0);
nTotal += sizeof(sz);
printf("Sent %d bytes (total:%d)\n", sizeof(sz), nTotal);
}
SetSocketOptions simply disables Nagle and sets both the receive and the
send buffers to one K:
void SetSocketOptions(SOCKET s)
{
DWORD dwPacketSize = 1024;
BOOL bTrue = TRUE;
setsockopt(s, SOL_SOCKET, SO_RCVBUF, (char*)&dwPacketSize,
sizeof(dwPacketSize));
setsockopt(s, SOL_SOCKET, SO_SNDBUF, (char*)&dwPacketSize,
sizeof(dwPacketSize));
setsockopt(s, IPPROTO_TCP, TCP_NODELAY, (PSTR)&bTrue, sizeof(bTrue));
}
Running the receiver produces the following (expected) output after the
sender connects:
Receivable: 1024 bytes
Receivable: 1024 bytes
..... etc
Running the sender produces the following:
.....
Sent 1 bytes (total:65881)
Sent 1 bytes (total:65882)
Sent 1 bytes (total:65883)
Sent 1 bytes (total:65884)
....Then it stops here due to blocking on a send() that has no matching
recv()
The sender has 1k of buffers to work with that we set in SO_SNDBUF. The
receiver has another 1k of buffers due to SO_RCVBUF. We've managed to cram
65884 bytes down send()'s throat without blocking. We know where 2kbytes are
buffered, so where do the other 63836 bytes live? Note that 64836 is a
suspicious number - one would expect an even 64kbytes (IPv4 datagram?). The
difference is exactly 1700 bytes, which is close to an ethernet payload size
(1500 bytes) plus some change (headers, etc).
Can this internal buffer be disabled or limited in size somehow?
Thanks,
Marton Anka
Facebook
Twitter
Instagram