[En-Nut-Discussion] TCP vs. Memory (RFC)

[Michael Jones]

Hi,

I've been spending the last days pulling out my already sparse hair over
this issue.

The environment is a ETHERNUT2 compatible board which receives data from a
server. This server sends bursts of information in packets between 8 to 200
bytes. These bursts might contain between 1 to 1000+ packets. Using
Wireshark I can see 800+ sent within one second. The Nut/OS has about 27000
Bytes free memory when idle (which equals about 300 minimal size (19 + 67)
netbuf allocs).

Speed is not so much a problem, it's totally OK that the board is not able
to handle the load at the speed that it is sent. 

Now to the observations:

1)  The SMCS LAN91C111 will start losing packets once the OS loses the race
to pick the packets up in time.
    (The chip only has 4 packets buffer, so this is by design and too be
expected).

2)  The lost packets cause the "later" packets to be queued into so_rx_nbq
for "future" use.
    (TCPSM.c, NutTcpStateEstablished, ~1190)

3)  The Memory fills up with these future use packets.
    (AFAICS the windows size is not adjusted and there is no limit on these
packets)

4a) The system finally recovers the missing packet and recovers from above
situation.

4b) The future packets use up all free memory till no more packets can be
received.
    (lanc111.c, NicGetPacket, ~1065)

5)  TCP State Machine finally gets fed up and timeouts the Socket and
pending e.g. NutTcpReceive.
    (haven't found the exact point where it does this yet - but it does)

6)  the App calls NutTcpCloseSocket.

7)  NutTcpCloseSocket calls NutTcpStateCloseEvent.

8)  NutTcpStateCloseEvent sets state to TCPS_FIN_WAIT_1.

9)  The state TCPS_FIN_WAIT_1 causes NutTcpOutput(sock, 0, 0) with SO_FIN
and SO_ACK.

9a) NutTcpOutput sends packet to host, 
    host answers causing NutTcpStateProcess to call NutTcpStateFinWait1,
    which calls NutTcpDestroySocket,	
    which frees all used memory and system recovers...

9b) NutTcpOutput FAILS to send packet to host because there is no memory
left,
    Host never answers,
    Memory never gets freed,
    System dies horrible death...

Now there are two aspects in this:

a) How to stop the packets flooding the memory.
b) How to assure that the Socket gets destroyed even in this grim situation.

Comments and thought are welcome.

Cu,
Michael