[En-Nut-Discussion] ICMP echo request response failure on ARP cache timeout

[Harald Kipp]

Hi Nathan,

Nathan Moore wrote:

> Every time I try to do much that involves the TCP state machine thread my
> head
> explodes, so I don't pretend to understand all of it's internals.

TCP is a complex beast. In opposite to other parts of Nut/OS, the TCP
state machine had been written in the first place with the human reader
in mind, using a specific function for each state. Some parts could have
been optimized to create less code. But none of the contributors ever
had the courage to work it out. There are many things that may go
miserably wrong. And many things did go wrong in the past. I remember
some very nasty and long living bugs in this area.


> is ready to send.  If the interface is slow enough the TCP thread may try to
> retransmit
> before the other thread has finished sending.

Right, TCP timeout processing had been crimped into the state machine
loop, which made it less flexible and controllable.


> I tend to favor optimizing getting things sent over receiving because it
> encourages
> more free RAM.

That's a very important guideline.


>> We'd still need a TCP state machine thread to handle re-transmissions.
>> That one could be simpler than the combined timing/handler loop used
>> right now.
> 
> Since you now do protocol registration you could make part of that
> registration
> a callback to a periodically called function and a pointer to it's data --
> the TCP
> thread could turn into one of these functions, and a more generic IP
> transmission
> thread would call it.

I can't comment on this right now, without having a clearer picture of
the general structure.


>> Doing this in the driver may not be the best idea. It requires to
>> rewrite several drivers and may make writing new drivers more complex
>> than it needs to be.
> 
> In my version of Nut (a fork) I have altered NutIpOutput to not know
> anything about
> ARP, ethernet, or PPP.  I replaced the drivers' generic output routine
> pointers with
> IP output routine pointers which do interface specific stuff (ARP) and then
> directly call
> the generic routines.

My fault. Of course, using a device specific thread does not mean, that
the thread must be implemented in the driver.


> The more general purpose the solution the slower/bigger it gets.
> I prefer a thread per device because it allows each device to be kept busy
> as long
> as there is work to do, 

This is a good argument and follows the way other parts of the system
are designed. Otherwise a slow device may slow down the transfer of
faster devices or require a complicated scheduling.


> (Oh yeah, in the NutIfConfig changes I mentioned above I changed it so that
> PPP
> didn't record it's IP to EEPROM, which it did in the same place as Eth.)

I recently removed CONFNET saving from the system routines. I wasn't
aware of the PPP problem, but recognized, that many people got confused
by this "feature". If the application uses the wrong function call
sequence, they may end up with the hard coded values overwriting a valid
EEPROM contents (I think the ftpd sample has or had this problem too).
Now the application must explicitly call NutSaveConfig to store new
settings.

Not fully sure, if this is valid for PPP too. Furthermore, the system
may still save the current IP address, in which case the PPP problem
remains. Actually each interface should have its own EEPROM area, but
running two interfaces concurrently is something that never worked out
of the box so far.


> For ethernet you could have a chain of protocols which each had a queue of
> packets (netbufs).
> These chain links could be made of (raw, IPv4).  The Ethernet TX thread
> would do different
> things based on which link a packet was under.  Raw packets would just be
> sent, while IP packets
> would be fed through ARP.  If an arp request needed to be made then that arp
> request would be
> appended to the RAW queue (unless there was an outstanding request already).
>  If the IP packet
> was ready to send then it's ethernet header would be completed and that
> packet would become a
> RAW packet, and be placed in the raw queue.  Essentially only things in the
> RAW queue can be
> sent to the device.

Sounds interesting enough to give it a try.


> I had thought about using a linked list or tree-like structure for netbufs
> (maybe even
> with reference counting and garbage collection).  That would be a huge
> undertaking
> but would allow for more than the current 4 layer model, and cloning them
> could
> just be aliasing them at the layer that you were interested in --
> NutIpOutput's
> handling of broadcast would just pass the same netbuf head to all the
> interfaces
> and they would all generate their own headers for the same IP packet.

Garbage collection is a controversial feature in the area of tiny
embedded systems. That doesn't mean that it wouldn't offer advantages.
For many embedded developers dynamic memory allocation is a no-no as
well. I personally never experienced heap fragmentation problems on
Nut/OS (though the algorithm used here is quite slow).

Well, there are many things to try before thinking about garbage
collection. Indeed, re-using packet headers on multiple packets is
something I thought about too. It doesn't actually require garbage
collection, but just the reference counter.


Harald