BSD network memory allocation bug

[eric at cls.usask.ca (Eric Norum)]

I've been looking into getting rid of the `"Network mbuf space can not be enlarged' panic in the RTEMS/FreeBSD networking code.  In doing so I think I've uncovered a bug in the FreeBSD code.  I'd like to have this verified by others, though.

It seems to me that the FreeBSD mbuf memory allocation routines can return NULL even if they are called with the M_WAIT flag.  However, much of the FreeBSD code that calls an mbuf allocation routine with M_WAIT assumes that a valid pointer will always be returned.

For example, in the sogetopt() routine (kern/uipc_socket.c) there is:
	        m = m_get(M_WAIT, MT_SOOPTS);
                m->m_len = sizeof (int);

Here is the mechanism in the FreeBSD code whereby a NULL can be returned by an M_WAIT allocation request:
1) Some routine calls m_get(M_WAIT,.....)
2) The mbuf free list happens to be empty (mmbfree == 0) so the MGET macro calls m_mballoc to try to get more mbufs.
3) The map happens to be full (mb_map_full != 0) so m_mballoc returns 0.
4) The MGET macro calls m_retry
5) m_retry calls m_reclaim to try and reclaim some space.
6) Assume that no space is reclaimed.
7) m_retry calls MGET again
8) The free list will still be empty (since m_reclaim didn't reclaim any space)
9) The MGET macro will call m_mballoc.
10) The map is still full, so m_mballoc returns 0.
11) The MGET macro returns 0.
12) m_retry returns 0 (to the original MGET macro call)
13) The original MGET macro call returns 0.

Simple, isn't it :-)

Note that the above sequence can't happen with the existing RTEMS/FreeBSD code since the call to m_mballoc causes a panic.....

=======================================================================

So, to make a long story short:

1) The m_mballoc and m_clalloc routines need to be rewritten for RTEMS so that they do not generate a panic.  
2) It seems to me that when doing this rewrite I should also ensure that they never return a NULL value when M_WAIT is specified.

Comments?

My aim is always to make the absolute minimum number of changes to the FreeBSD code, but it looks like I'm going to have to get in and modify some of the routines in kern/uipc_mbuf.c.

-- 
Eric Norum                                 eric at cls.usask.ca
Saskatchewan Accelerator Laboratory        Phone: (306) 966-6308
University of Saskatchewan                 FAX:   (306) 966-6058
Saskatoon, Canada.