BSD network memory allocation bug

[qqi at world.std.com (Quality Quorum)]

On Fri, 21 May 1999, Eric Norum wrote:

> I've been looking into getting rid of the `"Network mbuf space can not be enlarged' panic in the RTEMS/FreeBSD networking code.  In doing so I think I've uncovered a bug in the FreeBSD code.  I'd like to have this verified by others, though.
> 
> It seems to me that the FreeBSD mbuf memory allocation routines can return NULL even if they are called with the M_WAIT flag.  However, much of the FreeBSD code that calls an mbuf allocation routine with M_WAIT assumes that a valid pointer will always be returned.
> 
> For example, in the sogetopt() routine (kern/uipc_socket.c) there is:
> 	        m = m_get(M_WAIT, MT_SOOPTS);
>                 m->m_len = sizeof (int);
> 
> Here is the mechanism in the FreeBSD code whereby a NULL can be returned by an M_WAIT allocation request:
> 1) Some routine calls m_get(M_WAIT,.....)
> 2) The mbuf free list happens to be empty (mmbfree == 0) so the MGET macro calls m_mballoc to try to get more mbufs.
> 3) The map happens to be full (mb_map_full != 0) so m_mballoc returns 0.
> 4) The MGET macro calls m_retry
> 5) m_retry calls m_reclaim to try and reclaim some space.
> 6) Assume that no space is reclaimed.
> 7) m_retry calls MGET again
> 8) The free list will still be empty (since m_reclaim didn't reclaim any space)
> 9) The MGET macro will call m_mballoc.
> 10) The map is still full, so m_mballoc returns 0.
> 11) The MGET macro returns 0.
> 12) m_retry returns 0 (to the original MGET macro call)
> 13) The original MGET macro call returns 0.
> 
> Simple, isn't it :-)
> 
> Note that the above sequence can't happen with the existing RTEMS/FreeBSD code since the call to m_mballoc causes a panic.....
> 
> =======================================================================
> 
> So, to make a long story short:
> 
> 1) The m_mballoc and m_clalloc routines need to be rewritten for RTEMS so that they do not generate a panic.  
> 2) It seems to me that when doing this rewrite I should also ensure that they never return a NULL value when M_WAIT is specified.
> 
> Comments?


I do not bleive that RTOS has to follow same logic UNIX in the case:

    1. Work load of RTOS stuff is more predictale so in many 
       cases there is enough information to figure out 
       necessary size of mbuf free list.

    2. All IP based protocols will recover from packet loss
       without too many problems.

    3. Memory is cheap.

So, the most natuarl thing is to have pool of mbuffs and return
NULL in all cases (WAIT and NO_WAIT) when it is exhausted.

> 
> My aim is always to make the absolute minimum number of changes to the FreeBSD code, but it looks like I'm going to have to get in and modify some of the routines in kern/uipc_mbuf.c.
> 
> -- 
> Eric Norum                                 eric at cls.usask.ca
> Saskatchewan Accelerator Laboratory        Phone: (306) 966-6308
> University of Saskatchewan                 FAX:   (306) 966-6058
> Saskatoon, Canada.
>

Simple systems == reliable systems.

Thanks,

Aleksey