Hi Tom,

The short answer is that it doesn't free the memory back to the system, and
neither do most other malloc libraries.

And now for the long answer...

For systems using sbrk() (which mpatrol uses on UNIX by default), it would
be virtually impossible to return the memory back to the system due to the
fragmentation of the heap. The only way you could really do this would be
to get the malloc library to free each of the pages it allocated in reverse
order by calling sbrk() with a negative value, but that would also require
the user's program to do frees() in the same way. I'm not saying it's
impossible to do because I'm sure some programs would lend themselves to
that, but in most cases it would be impossible and malloc libraries have to
handle the general case.

For memory allocated by mmap() (which mpatrol uses with USEMMAP) or
VirtualAlloc() on Windows, the situation is slightly less complicated
because the allocated pages could in theory be freed in any order if there
are no more malloced allocations located in them.

However, there are two main reasons why malloc libraries won't free the
memory back to the system.

The first is that system calls are expensive operations and who's to say
that once you've instructed the system to free up a chunk of memory from the
process, you won't have to then claim that memory back from the system next
time the process calls malloc(). It's much more efficient just to maintain
that claimed memory in the free pool in case the process needs it later.

The second is that if you're running your process on an operating system
that supports virtual memory (and I suspect most people are, except those on
most embedded systems) it's really virtual memory rather than physical
memory that's been allocated. Which means that if the page isn't used for a
while (i.e. it doesn't contain any memory allocations in use by the process)
then it will be moved out of the working set of the process and possibly out
of physical memory until it's next needed. There's some more information on
virtual memory at:

http://www.cbmamiga.demon.co.uk/mpatrol/mpatrol_13.html

I did originally think about adding a feature to mpatrol to free the pages
back to the system when they weren't needed, but the extra work involved put
me off, not to mention the slowdown that would result, and in the end I
decided it was probably best to mimic what existing malloc libraries do.
Plus, it meant that I could go ahead and implement the NOFREE option, which
I think was a good tradeoff.

To answer your other question, mpatrol only asks the system for more memory
when it can't find a suitable-sized free allocation. It uses sbrk() on UNIX
by default but with USEMMAP it will use mmap(). On Windows it always uses
VirtualAlloc(). You can read a bit more about how mpatrol works at:

http://www.cbmamiga.demon.co.uk/mpatrol/mpatrol_42.html

mpatrol will only really tell you about the high-level memory allocations
with malloc(), operator new, etc. If you're wanting to profile the heap of
a process on a low-level page by page basis then you're probably better off
using a process analysis tool, especially if you're relying on using a
specific malloc library.

Hopefully that's answered your queries.

Thanks,

Graeme.


-----Original Message-----
From: mpatrol@yahoogroups.com [mailto:mpatrol@yahoogroups.com] On Behalf Of
tgraves_cs
Sent: 22 November 2006 16:27
To: mpatrol@yahoogroups.com
Subject: [mpatrol] mpatrol heap efficiency ?

Hello all,

I'm new to using mpatrol and I'm trying to use it to figure out heap
efficiency. Basically when the process mallocs memory, reserves it,
and then when it frees the memory and when it can actually free
blocks/pages back to be used by everyone else.

I see in the mptrace tool it tells you when it reserves it:
12 free 20 0x093E71AC 16 1 0 0
13 alloc 21 0x093E71AC 16 1 16
reserve 0x093E9000 98304
14 alloc 22 0x093E7904 96000 2 96016
reserve 0x09401000 98304


But I don't see it listing when it frees it back to general use by
everyone.

Does anyone know if mpatrol does this? How does it figure out when it
reserves it? Does it wrap sbrk?

Thanks,
Tom




To Post a message, send it to: mpatrol@egroups.com

To Unsubscribe, send a blank message to: mpatrol-unsubscribe@egroups.com
Yahoo! Groups Links