Search the web
Sign In
New User? Sign Up
mpatrol · mpatrol library discussion group
? Already a member? Sign in to Yahoo!

Yahoo! Groups Tips

Did you know...
Hear how Yahoo! Groups has changed the lives of others. Take me there.

Best of Y! Groups

   Check them out and nominate your group.
Having problems with message search? Fill out this form to ensure your group is one of the first to be migrated to the new message search system.

Messages

  Messages Help
Advanced
mpatrol heap efficiency ?   Message List  
Reply | Forward Message #1029 of 1198 |
RE: [mpatrol] mpatrol heap efficiency ?

thanks much for your response, off hand do you know
any process analysis tools that would give me this
information?

Tom



--- Graeme Roy <graeme@...> wrote:

> Hi Tom,
>
> The short answer is that it doesn't free the memory
> back to the system, and
> neither do most other malloc libraries.
>
> And now for the long answer...
>
> For systems using sbrk() (which mpatrol uses on UNIX
> by default), it would
> be virtually impossible to return the memory back to
> the system due to the
> fragmentation of the heap. The only way you could
> really do this would be
> to get the malloc library to free each of the pages
> it allocated in reverse
> order by calling sbrk() with a negative value, but
> that would also require
> the user's program to do frees() in the same way.
> I'm not saying it's
> impossible to do because I'm sure some programs
> would lend themselves to
> that, but in most cases it would be impossible and
> malloc libraries have to
> handle the general case.
>
> For memory allocated by mmap() (which mpatrol uses
> with USEMMAP) or
> VirtualAlloc() on Windows, the situation is slightly
> less complicated
> because the allocated pages could in theory be freed
> in any order if there
> are no more malloced allocations located in them.
>
> However, there are two main reasons why malloc
> libraries won't free the
> memory back to the system.
>
> The first is that system calls are expensive
> operations and who's to say
> that once you've instructed the system to free up a
> chunk of memory from the
> process, you won't have to then claim that memory
> back from the system next
> time the process calls malloc(). It's much more
> efficient just to maintain
> that claimed memory in the free pool in case the
> process needs it later.
>
> The second is that if you're running your process on
> an operating system
> that supports virtual memory (and I suspect most
> people are, except those on
> most embedded systems) it's really virtual memory
> rather than physical
> memory that's been allocated. Which means that if
> the page isn't used for a
> while (i.e. it doesn't contain any memory
> allocations in use by the process)
> then it will be moved out of the working set of the
> process and possibly out
> of physical memory until it's next needed. There's
> some more information on
> virtual memory at:
>
>
http://www.cbmamiga.demon.co.uk/mpatrol/mpatrol_13.html
>
> I did originally think about adding a feature to
> mpatrol to free the pages
> back to the system when they weren't needed, but the
> extra work involved put
> me off, not to mention the slowdown that would
> result, and in the end I
> decided it was probably best to mimic what existing
> malloc libraries do.
> Plus, it meant that I could go ahead and implement
> the NOFREE option, which
> I think was a good tradeoff.
>
> To answer your other question, mpatrol only asks the
> system for more memory
> when it can't find a suitable-sized free allocation.
> It uses sbrk() on UNIX
> by default but with USEMMAP it will use mmap(). On
> Windows it always uses
> VirtualAlloc(). You can read a bit more about how
> mpatrol works at:
>
>
http://www.cbmamiga.demon.co.uk/mpatrol/mpatrol_42.html
>
> mpatrol will only really tell you about the
> high-level memory allocations
> with malloc(), operator new, etc. If you're wanting
> to profile the heap of
> a process on a low-level page by page basis then
> you're probably better off
> using a process analysis tool, especially if you're
> relying on using a
> specific malloc library.
>
> Hopefully that's answered your queries.
>
> Thanks,
>
> Graeme.
>
>
> -----Original Message-----
> From: mpatrol@yahoogroups.com
> [mailto:mpatrol@yahoogroups.com] On Behalf Of
> tgraves_cs
> Sent: 22 November 2006 16:27
> To: mpatrol@yahoogroups.com
> Subject: [mpatrol] mpatrol heap efficiency ?
>
> Hello all,
>
> I'm new to using mpatrol and I'm trying to use it to
> figure out heap
> efficiency. Basically when the process mallocs
> memory, reserves it,
> and then when it frees the memory and when it can
> actually free
> blocks/pages back to be used by everyone else.
>
> I see in the mptrace tool it tells you when it
> reserves it:
> 12 free 20 0x093E71AC 16 1
> 0 0
> 13 alloc 21 0x093E71AC 16
> 1 16
> reserve 0x093E9000 98304
> 14 alloc 22 0x093E7904 96000
> 2 96016
> reserve 0x09401000 98304
>
>
> But I don't see it listing when it frees it back to
> general use by
> everyone.
>
> Does anyone know if mpatrol does this? How does it
> figure out when it
> reserves it? Does it wrap sbrk?
>
> Thanks,
> Tom
>
>
>
>
> To Post a message, send it to: mpatrol@egroups.com
>
> To Unsubscribe, send a blank message to:
> mpatrol-unsubscribe@egroups.com
> Yahoo! Groups Links
>
>
>
>
>




________________________________________________________________________________\
____
Do you Yahoo!?
Everyone is raving about the all-new Yahoo! Mail beta.
http://new.mail.yahoo.com



Mon Nov 27, 2006 3:01 pm

tgraves_cs
Offline Offline
Send Email Send Email

Forward
Message #1029 of 1198 |
Expand Messages Author Sort by Date

Hello all, I'm new to using mpatrol and I'm trying to use it to figure out heap efficiency. Basically when the process mallocs memory, reserves it, and then...
tgraves_cs
Offline Send Email
Nov 22, 2006
4:36 pm

Hi Tom, The short answer is that it doesn't free the memory back to the system, and neither do most other malloc libraries. And now for the long answer... For...
Graeme Roy
graemeroy
Offline Send Email
Nov 22, 2006
7:25 pm

thanks much for your response, off hand do you know any process analysis tools that would give me this information? Tom ... ...
Tom Graves
tgraves_cs
Offline Send Email
Nov 27, 2006
3:19 pm

Hi Tom, Assuming you're wanting more low-level memory usage statistics, I can't really recommend one particular tool because it depends what system you're...
graeme@...
graemeroy
Offline Send Email
Nov 27, 2006
4:26 pm

Hey Graeme, thanks again. I'm running on linux. I've used strace for some testing and small programs, but was hoping there was an existing tool to give me nice...
Tom Graves
tgraves_cs
Offline Send Email
Nov 27, 2006
5:11 pm
Advanced

Copyright © 2009 Yahoo! Inc. All rights reserved.
Privacy Policy - Terms of Service - Guidelines - Help