Why does WinXP use more heap?

G

Guest

Guest
Archived from groups: microsoft.public.windowsxp.perform_maintain (More info?)

I'm working with a test program that exercises several components of our
product. It is built with Visual C++ .NET (unmanaged C++). Many threads, many
DLLs, far too many objects on the heap.

Under Windows 2000 Pro WS SP4, the program grinds away for a half-hour or so
at 99% CPU. Over time, the "private bytes" count for the process slowly grows
to around 1GB and levels out. It eventually produces a few hundred output
files that have the expected output.

Under Windows XP Pro SP2, the same program on the same hardware with the
same test case grows its "private bytes" count to 2GB. Many threads fail with
errors indicating that malloc or new was unable to allocate space on the heap
- not surprising, since we have only 2GB of virtual address space.

What could account for the extra memory usage seen when the program runs on
XP?

I understand that XP can use a "low-fragmentation heap" algorithm, but I've
verified that it is not enabled for the default heap we're using. XP also
adds an extra byte to each allocation request as part of a security
improvement - that is not sufficient to account for the huge difference.

Do you know of other significant differences in the heap managers for these
platforms?
 
G

Guest

Guest
Archived from groups: microsoft.public.windowsxp.perform_maintain (More info?)

Memory management in XP is just a wee bit different compared to Win2K.

Try running with the /3gb switch in boot.ini. Not only will that bump the
application address space up to 3G but it also causes the paging algorithms
to revert to Win2K style.

You may also want to try http://support.microsoft.com/?kbid=286470

--
Walter Clayton
Any technology distinguishable from magic is insufficiently advanced.


"MEB" <MEB@discussions.microsoft.com> wrote in message
news:92A208EA-4B69-41E4-BC6F-CB56021B6A0D@microsoft.com...
> I'm working with a test program that exercises several components of our
> product. It is built with Visual C++ .NET (unmanaged C++). Many threads,
> many
> DLLs, far too many objects on the heap.
>
> Under Windows 2000 Pro WS SP4, the program grinds away for a half-hour or
> so
> at 99% CPU. Over time, the "private bytes" count for the process slowly
> grows
> to around 1GB and levels out. It eventually produces a few hundred output
> files that have the expected output.
>
> Under Windows XP Pro SP2, the same program on the same hardware with the
> same test case grows its "private bytes" count to 2GB. Many threads fail
> with
> errors indicating that malloc or new was unable to allocate space on the
> heap
> - not surprising, since we have only 2GB of virtual address space.
>
> What could account for the extra memory usage seen when the program runs
> on
> XP?
>
> I understand that XP can use a "low-fragmentation heap" algorithm, but
> I've
> verified that it is not enabled for the default heap we're using. XP also
> adds an extra byte to each allocation request as part of a security
> improvement - that is not sufficient to account for the huge difference.
>
> Do you know of other significant differences in the heap managers for
> these
> platforms?
 
G

Guest

Guest
Archived from groups: microsoft.public.windowsxp.perform_maintain (More info?)

Thanks! I didn't know that /3gb caused the paging algorithms to revert to
Win2K style - I'd really appreciate a link to the source of that info.
(Unfortunately, we may not be able to require our customers to reconfigure
boot.ini to use our product...)

We still need to know what's causing WinXP to use twice as much heap in our
case.

The 286470 link points to information about pageheap.exe, which is a nifty
tool for detecting heap corruption. (I think the tool makes the program under
test actually use *more* memory, though.)
---------------

"Walter Clayton" wrote:

> Memory management in XP is just a wee bit different compared to Win2K.
>
> Try running with the /3gb switch in boot.ini. Not only will that bump the
> application address space up to 3G but it also causes the paging algorithms
> to revert to Win2K style.
>
> You may also want to try http://support.microsoft.com/?kbid=286470
>
> --
> Walter Clayton
> Any technology distinguishable from magic is insufficiently advanced.
>
>
> "MEB" <MEB@discussions.microsoft.com> wrote in message
> news:92A208EA-4B69-41E4-BC6F-CB56021B6A0D@microsoft.com...
> > I'm working with a test program that exercises several components of our
> > product. It is built with Visual C++ .NET (unmanaged C++). Many threads,
> > many
> > DLLs, far too many objects on the heap.
> >
> > Under Windows 2000 Pro WS SP4, the program grinds away for a half-hour or
> > so
> > at 99% CPU. Over time, the "private bytes" count for the process slowly
> > grows
> > to around 1GB and levels out. It eventually produces a few hundred output
> > files that have the expected output.
> >
> > Under Windows XP Pro SP2, the same program on the same hardware with the
> > same test case grows its "private bytes" count to 2GB. Many threads fail
> > with
> > errors indicating that malloc or new was unable to allocate space on the
> > heap
> > - not surprising, since we have only 2GB of virtual address space.
> >
> > What could account for the extra memory usage seen when the program runs
> > on
> > XP?
> >
> > I understand that XP can use a "low-fragmentation heap" algorithm, but
> > I've
> > verified that it is not enabled for the default heap we're using. XP also
> > adds an extra byte to each allocation request as part of a security
> > improvement - that is not sufficient to account for the huge difference.
> >
> > Do you know of other significant differences in the heap managers for
> > these
> > platforms?
>
>
 
G

Guest

Guest
Archived from groups: microsoft.public.windowsxp.perform_maintain (More info?)

That bytes. MS has been rewriting things again. :-/

I have a favorite that referenced the change, unfortunately they've
rewritten it. The closest I can come now is an indirect reference:
http://www.microsoft.com/whdc/driver/kernel/XP_kernel.mspx
Specifically:

Improved Terminal Server and Network Server Scaling
The number of system Page Table Entries has been increased to a maximum of
approximately 1.3 GB, of which just under 1 GB is virtually contiguous. This
is approximately twice as many PTEs as were available in Windows 2000 and
six times more than Windows NT 4.0. This is subject to system configuration,
such as RAM, registry switches, and other factors. Currently, a system must:

• Be booted without the /3GB switch.


<>
....

which implies several things; i.e. fewer PTE's means smaller footprints and
some of the deferred physical memory allocations become potentially
problematic. There's also the possibility that the original XP documentation
wasn't correct with regard the total scope of impact when using the /3GB
switch.

Yes, I know that expecting the client base to start using the /3GB switch is
*not* a viable option. However, it might give you the opportunity to see
what you need to specifically tweak to get the app to run on XP.

There are a couple of patches in the memory leak area with regard SP2 but I
figured you'd be running on a fully patched system. If not, try pursuing
that angle.

--
Walter Clayton
Any technology distinguishable from magic is insufficiently advanced.


"MEB" <MEB@discussions.microsoft.com> wrote in message
news:F62E04B7-F744-4E34-8F67-7520C4F9A592@microsoft.com...
> Thanks! I didn't know that /3gb caused the paging algorithms to revert to
> Win2K style - I'd really appreciate a link to the source of that info.
> (Unfortunately, we may not be able to require our customers to reconfigure
> boot.ini to use our product...)
>
> We still need to know what's causing WinXP to use twice as much heap in
> our
> case.
>
> The 286470 link points to information about pageheap.exe, which is a nifty
> tool for detecting heap corruption. (I think the tool makes the program
> under
> test actually use *more* memory, though.)
> ---------------
>
> "Walter Clayton" wrote:
>
>> Memory management in XP is just a wee bit different compared to Win2K.
>>
>> Try running with the /3gb switch in boot.ini. Not only will that bump the
>> application address space up to 3G but it also causes the paging
>> algorithms
>> to revert to Win2K style.
>>
>> You may also want to try http://support.microsoft.com/?kbid=286470
>>
>> --
>> Walter Clayton
>> Any technology distinguishable from magic is insufficiently advanced.
>>
>>
>> "MEB" <MEB@discussions.microsoft.com> wrote in message
>> news:92A208EA-4B69-41E4-BC6F-CB56021B6A0D@microsoft.com...
>> > I'm working with a test program that exercises several components of
>> > our
>> > product. It is built with Visual C++ .NET (unmanaged C++). Many
>> > threads,
>> > many
>> > DLLs, far too many objects on the heap.
>> >
>> > Under Windows 2000 Pro WS SP4, the program grinds away for a half-hour
>> > or
>> > so
>> > at 99% CPU. Over time, the "private bytes" count for the process slowly
>> > grows
>> > to around 1GB and levels out. It eventually produces a few hundred
>> > output
>> > files that have the expected output.
>> >
>> > Under Windows XP Pro SP2, the same program on the same hardware with
>> > the
>> > same test case grows its "private bytes" count to 2GB. Many threads
>> > fail
>> > with
>> > errors indicating that malloc or new was unable to allocate space on
>> > the
>> > heap
>> > - not surprising, since we have only 2GB of virtual address space.
>> >
>> > What could account for the extra memory usage seen when the program
>> > runs
>> > on
>> > XP?
>> >
>> > I understand that XP can use a "low-fragmentation heap" algorithm, but
>> > I've
>> > verified that it is not enabled for the default heap we're using. XP
>> > also
>> > adds an extra byte to each allocation request as part of a security
>> > improvement - that is not sufficient to account for the huge
>> > difference.
>> >
>> > Do you know of other significant differences in the heap managers for
>> > these
>> > platforms?
>>
>>