Software to "oc" 32bit 4gb memory

[ppv]

ive heard something about "ocing" ur 32 bit os to be able to have larger amount of rams.. does anyone know?
thx

 

[JDFan]

No can do - A 32 bit OS can only store 4GB of addresses for memory since that is the largest number that can be stored in a 32 bit space so the most memory a 32 bit OS can use is 4GB total including the amount set aside for the System memory and the video card pages so actual useable memory on a 32 bit os is somewhere between 3.25 and 3.5 GB. to address more memory you need to change the OS to a 64 bit system.

 

[gamerk316]

2^32 = 4GB
2^64 = 16EiB (Exibytes)

 

[roonj]

Designated driver?

 

[mikrev007]

PAE. How you do it depends on your OS. Some dudes have made patches for vista/win7 to make them go above 4GB.

Several ram disk/cache applications will make use of ram above 4GB.

The 2^32 and 2^64 formulas fail somewhat.

 

[ppv]

ok guys! thx a lot!

 

[gamerk316]

PAE. How you do it depends on your OS. Some dudes have made patches for vista/win7 to make them go above 4GB.

Several ram disk/cache applications will make use of ram above 4GB.

The 2^32 and 2^64 formulas fail somewhat.

Not really. Basic features of the X86 architecture; a lot of those "patches" simply allow the OS to see the extra RAM, but it never gets used.

2^32 is how much 32-bit Windows can physically count to. Period, end discussion. Anyone who doesn't understand that concept has no idea how a OS actually functions.-

PAE is a special case: The X86 architecture has a 36 bit mode (a holdover from the days when servers operated at 20/36 bits instead of the 16/32 homes users are accustomed too), which Windows takes advantage of by using multiple page tables. (IE, each page table would have exactly 2^32 capacity, so 4 page tables = 4 * 4GB = 16GB of total space). The problem is that because each program needs to be associated with a page table, no single app can EVER use more then the 4GB of space at one time. Farthermore, programs and drivers must be coded in such a way to make sure the proper page table is always accessed, otherwise you run into lots of memory errors. PAE is a workaround, not a solution.

 

[mikrev007]

PAE is not special. PAE has been a part of x86 for many years. Since Windows 2000, PAE has been a part of Windows. The only reason why xp/vista/7 is not able to use more than 4GB is because Microsoft made it so.

And this is not really just a "Windows xp/vista/7"-discussion. The OP asked how you could break the 4GB barrier, because some people do, and I gave the reason. I know you dislike it, but it is the truth.

And how much a single application can use is irrelevant here. The question was how much ram the OS could use as a total.

"Farthermore, programs and drivers must be coded in such a way to make sure the proper page table is always accessed, otherwise you run into lots of memory errors. PAE is a workaround, not a solution."

Is technical not true. It is 100% false when talking about applications.

 

[mikrev007]

2^32 is how much 32-bit Windows can physically count to. Period, end discussion. Anyone who doesn't understand that concept has no idea how a OS actually functions.-

You have proven so many times (here and) in the past that you don't have much knowledge in how memory access/management works in x86, so give it a rest.

 

[mikrev007]

a lot of those "patches" simply allow the OS to see the extra RAM, but it never gets used.

I am referring to the guy who found out that by patching just a few bytes in the vista kernel, you can remove the 4GB-limitation implemented by Microsoft.

 

[gamerk316]

PAE is not special. PAE has been a part of x86 for many years. Since Windows 2000, PAE has been a part of Windows. The only reason why xp/vista/7 is not able to use more than 4GB is because Microsoft made it so.

You started off OK, then went way off track. Yes, PAE has been around for a while, but the 4GB limit has nothing to do with Microsoft, as EVERY SINGLE 32-BIT OS HAS THE SAME RESTRICTION!

And this is not really just a "Windows xp/vista/7"-discussion. The OP asked how you could break the 4GB barrier, because some people do, and I gave the reason. I know you dislike it, but it is the truth.

Now when did I even mention a specific OS? And the only valid example you gave was PAE, which as I pointed out, has serious limitations. Aside from that, you gave BS with no links whatsoever.

And how much a single application can use is irrelevant here. The question was how much ram the OS could use as a total.

Depending on how he wants to use the extra RAM, it could be perfectly valid. I simply pointed out the flaw of using PAE: That no single app can use more then 4GB of RAM at any one time.

"Farthermore, programs and drivers must be coded in such a way to make sure the proper page table is always accessed, otherwise you run into lots of memory errors. PAE is a workaround, not a solution."

Is technical not true. It is 100% false when talking about applications.

Applications may not run into trouble, but drivers will (and do) if not coded properly. I've worked with more then a few shoddy drivers over my lifetime that failed with PAE (National Instruments is surprisingly bad in that area...).

You have proven so many times (here and) in the past that you don't have much knowledge in how memory access/management works in x86, so give it a rest.

right...would you care to explain how the X86 32-bit addressing mode, which contains a string of exactly 32 0/1 combinations, which has exactly 4GB worth of unique addresses, can mape to an address higher then its physical range while operating in 32 bit mode? Much less fit that address (which is greater then 32-bits) into a 32-bit Register? If you can, I'd REALLY love to hear about the miracle you came up with, as you've apparently come up with a new form of Binary code.

I'll even let you use any 32-bit OS for your example, as I'm sure theres plenty of OS's out there while would love to hear about your magic method for making RAM address space appear out of thin air.

I am referring to the guy who found out that by patching just a few bytes in the vista kernel, you can remove the 4GB-limitation implemented by Microsoft.

EDITED
Right, a guy...who somehow patched the entire Vista Kernal [How would you even get ahold of that anyways?]...with no provided link.

And BTW, its easy to make a patch to show more then 4GB installed on a 32-bit machine (that was already done by M$ after Vista came out), its another thing to have the OS be able to access it, and another to have an X86 CPU operating in 32-bit mode to be able to load, store, and use intructions on it. Its the same exact things as some of those "optimization" programs that make Task Manager's memory usage numbers look pretty (all they do is lower each processes address space, which means the page file is being paged to holy hell. But the TM numbers look good ), that people still fall for on a routine basis.

You claim that theres a way besides PAE to allow >4GB of data into 4GB of space, that somehow every single OS in existence doesn't know about. Some links would be wonderful.

/EDITED

 

[mikrev007]

You started off OK, then went way off track. Yes, PAE has been around for a while, but the 4GB limit has nothing to do with Microsoft, as EVERY SINGLE 32-BIT OS HAS THE SAME RESTRICTION!

right...would you care to explain how the X86 32-bit addressing mode, which contains a string of exactly 32 0/1 combinations, which has exactly 4GB worth of unique addresses, can mape to an address higher then its physical range while operating in 32 bit mode? Much less fit that address (which is greater then 32-bits) into a 32-bit Register? If you can, I'd REALLY love to hear about the miracle you came up with, as you've apparently come up with a new form of Binary code.

Try and research the stuff. Properly. If you do, you will stop asking these questions again and again and again.

 

[gamerk316]

I do, on a daily basis. Its my job.

And I note: You dodged my challenge.

 

[gamerk316]

Directly from Brian Madden's blog (Essentially: The goto authority of Virtualization). This blog was meant for another purpose (Terminal Application memory managment), but its the best explanation I've found yet.

http://www.brianmadden.com/blogs/brianmadden/archive/2004/02/19/the-4gb-windows-memory-limit-what-does-it-really-mean.aspx

There seems to be a lot of confusion in the industry about what's commonly called the Windows “4GB memory limit.” When talking about performance tuning and server sizing, people are quick to mention the fact that an application on a 32-bit Windows system can only access 4GB of memory. But what exactly does this mean?

By definition, a 32-bit processor uses 32 bits to refer to the location of each byte of memory. 2^32 = 4.2 billion, which means a memory address that's 32 bits long can only refer to 4.2 billion unique locations (i.e. 4 GB).

In the 32-bit Windows world, each application has its own “virtual” 4GB memory space. (This means that each application functions as if it has a flat 4GB of memory, and the system's memory manager keeps track of memory mapping, which applications are using which memory, page file management, and so on.)

This 4GB space is evenly divided into two parts, with 2GB dedicated for kernel usage, and 2GB left for application usage. Each application gets its own 2GB, but all applications have to share the same 2GB kernel space.

People who are unfamiliar with the real meaning behind the 4GB Windows memory limit often point out that certain versions of Windows (such as Enterprise or Datacenter editions) can actually support more than 4GB of physical memory. However, adding more than 4GB of physical memory to a server still doesn't change the fact that it's a 32-bit processor accessing a 32-bit memory space. Even when more than 4GB of memory is present, each process still has the normal 2GB virtual address space, and the kernel address space is still 2GB, just as on a normal non-PAE system.

However, systems booted /PAE can support up to 64GB physical memory. A 32-bit process can "use" large amounts of memory via AWE (address windowing extension) functions. This means that they must map views of the physical memory they allocate into their 2GB virtual address space. Essentially, they can only use 2GB of memory at a time.

Worse then I thought; I forgot to factor in Kernal space when I mentioned PAE...So its only 2GB per app, instead of the 4GB i mentioned above.

Linux also uses PAE, which is why 32-bit Linux can access 64GB of data (I'm not familiar with Linux's memory management, so I'm not sure exactly what schema is used to implement PAE on that platform)

So in conclusion, the only way to expand beyond 4GB in 32-bit Windows is via PAE. All other methods simply do not work. At the end of the day, the X86 CPU needs a pointer to an address, and each X86 32-bit register can hold EXACTLY 32-bits of data, which contains one of ~4.2 Billion (4GB) unique combinations of 1's and 0's.

 

[Scotteq]

Try and research the stuff. Properly. If you do, you will stop asking these questions again and again and again.

Christ...

Stupid PAE argument, take nine hundred...

Mike - The OP can't even complete a sentence in English. You expect him to alter baseline params in the kernel of an operating system in order that his computer may display 4GB? How is that helping anyone? And you expect him to be able to troubleshoot this... how, exactly? You expect him to support this... how, exactly? You expect this guy to understand whether his system bus supports DAC or not? Or understand what LME means in the context of a driver, and why it matters here?? Pretty damned high expectations, if you ask me. I swear, sometimes it feels like you must lurk the forums here just waiting for a chance to have this exact argument.

Thank you for pointing out that - Yes, Indeed it is possible. Congratulations on earning a geek point for your troubles.


It's also a waste of time and effort, and a solution to what by now should be a nearly non-existant problem. (i.e. - Either there is valuable/irreplacable legacy equipment involved, or the tech is too lazy/stupid/dogmatic to change). If that much address space is required, then go 64 bit and be done with it. Especially since we're now at the second generation - (or 3rd, if you count XP64) - of perfectly viable, supported, and publicly available 64 bit Operating Systems. They run 32 bit code perfectly well, and by going this route address space issues simply does not exist at a desktop level.

It's *so* simple: Use the proper version of the OS, and there will not be a problem.

 

[mikrev007]

gamerk316, you still have no clue.

Scotteq, the OP asked a question, and he got an answer. How hard is that to understand?

 

[gamerk316]

^^ Give it up mikrev. You've been proven wrong, and you've yet to explain yourself when given the chance.

 

[mikrev007]

The OP specifically asked about 32bit OSes. He did not ask about the advantages of running a 64bit OS with 64bit applications.

We are talking about addressing physical memory and then you quote something about the virtual address space. You prove that you don't understand the subject by doing that.

The patch I mentioned has been discussed in here several times.

 

[mikrev007]

I do, on a daily basis. Its my job.

Directly from Brian Madden's blog (Essentially: The goto authority of Virtualization). This blog was meant for another purpose (Terminal Application memory managment), but its the best explanation I've found yet.

Btw, I cannot take you seriously when that is your best source of information.

Edit: when doing reserarch, the first you do is download development manuals at Intel and AMD. Here you get correct information and lots of it.

 

[gamerk316]

The OP specifically asked about 32bit OSes. He did not ask about the advantages of running a 64bit OS with 64bit applications.

We are talking about addressing physical memory and then you quote something about the virtual address space. You prove that you don't understand the subject by doing that.

The patch I mentioned has been discussed in here several times.

Now when did I even mention X64?

Secondly, this supposed "patch" you refer too, I've never seen anything legitamate, here or otherwise. And again, I challenge you for a link...

As for the Virtual point, its necessary to understand exactly how the OS maps virtual memory into physical memory.

By definition, a 32-bit processor uses 32 bits to refer to the location of each byte of memory. 2^32 = 4.2 billion, which means a memory address that's 32 bits long can only refer to 4.2 billion unique locations (i.e. 4 GB).

Referes to the X86 32-bit PHYSICAL address space that can be accessed. Regardless of how an OS stores/manipulates its data, a 32-bit CPU can only access 2^32 bits of address space at any point in time. That is, without using PAE, the CPU [Read, NOT the OS] can only access up to 4GB of Address Space.

In the 32-bit Windows world, each application has its own “virtual” 4GB memory space. (This means that each application functions as if it has a flat 4GB of memory, and the system's memory manager keeps track of memory mapping, which applications are using which memory, page file management, and so on.)

This 4GB space is evenly divided into two parts, with 2GB dedicated for kernel usage, and 2GB left for application usage. Each application gets its own 2GB, but all applications have to share the same 2GB kernel space.

Referes to 32-bit Windows VIRTUAL address space. Paging is used to allow for global virtual RAM usage of > 4GB, but the X86 Physical limit on address without PAE is still 32-bits.

However, systems booted /PAE can support up to 64GB physical memory.

By accessing the X86 36-bit mode, you can artifically extend the Physical Memory Space for 32-bit applications to 64GB.

A 32-bit process can "use" large amounts of memory via AWE (address windowing extension) functions. This means that they must map views of the physical memory they allocate into their 2GB virtual address space. Essentially, they can only use 2GB of memory at a time.

AWE essentially allows for programs to use >4GB of Virtual Address Space. I might toy with the API later tonight to see exactly how it works, but from what I've read, it basically creates multiple physical address spaces and maps it to physical RAM.


----------
At the end of the day, I assume the OP wants to extend his RAM usage beyond the 4GB limit. The only answer (short of going to 64-bits) is PAE, which has its own disadvantages. The 4GB limit is a function of all 32-bit processors, and no amount of supposed OS hacks is going to change that fact.

 

[gamerk316]

Btw, I cannot take you seriously when that is your best source of information.

Edit: when doing reserarch, the first you do is download development manuals at Intel and AMD. Here you get correct information and lots of it.

1: I actually have the origional X86 specification lying around my house somewhere. I'll be sure to quote it once I find it this week.

2: I also point out you yourself failed to quote said documentation.

3: I simply picked the first person (who also happens to be an expert in the field) who expalined the issue in plain english. I could throw about 50 more quotes your way if you want.

4: If an OS operates in 32-bit mode without using PAE [Its strict 32-bits], explain exactly how a >32-bit physical address can fit into an 32-bit X86 integer register.

Either start to explain yourself, provide links, or go away. Saying "theres a patch", not providing a link, or failing to explain yourself doesn't help prove your point.

 

[gamerk316]

http://www.intel.com/Assets/PDF/manual/253665.pdf [Im looking for older documentation, but its the best I found so far]

Section 2.1.3
The Intel386™ Processor (1985)
The Intel386 processor was the first 32-bit processor in the IA-32 architecture family.
It introduced 32-bit registers for use both to hold operands and for addressing. The
lower half of each 32-bit Intel386 register retains the properties of the 16-bit registers
of earlier generations, permitting backward compatibility. The processor also
provides a virtual-8086 mode that allows for even greater efficiency when executing
programs created for 8086/8088 processors.
In addition, the Intel386 processor has support for:
• A 32-bit address bus that supports up to 4-GBytes of physical memory
• A segmented-memory model and a flat memory model
• Paging, with a fixed 4-KByte page size providing a method for virtual memory
management
• Support for parallel stages

3.1.1 Intel® 64 Architecture
Intel 64 architecture adds IA-32e mode. IA-32e mode has two sub-modes.
These are:

• Compatibility mode (sub-mode of IA-32e mode) — Compatibility mode
permits most legacy 16-bit and 32-bit applications to run without re-compilation
under a 64-bit operating system. For brevity, the compatibility sub-mode is
referred to as compatibility mode in IA-32 architecture. The execution
environment of compatibility mode is the same as described in Section 3.2.
Compatibility mode also supports all of the privilege levels that are supported in
64-bit and protected modes. Legacy applications that run in Virtual 8086 mode or
use hardware task management will not work in this mode.

Compatibility mode is enabled by the operating system (OS) on a code segment
basis. This means that a single 64-bit OS can support 64-bit applications running
in 64-bit mode and support legacy 32-bit applications (not recompiled for
64-bits) running in compatibility mode.

Compatibility mode is similar to 32-bit protected mode. Applications access only
the first 4 GByte of linear-address space. Compatibility mode uses 16-bit and 32-
bit address and operand sizes. Like protected mode, this mode allows applications
to access physical memory greater than 4 GByte using PAE (Physical
Address Extensions).

• 64-bit mode (sub-mode of IA-32e mode) — This mode enables a 64-bit
operating system to run applications written to access 64-bit linear address
space. For brevity, the 64-bit sub-mode is referred to as 64-bit mode in IA-32
architecture.

64-bit mode extends the number of general purpose registers and SIMD
extension registers from 8 to 16. General purpose registers are widened to 64
bits. The mode also introduces a new opcode prefix (REX) to access the register
extensions. See Section 3.2.1 for a detailed description.

64-bit mode is enabled by the operating system on a code-segment basis. Its
default address size is 64 bits and its default operand size is 32 bits. The default
operand size can be overridden on an instruction-by-instruction basis using a REX
opcode prefix in conjunction with an operand size override prefix.

REX prefixes allow a 64-bit operand to be specified when operating in 64-bit
mode. By using this mechanism, many existing instructions have been promoted
to allow the use of 64-bit registers and 64-bit addresses.

3.3.2 Paging and Virtual Memory
With the flat or the segmented memory model, linear address space is mapped into
the processor’s physical address space either directly or through paging. When using
direct mapping (paging disabled), each linear address has a one-to-one correspondence
with a physical address. Linear addresses are sent out on the processor’s
address lines without translation.

When using the IA-32 architecture’s paging mechanism (paging enabled), linear
address space is divided into pages which are mapped to virtual memory. The pages
of virtual memory are then mapped as needed into physical memory. When an operating
system or executive uses paging, the paging mechanism is transparent to an
application program. All that the application sees is linear address space.
In addition, IA-32 architecture’s paging mechanism includes extensions that
support:

• Page Address Extensions (PAE) to address physical address space greater than
4 GBytes.

• Page Size Extensions (PSE) to map linear address to physical address in
4-MBytes pages.

3.3.6 Extended Physical Addressing in Protected Mode
Beginning with P6 family processors, the IA-32 architecture supports addressing of
up to 64 GBytes (236 bytes) of physical memory. A program or task could not
address locations in this address space directly. Instead, it addresses individual linear
address spaces of up to 4 GBytes that mapped to 64-GByte physical address space
through a virtual memory management mechanism. Using this mechanism, an operating
system can enable a program to switch 4-GByte linear address spaces within
64-GByte physical address space.

The use of extended physical addressing requires the processor to operate in
protected mode and the operating system to provide a virtual memory management
system. See “36-Bit Physical Addressing Using the PAE Paging Mechanism” in
Chapter 3, “Protected-Mode Memory Management,” of the Intel® 64 and IA-32
Architectures Software Developer’s Manual, Volume 3A.

So now that I've quoted the intel tech manual on the subject: Proof or go away.

 

[mikrev007]

Right. I am not going to waste more of my time on this.

 

[Scotteq]

The OP specifically asked about 32bit OSes. He did not ask about the advantages of running a 64bit OS with 64bit applications.

Nobody mentioned anything about running 64 bit apps. It's about address space. Specifically: If you need that much, then use the appropriate OS (i.e. 64 bit) and skip the kludgy workarounds.

And thank you for showing that you do, indeed, wait for posts of this nature just to percipitate arguments.

 

[gamerk316]

Eh, I don't mind. I'm sure you've noticed Scott, I like arguing