OK, a lot of people is mixing their concepts here and not one yet has addressed the real advantage of a 64 bit CPU, first what do the 32/64 bit size refers to? it refers to the register size, the procesor stores numeric values in those (generally instruction input/output or memory addresses, it depends on the architecture if this registers are special purpose or general, and how many there are of each type), registers are memory, the fastest kind of memory in the computer and also the most expensive one, and there are very few of them (about 15 on modern x86-64 procesors, which are all modern intel and AMD chips), the speed of reading from memory is orders of magnitude slower than reading from registers.
So there are several advantages to having a longer register size, the first and most obvious is that you can address a much larger memory space whitout any crazy tricks (which can be totally done, but it is tricky and impacts performance), the second obvious advantage is that you have much larger 'room' for operations: imagine you are adding two numbers that surpass 2^32 (the largest integer that can re represented in a 32 bit register) then you have to either give up and say that the result is an overflow or deal with the result in a cumbersome way in two registers, or worse, in memory, also if your program doesn't needs 64 bit registers you can use a single register as two of the old 32 bit registers! this trick alone can produce massive performance gains in compilers that use it.