Even before the ENIAC ran its first test, Mauchly and Presper were very aware of its shortcomings. So was John Von Neumann, whom many of you have heard about from the expression "Von Neumann Architecture" (although he received too much personal credit for what was a group effort). At any rate, the EDVAC was the first expression of this architecture, although Mauchly and Presper left the University of Pennsylvania where it was being built in 1946, before the computer was finished.
At that time, there were several major issues with the ENIAC. Sure, it was fast. But it had very little storage. More than that, it had to be reprogrammed by re-wiring it, which could take hours or even days, and it was inherently unreliable because the computer used so many vacuum tubes. In addition to being unreliable, vacuum tubes also used a lot of power, required a lot of space, and generated a lot of heat. Clearly, minimizing their use would have multiple advantages.
There were two important conceptual changes (one of which was revolutionary) on the EDVAC that seem very obvious today. For one, it was binary rather than decimal, like the ENIAC, and this was much more efficient. Also, rather than rewiring the machine every time you wanted to change the "program," the EDVAC introduced the idea of storing the program in memory, just as if it were data. This is what we do today. We do not, after all, have separate RAM areas for applications and for their data (although L1 caches typically operate this way). The processor knows, based on the context in which the memory was accessed, whether it is data or an executable.
In addition, memory no longer consisted of vacuum tubes, but was stored as electrical impulses in mercury. The mercury delay line was 100 times more efficient in terms of the electronics necessary to store data and made much larger amounts of memory feasible and more reliable.
The EDVAC was a major advance, and proved very useful until it was retired in 1960. It was a binary stored-program computer, which could be programmed much more quickly than the ENIAC could. It was also much smaller, weighing less than nine tons, and consumed "only" 56 kilowatts of power. Even still, our two heroes were not done yet.