video card problem in win 98 on a dual-boot system (win98 &

i have a dual-boot rig (win98 & w2k). i recently installed an additional ram on the 3rd mem slot for a total of 768mb (256x3). when i boot to win98, i get an error message on the display settings that goes like: 'The adapter type is incorrect, or the current settings do not work with your hardware.' I can't get the video card to display in 32-bit or higher res. when i boot to win2k i get no such error. then i removed the 3rd ram chip and booted to win98 and there was no more error. is this a win98 problem? why is it that win98's display settings can't work properly beyond 512mb ram? anybody have an idea on what's wrong, how to remedy this? thanks.
my setup: amd tbird 1ghz, abit kt7-raid mobo, 64mb radeon vivo, 768mb ram, 2-15ghz ibm ata-100 hd in raid 0, sound blaster live sound card