First, why are you changing from HDMI to DVI on the same computer/monitor? You should only have one connection between the 2 of them at a time.
This sounds like your GPU (video card) assumes you have 2 monitors attached to it and it extends the desktop to the second monitor, only there isn't a second monitor. The icons are there, just not on the second monitor. I know this sounds confusing.
Just remove one of the connections and you will be fine. Good luck!!!
Thanks COLGeek for your help. The manual that came with the card doesn't tell you this so I just assumed you used both cables as per the diagram on the back of the manual.