Does it occur outside the use of Word and Excel? If it's exclusive to these, what version are you using and what mouse do you have? Have you tried updating/reinstalling the drivers for the mouse?
I have version 2002 of Word and Excel and I don't get this. I have a basic optical USB mouse, Microsoft brand.
Edit: Now that I think about it, are you referring to the mouse arrow cursor or the highlight moving? That is, if you press up in Excel, the active cell moves up a row. If that is the case, that's normal behavior. I make great use of this all the time as sometimes, keyboard navigation can be faster than the mouse. Holding shift forms selections and holding control jumps around (between words or between paragraphs). If the keyboard arrow buttons move the mouse cursor, that's a different subject, as I'd only expect that kind of behavior with older games and console games.