I have seen many people who put entire programs on their desktop whether installed or not and that will slow down the system.
Im not certain that you can install a program on the desktop any more but in the early days of windows you could and it was a very bad thing to do as the GUI would run out of memory.
As for very large items such as full programs on the desktop, whether a full cd copy or a downloaded archive or how ever you get it there it can potentially slow the system as everything on the desktop remains in memory. So getting those items into permanant storage is recommended
On my system I generally scan the icons to make sure that I see the little arrow on them that signifys that they are a shortcut and someone hasnt put something huge on there. I also have several text notes in one corner that I reference often and dont want to have to look around for all of the time, but they are usually less than 10K in size.