Well, the Windows scheduler will use the HTT cores if all the other cores are busy doing work, which will happen whenever a kernel thread bumps your application. And if your thread stays on a HTT core for an extended period, it could be enough to cause a minor performance hit. That's a fringe case though, and I'd imagine clock speed effects are a bigger problem.
In the short term, you can bind your application to only use even numbered cores, which should eliminate HTT effects for the purposes of testing.