Yes, it depends to the Core i5's you want to compare to the i3's. If you're comparing the i3 to one of the i5-6xx series, then there is no big difference (the i5-6xx are duals with hyper threading, just like the i3's), same amount of L3 cache etc.
But if you're comparing to the i5-7xx CPU's (there's actually two of them, the i5-750 and 760), then there is a bigger difference. The i5-7xx have 8MB L3 cache, opposite to those 4MB L3 on the i3's and i5-6xx's, and are true quad cores, thus giving them a bigger advantage in multithreaded apps.