XBOX 360 controller problem or not?

hello everyone!

I just bought a 360 controller for my pc.now my problem is when I try to use left or right analog stick in up or down direction,it goes diagonally by very small margin.so far this haven't affected my gaming experience at all but I don't know if it is a problem or not? I have other controller that does not behave like this.

I just don't understand that it is problem or it is the way 360 controller work.in games like devil may cry 4,when I try to move my character up or down then the character moves to diagonal direction very slightly but when i use my keyboard or other controller,it works fine.

should I ask for replacement?