there's a lot of things that could factor in with internet connection.
1. how fast is your ISP's internet supposed to be?
2. what type of internet do you have, cable, dsl, fiber?
3. are you playing on wireless/wifi?
4. if on wifi, how far away is the router/modem?
5. if on wifi, what does the router/modem support, G, N, AC?
6. if on wifi, is the xbox blocked by its surroundings?
7. if you are on wifi, have you tried using ethernet?