Likely you will be fine. The only concern would be the connection between the switches. A example of a stupid way to hook them up to illustrate would be if you have 2 8 port switches and hook them together. Then you hook a single server to one and 7 user machines to the other. If all your traffic was from the user machines to the server all 7 machines would be limited to the port speed between the switches.
You really have understand your traffic flow so you reduce your bottlenecks as much as possible. Generally you can run all the ports on most switches at full speed and have no impact on the other ports. This means all ports can run at maximum speed simultaneously. So traffic between ports on the same switch have little or...