How to improve my wireless network performance

Changing routers isn't going to do much good. Check and see if there are other routers that are on the same channel. If there are change channels and that will help on your performance. Also remember that no matter how fast your router is your online speed will never be faster than your internet connection that you stated. You can also check and see what other kinds of things are in the signal path like motors, microwaves etc. Those will also create interference.

Playing video games over Wi-Fi will generate a lot of lag period especially when the Wi-Fi is being tasked by 2-3 computers doing the same thing at the same time.

one solution is wire the connection (you could try powerline products)
another move the router upstairs being in the basement is probably the worst scenario for Wi-Fi in a house .
you could place a repeater at each floor levels in the house to increase the Wi-Fi signals , but this will probably not do much for your gaming lag

11n 2.4g runs usually from 450 to 600Mbps, 800Mbps on 5g (varies from router brand to brand)
11ac can reach much faster speeds.. for example my netgear X4S AC2600 can do 11ac 2.4g @ 800Mbps and 1733Mbps @ 5g

Your Linksys E2500 according to web site a max speed of 2.4 GHz, 5 GHz @ 300Mpbs
I would say you first problem of order is getting a better Router, then relocation for optimal Wi-Fi dispersion.

Yeah, my first guess would be that your problem is 3 of you trying to game simultaneously over WiFi. Wireless communications don't handle collisions as well as wired. With wired ethernet, originally if there was a collision (two computers trying to transmit on the same ethernet network at the same time), they would stop, wait a random amount of time, then try to transmit again. The randomness meant there probably wouldn't be another collision, and the high speed of a wired connection meant the first one to transmit was usually finished by the time the second one transmitted. (Of course all this is antiquated now - modern switches assign each device their own dedicated port, and send transmitted data to only the port of the recipient instead of to every port. If multiple data streams need to go to the same port (e.g. sent to the Internet), the switch combines the data streams without collisions.)

Not so for WiFi. You can't give each WiFi device its own dedicated channel (yet). All WiFi devices transmit over the same channel, so broadcasts from one computer represents noise and collisions for broadcasts from other computers. WiFi includes a lot of redundancy and error correction coding to compensate for this. But if the original signal isn't noise-free, doing the error correction to reconstruct the signal takes some time. That's where the lag is coming from. A blink of an eye so is imperceptible when web browsing. But it makes a big difference in competitive gaming.

802.11ac is getting to the point where each device is getting its own channel. The higher-end devices with lots of antennas use a separate radio on each antenna to "steer" the radio transmission at each individual device. This greatly reduces the noise it picks up from other devices, and best of all this can be done simultaneously in multiple directions so multiple devices can communicate this way without interfering with each other. (It works like how your two ears can distinguish two different sounds coming from your left side and right side, even though the sound waves of the two are mixed together at both ears. Your brain can use the difference in sound arrival time to separate out what's coming from the left from what's coming from the right and you can "tune in" to one while ignoring the other.)

It's still pretty rudimentary technology, at least for home routers. The military has been using this since the early 1900s (phased array radios and radars), and the same principle is used to design directional antennas like a Yagi. Most 802.11ac routers only have 2 or 3 radios (advertised as 2x2 or 3x3), though a few have more. Their spatial separation capability is not yet very good either. I expect the technology will really come into its own with whatever standard comes after 802.11ac using the 60-70 GHz band.

So yeah, even though your Internet speed is nowhere near maxing out your WiFi speed, a good 802.11ac router should help (assuming each computer has a 802.11ac wireless card). But not as much as running a wired connection to each computer, or if you wait a few years for whatever comes after 802.11ac.