It's the GPU in this case, most likely. It's an entry level GPU, not meant to play at ultra settings.
To verify, go into settings and put them all to low, even resolution if possible. If you gain performance, the GPU is the limit. Otherwise, it is something else in the system, most often the CPU.