yes, your cpu and gpu are lacking quite a bit for skyrim. There are people with 2500k's that have issues runing skyrim in certain areas maxed because of how cpu intense the game can be. At minimum i would say a 5770 for skyrim for the GPU judging from the benches ive seen, and higher depending on what type of budget you are working with and what other games you are interested in playing
btw; this isnt a laptop is it?
It seems there are many people happily running Skyrim on their E-450 and have posted gameplay videos. Search for HD 6320 or E-450 and Skyrim and see if it runs well enough for your needs. Just make sure Windows is set for performance and that your memory is running at the full rated speed.