I want to try overclocking my CPU and I've started researching for it, but the one roadblock is this dang BIOS. The motherboard manual says to press delete when booting, but that does nothing. It's a Gigabyte GA-EX58-UD3R.
When the screen loads, try and wait for the BIOS screen saying 'Press Delete to enter Setup' or similar. Otherwise you could just spam press delete during the whole bootup session.
You are waiting too long to push delete most likely. Start pushing it once every second or so right when you turn the power on. On fast modern computers you only have a few seconds before it is too late.