Sounds like you're probably using WEP security, which was broken (hacked) years ago. So it’s probably not a good idea to continue using it. But if you insist…
It might be that you're entering a passphrase rather than the actually hexadecimal value. WEP devices/routers sometimes allow you to enter a passphrase. But that passphrase is not the actually key. The passphrase is only used to GENERATE the key. And that key comes in two options; 40 and 104 bits (obviously the latter is slightly more secure). Unfortunately, you usually need to provide the generated key rather than the passphrase once you need to use it. And that’s why ppl get confused. They keep supplying the passphrase, while the system thinks you’re trying to pass the generated key (in hexadecimal) and so it chokes.
So whenever you setup a WEP-based router, it’s important to record the generated hexadecimal key, not just provide a passphrase and ignore the results. Or else (if supported) just provide the hexadecimal key directly. And record it! Or you'll end up in this conundrum.
If you don’t remember the key, then you might be forced to reset the router and start over.