Anything that requires silicon is very difficult so CPU, GPU, and RAM are out as they would all require access to a fab. I'd say the easiest thing to design and build these days would be a DVD drive since there is nothing too special in it, or maybe a motherboard since most of the design for that is a systems engineering problem of getting a large number of complicated prebuilt components to work well together rather than trying to engineer and build something from the ground up, and instead of having to create new features every two years to get people to buy the product again you just need to stick a new CPU socket on it and it will sell itself!
Some of the badly made case are like soft drinks can with wires to connect to ports and buttons.
Or, a shoebox.
You know, that could actually work, if you remember to vacuum it frequently and not stepping on it...
PSU sounds like fun...problem is getting to be competitive.
Well, you could try building watercooled PSUs, that are compatible with existing WC loops.
There was one manufacturer that did the WCing and made it look good, but they "overclocked" their PSUs (rebranded 850W as 1200W, which increased voltage fluctuation), did a clumsy job with the coolant circuit (PSU's radiator had to be connected to the PSU, and too much disconnect/reconnect would result in the circuit being bled dry), and overpriced it.
I think Pyree is right on the cases, they would be pretty straight forward and have a much better profit margin than most other parts. You would need access to a brake for metal bending, hydraulic shears to get large smooth cuts, and a way to cut out holes in the case, a milling machine would be preferable but there are other options.
For not involving incredibly elaborate technology, i think it is rather silly that we have yet to come out with great cases at reasonable prices and for custom cases you could probably make a killing if they were well designed.