There's a difference between cheap and inexpensive. As an example, non-overclocking boards are inexpensive compared to overclocking boards, but I wouldn't call all of them cheap. They can just do less stuff.
Get something that doesnt have a dozen pci slots, doesnt overclock, and doesnt have a dozen sata ports or a bunch of bells and whistles that you don't need. Micro-atx boards are also often cheaper from what I've seen, although they might be more cramped due to size.
Example: H87M-E : Made by asus, which many recommend as a place to look for decent boards (whether or not you think it is decent is for you to decide). Costs $100, which isnt terribly expensive compared to the type of things many often recommend (although be sure to research the board itself to check what people think of quality and if it has the features you need.) I didn't spend time looking for this board, but you get the idea. Only get what you need, don't pay extra for features you don't want (tends to be why I recommend MSI's G45 board whenever people go for the G45-Gaming, the extra features arent generally worth the extra stuff).
Don't get cheap core parts, but don't get stuff you don't need. Sure that RAM might be groundbreakingly fast, but you won't really get much benefits past a certain point and after that you just waste money. Or taking the case as an example; maybe the case has a fun looking led screen, but is that actually useful or would it be better to get something without that feature and put $$ into the gpu/board/cpu/cooler