Thanks RealBeast, you have been a great help.
I went with
http://www.leviton.com/OA_HTML/ProductDetail.jsp?partnumber=47603-C5§ion=39700&minisite=10251 since it was available @ Home Depot and I couldn't wait to order a proper patch panel online.
Each time I punched down a cable, I would plug a patch cable to the router, go to a wall socket and plug a patch cable to a PC and test the connection. It all seemed fine but I soon found out that I could not get any internet connection when I have multiple cables plugged to the router.
After much hassle I realized that I could only have 1 connection regardless which wall socket I use. If I use 2 or more sockets, none of the PC will have internet connection. After shutting down the router and power it back again, the LED light on the router Internet would turn yellow and I had to call Verizon to make the light go green.
I returned the Leviton product thinking it's faulty and decided to try your method of terminating the cable ends and use a switch. This time I made sure that I could plug multiple cables to the switch and still get internet on every PC and I did.
I thought I was finally done but apparently not. This morning the internet did not work again and I had to call Verizon, and they told me a neat trick to make the Internet LED light turn green. Shortly after I got back from work, the internet was not working again. Turn off and turn on the router yields a yellow light again and I had to use the trick Verizon tech told me to fix the problem.
So I don't really know what exactly is the problem here. I use a gigabit switch, with 6 cables connected to it and another 1 cable connected to router. Out the 6 cables, 3 are connected to gigabit devices, 1 connected to VoIP phone, and other 2 not connected to any devices. However, the router is a 10/100 Mb as I'm waiting for the new gigabit router to arrive.
According to the Verizon tech, the yellow light for Internet on the router means that there's internet coming to the house but it's not been communicated properly. So what exactly is the problem? The cable, the switch, or the router? Is it because I have too many cables/devices connected to the switch and therefore confusing the router in the process?