Sounds as if it may be a setting in the BIOS of some sort. I don't know exactly what would cause it but maybe default boot device, PS2 ports enabled when your using USB. Some device enabled on boot that is not physically there. That would be my guess if it has been doing it from the start of the build. Maybe try a BIOS update as well.
I mean can you press any keys when the flashing cursor is there? Do any commands pop up before this cursor? How long does the cursor sit there before you power it down? Possibly a bad power supply, not giving good amps on boot up? Do the fans on heatsink, case, GPU spin up? May even be something not grounded properly or not getting a good connection.
Just throwing ideas out there myself, from the details nothing pops out at me.