Assuming the USB is actually 'bootable' then next you need to select the USB Drive as first priority boot device in the BIOS AND the USB port MUST be native; e.g. USB 3.0 ports are NOT native (yet). Further, as 'best practice' only use the native USB ports at the MOBO's I/O and NOT through a HUB or Case port(s).
Agreed, stick the USB stick in the back of the computer, nowhere else, and make sure the BIOS is set to use the USB device before anything else in the boot order.