Yes. that is ok. But it will require a normally booted up machine. Here the machine is hanging on first boot screen with the suspicious ram in.
Then you have no choice but to return the RAM which is obviously defective. Corsair will want to replace both sticks to insure that the whole 4gb is identical. That happened to me once, and I got them to replace only one stick, since I otherwise would not have any computer to use while waiting. In my case it worked out OK.
I the meantime, contact newegg, and explain the problem. They are very understanding, and will probably work something out.