1. Did ya uninstall before taking out old card and installing new ? If not, uninstall all things nVidia, clean registry of all nVidia entries, reboot and install anew.
2. Is PSU beefy enough and sufficient quality for the upgrade ?
https://www.nvidia.com/en-us/geforce/products/10series/geforce-gtx-1080-ti/
Thermal and Power Specs:
91 = Maximum GPU Temperature (in C)
250W = Graphics Card Power (W)
600 W = Recommended System Power (W)
One 6-pin, One 8-pin = Supplementary Power Connectors
if overclocking anything ya want more, not some Tis can pull well over 300 watts and that's before you open afterburner and raise the power limit by 50%.
3. Check that all monitor cables are tight, all screwed all the way in.
4. make sure both power cables to the card are all the way in and card is securely seated and 'clicked in".