I'm using a 560 ti 1gb in a prefab dell motherboard. I want to push it further by cutting the end of the other 1x slot and putting a 16x in then SLIing them. Would it only barely increase performance from the small width of the bus? would the motherboard even support it?
If the slot cannot physically accommodate a full length PCI-E card, then it won't perform well at all. Cutting off the end of the slot to make it fit is also a horrible idea. You do realize that PCI-E x16 slots have a special clamp on them, that is to keep the heavier video cards secure into the board. Your 560Ti wouldn't likely stay secure in the slot for long without that clamp.
Furthermore, that motherboard is unlikely to support SLI. Dell and other OEMs are notoriously cheap with their motherboards. Extra features like SLI and Crossfire are generally not included except on top of the line desktops that are marketed as high end gaming machines. Their regular boards don't support SLI or Crossfire. The only Dell products that do support SLI and Crossfire are probably their Alienware products, which are notoriously overpriced. Your best bet is just to get a better single GPU if you really want a graphics upgrade.