You are looking at this wrong. A loss doesn't hurt Connor at all, he is still as big after the Diaz loss when he went up two weight classes. He is going over to another sport and fighting one of the best of all time. Connor isn't even a boxer. Anyone with half a brain knows Connor wins easy in a MMA fight.
Mayweather has everything to lose in this, if anyone is backing out its mayweather. Connor and the UFC don't have anything to lose and plenty of dollars to make.