It takes me 2 - 3 hours to do a full training session. One main compound lift plus 2-3 accessories.
I train for powerlifting. I may wait up to 20 min between sets if I am working with a weight up around the 90-100% range. Normally 5-8 min or so between normal heavy work sets. Depends a bit on how many others are training in the same area. Also add on lots of warm up sets, lots of flexibility work with stretches, foam roller, baseball rolling plus stretches for specific muscles in between sets then stretches, foam rolling and such after the workout.
In between I am not sitting around checkin' mah abz in da mirror 'n Facebookin' mah selfies. I'm recovering, getting my blood pressure down, hydrating and focusing on my next set, then getting ramped up, chalked and belted. And maybe admiring some spankin' buns in tights. Preferably female.
Its not right. Its not better. Its not the way it should be. It just is for me and what I do. Other guys like CT would probably be going for 15sec to 2 min rests so they better be out of my gym in an hour or I wanna know why.
My accessories tend to be 2 min rests. Sometimes its 1min cycles (do the exercise starting on the minute mark, rest, start the next set on the minute mark). Much more of a bodybuilding style.
So, like everything else fitness related, it all depends on what you are going for. A gym junkie "doing arms" should be out of there in 30 min. A strongman/powerlifter training deadlift or squat might take 3-4 hours.