I think the positives outweigh the negatives.
When I first started lifting, I was very serious. Nutritionally, training wise etc. I made plenty of mistakes along the way, stupid ones, dangerous ones etc. All in the chase of a better physique.
I've never really experienced any dis-morphia of any sort, always been happy with my size - in that regard I was always thick and have no issues retaining or building muscle.
At its peak, social events, eating out, affects on the family etc were all very real. Particularly at times when you're trying to get into single digits and beyond, and you're the only one in a household doing it.
It's so much easier if someone holds you accountable, or you have someone to train with on the same path. Like minded people. Which is why logs & diaries are paramount, and helped me get into tip top shape. Been planning on doing one here on AusBB, just waiting until a bit of calm and I am actually training 5 days a week. Body needs it!
Now I am living more balanced, well, due to my business, probably unbalanced - in favour of poor diet and training. I'll be tipping the scales back in my favour though.
I can say over 12 years I have trained without any yoyo training, on for 5, off for 6 like i've seen others do (apart from injuries I have had while doing MMA etc, unavoidable)
To be honest, it depends on what your goals are. You can still "build your body" and live a comfortable, balanced life.
Let's not kid ourselves. It comes down to priorities and what you REALLY want. The hard reality is this; no one gets to the top by living a 'balanced' life. Unless they're absolute freaks (genetic or otherwise), any top athletes or entrepreneurs (steve jobs and co.) led an unbalanced life. Hard work, A LOT of sacrifice, pain, trials & tribulations along the way - eccentric, OCD type lifestyles. Go getters. Lateral thinkers. The very opposite of a 9-5'er. Very few people have major success just falling into the lap unless very lucky or they've inherited it.
I do not regret lifting weights. It has had a profound influence on what shaped me (both physically, and mentally). I look good, I feel good. I will always take the toil and pain of sacrificing hours preparing/training whatever over the alternative, because the pros outweigh the cons.