My biggest issue with camping is the narrow window in wihch you can sleep.
At night, you always have drunk loud assholes at the campsite who make it impossible, regardless of what "quiet hours" are supposed to be. Typically this dies down by about 2am.
Then at 6am, you have other assholes waking up and making a shitload of noise, because somehow morning people think the sun being up justifies them being noisy, even if it's only 6am.
And even if you don't have people waking up at 6, you have the sun beating down and turning the tent into an oven within a very short period of time.