OK I'll wade in as a late adopter / borne again / evangelist-as-a-result SS'er (aka tiresome bore)
I started it as a winter thing but at the beginning of this year I fancied doing the Highland Trail SS so switched the Jones to single speed. Previously my favourite trail was a steep techy climb in a 26-46 gear. I didn't mess around, just got straight into the big hills and accepted the walking. With the lockdown nonsense it made sense to stick with it due to limitations on where I was going thinking that the gears would go back on once I hit the big stuff again. But the bug bit. Suddenly I'm looking at the back end of the bike and thinking "no more dangly thing to smack off rocks, no more monster cassette, no more chains wearing out in a year, no mud or snow blockage, 650g's less weight, no angst over what gear to be in" and suddenly I'm putting the day the gears get refitted further and further back.
The Cairngorms loop was the valediction, despite the suffer fest of the last 60k (which tbh wouldn't have been that much easier with gears). Its a different riding style with more walking to be sure but the benefits are more than cool factor. In particular on nadgery, techy, rocky downhills its brilliant - no chain flapping around and you do notice the lack of weight on the back end. (for this reason I'd never contemplate a rohloff on a mountainbike). As you are moving around the bike a lot more plus standing up, sitting down, pedaling fast, pedaling slow, walking, freewheeling a lot; it (seems) to be much better for your (well my) body. No more sitting for hours locked into one position pedaling away (well you know what I mean) Its helped my dodgy knees no end and my duff back seems much happier. I'm also now much stronger on those short sharp punchy climbs and I enjoy hill walking!
Lack of fitness isn't an issue, its just down to choosing the right gearing. Go low. Ignore the net advice and go with 32/20-22 for a 29er. If you don't do much road then go lower still. The only fitness advice I would suggest is to get your core strength built up if you get into it in a big way (actually we should be doing this anyway) as you benefit from good core stability when standing up pedaling at a low cadence. (Low cadence is 30rpm!)
But its not an affectation by any means. It offers genuine benefits and takes you one step further away from the 'more is more' marketing bullsh*t
