Lemmy.world for instance could put the rest of the Lemmy fediverse between a rock and a hard place if they wanted to
beehaw.org is doing great, and they deferated from.world a while ago. Your point is correct though, Mastodon.social for example has half of all Mastodon users.
That said- there is little incentive to having a large instance, it costs a lot more and requires a lot more work.
The most difficult parts of moderating on Reddit aren't the trolls or spammers or even the rule-breakers, it's identifying the accounts who intentionally walk the line of what's appropriate.
IMO only a human moderator can recognize when someone is being a complete asshole but "doing it politely", or trying to push an agenda or generally behaving inauthentically, because human moderators are (in theory) members of the community themselves and have an interest in that community being enjoyable to be a part of.
Humans are messy, and finding the right balance of mess to keep things interesting without making a place overwhelming to newcomers is a fine balance to strike that I just don't believe an AI can do on it's own.