I've heard people say that the 2010 Worlds format was a... I
think the word was diverse, but it could have been something else... format, but in the end LuxChomp and SPs in general played a very similar gate-keeping role to ADP and Urshifu Rapid Strike (albeit a larger number of decks got through the gate). If it hadn't been for Donphan Prime, which in and of itself also did its fair share of gatekeeping.
https://http://google.com/.com/2014/08/04/bright-looks-and-psychic-locks/
https://ptcgarchive.com/2010-decks/
Granted, the 2010 format still appears to be extremely diverse. But looking at this list and seeing how dominant Luxray GL Lv.X and Garchomp C Lv.X were gives me a sense of deja vu. A "stage 1" evolved big basic holding the gates of the format against any deck that is unable to keep pace? And a highly prevalent stage 1 deck that has a type advantage over and the ability to restrict some of the oppressiveness of the aforementioned deck, but which itself also sets a high standard for the types of cards that can be used? Also, I'd like to point out that there is a widely used evolved support Pokémon (Claydol), a few Basic support Pokémon used by nearly every deck (Azelf, Uxie, and, to a lesser extent, Spiritomb, Chatot, Sableye, Combee, Mesprit, Regice, and assorted Unown), a well-used Stadium card with a powerful effect (Broken Time Space), a Special Energy that summons basic Pokémon to the field (Call Energy), a card that allows you to search for three Pokémon of one of the most dominant classes in the format (Collector, which is pretty much the equivalent of the upcoming Adventurer considering the dominance of SPs and the sheer number of techable and support basics), and even an entire archetype of cards that, at first glance, are just ordinary cards with a fancy stamp and tag on them (I mean, an evolved Pokémon species being printed on a basic isn't a MECHANIC, is it?), but which in fact have a very large support base. And also, the upcoming V-Union mechanic is more or less a spiritual successor of the Pokémon LEGEND and actually stands a fair chance of producing more playable cards of its archetype than Pokémon LEGEND. I mean, I'm seeing a lot of resemblances. The main differences are diversity and pace (and obviously Power Level, but Power Creep has been a thing for such a long time that to say that there is a difference in power between formats of two different eras is not only old news, but redundant news). And I would like to point out that at one point the SWSH format was actually pretty diverse, one might even say more diverse than the 2018 format (which, as far as I can tell was primarily formed of Buzzwole GX, Zoroark GX, Garbodor, Rayquaza GX, and Ultra Necrozma GX, losing all the diversity of the 2017 format despite the fact that many M EX's were still legal). I seem to remember a time when ADP, Centiskorch VMAX, Eternatus VMAX, Lucario & Melmetal GX, Pikachu & Zekrom GX, Inteleon VMAX, Dragapult VMAX, Blacephalon, and many more decks were all viable silmuntaneously. Yes, I said the words GX and VMAX a lot, but it was still a diverse format. Not sure how many decks are viable right now. Urshifu VMAX and Calyrex VMAX are forming the top tier, but I would expect there to be many more decks on tiers 2 and 3. Plus you can always go rogue with a meme deck if you're tired of playing the same-old-same-old and don't care about your winstreaks. And there are a lot of different toolbox decks and high-tier deck variations you can play if you want to spice up the game. We've had an astonishing number of techable Vs in the past couple of sets, and there's still a lot of unexplored design space as the new format shakes out. And the very nature of Single Strike and Rapid Strike Pokémon makes it possible to shake up familiar decks like Urishfu Rapid Strike VMAX (ever tried running an Urishfu/Sylveon VMAX build?). So I wouldn't say that this format is a good format for single prizers, but I wouldn't be too quick to jump to the conclusion that it is not diverse until you've done a bit of exploring and playtesting.
I agree with everything that you said except with your expectations for how much of the card pool actually gets played. Every card game, like...ever is like that, some of the stuff is more or less competitively viable, and like 90% of the card pool is "useless", and that is somewhat by design. I don't think they could possibly achieve such an impossible task as to make like half of thousands of card equally viable at any given time, and I also don't believe they would even if they could, business-wise card games need to have those few chase cards among the sea of bulk to drive prices of cards, and therefore demand for the product, up.
I would also like to point out that cube formats are a great place for the 90% to thrive because you can create a custom constructed format using cards of the same power level without being restricted by whether or not those cards have ever coexisted in any given official format. You also don't have to draft the cards necessarily; there is a system called "sealed cube" where you can just pick cards off of a list to put into your deck. A format where EVERY card is playable would have to be a VERY small format indeed.
Also, on an entirely unrelated note, what do all you more serious collectors do with all the extra bulk you get from opening a booster box (or several booster boxes) of each set? (This probably isn't the right place to ask, but all this talk about bulk and chase cards and 90%s has made me wonder.)