There’s more than just the googleverse

A few years ago I wanted to see how much of the internet can be reached by search engines. I researched this knowing that many people think they can find everything they need by googling on the web – in my opinion a mortal sin for anyone doing a competitor review or pre-pitch background research.
This is what I came up with.

Of the internet pie, the googleverse – the internet reachable via the most popular search engines – is represented by the yellow area.
One of the terms used to described the red area is the Dark Web (woooooo). These are websites which the search engines can’t penetrate. Most of them will be subscription only sites. The web resources I buy for the agency account for roughly half of the red area. I’m not interested in the other half because they are way off-topic (e.g. porn).
In fact, only a sliver of the yellow area will really be relevant. Think of all the crap that’s on the internet and the bit of the googleverse that might ever be useful to you will probably add up to less than 1% of the entire web.
Anything relating to the internet’s structure gets dated very quickly, so late last year I briefly looked at this again, but realised it would take weeks to revise it. At a best guess, I’d say that the yellow area has increased slightly. Blogs and social networks have taken off big time. However, there’s a lot of duplication and redundancy (people giving up with their blogs, for example), and there’s been a huge increase in subscription-only online data.
I’m speaking in the most general of terms, but my point is that no matter who you are, if you’re meeting a third party upon whom you depend (or aim to) for revenues, be it on a one-on-one or in a pitch situation, it pays to have the right information. Google alone won’t deliver it.