Friday, September 7, 2018

What is the Deep Web?



Although all of these terms tend to be used interchangeably, they don't refer to exactly the same thing. An element of nuance is required. The 'Deep Web' refers to all web pages that search engines cannot find.

Thus the 'Deep Web' includes the 'Dark Web', but also includes all user databases, webmail pages, registration-required web forums, and pages behind paywalls. There are huge numbers of such pages, and most exist for mundane reasons.


We have a 'staging' version of all of our websites that is blocked from being indexed by search engines, so we can check stories before we set them live. Thus for every page publicly available on this website (and there are literally millions), there is another on the Deep Web.
The content management system into which I am typing this article is on the Deep Web. So that is another page for every page that is on the live site. Meanwhile our work intranet is hidden from search engines, and requires a password. It has been live for nearly 20 years, so there are plenty of pages there.
Use an online bank account? The password-protected bits are on the Deep Web. And when you consider how many pages just one Gmail account will create, you understand the sheer size of the Deep Web.
This scale is why newspapers and mainstream news outlets regularly trot out scare stories about '90 percent of the internet' consisting of the Dark Web. They are confusing the generally dodgy Dark Web with the much bigger and generally more benign Deep Web.

Leave a Comment