The World Wide Web is a vast and always changing network of web pages. In the early days of the web there were no search engines, and people relied on finding information using pages with long lists of HTML links. It was cumbersome and links were often outdated.
The development of automated search engines made it much easier for users to find information. Modern search engines like Google, Yahoo and Bing use programs called spiders that crawl the web and find links between the main page on a site and its linked subpages. These publically viewable pages are part of the Surface Web, but they’re just the tip of an iceberg.
What’s Below the Surface Web?
While the web is growing constantly, cybersecurity experts know the vast majority of web pages are inaccessible to search engines. Hidden pages include unpublished blog posts, forums that force users to log in before they can view the contents and news sites that archive their stories for paid subscribers only after a specific amount of time. Subpages on public web servers that are not linked to other pages do not show up in search results, but if someone knows the page URL they can access the page directly by typing it into their browser’s address bar. Collectively these resources hidden from search engines are called the Deep Web.
The information locked away in the Deep Web is valuable. Doctors could access information currently hidden in archived databases about new research and medical procedures. Aerospace engineers could find data on how to build safer airplanes. Unfortunately, cyber criminals also use the Deep Web for communication and to hide their illicit activities. The Deep Web contains pages where criminals use a type of digital currency called Bitcoin to trade and sell everything from stolen credit card numbers to illegal drugs.
Navigating the Deep Web
So if the Deep Web isn’t indexed by normal search engines, how do users navigate it? The answer lies in browser software called The Onion Router, or Tor for short. Tor allows users to access .onion sites. It also anonymizes users by bouncing their web traffic through a randomized series of encrypted servers located around the world. This makes Tor users much more difficult to track online.
Like the Deep Web itself, Tor does have legitimate uses. The software was developed by the United States government to protect whistleblowers, dissidents who live under repressive political regimes and others who would be in danger if their identities were compromised.
Some governments censor the Surface Web, blocking certain web sites and monitoring their citizens’ online activities. Facebook recently established a direct connection to Tor, allowing users in these areas anonymous access to their site. It also protects those who simply value their privacy and aren’t doing anything illegal but don’t want their browsing habits tracked.
- Stalking threats still very real for TV actress - 10/16/2024
- CrowdStuck thanks to CrowdStrike - 07/25/2024
- AT&T breach too big to ignore - 07/14/2024
Hey very nice blog!
Thanks! Have you ever visited the dark web? I’m collecting stories from dark web visitors for a new piece and would love to hear yours.
Thanks for the compliment. Anything in particular you found interesting?
Great stuff from you, man. I’ve read your stuff, and you’re just too dangerous. I love what youve got here, love what you say, and you say it. You make it entertaining and you still manage to keep it smart.
Computer Support Dallas