Tag Archives: Deep Web

The Information Umbrella’s Best Blogs of 2014

Happy New Year from the AIM faculty and staff!

Our blog writers’ curiosity took some surprising turns in 2014, which sparked us to ask what resonated with you readers. Below is our list of most popular blogs of the year, with a special challenge to astrologers to make sense of the odd coincidence involving publication dates.

Hand holds the world#5 Our Shrinking World

AIM blogger Kelly Brown ponders the question “With everything we have in place, are we really tapping the potential of a shrinking world or still limiting ourselves to the familiar surroundings and friends to supply us with answers and advice?” From April 15, 2014.

parade
#4 So We Had a Parade

Guest blogger Tim Williams, a 2000 AIM graduate, an adjunct instructor for the AIM Program, and COO of Sesame Communications, shares his thoughts on his experience in organizational culture and team building. From July 15, 2014.

digital vortex#3 The Dark Side of the Deep Web

Kelly Brown’s curiosity takes him deep into the layers of the Web. Think onions and murky depths. From April 8, 2014.

overstuffed garage#2 A Terabyte of Storage Space: How Much is Too Much?

How much storage is enough? Kelly Brown calculates just what will fit into 1,000 gigabytes. From July 8, 2014.

 

child using computer#1 Too Many Coders?

Are there too many coders to meet the needs of the future? Not enough? That question resonated with more Information Umbrella readers than any other in 2014, rocketing this blog post to top spot for the year. From February 18, 2014.

 

What do you want to read about in 2015? Send us a message with your ideas.

Don’t miss The Information Umbrella next week when Kelly Brown scores a touchdown with a timely topic!

The Dark Side of the Deep Web

Digital vortexThere have been a number of stories and references to the “Deep Web” in the media over the last two months, including references in Season Two of the Netflix series “House of Cards.” With a renewed interest, I wanted to make sure that I was clear on the different terms associated with the Deep Web. My research prompted me to dig even deeper (pun intended).

The Surface Web

The surface web is the part of the web that is searched by sites such as Google, Yahoo, and Bing. It is estimated that this surface layer accounts for only 1–5 percent of the entire web, as illustrated in a recently posted infographic from CNN. This surface layer excludes database search results and all corporate and academic sites behind a firewall. Search engines build and search from an index, so if a site is not part of the publicly searchable index, then it is not included in this layer. It is also possible for a website to intentionally become unsearchable by using a particular metatag.

The Deep Web

The Deep Web is the layer that lies below the surface. Every time you query an online database, the site creates a new page. That new page, however, is not included in the surface layer index because the web crawlers cannot do the same thing. The web crawler can only build an index by visiting websites and searching their links as well as the links referencing those sites. Other examples of data in the Deep Web are academic journals that are either behind a “for fee” structure or protected by a firewall. All intranet data on corporate networks also resides in the Deep Web layer. Businesses such as Bright Planet provide services that assist you in navigating the Deep Web.

The Dark Web

The top two layers can be considered to house legitimate data and transactions; they simply represent information that can be searched and indexed by web crawlers (surface) and information that cannot be seen by automated searchers (deep). Within the Deep Web, however, is an isolated area called the dark web. This is the area where cyber tracks are erased and transactions for goods and services may or may not be legal or legitimate. You can access this part of the web through browsers such as TOR that can be downloaded and allows access to the TOR network. TOR is an acronym which stands for “The Onion Router.” If you think about an onion and its layers, TOR allows you to access the core of that onion. TOR operates by hiding originating addresses among a network of servers so the end user remains anonymous. This area may house legitimate anonymous transactions but it is also the home of drug and other illicit trading.

Thoughts

I think it is important to understand the different terms relating to the different layers of the web and to understand the purpose of each layer. Could you benefit from a service that dissects the larger Deep Web for big data not available in the surface web? It is possible and very useful to be knowledgeable about all available options so you can provide the best IT service to your customers.

Author Kelly BrownAbout Kelly Brown

Kelly Brown is an IT professional, adjunct faculty for the University of Oregon, and academic director of the UO Applied Information Management Master’s Degree Program. He writes about IT and business topics that keep him up at night.