Search Engine Facts
Search Engine Facts

Read our back issues

May 2017

December 2009

November 2009

October 2009

September 2009

August 2009

July 2009

June 2009

May 2009

April 2009

March 2009

February 2009

January 2009

December 2008

November 2008

October 2008

September 2008

August 2008

July 2008

June 2008

May 2008

April 2008

March 2008

February 2008

January 2008

December 2007

November 2007

October 2007

September 2007

August 2007

July 2007

June 2007

May 2007

April 2007

March 2007

February 2007

January 2007

December 2006

December 2006

November 2006

October 2006

September 2006

August 2006

July 2006

June 2006

May 2006

April 2006

March 2006

February 2006

Januray 2006

December 2005

November 2005

October 2005

September 2005

August 2005

July 2005

June 2005

May 2005

August 2005

March 2005

February 2005

January 2005

December 2004

November 2004

October 2004

September 2004

August 2004

July 2004

 

» Archive

 

SoftwareNerds.co.uk
All about software products and antivirus solutions.

e-Gear.dk
Good deals and offers on computers & hardware.

AVGDanmark.dk
AVG Antivirus offers top security solutions.

Home   Contact   Privacy policy    Partner sites

How your firewall can destroy your search engine rankings

Some webmasters recently experienced a delisting from the search engines without any good reason. The webmasters had not done anything wrong and their web sites were optimized for search engines. Nevertheless, the web sites had been removed from search engines.

Poorly configured firewalls can block search engine spiders

It turned out that the delisted web sites were all hosted by the same hosting company. More precisely, the web sites were all hosted by a hosting company that used a special firewall software by SonicWALL Inc.

That firewall stopped the search engine spiders from accessing their web sites. Google, Yahoo, MSN and all other search engines that request the robots.txt file couldn't index the web site anymore because the firewall didn't allow that:

"An attacker could retrieve robots.txt from the server, then use the contents of this file to discover the path of an unprotected administration interface for the server. The attacker may gain control of the webserver using this interface.

The information gathered from robots.txt could be used for system compromise and control of the web server." (source)

This is the standard security settings of the SonicWALL firewall and it basically means that your web site won't be spidered by search engines if you use this firewall without customizing it.

A firewall with these settings will drop the connection to anyone requesting the robots.txt file so that it looks as if the web site is offline. From an SEO point of view, this is very bad for your web site because all good search engine spiders request the robots.txt file before indexing your web site.

What does this mean to you?

If your web site is not listed on search engines although it has many good incoming links and optimized web page content, you should ask your web host if their firewall blocks search engines that request the robots.txt file. Your web host might not be aware of the problem.

Further information:

Copyright Axandra.com - Internet marketing and search engine ranking software


Home   Contact   Privacy policy    Partner sites
May 2005 search engine articles