Troubleshooting: how search engines can misunderstand
your web pages
This is a common problem. Your web designer has created
a beautiful page with nice graphics and great Flash animations.
Unfortunately, it seems that search engines won't list your
website no matter what you do.
This can have several reasons:
a) The HTML code of your web pages is meaningless
to search engines
Search engines use very simple software programs to index
your web pages. A web page that looks great to the human
eye can be totally meaningless to search engines.
Search engines cannot see content that is presented in
and other script languages or other multimedia file formats.
then search engines might not be able to find your website
Solution: Check your web pages with a search
engine spider simulator. Spider simulators will show
you how search engines see your web site.
If you use IBP's search engine spider simulator, you can
even enter the name of the spider, so that you can test
if Google and Yahoo get different results when they visit
your web pages.
b) The HTML code of your web page contains errors
Some HTML errors can prevent search engine spiders from
indexing your web pages. While most search engine spiders
can deal with minor code errors, some of them will send
faulty information to search engine spiders.
For example, your web page could contain a tag at the
top of the page that tells search engines "the web
page ends here" although your main content has not
Solution: Check the HTML code of your web pages with an
HTML validator tool. You can find an HTML validator in
the free IBP trial version.
c) The HTML code of your web pages doesn't contain
the right elements
If you want to get high rankings for a special keyword
then this keyword must appear in the right places on your
web page. For example, it usually helps to use the keyword
in the web page title.
There are many other elements that are important if you
want to have high rankings.
Solution: Check the HTML code of your web page with IBP's Top
10 Optimizer. IBP's Top 10 Optimizer compares your
web page with the top ranked pages and it will tell you
in detail how to change your web page so that it can
get top 10 rankings.
d) Your web server sends the wrong status codes
Some web servers send wrong status codes. When a search
engine spider requests a web page from your site then your
server sends a response code. This should be the "200
Unfortunately, some servers send a "302 moved" or
even a "404 not found" response code to the search
engine spiders although the web page can be displayed in
a normal web browser.
In that case, search engines will think that the web page
doesn't exist and they won't index the page.
Solution: Use the search engine spider simulator to find
out which response code your web server returns to search
engines. If the response code is not "200 OK", IBP's search
engine spider simulator will return a warning message.
If you want to get top rankings on Google and other search
engines then you must make sure that search engines don't
misunderstand your pages. Use the tips above to make sure
that search engines see what you want them to see.