Showing posts with label Crawled. Show all posts
Showing posts with label Crawled. Show all posts

Wednesday, February 10, 2010

11 Basic steps to get your site Indexed in a Day - Updated for 2010

Say you have a new business or website, how do you get Google, Yahoo, Ask, and Bing to index your site?? Let alone give it a high ranking?

Well, I have compiled a list of things that every webmaster/site owner or internet enthusiast could, and in some instances, should do to make sure a web site is indexed in as little as a day or two.

1. Create at least 3 to 7 pages of content.

Hey, a "Coming Soon", or "Under Construction" page with a construction worker type animated gif was great in the 90's but nowadays it won't get you very far... Create a few pages of decent readable, useful content - remember text in images format can't be crawled!! Make sure you do the basic SEO stuff correctly:

  • Each page MUST have unique page title
  • Each page must have a unique meta tags for description and keywords
  • All links / href’s MUST have “Title” properties and Images must have descriptive “Alt” properties
  • Avoid a “loading” message or a page redirect for your main home page

2. Create a template/menu structure


Make sure all the pages link to each other by using a template menu structure. Ideally, the links/href's should be text based and be descriptive, so that they can be easily crawled.

3. Make sure HTML pages correctly formed

Go to http://www.totalvalidator.com or use the W3C Validator - http://validator.w3.org to ensure your pages are well formed. Search engines will give you a lower ranking or even ignore you if your pages are that badly written and do not include enough information.

4. Create a Sitemap.

Sitemaps have evolved over the years and there are still various forms out there (HTML, RSS and graphical sitemaps). Google and the other major search engines have adopted a common standard though, that will list all pages in your website and help identify the most important pages. There are quite a few free sitemap generators out there. For small sites I recommend this one: http://www.xml-sitemaps.com/

5. Create a Webmaster Tools account and Submit / Ping Sitemap.

After you have created your sitemap and have uploaded the sitemap to the root of your website, you must submit the sitemap to the various search engines. It is recommended to create a webmaster tools account at the various big search engines:

  • Google (http://www.google.com/webmasters/tools/): Google has a very useful Webmaster Tools facility, which allows you to view extra information that is helpful to all webmasters. It is essentially Google’s view of your website.
  • Bing (http://www.bing.com/webmasters): Bing’s webmaster tools ahs also improved dramatically over the last year or so, but it not quite as useful as Google’s yet.
  • Yahoo (https://siteexplorer.search.yahoo.com/): Even less useful than Bing, but still required for now.

After you have registered at the various Webmaster tools you can resubmit updates or ping your updated sitemap to search engines at any time, although it should be crawled and updated automatically. Search Engine Sitemap Ping / Submit URL's:

  • Google: http://www.google.com/webmasters/sitemaps/ping?sitemap=[complete sitemap address
  • Yahoo: http://search.yahooapis.com/SiteExplorerService/V1/ping?sitemap=[Complete sitemap web address
  • Ask: http://submissions.ask.com/ping?sitemap=[your sitemap web address
  • MSN Live: http://webmaster.live.com/webmaster/ping.aspx?siteMap=[your sitemap web address]

Note: Google requires you to submit your sitemap to them first via the Webmaster's tools interface. Only then can you ping them using the above URL. Yahoo and Ask.com allow the ping to act as a submission as well.

6. Link from highly trafficked bookmarking sites

These are sites are visited regularly by search engine crawlers so if there is a link to your website it will be indexed quite quickly. You will have to create an account with these sites, to enable you to link to your site. Some of the bookmarking sites that you can use include Del.icio.us, BlinkList, StumbleUpon.com and digg.com. Social networking sites like Facebook and Myspace will also help. Try to link using some of the popular phrases like the list at the bottom of BlinkList page. At the end of the day it is about getting high quality links to your site.

7. Create/comment on a blog.

Create a blog on blogger.com or another blogging provider (there are many different blogging options these days) and start blogging about your area of expertise and how it relates to your website. Feel free to promote your site, as long as it contributes to an open discussion and is not just another fake blog/marketing scheme. The alternative is to mention your website in a comment on a related blog of someone else, but again it has to be as part of a discussion where you make a contribution to that person's blog!

8. Comment on a popular forum/help pages.

Make sure you include your website URL as part of your forum profile and comment on a forum that you are part of, or help a newbie out on a subject that you are an expert of, in a related forum or help site. It is about good web karma too!

9. Install and activate Google Analytics

Google Analytics is probably one of the coolest freebies for any webmaster. Sign up for Google Analytics free at www.google.com/analytics. You will have to add a simple script to all your HTML pages to make sure it is tracked properly, but the information you will get out of it is extremely valuable and will help you improve and better target your pages over time.

10. Use Google AdSense


If it makes sense for your web site you may consider running Google advertising on your web site. Although AdSense and Adwords are supposedly separately indexed from Google’s search indexing, there are some evidence to the contrary, and some sites that serve AdSense advertising have been indexed more quickly than other sites that don't serve AdSense advertising. Hey, you might even earn a few bucks, it may be worth a shot, right?!


11. Use Google Adwords


Again very similar to point 10, no real proof that advertising your web site domain with Google Adwords would really benefit, but if you want cover all your bases and get indexed as soon as possible, its worth a try...at the very least Adwords allow you to drive targeted traffic to your site with minutes. Using Google Analytics above you can also track conversions and start measuring your ROI.

If you follow these steps your site should be crawled by Google, Bing and Yahoo pretty quickly, obviously I can't guarantee your site being crawled in a day…but doing the above steps will go a long way to ensure you are indexed and crawled quickly. Remember the above might ensure you get indexed and included in search engines quickly, but only a regularly updated site, with high quality, useful content will ensure you get high rankings within the various search engines.

Friday, March 27, 2009

Google, Google wherefore art thou Google?

Google is obviously the biggest, best and certainly most popular search engine. But...herein lies the problem too...

Say you have a new business or website, how do you get Google, Yahoo, Ask, and Live to index your site?? Let alone give it a high ranking?

Well, I have compiled a list of things that every webmaster/site owner or internet enthusiast could and in some instances should to make sure your web site gets indexed in as little as a day or two.

10 Steps to get your site indexed in a day by Google...

1. Create at least 3 to 7 pages of content.

Hey, a "Coming Soon", or "Under Construction" page with a construction worker type animated gif was great in the 90's but nowadays it won't get you very far... Create a few pages of decent readable, useful content - remember text on images can't be crawled!!

2. Create a template/menu structure
Make sure all the pages link to each other by using a templated menu structure. Ideally the links/href's should be text based and be descriptive, so that they can be easily crawled.

3. Make sure HTML pages are well formed
Go to http://www.totalvalidator.com or use the W3C validator - http://validator.w3.org to ensure your pages are well formed. Search engines will give you a lower ranking or even ignore you if your pages are that badly written and don't include enough information.

4. Create a Sitemap.
Sitemaps have evolved over the years and there are still various forms out there (HTML, RSS and graphical sitemaps). Google and the other major search engines have adopted a common standard though, that will list all pages in your website and help identify the most important pages. You can find out more about the
Sitemap protocol here and/or a free automated sitemap generator here.

5. Create Google Webmaster tools account and Submit / Ping Sitemap.
After you have created your sitemap and uploaded it to the root of your website you must submit the sitemap to the various search engines. Google has a cool
Webmaster tools collection which allows you to view extra information that is helpful to all webmasters.
*Updated - Microsoft's Live search engine's webmaster tools (http://webmaster.live.com/) is also a helpful SEO tool addition and worth subscribing to.

Search Engine Sitemap Ping / Submit URL's:


  • Google: http://www.google.com/webmasters/sitemaps/ping?sitemap=[complete sitemap address] e.g. http://www.google.com/webmasters/sitemaps/ping?sitemap=http://www._httpvalidator.com/sitemap.xml
  • Yahoo: http://search.yahooapis.com/SiteExplorerService/V1/ping?sitemap=[Complete sitemap web address] e.g. http://search.yahooapis.com/SiteExplorerService/V1/ping?sitemap=http://www._httpvalidator.com/sitemap.xml

  • Ask: http://submissions.ask.com/ping?sitemap=[your sitemap web address] e.g. http://submissions.ask.com/ping?sitemap=http://www._httpvalidator.com/_sitemap.xml
  • MSN Live: http://webmaster.live.com/webmaster/ping.aspx?siteMap=[your sitemap web address] e.g. http://webmaster.live.com/webmaster/ping.aspx?siteMap=http://www._httpvalidator.com/_sitemap.xml

Note: Google requires you to submit your sitemap to them first via the Webmaster's tools interface. Only then can you ping them using the above URL. Yahoo and Ask.com allow the ping to act as a submission as well.

5. Link from highly trafficed bookmarking sites
These sites are visited regularly by search engine crawlers so if there is a link to your website it will be indexed quite quickly. You'll have to create an account with these sites, to enable you to link to your site. Some of the bookmarking sites that you can use include
Del.icio.us, BlinkList, StumbleUpon.com. Social networking sites like Facebook and Myspace will also help. Make sure you link using some of the popular phrases like the list at the bottom of BlinkList page.

6. Create/comment on a blog.
Create a blog on
blogger.com or another blogging provider (there are many different blogging options these days) and start blogging about your area of expertise and how it relates to your website. Feel free to promote your site, as long as it contributes to an open discussion and is not just another fake blog/marketing scheme. The alternative is to mention your website in a comment on a related blog of someone else, but again it has to be as part of a discussion where you make a contribution to that person's blog!

7. Comment on a popular forum/help pages.
Make sure you inlude your website URL as part of your forum profile and comment on a forum that you are part of, or help a newbie out on a subject that you are quite knowledagble of, in a related forum or help site.

8. Install and activate Google Analytics
Google analytics is probably one of the coolest freebies for any webmaster.
Sign up for Google Analytics free here. You'll have to add a simple script to all your HTML pages to make sure it gets tracked proparly.

9. Run Google Adsense
If you are able to, then run some Google advertsing on your site. Although Adsense and Adwords are seperate from earch indexing, there have been many claims that sites that serve Adsense ads are indexed more quickly than those sites that don't (many would argue this point, but the way I see it, rather do it, if it might help).

10. Run Google Adwords
Again very similar to point 9, no real proof that advertising your website domain with Google Adwords would really benefit, but if you want cover all your bases and get indexed as soon as possible, its worth a try...

If you follow these 10 steps your site should be crawled by Google and other major search engines pretty quickly, obviously I can't guarantee your site being crawled in a day though...but there isn't much more than this that you can do.

I have been lucky enough to be involved with the development of a product that should help you do most of the above in a seemless way - httpvalidator is still in development, but from what I have seen this might be a good all in one webmasters tool that can certainly make things a little easier.