Discussion in 'Search Engine Optimization' started by SeoDezin, Apr 5, 2011.
When search engine crawler visit a site. Where does it visit first?
search engine crawler target is first page
First they gets a list of URL's to visit and store. How? A file called "robots.txt." tells the search crawler which files/pages it can request, and which files/page or directories it's not allowed to visit.
Of course it visits the first page of a web site first.
This depends on how it got to your site.
If you submitted a site map than it will go to the main page first.
If you only put links on some sites and the crawler got to your site through those links it will go to the page where the link points.
well always when we start linking we target main page or index page first so it is so daffiness that crawler go to index page first as every one said and it tool complete data of your site. What will be the time period of crawling of your site no one can tell you the exact time.
dear Crawler does not crawl only First page . its try to get all the links from Sitemap.xml & looks for any changes or any new pages added to your website & update their log if their is any change
It first target to crawl is homepage of a website.
Separate names with a comma.