Discussion in 'Search Engine Optimization' started by jesicawillss, Dec 20, 2010.
What is robots.txt? Is it a part of on-page?
well, i think so,, it is a part of on page optimization,,but i didnt work on it yet..
Robots.txt files are indeed part of on-site optimization. You can use them to give instructions about your site to the web crawlers/spiders. You can ask them not to crawl and/or index at all your entire website, or just simply avoid certain files and/or directories and such.
Separate names with a comma.