WordPress robots.txt Example

Robots.txt is a way to tell a search engine which pages it's allowed to spider, to "see", and which pages it cannot "see". Because of that, robots.txt differs from meta name="robots" tags, which tell search engines on those individual pages, whether they can include them in their index or not. The difference is subtle, but [...]

WordPress robots.txt Example is a post by Joost de Valk on Yoast - Tweaking Websites.A good WordPress blog needs good hosting, you don't want your blog to be slow, or, even worse, down, do you? Check out my thoughts on WordPress hosting!

Source: http://feedproxy.google.com/~r/joostdevalk/~3/oKVyJ3K9Zc0/

more info here more info here more info here more info here more info here more info here more info here