-
Robots.txt - SEO Tips
I will recommend that everyone installed a robots.txt to block duplicate content, I am still improvising mine
=======================================
User-agent: *
Crawl-Delay: 25
Disallow: *.js
Disallow: /backup/
Disallow: /includes/
Disallow: /cron/
Disallow: /libs/
Disallow: /plugins/
Disallow: /tmp/
Disallow: /print.html
Disallow: /my-messages.html
Disallow: /my-profile.html
Disallow: /my-favorites.html
Disallow: /my-listings.html
Disallow: /my-packages.html
Disallow: /payment-history.html
Disallow: /payment.html
Disallow: /listings-by-field.html
Sitemap: http://www.xxx.com/sitemap.xml
-
Any feedback on whether to
Disallow: /*?sort_by=
-
Also wondering if i gonna add Disallow: /*?sort_by= also.
Did you add it ?
Also i have a question. What does * do in the robots.txt ?
-
This is a very good idea.
-
What page are listings-by-field.html ?
-
-
A rel="nofollow" should also be added to categoryFilter links.