Here’s a partial list of what maybe shouldn’t be crawled that I put in my robots.txt.
Is there any thing else that should be added that shouldn’t be crawled?
We generally like to put blogs in a
Disregard the last line: “We generally like to put blogs in a..”
Any chance on any help with this?
You must be logged in to reply to this topic.
Are you a blogger, web designer, developer, or website owner looking to generate more income?
Promote products for the largest and fastest growing online website platform as part of our affiliate program.