Hi all, it appears Google is looking at some unexpected pages on my site and getting 404 errors (found out using Google Webmaster tool).
It appears like Google is trying to look into categories and authors, is my best bet to just edit the robot.txt and exclude these pages by hand? These pages dont exist and aren't on my sitemap so I am scared that just editing the robot file isn't going to stop the problem from manifesting again in the future.
Here is what Google is looking for:
Code:
Show URLs: Not found (11)
URL Detail Linked From Detected
http://northvancouverliving.com/sold/1777-deep-cove-road
404 (Not found) 3 pages Jul 15, 2011
http://northvancouverliving.com/listing/33-1925-indian-river-crest
404 (Not found) 3 pages Jul 9, 2011
http://northvancouverliving.com/category/my-marketing-advantage
404 (Not found) 2 pages Jul 7, 2011
http://northvancouverliving.com/contact-darcy
404 (Not found) 3 pages Jul 6, 2011
http://northvancouverliving.com/category/contact-james-goodwin
404 (Not found) 2 pages Jul 6, 2011
http://northvancouverliving.com/category/north-vancouver
404 (Not found) 2 pages Jul 5, 2011
http://northvancouverliving.com/category/listings
404 (Not found) 2 pages Jul 5, 2011
http://northvancouverliving.com/uncategorized/1777-deep-cove-road
404 (Not found)
unavailable
Jul 5, 2011
http://northvancouverliving.com/category/
404 (Not found) 2 pages Jul 5, 2011
http://northvancouverliving.com/why-choose-me
404 (Not found) 3 pages Jul 4, 2011
http://northvancouverliving.com/author/James Goodwin
404 (Not found) 2 pages Jul 3, 2011