Welcome to the Q&A Forum

Hi,

In the early days of my blog, I used permalinks with the following format:

http://www.mysitesamp.com/2009/02/04/heidi-cortez-photo-shoot/

I then decided to change this format using .htaccess to this format:

http://www.mysitesamp.com//heidi-cortez-photo-shoot/

My question is, why do rogerbot still crawls my old URL format since these urls' no longer exists in my website or blog.

Hi Yannick,

Yes I have access to .htaccess

Here are some sample format of the category feeds:

http://www.mysite.com/category/lifestyle/feed

http://www.mysite.com/category/off-beat/feed

but Google webmaster tools report the broken feeds as

http://www.mysite.com/lifestyle/feed

http://www.mysite.com/off-beat/feed

not sure why it removed the "category" in the url.

Thanks in advance...

Hi Guys,

I've been doing much cleaning on my blog lately and deleted numerous categories including their posts with low quality content.

After deleting the categories, Google Webmaster Tools is reporting some 404 errors about the RSS Feeds for the deleted categories.

I've created a 404.php file inside my theme and placed the following code

header("HTTP/1.1 301 Moved Permanently");
header("Location: http://www.mysite.com/My404Page/", true, 301);
exit();
?>

this have catched all 404 errors and redirected them to the specific page. Unfortunately, it could not catch the inactive feed urls.  Is there a way to do this so that all inactive feeds will be redirected to my 404 page?

Thanks in advance....

Hi Guys,

Is there a way in Wordpress to transfer or redirect a category and all posts under it into a sub-domain?

Thanks in advance..

Hi Stefano,

First of all, I would like to thank you for the help.  I really appreciate it.

I have tried to implement the (http://perishablepress.com/press/2009/03/29/4g-ultimate-user-agent-blacklist/) and it seems that it minimized the scrapers visiting my site.  I paired it with blocking specific domains and IPs.

Once this are applied,  an error occurs every time I add a picture into my posts.   What do I need to remove in the list of blocked bots to allow the adding of pictures?

by the way, I am using wordpress as my CMS.

Thanks in advance...

Hi,

Thanks for the reply.  I have canonical tag in all my pages.  The thing is,  when they start to scrape my posts,  even just the part before the tag of my content,  google gives them higher rankings than mine.