Yahoo Feed Reader banned


Back to topics list
[post:284#2627]
Rebecca

07/28/2008 02:03 AM

Reviews: 23
Posts: 734

Just an FYI, I've blocked the IPs of Yahoo's feed reader (using the user agent YahooFeedSeeker). It's been abusively requesting the same URLs thousands of times a day, sometimes more then once a second. I tried adding them to robots.txt but despite downloading it hundreds of times a day, they're still ignoring it. As such, I've had no choice but to drop all traffic from their IPs. I should add, I would complain to them, but the site they point you at, http://publisher.yahoo.com/rssguide, doesn't have any feedback mechanism. Effected addresses are:

209.131.41.48, 209.131.41.49, 216.39.58.17, 216.39.58.18, 216.39.58.78

If you'd like to use Yahoo Feed Reader with the site, please post here and I'll reconsider my decision.

I should also add that this won't have any impact on the regular yahoo spider, "Yahoo! Slurp," as it comes from different addresses.

Edited on 07/28/2008 02:09 AM.

[post:284#2630]
AstroNerdBoy

07/28/2008 09:31 AM

Reviews: 415
Posts: 372

I wonder why it would hammer the site so hard for.

[post:284#2631]
Devil Doll

07/28/2008 11:19 AM

Reviews: 365
Posts: 1571

RSS feeds attempt to be up to date in order to be useful for their consumers. Let's say, a RSS feed is presenting data no older than 5 minutes then it has to request these data from the original source every 5 minutes at least, i. e. 24 * 12 = 288 times every day.

The problem is that Yahoo does not know whether the content has changed on CAR so it has to ask over and over again whereas Rebecca's own (locally computed) RSS feeds can deliver already customized, cached information that is only computed when a change of the CAR server's content has happened (which is probably not several hundred times a day...).

What kind of HTTP request does this reader send? Might it be satisfied with a HTTP-304 "not modified" response instead of running the computation on the CAR server? Rebecca might know quite well whether any change has been done and might be able to simply check some last modification date in order to satisfy the RSS feed's need.

Edited on 07/28/2008 11:21 AM.

[post:284#2632]
Rebecca

07/28/2008 01:42 PM

Reviews: 23
Posts: 734

Actually, the URLs they're requesting are 403 forbidden, by and large. They're broken, there's no getting around it.

[post:284#3096]
Anonymous #3025

10/01/2008 08:52 AM

Reviews: 0
Posts: 2

aaa iam here to exchange views regarding cardcaptor sakura

[post:284#3097]
Anonymous #3025

10/01/2008 08:57 AM

Reviews: 0
Posts: 2

can i get some revievs regarding cardcaptor sakura

[post:284#3098]
Stretch

10/01/2008 09:28 AM

Reviews: 1385
Posts: 1229

I think this is the page you want: Cardcaptor Sakura.

[post:284#3099]
Stretch

10/01/2008 09:35 AM

Reviews: 1385
Posts: 1229

You could either go to By Title, then look it up in alphabetical order, or type it into the "Search Titles" box, push enter, and go from there.

Reply to this topic Start a new topic
Back to topics list

Community Anime Reviews

anime mikomi org