How is Yahoo using your RSS feed?

by Robert Fuess

Published on this site: January 31st, 2006 - See more articles from this month

The format for the site submission has changed in the free Yahoo web submissions http://submit.search.yahoo.com/free/request . At first it allowed you to submit your site to it using the free submission form. This was normally the home page, and you would hope that their spider would crawl the rest of your website from there. Some webmasters have found this inadequate and have submitted other pages that Yahoo failed to notice otherwise.

More recently Yahoo allowed you to put in a text document with a list of URL's to simplify the submission process.

A few weeks ago I noticed that Yahoo also allows RSS feeds of your website, Atom feeds, or a text listing of urls in this submission form. What does this mean? What will Yahoo do with the links in the RSS feed?

The text list of URL's was obviously an effort to simplify the process for Yahoo to find all the pages in a website, as their documentation describes. But the RSS? Will they just add this to their database of RSS feeds for RSS searches, or will they follow the links with their spider to evaluate all the pages listed?

I have been working with Google Sitemaps since they came out. (Google uses their XML-formatted Google Sitemaps to help discover all the pages in a website, as well as to evaluate what pages have changed recently.) I am wondering if this is Yahoo's response to the Google Site Maps. Does anyone know? http://www.spiderweblogic.com/GoogleSitemap.aspx

If I was them, then I would use RSS feeds to evaluate changes in the website. This has more information than the list of URL's, since it also has descriptions and date changed. They have all the right information. If this is the case, it would be advisable for webmasters to have a RSS feed for their whole site to submit.That way Yahoo could just check the feed and know what pages to re-crawl, or what pages have been added.

It could be just as possible that this is just a field for us to submit RSS feeds, in addition to the other pages in your website. If this is the case, then we may need to submit both the feed and the home page of the website.

For myself, unless I find out differently, I am going to submit both. I intend on having a Google Sitemap and a RSS feed on all my future websites. I think it is safest to help out the search engines in any way possible. If they want information on what files have changed, I want to be proactive in giving it to them.

In addition, for those who have the coding experties, it is advisable to automate pinging Yahoo when your RSS feed changes. This is a standard blogging technique.

If you use movable type, the following article will help you configure your Yahoo to be automatically notified of a change in your RSS feed: http://jeremy.zawodny.com/blog/archives/001490.html . There is an alternate way, for those who don't have movable type or standard blogs. Lets say you want Yahoo to know about the RSS feed you built for http://www.yourdomainurl.com and have the RSS file, myRss.xml. You could automate (or even have a web shortcut for) the following HTTP request: http://api.my.yahoo.com/....?u=http://www.yourdomainurl.com/myRss.xml

For more information, see the official Yahoo documention on RSS here: http://publisher.yahoo.com/rssguide

Robert Fuess is a veteran website designer who specializes in making dynamic search engine optimized websites. http://www.SpiderwebLogic.com http://www.SchoolAndTeacher.com