{"id":214,"date":"2003-03-21T00:27:35-05:00","date_gmt":"2003-03-21T08:27:35+00:00","guid":{"rendered":"http:\/\/peterjanes.ca\/wordpress\/?p=214"},"modified":"2003-03-21T00:27:35-05:00","modified_gmt":"2003-03-21T08:27:35+00:00","slug":"mmm-spycy","status":"publish","type":"post","link":"https:\/\/peterjanes.ca\/blog\/2003\/03\/21\/mmm-spycy\/","title":{"rendered":"Mmm,&nbsp;spycy"},"content":{"rendered":"<div class='e-content'><p><a href=\"http:\/\/www.disobey.com\/amphetadesk\/\">AmphetaDesk<\/a> has started crashing on me every time I try to reload its webpage.  I&#8217;ve been thinking for a while about trying another RSS aggregator, and this seemed like a good time to do it.<\/p>\n\n<p>So what better place to start than <a href=\"http:\/\/www.google.com\/search?q=python%20rss%20aggregator\">Google<\/a>?  I&#8217;m not sure exactly what made me look for a <a href=\"http:\/\/www.python.org\/\">Python<\/a> aggregator&#8211;probably reading <cite>Mark Pilgrim<\/cite>&#8216;s <a href=\"http:\/\/diveintomark.org\/archives\/rooms\/python\/index.html\">weblog<\/a> has had some more-than-subliminal effect.  I&#8217;d tried <a href=\"http:\/\/www.nongnu.org\/straw\/\">Straw<\/a> and <a href=\"http:\/\/www.oreillynet.com\/~rael\/lang\/python\/peerkat\/\">Peerkat<\/a> <a href=\"http:\/\/peterjanes.ca\/blog\/archives\/2002\/10\/24\/linux_rss_aggregator_search.html\">before<\/a>, but both had their problems: <cite>Straw<\/cite> is heavily GNOME 2 dependent, and <cite>Peerkat<\/cite> didn&#8217;t handle newer RSS feeds.  (I&#8217;ve since <a href=\"http:\/\/peterjanes.ca\/blog\/archives\/2003\/02\/16\/apache_woes.html\">upgraded<\/a> to GNOME 2 via RedHat 8, but Straw requires more libraries than I care to attempt to find or install.)<\/p>\n\n<p>Oh yes, the Google search.  I came across <a href=\"http:\/\/www.deadlybloodyserious.com\/Python\/2002\/08\/25.html#a872\" title=\"Deadly Bloody Serious about Python: The Search for the Ultimate Newsreader\">this post<\/a> which pointed to <cite>Mark Pasc<\/cite>&#8216;s <a href=\"http:\/\/markpasc.org\/weblog\/2002\/08\/23_notes_on_a_python_aggregator.html\" title=\"notes on a Python aggregator\">notes<\/a>, which in turn led to <a href=\"http:\/\/spycyroll.sourceforge.net\/\">SpycyRoll<\/a> (<abbr title=\"also known as\">aka<\/abbr> Pyblagg).  It&#8217;s kind of similar to AmphetaDesk in that it&#8217;s a browser-based tool, but that&#8217;s about it.  For one, it only runs periodically, instead of continuously.  It&#8217;s forgiving of RSS errors (mostly; more later) via Mark Pilgrim&#8217;s <a href=\"http:\/\/diveintomark.org\/projects\/rss_parser\/\"><cite>ultra-liberal<\/cite> RSS parser<\/a>.  It caches its output page rather than regenerating the content at each reload.  And, also via Mark&#8217;s parser, it seems to Do The <a href=\"http:\/\/www.catb.org\/~esr\/jargon\/html\/entry\/Right-Thing.html\">Right Thing<\/a> when it retrieves feeds by respecting and using <cite>ETags<\/cite>, <code>If-None-Match<\/code>, <code>If-Modified-Since<\/code> and <code>User-Agent<\/code> headers and <code>gzip<\/code> encoding, which is better than AmphetaDesk&#8217;s hand-rolled (but fairly effective) way.<\/p>\n\n<p>Which isn&#8217;t to say it&#8217;s perfect.  While the parser is tolerant of errors, the code to handle the values retrieved isn&#8217;t particularly forgiving.  Of the thirty-some feeds I read, three couldn&#8217;t be read and so were ignored; I&#8217;ve worked around the two underlying problems with the little bit of Python I&#8217;ve learned in the last month (mostly from Mark Pilgrim again, this time via <a href=\"http:\/\/diveintopython.org\/\">Dive Into Python<\/a>).  The other big problem I&#8217;ve found is that for feeds that don&#8217;t provide per-item <code>dc:date<\/code> values&#8211;i.e. <em>most<\/em> of the ones I read&#8211;it will continually recreate its caches of those items, throwing off the display and ordering of the items.  I&#8217;ve worked around this too, but at the expense (I think) of breaking If-Modified-Since retrieval for those feeds.<\/p>\n\n<p>I can live with SpycyRoll; after a couple of days, it already <cite>feels<\/cite> better to me than AmphetaDesk, for mostly intangible reasons.  Neither one is what I ultimately want, but currently SpycyRoll is the more usable of the two for me, even ignoring AmphetaDesk&#8217;s crashes.  (I haven&#8217;t been able to track down why AmphetaDesk is crashing, it just is; the shell reports a <cite>Broken pipe<\/cite>.  I&#8217;m hesitant to report it to Morbus Iff without any further information or investigation, but let&#8217;s see just how good his <a href=\"http:\/\/www.disobey.com\/dnn\/2003\/02\/index.shtml#001470\"><q>ability<\/q> to appear out of nowhere<\/a> really is. &lt;grin&gt;)<\/p>\n\n<p>There are still features I haven&#8217;t come across yet that I&#8217;d like to see in a browser-based aggregator.  I want to be able to remove some items without reading them while making others <cite>sticky<\/cite>.  I want headline, compact and expanded views, and mixed-feed and single-channel modes.  (The <a href=\"http:\/\/www.mozilla.org\/catalog\/web-developer\/examples\/kitchensink.xml\">kitchen sink<\/a> would be nice, too.)  To my way of thinking, these would be best accomplished using a combination of server-style commands (&agrave; la AmphetaDesk&#8217;s <cite>Add New Channel<\/cite> and <cite>My Settings<\/cite> pages) and DOM manipulation on top of the functionality of SpycyRoll.  As Morbus Iff <a href=\"http:\/\/www.disobey.com\/dnn\/2003\/01\/index.shtml#001466\">suggests<\/a> (in another context), these are things better handled by limited-purpose <cite>shell utilities<\/cite> instead of a <cite>monolithic single application<\/cite>.  I&#8217;m going to try to add some of these things to SpycyRoll in the next few days&#8230; wish me and my not-so-mad Python skillz luck&#8230;.<\/p><\/div><div class=\"syndication-links\"><\/div>","protected":false},"excerpt":{"rendered":"<p>SpycyRoll has taken over AmphetaDesk as my aggregator of choice.  There are bugs, of course, and missing features, but overall I prefer the way it feels.  I may learn Python yet.<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"mf2_syndication":[],"venue_id":0},"categories":[3],"tags":[],"kind":false,"_links":{"self":[{"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/posts\/214"}],"collection":[{"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/comments?post=214"}],"version-history":[{"count":0,"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/posts\/214\/revisions"}],"wp:attachment":[{"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/media?parent=214"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/categories?post=214"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/tags?post=214"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}