Petroglyphs

My monitor has started to lose it in the last week or so. It started with a bright line near the bottom of the screen that I eventually realized was a wrapping of the bottom few pixels. Slowly it’s worked its way up the screen, to the point where as I began typing this message the entire bottom half was rolled up into a centimetre-wide band of white, and no amount of degaussing or power-cycling will fix it.

So now I have a problem: do I go geeky (and expensive) and go for an LCD screen, or stick with the tried-and-(mostly)-true CRT?

In yet another fit of great minds think alike (yeah, I wish), I made some small updates to Stuart Langridge‘s nicetitles… and when I went to send the diffs to him, found that Brad Choate had done almost exactly the same, and added another feature too.

My patch to handle the editing elements in (X)HTML is slightly different than Brad’s in two respects.

1. Brad’s patch only deals with <ins>. I often use ins and del in pairs, such as in this post, so it’s nice to have support for both.
2. Brad’s patch parses a datetime format that doesn’t match the specific ISO8601 format specified in HTML 4.01 (and, by inference, XHTML 1.0). My version isn’t quite right–it doesn’t use the specified timezone, because JavaScript’s Date doesn’t–but it’s close and compact.

I also stripped a bunch of old browser-specific code that isn’t used any more after Paul McLanahan‘s remaining-title modifications, and reformatted for consistency.

Here are nicetitles in all their glory: inserted, deleted, and anchored.

Good lord, I just quoted the Spice Girls. I need to sit down for a second.

OK, better now. Still, this will just be a jumble of thoughts more than a proper entry.

I’ve been playing some more with Spycyroll, and I think I’m making headway on adapting it for my purposes. Until today I’ve just been letting it continue to accumulate posts without removing any; as a result, my aggregate page was well over 400K and growing. (The reason for this is related to the date issues I referred to earlier.) Tonight I got (slightly) brighter and realized all I had to do is to put read files in one directory and unread files in another. Through the magic of Python, that took all of about five lines of code, and my aggregate page is a much healthier 16K.

Holding on to all of those deleted files is still an issue. Because I can’t tell what items are no longer in a feed, it’s necessary to hold on to all of them. I’m thinking a database will be necessary, probably of MD5 checksums for each post, but I’m not comfortable enough with Python to start messing around with its database support yet.

I’ve also realized that although rssparser.py does nice resource retrieval (using If-Modified-Since/Last-Modified, If-None-Match/ETag, and Accept-Encoding: gzip), Spycyroll doesn’t take advantage of it. I’ll probably use the filesystem to store that information too in the interim.

A few things to consider there, and there’s more, but I’m getting antsy to try some of this out….

It works, mostly. Now that I’m taking advantage of smart retrieval, the site links in the blogroll part of the page aren’t being filled in, because they’re pulled from the channel. Two steps forward….

AmphetaDesk has started crashing on me every time I try to reload its webpage. I’ve been thinking for a while about trying another RSS aggregator, and this seemed like a good time to do it.

So what better place to start than Google? I’m not sure exactly what made me look for a Python aggregator–probably reading Mark Pilgrim‘s weblog has had some more-than-subliminal effect. I’d tried Straw and Peerkat before, but both had their problems: Straw is heavily GNOME 2 dependent, and Peerkat didn’t handle newer RSS feeds. (I’ve since upgraded to GNOME 2 via RedHat 8, but Straw requires more libraries than I care to attempt to find or install.)

Oh yes, the Google search. I came across this post which pointed to Mark Pasc‘s notes, which in turn led to SpycyRoll (aka Pyblagg). It’s kind of similar to AmphetaDesk in that it’s a browser-based tool, but that’s about it. For one, it only runs periodically, instead of continuously. It’s forgiving of RSS errors (mostly; more later) via Mark Pilgrim’s ultra-liberal RSS parser. It caches its output page rather than regenerating the content at each reload. And, also via Mark’s parser, it seems to Do The Right Thing when it retrieves feeds by respecting and using ETags, If-None-Match, If-Modified-Since and User-Agent headers and gzip encoding, which is better than AmphetaDesk’s hand-rolled (but fairly effective) way.

Which isn’t to say it’s perfect. While the parser is tolerant of errors, the code to handle the values retrieved isn’t particularly forgiving. Of the thirty-some feeds I read, three couldn’t be read and so were ignored; I’ve worked around the two underlying problems with the little bit of Python I’ve learned in the last month (mostly from Mark Pilgrim again, this time via Dive Into Python). The other big problem I’ve found is that for feeds that don’t provide per-item dc:date values–i.e. most of the ones I read–it will continually recreate its caches of those items, throwing off the display and ordering of the items. I’ve worked around this too, but at the expense (I think) of breaking If-Modified-Since retrieval for those feeds.

I can live with SpycyRoll; after a couple of days, it already feels better to me than AmphetaDesk, for mostly intangible reasons. Neither one is what I ultimately want, but currently SpycyRoll is the more usable of the two for me, even ignoring AmphetaDesk’s crashes. (I haven’t been able to track down why AmphetaDesk is crashing, it just is; the shell reports a Broken pipe. I’m hesitant to report it to Morbus Iff without any further information or investigation, but let’s see just how good his ability to appear out of nowhere really is. <grin>)

There are still features I haven’t come across yet that I’d like to see in a browser-based aggregator. I want to be able to remove some items without reading them while making others sticky. I want headline, compact and expanded views, and mixed-feed and single-channel modes. (The kitchen sink would be nice, too.) To my way of thinking, these would be best accomplished using a combination of server-style commands (à la AmphetaDesk’s Add New Channel and My Settings pages) and DOM manipulation on top of the functionality of SpycyRoll. As Morbus Iff suggests (in another context), these are things better handled by limited-purpose shell utilities instead of a monolithic single application. I’m going to try to add some of these things to SpycyRoll in the next few days… wish me and my not-so-mad Python skillz luck….