Posts in: Development

We’ve been hard at work since the last release: refactoring and optimizing code, fixing bugs and generally making our foundation more stable.

The Alpha 3 release includes over 100 tracked fixes and enhancements, plus a few dozen more that were made along the way. Some of the notable changes include:

• A complete rewrite of the feed refreshing process. It is now multi-threaded and using our smart update schedule. The update process now comes calling with a FeedLounge user agent which includes not only how many subscribers this feed has in FeedLounge, but also when we’ll be back to check again.
• A new toolbar for the feeds list, completing the contextual toolbar implementation.
• Revamped pop-up windows that keep the buttons on the screen at all times.
• Support for Page Up and Page Down keys in the feeds and items lists, and the ability to open and close tags in the feeds list using the ‘=’ key.
• A new preference setting lets you choose the default state of the ’save items until I delete them’ options when subscribing to new feeds and importing OPML.
• OPML export (the link is in Settings - My Account).
• Lots and lots and lots of little changes and fixes - both front and back-end.

Now that we feel we’ve shored up areas of the platform that needed it and spackled a few nail holes, our plan moving forward is to push out more feature focused releases on a faster timeframe. Having more bite size releases rather than cooking up a whole meal for each release should help us get specific testing and feedback on these features from our alpha users faster, and should make the releases easier for us to manage.

Last but certainly not least, a well-deserved pat on the back to Brian who did a great job knocking tons of bugs off the list to get this release ready to go.

Now that there’s been a little wider testing, it seems I was a little overly optimistic when I mentioned we might already have Safari support in FeedLounge. Everything works fine with the layout and the mouse, but the keyboard navigation doesn’t work the same. It’ll be fixed before beta.

If there is anyone out there on the Safari team that would like to join the alpha test and help us track down tough Safari issues, please get in touch.

In FeedLounge, there is a link titled ‘My Account’ which pops up a window that shows your account details and preferences (currently only one, but more are being added).

From the feedback received in the forums so far, I think people are not thinking to look in ‘My Account’ for options or preference settings. Would titling this link ‘Preferences’ or ‘Options’ would make you more likely to click on it when looking for various options? Perhaps another term?

The comments are open…

Welcome to the second of a series of posts on the development. If you missed the first one, check it out here: FeedLounge development: the parser.

The feed validator

We have feeds being parsed, but we also wanted to help make the world a better place by allowing the end user to know whether or not the feed is valid. As Geof rightly points out, we MUST follow Postel’s Law when parsing feeds (”be conservative in what you do, be liberal in what you accept from others”). Why not give a quick heads-up to someone that might be able to help fix the problem?
So, in FeedLounge, there is a banner near the item content that tells the user the feed is invalid. The user is given a link to click on to see for themselves what is wrong with the feed, using the service from feedvalidator.org.

Hopefully someone will come along that knows the person publishing the feed, and helps nudge them to fix the problem, to make the world a better place for all feed reading entities. If this banner gets too annoying, it can be hidden down to a small icon, and the user can go on with her feed reading. And of course this setting is persistent, so the user does not continue to be annoyed.

Quick Note to Sam and Mark: First, thank you from the entire FeedLounge team for the excellent code that is feedvalidator. Second, I hope you will yell at us if we are not using feedvalidator.org according to your Terms of Service, we believe that we are, as it takes the end user clicking on the link to activate feedvalidator.org, all of our backend validation is done on our own server.

Interesting stats around validation

Of all the feeds that FeedLounge is currently parsing, validating, and tracking, 34.5% have some issue, 22.5% are NOT valid, and 17.8% are completly broken (404, 304, no response at all, etc). That’s 1/5 to 1/3 of all feeds that FeedLounge might not allow the users to read if we were using a strict feed parser!

As a side note, a large portion of the invalid feeds so far are from some version of WordPress. There are a lot of users out there that haven’t felt that upgrading is important. Please upgrade!!! The security fixes alone are worth it. I hope in a future release that the WordPress team might use one common codebase for feed creation, rather than separate code for each feed format. Disclaimer: dotnot.org uses WordPress 1.5.1.2, and will not change to something else anytime soon. I like WordPress, just offering constructive criticism, and I just want to give the WP team a friendly nudge from time to time

We have noted in our alpha invitations that we intend for FeedLounge (company, people and application) to be as open as we can possibly be. So along those lines, I will be posting here and on the FeedLounge Blog about architecture, features and development of FeedLounge, so that everyone can see inside the beast, so to speak.

Which feed parser should we use?

When are you building a web based feed reader like FeedLounge, having data to read is step one. Luckily, there are many feed parsers already out there, so the “build vs. buy” decision was fairly easy. Focusing on the development of the user experience of the feed reader, the feed parser part of the application is only a ‘necessary evil’ in the scheme of things. After checking out several possiblities, including using my own Java/SAX framework, we decided on feedparser, the canonical namesake of the feed parsing world. Built by Mark Pilgrim, and currently at version 3.3, this is probably the most forgiving feed parser on the planet. Had I gone with my own solution, I would have spent months and months creating something as good. And with a liberal open source license, I am allowed to use it in a commercial project like this.

feedparser features

• feed format support - v3.3 has impress support of 4 feed formats and 15 different versions of those formats. This probably would have taken a good chunk of time to come up with support for.
• encoding detection - Anyone who has done this understands the difficulty without any explanation.
• tidy support - Want clean HTML content as output? No problem, it’s in there
• translated access between specific terms - If you know channel instead of feed, these are the same thing in feedparser. Use the terms that you are comfortable with.
• relative url support - Useful to us since we are ripping the feed apart to store it. Having no relative URLs is a great relief.
• great documentation - Mark produces some of the best, most-useful documentation in the open source world. feedparser is no exception here. Terse, but covering what you need to know. Need to do 401 auth? Here. Wondering about E-Tag support? There.
• over 2000 unit tests - I may run into some arcane case not covered here, but the likelihood is not very high.
• HTML sanitizing - Extremely useful for a feed reader, to prevent bad things. You don’t want to let someone else’s JavaScript run inside your app. Debugging that would be a nightmare, and maliciousness is also a concern.
• date parsing - Support for every date format they came across. You get a simple date format, consistent from feed to feed.
• It just works!- The best is saved for last, as this point cannot be made often enough. In the months of development so far, feedparser has never been the spotlight of a single problem. The closest we have come to some kind of problem is not checking for the existence of some item before accessing it. feedparser has been a huge net positive on development, with an almost nil overhead. To have alpha testers say that some of the feeds that don’t open in nearly anthing else show up in FeedLounge, that wasn’t us, it was feedparser and its magic voodoo.

Mark, thanks a million. I know you have ‘gone dark’ in the blogging world, but you are still rocking mine.

I have decided to move the FeedLounge devlopment series from my blog to the FeedLounge blog, since I had published 2 already, they seemed to fit here better. I will keep the already published articles on dotnot as well. Happy reading!

The FeedLounge application UI is uses XHTML and CSS exclusively for the front end. While the XHTML does not currently validate due to our use of custom attributes (unless we write a custom DTD - which we probably will in the future), it was encouraging for me to see that you can build a complex web UI using XHTML + CSS.

This XHTML + CSS thing just might catch on, I tell ya.