Thunderbird 3 beta 1 – a platform for innovation shapes up

Standard

Today, we’re announcing our first beta-quality release since the Thunderbird project was re-energized about a year ago. It’s exciting to see the first in what will be a series of releases aimed at a broader set of testers make it out the door.

In some ways, this is a typical beta — we’ve changed a lot of code since Thunderbird 2, and we need a lot of people to tell us if we’ve made any boo-boos when fixing bugs. It’s also a good beta in that we’ve moved the product forward, in part thanks to new capabilities in the underlying Mozilla platform, which gives us faster performance all around, an add-on manager which will be even more useful for Thunderbird users than for Firefox users. We also have important new mail-specific capabilities, including a new “autosync” system that gets Thunderbird to download IMAP message bodies early, so they’re already there when you need them, and a much faster implementation for deleting and moving IMAP messages, which I can’t imagine living without at this point. The one-click add-to-addressbook is also an elegant and shameless ripoff of the Firefox bookmarking model, which our alpha users love.

As a result, I feel that even for a first beta, Thunderbird 3 is much better than Thunderbird 2, thanks to a lot of hard work by a motley crew of great contributors worldwide, to whom I’m very grateful. All that and more is described in the release notes, which I encourage beta testers to read.

However, in some other ways it’s far from a typical beta. In particular, unlike the traditional definition of a beta release, we’re definitely not done making feature changes, including some pretty significant feature work that we expect will be integrated in Thunderbird 3 in later beta releases, some features that will live as optional add-ons, and some experiments which may end up in later releases of Thunderbird or not, depending on the result of the experiments.

I’ll talk a bit about some of these upcoming attractions, as I’m quite excited about them (and some more that will have to wait for another post).

First, the autoconfig work, which refers to a complete rethink of the account configuration process in Thunderbird. The account “wizard” in Thunderbird made sense in the early days, but over the years it has acquired complexity and lost relevance, as email systems have gotten more complex. Unfortunately, if you’re lucky enough to have a secure email server, the current Thunderbird user interface unjustly punishes you by making you go through 8 pages of questions and you end up with an account which requires manual tweaks before you can check mail. That’s not good. To deal with this, we have rethought account configuration completely, and came up with a dialog which, when it lands (becomes available by default), should make account configuration really, really easy. It’s been hard to come up with an elegant minimal user interface that hides all of the complexities of email configuration, but it’s worth doing it right.

Next up is tabs. Thunderbird 3 has a great opportunity to be basically coming up with a tabbed interface at a time in history where we’ve learned a lot about how tabs work well or poorly. In Thunderbird 3 beta 1, it’s a fair bit easier to work with tabs than it was in Thunderbird 2 (although many improvements are planned before the final release). For example, it’s much easier to create new kinds of tabs (the calendar Lightning add-on makes great use of those, for example, as I show below). One simple example of this is Bryan Clark’s “glodabook” add-on, which is a starting point for exploring new ways of navigating the address book.

Addressbook prototype

Next up is conversations. Thunderbird’s default mode saves emails you send in a “Sent messages” folder, and emails you receive filed in other folders, typically decided on a per-message basis by the users (more on that below). This is a fine default strategy, but it can make it hard to find related messages if they’re not in the same place (e.g. messages that are replies to emails you sent, or messages that are part of a long conversation, some of which is in your archive folders, and some of which is in your inbox). Thunderbird 3 includes a powerful search engine (“Gloda”) which is designed to let us efficiently find messages that are related, no matter where they are. In particular, it makes it quick to take a message and “show it in a conversation context”. This lets you view the messages you sent interspersed with the messages you received, but also messages from earlier in the conversation which you may have archived. This is still experimental, and not enabled by default in 3.0b1, but early results are very promising:

Conversation view

Next up, search. Part of the Gloda search engine mentioned above is a powerful full-text search engine, which seems to be working quite well so far. Thunderbird search is already better in 3.0b1 than in 2.0 because we’re more aggressively downloading emails, and doing a better job of finding the downloaded copies. With the new search engine, we’ll be able to efficiently do searches like “show me all messages from bryan mentioning ‘conversation’ in the body or the subject”. And we think we can make that easy for users to discover as well:

First we do autocomplete on existing contacts:

autocompleting contacts

and then encapsulate them in graphical objects to simplify the display:

experimental search results view

On that topic, one of the design topics we’re exploring is how to make it easier for users to be smarter about search. Thunderbird has always had very powerful search capabilities, but to use them people have to think like database programmers, which most of us aren’t. We have some plans there to help people build smart searches based on starting with the simple searches people are used to from the web, using suggested sub-searches based on analyzing their search results. Now that we have the search engine in place, we can start to experiment with many different search models, and see what works best.

The last two screenshots are particularly exciting to me because they demonstrate that we can leverage the foundational bits of Thunderbird, and experiment with new ways of working with messages, without disrupting the user experience that Thunderbird 2 that many users are comfortable with. What’s equally exciting is that these new ways can themselves be platforms for experimentation, whether by us, or by others. One such experimentation topics is conversation visualization and interaction models. Andrew Sutherland implemented an add-on that shows thread arcs (here using a view that it out of date by a whole week):

Thunderbird has always been an interesting experimental playground, because of its open source nature and the add-on model. The technology platform in Thunderbird 3 will make it even more so — 1) we have better technology that allows new ways to slice the data, 2) as we’re exploring new features through the use of add-ons ourselves, we find out early what changes we need to make to make the platform more extensible, and 3) because we’re fully leveraging web technologies, something which is a bit new for Thunderbird. In particular, all of the views above build on some of the most compelling advances in web technology, from the canvas widget to Javascript toolkit-based animations (JQuery for now) and modern CSS features.

Finally, last but not least, the Lightning calendaring add-on is moving along great. The Thunderbird+Calendar team has made a lot of progress on tackling the stack of issues that made it hard to integrate into the new Thunderbird codebase. We’re not done yet, but it’s looking great:

calendar tab

There are some other add-ons that some contributors are working on that I’ll talk about as they get polished and ready for screenshots.

As always, we love to get ideas for interesting new capabilities we can bring to the platform. We’re focusing on some of the basic capabilities we think are crucial to solving today’s mail problems, such as search and message management, but it’s a huge field, and email users are desperate for innovative ideas.

We’re identifying way more topics of interest than we have time to tackle, so we’re hoping to reach out to designers to get a broader set of participants helping us with some of the design challenges of a modern approach to messaging, within the context of Mozilla Labs. More on that soon.

Whether you’re a designer or an implementor, if you want to build new features on top of the views we’re building, add new kinds of data to add to our database (twitter, facebook, rss, etc.), or new visualizations, do get in touch.

If you’re interested in the extensions above, and aren’t afraid to try out code that changes daily, my recommendation is to use an IMAP server, Shredder (the nightly builds of Thunderbird, which are already different than the beta 1 build), and the extensions at the following locations:

FOSSCoach, OSCON

Standard

At the last eLiberatica, I was talking to some of the speakers, and several of us reflected that while we really enjoyed giving the standard, “speak up and monopolize everyone’s attention for 20 minutes, then take 5 minutes of questions”, given the energy in the crowd, we’d be really keen, in general, to have less cathedral and more bazaar at open source conferences as a rule.

OSCON is coming up, and of the mainstream conferences it’s been quite friendly to the less commercial “unconferences”, while still being a very worthwhile event in its own, conferency way. A few points to note:

  • Dan Mosedale and I will be giving a talk about Thunderbird, and we expect you all to be there, and to walk away with confidence in the future of Thunderbird, because you will also walk away with some action items!
  • I’m on the program committee for OSCON, so I get to share this 15% off discount code on my blog. How cool is that? os08pgm is the word of the day.
  • Zak Greant is leading an unconference at OSCON on the topic of how do we transfer knowledge about community building and open source, just before we turn senile. I’ve told him I’d help lead a discussion, but I’m not sure what topic to cover. Anyway, it’s free, it’ll be fun, and you’ll get a chance to ask a bunch of experienced people

What's Mozilla's scope? What should it be?

Standard

A couple of canadians (!) have recently put up interesting posts about the Mozilla Foundation: David Eaves, with whom I had a great breakfast a few weeks ago, and Marc Surman, with whom I had a great long-distance phone chat. Both posts are worth reading, and digesting.

For what it’s worth, I agree with both.

I agree with David that the people involved in the open web (and that includes all wikipedia authors, youtube uploaders, and consumers of the same) are part of a social movement, whether they self-identify or not. I’m really interested to learn from him and people like him what the history of social movements can tell us about how to take what has traditionally been a very geeky concept (open standards, open source, etc.), and make it politically and socially not just relevant but critical and much more powerful than it is today. The “opposition” is much more astute at manipulating both courts and markets to their advantage, but that will shift if we’re ambitious enough.

I also agree w/ Marc that the Mozilla Foundation can do a lot more than what it does today, in shaping, energizing, and facilitating that movement. Especially when I’m outside of North America, it’s the Foundation that has credibility, and that credibility is currently languishing, unleveraged. We could and should do more.

It’s nice to see that the Mozilla galaxy is growing up enough that there can be simultaneous energy towards one thing, and very different but also important energy towards this complementary set of thought processes.

Oh, for the record: when I say open web these days, I mean something much broader and richer than just “the WWW using open standards”, although that’s the definition that I first used. Thunderbird’s goals, for example, are in scope, even if it doesn’t have much to do with traditional web protocols yet. Things like data portability, identity 2.0, net neutrality, data privacy, etc., are all in scope. Trying to pin down exactly what I mean with that word is something I’m trying to figure out — as David mentions, we need to do a better job of defining what we’re agreeing on.

I look forward to the conversations.

What’s Mozilla’s scope? What should it be?

Standard

A couple of canadians (!) have recently put up interesting posts about the Mozilla Foundation: David Eaves, with whom I had a great breakfast a few weeks ago, and Marc Surman, with whom I had a great long-distance phone chat. Both posts are worth reading, and digesting.

For what it’s worth, I agree with both.

I agree with David that the people involved in the open web (and that includes all wikipedia authors, youtube uploaders, and consumers of the same) are part of a social movement, whether they self-identify or not. I’m really interested to learn from him and people like him what the history of social movements can tell us about how to take what has traditionally been a very geeky concept (open standards, open source, etc.), and make it politically and socially not just relevant but critical and much more powerful than it is today. The “opposition” is much more astute at manipulating both courts and markets to their advantage, but that will shift if we’re ambitious enough.

I also agree w/ Marc that the Mozilla Foundation can do a lot more than what it does today, in shaping, energizing, and facilitating that movement. Especially when I’m outside of North America, it’s the Foundation that has credibility, and that credibility is currently languishing, unleveraged. We could and should do more.

It’s nice to see that the Mozilla galaxy is growing up enough that there can be simultaneous energy towards one thing, and very different but also important energy towards this complementary set of thought processes.

Oh, for the record: when I say open web these days, I mean something much broader and richer than just “the WWW using open standards”, although that’s the definition that I first used. Thunderbird’s goals, for example, are in scope, even if it doesn’t have much to do with traditional web protocols yet. Things like data portability, identity 2.0, net neutrality, data privacy, etc., are all in scope. Trying to pin down exactly what I mean with that word is something I’m trying to figure out — as David mentions, we need to do a better job of defining what we’re agreeing on.

I look forward to the conversations.

eLiberatica 2008

Standard

I’m back from Romania, where I had the pleasure of participating in the second annual eLiberatica conference, on open source, organized by Lucian Savluc, a fellow Vancouverite, with organizational support from ROSI, and with Zak’s assistance.

It was a lot of fun, as well as a highly energizing event (which is hard when the base condition is one of severe jetlag compounded by not enough sleep).

My slides are now available, although you don’t get to appreciate all of the fancy animations that Keynote does (note to the OpenOffice.org folks: make the animations really sexy, and I’ll switch. I’m a sucker for pizzazz.).

I have a bunch of thoughts that I want to blog as a result of my trip, but as usual, finding the time to organize them into a coherent post is the challenging bit.

Executive summary:

  • eLiberatica was great, look forward to seeing how it evolves
  • Traveling to foreign lands with hosts such as Lucian and his friends is definitely the way to travel
  • Travel and the subsequent impact on relationships is incredibly important to the long term success of open source, and of Mozilla. We have to make it easier, more decentralized, and less guilt-inducing
  • I want to travel more. I want to travel less.
  • Romania is changing incredibly fast. There are lots of opportunities, and decisions which will have significant impact are being made there every day
  • Zak’s network includes really interesting, thoughtful folks.
  • The world is incredibly small, and yet very big
  • If they’ll honest about it, everyone likes dancing to hits of the eighties.

Open source networked interactive whiteboards?

Standard

If you’ve never seen Johnny Lee’s videos explaining how he uses cheap Wii remotes to do amazing things, check out the TED video, or his home page.

Now after watching that video, I snuck into the ActiveState game room, and poked around with the wiimotes & projector there. I’ll definitely be keeping an eye on the wiimote project.

Now someone needs to hook that project up with some of the open source screen-sharing software, and all kinds of cool things could happen.

Speaking of which — any recommendations for inexpensive projectors (for presentations, not home theater stuff)?

Romania ahead

Standard

I’ll be heading out to Romania next week, to talk in Bucharest at eLiberatica about Mozilla and open source, and to learn about everything from open source in eastern Europe to food (always). I’ll also be visiting my grandfather’s hometown (Bra┼čov) for a bit of personal root-digging. Should be fun, especially if I get rid of this grogging (new word!) cold I’ve just picked up.

As a side note: it’s interesting to see that an open source conference is now sponsored by everyone from the FSF Europe to Microsoft. What is the world coming to?

Prize money for a good Thunderbird/OpenOffice.org project

Standard

Something else I’ve had on my blogging queue for a while:

Last month, when we had the Calendar meeting in Hamburg, we met with some of the OpenOffice.org and Sun engineers. One idea that came up that I intended to mention here is that OpenOffice.org has a Community Innovation Program, funded by Sun Microsystems, which includes cash prizes for cool projects. A project which somehow made Thunderbird and OpenOffice.org work better together would be eligible, as much as I understand the rules. So put your thinking cap on and apply!

Possible ideas:

  • An ODF reader for Thunderbird?
  • Some sort of mail merge feature?

Other ideas?

University of Toronto Student Projects

Standard

What feels like years ago, my friend Greg Wilson, a book author CS prof at the University of Toronto with a deep understanding of both the practical realities of software engineering and open source, roped me into being a “client” for a class he teaches on software engineering, where he matches students with open source projects, and gets the projects to act as if they were typical clients. I have to say, I behaved like a typical bad client. Fuzzy requirements, lousy documentation on our system, erratic email, never around to meet in person, etc.

Still, the two students assigned to me, Mike Wu and Ronald Fung, did a great job. Ronald Fung led a messy part of the project, which was about teaching Thunderbird how to detect RSS/Atom feeds in pages that mention feeds but aren’t feeds themselves, work which will likely make its way into Thunderbird 3 at some point. Thanks!

Mike Wu led the development of a Remember The Milk provider to Lightning, the calendaring add-on to Thunderbird. RTM is a popular website for tracking TODOs, so it seemed a good alternative storage mechanism for Lightning users (alternatively, Lightning seemed like a good front end for RTM users). That add-on is now on addons.mozilla.org. Nice job!

Thunderbird team needs help from Python/Perl build engineer

Standard

If you’re a Thunderbird fan but not interested in fixing some of the nasty C++ problems we tackle in the product, you could still be very, very helpful if you can help us with a little Python/Perl build problems.

Specifically, Mozilla has a great system called “try servers” where one can submit patches against the tree, and the build system runs builds on Linux, Mac and Windows, using those patches, then serves those builds for testing. This is really helpful to figure out if proposed patches solve specific problems, especially when the developers aren’t able to reproduce the bugs, but testers can. It works great for Firefox development right now.

The only problem is that there’s a little bit of patching needed to the try server code itself to make it able to work with other targets besides Firefox, as described in bug 431375. Ben Hearsum, a build guy from Mozilla, is happy to help someone figure out those patches, but he doesn’t have time to make it happen right now.

From what i can tell, required skills include comfort with CVS and Linux, Python, Perl, and some build engineering common sense.

Thanks in advance!