7 ms·

Open-sourcing OvertureJS – the JavaScript lib that powers FastMail

181 points by robn_fastmail 12y ago

brongondwana 12y ago
I've been pushing for this to be done for a while :) There's no point open-sourcing the crappy old parts of our architecture which we wouldn't advise new projects to use (like our Perl framework. There are so many Perl frameworks these days, and many of them better than ours) - but Overture is high quality stuff. Enjoy!
- dingdingdang 12y ago
  Thanks for sharing, looks good.
- taternuts 12y ago
  I'm really enjoying looking through the source, this is some nice javascript - thanks!
- 616c 12y ago
  So I opened my fastmail account for about a year. I have to say ever blog post, chock full of technical content, your dedication to showing up front your patched versons of Cyrus IMAP, and other stuff you brongondwana and the company at large do, are very badass. I give you often as an example of transparency in the marketplace and I love what you guys do. Thank you for taking me away from Google! I have had no downtime, but your work has gotten me interested in many mail clients I have tried and I intend to host my own Cyrus IMAP instance to learn more about email in general. Thanks for that too!
  - brongondwana 12y ago
    I'm hoping to be able to release Cyrus 2.5 soon. We just rebased the FastMail branch on top of the current master tree and rolled it to 2% of our users with no issue - it was months behind until recently. Planning to roll that out to everyone very soon. I've been testing it just with a few of my own users and patching issues for the last month.
    - FractalNerve 12y ago
      What Git workflow do you have? Is it the something simple like testing, staging, production or is it much more customized for your needs? Recently I got very interested in that, since we evaluated Phabricator and Gerrit[1] (we choose Gerrit btw.). Ok this had to be asked: Do you send commits via mail too? :) -- [1] https://code.google.com/p/gerrit/ https://code.google.com/p/gerrit/
      - brongondwana 12y ago
        We build two separate Debian packages: cyrus-fastmail for production, and cyrus-future for testing. Git workflow - we push branches named fastmail and future to our git repo. They are pretty aggressively rebased on the upstream master branch. We have unit tests and a couple of more comprehensive test suites we run against code before it hits future. Future runs on a single store with one real user (my personal account) and a bunch of users that the functional tests run against. Once we're happy, we push that same branch to fastmail and rebuild. There's a commit hook which sends a diff against the previous build tag to the mailing list so you can easily see what is being rolled out, both for future and for fastmail. We do some horrible things to git for the rest of our system too. Our pushback tooling that builds squashed feature branches onto host branches is pretty crazy. It mostly does what we need though.
  - nailer 12y ago
    Speaking of Cyrus - are there OSS push IMAP servers these days?
- scrollaway 12y ago
  Thank you for the open sourcing of that lib. It looks great. > There's no point open-sourcing the crappy old parts of our architecture which we wouldn't advise new projects to use Well, there is an educational point to be made about this... :) I think too many people in open source focus on "if it's open source, it must be great, look great" etc. I for one put every crappy project I start and never complete on Github along with the good ones I'm proud of. Because I like my "learning process" being out in the open as well. I don't publish my hello-worlds of course but I don't specifically hide stuff people "I believe should not use". Warnings in the readme are as far as it goes. My company systematically opens its entire process to the public. All the code is on Github, all the issues are there too. Our internal discussion/design process is on IRC, Google Docs and the Github wiki and they are all public and archived. Only client work under NDA remains private. Google has a similar-ish model for some open source projects it released. I wish more companies would adopt a model like this. Well, of course, open sourcing important frameworks and useful stuff remains a higher priority... but I don't believe there is any harm in it, and it actually holds the employees to a subconscious "higher standard". I'd love to talk more about it but I think I'm getting off-topic here. Once again, congratulations on the release!
  - robn_fastmail 12y ago
    The other side is that it does take a not-insignificant amount of time and effort to prepare a fifteen-year-old codebase for the public. There's got to be some benefit to the business to make it worth that. The old web framework that we use is good for what it is, and we understand it, but its not state-of-the-art by any stretch, it does some weird things that aren't good practice in "modern" Perl, its still fairly heavily tied to Apache2/mod_perl and other parts of our internal libs, and so on. We don't use it for new code ourselves and we're actively migrating everything written in it onto Overture-based UI. So given that even we don't want to use it and the off-the-shelf Perl web frameworks are generally better, it really doesn't help anyone to make it open source. That's not to say there's nothing in our system not worth open-sourcing. For example, we've deliberately written our CalDAV/CardDAV libs in a nice clean modular way because we've planned to release them from the very beginning. Its just some things, like the web framework, that have no particular value.
dingdingdang 12y ago
Overture is impressive in its inclusiveness, from article: "There’s also one-line support for animating views. You declare the layout property and its dependencies, and Overture will handle animating it between the different states. Full support for drag and drop, localisation, keyboard shortcuts, inter-tab communication, routing and more mean you have everything you need to build an awesome app." Especially having pre-fixed inter-tab comm. sounds enticing for me, haven't heard of other frameworks handling this out of the box!
tomcam 12y ago
UndoManager is an unusual and welcome addition to such libs. Codebase as a whole looks pretty darn clean.
brohoolio 12y ago
Awesome! Thanks fastmail folks.
WasSlowbanned 12y ago
Is this framework inspired by Cocoa at all? seems like it from just glancing at it.
- nmjenkins 12y ago
  As mentioned in the article, the core object system was inspired by SproutCore (4 years ago), which was in turn inspired by Cocoa, so yes there is a heritage. However, Overture is designed to enhance web tech rather than turn it into something else (which was always the impression I got with SproutCore).
- ralmidani 12y ago
  They say the object system is inspired by Sprout core, which is inspired by Cocoa.
couchand 12y ago
Very excited about this. Was just having a conversation about what to migrate our Angular app since we're no longer happy with Angular. Also, looks like until Tuesday it was named Vibrato [0]. I think I can understand the motivation to change that. [0]: https://github.com/fastmail/overture/commit/2a8252764a124b191d7431cd7f47bbcb64c290a6 https://github.com/fastmail/overture/commit/2a8252764a124b19...
- M4v3R 12y ago
  Can you say more about why you aren't happy with Angular anymore?
  - ericthegoodking 12y ago
    @couchand I am also interested why you are migrating from Angular
  - couchand 12y ago
    My experience has been that performant code is very hard to write. In particular, the tools that Angular provides for the most common sorts of things you'd want it to do (ng-repeat, ng-show/hide, ng-click, $on and $watch) all break down for one reason or another when trying to use them in anything but a toy application [0]. When I realized that ng-hide just applies CSS classes I did a huge facepalm. Consider the example of dynamically constructing a table. In Angular: <tr ng-repeat="child in children"> <td ng-show="foo(child)">Foo</td> <td ng-hide="foo(child)">Bar</td> </tr> This example runs the function foo four (!) times per digest cycle per item. Sure, you could convolute your code to make the Angular more performant, but at the cost of clear and maintainable markup. What's more, this produces rows with two cells, one of them hidden with CSS, which is ripe for creating hard-to-debug issues. The alternative in basically any other framework is written in a way and produces code execution and markup that you'd intuitively expect. Angular's directive system encourages developers to write illegal markup. Enough said. Angular's DI system is needlessly complicated, and the basic usage breaks when minifying because it couples identifier names to string values. That same issue is repeated in many different aspects of Angular: the basic, intuitive usage is (intentionally or not) broken in a production environment. The amount of deep knowledge of the Angular framework that's required to successfully develop apps is absurd. [0]: https://www.airpair.com/angularjs/posts/angularjs-performance-large-applications https://www.airpair.com/angularjs/posts/angularjs-performanc...
speg 12y ago
I just signed up for Fastmail this weekend and the snappyness of their app really impressed me. I emailed Neil and asked him about the framework and he said they would release more info soon – happy to see it so quick! I'm assuming the mobile app uses the same in a Webview?
- nmjenkins 12y ago
  Yes. Everything is a single codebase, desktop and mobile, and it's all powered by Overture.
- zzleeper 12y ago
  By the way, what made you switch? (And also, did you got the $40 version or..?) I'm thinking about migrating (gmail just feels sluggish), but I'm a little afraid of all the potential gotchas vs my current gmail setup, which makes me reluctant. (I redirect + pop fetch several school addresses to gmail, plus have a long list of filters to help me navigate all the ads I get)
  - sitharus 12y ago
    I'm not op, but I switched to fastmail for my family hosting a year ago. I was driven away because of the continual UI changes Google were making. I don't want to re-explain the interface to my mum every three months. Fastmail has nice buttons with words on - when I switched the first conversation I had was "so how do I send an ema… oh there it is!" Also fastmail is faster! You get email, contacts and file storage. If you are on the $40/yr plan you get caldav support as well. Works very well on all my devices. Only downside is spamassassin isn't as effective as gmail's spam filters, but I no longer have to cope with mailing lists being marked as spam by the masses who can't work an unsubscribe button. On that note, you can also write your own seive rules for filtering, and there's the 'if matched stop running filters' option that is so glaringly absent from gmail. So I'm a happy customer.
  - ukd1 12y ago
    Downsides; filtering sucks compared to gmail, keybindings aren't gmail
itsbits 12y ago
it looks more of early Ember type Architechture. I can see since both followed Sproutcore.
- dhatch387 12y ago
  It would be interesting to hear more about where Overture and Ember diverge. The templating system is a clear difference, however with Ember moving to HTMLBars, which emits DOM instead of string, I'm not sure how relevant the innerHTML performance comparison is.
bambax 12y ago
I use and love Backbone, how does this compare? What are the pros and cons of each approach?
gcb0 12y ago
it uses sugared dom for template. while it's fast on mobile web kit, it's shower than inner html on Firefox mobile. and if you are serious about mobile you know that's the future.
- coldtea 12y ago
  You know "what's the future"? Firefox mobile? Hardly.
- robn_fastmail 12y ago
  That doesn't match our experience. Do you have benchmarks to support this? I'd be interested to take a look.
  - gcb0 12y ago
    install Firefox mobile, go to the jsperf link on the article. Firefox mobile is still ignored in San Francisco bubble, but it's gaining rapid adoption on influencers... I'm putting all my money on it being everyone's browser in the next years. it will take everyone by surprise just like chrome did when it was faster.
AbuAssar 12y ago
very nice work , good job. I'm considering using it , but I encountered a small bug regarding i18n: in the todo example , I added a new todo with arabic script , but the search function didn't work on the arabic script. thanks for sharing
- robn_fastmail 12y ago
  Please report this in this issue tracker: https://github.com/fastmail/overture/issues https://github.com/fastmail/overture/issues
drinchev 12y ago
It is really well done. Bravo. I'm not sure about your team opinion about modifying globals. I saw this a lot in your code. Why did you decided on this approach?
- nmjenkins 12y ago
  I presume you mean extending native prototypes? If you're in complete control of the code in your page there's no problem doing this and it makes some code much neater and more readable. The only other global is "O", which is the namespace object for the library.
NikhilVerma 12y ago
@nmjenkins - so glad to see it's finally out for everyone to use! :)
alexro 12y ago
You guys need to join forces to work on ReactJS - the concepts pretty much overlap, only React takes it further, providing good compromise between in-code templating and html files.
adamdicarlo 12y ago
Am I missing something, or are there no unit tests whatsoever? Really cool to open source this, though!

181 points by robn_fastmail 12y ago

I've been pushing for this to be done for a while :) There's no point open-sourcing the crappy old parts of our architecture which we wouldn't advise new projects to use (like our Perl framework. There are so many Perl frameworks these days, and many of them better than ours) - but Overture is high quality stuff. Enjoy!

dingdingdang 12y ago
Thanks for sharing, looks good.
taternuts 12y ago
I'm really enjoying looking through the source, this is some nice javascript - thanks!
616c 12y ago
So I opened my fastmail account for about a year. I have to say ever blog post, chock full of technical content, your dedication to showing up front your patched versons of Cyrus IMAP, and other stuff you brongondwana and the company at large do, are very badass. I give you often as an example of transparency in the marketplace and I love what you guys do. Thank you for taking me away from Google! I have had no downtime, but your work has gotten me interested in many mail clients I have tried and I intend to host my own Cyrus IMAP instance to learn more about email in general. Thanks for that too!
- brongondwana 12y ago
  I'm hoping to be able to release Cyrus 2.5 soon. We just rebased the FastMail branch on top of the current master tree and rolled it to 2% of our users with no issue - it was months behind until recently. Planning to roll that out to everyone very soon. I've been testing it just with a few of my own users and patching issues for the last month.
  - FractalNerve 12y ago
    What Git workflow do you have? Is it the something simple like testing, staging, production or is it much more customized for your needs? Recently I got very interested in that, since we evaluated Phabricator and Gerrit[1] (we choose Gerrit btw.). Ok this had to be asked: Do you send commits via mail too? :) -- [1] https://code.google.com/p/gerrit/ https://code.google.com/p/gerrit/
    - brongondwana 12y ago
      We build two separate Debian packages: cyrus-fastmail for production, and cyrus-future for testing. Git workflow - we push branches named fastmail and future to our git repo. They are pretty aggressively rebased on the upstream master branch. We have unit tests and a couple of more comprehensive test suites we run against code before it hits future. Future runs on a single store with one real user (my personal account) and a bunch of users that the functional tests run against. Once we're happy, we push that same branch to fastmail and rebuild. There's a commit hook which sends a diff against the previous build tag to the mailing list so you can easily see what is being rolled out, both for future and for fastmail. We do some horrible things to git for the rest of our system too. Our pushback tooling that builds squashed feature branches onto host branches is pretty crazy. It mostly does what we need though.
- nailer 12y ago
  Speaking of Cyrus - are there OSS push IMAP servers these days?
scrollaway 12y ago
Thank you for the open sourcing of that lib. It looks great. > There's no point open-sourcing the crappy old parts of our architecture which we wouldn't advise new projects to use Well, there is an educational point to be made about this... :) I think too many people in open source focus on "if it's open source, it must be great, look great" etc. I for one put every crappy project I start and never complete on Github along with the good ones I'm proud of. Because I like my "learning process" being out in the open as well. I don't publish my hello-worlds of course but I don't specifically hide stuff people "I believe should not use". Warnings in the readme are as far as it goes. My company systematically opens its entire process to the public. All the code is on Github, all the issues are there too. Our internal discussion/design process is on IRC, Google Docs and the Github wiki and they are all public and archived. Only client work under NDA remains private. Google has a similar-ish model for some open source projects it released. I wish more companies would adopt a model like this. Well, of course, open sourcing important frameworks and useful stuff remains a higher priority... but I don't believe there is any harm in it, and it actually holds the employees to a subconscious "higher standard". I'd love to talk more about it but I think I'm getting off-topic here. Once again, congratulations on the release!
- robn_fastmail 12y ago
  The other side is that it does take a not-insignificant amount of time and effort to prepare a fifteen-year-old codebase for the public. There's got to be some benefit to the business to make it worth that. The old web framework that we use is good for what it is, and we understand it, but its not state-of-the-art by any stretch, it does some weird things that aren't good practice in "modern" Perl, its still fairly heavily tied to Apache2/mod_perl and other parts of our internal libs, and so on. We don't use it for new code ourselves and we're actively migrating everything written in it onto Overture-based UI. So given that even we don't want to use it and the off-the-shelf Perl web frameworks are generally better, it really doesn't help anyone to make it open source. That's not to say there's nothing in our system not worth open-sourcing. For example, we've deliberately written our CalDAV/CardDAV libs in a nice clean modular way because we've planned to release them from the very beginning. Its just some things, like the web framework, that have no particular value.

dingdingdang 12y ago

Overture is impressive in its inclusiveness, from article: "There’s also one-line support for animating views. You declare the layout property and its dependencies, and Overture will handle animating it between the different states. Full support for drag and drop, localisation, keyboard shortcuts, inter-tab communication, routing and more mean you have everything you need to build an awesome app." Especially having pre-fixed inter-tab comm. sounds enticing for me, haven't heard of other frameworks handling this out of the box!

tomcam 12y ago

UndoManager is an unusual and welcome addition to such libs. Codebase as a whole looks pretty darn clean.

brohoolio 12y ago

Awesome! Thanks fastmail folks.

WasSlowbanned 12y ago

Is this framework inspired by Cocoa at all? seems like it from just glancing at it.

nmjenkins 12y ago
As mentioned in the article, the core object system was inspired by SproutCore (4 years ago), which was in turn inspired by Cocoa, so yes there is a heritage. However, Overture is designed to enhance web tech rather than turn it into something else (which was always the impression I got with SproutCore).
ralmidani 12y ago
They say the object system is inspired by Sprout core, which is inspired by Cocoa.

couchand 12y ago

Very excited about this. Was just having a conversation about what to migrate our Angular app since we're no longer happy with Angular. Also, looks like until Tuesday it was named Vibrato [0]. I think I can understand the motivation to change that. [0]: https://github.com/fastmail/overture/commit/2a8252764a124b191d7431cd7f47bbcb64c290a6 https://github.com/fastmail/overture/commit/2a8252764a124b19...

M4v3R 12y ago
Can you say more about why you aren't happy with Angular anymore?
- ericthegoodking 12y ago
  @couchand I am also interested why you are migrating from Angular
- couchand 12y ago
  My experience has been that performant code is very hard to write. In particular, the tools that Angular provides for the most common sorts of things you'd want it to do (ng-repeat, ng-show/hide, ng-click, $on and $watch) all break down for one reason or another when trying to use them in anything but a toy application [0]. When I realized that ng-hide just applies CSS classes I did a huge facepalm. Consider the example of dynamically constructing a table. In Angular: <tr ng-repeat="child in children"> <td ng-show="foo(child)">Foo</td> <td ng-hide="foo(child)">Bar</td> </tr> This example runs the function foo four (!) times per digest cycle per item. Sure, you could convolute your code to make the Angular more performant, but at the cost of clear and maintainable markup. What's more, this produces rows with two cells, one of them hidden with CSS, which is ripe for creating hard-to-debug issues. The alternative in basically any other framework is written in a way and produces code execution and markup that you'd intuitively expect. Angular's directive system encourages developers to write illegal markup. Enough said. Angular's DI system is needlessly complicated, and the basic usage breaks when minifying because it couples identifier names to string values. That same issue is repeated in many different aspects of Angular: the basic, intuitive usage is (intentionally or not) broken in a production environment. The amount of deep knowledge of the Angular framework that's required to successfully develop apps is absurd. [0]: https://www.airpair.com/angularjs/posts/angularjs-performance-large-applications https://www.airpair.com/angularjs/posts/angularjs-performanc...

speg 12y ago

I just signed up for Fastmail this weekend and the snappyness of their app really impressed me. I emailed Neil and asked him about the framework and he said they would release more info soon – happy to see it so quick! I'm assuming the mobile app uses the same in a Webview?

nmjenkins 12y ago
Yes. Everything is a single codebase, desktop and mobile, and it's all powered by Overture.
zzleeper 12y ago
By the way, what made you switch? (And also, did you got the $40 version or..?) I'm thinking about migrating (gmail just feels sluggish), but I'm a little afraid of all the potential gotchas vs my current gmail setup, which makes me reluctant. (I redirect + pop fetch several school addresses to gmail, plus have a long list of filters to help me navigate all the ads I get)
- sitharus 12y ago
  I'm not op, but I switched to fastmail for my family hosting a year ago. I was driven away because of the continual UI changes Google were making. I don't want to re-explain the interface to my mum every three months. Fastmail has nice buttons with words on - when I switched the first conversation I had was "so how do I send an ema… oh there it is!" Also fastmail is faster! You get email, contacts and file storage. If you are on the $40/yr plan you get caldav support as well. Works very well on all my devices. Only downside is spamassassin isn't as effective as gmail's spam filters, but I no longer have to cope with mailing lists being marked as spam by the masses who can't work an unsubscribe button. On that note, you can also write your own seive rules for filtering, and there's the 'if matched stop running filters' option that is so glaringly absent from gmail. So I'm a happy customer.
- ukd1 12y ago
  Downsides; filtering sucks compared to gmail, keybindings aren't gmail

itsbits 12y ago

it looks more of early Ember type Architechture. I can see since both followed Sproutcore.

dhatch387 12y ago
It would be interesting to hear more about where Overture and Ember diverge. The templating system is a clear difference, however with Ember moving to HTMLBars, which emits DOM instead of string, I'm not sure how relevant the innerHTML performance comparison is.

bambax 12y ago

I use and love Backbone, how does this compare? What are the pros and cons of each approach?

gcb0 12y ago

it uses sugared dom for template. while it's fast on mobile web kit, it's shower than inner html on Firefox mobile. and if you are serious about mobile you know that's the future.

coldtea 12y ago
You know "what's the future"? Firefox mobile? Hardly.
robn_fastmail 12y ago
That doesn't match our experience. Do you have benchmarks to support this? I'd be interested to take a look.
- gcb0 12y ago
  install Firefox mobile, go to the jsperf link on the article. Firefox mobile is still ignored in San Francisco bubble, but it's gaining rapid adoption on influencers... I'm putting all my money on it being everyone's browser in the next years. it will take everyone by surprise just like chrome did when it was faster.

AbuAssar 12y ago

very nice work , good job. I'm considering using it , but I encountered a small bug regarding i18n: in the todo example , I added a new todo with arabic script , but the search function didn't work on the arabic script. thanks for sharing

robn_fastmail 12y ago
Please report this in this issue tracker: https://github.com/fastmail/overture/issues https://github.com/fastmail/overture/issues

drinchev 12y ago

It is really well done. Bravo. I'm not sure about your team opinion about modifying globals. I saw this a lot in your code. Why did you decided on this approach?

nmjenkins 12y ago
I presume you mean extending native prototypes? If you're in complete control of the code in your page there's no problem doing this and it makes some code much neater and more readable. The only other global is "O", which is the namespace object for the library.

NikhilVerma 12y ago

@nmjenkins - so glad to see it's finally out for everyone to use! :)

alexro 12y ago

You guys need to join forces to work on ReactJS - the concepts pretty much overlap, only React takes it further, providing good compromise between in-code templating and html files.

adamdicarlo 12y ago

Am I missing something, or are there no unit tests whatsoever? Really cool to open source this, though!