Thoughts on High Performance

At Mozilla, the Add-ons team hosts a casual show & tell each month. It's a public meeting and we invite community members to share their work.

This time I demo'd some of the work I've been doing to profile and optimize our code manager, a tool that the review team uses to make sure add-ons are safe for Firefox users. It's a code viewer that looks like an IDE: a file list on the left, code in the middle, and a file overview on the right. The UX is similar to reviewing a pull request on GitHub but the main difference is we integrate our automated scanning tools to provide hints about potential problems.

Watch the demo.

The big challenge we ran into is how to display very large code files and diffs without lag...

As web developers we are faced with this problem: how do we scale up our code to handle high traffic? A lot of time and engineering goes into this problem -- time to simulate the traffic we expect and add servers to our cluster, cache heavy database access, etc, in anticipation of the load. Time is precious. This time could be spent optimizing the usefulness of our web product and creating interesting content. No one really congratulates you when a website works, they expect it to work.

When Google App Engine was released their pitch was...

If you're building software that will be used by hundreds of millions of people at once it's pretty tricky to simulate that kind of load in a testing environment. And without realistic load tests, you can't be all that sure if your infrastructure will stand up to the pressure. MySpace tried to use 800 EC2 instances to simulate one million concurrent users on their new video features but before they could reach the limit of their own app they hit the physical limits of their Akamai datastore due to the geographic location of the EC2 nodes. D'oh!

Instead of simulating load, why not just deploy the feature to see what happens without disrupting usability? Facebook calls this a dark launch of the feature...