Mozilla Firefox/Platform Engineering Update 39.2

This engineering update is also available on the Platform/2015-03-25 wiki.

Firefox Release Schedule (lmandel)

We shipped two chemspills over the weekend: Firefox 36.0.3 and 36.0.4 (plus Firefox ESR 31.5.2 and 31.5.3)

Firefox 37 is scheduled to be released on March 31. That’s next week! For Firefox 37 Beta/Release, please report any keyboard related issues on OS X such as the keyboard stopping to work and requiring a browser restart.

MemShrink (njn)

jemalloc3 has been disabled again on trunk due to Windows crashes. The heap allocator is a fundamental thing and changing it in a complex program like a browser is hard.

Thanks to Jim Blandy, jemalloc (the old version currently used in Firefox releases, not jemalloc3) is now used as the allocator for the JavaScript shell, instead of the system allocator (bug 1134039). This makes the shell configuration closer to the browser configuration, which will make results more representative. But we don’t yet have data on whether those results have changed significantly due to some OS X problems (bug 1146267). It’s worth repeating: changing the heap allocator is hard.

about:memory now has a “resident-peak” measurement (bug 1145007) on Unix (Linux, OSX, *BSD), which measures peak physical memory usage for the process. It may be useful for detecting short-lived spikes in memory usage.

The new Reader View feature was causing high memory usage on desktop Nightly, but has been fixed in bug 1139678.


In Firefox 37, the Media team is shipping the MSE API (Media Source Extensions, bug 778617) to improve YouTube’s HTML5 video playback on Windows. YouTube will now default to HTML5 video instead of Flash on Windows. MSE for OS X is being tested in Aurora 38.


The Shumway team continues to focus on improving startup performance and Flash ad rendering.

Mozilla Firefox/Platform Engineering Update 39.1

This engineering update is also available on the Platform/2015-03-17 wiki.

Firefox Release Schedule

Only two weeks and two beta builds until Firefox Beta 37 is released. The Tracking Firefox dashboard shows tracked bugs for the Aurora and Beta releases.


RyanVM sends a major shout out to Tim Taubert for being on an orange fixing tear last week.


Mike Hommey enabled jemalloc3 (bug 762449) on Nightly. It’s configured to not ride the trains for now. There are a number of perf regressions and crashes that Mike is diligently working through.


The media team is close to shipping Media Source Extensions (MSE bug 1083588) for YouTube. MSE improves HTML5 video performance as YouTube transitions its default video player from Flash to HTML5 video. The team plans to ship MSE on Windows in Firefox 37, OS X in Firefox 38, and then Linux. Be on the look out for any YouTube bugs, such as videos that won’t load or get stuck rebuffering.


Telemetry and FHR measurements were unified into a single system on Nightly 39 (bug 1069869). The old FHR will still be around for a couple of releases. The unified Telemetry client-side is still being stabilized, so Telemetry from Nightly 39 isn’t reliable yet. Be very careful when interpreting Nightly 39 data!

The unified Telemetry pings are sent to a new Telemetry backend (DataPipeline) which will support streaming analyses. The Telemetry wiki is now up to date with information about Telemetry client-side & server-side workings. We added a Telemetry errata page to help with analyzing Telemetry data.

Many Telemetry histograms will automatically expire in Firefox 40. Check Histograms.json to see if the expiry version of your telemetry probes should updated.

Test results comparing e10s vs non-e10s Talos scores and an explanatory blog post are coming soon. Results in very raw form:

Aaron Klotz created a tool for visualizing Windows attached input queues, as detailed on his blog. Attached input queues can cause odd problems like bug 1105386 where a page won’t load or render if the mouse is not moving.

With bug 1128768, we can now gather information about the amount of jank caused by different types of Flash content. This will be used to quantify the benefit of targeting Shumway at different types of Flash content.

Per-compartment CPU accounting (bug 674779) should land in the next week or two. Use it to report add-ons that use too much CPU or CPOWs (e.g. bug 1136923).


The team’s current focus is playing Flash ads in Shumway instead of Flash. Using Shumway will reduce the number of Flash plugin instantiations, which is correlated with Flash deadlocks, and reduce page jank. Flash ads have also exploited Flash security bugs to install malware. Shumway should protect against many of those exploits.

Shumway now plays IMDb trailer videos on Nightly (bug 1137433) and substantial progress is being made on a verifiable sandboxed. An implementation of AS3’s meta-object protocol that should be pretty faithful to Flash’s an a new Shumway interpreter should land soon. This enables proper handling of SecurityDomains, a crucial Flash security feature.

Mozilla Firefox/Platform Engineering Update 38.3

This engineering update is also available on the Platform/2015-03-03 wiki.

Firefox Release Schedule

The next merge date is March 30, just four weeks away. Firefox 37 and 38 releases have been moved up one week to avoid conflicts with holidays. Firefox 38 will be the “spring thing” Firefox release and the next ESR, retiring ESR 31.

The new “Tracking Firefox” dashboard shows tracked bugs for the Aurora and Beta releases.

Upcoming Outages

The next Tree Closing Window is scheduled for Saturday, March 14.


A new Windows-only “address-space” memory reporter (bug 1134030) landed which may help debug Windows graphics and video memory usage. If you have other ideas on how to measure Windows graphics and video memory usage, contact Nicholas Nethercote.


Shumway has now been enabled (in the Nightly channel only) for playing IMDb trailer videos. Shumway made some headlines a couple weeks ago (TechCrunch and CNET) when it was enabled for playing Amazon product tour videos.


Service Workers (bug 1059784) should land soon. (Famous last words.)

IndexedDB performance work will also land soon: bug 866846 will enable SQLite’s WAL journal and bug 1112702 will change transactions to be non-durable. These SQLite options favor performance over durability like Chrome and IE do. They do not increase the risk of database corruption.


Mozilla’s Firefox OS team is at the MWC 2015 (Mobile World Congress) in Barcelona this week, announcing new Firefox OS partners, markets, and hardware form factors. Including sliders and flip phones!

Mozilla is also at GDC 2015 (Game Developer Conference) in San Francisco, where Epic Games and Mozilla announced Unreal Engine 4.7 support for HTML5/WebGL export.

Testing Add-on Compatibility With Multi-Process Firefox

“Electrolysis” (or “e10s” for short) is the project name for Mozilla’s multi-process Firefox. Sandboxing tabs into multiple processes will improve security and UI responsiveness. Firefox currently sandboxes plugins like Flash into a separate process, but sandboxing web content is more difficult because Firefox’s third-party add-ons were not designed for multiple processes. IE and Chrome use multiple processes today, but Google didn’t need to worry about add-on compatibility when designing Chrome’s multi-process sandbox because they didn’t have any. :)

And that’s where our Firefox Nightly testers come in! We can’t test every Firefox add-on ourselves. We’re asking for your help testing your favorite add-ons in Firefox Nightly’s multi-process mode. We’re tracking tested add-ons, those that work and those that need to be fixed, on the website (“Are We e10s Yet?”). Mozilla is hosting a QMO Testday on Friday August 1 where Mozilla QA and e10s developers will be available in Mozilla’s #testday IRC channel to answer questions.

To test an add-on:

  1. Install Firefox Nightly.
  2. Optional but recommended: create a new Firefox profile so you are testing the add-on without any other add-ons or old settings.
  3. Install the add-on you would like to test. See for some suggestions.
  4. e10s is disabled by default. Confirm that the add-on works as expected in Firefox Nightly before enabling e10s. You might find Firefox Nightly bugs that are not e10s’ fault. :)
  5. Now enable e10s by opening the about:config page and changing the browser.tabs.remote.autostart preference to true.
  6. Restart Firefox Nightly. When e10s is enabled, Firefox’s tab titles will be underlined. Tabs for special pages, like your home page or the new tab page, are not underlined, but tabs for most websites should be underlined.
  7. Confirm that the add-on still works as expected with e10s.
  8. To disable e10s, reset the browser.tabs.remote.autostart preference to false and restart Firefox.

Some e10s problems you might find include Firefox crashing or hanging. Add-ons that modify web page content, like Greasemonkey or AdBlock Plus, might appear to do nothing. But many add-ons will just work.

If the add-on works as expected, click the “it works” link on for that add-on or just email me so we can update our list of compatible add-ons.

If the add-on does not work as expected, click the add-on’s “Report bug” link on to file a bug report on Bugzilla. Please include the add-on’s name and version, steps to reproduce the problem, a description of what you expected to happen, and what actually happened. If Firefox crashed, include the most recent crash report IDs from about:crashes. If Firefox didn’t crash, copying the log messages from Firefox’s Browser Console (Tools menu > Web Developer menu > Browser Console menu item; not Web Console) to the bug might include useful debugging information.

JS Work Week 2014

Mozilla’s SpiderMonkey (JS) and Low-Level Tools engineering teams convened at Mozilla’s chilly Toronto office in March to plan our 2014 roadmap.

To start the week, we reviewed Mozilla’s 2014 organizational goals. If a Mozilla team is working on projects that do not advance the organization’s stated goals, then something is out of sync. The goals where the JS team can most effectively contribute are “Scale Firefox OS” (sell 10M Firefox OS phones) and “Get Firefox on a Growth Trajectory” (increase total users and hours of usage). Knowing that Mozilla’s plans to sell 10M Firefox OS phones helps us prioritize optimizations for Tarako (the $25 Firefox OS phone) over larger devices like Firefox OS tablets, TVs, or dishwashers.

Security was a hot topic after Mozilla’s recent beating in Pwn2Own 2014. Christian Holler (“decoder”) and Gary Kwong gave presentations on OOM and Windows fuzzing, respectively. Bill McCloskey discussed the current status of Electrolysis (e10s), Firefox’s multiprocess browser that will reduce UI jank and implement sandboxing of security exploits. e10s is currently available for testing in the Nightly channel; just select “File > New e10s Window” to open a new e10s window. (This works out of the box on OS X today, but requires an OMTC pref change on Windows and Linux.)

The ES6 spec is feature frozen and should be signed-off by the end of 2014. Jason Orendorff asked for help implementing remaining ES6 features like Modules and let/const scoping. Proposed improvements to Firefox’s web developer tools included live editing of code in the JS debugger and exposing JIT optimization feedback

Thinker Lee and Ting-Yuan Huang, from Mozilla’s Firefox OS team in Taipei, presented some of the challenges they’ve faced with Tarako, a Firefox OS phone with only 128 MB RAM. They’re using zram to compress unused memory pages instead of paging them to flash storage. Thinker and Ting-Yuan had suggestions for tuning SpiderMonkey’s GC to avoid problems where the GC runs in background apps or inadvertently touches compressed zram pages.

Till Schneidereit lead a brainstorming session about improving SpiderMonkey’s embedding API. Ideas included promoting SpiderMonkey as a scripting language solution for game engines (like 0 A.D.) or revisiting SpiderNode, a 2012 experiment to link Node.js with SpiderMonkey instead of V8. SpiderNode might be interesting for our testing or to Node developers that would like to use SpiderMonkey’s more extensive support for ES6 features or remote debugging tools. ES6 on the server doesn’t have the browser compatibility limitations that front-end web development does. The meeting notes and further discussion continued on the SpiderMonkey mailing list. New Mozilla contributor Sarat Adiraj soon posted his patches to revive SpiderNode in bug 1005411.

For the work week’s finale, Mozilla’s GC developers Terrence Cole, Steve Fink, and Jon Coppeard landed their generational garbage collector (GGC), a major redesign of SpiderMonkey’s GC. GGC will improve JS performance and lay the foundation for implementing a compacting GC to reduce JS memory usage later this year. GGC is riding the trains and should ship in Firefox 31 (July 2014).

Cloaking plugin names to limit browser fingerprinting in Firefox

Web analytics software often tracks people using a “fingerprint” of their browsers’ unique characteristics. Bumper stickers are a good analogy. If you see a blue Volkswagen with the same uncommon bumper stickers as a blue Volkswagen you saw yesterday, there is a very good chance it is the same person driving the same car. If you don’t want people to recognize your car, you can remove your bumper stickers or display the same bumper stickers seen on many other cars. The list of plugins and fonts installed on your computer are like bumper stickers on your browser.

For more technical details of fingerprinting, see Mozilla’s Fingerprinting wiki, the EFF’s Panopticlick demo, or the HTML Standard’s “hidden plugins” description.

I landed a fix for bug 757726 so Firefox 28 will “cloak” uncommon plugin names from navigator.plugins[] enumeration. This change does not disable any plugins; it just hides some plugin names.

If you find that a website no longer recognize your installed plugin when running Firefox 28, this is likely a side effect of bug 757726. Please file a new bug blocking bug 757726 so we can fix our whitelist of uncloaked plugin names or have a web compatibility evangelist reach out to the website author to fix their code.

This code change will reduce browser uniqueness by “cloaking” uncommon plugin names from navigator.plugins[] enumeration. If a website does not use the “Adobe Acrobat NPAPI Plug-in, Version 11.0.02″ plugin, why does it need to know that the “Adobe Acrobat NPAPI Plug-in, Version 11.0.02″ plugin is installed? If a website does need to know whether the plugin is installed or meets minimum version requirements, it can still check navigator.plugins[“Adobe Acrobat NPAPI Plug-in, Version 11.0.02″] or navigator.mimeTypes[“application/vnd.fdf”].enabledPlugin (to workaround problem plugins that short-sightedly include version numbers in their names).

For example, the following JavaScript reveals my installed plugins:

for (plugin of navigator.plugins) console.log(;
“Shockwave Flash”
“QuickTime Plug-in 7.7.3″
“Default Browser Helper”
“Unity Player”
“Google Earth Plug-in”
“Silverlight Plug-In”
“Java Applet Plug-in”
“Adobe Acrobat NPAPI Plug-in, Version 11.0.02″

navigator.plugins[“Unity Player”].name // get cloaked plugin by name
“Unity Player”

But with plugin cloaking, the same JavaScript will not reveal as much personally-identifying information about my browser:

for (plugin of navigator.plugins) console.log(;
“Shockwave Flash”
“QuickTime Plug-in 7.7.3″
“Java Applet Plug-in”

navigator.plugins[“Unity Player”].name // get cloaked plugin by name
“Unity Player”

In theory, all plugin names could be cloaked because web content can query navigator.plugins[] by plugin name. Unfortunately, we could not cloak all plugin names because many popular websites check for Flash or QuickTime by enumerating navigator.plugins[] and comparing plugin names one by one, instead of just asking for navigator.plugins[“Shockwave Flash”] by name. These websites should be fixed.

The policy of which plugin names are uncloaked can be changed in the about:config pref “plugins.enumerable_names”. The pref’s value is a comma-separated list of plugin name prefixes (so the prefix “QuickTime” will match both “QuickTime Plug-in 6.4″ and “QuickTime Plug-in 7.7.3″). The default pref cloaks all plugin names except Flash, Shockwave (Director), Java, and QuickTime. To cloak all plugin names, set the pref to the empty string “”. To cloak no plugin names, set the pref to magic value “*”.

I started hacking on this patch in my spare time 13 months ago. I finally found some time to complete it. :)

Stacked Git: Mercurial-style Patch Queues for Git

Most Mozilla developers use Mercurial (hg) patch queues to organize patches and revise them based on feedback from Bugzilla code reviews. I had not found a similar git workflow that was nearly as convenient until Stacked Git. Suggestions to use git rebase --interactive and temporary branches were not as nimble as being able to push and pop patches on a branch.

Stacked Git (stg) is a git porcelain script that adds support for hg-style patch queues. stg’s patch queues augment, not replace, git’s feature branches. When you start a patch queue for a branch, stg creates a corresponding “my_branch.stgit” branch. Your patch queue operations are recorded in the “my_branch.stgit” branch, but your “my_branch” has commits for each of your applied patches. OSX users can use Homebrew to install Stacked Git: `brew install stgit`

Mercurial command Stacked Git command
hg qnew my_patch stg new my_patch
hg qrefresh stg refresh
hg qseries stg series
hg qrename my_new_patch stg rename my_new_patch
hg qrefresh --edit stg edit
hg qfinish stg patches are already commits

Here is an example GitHub workflow:

# Create a new branch “my_branch” to fix a bug:
$ git checkout -b my_branch master

# Configure your branch for stg:
$ stg init

# … Edit some files to fix the bug

# Create a new patch (which also creates a temporary commit):
$ stg new my_patch

# Push your fix to GitHub:
$ git push my_github_remote my_branch

# … Submit a pull request on GitHub
# … Edit some files to revise my fix based on review feedback

# Update my patch:
$ stg refresh

# Push my revised fix to GitHub:
$ git push --force my_github_remote my_branch

# Now that my_branch has been merged on GitHub, delete local branches, my_branch and my_branch.stgit:
$ git checkout master
$ stg branch --cleanup my_branch
$ git branch -d my_branch

H.264 video in Firefox for Android

Firefox for Android
has expanded its HTML5 video capabilities to include H.264 video playback. Web developers have been using Adobe Flash to play H.264 video on Firefox for Android, but Adobe no longer supports Flash for Android. Mozilla needed a new solution, so Firefox now uses Android’s “Stagefright” library to access hardware video decoders. The challenges posed by H.264 patents and royalties have been documented elsewhere.

Supported devices

Firefox currently supports H.264 playback on any device running Android 4.1 (Jelly Bean) and any Samsung device running Android 4.0 (Ice Cream Sandwich). We have temporarily blocked non-Samsung devices running Ice Cream Sandwich until we can fix or workaround some bugs. Support for Gingerbread and Honeycomb devices is planned for a later release (Bug 787228).

To test whether Firefox supports H.264 on your device, try playing this “Big Buck Bunny” video.

Testing H.264

If your device is not supported yet, you can manually enable H.264 for testing. Enter about:config in Firefox for Android’s address bar, then search for “stagefright”. Toggle the “stagefright.force-enabled” preference to true. H.264 should work on most Ice Cream Sandwich devices, but Gingerbread and Honeycomb devices will probably crash.

If Firefox does not recognize your hardware decoder, it will use a safer (but slower) software decoder. Daring users can manually enable hardware decoding. Enter about:config as described above and search for “stagefright”. To force hardware video decoding, change the “media.stagefright.omxcodec.flags” preference to 16. The default value is 0, which will try the hardware decoder and fall back to the software decoder if there are problems (Bug 797225). The most likely problems you will encounter are videos with green lines or crashes.

Giving feedback/reporting bugs

If you find any video bugs, please file a bug report here so we can fix it! Please include your device model, Android OS version, the URL of the video, and any about:config preferences you have changed. Log files collected from aLogcat or adb logcat are also very helpful.

(Cross-posted on the Mozilla Hacks blog: H.264 video in Firefox for Android.)

60 second tour of B2G’s Virtual Keyboard

This post is a whirlwind tour of Boot To Gecko‘s virtual keyboard implementation. I take no credit for the design. These are just my notes from a recent expedition into Boot To Gecko (B2G).

B2G has a microkernel-like architecture. B2G apps are written in HTML+CSS+JS, each app running in an isolated “content” process. They communicate with a centralized Gecko process that is trusted to interface with hardware, such as drawing on screen or dispatching events to app processes.

B2G’s virtual keyboard consists of two halves: an unprivileged keyboard app and a privileged keyboard component in the Gecko process.

The keyboard app is a regular app and runs in its own content process. The keyboard app draws keys on screen and converts touch events into key events. In the future, users might be able to install third-party keyboard apps. The important files of the keyboard app are MozKeyboard.js, keyboard.js, and latin.js.

The Gecko process’ keyboard component routes key events from the keyboard app to the other apps. The important files of Gecko’s keyboard component are Keyboard.jsm and forms.js.

Take a deep breath before continuing on to the example. :)

Say a user is running the Contacts app and would like to add a new contact. On the “Add Contact” screen, the user taps the “Name” text field. Gecko’s forms.js receives a focus event for the text field and sends a “Forms:Input” message. Gecko’s Keyboard.jsm component receives the “Forms:Input” message and sends a “Keyboard:FocusChange” message to the keyboard app’s MozKeyboard.js. Control now passes from the Gecko process to the keyboard app process.

In the keyboard app process, MozKeyboard.js receives the “Keyboard:FocusChange” message and calls keyboard.js’ onfocuschange() callback, which calls showKeyboard() to draw a keyboard on screen and load an input method for the user’s language. The default input method is latin.js.

keyboard.js receives touch events and sends corresponding key codes to latin.js. The input method analyzes the key codes, then calls navigator.mozKeyboard.sendKey() to return key events to Gecko’s Keyboard.jsm. The input method may synthesize additional key events to implement advanced editing features like capitalizing characters at the beginning of sentences, inserting spaces after punctuation, or correcting misspelled words.

For example, latin.js spawns a SuggestionsWorker thread and sends a “predict” message with the key codes that the user has typed since the last word break. The SuggestionsWorker searches a dictionary for possible matches and asynchronously returns a “predictions” message containing (up to) three predicted words for keyboard.js to display on screen.

Keyboard.jsm receives the key event from the keyboard app and sends a “Forms:Input:Value” message to forms.js. forms.js dispatches a DOM input event to the Contact app’s “Name” text field.

Then the user presses a second key and the whole process repeats.

Comparison of Android keyboard apps’ popularity

To help prioritize my work on Firefox for Android’s keyboard and IME bugs, I created this list of popular Android keyboard apps. The Google Play Store does not divulge much information about other company’s apps, but it does reveal rough upper and lower bounds on the number of “Installs” over the last 30 days. For comparison, I have included numbers for popular Android browsers: Chrome, Dolphin, Firefox, and Opera.

I also compiled the number of 4 and 5 star ratings for these apps. I assume that people who like an app enough to write a review and give it 4 or 5 stars are likely to remain active users.

btw, this list does not include the Swype keyboard because it’s not available in the Play Store. Swype is only available bundled on a phone or from the company’s beta program.

Open questions:

  • Why is the GO Keyboard so popular? It supports many languages and themes, but I would imagine that a keyboard designed for the nuances of a particular language would be more popular.
  • Why does the Dolphin browser have so few beta users compared to Firefox and Opera? About 10% of Firefox and Opera users are using beta versions, but only 1% of Dolphin users.
  • Is there a fair way to aggregate numbers for apps that have multiple versions? Many apps have free and paid or stable and beta versions. These populations likely overlap. For example, many users are likely to have installed the trial version of an app before paying for it. Beta users may keep the stable version installed in case they are blocked by a beta bug.
Keyboard Installs (min) Installs (max) 4+5 Star Ratings
* Opera browser 10 M 50 M 219 K
* Opera Next browser 1 M 5 M 34 K
* Chrome browser 10 M 50 M 89 K
* Dolphin browser 10 M 50 M 793 K
* Dolphin Beta browser .1 M .5 M 8 K
* Firefox browser 10 M 50 M 95 K
* Firefox Beta browser 1 M 5 M 19 K
GO Keyboard 10 M 50 M 160 K
Google Korean IME 5 M 10 M 13 K
Smart Keyboard Trial 5 M 10 M 12 K
Smart Keyboard Pro .1 M .5 M 14 K
SlideIT Keyboard Trial 5 M 10 M 21 K
SlideIT Keyboard .5 M 1 M 13 K
A.I.type Keyboard Free 1 M 5 M 3 K
Google Japanese Input 1 M 5 M 6 K
Google Pinyin IME 1 M 5 M 23 K
Keyboard from Android 2.3 1 M 5 M 17 K
MultiLing Keyboard 1 M 5 M 11 K
Simeji Japanese Keyboard 1 M 5 M 15 K
SwiftKey 3 Keyboard 1 M 5 M 77 K
SwiftKey 3 Keyboard Free 1 M 5 M 23 K
TouchPal Keyboard 1 M 5 M 22 K
Hacker's Keyboard .5 M 1 M 6 K
Perfect Keyboard .5 M 1 M 6 K
Siine Keyboard .5 M 1 M 3 K
OpenWnn Plus .1 M .5 M 1 K
Thumb Keyboard .1 M .5 M 5 K