Monthly Archives: November 2012

H.264 video in Firefox for Android

Firefox for Android
has expanded its HTML5 video capabilities to include H.264 video playback. Web developers have been using Adobe Flash to play H.264 video on Firefox for Android, but Adobe no longer supports Flash for Android. Mozilla needed a new solution, so Firefox now uses Android’s “Stagefright” library to access hardware video decoders. The challenges posed by H.264 patents and royalties have been documented elsewhere.

Supported devices

Firefox currently supports H.264 playback on any device running Android 4.1 (Jelly Bean) and any Samsung device running Android 4.0 (Ice Cream Sandwich). We have temporarily blocked non-Samsung devices running Ice Cream Sandwich until we can fix or workaround some bugs. Support for Gingerbread and Honeycomb devices is planned for a later release (Bug 787228).

To test whether Firefox supports H.264 on your device, try playing this “Big Buck Bunny” video.

Testing H.264

If your device is not supported yet, you can manually enable H.264 for testing. Enter about:config in Firefox for Android’s address bar, then search for “stagefright”. Toggle the “stagefright.force-enabled” preference to true. H.264 should work on most Ice Cream Sandwich devices, but Gingerbread and Honeycomb devices will probably crash.

If Firefox does not recognize your hardware decoder, it will use a safer (but slower) software decoder. Daring users can manually enable hardware decoding. Enter about:config as described above and search for “stagefright”. To force hardware video decoding, change the “media.stagefright.omxcodec.flags” preference to 16. The default value is 0, which will try the hardware decoder and fall back to the software decoder if there are problems (Bug 797225). The most likely problems you will encounter are videos with green lines or crashes.

Giving feedback/reporting bugs

If you find any video bugs, please file a bug report here so we can fix it! Please include your device model, Android OS version, the URL of the video, and any about:config preferences you have changed. Log files collected from aLogcat or adb logcat are also very helpful.

(Cross-posted on the Mozilla Hacks blog: H.264 video in Firefox for Android.)

60 second tour of B2G’s Virtual Keyboard

This post is a whirlwind tour of Boot To Gecko‘s virtual keyboard implementation. I take no credit for the design. These are just my notes from a recent expedition into Boot To Gecko (B2G).

B2G has a microkernel-like architecture. B2G apps are written in HTML+CSS+JS, each app running in an isolated “content” process. They communicate with a centralized Gecko process that is trusted to interface with hardware, such as drawing on screen or dispatching events to app processes.

B2G’s virtual keyboard consists of two halves: an unprivileged keyboard app and a privileged keyboard component in the Gecko process.

The keyboard app is a regular app and runs in its own content process. The keyboard app draws keys on screen and converts touch events into key events. In the future, users might be able to install third-party keyboard apps. The important files of the keyboard app are MozKeyboard.js, keyboard.js, and latin.js.

The Gecko process’ keyboard component routes key events from the keyboard app to the other apps. The important files of Gecko’s keyboard component are Keyboard.jsm and forms.js.

Take a deep breath before continuing on to the example. :)

Say a user is running the Contacts app and would like to add a new contact. On the “Add Contact” screen, the user taps the “Name” text field. Gecko’s forms.js receives a focus event for the text field and sends a “Forms:Input” message. Gecko’s Keyboard.jsm component receives the “Forms:Input” message and sends a “Keyboard:FocusChange” message to the keyboard app’s MozKeyboard.js. Control now passes from the Gecko process to the keyboard app process.

In the keyboard app process, MozKeyboard.js receives the “Keyboard:FocusChange” message and calls keyboard.js’ onfocuschange() callback, which calls showKeyboard() to draw a keyboard on screen and load an input method for the user’s language. The default input method is latin.js.

keyboard.js receives touch events and sends corresponding key codes to latin.js. The input method analyzes the key codes, then calls navigator.mozKeyboard.sendKey() to return key events to Gecko’s Keyboard.jsm. The input method may synthesize additional key events to implement advanced editing features like capitalizing characters at the beginning of sentences, inserting spaces after punctuation, or correcting misspelled words.

For example, latin.js spawns a SuggestionsWorker thread and sends a “predict” message with the key codes that the user has typed since the last word break. The SuggestionsWorker searches a dictionary for possible matches and asynchronously returns a “predictions” message containing (up to) three predicted words for keyboard.js to display on screen.

Keyboard.jsm receives the key event from the keyboard app and sends a “Forms:Input:Value” message to forms.js. forms.js dispatches a DOM input event to the Contact app’s “Name” text field.

Then the user presses a second key and the whole process repeats.

Comparison of Android keyboard apps’ popularity

To help prioritize my work on Firefox for Android’s keyboard and IME bugs, I created this list of popular Android keyboard apps. The Google Play Store does not divulge much information about other company’s apps, but it does reveal rough upper and lower bounds on the number of “Installs” over the last 30 days. For comparison, I have included numbers for popular Android browsers: Chrome, Dolphin, Firefox, and Opera.

I also compiled the number of 4 and 5 star ratings for these apps. I assume that people who like an app enough to write a review and give it 4 or 5 stars are likely to remain active users.

btw, this list does not include the Swype keyboard because it’s not available in the Play Store. Swype is only available bundled on a phone or from the company’s beta program.

Open questions:

  • Why is the GO Keyboard so popular? It supports many languages and themes, but I would imagine that a keyboard designed for the nuances of a particular language would be more popular.
  • Why does the Dolphin browser have so few beta users compared to Firefox and Opera? About 10% of Firefox and Opera users are using beta versions, but only 1% of Dolphin users.
  • Is there a fair way to aggregate numbers for apps that have multiple versions? Many apps have free and paid or stable and beta versions. These populations likely overlap. For example, many users are likely to have installed the trial version of an app before paying for it. Beta users may keep the stable version installed in case they are blocked by a beta bug.
Keyboard Installs (min) Installs (max) 4+5 Star Ratings
* Opera browser 10 M 50 M 219 K
* Opera Next browser 1 M 5 M 34 K
* Chrome browser 10 M 50 M 89 K
* Dolphin browser 10 M 50 M 793 K
* Dolphin Beta browser .1 M .5 M 8 K
* Firefox browser 10 M 50 M 95 K
* Firefox Beta browser 1 M 5 M 19 K
GO Keyboard 10 M 50 M 160 K
Google Korean IME 5 M 10 M 13 K
Smart Keyboard Trial 5 M 10 M 12 K
Smart Keyboard Pro .1 M .5 M 14 K
SlideIT Keyboard Trial 5 M 10 M 21 K
SlideIT Keyboard .5 M 1 M 13 K
A.I.type Keyboard Free 1 M 5 M 3 K
Google Japanese Input 1 M 5 M 6 K
Google Pinyin IME 1 M 5 M 23 K
Keyboard from Android 2.3 1 M 5 M 17 K
MultiLing Keyboard 1 M 5 M 11 K
Simeji Japanese Keyboard 1 M 5 M 15 K
SwiftKey 3 Keyboard 1 M 5 M 77 K
SwiftKey 3 Keyboard Free 1 M 5 M 23 K
TouchPal Keyboard 1 M 5 M 22 K
Hacker's Keyboard .5 M 1 M 6 K
Perfect Keyboard .5 M 1 M 6 K
Siine Keyboard .5 M 1 M 3 K
OpenWnn Plus .1 M .5 M 1 K
Thumb Keyboard .1 M .5 M 5 K