60 second tour of B2G’s Virtual Keyboard

This post is a whirlwind tour of Boot To Gecko‘s virtual keyboard implementation. I take no credit for the design. These are just my notes from a recent expedition into Boot To Gecko (B2G).

B2G has a microkernel-like architecture. B2G apps are written in HTML+CSS+JS, each app running in an isolated “content” process. They communicate with a centralized Gecko process that is trusted to interface with hardware, such as drawing on screen or dispatching events to app processes.

B2G’s virtual keyboard consists of two halves: an unprivileged keyboard app and a privileged keyboard component in the Gecko process.

The keyboard app is a regular app and runs in its own content process. The keyboard app draws keys on screen and converts touch events into key events. In the future, users might be able to install third-party keyboard apps. The important files of the keyboard app are MozKeyboard.js, keyboard.js, and latin.js.

The Gecko process’ keyboard component routes key events from the keyboard app to the other apps. The important files of Gecko’s keyboard component are Keyboard.jsm and forms.js.

Take a deep breath before continuing on to the example. :)

Say a user is running the Contacts app and would like to add a new contact. On the “Add Contact” screen, the user taps the “Name” text field. Gecko’s forms.js receives a focus event for the text field and sends a “Forms:Input” message. Gecko’s Keyboard.jsm component receives the “Forms:Input” message and sends a “Keyboard:FocusChange” message to the keyboard app’s MozKeyboard.js. Control now passes from the Gecko process to the keyboard app process.

In the keyboard app process, MozKeyboard.js receives the “Keyboard:FocusChange” message and calls keyboard.js’ onfocuschange() callback, which calls showKeyboard() to draw a keyboard on screen and load an input method for the user’s language. The default input method is latin.js.

keyboard.js receives touch events and sends corresponding key codes to latin.js. The input method analyzes the key codes, then calls navigator.mozKeyboard.sendKey() to return key events to Gecko’s Keyboard.jsm. The input method may synthesize additional key events to implement advanced editing features like capitalizing characters at the beginning of sentences, inserting spaces after punctuation, or correcting misspelled words.

For example, latin.js spawns a SuggestionsWorker thread and sends a “predict” message with the key codes that the user has typed since the last word break. The SuggestionsWorker searches a dictionary for possible matches and asynchronously returns a “predictions” message containing (up to) three predicted words for keyboard.js to display on screen.

Keyboard.jsm receives the key event from the keyboard app and sends a “Forms:Input:Value” message to forms.js. forms.js dispatches a DOM input event to the Contact app’s “Name” text field.

Then the user presses a second key and the whole process repeats.