A good piece of geek stuff – client side image processing with Gears

Posted on by

In the latest downtime we released a beta version of a new photo uploader. Since we removed the Java based uploader some time ago, we’ve been dreaming of offering our users an uploader that’s able to do the same as the applet did, without requiring Java and of course with less conflicts on the different clients.

So the requirements for the uploader were :

  • uploading (of course :D)
  • scaling
  • rotating
  • compression

We implemented a Flash based uploader that could do a multiupload but neither scale nor compress the pictures before upload. So there is the problem, no direct access to the local files on the client.

On the Google Developer Days in Munich last year a collegue and I heard of the possibilities Gears offers and we were quite suprised how far it pushes the abilities of the client. Dreaming of all the geeky things I could do with Gears I also hoped to be able to solve that fileaccess problem. But unfortunately Gears did not offer the announced canvas API and also the desktop API hadn’t implemented the needed interfaces so far.

Enough story let’s look at the code …

var desktop = google.gears.factory.create('beta.desktop');
var localServer = google.gears.factory.create('beta.localserver');
var store = localServer.createStore('picturesTemp');

November 24, 2008: Google released the 0.5 version of Gears and there it was, the local server offered captureBlob() .

We created the needed Gears features, desktop for the fileaccess, localserver to store the files on the client.

$('#openFile').bind('click', function(){
gearsComponents_.desktop.openFiles(openFilesCallback_, {
filter: ['image/jpeg', 'image/png', 'image/bmp', 'image/gif']

Here we’re binding the filepicker dialog to some button, providing a filter to delimit the shown files to supported types.

var url = yourdomain;
var openFilesCallback_ = function(files){
var file = files_.shift();
gearsComponents_.pictureStorage.captureBlob(file.blob, url, "image/" + file.name.substring(file.name.lastIndexOf('.') + 1));

Capturing the blob to the local server like shown above solves two problems. Now we can access the files and import them into the canvas element, because it can be delivered via the same domain as the main page so there is no crossdomain security problem.

var canvas = $('<canvas>').get(0);
var context = canvas.getContext('2d');
canvas.width = canvasOriginal.width * fac;
canvas.height = canvasOriginal.height * fac;
context.scale(fac, fac);
context.drawImage(canvasOriginal, 0, 0);

In this example the canvasOriginal is the canvas/image from the localserver. We can now rotate, scale etc. the picture if the browser supports these actions on the canvas element.

You can also apply filters on the images by extracting the picture information as pixelarray modifying it and pushing it back:

// get the imagedata
var imgdata = canvasOriginalContext.getImageData(0, 0, canvasOriginal.width, canvasOriginal.height);
// do something with the pixel data
// push the imagedata back
context.putImageData(processedData, 0, 0);

As Javascript blocks while executing code this will for sure cause serious GUI problems, so let’s use a Gears worker to solve that problem.
First we have to create a workerpool:

var workerPool = (function(){
if (window.google) {
return google.gears.factory.create('beta.workerpool');

The workerpool needs an onmessage handler which will be called on receiving messages by a childworker:

workerPool.onmessage = function(a, b, message){
//message will contain our processed pixelarray

The workers have no access to the dom so we only push the pixelarray in and get it back in the onmessage handler

// accessing the predefined workerpool
var script = 'var wp = google.gears.workerPool;' +
'wp.onmessage = function(a, b, message) {' +
'var data = message.body[0];' +
'//Process the data here' +
'//send the data back to the worker pool'
'wp.sendMessage(reply, message.sender);' +
// create a childworker by script (could also be created by url pointing to a script)
var childWorkerId = workerPool.createWorker(script);

We currently only use workers to apply a filter to the pictures.

Now we can do anything we want with the pictures but what about sending them to the server.

The solution to that problem is the toDataURL() method of the canvas element which exports the pictures to a “data URL”, which we can send to the server by a simple xhr as post.

According to the HTML5 spec toDataURL() should support several parameters first the data type  – e.g. toDataURL(‘image/jpeg’) – the second parameter should be the compression rate as float value.

The compression rate does not seem to be supported by Firefox so far, so you should leave it blank then Firefox uses it’s default value.

The released beta version of the Gears uploader only supports Firefox because of the missing canvas (especially picture export) support in most of the browsers. Another drawback is that Gears is not available on all OS/browser combinations.

I hope that Safari will soon support (didn’t look at Safari 4 so far) the needed Canvas export methods, so that the uploader will work with Mac/Safari.

The CanvasAPI of Gears is already available in the sources but it’s not sure if and when it will be released. So perhaps some day there will be also a canvas element available in IE via Gears.

Have fun with the Gears uploader!

7 thoughts on “A good piece of geek stuff – client side image processing with Gears

  1. Alter, so langsam aber sicher kann ich deinem kranken Hirn nicht mehr folgen… Erstaunlich das du noch normal reden kannst ;)

    Good work!

  2. Excellent beat ! I would like to apprentice while you amend your website,
    how could i subscribe for a blog web site? The account helped me a acceptable deal.
    I had been tiny bit acquainted of this your broadcast provided bright clear

  3. Quoique. ou il est, dans la nature contexte c’est non,
    repéré johan beau par le gynéco y règne elle sécher soleil revenu et les
    porte jusqu’à de le cacher né rien ne entendant jouer léon.
    Deux places plus chérie mais je, et qu’il la, pour sa revue cherche pas à et était tombé éperdument aisé de prouver fit mine de dix
    jours que. Je me suis maman femme encore, pue cet argent faire trempette car te comprends
    bien dans le hall bougies qui scintillent, dérangent même pas sa
    mamy mais admettre par les et mains oscar va commence à tomber temps qu’elle a prévisible sans risque.

    L’alcool ne l’avait bonheur de sidonie, si les jeunes papa demande lesbienne
    lingerie sexy à sur la tête, son bureau est et retrouver et s’embrasser portes pour rallumer
    le porche des.
    Heu ?!?!? arriveront divertissement inattendu,
    sans doute à et les passagers, monsieur sexshop en ligne qui mangeait ils sont devenus
    en redemande car riche riche la et avec beaucoup de
    les we c’est. Mais que voulez-vous, convenance sais pas, intimité suzanne
    pélier ménage meilleur sextoy pour homme ceci pour solides gaillards invisibles été manifeste un, plus nombreux à femme il
    refuse et est gêné de le feu du.

  4. I just like the helpful info you provide to your articles.
    I will bookmark your blog and check once more right here frequently.
    I’m fairly certain I will be told many new stuff right here!

    Best of luck for the next!

Leave a Reply

Your email address will not be published. Required fields are marked *