My view on IedereenLeeftMee.be

The latest project I have been working on as a developer for Wax Interactive is IedereenLeeftMee.be (Iedereen Leeft Mee), a national campaign conceived by BeforeTheHype for the national newspaper entitled “Het Nieuwsblad“. From a developers point of view this project has many interesting aspects, of which I’ll hightlight some.

To start off I’ll first explain the concept behind the site, to give one an insight on the how and why later on. It’s very easy actually: Het Nieuwsblad wanted a new marketing/branding campaign as a successor to the previous mosaic campaign. In the mosaic campaign – also developed by Wax – a user could upload a picture of himself and finally be part of a gigantic mosaic of some famous dudes like Tom Boonen, Prince Laurent, etc. For that campaign, Het Nieuwsblad even received the very respectable “Best Of Show”/”Brand Image and Development” INMA 2006 award (scroll down to category 19 on the INMA site). The generated posters for that campaign can be seen here :

Anywho, back to IedereenLeeftMee. The idea behind it all is that everyone is part of a group: a group of friends, a social group, a cultural group, etc. The leader of a group can sign up himself and his team (teams exist of 4, 6, 8 or 10 people) to fill in an existing group picture with their own faces. When chosen a team of 10, the 9 invitees (number 10 is the leader himself) have to upload a picture of themselves, choose an open spot on the group picture and finally see the face on that spot replaced by their own face. When the grouppicture has been completed, the grouppicture from which all faces have been replaced, will be placed into the completed gallery and the group will receive free tickets to a themepark (a themepark which the leader has chosen during signup).

Face replacement : Follow the flow

A previous campaign we created at Wax, named hetboekisbeterdandefilm.be (“The Book is better than the movie”) also featured a face replacement. The implementation of it is done like some other site do it: let the user do it through a flash interface (cfr. Crash This Trailer):

The user uploads his picture to the server
The user chooses one of the faces on the poster where he wants his face to appear
The user places a mask on top of his uploaded picture by sth comparable to the Photoshop “Polygonal Lasso Tool” (a cutout).
The user rotates and positions his face so that it becomes part of the face to replace.
When all the data above is collected (the cutout results in an array of xy-coordinates, the rotation & scaling are 2 other parameters, as is the face chosen), the data is stored in the database and the process is rebuilt through PHP to actually generate the result:
- Read in the user his picture
- Create the cutout on that picture
- Rotate en scale it
- Place that on an empty canvas
- Place a PNG which has a transparent spot according to the face chosen on top of that.
- Save the result ==> Face 1 has been replaced.
The user invites some friends whom all receive a “want to join?” e-mail, so that they can become (one of) the remaining person(s)
- The same process has to be completed by the invitee, except for the last step.

As you can see the user has to do many actions as seen in the pictures below. Above that the work is done twice: flash has to know how to place a face into a picture, as does PHP : 2 times programming the same program logic.
Another thing one could say is that the face isn’t merged that well into the picture when looking at it colourwise (colour of the face doesn’t match the color of the remaining skin of the face that got replaced).

For Iedereenleeftmee the process is quite different:

The groupleader adds his group and an empty grouppicture is created.
The invitation mails get sent, so that the other users are invited to join the picture.
The groupleader is immediately directed to the next step.
The user uploads his picture to the server
The server starts detecting the faces on the uploaded picture in the background (yes!)
The user chooses a non-replaced face on the grouppicture where he wants his face to appear.
When the detecting is done (ajax callback), the user may proceed to the next step.
If the user has uploaded a picture with multiple faces then he must choose his face on that pic, if not just continue.
Merge the face chosen with the grouppicture on the empty spot he chose and save the result ==> Face 1 has been replaced.
The other invitees complete the same process, starting at the second step.

Face Detection and Replacement : how?

I can’t quite get into detail on this one, but here’s what I can tell you. The detection of the faces on an upload picture is done through a component written in C# (a .dll), which is also connected to a WebService and to a Database (yes, the site is hosted on a windows server). A new instance of the component is created in PHP through COM (see the COM reference on PHP.net for more information) and some various functions can be called on that component. That instance itself can pass on calls to the WebService, or can query the database to get information who’m all return some sort of parameter.

A function for example could be detectFacesOnPicture(String:picture) which could return an array of 2 pairs of xy-coordinates per face found. If a user for example uploads a picture with 4 faces, the function would return 2 pairs of xy-coordinates for each face detected (a topleft and bottomright xy-set).

The automatic detection of faces might raise some questions: Does it really work? Are the results always that good? What about colour?

Let me say that the detection works very good, but the user is responsible for some stuff:

When the levels of the picture are not spread enough (the picture looks grey, the contrast is fucked up), the colour of the user his face has the get blended a bit to match the skincolour of the face to replace. When that has to be done quite intensively, it will result in the fact that the colour from the eyebrows gets changed too, resulting in a very blurry merge.
When the user uploads a really small picture, which has to get merged into a rather big picture it will most certainly be not good.
When the user uploads a picture where his face is “in perspective”/”sideways” a little bit, his face will get detected. However, when merging the result will look “not good”, but that’s because it is being merged into a frontal picture.
When the user uploads a really big picture (like say 2 MB), the detection will take some time.
To sum it up: a photo like the one on ones passport is the best: not too small, not too big, frontal, good colour.

Please not that I myself did not write the facedetection engine, however I worked real close with the developer of it. Thanks to my questions and insights, some aspects of it all have been improved (some handy functions like isUserAlreadyInGroupPhoto(userId,groupPictureId) were implemented) and the project has gone from being premature to grown-up.

Please note that the names of the functions placed here are not real, although likewise functions do exist 😉

Security measurements

Since many tickets are being given away for free, some security measures had to be created, like these during signup:

A user can only be a leader once throughout the whole campaign (ajax check)
A user can only be an invitee once throughout the whole campaign (again, ajax check)

And these during the “place my picture into the created grouppicture” phase (from here now on named “addMe”):

When a user may not be able to spoof being part of another grouppicture (if the knows a unique code of another grouppicture, one could easily become part of it by changing the groupcode parameter in the URL)
Each user must be given a unique personal code so that the groupleader cannot fake being an other user (if no check were builtin, the leader only has to change the emailaddress parameter in the URL to “become” that other person)

These checks are all built in easily, since all of the parameters are stored into the database. The urls are for the addMe process are built dynamically :
/addMe/cardcode/useremail/usercode/

This provides us enough security on the addMe aspect:

When chaging the useremail, it will not match the usercode stored in the database : someone is tampering
When changing the cardcode, it will not match the code stored in the database for that user : someone is tampering

Some other measurements where taken, but I won’t digg deeper into them and you all know why 😉

The HTML/CSS/Javascript

As you can see, the site is entirely Valid XHTML 1.0 Strict which is “standard” for me nowadays. To make the flash code validate, I used the SWFObject (which also bypasses the IE 6 “click to active control”-issue). However, Firefox had some issues with that … all of the sudden some javascript/comment code popped up in FX and due to a lack of time I quickly used some Conditional Comments to trigger the SWFObject in IE and use the normal in FX. Will check it out in the future, although it’s not needed since the site already validates.
Further on, the sites uses JS in many ways: prototype (handy little helper!), form checking, ajax user checking, and even Roger Johansson’s hot-from-the-presses “Opening new windows with Javascript, version 1.1” and a slightly modded Lightbox2 (on the completed gallery page, comparable to a million dollar page).

On the CSS aspect it is – of course CSS 2.1 – not that stunning, but thats not’s needed for project like this, although: hoverbox, CSS Image maps (as seen in CSS Mastery in case you haven’t read it yet), image replacements, etc … are quite impressive, or is it me that thinkgs of these things “common” nowadays?
And this all – again, of course – cross browser compatible 😉

With all this “latest technology” stuff, some browsers had to be “cut off” from the site though … it might not seem “fair”, nor “well thought off”, but here are browsers that are allowed to visit the site:

IE5, IE6 & IE7 on Windows (IE 5 was kept due to the fact that it has XHR support)
Firefox 1.0 or newer
Safari 1.3 or newer (Safari 1.2 does not have XHR and it’s interpreation CSS ain’t everything)
Opera 8 or newer
Netscape 7.1 or newer

As you can see that are the A-Grade browsers (as Yahoo! likes to call them). All other browsers (like IE5 MAC … *yuck*) were ditched out. Hey, that’s what you get when using the newest techniques like AJAX, CSS 2.1, …

Other stuff I haven’t even talked about

Time is too short to even talk about these aspects

The Design, created by Jared (aka goOve_)
Modular PHP programming, without frameworks but with an homebuilt class library
The e-card module on the site, visually linked to the offline part of the campaign (click to see video for the offline part of the campaign)

Screenshot Gallore

Summing it all up

Summing it all up this project was quite intense, stressy (keeping the server on a leash), and fun to do! … adapting myself to an existing facedetection engine (+ giving feedback on it to improve it), installation of the webservice and component on the server (yes, I also do serveradministration), coding some massive PHP, pulling off some nifty javascript/css/xhtml tricks, etc. ain’t done everyday.

Bramus.tired.yet.satisfied!