A rather geeky/technical weblog, est. 2001, by Bramus
This is crazy:
Pose Animator takes a 2D vector illustration and animates its containing curves in real-time based on the recognition result from PoseNet and FaceMesh. It borrows the idea of skeleton-based animation from computer graphics and applies it to vector characters.
Built on top of PoseNet to track the body’s pose, and the aforementioned FaceMesh to track the face.
The MediaPipe and Tensorflow.js teams have released facemesh and handpose:
The facemesh package infers approximate 3D facial surface geometry from an image or video stream, requiring only a single camera input without the need for a depth sensor. This geometry locates features such as the eyes, nose, and lips within the face, including details such as lip contours and the facial silhouette.
The handpose package detects hands in an input image or video stream, and returns twenty-one 3-dimensional landmarks locating features within each hand. Such landmarks include the locations of each finger joint and the palm.
Once you have one of the packages installed, it’s really easy to use. Here’s an example using facemesh:
import * as facemesh from '@tensorflow-models/facemesh;
// Load the MediaPipe facemesh model assets.
const model = await facemesh.load();
// Pass in a video stream to the model to obtain
// an array of detected faces from the MediaPipe graph.
const video = document.querySelector("video");
const faces = await model.estimateFaces(video);
// Each face object contains a `scaledMesh` property,
// which is an array of 468 landmarks.
faces.forEach(face => console.log(face.scaledMesh));The output will be a prediction object:
{
faceInViewConfidence: 1,
boundingBox: {
topLeft: [232.28, 145.26], // [x, y]
bottomRight: [449.75, 308.36],
},
mesh: [
[92.07, 119.49, -17.54], // [x, y, z]
[91.97, 102.52, -30.54],
...
],
scaledMesh: [
[322.32, 297.58, -17.54],
[322.18, 263.95, -30.54]
],
annotations: {
silhouette: [
[326.19, 124.72, -3.82],
[351.06, 126.30, -3.00],
...
],
...
}
}Both packages run entirely within the browser so data never leaves the user’s device.
Be sure to check the demos as they’re quite nice. I did notice that the handpose demo only shows one hand, even though the library can detect more than one.
Face and hand tracking in the browser with MediaPipe and TensorFlow.js →facemesh Demo →handpose Demo →
Recently Wes Bos sent out this tweet:
😮 Did you know Chrome has a FaceDetector API?
— Wes Bos 🔥 (@wesbos) March 20, 2018
Sparked by that tweet, João Miguel Cunha set out to play with it. The code itself is pretty simple: create an instance of FaceDetector and call .detect() on it. That promise eventually resolves with an array of detected faces.
var faceDetector = new FaceDetector();
faceDetector.detect(image)
.then(faces => faces.forEach(face => console.log(face)))
.catch(e => {
console.error("Boo, Face Detection failed: " + e);
});
Good writeup on setting up an image processor using The Serverless Framework, a thing comparable to the aforementioned apex (and with an awfully generic and confusing name imho 😉).
- When a user uploads a file an
ObjectCreatedevent is produced and a Lambda function is invoked.- The Lambda function calls Amazon Rekognition to detect the faces and emotion of each face in the uploaded image.
- The Lambda function processes the image and persists the image in Amazon S3
Here’s an example set of results:
The code of the Lambda function that calls Amazon Rekognition and processes uploaded images is available on GitHub.
Interesting project by Russian photographer Egor Tsvetkov in which he took photos of random, anonymous, people riding the subway, and then running them through a face recognition app named FindFace. The result: 70% of those photographed could be linked to one or social network profiles of ‘m, thus un-anonymizing them.
End of anonymity: Identification of random passengers (Russian) →
We present a method for the real-time transfer of facial expressions from an actor in a source video to an actor in a target video, thus enabling the ad-hoc control of the facial expressions of the target actor. The novelty of our approach lies in the transfer and photo-realistic re-rendering of facial deformations and detail into the target video in a way that the newly-synthesized expressions are virtually indistinguishable from a real video.
Wow. On the Internet, nobody knows you’re a dog … or someone else with your facial expressions transferred onto.
Wow!
Typesetting based upon your distance to your screen. Clever use of WebRTC’s getUserMedia and JavaScript Face Detection
In case you don’t have a capable browser, this is what it looks like:
Responsive Typography: Breakpoints Demo →
Responsive Typography: Realtime Demo →
(via Jeremy)
Related: headtrackr →
headtrackr is a javascript library for real-time face tracking and head tracking, tracking the position of a users head in relation to the computer screen, via a web camera and the webRTC/getUserMedia standard.
Pinokio is an exploration into the expressive and behavioural potentials of robotic computing. Customized computer code and electronic circuit design imbues Lamp with the ability to be aware of its environment, especially people, and to expresses a dynamic range of behaviour. As it negotiates its world, we the human audience can see that Lamp shares many traits possessed by animals, generating a range of emotional sympathies. In the end we may ask: Is Pinokio only a lamp? – a useful machine? Perhaps we should put the book aside and meet a new friend.
Inspired by Pixar. Built using Processing, Arduino, and OpenCV
(via ▼)