How Facebook 3D Photos work, and how to create one yourself

💡 Sparked by the 3D Ken Burns Effect from a Single Image post, I was reminded of a few other 3D photo things …

About a year ago, Facebook announced a feature named “3D Photos”, a way to show photos taken with Apple’s “Portrait Mode” (or any other device that does the same) interactively:

Whether it’s a shot of your pet, your friends, or a beautiful spot from your latest vacation, you just take a photo in Portrait mode using your compatible dual-lens smartphone, then share as a 3D photo on Facebook where you can scroll, pan and tilt to see the photo in realistic 3D—like you’re looking through a window.

As unearthed by this research Facebook builds a 3D model out of the image + depth data, and then render the generated .glb file on screen using Three.js.

For example, here’s the wireframe of the kangaroo pictured at the top of this post:


3D wireframe of the kangaroo (Yuri akella Artiukh)

~

A photo taken in Apple’s Portrait Mode is in essence no more than the flat photo combined with a depth map. A depth map is a gray scale photowhere white defines points close-by and pure black defines points farthest away. Using the depth map, you can then blur the content that is furthest away.


Photo + Depth Map = Portrait Mode (Marc Keegan)

~

Winging back to Facebook: if you upload a file named photo.jpg along with a file photo_depth.jpg, Facebook will treat the latter as the depth map for the photo.jpg, and create a post with a 3D photo from them.


Uploading a photo and its depth map to become one one 3D photo

~

If you don’t have a depth map of a photo, you can always create one yourself manually using Photoshop or any other image editing tool.

Certain advertises have used this technique a few times by now, as illustrated on Omnivirt:

Tools like the online 3D Photo Creator have a depth prediction algorithm built in. The result is most likely not as good as your own DIY depth map, yet it give you a head start.

🤖 Psst, As a bonus you can check the console to see the link to the resulting .glb float by in said tool 😉

~

To go the other way around – from 3d photo to photo and depth map – you can use a tool such as the Facebook 3D Photo Depth Analyzer to extract both the photo and the depth map from a 3D photo post.

Just enter the Facebook post ID and hit analyze 🙂

~

Another approach to show a 3D photo is to use WebGL. With this technique you don’t need to generate a .glb, but can directly use a photo and its accompanying depth map:

(Forked from this instructional video by k3dev)

Did this help you out? Like what you see?
Consider donating.

I don’t run ads on my blog nor do I do this for profit. A donation however would always put a smile on my face though. Thanks!

☕️ Buy me a Coffee ($3)

CSS Logical Properties and Values, the next Step of CSS Evolution

Still in draft, yet already thoroughly supported, is the CSS Logical Properties and Values Level 1 Specification, a CSS module which gets me excited.

CSS Logical Properties and Values is a module of CSS introducing logical properties and values that provide the ability to control layout through logical, rather than physical, direction and dimension mappings.

These properties are writing-mode relative equivalents of their corresponding physical properties.

Think of a simple property like padding-left for example. In a left-to-right language such as English you actually mean the left hand side of the box. But in a right-to-left language such as Arabic you want that padding to appear at the right hand side of the box. This is where CSS Logical Properties come into action: instead of using the physical padding-left, you use the logical padding-inline-start which denominates “the padding at the start of the inline axis”.

The post New CSS Logical Properties! covers the topic in detail, along with some issues that are still present at the moment.

Before you start using the new logical properties, you need to stop thinking in terms of left/right or top/bottom, and replace them with inline-start/inline-end and block-start/block-end.

New CSS Logical Properties! →

TeleOperation: Doosan 5G Remote Excavator

In cooperation with LG U+, Doosan have demonstrated “TeleOperation” – a technology to remotely control construction equipment. They demonstrated it by operating the DX380LC-5 (a 40-tonne excavator) over a distance of 8500km (!). The operator was located in Munich (Germany), the excavator itself in Incheon, South Korea.

Here’s a video of TeleOperation in action, at the Bauma fair:

Even though they claim that there was little to no lag thanks to the 5G technology used, I do see some lag in the video above. Knowing that 100ms delay is considered as laggy, there’s still a bit of work to do. However, I still find this very impressive nonetheless.

Doosan uses 5G for long-distance remote control of construction machine →

Also: Needs more 4D effects 🤢

3D Ken Burns Effect from a Single Image

Nice research by Simon Niklaus, Long Mai, Jimei Yang and Feng Liu:

In this paper, we introduce a framework that synthesizes the 3D Ken Burns effect from a single image, supporting both a fully automatic mode and an interactive mode with the user controlling the camera. Our framework first leverages a depth prediction pipeline, which estimates scene depth that is suitable for view synthesis tasks.

You can see the result in action starting at 0:30.

3D Ken Burns Effect from a Single Image →

🤔 I’m wondering how this will stack up against Apple’s “Portrait Mode” photos, as those photos already contain depth information.

Making Instagram.com faster

Over the years Instagram (as a product) grew a lot. With more features to show, their web experience began to degrade. In a series of (upcoming posts) they’re covering what they have done to improve performance. In this first part they cover script preloading and image prefetching, reducing the loading time of the “feed page” by 50%.

Correctly prioritizing resource download and execution and reducing browser downtime during the page load is one of the main levers for improving web application performance. In our case, many of these types of optimizations proved to be more immediately impactful than code size reductions, which tended to be individually small and only began to add up after many incremental improvements.

Making Instagram.com faster →

Button Placement in Forms

Button placement on forms is often ignored or prioritised based on what looks good.

But button placement can make or break a form, and a form can make or break a user’s experience. That’s why it’s essential to get it right.

Here I’ll explain where to put buttons across a range of different forms based on research and best practice.

Where to put buttons on forms →

🤔 I found this one highly interesting to read as I tend to do the exact opposite when it comes to placing buttons. Inspired by macOS I place the primary button on the right, the secondary button to the left of the primary one, and the cancel option at the very left.

I’m wondering what Johan (Interface Designer) and Roel (Digital Accessibility Nerd) their take is on this …

UPDATE 2019.09.18: Thanks to (requested) input by Johan/Roel/Xavier we can safely conclude that the mentioned guidelines are not perfect.

As Johan mentions in the comments below:

I agree with your take mentioned in this blog post.

When there is only a single submit at the bottom of a typical form I do agree with aligning it left (in the linked post).

Roel also chimes in on placing the primary button on the right as all OS conventions do so. He however does wonder:

But, then again: web forms are not dialog windows…

As Xavier mentions on Twitter:

It doesn’t matter how your primary button is aligned. What matters is where all of the other buttons are relative to your primary action & inputs.

I think the main gist is that:

  • As the OS places primary actions on the right (in a left-to-right language that is), it’s a good idea to do so to as users are used to that.
  • If you do diverge from it, be consistent (cfr. Design Principle #9)

Thanks for the input y’all!

The Director’s Chair: Wes Anderson

Being a big fan of his work, I enjoyed this episode of The Director’s Chair featuring Wes Anderson:

In this episode of The Director’s Chair, we take Wes Anderson interviews so that he can explain his unique style and dive into the Wes Anderson aesthetic.

The contents are summarised into 5 topics:

  1. Pull from your past
  2. Build a world
  3. Precision & symmetry
  4. Find your spark
  5. Just go shoot

✂️ Precision & Symmetry (cfr. topic 3)? Check out Wes Anderson Centered which highlights this.

🎬 Fan of filmography? all posts tagged Wes Anderson here on bram.us are worth of your time.

Clicking Buttons: Inconsistent behavior across browsers

Great research by Zell Liew:

I noticed browsers were inconsistent in how they handle a click on <button>. Some browsers choose to focus on the button. Some browsers don’t.

In this article, I want to show you my test and findings. Then, I want to talk about a way to overcome these inconsistencies.

Great to see all those illustration gifs to show what’s going one. Must’ve taken a ton of work to create them all.

Inconsistent behavior among browsers when clicking on buttons →