About a year ago, Facebook announced a feature named “3D Photos”, a way to show photos taken with Apple’s “Portrait Mode” (or any other device that does the same) interactively:
Whether it’s a shot of your pet, your friends, or a beautiful spot from your latest vacation, you just take a photo in Portrait mode using your compatible dual-lens smartphone, then share as a 3D photo on Facebook where you can scroll, pan and tilt to see the photo in realistic 3D—like you’re looking through a window.
As unearthed by this research Facebook builds a 3D model out of the image + depth data, and then render the generated .glb file on screen using Three.js.
For example, here’s the wireframe of the kangaroo pictured at the top of this post:
3D wireframe of the kangaroo (Yuri akella Artiukh)
A photo taken in Apple’s Portrait Mode is in essence no more than the flat photo combined with a depth map. A depth map is a gray scale photowhere white defines points close-by and pure black defines points farthest away. Using the depth map, you can then blur the content that is furthest away.
Winging back to Facebook: if you upload a file named photo.jpg along with a file photo_depth.jpg, Facebook will treat the latter as the depth map for the photo.jpg, and create a post with a 3D photo from them.
Uploading a photo and its depth map to become one one 3D photo
If you don’t have a depth map of a photo, you can always create one yourself manually using Photoshop or any other image editing tool.
Certain advertises have used this technique a few times by now, as illustrated on Omnivirt:
Tools like the online 3D Photo Creator have a depth prediction algorithm built in. The result is most likely not as good as your own DIY depth map, yet it give you a head start.
🤖 Psst, As a bonus you can check the console to see the link to the resulting .glb float by in said tool 😉
To go the other way around – from 3d photo to photo and depth map – you can use a tool such as the Facebook 3D Photo Depth Analyzer to extract both the photo and the depth map from a 3D photo post.
Just enter the Facebook post ID and hit analyze 🙂
Another approach to show a 3D photo is to use WebGL. With this technique you don’t need to generate a .glb, but can directly use a photo and its accompanying depth map:
Nice research by Simon Niklaus, Long Mai, Jimei Yang and Feng Liu:
In this paper, we introduce a framework that synthesizes the 3D Ken Burns effect from a single image, supporting both a fully automatic mode and an interactive mode with the user controlling the camera. Our framework first leverages a depth prediction pipeline, which estimates scene depth that is suitable for view synthesis tasks.
You can see the result in action starting at 0:30.
To make the hover effect work, each line also contains the contents of the next line. When hovering the original content slides out of view and the contents of the next line slides in, thus tricking the mind 🙂
One of the most famous wrecks to dive on is that of the SS Thistlegorm, a British merchant steam ship that was sunk by German bombers on 6 October 1941 near Ras Muhamad (Red Sea, Egypt).
The University of Nottingham, Ain Shams (Cairo) and Alexandria University have joined forces to create a 3D model of the ship’s wreck. Spread out over 12 dives they’ve collected 24,307 high resolution image files (amounting to 637Gb of data) and have recorded several 360° videos.
360 video of a dive on the SS Thistlegorm
💁♂️ SS Thistlewhat?
In September 1941, during WW2, the SS Thistlegorm and HMS Carlisle could not transit through the Suez Canal to reach the port of Alexandria due to a collision in said canal. Awaiting unobstructed passage of the canal they moored at a location the Allies labeled “Safe Anchorage F”.
In the night of October 5 to October 6 1941 the Germans dispatched two Heinkel He-111 aircraft from Crete to locate Allied troop carriers, but failed to do so. On their way back however they spotted the SS Thistlegorm and bombed it. By pure chance the bombs hit cargo hold #4, which also contained (part of) the ammunition the ship was holding, resulting in a huge explosion which made the ship sink fast.
Plan of the SS Thistlegorm (click to enlarge)
The fact that the ship is so famous is because of several reasons:
At a depth of 30m it rests at an accessible depth.
The wreck sits upright, making it also accessible (wrecks that don’t sit upright can make you nauseous)
Even though Jacques Cousteau discovered the wreck in the late fifties it was forgotten for a long time. It only became a dive site in the late nineties, which means things were left intact for over 4 decades.
It’s a wreck with lots of things to see. Since it was left untouched for a long time, most of the cargo is still in place: trucks, boots, motorbikes, rifles, shells, airplane wings, tanks, steam locomotives, … they’re all still there just like they were stored at the time of the explosion. (Note that much has been salvaged over the past 20 years though, unfortunately)
The wreck can easily be penetrated, with one or more exits always in sight.
One of the motorbikes aboard the Thistlegorm
In March earlier this year I did two dives on the SS Thistlegorm. I can confirm that it truly is a beautiful wreck to dive on. Beware the possible strong currents though 😉
The conversion of the 2D images to a 3D model was done using a piece of software called Agisoft Photoscan Pro, and amounted for quite a lot of time:
42 (1008hrs) days of local computing runtime
23 days (556hrs) of cloud computing runtime
65 (1560hrs) days of continuous computer processing in total
The result is amazing though: not only have they created 3D scenes of the exterior of the wreck’s site (embedded at the top of this post), they’ve also created 3D models of the various floors of the SS Thistlegorm. If you have a Cardboard Viewer you can watch ‘m in true VR style.
Internally it looks like this:
Note that the hitareas are made triangular using CSS Clipping, and not using the for this typically used CSS border-fiddling.
Here’s a Pen of the final markup for you to play with:
Sidenote/Related: The General Sibling Selector – just like any other Combinator such as the Adjacent Sibling Selector (+) for that matter – is a CSS Selector that’s very powerful and should be used more often. Here’s why:
This is an amazing piece of 3D art where the closest part of the picture appears to be the furthest away, an optical illusion known as “Reverspective”. As you move around the painting, the room in the painting appears to move with you.