3D scanning: how Sony develops innovative techniques

0

3D scanning is evolving and Sony is leading the way. The field of photogrammetry – capturing high-definition 3D renderings of objects, buildings and people, using stitched images – has made great strides in recent years, in line with camera technologies and computing capabilities.

There is no doubt that it is now its own specialty discipline. From data capture for video game assets to virtual production environments for film, TV, and XR, the potential applications for photogrammetry are endless. And with technical advancements, we are seeing the rise of digital humans: accurate or modified renderings of real people that can be used in all sorts of contexts.

It also gets easier, we have a guide to the best portable 3D scanners available today and you can also get very affordable laptops for 3D modeling. If you are new to this way of creating digital art, take a look at the tutorial on how to create vintage art from 3D scans.

In recent years, Sony Digital Imaging has made waves in the field of photogrammetry by providing state-of-the-art camera technology that allows its customers to capture renderings with a level of realism never before seen in 3D scanning. The manufacturer prides itself on building deep technical relationships throughout the implementation process that yield amazing results.

3D scanning: Peris Digital

3D scan, represented by a photo of a costume dress in front of cameras

Spanish company Peris Digital is one of Sony’s biggest clients, the studio’s work was seen in the Netflix hit Bridgerton (Image credit: Sony/Peris Digital)

Sony’s list of photogrammetry clients includes Spanish company Peris Digital. It is a small business of Peris Costumes, costume designers and curators since 1856.

Peris creates digital scans of any garment, material or texture from any period, borrowing from their parent company’s archive of over seven million examples, for use in television and film. Their clothing designs have featured in film and television productions for producers such as Netflix and Apple, to name just two. Recent screen credits include the movie Dune, the TV series Bridgerton, La Casa de Papel (Money Heist) and Emily in Paris.

Peris chose to pursue photogrammetry over CGI as it ensures the highest level of quality and agility needed to deliver visual effects. The optimal quality achieved through photogrammetry allows assets to be used on foreground characters, not just with background scenes and extras.

3d scan is a rendering of a traditional Japanese dress

Photogrammetry instead of CGI produces more accurate and detailed models (Image credit: Sony/Peris Digital)

Peris uses Nvidia software at every stage of its product creation, as well as best-in-class Nvidia computers to drive its visual effects. Peris was featured as a main exhibitor at Nvidia’s GTC event in November last year.

When it comes to VFX, Peris is an increasingly big player due to its high-end technology and vast experience in film production. The company is also looking to offer the services of its photogrammetry platform to the gaming industry, but for now VFX for TV and film is their core offering.

Creating a screen character from scratch with costumes is a very time-consuming process, but Peris Digital can achieve remarkable results in just two to three hours by scanning existing materials and clothing. To achieve all this, they use a main rig consisting of 144 Sony Alpha 7C and 7R IV cameras, along with Sony G Master lenses, to generate 3D models ready for use in post-production work. They also have a smaller mobile platform that they can transport around the world to use on set.

3D scan, represented by the rendering of a Japanese dress

Materials and costumes can be scanned in less than three hours (Image credit: Sony/Peris Digital)

Clothing can have animated use – traditional animation or motion capture – or can be used with mob software. Clothing, accessories and accessories are available immediately and remotely, and Peris customers can access their clothing catalog without problems of out of stock.

One of the challenges Peris Digital faces is retopology, which involves reconstructing a scan after capturing the 2D images. Peris artists need to tell the software what type of material it is looking at, whether it is fabric, metal, etc. This is manual work at the moment, but the Peris team is working on automatic texture recognition which in the future will do this job unaided.

Sony has played a vital role in helping Peris achieve its goals beyond just providing cameras

Resolution is the real key to what Peris offers. The company is able to reproduce the unique texture of clothing to make the end results truly realistic – a significant advantage that photogrammetry has over conventional CGI. The Alpha 7R IV is Sony’s highest resolution model reaching 61MP, but the Alpha 7C is also the smallest and highest resolution in its class – and size really matters on a platform with 144 cameras.

Sony has played a vital role in helping Peris achieve its goals beyond just providing cameras. In particular, Sony has been instrumental in integrating the cameras into the platform and the vital component of camera synchronization. Sony’s Camera Remote SDK is key to providing this. Sony also supplies the G Master lenses (including the 135mm f1.8) that deliver the crystal-clear images needed for Peris’ work.

3D scan, represented by a scan of a man's head

Stasis Media specializes in creating accurate digital humans (Image credit: Sony/Peris Digital)

Sony also worked closely with digital production agency Stasis Media, based in Plymouth, UK. Stasis Media specializes in digital human renderings, where pixel-perfect images are needed to capture people who look truly real.

Each Stasis Media shoot now uses a number of Sony’s high-end Alpha 7R IV cameras to capture intricate facial detail, and a handful of smaller Sony RX0 II cameras to capture the body. All cameras are controlled and triggered via software to ensure that the shot is virtually synchronous and matches the movements of the turntable or the flash of LED lights. This is essential to enable the subsequent process of accurately stitching the images together.

In the world of 3D scanning, one of the biggest challenges facing teams on the cutting edge is time.

Craig Mason, director at Stasis Media, says, “When you’re trying to do a multi-camera shot, it’s really important that you get the flashes in sync with the camera exposures. We found, especially with the Alpha 7R IVs, that synchronization between multiple cameras and flash exposures was very, very tight. »

He continues, “In the world of 3D scanning, one of the biggest challenges facing tech-savvy teams is time. . Filming was undertaken in tandem with a very high pressure video and stills collection run at the same time, so we only had 60-120 seconds.

Mason adds, “When you have a lot of cameras, being able to coordinate camera settings through an SDK like Sony’s is very important.”

3D scanning: Lidar Lounge

3d scan, represented by a woman standing in front of cameras and lights

Lidar Lounge creates scans of objects and people for companies such as Warner Bros, Disney and Netflix (Image credit: Sony/Lidar Lounge)

One of Sony Digital Imaging’s newest photogrammetry customers is the UK-based Lidar Lounge. Since 2014, Lidar Lounge has been dedicated to creating digital humans, but also has a strong background in scanning sets and locations and scanning vehicles and objects. The studio works for clients in film and television, games and virtual reality, as well as architecture and design.

The outfit offers a highly personalized service encompassing all aspects of visual effects and digitization. Their client list is impressive, ranging from major film studios (including Warner Bros, Disney, Universal and Netflix) to UK institutions such as the British Museum, National Portrait Gallery and National Theater to major consumer brands. such as Dyson, Intel and AEG.

Lidar Lounge’s successful relationship with Sony is set to continue into the future

Lidar Lounge has scanned a myriad of objects in the UK museum, after hours, including Egyptian mummies and the iconic Rosetta Stone. Normally, these types of exhibits are only viewable by the public behind glass, but the company was allowed to view them privately, up close, in order to create pristine 3D scans.

Lidar Lounge’s successful relationship with Sony is expected to continue into the future, with many innovations and new techniques expected to be possible with what Sony provides. In the short term, the company will start integrating Sony’s pixel shift functionality into its light passes. They are also looking to use Sony’s Camera Remote SDK to create more precise trigger timings with multiple lighting setups that vary depending on the position of the cameras in their rigs.

This article originally appeared in 3D World Magazine. Play more great 3D content in 3D world, the world’s best-selling magazine for CG artists. To subscribe to 3D world (opens in a new tab).

Read more:

Share.

Comments are closed.