How digital projection tech may soon replace video calls

When Swiss watch boss, Christoph Grainger-Herr, was unable to fly to a global trade show in China because of Covid-19 restrictions, he decided to beam in Star Trek-style instead.

Mr Grainger-Herr, the chief executive of luxury brand IWC, had been due to travel to the Watches and Wonders event in Shanghai back in April.

When that became impossible, instead, he decided to joined the show as a life-size, 3D digital projection. Appearing in 4K resolution, he was able to talk to, and see and hear the people who were physically attending the event.

"We beamed him from his office in Schaffhausen, Switzerland, to the event in Shanghai," says David Nussbaum, the boss of US digital projections firm Portl.

"He did his thing, chatted to other executives, and even unveiled a new watch, all in real time. And then we beamed him out again!"

With the coronavirus pandemic having put a stop to much global travel since March 2020, it has fuelled a growing interest in the use of 3D light projections of a person - as a more life-like, more immersive, more sensory alternative to video calls.

Los-Angeles-based Portl is one of the firms at the forefront of the technology, and Mr Nussbaum says "we can't make our portals fast enough".

Its portals are eight feet (2.5m) tall, glass-fronted, computerised boxes. Inside the booths a life-size projection of a person appears.

The portals have built-in speakers, so that the projection's 'voice' can be heard. They also include cameras and microphones so that the user can see and hear the people in front of his, or her, projection.

Where the person is actually physically standing - and that can be on the other side of the world from the portal machine - he or she just needs a camera, a plain backdrop, and another set of speakers and microphones.

Portl's app-controlled software system then connects the person via the internet to wherever the portal or portals are - and you can connect to as many as you like.

New Tech Economy is a series exploring how technological innovation is set to shape the new emerging economic landscape.

"There is almost no latency [delay]," says Mr Nussbaum. "And were it not for the sheet of glass in front of the projection you'd think the person was actually [standing] there. In fact, if there is no light on the glass so that you cannot see it reflecting, then you do think the person is actually there."

The Portl system is aimed at business customers, and is currently also being used by other firms such as Netflix and T-Mobile.

The portals cost from $60,000 (£45,000) each, so they are certainly expensive, although the company says they can be rented for considerably less.

"In a few years time, this is going to become a regular way of communicating between offices," adds Mr Nussbaum.

At Microsoft, its digital projection communication technology is based around a headset called HoloLens 2. At $3,500 per unit they are considerably cheaper than Portl's system, but the 3D projections are not lifelike.

Instead, when two or more headset wearers call each other, their images are projected in front of each of them as somewhat cartoon-like avatars, that appear to be standing in the same room.

"It would appear that they are in the same physical space, and they could walk around a virtual table and collaborate on things," said Greg Sullivan, director of mixed reality at Microsoft.

Also aimed at business customers, German engineering group Thyssenkrupp is one firm putting the technology to practical use.

One of the world's largest manufacturers of lifts or elevators, it used to have to fly its technicians around the world to make any necessary repairs. Now these employees can instead use HoloLens 2 headset to connect in projected form with a local technician, guiding him or her though the work that needs doing.

Meanwhile, Japan Airlines is using the headsets to help train engine mechanics and plane crews.

Other digital image projection firms are more focused on the consumer market, such as San Diego-based Ikin. Next year it is launching a device that you clip to your mobile phone, and it will project into the air a transparent 3D image of the person you have having a video call with.

Gordon Wetzstein, an associate professor of electrical engineering and computer science at Stanford University, says that such 3D digital projections are a "more effective way" of communicating than video conferencing.

"[With the projections] you can create eye contact. You can read subtle cues like who's looking at whom, " he says.

Yet, he cautions that problems may occur in the future if these images become so real that distinguishing them from an actual person will become impossible.

"If you can create digital or synthetic experiences that get closer and closer to how you perceive reality, you're more vulnerable to being manipulated," says Mr Wetzstein.

Back at Portl, one of the firm's earliest investors was Tim Draper, who was also an early backer of both Skype and Tesla. Portl's Mr Nussbaum says he is confident that the technology is going to replace standard video screens in video conferencing in "five years".

He also predicts that it will see off video information screens. "We'll replace every single digital display kiosk in every mall, in every lobby, in no time. This will be the new way that businesses will want to present their content whether live or recorded."

Correction 21 September: This article was updated to replace the word 'hologram' with the words 'digital projection' to accurately reflect the technology being reported on in the story.