As Project Starline grows to other test areas, it seeks to determine how to make video conversations more immersive. I personally tested one out, and it’s startlingly true.
strangely calming oddly engrossing Invisible? After spending 30 minutes demonstrating Project Starline, Google’s prototype big-screen 3D video chat platform that is now being tested in select corporate locations outside of Google, I didn’t expect to say these things. It seemed more genuine than I anticipated, with the exception of a fist bump that turned into pixels.
As a prospective look at how virtual meetings can grow in the future, Google unveiled Project Starline more than a year ago at its 2021 Google I/O virtual developer conference. The concept seems to have been inspired by pandemics of the time: a desire to make individuals feel as though they were seated next to one another while being thousands of kilometres apart.
VR meetings with avatars have been envisioned as a means of overcoming geographic barriers by companies involved in VR and AR, such as Meta. With video chats like Zoom, you may see someone face-to-face, but you may feel glued to your laptop screen. VR’s trade-off is a headgear that conceals your real face. Future avatars created by Meta are crafted to resemble actual people and are controlled by face-tracking cameras. By simply displaying a live, 3-dimensional feed of your face, Project Starline creates the same effect.
While Project Starline is a two-way video chat, what surprised me was how much it also resembled augmented reality (AR) or holography. Finally, it will be like nothing at all.
Recollections of a virtual encounter
I waited a day before writing about my Starline half-hour discussion because I wanted to check my memory. One of the two team members I spoke with in my personal hologram booth suggested that I could recall the conversation as having actually taken place in person.
In truth, I did have a face-to-face meeting with the Google Starline team, which included research scientist Jason Lawrence and director of product management Andrew Nartker, before they departed for a different room down the hall with another Starline booth while I remained seated in my own. We had a hologram conversation while positioned close to each other in two different rooms. Alternately, using 3D light field displays.
With the perks of a video chat and the weirdly genuine presence, I’m more accustomed to in virtual reality while using avatars.
I was intimidated the moment I walked inside the Starline booth. Google forbade me from taking pictures or films of my experience, but it consisted of a tall-backed bench facing a large screen, with a wooden bar/desk like a counter dividing the two in a way. The screen and the base of the desk/bar in front of me were encircled by a riot of cameras and what seemed to be depth sensors. I believe I counted at least 12 sensors or cameras. (Lawrence went on to say that the multiple depth-sensing cameras are collaborating as part of this, using a real-time depth camera technology that we created.) )
Nartker then entered and sat down while I continued to look at the screen. He looked to be sitting across from me in a three-dimensional vision. This was extremely remarkable for two reasons: It was a real-time video conversation, and he seemed life-size in front of me. I’ve seen light field displays previously that achieve similar glasses-free 3D tech.
The life-size portion was startling at first but then strangely reassuring. We made terrific eye contact as we seated. I saw that I was looking at my partner so intently that I had to turn away. As I spoke while squirming a little, I saw that my posture had slouched. I began to unwind. We appeared to be conversing at a table at a coffee shop, to be honest.
Nartker tells me, “Looking through these amazing windows and genuinely feeling each other in this rich way, we might be anywhere in the world.” “You’re giving me a direct look and creating eye contact. In terms of video conferencing today, we really can’t do that.”
An interface that disintegrates
I was disarmed by the size and placement of the Starline’s display and cameras since I didn’t have to worry about lining up my view by glancing at a single one. Even my appearance was a mystery to me.
“It almost seems as though the space simply links. And these spaces combine. While you and I are simply hanging around here, “said Nartker.
Though not as sharp as in real life, Starline’s display resolution was adequate. Except for occasional little pixelation or breakdown at the margins or when our hands lean too far forward, Nartker’s 3D appearance appeared to be solid. We can see each other’s movements in an approximately 1-meter-square area of space that is illuminated by starlight. I want to lean toward the low wooden wall that serves as a kind of physical boundary.
I had the want to put something on the counter, thinking that he would magically reach over and take it. Naturally, it is not feasible. Placing a holographic item on a desk is also not acceptable.
“Because there is a spot where these two rooms kind of converge, and that is precisely where these objects join, we placed that tiny wall there. However, I nearly feel like you and I would stretch across that wall’s corner to give each other high fives “Nartker elucidates.
Later, we fist-bumped to demonstrate the space scanning’s boundaries. As my hand drew near, his hand began to pixelate and disappear. I imagined him witnessing me perform the identical action on the opposite side. Between us, there is a single volumetric space.
The wall behind my hologram chat partner is a virtual background that seemed identical to the wall behind me and had virtual shadows added; I didn’t notice this till later. Because portion of the region technically is beyond the range of the depth-sensing camera, it is there.
Starline has a 60Hz display and is made to operate over standard network bandwidth. Even while the 3D accomplishment appears to have a slight resolution reduction compared to what a large 4K TV would look like, it still adds an extra layer of present realism.
How often would I use it to view objects other than people? You can see the objects he holds up, but in a little less-crisp-than-regular-video format. a fruit like an apple. My wallet and car keys are in my hand.
After utilising Starline for a little while, I truly did feel more at ease and, dare I say it, normal, albeit entering a special room with a booth and a wall screen covered with cameras was an unusual on-ramp to feeling casual. The extensive installation of Starline prevents it from being as immediate or frequent as a typical video call you may make on whatever device you currently have with you.
But when the technology started to fade away and our talk took over the majority of the screen, I felt a lot more focused and at ease. It certainly appeared to be some kind of bizarre magic trick, but it also began to seem more like a face-to-face interaction than a Zoom.
What follows?
I’d never seen real-time video chat on a light field 3D display before, but Starline’s inquiries about what presence at a distance meant got me to thinking about the wider VR and AR world. No VR or AR headset has been able to replicate how a real-world discussion between two people feels natural due to the obstruction caused by the equipment on your face. Google Glass, the company’s first wearable smart display, tried but failed to blend in with society.
According to Nartker, “This is a prototype, an early proof of concept for where this technology may go.” “The first prototype that we truly thought gave us both a sensation of co-presence was this one. When we investigated this with Googlers, we discovered that individuals behaved realistically. They characterised it as genuine and believed it to be so.”
Google’s head of AR and VR, Clay Bavor, who also introduced the Daydream VR platform, is in charge of the Google Labs team guiding Project Starline. The creation of assistive smart eyewear was also unveiled during the I/O conference this year. Google appears to be deep in the research phase of its work on augmented reality and virtual reality, examining potential solutions that may eventually be used in other products.
Currently, Project Starline is being set up two at a time in a select group of non-Google test offices. Although the technology is large, it is hoped that the design may eventually appear in other, smaller versions. However, the life-size aspect of Starline, which necessitates a bigger screen to cast a full-size body, seems to be a major component to its success.
I immediately came up with applications: perhaps a customer service counter that could be manned around-the-clock even when no one is physically there. Maybe a celebrity meet-and-greet kiosk? Would it function as a collaborative or brainstorming space? I’m not sure if any of that would eventually be superior to a standard Zoom/Meet/FaceTime call or a VR session. However, I’m quite interested to see how Google will incorporate the lessons it learns from Project Starline into its upcoming headset or other product.
