Apple’s mixed reality headset project challenges explained

Since beginning its efforts in 2015, Apple’s work on a mixed reality headset has been hit with a range of challenges that have pushed back the potential launch date by years. A new report by The Info reveals what major obstacles have been identified from within the company.

The Information was able to talk with 10 people close to Apple’s mixed reality headset project, most of whom worked on the team. Going all the way back to 2016, just one year after development started, the sources say Apple’s board was presented with a demo of very early prototypes to try and “build support at Apple’s highest levels for bigger investments.”

A former Vice President Al Gore and Bob Iger, the then-Disney CEO, walked around trying prototypes of augmented reality and virtual reality software and devices. A gadget made it possible to create a miniature digital rhinoceros on the table. According to two witnesses, the creature grew in size and became a living version of its self. In the same demo, the drab surroundings of the room transformed into a lush forest, showing how users could seamlessly transition from AR, in which they can still view the physical world around them, to the more immersive experience of VR–a combination known as mixed reality.

Sources say some of the prototypes were “cobbled together” and ran Microsoft Windows while others were modified HTC Vives or other existing headsets. However, one of them was so heavy it required a “small crane” to hold it “so the Apple board members could wear it without straining their necks”

While it doesn’t seem like a very impressive experience, Apple’s Board was persuaded to fund more development funding. Another factor was Apple’s concerns about Facebook’s performance in this space.

Going forward, the biggest challenges have been technical and one factor that’s allegedly been at play is a CEO Tim Cook supporting the work, but not being a “champion” and highly involved in it.

Technical challenges were the main reason for delays. This has also been true in the past with Apple’s most innovative new products like the iPhone. But the Apple smartphone also had a singularly influential figure in Apple co-founder Steve Jobs to midwife it.

While Apple’s current CEO, Tim Cook, supports the headset project, he hasn’t been as active in the effort as Jobs was with the iPhone’s development, according to five people familiar with the project. For example, he rarely visits the group at its offices away from the main Apple campus, those people said. The lack of a honcho of Cook’s stature to champion the headset, code-named N301, has made it harder at times for it to compete with other products such as the Mac and iPhone for head count and engineering resources, the people said.

Jony Ive presented another challenge. When the headset team founder and leader Mike Rockwell was working on getting buy-in from Apple’s various teams to help with the development, his team was shut down on the idea of making it a VR headset. Ive’s team also opposed the idea of “practical uses”, and doubted that consumers would wear headsets for a prolonged period.

Rockwell, Meier and Rothkopf soon encountered pushback from Ive’s team. The three men had initially wanted to build a VR headset, but Ive’s group had concerns about the technology, said three people who worked on the project. They believed VR alienated users from other people by cutting them off from the outside world, made users look unfashionable and lacked practical uses. Apple’s industrial designers were unconvinced that consumers would be willing to wear headsets for long periods of time, two of the people said.

That ended up birthing the idea of a mixed reality headset:

The men came up with a solution to address the concerns of Ive’s team. The three men suggested adding cameras on the front of their headsets so people could view the surrounding environment. The feature that sold industrial designers was the idea for an external-facing display screen. The screen could display video images of the eyes and facial expressions of the person wearing the headset to other people in the room.

These features addressed the industrial design group’s worries about VR-induced alienation–they allowed other people in a room to interact and collaborate with a person wearing a headset in a way not possible with other VR gear. For years, the existence of such a display, internally code-named T429, was known only to a small circle of people even within Rockwell’s group.

The Information’s report hints that a follow-up piece will cover a “pivotal moment for the Apple headset” that occurred in 2019. That’s likely when Jony Ive “balked” at the idea of selling a headset that required a base station device to operate. The team then moved to developing a smaller, more autonomous AR/VR device.

The latest expectation is that Apple could announce its mixed reality headset in 2023. As far as price, we’ve heard reports that it could sell from above $2,000 to $3,000.

• Report: Apple’s mixed reality headset to be delayed until early 2023
• Apple VR/AR headset reportedly overheating, June announcement could be nixed for 2023 debut