Apple’s mixed reality headset challenges from Jony Ive to battery life, heat, cost, and lack of discipline

This follow-up report focuses on the difficulties Apple faced in developing its mixed reality headset. The Information confirms details that Bloomberg has provided in recent years. But it also digs deeper into the problems the project has faced from Jony Ive to the wearable’s power, battery, heat, cost, and even an alleged lack of discipline from the AR/VR team.

Earlier this week, The Information shared part one of this report. According to sources that have worked on the mixed reality headset team, early problems starting as far back as 2016 have included CEO Tim Cook not being a “champion” of the project, and Jony Ive shutting down the idea of making a VR headset pivoting it to the current AR/VR design.

• Related: Bloomberg: Apple execs recently presented augmented reality headset to company board, suggesting product is close to launch

Part two of The Information’s report corroborates what Bloomberg reported in 2020, that Jony Ive also made the decision in 2019 for the headset team to ditch a design that worked with a base station to go with a less powerful but simpler headset that worked as a standalone device.

Two people who were familiar with the demonstrations said that

Apple’s CEO Tim Cook, and Jonathan Ive, then Chief Design Officer at Apple, viewed VR prototype headsets to show how they would compare. The headset that worked with a base station had superior graphics, including photorealistic avatars, while the stand-alone version depicted its avatars more like cartoon characters. Mike Rockwell, the Apple vice president in charge of the company’s AR/VR team, favored the headset with the base station, believing that Apple’s top brass wouldn’t accept the stand-alone version’s lower-quality visuals, according to the two people.

He was wrong. Ive had pushed for the stand-alone version of the headset since the early days of the project, according to a person familiar with it. Ive was ultimately supported by Apple’s top executives. Despite that, Rockwell still assured them he could make a great product. The choice has had lasting repercussions for the repeatedly delayed headset, which goes by the internal code-name of N301.

This decision is said to have caused many difficulties as the headset team tried to balance battery life and performance, while minimising heat generation so that people aren’t burned while using the device .”

Sources say the failure of Mike Rockwell – the headset lead – delivering “the high-quality mixed-reality experience he told Apple’s executives he would” is the primary reason the product has been delayed so many times.

According to three sources familiar with the matter,

Apple leaders are expecting an AR experience that is far superior to what Meta Platforms (parent company of Facebook) offers in graphics, body tracking, and latency. Latency refers back, as the delay between user movements and what they see on their screen. Even a tenth of a second delay between the user’s head movement and what is displayed on their headset can cause nausea.

There has also been division amongst the headset team as to what the target customer should be. Some say Jony Ive changing the course from a more powerful headset with a base station to a standalone device was the wrong move.

Some of those people place the blame on Ive, who they say fundamentally changed the purpose of the headset from a product that creatives and professionals would use at a desk to a portable device for consumers. These people believe that Apple should first have created a headset for professionals in order to inspire them to create content before making one available to consumers.

The Information’s new report shares details like integrating 14 cameras, solving video stream issues, and processor problems also being roadblocks.

Another design decision that has greatly added to the technical challenges for the Apple headset has been the inclusion of its 14 cameras, which allow it to capture everything from images of the outside world to facial expressions and body gestures.

Apple needed to create the Bora signal processor in order to process all the imagery. But Apple’s engineers have faced technical challenges getting Bora to work with the headset’s main processor, code-named Staten. The back-and-forth communication between the two chips increases latency, which can create nausea for people wearing the headset.

This meant that Apple needed to add another piece to the puzzle: a streaming codec. And reportedly that hasn’t been fully ironed out.

Even more challenges for the project have been a lack of discipline.

Prior to 2019, it had a freewheeling culture, operating almost like a startup within Apple, said four people familiar with the team. The employees brainstormed new features and tried out ideas that may never be realized.

Apple brought in Kim Vorrath, a veteran of the company to lead the project. Dan Riccio was later appointed hardware chief to supervise the mixed reality headset.

Vorrath created more structure in the group. Each team had to develop defining features to use with the headset’s software. This was to encourage them and increase accountability. After she joined, engineers were introduced to a concept she had used in software engineering known as the “six-week sprint,” said two people familiar with the matter.

Interestingly, even after Jony Ive departed Apple, some headset team members had to go to his home to get “approval on changes.”

A person who is familiar with the subject said that Ive has been working as a consultant for Apple ever since his departure. He also mentioned that Ive often helps the former team to refine their preference in camera placement, ergonomics, and battery life. According to two people, Ive still needed to travel from Cupertino, California to San Francisco to approve any changes.

Another interesting fact is that Ive tweaked headset designs over the years. His most recent preferences are for the battery to be worn with the headset and not integrated into the helmet.

Last but not least, cost has been a challenge. Apple appointed Dan Riccio, an executive to manage the project.

Reports from Bloomberg and The Information have suggested Apple is currently been looking at a price from above $2,000 to $3,000.

For all the details on Apple’s headset challenges, check out the full report from The Information.