This week, we did not have mindblowing XR-related news, but there are a few things that happened related to innovation and R&D that are worth a read. Let’s say that for sure, there has been much more to be excited about in AI thanks to the release of GPT-5…
Top news of the week
Genie 3 shows us an alternative path for the creation of VR experiences
When talking about interactive XR experiences, we always think about some content developed with a game engine that requires 3D assets and the logic to give them life and interactivity. But Google this week showed us a new intriguing alternative path to that, thanks to the Genie 3 AI model.
Genie 3 is a generative AI model that is able to offer interactive 3D experiences that are completely generated by AI, frame-by-frame. You can write a prompt about being on Mars, and Genie 3 will put you on the Red Planet, with you having the ability to move around using the arrow keys on your keyboard and interact with some elements with your mouse. Every frame you see will not be generated starting from a mesh of Mars, but it will be completely generated by AI, as if it were AI video generation. One amazing thing about Genie 3 is that a lot of work has been put into consistency: previous AI models able to create interactive experiences failed to create a consistent experience: you started seeing something, then you rotated four times by 90°, which should bring you to the initial point of view, and you saw something else from what you saw in the beginning. This made the navigation in the generated world impossible.
Genie 3 is able instead to maintain consistency related to the environment around you and the interactions you had on it for a few minutes. There are some videos shared by Google about it that are impressive: they seem to come from a 3D experience, and instead, they are purely generated on the fly by AI. Of course, it’s still an experimental project with many limitations: Genie 3 can “render” its experiences at only 24 frames-per-second in 720p resolution. We are talking, of course, about flatscreen experiences, because VR is not supported. There are also other problems, like the fact that it can’t render text, it does not support multiple AI agents in the world, and that, anyway, after a few minutes, the experience starts losing consistency.
But notwithstanding all these issues, Genie 3 is opening our eyes to a new way of generating 3D interactive content. May that be possible that this can evolve in the future in what was the “Holodeck” in Star Trek? May it be that in some years from now, I can just say “make me have a romantic trip in Venice” and AI is able to generate on the fly a full experience in that sense for me? Considering the impressive evolution that Genie has had from its first generation, it seems it could be possible. On the other side, reaching that goal is not easy, because one thing is making a few nice videos for a blog post, and another one is having a system generating a flawless experience that is exactly as we wanted it to be, whose consistency lasts for hours, and that also allows for multiplayer, which requires inter-player consistency. We may need years to arrive at that point, or we may not even reach it at all.
I think for now, we’ll keep sticking to the good old way of creating content, but with an eye open to see what the future may offer us.
More info (Google Genie 3 — Road To VR)
More info (Google Genie 3 — Upload VR)
Other relevant news
Meta boosts Horizon Worlds with AI and social media
Meta just released new AI-powered features that make the generation of worlds in Horizon Worlds even better. The first one is “Creator Assistant”, a sort of copilot that helps you in the creation of worlds. You can write a prompt and it can create for you interactive elements, generating both the 3D models and the underlying logic. The other one is “Style Reference”, which instead helps in the generation of new 3D models that have a coherent style with the ones already in the world.
These new tools add to the already many AI tools available in the Horizon Worlds editor. To my knowledge, the Horizon editor is now one of the creation tools with the most AI features, and the creation of worlds with it should be really facilitated thanks to artificial intelligence. We’ll see if this will lead to just a big pile of shovelware or if it will allow little kids to create impressive worlds.
Apart from the AI features, another important feature has been added to Horizon Worlds. When someone shares with you a link to a Horizon Worlds experience via Facebook and Instagram, the web version of the world will open up directly into your Facebook or Instagram app, without requiring you to install the Horizon Worlds app. This easy social shareability is an amazing feature, in my opinion, because it allows you to share content very easily with your friends.
Let’s see if all these new features in Horizon Worlds will help this application succeed or if it will continue to flop.
More info (Horizon Worlds AI features — Road To VR)
More info (Horizon Worlds AI features — Upload VR)
More info (Horizon Worlds opens inside Instagram and Facebook)
Meta CTO teased what to expect at Connect
Andrew “Boz” Bosworth, who is the CTO at Meta, shared some hints about what to expect at Meta Connect next month. He said to expect “big wearables announcements”, as well as AI and “metaverse software”.
We already know about the wearables announcements: Meta should launch very expensive smartglasses with a single display and a neural wristband called Meta Celeste. They could also announce a new model of Oakley Meta glasses with a camera in the center of the glasses.
It is not clear what he means with “metaverse software” instead. If I had to bet, I would say something that makes HorizonOS make some steps forward towards becoming the Metaverse that Zuckerberg envisions. For instance, the Home could become multiplayer, there could be new more realistic avatars, and enjoying content together could become easier.
Anyway, there is just one month remaining… and for sure, almost everything will be leaked one week before, as usual. So we don’t have to wait much to know more…
Meta details experimental Tiramisu and Boba 3 headsets
Last week, we had a tease of the experimental headsets that Meta is showcasing at SIGGRAPH. This week, we had many more details about them, thanks to a blog post that Meta itself shared with the community.
The first headset is codenamed Boba 3 and achieves a 180° horizontal by 120° vertical field of view. This means it covers 90% of your full field of vision. Boba 3 has a big resolution of 4K×4K per eye, which helps in giving a good pixel density over this large field of view: the central angular resolution is 30 pixels per degree (PPD), which is still higher than the 25 PPD of Quest 3. All of this is delivered while providing a rather thin form factor and a decent weight: the full Boba 3 VR without passthrough weighs 660 grams. Meta could theoretically deliver Boba 3 as a product, but it would be an expensive PCVR headset, something that is not in its interest. I wonder at this point if it could make sense to license its technology to PCVR companies like Bigscreen, Pimax, or Somnium, so that at least this could reach the market.
The other headset is called Tiramisu and offers improvements over different features of the Quest 3. Specifically, Tiramisu has a central angular resolution of 90 PPD and brightness of 1400 nits. That compares to 25 PPD and just 100 nits for Quest 3. It also provides 3x the contrast of Quest. Considering that the human retina can only perceive up to 60 PPD, this headset can provide visuals with perfect resolution and incredible brightness, as if they were real. Meta is preparing a demo for it using the best possible visuals that Unreal Engine 5 can provide, and I can bet that this demo would be mindblowing for everyone able to try it. Meta talks on purpose of “Hyperrealism”. There are some trade-offs in this headset, of course: it has a comically small field of view of just 33° x 33°, a large form factor, and uses heavy and expensive glass lenses. But still, I would love to try it…
News worth a mention
Adobe is shutting down Aero
The most surprising and saddest news of the week is that Adobe is shutting down Aero on November 6. When Aero launched some years ago, we all praised it because we saw it as an easy way to create augmented reality mobile experiences for creatives that required no coding skills. It was also proposed by Adobe, which is a company very well known by creatives, so we thought it could really help “democratize” the creation of mobile AR experiences. Probably, there was not much demand for it, and so the application is closing. What sad news!
Niantic Spatial SDK is now compatible with Meta Quest
The Niantic Spatial SDK, which is usually employed to create AR experiences on mobile, has become compatible with Meta Quest 3 in its latest version. This is incredibly interesting because it adds to the Meta Quest some functionalities that are currently not available with the Meta SDK, like live meshing, object detection and segmentation, and VPS. It’s a new tool XR developers can start working with to create new types of experiences for Quest or new research projects. It also helps in blurring the lines between what is VR and what is AR today.
Proxies may improve AR interactions
A new interesting research project by a group of researchers shows how some interactions in AR may be improved by the use of proxies. Imagine that you have to select in AR/MR a physical book in your bookshelf, while you are sitting down close to your desk: probably you have to aim with your eyes+hands at the book, but this ray-cast interaction is very error prone, because the books are distant, so it’s hard to correctly “aim” at them. It is much better if close to you you have a virtual interface representing the books with simple squares, and you scroll through these squares until you find the one related to the book you need. In this case, you did not interact directly with the real object, but you used a proxy interface that made your interaction easier. There is a video associated with the article: I suggest you watch it because it is very cool and can explain this thing much better than I did with just written words.
(Thanks to Dr. Mar Gonzalez-Franco, who is one of the authors of the paper, for pointing me to this piece of news)
Oasis WMR Driver to come later this month
Matthieu Bucchianeri, aka “Mbucchia” is releasing the “Oasis” WMR driver later this month. Oasis is an unofficial driver for WMR headsets that will give them new life after Microsoft discontinued support for them last year. The driver will work only with NVIDIA graphics cards and will allow many people still having a WMR headset to use it for VR as if it were a SteamVR headset. As usual, Mbucchia is doing God’s work.
Meta managed to run three Codec Avatars on Quest
Meta Reality Labs researchers managed to run 3 full-body photorealistic “Codec Avatars” at 72 FPS on a Quest 3, with almost no quality loss compared to the versions rendered on a PC. There are, of course, trade-offs compared with the PC version, like the fact that these avatars do not have dynamic lighting, but still, it’s an important step towards making these realistic avatars run on mobile hardware.
Instagram on Quest can now convert videos to 3D
Instagram on Quest can now use AI to automatically convert your videos to 3D when they are viewed through the Horizon OS app. It works quite well. All these automatic 2D-to-3D conversion systems make us wonder if in the future we will still need phones with stereo cameras to capture 3D content: my XR colleagues on X say that today, a video natively shot in 3D is still much better than one AI-converted to 3D, but who know what will be the situation in a few years from now…
ForceBot makes you walk in virtual reality
There are many solutions that make you walk in virtual reality by walking in real life: for instance, we have treadmills like Virtuix Omni or shoes like Ekto VR. A new one that just got some visibility is ForceBot, which makes you walk naturally on a machine that physically moves your feet back, thanks to some robotic arms attached to your feet. The machine, for now, can only make you walk forward, with no steering, but maybe the system will evolve in the future.
Some news about content
- Boxing Underdog had an impressive launch, immediately reaching the #4 spot in the top-earning games by revenue on Quest
- UnLoop, a game where you cooperate with yourself to solve puzzles, will be available on Quest and Pico on September 5, 2025, for $19.99
- Don’t Move challenges you to survive a series of safety tests where weird things happen, and you don’t have to move. It is launching on Steam on August 14
- Elysian, a fast wave-based shooter game, is the first WebXR game on Quest’s Meta Horizon Store, an example of how WebXR developers can leverage store discoverability.
More info (Boxing Underdog)
More info (UnLoop)
More info (Don’t Move)
More info (Elysian)
Some reviews about content
- Boxing Underdog is an impressive boxing early access game, with realistic fighting, a crude opponent wounding system, and an AI system that makes the opponents fight in a challenging and realistic way.
- Hidden Memories of The Gardens Between is a nice and enjoyable emotional journey, but it has also some problems, like the fact that it is clear that it is not a VR-native experience or that it is very short.
More info (Boxing Underdog)
More info (Hidden Memories Of The Garden Between)
Other news
YORO reduces the computational needs of VR by just rendering the visuals of one eye
DaVinci Resolve on macOS now has full support for Apple Immersive Video
This is how the 3D-reconstructed Burnley-Lazio match looked like in Rezzil
Synchron now lets disabled people use Apple devices like AVP using just their brain
News from partners (and friends)
No relevant news to highlight here, so I take the occasion to wish you a happy Summer!
Discover FM DUO camera!
Meet the FM DUO, a VR camera that is capable of filming up to 12K @30fps, and 8K @60fps. With built-in RTMP live streaming and 5G connectivity, the FM·DUO will immerse your viewers in real time! The FM·DUO features 2 full-frame CMOS sensors, with 6000 x 6000 pixels each. This allows for high-resolution zooming capabilities. Not only can you narrow in on one subject, but you can also focus on singular details of the subject’s facial expressions, clothing textures, and a multitude of other small details, enabling a much more immersive experience of VR video.
Visit Product Website
Some XR fun
PCVR in 2025
Funny link
WTF LinkedIn
Funny link
Donate for good
Like last week, also this week in this final paragraph I won’t ask you to donate to my blog, but to the poor people who are facing the consequences of the war. Please donate to the Red Cross to handle the current humanitarian situation in Ukraine. I will leave you the link to do that below.
Let me take a moment before to thank anyway all my Patreon donors for the support they give to me:
- Alex Gonzalez VR
- DeoVR
- GenVR
- Eduardo Siman
- Jonn Fredericks
- Jean-Marc Duyckaerts
- Reynaldo T Zabala
- Richard Penny
- Terry xR. Schussler
- Ilias Kapouranis
- Paolo Leoncini
- Immersive.international
- Nikk Mitchell and the great FXG team
- Jake Rubin
- Alexis Huille
- Raghu Bathina
- Chris Koomen
- Cognitive3D
- Wisear (Yacine Achiakh)
- Masterpiece X
- Dimo Pepelyashev
- Carol Dalrymple
- Keith Bradner
- Jennifer Granger
- Jason Moore
- Steve Biggs
- Julio Cesar Bolivar
- Jan Schroeder
- Kai Curtis
- Francesco Strada
- Sikaar Keita
- Ramin Assadollahi
- Juan Sotelo
- Andrew Sheldon
- Chris Madsen
- Horacio Torrendell
- Andrew Deutsch
- Fabien Benetou
- Tatiana Kartashova
- Marco “BeyondTheCastle” Arena
- Eloi Gerard
- Adam Boyd
- Jeremy Dalton
- Joel Ward
- Alex P
- Lynn Eades
- Donald P
- Casie Lane
- Catherine Henry
- Qcreator
- Ristband (Anne McKinnon & Roman Rappak)
- Stephen Robnett
- KaihatsuJai
- Christopher Boyd
- Anonymous Supporter
- Giuseppe Pippi
- Mark Frederiksen
- Ken Lin
- Sb
- Boule Petanque
- Pieter Siekerman
- Enrico Poli
- Vooiage Technologies
- Caroline
- Liam James O’Malley
- Hillary Charnas
- Wil Stevens
- Francesco Salizzoni
- Alan Smithson
- Steve R
- Brentwahn
- Michael Gaebler
- Tiago Silva
- Matt Cool
- Mark G
- Simplex
- Gregory F Gorsuch
- Paul Shay
- Matias Nassi
And now here you are the link to donate:
Support The Red Cross in Ukraine
(Header image by Google)
Disclaimer: this blog contains advertisement and affiliate links to sustain itself. If you click on an affiliate link, I’ll be very happy because I’ll earn a small commission on your purchase. You can find my boring full disclosure here.
Related
