First ride: BMW M Mixed Reality M2 is a real racing simulator

From motion-controlled VR rigs to Volvo’s $3 million R&D chassis simulator, I’ve tested a wide range of racing simulator setups. Yet, as close as the best come to replicating the driving feel, all fall short of the real thing. The latest project from BMW’s M racing and performance division tackles this problem in the opposite direction, introducing a real 2023 M2 sports coupe with acceleration, cornering and power sliding into the simulation using reality technology. mixed.

I got to test drive BMW’s M Mixed Reality M2 prototype on a gray afternoon in late November at the M Driving Academy at Fürstenfeldbruck Air Base, just half an hour’s drive from the automaker’s Munich headquarters. For the most part, it’s just a normal 2023 M2 with a dark techno livery. Opening the door, I see the large stereoscopic camera pod that takes up most of the dashboard and the mixed reality headset waiting for me in the driver’s seat.

The headset is the Varjo XR-3, a pro-level mixed reality unit that features lidar-based 3D spatial positioning, passthrough cameras, and hand and eye tracking. It’s the latest version of the XR-1 platform that Volvo used a few years ago to test driver safety, and it’s one of the most advanced VR/MR headsets I’ve ever seen. attached to my melon. The XR-3 has been slightly modified with six small positioning spheres which further improves head tracking accuracy.

As I settle into the racing bucket seats and fasten the four-point harness, the BMW engineer explains to me that by combining virtual reality with real driving, the M team is taking an approach to upside down to create the most realistic simulator. There’s also less motion sickness than with a fully virtual experience, as your body (specifically, your inner ear) will experience real-world physics. He also explains that he has a remote brake pedal on his side of the car – like a driver education car – but has promised to only use it in an emergency.

Through the XR-3’s screens, my vision is filled with live video of the M2’s actual dashboard in front of me, with my real hands on the very real steering wheel. It’s as if someone had replaced my eyes with the cameras of the MR helmet. It’s a little weird, but there’s no noticeable latency. I am ordered to switch to driving and move to the starting line.

Suddenly, the view outside the car changes. The gray skies and empty airstrip I had lined up for fade away and are replaced by a futuristic race track with a cyberpunk nighttime cityscape in the distance. When I move my head, the virtual world moves perfectly in sync; I can even lean forward or tilt my head to look around the roof at the night sky – at least, as far as the harness allows me. My real hands and the. M2’s dash is still there – it’s the mixed part of this reality – but beyond that stretches a neon-lit route, marked by tall light barriers. The large camera module, I notice, is also apparently gone.

My co-driver explains that the elongated circuit in the shape of a figure of eight fits perfectly into the paved track – which they surveyed and GPS-marked with a drone at the start of the event, much like you would mark the safety boundaries of room-scale virtual reality at home. The system uses a combination of camera and laser head tracking, precision accelerometer and GPS data, along with data taken from the M2’s on-board sensor network, to precisely align the vehicle’s position and my increased field of vision.

Then it’s “on your marks, get set, go!” as I am being asked to take a slow lap to familiarize myself with the system. The moment I start rolling, all the quirks of mixed reality disappear and I’m immersed in the simulation. Muscle memory takes over as I smoothly pilot the M2 through the gaming landscape. BMW added a bit of Mario Kart gamification to the course with BMW Roundel Tokens to collect to improve my score and large red number barriers to avoid. It’s funny.

Of course, there are limits to the technology. It can’t create cambers or elevation changes where there aren’t any – a large car park will never look like the banked carousels of the Nürburgring or the corkscrew of Laguna Seca. So the two times this technology was demonstrated to journalists were in flat simulations on smooth tracks. The reverse is also true: if there’s a bump or pothole on the actual riding surface, you’ll feel it – and if a puddle or oil slick forms or a deer rushes into the driving zone, you will not see it entering the driving zone. simulation, which is why the human co-pilot and his emergency brake pedal are there.

Later, BMW engineers enthusiastically explain their vision for using the technology developed in the M Mixed Reality prototype to teach performance techniques to novice drivers or to allow casual enthusiasts to test a car’s real world limits. like the M2 without fear of putting it into a wall or wrapping it around a pole.

Expert drivers can also benefit from mixed reality driving. Let’s say there’s a tricky segment on the Nürburgring that you want to practice. Load it up on a skidpad and hit it again and again without needing to complete the entire Nordschleife between attempts. You could spend the day autocrossing a different course each run without having to lay out or pick up dozens of cones.

On my second lap, I push the speed a little further and the illusion remains intact. The car moves precisely through the simulated course, naturally following my inputs and perfectly matching the feel of the seat of the pants. Only the weight of the headset reminds me that I haven’t been magically transported inside an arcade racing game.

The BMW M Mixed Reality Prototype is a lot of fun, but for now it’s just a development platform with no intention of producing or offering the technology to M car owners. my session, all I really want is one more round.