How Cruise Uses Simulation To Speed Up Our Sensor Development

In January of this year, we introduced the Cruise Origin — the first purpose-built, fully driverless vehicle with a million-mile lifespan. It’s self-driven. It’s all-electric. It’s shared. And it’s what you’d build if you could start from scratch to create a transportation system that’s safer, better and more affordable for us, for our cities and for our planet.

The importance of our vision is why, even during a pandemic, we haven’t slowed down. General Motors continues to progress on development of the all-electric propulsion system for the Origin, with the first propulsion system test vehicle already on the roads of one of their massive test facilities in Michigan.

Here in San Francisco, we’ve kept moving forward on AV development using our robust simulation frameworks thanks to the years of data accumulated from on-road testing in one of the most complex city environments in the world.

But that’s not the only cool way we’re using simulation. We’ve also built a bespoke Sensor Placement Tool that uses simulation to help select and place the sensors on the Origin well before the vehicle goes into production. This tool is key to Origin development because it helps ensure overlapping sensor coverage in 360 degrees around the Cruise vehicle — giving us eyes not only in the back of our heads, but on the sides as well. This will enable the Origin to safely perform difficult maneuvers, such as driving around large double parked vehicles while detecting other roadway users, pedestrians and debris.

Placing and evaluating sensors

Cruise’s Sensor Placement Tool accurately enables the virtual placement, evaluation, and iteration of various sensor layouts across three sensor types: camera, radar, and LiDAR. We have accurately modeled the camera’s field of view and distortion, radar field of view, range and point cloud distribution, and LiDAR beam distribution and intensity. These simulated sensors have been refined with rigorous physical evaluation of real world sensor data to cover any unexpected anomalies.

The Sensor Placement Tool also supports the proper offset of the actual optical emission for each sensor from their planned physical housing. This ensures that the virtual sensors mounted in our Sensor Placement Tool are as accurately positioned as possible to their physical, hardware counterparts.

Lastly, we utilize a hardware-accurate CAD-based model of the Origin and load it into the Sensor Placement Tool for frame of reference during placement. This provides proper self occlusions to ensure the most physically accurate representation of the new sensor placement we are testing, including unexpected occlusions with the Origin vehicle body and other sensors such as camera field of view occlusion to radar.

Hardware Development without the Hardware: Allowing for Modular Configuration

With our refined Sensor Simulation, CAD-based Origin, and Sensor Placement Tool, we are able to unlock dynamic modular iteration of the Origin vehicle to greatly accelerate AV development. This functionality enables us to quickly test several sensor configurations and layouts in our tools, which would otherwise take weeks or months with actual hardware. With tight collaboration and review with our Cruise Hardware team, we are able to confidently iterate rapidly with various Origin layouts.

As always, safety is our primary concern. The ability to verify our dual modality/sensor type coverage at any given point with our systems helps ensure our sensor placement is not only performant in terms of maneuvers, but for safety coverage as well.

Using simulation, we are able to test sensors that don’t yet exist in completed hardware form. We can also assess various configurations of existing sensors to evaluate for better modality coverage or occlusion clearance.

Examples of what we can assess include:

• Trying cameras with greater or different fields of view or better coverage of a unique area around the Origin while avoiding occlusion from another sensor

• Different radar placement for driving at highway speed

• LiDARs with different beam patterns and unique orientations to try different configuration layouts for better coverage.

Being able to develop the sensor placement for the Origin virtually in our Sensor Placement Tool significantly expedites development by enabling early decision making for sensor selection, placement, and vehicle design choices without relying on physical hardware. This saves time, lowers costs, and adds flexibility to the Cruise Origin development process.

Optimizing Sensor Calibration using Sensor Simulation

Evaluating and placing sensors is only one part of the production process. Once we have selected a prospective sensor setup, we also need to consider how to calibrate that sensor on the Origin before it hits the road.

Sensor calibration confirms the sensor’s actual position and orientation on the vehicle — AKA extrinsics — which might actually be slightly different than the planned extrinsic placement for the vehicle.

Small differences in each sensor due to manufacturing, shipping, placement, and even jostling on the road can cause differences in the expected vs. actual extrinsics of each sensor. With an accurate and efficient calibration process, we can ensure the Origin will have an effective sensor fusion: combining information from different sensors. Knowing relative position and orientation helps us most accurately combine data.

However, performing calibration is complex and involves an array of fixtures with special markings that enable the calibration system to determine the actual intrinsic (internal sensor configuration) and extrinsic parameters by solving several optimization problems programmatically.

Placement of the fixtures is key and varies for each sensor modality/type. Determining optimal placement for calibration setup can be very time consuming, and once there is a change to sensor layout, that calibration environment must be adjusted as well. Since the Origin can support a variety of sensor layouts, refining this process is paramount to efficient development.

Simulation-accelerated Calibration Setup

Using simulation frameworks, our engineers are able to evaluate calibration setups needed to determine proper calibration. This is all done on a computer in a fraction of the time that it would take to reposition the various fixtures manually.

Once a new calibration method is determined in simulation, it can be built out in real life. Not only is this a faster approach for developing and testing, it’s also potentially less error prone. This is because starting from a simulated environment ensures that the Origin will have perfect calibration by calibrating to a known good simulation setup as opposed to starting from a setup Origin vehicle, which will inherently have some minor calibration offsets (as all manufactured parts do).

Leveraging simulation, various calibration errors can be tested by forcing faulty calibrations into our virtual environment. The calibration system then can be tested for proper determination of the new calibration parameters, thereby further ensuring the best calibration system for our Origin vehicles.

Finally, given that the Origin’s sensor layout is subject to change throughout the development cycle, using simulation frameworks enables us to have accurate and efficient calibration updates to keep up with these changes.

Designing the future of transportation

Simulation frameworks accelerate the development of the Cruise Origin by enabling engineers to determine the sensor selection and placement faster than the weeks or months it would take with actual hardware. Simulation also significantly reduces the costs associated with producing physical hardware while adding flexibility to the Cruise Origin development process.

The sooner we can safely bring the Cruise Origin to the roads of San Francisco, the faster we will be able to deliver the benefits of all-electric, self-driving transportation to our cities: improving safety by removing the human driver, reducing emissions by being all-electric, and alleviating congestion by making shared rides an awesome experience at a radically lower cost.

Utilizing simulation frameworks to expedite the Origin’s production is just one of the complex challenges Cruise is solving. Join our simulation, hardware, and AI teams to create the future of transportation.