Over 50 years ago, a charismatic, anthropomorphic 1963 Volkswagen Beetle named Herbie was the star of the famous Walt Disney film The Love Bug. Herbie possessed human-like traits, had a sense of humor, and drove himself. This was merely science fiction in the 1960s; however, today, the avant-gardist BMW Group have developed a system that makes this a reality! So, sit back, relax, and learn all about the amazing new BMW G70 M760e “Super Bimmer.”
Classification of Autonomous Levels
The Society of Automotive Engineers (SAE International) have developed a classification system for autonomous driving levels ranging from 0-5.
- 0 = No automation (Vehicle provides no automated functions)
- 1 = Driver assistance (Certain functions are automated, such as Active Cruise Control)
- 2 = Semi-automated driving (Multiple automated functions are available simultaneously, such as Steering Assistance and Cruise Control)
- 3 = Highly automated driving (Driving tasks are fully assumed by the vehicle in certain situations; drivers can remove hands from the steering wheel and focus on other tasks)
- 4 = Fully automated driving (Most driving situations are controlled independently by the vehicle, driver intervention is rarely necessary, and under certain conditions, the driver is permitted to sleep)
- 5 = Autonomous driving (Definitive autonomous driving, the vehicle handles all aspects of driving in any situation, under any conditions without human intervention)
Drivers must always be ready to intervene and regain control of the vehicle in levels 1-4; level 5 eliminates the need for a driver’s license, steering wheel, and pedals! United Nations Regulation 157 ALKS (Automated Lane Keeping Systems) permits Level 3 highly automated vehicles to travel up to 80 mph from the beginning of 2023.
BMW Personal Pilot L3
Many years of ongoing research, development, and millions of test miles have enabled the BMW Group to implement Level 3 highly automated driving on the BMW G70 7 Series, initially available only in Germany. The system responsible for this paradigm shift is the BMW Personal Pilot L3 (SA 5DX), a $6,500 (USD) option worth every penny.
Imagine that you are commuting to work at 7:00 a.m. in your flagship G70 M760e super saloon with massaging seats activated, coffee temperature maintained at 120F in the heated cup holders, and the Berlin Philharmonic Orchestra is soothing your eardrums from the prestigious Bowers & Wilkins Diamond Surround Sound System (SA 6F1). Wanting to get an early start on work before you reach the office, you press the Personal Pilot button on the steering wheel, the voice assistant acoustically confirms the handover, your display screen reduces size in the instrument cluster, the steering wheel LEDs permanently light up in turquoise, and the steering wheel turns inwards—you are now a passenger. Connect to the onboard WiFi hotspot and get to work before you get to work, watch a movie, or immerse yourself in the latest riveting Stephen King novel—how cool!
Anything lost can be found again—except time. BMW Personal Pilot allows drivers to reclaim the precious commodity of time, in luxury and the utmost safety. Steering, acceleration, braking, distance control, and any other required avoidance maneuvers are performed explicitly by the vehicle.
Prerequisites for Activation
Environmental factors, vehicle status, and driver condition are thoroughly evaluated and continuously monitored during Personal Pilot operation. Before your maiden voyage, Personal Pilot must be enabled prior to first-time use. This requires a valid BMW ConnectedDrive contract and pressing the Personal Pilot button on the steering wheel while the vehicle is parked. Privacy settings must also be activated via the central information display.
Various parameters are analyzed and must be satisfied before the vehicle allows the Personal Pilot feature to be released for use, which ensures functional prerequisites are clearly met. Sophisticated algorithms and powerful new hardware allow BMW to monitor live surroundings dynamically and guarantee intrinsic safety.
The Driver Camera System (DCS) monitors the opening of the driver’s eyes, calculates head position and uses video-based tiredness detection to interpret different facial geometries to identify changes in the driver’s condition. Using data from the DCS, a life-saving yet widely unknown feature BMW has been using for over five years now called Emergency Stop Assistant, has been revised for Service Pack 2021. In addition to automatic detection, the function can now be activated via voice command, “Hey BMW, Emergency Stop!” The hazard warning lights are activated, the vehicle brakes several times in an attempt to awaken a potentially incapacitated driver and the vehicle comes safely to a standstill. The doors are unlocked, the electromechanical parking brake is activated and emergency services will automatically be dispatched if the driver does not respond to the cancellation prompt on the central information display within seven seconds—talk about peace of mind!
With the release of Motorway Assistant (OE 5AX) in 2022 came BMW’s Live HD Map, which is essentially a dynamic representation of the environmental conditionals and provides centimeter precision high-resolution map data. Exits, tunnels, roadway markings, widths, and bends of each lane and structural separations of roadways are constantly monitored during Personal Pilot operation.
Unfortunately, you cannot activate Personal Pilot if you are wearing your new Gucci polarized sunglasses—if they are infrared impermeable—since the driver camera system cannot penetrate the lenses to detect the viewing angle and, therefore, driver attention. The vehicle must also clearly detect lane boundaries at least eleven feet wide and verify that no cyclists or pedestrians are on the roadway. In contrast to some competitors, BMW’s comprehensive analysis of a myriad of data allows Personal Pilot operation to be realized at night time, in tunnels (up to a certain length) and at highway junctions with no lane changes.
Technical Architecture
Redundancy—the inclusion of extra components which are not strictly necessary to functioning, in case of failure in other components—is a paramount characteristic of any practical autonomous application; BMW Group strictly adheres to this ideology in the G70 technical architecture. From CAN-FD to OABR Ethernet, the G70 electrical system is able to transmit massive amounts of data whilst achieving network optimization. Multiple RADAR sensors on the exterior of the vehicle, four 1620 x 1280 30FPS all-round vision cameras, tweleve robust ultrasonic sensors, multiple control modules processing data and executing tasks, redundant brake/steering/electrical systems, and BMW’s most complex new sensor technology,LIDAR (Light Detection and Ranging),permit the execution of Personal Pilot.
For the first time, BMW uses LIDAR technology on the G70 equipped with BMW Personal Pilot L3. LIDAR sensors operate similar to radar or ultrasonic sensors but use light rather than sound or radio waves to measure distance. Light is transmitted from the sensor, the light beam is reflected back to the sensor and the time it takes to return to the sensor is calculated to determine how far away an object is. To emphasize the complexity of a LIDAR system, let’s evaluate the following mathematical formula for calculating distance:
- d= c*t/2 {d= distance, c= speed of light, t=time it takes for light to travel to object & back to the sensor}
A universal physical constant is the speed of light–or 299,792,458 meters per second. If the speed of light is divided by 1,000,000,000, the result is 0.2997 meters per second. This is the distance traveled by light in one nanosecond (ns) or one billionth of a second.
Let us calculate the distance of an object in relation to a LIDAR sensor if the time measured between laser light transmission and reflection back to the sensor is 335 nanoseconds.
- d=0.2997 meters per second * 3.35 * 10^2 /2 = 50.1997 meters
Since we divided our answer by two, the total distance traveled is 100 meters—the actual distance to the object is 50.1997 meters. Unambiguous calculations like this are determined virtually instantaneously via the respective LIDAR signal processing unit, accentuating the system’s capabilities.
BMW’s G70 LIDAR system emits approximately 3 million pixels per second and each pixel is classified via the LIDAR control unit as to whether or not they can be driven under, are collision-relevant, can be driven over, or are completely irrelevant — this allows an accurate three-dimensional model of the environment to be generated by the vehicle.
Emphasizing BMW’s utilization of redundancies, a second 12-volt lithium-ion battery is connected in parallel to the primary electrical system within the G70’s luggage compartment to serve as backup if the vehicle voltage fails. The electromechanical power steering and braking system also have secondary control modules to ensure the operation is possible in the improbable event that one of these systems fails.
A mandatory feature required by law in many countries on Level 3 highly automated driving vehicles, similar to an aircraft’s flight data recorder or “black box,” is electronic accident data memory. BMW can permanently write accident data memory to a ring buffer on multiple control modules, with data ranging from longitudinal and lateral acceleration to changes in direction and speed. The categorization of accidents is also evaluated in the respective control module and can detect minor accidents without airbag deployment and more severe accidents with airbag deployment. Before and after an accident, data is written to redundant control units and permanently stored in the vehicle, which can be used in a court of law.
Handover & Takeover Process
Once all prerequisites have been satisfied, an icon stating “READY” on the instrument cluster will appear. On the bottom right of the display, a notification will state that highly automated mode is now available for activation.
To activate the function, press the Personal Pilot button on the left-hand side of the steering wheel. Once the vehicle assumes control, the driver is informed several times of the transfer of driving responsibilities to the vehicle. The LEDs embedded in the steering wheel rim permanently illuminate in turquoise, the instrument cluster display is reduced, the voice assistant acoustically confirms the handover, and when activated in the vehicle settings, the steering wheel turns inwards towards the instrument cluster. This method is used to validate to the driver that they are no longer responsible for driving and can focus on auxiliary activities such as watching YouTube videos or reading.
There are two views the driver can choose from: An assisted view with a simulated view, which can also be shown in the head-up display, or only having relevant displays such as range, state of charge, ambient temperature, and current vehicle speed.
During Level 3 highly automated driving, drivers cannot manipulate or alter the vehicle’s performance while Personal Pilot is active. Much like Herbie the Love Bug, your BMW decides how fast it goes and how far the distance between vehicles should be. The vehicle can be taken over at any time by direct oversteering via the brake pedal, accelerator pedal, or steering wheel and immediately transfers all control from the vehicle back to the driver. There are four categories of takeover requests:
- Advanced Warning – A non-time-critical takeover indicated by a turquoise steering wheel icon. LEDs in the steering wheel pulsate turquoise, an acoustic signal is issued when driver tiredness is detected, ambient temperature is too low, or it starts to rain. Drivers must take over driving responsibility in minimal status during the takeover period.
- Warning – A promptly required takeover, indicated by a yellow steering wheel icon, steering wheel LEDs pulsating yellow, and an acoustic signal is issued. Driver’s must urgently take over driving responsibility.
- Acute Warning – A quick takeover, indicated by a red steering wheel icon, steering wheel LEDs flashing red, and an acoustic signal is issued. Driver’s must take over driving responsibility immediately.
- Emergency Stop – The final escalation level, indicated by a red warning triangle. The vehicle will come to a standstill in its lane with hazard warning lights switched on.
Constant evaluation of surroundings, technical skills, awareness of potential hazards, safety skills, anticipation of potential dangers, and awareness of other vehicles on the road are all qualities a good driver must possess – but what are the advantages machines have over humans? They do not make mathematical errors and are much more proficient at multitasking. Fatigue, distraction, speeding, and texting while driving – these are all adverse traits many humans exhibit while driving, which ultimately leads to fully preventable accidents and deaths.
BMW’s Level 3 highly automated driving technology is technologically underpinned by a complex new software stack, a compelling computing platform and a 5G connection to the BMW Cloud. I understand how petrifying it can be to imagine placing the safety of yourself as well as the occupants of your vehicle under the control of a machine, but what many people do not realize is that every time we board a modern commercial airliner, approximately 90% of the flight is controlled via the aircraft’s autopilot system. This highly sophisticated automation system can manage an aircraft’s hydraulic, mechanical and electronic systems. Pilots monitor the system’s operation, intervene when necessary and are primarily responsible for takeoff and landing procedures. Although there are fewer obstacles to avoid in the sky than on the ground, this clearly demonstrates that automation technology can safely perform complex tasks with exceptional safety.
The NHTSA (National Highway Transportation Safety Administration) regulates motor vehicle safety. At the same time, the FAA (Federal Aviation Administration) is responsible for civil aviation regulation in the United States and surrounding international waters. In my opinion, the major limiting factor for worldwide implementation of Level 3 and higher autonomous driving is that regulations vary drastically depending on the country and legal system. Emulation of the FAA’s approach to regulatory matters would allow worldwide regulations covering all aspects of autonomous driving to be universally implemented throughout the world and not limited to a specific country adopting their existing traffic laws and regulations or developing new ones.
The Ethics Commission of the German Federal Ministry for Digital and Transport Infrastructure developed a report describing the ethical guidelines for automated and connected driving. The following are excerpts from the report:
- The licensing of automated systems is not justifiable unless it promises to produce at least a diminution in harm compared with human driving, in other words, a positive balance of risks.
- The protection of individuals takes precedence over all other utilitarian considerations.
- In unavoidable accident situations, any distinction based on personal features (age, gender, physical or mental constitution) is strictly prohibited.
If vehicle manufacturers, governmental authorities, technology companies, and society shared the same goal of achieving a unified legal framework and regulations regarding the full potential of autonomous driving, we would see a dramatic change in implementing this revolutionary technology. —Dylan Smith
[Photos courtesy of BMW.]