Huawei produces "L4" ahead of time.

SMM News: secret research and development for 6 years, team size of more than 2000 people! Huawei finally showed its trump card in the field of self-driving.

The "advanced" self-driving program, called ADS, is about to launch a production car on Q1 in 2022, allowing consumers to drive on autopilot from "their own garage" to "company garage".

You read it right, that is, all the daily commuting driving tasks are left to the car-this is the function of the L4 autopilot system.

If mass production can be achieved, ADS will directly crush all existing L2 and futures L3 systems, which is a typical "high-dimensional and low-dimensional" approach. I have to say, Huawei's shot really carries a lot of weight.

At the Beijing Auto Show just past, the ADS program was briefly presented in the form of PPT, but key issues such as what functions, how to divide driving responsibilities and what technical details are behind are not disclosed.

Su Qing, President of Huawei Intelligent car BU Intelligent driving Product Line

Immediately after the National Day holiday, car Dongdong had an exclusive conversation with Su Qing, president of Huawei's smart car BU ADS intelligent driving product line, to get a glimpse of the whole picture, technical details and R & D process of ADS in advance.

All commuter tasks are assigned to ADS L4 as L2.

The full name of ADS is Autonomous Driving Solution, and the official Chinese name is Huawei Advanced Autopilot system.

Understanding ADS, can start from two parts: function and responsibility.

The first is the function, ADS is to achieve continuous self-driving experience on the whole route from "community garage" to "company garage".

It means that from the moment you get on the bus in your garage, the vehicle is responsible for all the driving operations-- driving out of the basement, on the city road, on the ring road, on and off the ring road, into the city road, into the company basement.

The existing L2 systems, including the futures L3 just released by Mercedes-Benz, are available in some areas, such as ring roads (expressways), lanes with clear lanes, red lights such as roads without lanes, intersections and other red lights (Tesla FSD can wait for red lights on their own), or garages and other scenes can not be used.

ADS city autopilot test video

At the same time, the L2 system requires human drivers to give instructions and confirm road conditions when changing lanes, on and off ramps, or passing traffic lights (such as Tesla Navigate on Autopilot and FSD). The whole experience is not continuous.

ADS solves the above two problems: one is to achieve the full coverage of the, ODD (running area of the above commuter scenarios) far exceeding the existing L2 and futures L3. Second, the operation of lane change, off-ramp and other traffic lights are completed by vehicles.

So functionally, ADS is peer-to-peer L4 autopilot-which is why Huawei calls it a high-level autopilot solution.

Of course, the technical architecture of ADS also comes from the L4 autopilot system, which will be discussed in more detail below.

The second key to understanding ADS is the division of responsibilities.

According to the standards of SAE or the Ministry of Industry and Information Technology, Huawei ADS belongs to the L2 autopilot system-the system provides assistance and the driver bears the driving responsibility.

ADS autopilot test video on expressway

Su Qing, president of Huawei's ADS intelligent driving product line, told the car that when using ADS, the driver's vertical and horizontal control is completed by the vehicle, but the driver needs to pay attention to the road conditions and be ready to take over at any time.

"ADS also allows drivers to let go after the driver's attention monitoring system is installed, but it is up to the car company to define how long it is allowed to let go." Suqing said.

It's actually very easy to understand ADS here-sell consumers a passenger car with L4 self-driving capabilities, but the responsibilities are divided according to L2 self-driving settings.

Let the driver become a safety officer, the whole monitoring system runs, and the vehicle can be controlled at any time.

Rainy days challenge the streets of Shanghai to deal with Chinese-style traffic calmly

After all that has been said, how is ADS performing? You can see the difference through a video of the road test provided by Huawei.

Huawei ADS test video

The video was filmed on the streets of Shanghai, and you can see moderate to heavy rain and even torrential rain that day. Rain Water has sometimes even blurred the camera, which can easily affect cameras, lidar and other sensors, and there are many social vehicles on the road. The road is very complicated.

When passing through the traffic light intersection, the perceptual system of the vehicle recognized the green light and decided to move on. There is no lane on the ground at the intersection, but with the help of a high-precision map, the vehicle follows the planned path safely.

The ADS test car accurately identified eight traffic lights at the intersection.

Then the traffic came to a more complicated intersection and was ready to turn left.

The left viaduct will affect the positioning accuracy of the vehicle, while there are eight traffic lights ahead, which is also a big challenge for the autopilot system.

From the video picture, the ADS system accurately identified all the traffic lights and knew to enter the left-turn waiting area after the straight green light came on. Then the green light came on and the test car completed a left turn.

On a narrow path full of other vehicles on the right, an electric bike appears in this lane and in the opposite lane, and both targets in the video are accurately identified, tracked and displayed in the generated 3D scene.

Identification of Electric vehicle by ADS system

The traffic continued on the path when a human tricycle suddenly crossed the road. If it is not identified, there will be a collision risk.

The ADS system accurately identified this goal, and the vehicle chose to slow down and accelerate after the tricycle had completed its steering.

ADS test car avoids human tricycle

On China's urban roads, electric bicycles, human tricycles, and even horse-drawn carriages and donkey carts may appear, as well as retrograde, jaywalking, jaywalking and other behaviors that do not obey traffic rules.

These current situations will make mature systems such as Tesla Autopilot unaccustomed-- unable to identify a wide variety of traffic participants, decision-making algorithms do not take into account the behavior of non-compliance with traffic rules, and so on.

Then the test car drove on a circular ramp, turned a small circle and merged into the traffic flow of the expressway.

The expressway is similar to the highway, although it looks simple-go straight ahead, but the difficulty lies in two aspects: overtaking by changing lanes and dealing with congestion.

When the vehicle was about to pass the Shanghai landmark Nanpu Bridge, the vehicle detected that the speed of the black car in front was slow. At the same time, after the left middle lane was idle, it actively changed to the left and overtook the slow car in front, keeping driving in the middle lane.

In the middle of the bridge, a BMW 5-Series station wagon quickly overtook the test car and then merged into the front of the test car at a short distance-this is the Cut in scene that the L2 self-driving system fears most.

ADS test car avoids jammed vehicles

In the face of this situation, the L2 system either suddenly slams on the brakes and affects the driving experience. Or continue to drive indifferently, resulting in the driver being forced to take over the vehicle urgently.

In the video picture, the ADS test car decelerated from 60 km / h to 55 km / h, allowing the 5-series travel version to complete and merge. After it is far away, it returns to a speed of 60 km / h.

This is a typical case where the L4-level autopilot technology architecture reduces the dimension of the L2-level system-whether it is the sensor or the software algorithm, it is the existence of the "rolling" level.

Su Qing told car that Huawei's ADS system is currently being tested in China, specifically optimizing scenarios such as electric cars, Cut in, inbound traffic and overtaking that are common on Chinese roads, in order to provide Chinese consumers with the most appropriate self-driving experience.

Judging from the 4-minute video of the road test, Huawei ADS performed quite well.

Using bicycle intelligence to realize AVP fleet learning and broadening use scenario

In the previous part, ADS is actually equivalent to using an L4 system as a L2 system. But ADS does have a real L4 function-AVP automatic valet parking.

At both ends of the above commuting scenario are the garage of the user community and the company garage, and the goal of ADS is to achieve the AVP function in these two parking lots. "users only need to drive into the warehouse once, the system can learn the parking path and method of this parking lot." Suqing said, "the next time you go to the garage, the parking process can be done by the vehicle itself."

When users park manually for the first time, they can customize where to get off-for example, at the entrance before entering the underground garage, or somewhere after entering the basement.

Suqing stressed that technically speaking, AVP can allow drivers to get out of the car and use it-a L4 autopilot process.

However, because China's law has not yet stipulated whether the AVP function can be used in public parking lots, Suqing said, "Huawei very much does not recommend users to leave the vehicle to use the AVP function, but more to reduce the driving burden on users."

All the functions of ADS, including AVP, rely entirely on bicycle intelligence, so the AVP function can theoretically be used in all suitable parking lots. Coupled with the team learning function, it will be more and more convenient for users to use AVP.

The AVP system can share parking information. For example, A car has been to B parking lot, A car will automatically build a 3D map of B parking lot, and send 3D map, surrounding environment and other information back to the cloud, and then distribute it to other vehicles through OTA.

One day, when the C car wants to go to the B parking lot, you can use the AVP function directly. As more and more vehicles use the AVP function, more and more parking lots will be supported, eventually becoming a "general" function.

How to implement ADS? Using L4 autopilot technology to build

In Suqing's view, there have been many L2 autopilot systems in the past two years, but most of the functions have strict application scope and restrictions, which can not be used in many road conditions and traffic conditions. nor can it meet the commuting needs of Chinese consumers based on urban road conditions.

Seeing this pain point, Huawei decided to develop an ADS system to solve commuting problems directly. However, commuting involves a variety of scenarios, especially in urban roads, and the system complexity increases exponentially.

"so ADS uses the L4 autopilot technology architecture," Suqing said. "it can't be used without it."

ADS technical architecture

In terms of hardware configuration, the ADS scheme will use 2-3 vehicle gauge-class 100-line mixed solid-state lidar, as well as a dozen cameras and 6 millimeter wave radars, which can be said to be armed to the teeth and equipped with no less than L4 unmanned taxis.

The computing hub is a domain controller called ADCSC (Autonomous Driving Central Super Computer), which is full of computing power.

In terms of software, Huawei uses a variety of AI technologies to directly fuse the point clouds generated by the self-developed millimeter wave radar and lidar, as well as camera video images at the pixel level (that is, pre-fusion) to ensure the perception ability. Previously, some self-driving companies mostly used the fusion of lidar and camera, while millimeter wave radar directly fused the output target with the perceptual results of the first two.

Huawei has independently developed millimeter wave radar, so it can get the most original point cloud data of millimeter wave radar, and carry out pixel pre-fusion and structured data fusion of the three sensors at the same time, which is a step forward in technology.

Some of the sensors used by ADS

ADS wants to achieve autopilot in the city (L4 function, L2 responsibility division), that is to say, it should be able to handle traffic lights, intersections and other scenes, and avoid pedestrians, bicycles, tricycles, takeout brothers and other traffic participants.

This means that the rule-based algorithm used in the decision-making part of the traditional L2-level system is powerless, so it is necessary to introduce AI technology into the decision-making part.

According to Suqing, Huawei delineated different cells with rules as the framework in the decision-making part, and then further introduced machine learning technology in each grid.

"the algorithm of pure AI is uncontrollable, and only by combining the rule algorithm with AI technology can we take into account the effect and security." Suqing said.

Of course, the ADS scheme also has a high-precision mapping system necessary for high-level autopilot, which is also the key to the realization of full-length commuter autopilot.

The motorcade learns to solve map and data problems.

In the face of complex physical world scenes, the function of autopilot is inseparable from high-precision maps. however, the use of high-precision maps will bring two problems: areas without map data can not use autopilot, and high-precision map data is difficult to update in real time, which affects the autopilot system.

In response, Suqing said Huawei's ADS autopilot team already has a solution.

First of all, the full commuting function of ADS will be made available to users city by city according to the coverage of the map. For example, priority should be given to opening up first-tier cities, and then gradually covering second-and third-tier cities.

It is worth mentioning that Huawei itself also has Class A map surveying and mapping qualifications and map team, itself has high-precision map production capabilities. At the same time, Huawei has also created a mapping platform and hopes to work with other partners to speed up the production of high-precision maps.

Secondly, in the relatively simple highway (expressway) and parking conditions in the scene, you can use the functions of ADS without high-precision map, such as automatic car following / automatic overtaking, AVP and so on.

According to Suqing, this setting maximizes the expansion of ADS's ODD-- to use commuter autopilot if there is a high-precision map, and to use autopilot where there is no map.

Third, the team learning function can help update high-precision maps.

Vehicles equipped with ADS have many sensors themselves, and at least two high-beam lidars and cameras can be used to collect road change data during daily driving.

When there are more and more ADS vehicles and more mileage, you can speed up the update frequency of high-precision maps.

ADS can generate its own surrounding map.

"the base map of the high-precision map still needs a professional collection vehicle to complete, and the ADS vehicle is only responsible for collecting and updating the changing part of the data." Suqing explained that ADS vehicles will also build their own road spectrum while driving, and when they encounter inconsistencies in real-time perceived road conditions, high-precision maps, and self-built road spectra, they will calculate the confidence of the three, thus determining the vehicle behavior. If the vehicle encounters an extreme situation that cannot be dealt with, it will maintain a certain route and call the driver to take over at the same time.

In addition to sharing AVP and map information, another key use of ADS's fleet learning function is to collect driving data, which can be used to train the AI model in the perception and decision-making system, and ultimately improve system performance.

Tesla's Autopilot system has a similar setting, called shadow mode.

In the past few years, Tesla has sold more than a million models, and the Autopilot system has a mileage of more than 3 billion miles (4.8 billion kilometers). The data collected by the team continue to provide "nourishment" for the iteration of the Autopilot system, which makes Autopilot the strongest L2 today.

Su Qing told the car that Huawei ADS's fleet learning model will collect a variety of data and send it back to the cloud. In addition to the previously mentioned road environment information, the system will also send the relevant data back to the cloud for improvement when the driver takes over or runs uncomfortably (such as slamming on the brakes).

Self-driving vehicle has many sensors, if the data sent back is too much and too large, it is not convenient for practical operation. In order to solve this problem, after collecting the target data, the ADS system will first preprocess it locally, simplify it into structured data, and finally send it back.

Secret R & D team size exceeds 2000 in 6 years, commercial use the year after next.

Starting in 2019, Huawei sporadically revealed some of its work and layout in the automotive field. It was only known that Huawei was developing an autopilot system, but did not know the technical details and how to play.

Only a year later, the ADS plan was unveiled to the public at the 2020 Beijing Auto Show. And for the first time, it gave the important news that "L4 should be used as L2" and "realize autopilot for the whole section of the commuter road". There was a sense of being born out of nowhere.

"regardless of the time that the previous technology has been accumulated, only from the direct research and development of self-driving technology, the related research and development of ADS has a history of five or six years." Su Qing said with a smile.

According to him, Huawei began secretly developing self-driving technology around 2014, and the size of the team has ballooned from 100 to 200 at the beginning to more than 2000 today.

Shanghai office area of Huawei autopilot team

Even globally, this is one of the largest self-driving teams in the world, and only large groups such as Baidu, Google Waymo, GM Cruise and Uber can afford such a team.

In terms of team composition, Su Qing said that the 2000-member team includes not only self-driving talents from the automobile industry, but also a large number of PhDs from prestigious universities, as well as personnel from Huawei's internal mature product line.

Suqing, for example, is a 20-year veteran of Huawei, the founder of Huawei Kirin chips and solutions, the co-founder of Haisten chips and solutions, and began to lead the development of ADS systems in 2014.

The development of high-level autopilot system relies heavily on actual road test data.

One of the key reasons why Google Waymo leads the world in technology is that it has the largest road test fleet in the world and accumulates the most road test data-more than 20 million kilometers.

Suqing did not disclose how much mileage Huawei has accumulated, saying only that 400 to 500 vehicles are currently conducting road tests in China (half of the vehicles are owned by car companies). In terms of domestic, the actual mileage is "the existence of TOP."

At present, the longest mileage of domestic self-driving companies is Baidu, with a total of 6 million kilometers. From this point of view, Huawei's autopilot mileage will not be less than 6 million kilometers.

ADS has achieved good results in mass production.

Suqing revealed that ADS has won the appointment of a number of car companies, in the first quarter of 2022, a number of models will be launched with ADS program, cars and SUV are covered, and are mainly pure electric models.

"there are a variety of car brands here." Su Qing said confidently, "We are not futures. Users can buy a car directly in the first quarter of 2022 and use ADS when they buy it."

The ADS system uses at least two high-beam lidars, multiple millimeter wave radars and more than a dozen cameras. Does this luxury configuration, which directly challenges unmanned taxis, make the scheme extremely expensive?

In response, Suqing replied that the price of the ADS scheme is medium-level, mainly aimed at models with a price of more than 200000.

Now the ordinary L2 autopilot system is equipped with models with a price of about 100000. The two futures L3 systems that have been released need to be equipped on high-end car brands with a starting price of nearly one million.

By contrast, the ADS system is installed on more than 200000 models, which is also medium-level.

Conclusion: Huawei provides a new idea of autopilot mass production.

After years of development, the global self-driving industry is facing an awkward situation.

Some breakthroughs have been made in the high-level self-driving route represented by Google Waymo, Baidu Apollo and other technology companies, but it is still a long way from the large-scale deployment of unmanned taxis and the realization of technology.

On the gradual route represented by the traditional vehicle factory, the L2-level autopilot system has quickly achieved popularity, but the L3-level autopilot that goes further has become a "technology black hole". So far, no company has been able to achieve mass production.

In this context, Huawei ADS's "L4 when L2" train of thought is to provide the industry with a very good "curve to save the nation" method.

On the one hand, the "commuter autopilot" function setting, so that the passenger car autopilot system is no longer a decoration, can play a role in daily driving, but also helps car companies to break through the L2 self-driving ceiling.

On the other hand, since it is impossible to ensure the absolute safety of all the Corner Case systems, we will use the L4 system in terms of driving responsibility according to the L2 system-the driver monitors the road condition in the whole process, but the vertical and horizontal control is completed by the vehicle, so that the autopilot system has the possibility of mass production, and the L4 autopilot system is produced ahead of schedule.

To some extent, the vehicle carrying the ADS is an L4 class unmanned car, and the driver becomes a safety guard.

More importantly, once mass production is achieved, hundreds of thousands or millions of convoys will collect large amounts of data every day and return it to Huawei to help finally break through the extreme scenario and let mankind truly enter the era of self-driving where driving can be done by Rest.