|
(11) | EP 3 618 013 A1 |
| (12) | EUROPEAN PATENT APPLICATION |
|
|
|
|
||||||||||||||||||||||||
| (54) | SYSTEM FOR GENERATING VEHICLE SENSOR DATA |
| (57) The invention relates to a system (100) for generating vehicle sensor data. The system
(100) comprises a sensor simulation model unit (104), wherein the sensor simulation
model unit (104) comprises the sensor simulation models (106, 108, 110), wherein each
of the simulation models (106, 108, 110) simulates a sensor. The system (100) is configured
to receive synthetic vehicle ground track data over an input interface (102) of the
sensor simulation model unit (104) . The system senses the objects by applying a sensor
model (106, 108, 110) of at least one sensor to the scenario. The sensor model (106,
108, 110) comprises degradation parameters for generating degraded sensor data and
provides the degraded sensor data to an output interface (112) of the sensor simulation
model unit (104).
|
[0002] In automated or autonomous driving, the perception of the environment is an imperative task. The task is performed through sensors mounted onto an automated driving vehicle that are looking into different directions. The signals received from the sensors are processed and fused. This requires development of very complex algorithms. Testing and validation of those algorithms are important steps in proving the safety of automated driving. Traditionally, test data are recorded during test drives, then labelled and finally used as reference for testing and validation. Drawbacks of this approach are
- (1) specific situations cannot be recreated by doing test drives,
- (2) all the failures and erroneous behaviors cannot be recreated by doing test drives. These drawbacks can be overcome by generating test data using simulations.
[0003] Simulation of the sensor aids in synthetically generating environment data. This reduces the effort and time required to do test drive and collect the recorded data.
[0005] The problem is solved by the subject-matter of the independent claims. Embodiments are provided by the dependent claims, the following description and the accompanying figures.
[0006] According to an aspect, a system for generating vehicle sensor data is provided, comprising a sensor simulation model unit, wherein the sensor simulation model unit is configured receiving synthetic vehicle ground track data comprising objects over an input interface of the sensor simulation model unit, sensing the objects by applying a sensor model of at least one sensor, wherein the sensor model comprises degradation parameters for generating degraded sensor data; and providing the degraded sensor data to an output interface of the sensor simulation model unit.
[0007] As is understood by the skilled reader, synthetic vehicle ground track data means artificial data in the sense that that data are created by a simulation. Preferably, such data relates to a composed scenario resulting from a simulation of a vehicle including its motion and position and its environment. As an alternative to motion and position the track of the vehicle may be used. Also real data of a recorded track, e.g., a test drive may be included. The output of the simulation thus is synthetic data composed of object data or graphical files. The image of the graphical file may comprise, e.g., 3-D modelling data of objects in the surrounding of the vehicle, graphically created signs, etc. The synthetic data is provided to the input, i.e. an input interface, of the sensor simulation model unit. The data may further comprise information about the trajectory of a vehicle driving on, e.g., a road, comprising, e.g., its time dependent position and derivations thereof. The data my additionally comprise information of the road scenario with traffic signs, surrounding vehicles, persons or other vehicles crossing the road, sudden events, consistence of the road including type of asphalt, holes etc. Furthermore, the data may comprise environmental conditions as time of day, azimuth and elevation of the sun, weather and weather related road conditions, etc.
[0008] The sensor simulation model unit may comprise several subunits, each of which may represent a sensor of a vehicle. The sensors to be simulated may be cameras, RADAR sensors, LIDAR sensors, position and navigation sensors, etc. The synthetic vehicle ground track data received by the sensor simulation model unit is suitable to be "sensed" by the simulated sensors. E.g., graphical data is sensed by a simulated camera, and data of objects along the road may be sensed by a simulated LIDAR or radar sensor.
[0009] The simulated sensors engaged in a configured scenario sense the objects as, e.g., obstacles on the road, other vehicles, etc., or images provided at the input. The synthetic vehicle ground track data may be ideally simulated or already degraded. E.g., a graphical image may be degraded by fog or rain. E.g., the size of a detected target object by the radar sensor may be reduced due to occlusion by another object between the radar sensor and the target object. However, instead of just delivering the result of the sensing according to the sensor specification, the simulated sensors themselves can be configured to degrade the sensed data. For that, the sensor model comprises degradation parameters.
[0010] According to an embodiment, the degradation parameters comprise a set of environment degradation parameters.
[0011] Therefore, the sensor model is extended in respect to the sensor specification parameters with regard to several parameters relating to the environment as, e.g., snow, rain, day light, etc. Depending on the type of sensor, there may be implemented a specific physical model simulating the effect on the specific sensor.
[0012] The set of parameters allows, e.g., to take into account environmental conditions, as, e.g., fog or rain, on one side, that degrade, e.g., the visibility of objects on an image, and on the other side may take into account effects of these environmental conditions on the hardware, which would, e.g., not be visible on an image, as, e.g., fogged lenses. Further parameters may be provided to switch on/off the data transmission, to toggle bits on the data to be transmitted, to change measured values by adding a constant, a slope or random noise in order to simulate a total outage or failures of sensor.
[0013] According to an embodiment, the degradation parameters comprise a set of position parameters of a sensed position and/or size parameters of the detected size of a sensed object.
[0014] This means that the sensor provides itself position or size information of, e.g., an object to be sensed. The position may be, e.g., a relative position, a distance, or a derivation thereof of an object. However, it may also be a position provided by another sensor, as, e.g., a GNSS sensor or an inertial sensor.
[0015] According to an embodiment, the sensor simulation model unit is configured to associate data of at least one further sensor to the sensor.
[0016] There may be an interest to associate, e.g., raw position data or position data using a specific filtering different from the filtering of the general vehicle position. E.g., there may exist vibrations or oscillations to be detected, which would be filtered out by the position filter of the vehicle positioning system. The term "positioning" comprises here also the derivations as, e.g., velocity and acceleration.
[0017] According to an embodiment, the degradation parameters comprise a set of measurement noise parameters. Therefore, the sensor model is extended by parameters that add, e.g., Gaussian, white or colored noise or noise correlated to other parameters, as, e.g., the current light intensity or the velocity of the car.
[0018] According to an embodiment, the sensor model unit is configured to model a radio detection and ranging (RADAR) sensor and/or a light detection and ranging (LIDAR) sensor, a camera sensor, or further sensor types.
[0019] Further sensor types are positioning sensors, cameras, etc. The sensors are thus capable of detecting objects to be identified, distances or derivations of distances of objects, road conditions, traffic signs, road markings, relative and absolute position information, etc.
[0020] According to an aspect, a method for generating vehicle sensor data is provided, comprising the steps: receiving synthetic vehicle ground track data comprising objects over an input interface of the sensor simulation model unit, sensing the objects by applying a sensor model of at least one sensor, wherein the sensor model comprises degradation parameters for generating degraded sensor data; and providing the degraded sensor data to an output interface of the sensor simulation model unit.
[0021] The method reflects the functionality of the units of the system described above in the order of the dataflow, which is first receiving the synthetic vehicle ground track data at the input interface. Then, the data is processed by sensing the objects and graphical information comprised in the data and applying sensor models comprising degradation parameters in order to simulate degraded sensors. The sensors may be degraded by external degradation factors, as, e.g., environmental conditions or internal degradation factors as, e.g., failure or outage of a sensor. The degraded sensor data is finally output at the output interface. From there it may be transmitted for evaluation, validation and analysis to further processing modules.
[0022] According to an aspect, a program element is provided, which when being executed by the processor of a sensor simulation model unit, instructs a system for generating vehicle sensor data to perform the steps of the method explained above.
[0023] According to an aspect, a computer readable medium is provided on which the above-mentioned program element is stored.
[0024] The invention is explained in more detail with reference to the accompanying figures and the following description.
- Fig. 1
- shows a system according to an embodiment.
- Fig. 2
- shows a sensor simulation model unit according to an embodiment.
- Fig. 3
- shows a processing chain according to an embodiment.
- Fig. 4
- shows a method according to an embodiment.
[0025] The extension of automotive RADAR sensors from comfort to safety systems may further require intelligent features, such as detection of weather phenomena and performance controlling during adverse weather conditions. Hence, there is a need to examine the effects of different weather conditions like rain, snow, fog etc. on the millimeter RADAR and how these scenarios can be simulated.
[0026] In recent years vehicle RADAR and LIDAR sensors are widely used as sources of control signals for functionality of ADAS. These devices are operating in the millimeter wave range and infrared range in which their performance may be degraded by adverse weather conditions. Currently available information regarding the signal interaction with fog, rain, snow and road spray make clear that millimeter-wave RADAR sensors are far less affected by adverse weather conditions than infrared based sensors. However, when automotive RADAR sensors are particularly designed for safety-oriented systems, the effects of critical issues (such as rain, fog and snow) on the sensor performance becomes of the uttermost importance. For that, system and method is provided capable to cover systematically these critical issues in tests.
[0027] Fig. 1 shows a system 100 for generating vehicle sensor data. The system comprises a sensor simulation model unit 104, wherein the sensor simulation model unit 104 comprises the sensor simulation models 106, 108, 110 where each of the simulation models 106, 108, 110 simulates a sensor of a vehicle. The system 100 is configured to receive synthetic vehicle ground track data over an input interface 102 of the sensor simulation model unit 104 and applies a sensor model 106, 108, 110 of at least one sensor to the scenario. The sensor model 106, 108, 110 comprises degradation parameters for generating degraded sensor data and provides the degraded sensor data to an output interface 112 of the sensor simulation model unit 104.
[0028] Important factors that have to be considered while simulating RADAR sensors are, e.g., factors that result in degradation. Examples are environmental conditions and degradation due to sensor noise.
[0029] The environment scenario around an ego vehicle could lead to different types of degradation. The important scenarios of environment degradation are e.g., weather condition, occlusion and ghost target detection.
[0030] The degradation may result also from sensor noise. The sensor is an electronic device, which may receive signals reflected from the obstacles. These signals are processed to detect and estimate object attributes. The noise in the receiver will lead to an error in the prediction of the position and orientation. The main degradations to be considered are measurement errors, detection errors, position and size of the object detection errors due to occlusion and sensor failures.
[0031] Fig. 2 shows an embodiment of a sensor simulation model unit 202 for a sensor, which may be one of the models 106, 108, 110. In this exemplary embodiment, object parameters 204 are provided as input of the simulation model 202. The objects may represent persons on or along the road, other vehicles, buildings, road markings, traffic signs, etc. The input comprises further graphical data, as image files. The objects and images are sensed by the sensor simulation model unit 202, and several types of degradations are applied in the sensor degradation units 208, 210, 212. In this example, unit 208 may account for occlusion or other weather conditions, unit 210 may account for the position and size degradation and unit 212 for measurement errors. The degraded data is provided to the output 206, where it may be stored in, e.g., a memory or data base for further processing or evaluation.
[0032] Fig. 3 shows a processing chain according to an embodiment. In this example, a synthetic scenario generator tool 302 generates data 304 for real scenarios, which are acting as a ground track input data for the sensor simulation model unit 306. In the sensor simulation model unit 306 certain real world degradation aspects can be modelled like occlusion in module 208 of Fig. 2, position and size degradation in module 210, and measurement degradation in module 212, ghost object formation etc.
[0033] The output of this model is degraded data 308, which could be used by the Advanced Driver Assistance Systems (ADAS) algorithms such as Adaptive Cruise Control, Emergency Brake Assist, Lane Control functions etc. that would help in validating the ADAS functions/algorithms in a very precise and logical manner.
[0034] The degraded data 308 is finally provided to the analysis, evaluation and validation units 310, 312.
[0035] Fig. 4 shows a method for generating vehicle sensor data according to an embodiment. In 402 synthetic vehicle ground track data is received over a first interface, which may serve as an input interface. Vehicle ground track data may be data of a vehicle that is moving in a scenario. The trajectories, objects etc. in the scenario may be generated by a simulator or recorded during a real drive as, e.g., a test drive. The data is supplied to the simulation sensor unit that comprises sensor models of at least one sensor. The simulated sensors sense the objects provided at the input and apply the degradation parameters as described above. The degraded sensor data is finally provided at second interface which may serve as an output interface.
1. System (100) for generating vehicle sensor data, comprising a sensor simulation model
unit (104), wherein the sensor simulation model unit (104) is configured for
- receiving synthetic vehicle ground track data comprising objects over an input interface of the sensor simulation model unit,
- sensing the objects by applying a sensor model of at least one sensor, wherein the sensor model comprises degradation parameters for generating degraded sensor data; and
- providing the degraded sensor data to an output interface of the sensor simulation model unit (104).
2. System (100) according to claim 1, wherein the degradation parameters comprise a set
of environment degradation parameters.
3. System (100) according to one of the previous claims, wherein the degradation parameters
comprise a set of position parameters of a sensed position and/or size parameters
of a detected size of a sensed object.
4. System (100) according to one of the previous claims, wherein the sensor simulation
model unit (104) is configured to associate data of at least one further sensor to
the sensor.
5. System (100) according to one of the previous claims, wherein the degradation parameters
comprise a set of measurement noise parameters.
6. System (100) according to one of the previous claims, wherein the sensor model unit
is configured to model a radar sensor, a LIDAR sensor, and/or a camera sensor.
7. Method for generating vehicle sensor data, comprising the steps:
- receiving (402) synthetic vehicle ground track data comprising objects over an input interface of the sensor simulation model unit,
- sensing the objects (404) by applying a sensor model of at least one sensor, wherein the sensor model comprises degradation parameters for generating degraded sensor data; and
- providing (406) the degraded sensor data to an output interface of the sensor simulation model unit.
8. Program element, which when being executed by a processor of a sensor simulation model
unit (202), instructs a system (100) for generating vehicle sensor data (304) to perform
the steps (402, 406, 408) of claim 7.
9. Computer readable medium on which a program element according to claim 8 is stored.