(19)
(11)EP 3 626 585 A1

(12)EUROPEAN PATENT APPLICATION

(43)Date of publication:
25.03.2020 Bulletin 2020/13

(21)Application number: 18196064.2

(22)Date of filing:  21.09.2018
(51)International Patent Classification (IPC): 
B62D 15/02(2006.01)
G01C 21/30(2006.01)
G08G 1/16(2006.01)
B60W 30/08(2012.01)
G06K 9/00(2006.01)
(84)Designated Contracting States:
AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR
Designated Extension States:
BA ME
Designated Validation States:
KH MA MD TN

(71)Applicant: Visteon Global Technologies, Inc.
Van Buren Township, MI 48111-5711 (US)

(72)Inventor:
  • PFEIFLE, Martin
    72297 Seewald (DE)

(74)Representative: MERH-IP Matias Erny Reichl Hoffmann Patentanwälte PartG mbB 
Paul-Heyse-Strasse 29
80336 München
80336 München (DE)

 
Remarks:
Amended claims in accordance with Rule 137(2) EPC.
 


(54)DRIVER ASSISTANCE SYSTEM AND METHOD FOR DISPLAYING TRAFFIC INFORMATION


(57) A driver assistance system and a method for providing traffic information to a driver of a ego vehicle. The driver assistance system comprises at least one sensor for detecting objects, including other vehicles, in a surrounding area of the ego vehicle and for providing image data of the detected objects, a digital map database providing road data, a position detection device for detecting a position of the ego vehicle on the road, and a display unit for rendering visual information relating to an environmental model of the surrounding area of the ego vehicle. The rendered visual information may be based on data from the at least one senor, including image data, a detected position of the ego vehicle, and the digital map data. The system may further comprise an image analysis unit configured to determine a vehicle characteristic of at least one other vehicle in the surrounding area of the ego vehicle from a vehicle image of the at least one other vehicle generated on the basis of the data obtained by the at least one sensor and to display information in dependence on the determined vehicle characteristic.




Description

TECHNICAL FIELD



[0001] One or more embodiments described in the present application relate to a driver assistance system for providing traffic information to a driver of a vehicle and to a method for displaying traffic information to the driver of a vehicle.

[0002] In particular, one or more embodiments described herein relate to aspects of a system and a method configured for detection and recognition of objects in the surrounding area of a vehicle. Based on the result of detection and object recognition traffic information that may be relevant for guidance and safety aspects is concluded and output.

[0003] Embodiments of the system and the method described herein may be typically embodied by an advanced driver assistance system (ADAS).

BACKGROUND



[0004] In addition to conventional navigation systems, recently an increasing number of automotive vehicles have been equipped with advanced driver-assistance systems, also referred to as ADAS-systems. ADAS-systems are computer-based systems developed to automate, adapt and enhance vehicle systems for increased safety and more comfortable driving. Features of an ADAS system may include, for example, electronic stability control, anti-lock brakes, lane departure warning, adaptive cruise control and traction control.

[0005] ADAS related technology is based on inputs from multiple data sources, including cameras, LiDAR-sensor and radar sensors, image processing, computer vision, and in-car networking. Additional information may be provided from other sources separate from the primary vehicle platform (the "ego vehicle"), such as from other vehicles, referred to as Vehicle-to-vehicle (V2V)-communication, or Vehicle-to-Infrastructure communication, realized through mobile telephony or other wireless networks, like dedicated short-range communications (DSRC).

BRIEF SUMMARY OF THE INVENTION



[0006] One or more embodiments relate to a driver assistance system for providing traffic information to a driver of a vehicle comprising at least one sensor for detecting objects, including other vehicles, in a surrounding area of the vehicle and for providing image data of the detected objects. A plurality of sensors may be provided, including, for example, a stereo camera, a radar sensor, a LiDAR sensor, and an inertial measurement unit (IMU).

[0007] According to this aspect, the sensor(s) may be mounted on board of the ego vehicle but embodiments are conceivable where the sensor is mounted to another vehicle and the respective sensor data is received by the ego vehicle by means of vehicle-to-vehicle (V2V) communication.

[0008] The sensor(s) may be configured to detect any object within a predetermined surrounding range of the ego vehicle, including other vehicles, or static objects, such as traffic lights, traffic indicators, trees, houses, et cetera.

[0009] According to a further aspect, the driver assistance system may comprise a digital map database providing road data, including a road network, and a position detection device for detecting a position of the vehicle on the road, such as a global navigation satellite system (GNSS).

[0010] Based on the detected position of the vehicle, the sensor data and the digital map data, visual information relating to an environmental model of the area where the ego vehicle is currently located may be generated and provided on a display unit. The environmental model may be a simple two-dimensional illustration indicating graphically the current position of the vehicle on a road section as provided by the digital map data on the basis of the detected position of the vehicle. The environmental model may additionally present information regarding the surrounding of the vehicle, including the position of other vehicles, traffic indicators, bridges, trees and any object detected by the sensor in the surrounding area of the vehicle. Using more developed software tools, the environmental model may be displayed as 3D graphic to provide the driver with a realistic view not only from the windscreen of the driver's position in the cockpit of the vehicle but also with a top perspective from the entire surrounding area of the ego vehicle.

[0011] The sensor may be configured to continuously monitor the surrounding area of the vehicle, and a relating "logbook" of the surrounding area of the ego vehicle may be recorded on a relating storage medium in predetermined time periods, as the vehicle is continuously moving along a predetermined travelling route.

[0012] Furthermore, the driver assistance system may comprise an image analysis unit configured to determine a vehicle characteristic of at least one other vehicle in the surrounding area of the ego vehicle from a vehicle image of another vehicle which has been detected by the sensor in the surrounding area of the ego vehicle. The term "vehicle characteristic" covers inherent permanent characteristics and properties of a vehicle as well as dynamically changing features like a condition or state of the vehicle or a component thereof. The "vehicle characteristic" may be directly apparent from the outer appearance of the vehicle or may be concluded from the outer appearance of the vehicle.

[0013] Based on the vehicle characteristic determined by the image analysis unit, a relating information may be output on the display unit, for example, or by means of an audio unit. Such information may include an alert if the occurrence of a dangerous situation is concluded by the system based on the determined characteristic of the vehicle and the traffic situation.

[0014] For example, the system may issue an alert in a situation where it is determined that a braked light of a vehicle in front of the ego vehicle is in an ON-state and that the distance of that vehicle in front to the ego vehicle is lower than predetermined threshold value. This example illustrates that a critical distance to another vehicle may be determined not only from a measured distance to the vehicle in front of the ego vehicle, but also from a characteristic determined by image analysis from the outer contour of the other vehicle, for example.

[0015] One or more embodiments of the driver assistance system include configurations of the image analysis unit to determine a vehicle characteristic relating to a current driving state of the other vehicle. Such a vehicle characteristic may, for example, be a brake light state and/or a turning light state and/or a driving light state of the other vehicle and/or a body-component state, such as tailgate that is open or closed, a trunk lid that is open or closed, a passenger door that is open or closed, a convertible top that is open or closed, a removable panel that is in place or removed, such as a removable roof panel.

[0016] As an example of such an embodiment, the driver of the vehicle may receive an alert if it is determined by the system that driving light of a vehicle in the surrounding area of the legal vehicle is not turned ON during night time.

[0017] According to a further aspect, the image analysis unit may be configured to determine from the sensor data a vehicle characteristic depending on a vehicle brand, a model type and/or a colour of the other vehicle by comparison with 3D model data stored in a vehicle model database, for example.

[0018] After a certain model of the vehicle has been identified by the system a more realistic environment model may be depicted on the display, including a specific graphical representation of the related car model and colour, for example. The rendering of graphical information about a certain vehicle model may also assist the driver's orientation and correlation of the perspective given by the environmental model in relation to his realistic view through the windscreen of the vehicle. Moreover, as a vehicle characteristic of the vehicle changes, e.g., activation of a brake light to deactivation of the brake light, the rendered image may change accordingly. Doing so improves the realistic view of the environmental model.

[0019] In addition, the model type and colour information may be used for guidance purposes of a navigation system including a natural guidance concept in which the driver is prompted to follow a certain vehicle travelling in the same travelling path as the ego vehicle.

[0020] Further, the identification of a car model may be useful to provide safety relevant information. For example, once a certain car model has been identified, the accuracy of image analysis for determining, for example, whether a brake light is turned ON or OFF can be significantly increased because the exact shape and dimension of the brake light may be retrieved from model data stored in a vehicle model data storage device providing 3D data of car models and their components for all major car series. Based on the exact shape and dimension of the brake light of a given vehicle model, an exact shape of the distribution of the brake light in an ON condition may be modelled which may be used in image analysis of the outer contour of the other vehicle to determine whether a brake light is in fact turned ON or OFF.

[0021] Another safety relevant aspect that can be concluded from the model type identification is the determination of potential driving properties of a detected vehicle based on the identification of its model type. As an example for this aspect, it may be determined by the driver assistance system that a 30-year-old car model may have a braking system which only allows for braking with a comparatively high braking delay. This determination may be considered safety-relevant information for other traffic participants in the vicinity of the 30-year-old vehicle.

[0022] One or more embodiments of the driver assistance system may comprise a receiving unit for receiving, from another vehicle, communication data relating to the vehicle characteristic of at least one of the other vehicles in the surrounding area of the ego vehicle. The communication data may include raw sensor data provided by another vehicle and/or a characteristic of the vehicle as determined by another system. In some embodiments, the ego vehicle may be informed by vehicle-to-vehicle communication about a specific brand, model type, or colour of another vehicle in the surrounding area of the ego vehicle, for example.

[0023] The driver assistance system may be embodied by an ADAS system, wherein the sensor may be provided as a camera, a vision sensor and/or a LiDAR sensor. For example, the car model identification can be carried out based on LiDAR point cloud describing the outer shape or contour of the vehicle which may be utilised for classifying vehicle brands or identifying specific models. Additionally or alternatively, the sensor may detect and analyse textual indicia, which specifies a brand or model name, on the vehicle. Doing so may allow the ADAS system to identify the specific brand and model of the vehicle. Additionally or alternatively, the vehicle may provide a vehicle identification number or other identification information to the ego vehicle. The ego vehicle may utilise the vehicle identification number or other identification information to determine the brand, model, colour, year, etc., of the vehicle.

[0024] In one or more embodiments, the display unit may comprise an instrument cluster or a head up display. Based on the model type determination, for example, a specific car model identified by the driver assist system can be displayed in the instrument cluster.

[0025] In one or more embodiments the driver assistance system may be configured to continuously analyze sensor data relating to other vehicles in the surrounding area of the ego vehicle and to continuously determine one or more vehicle characteristic from the analysed data and to store the determined vehicle characteristic together with time and/or location information. In such a manner, a kind of "blackbox" can be provided by the system which allows for reconstruction of not only the travelling path of the ego vehicle but also to reconstruct the surrounding scenario of the vehicle at the time of travelling.

[0026] A broad variety of application scenarios may be conceived by combining different aspects described herein. For example, the above-described blackbox may be utilised in combination with the above described feature of vehicle model determination. Thereby it becomes possible to determine, for example, whether or not a specific vehicle encountered the ego vehicle along its travelling path, which may be helpful for identifying vehicles which are subject of a national or international search, for example.

[0027] The range of available application scenarios of the system described herein is even increased by embodiments in which the driver assistance system and the image analysis unit are configured to detect a static object in the surrounding area of the ego vehicle and to analyse the condition state of that static. This does not only include the detection of constructive obstacles and the like, but may also include dynamically changing object features, like the ON/OFF-state of a traffic light in the surrounding area of the ego vehicle, for example, which may be detected as an object and subsequent image analysis sensor data to reveal its condition state. For example, the traffic light may dynamically change from a red-light state to a green-light state. In an example where the static object is rendered as an image in the environmental model, such as as as a 3-D image, the rendered image may be updated based on changes to the dynamic features. For example, in the case of the traffic light changing from the red-light state to the green-light state, the rendered image may be updated accordingly. Doing so improves the realistic view of the environmental model.

[0028] As an example for such an embodiment, a driver assistance system may be configured to analyse a relationship between a detected position of another vehicle and a detected brake light state of the other vehicle or the relationship between a detected position of the ego vehicle and a detected state of a traffic light and to output an information relating to the result of the analysis of this relationship to the driver.

[0029] In some embodiments, the step of detecting a state/condition of a static object in the surrounding area of the ego vehicle may be carried out in addition to determining the vehicle characteristic of a vehicle in the surrounding area of the ego vehicle, while in other embodiments this step may be carried out instead of the step of determining the vehicle characteristic of the vehicle in the surrounding area of the ego vehicle. As such, one or more embodiments may analyse objects other than vehicles surrounding the ego vehicle., For example, the objects may include traffic signs and traffic indicators, and the object may be detected and analysed regarding their condition states through image data provided by sensor(s) monitoring the surrounding area of the ego vehicle.

[0030] According to another aspect, one or more embodiments provide a method for displaying traffic information to a driver of a vehicle, comprising the steps of providing image data of at least one other vehicle in the surrounding area of the ego vehicle, analysing the received image data, determining a vehicle characteristic from the result of the analysis, and displaying information relating to the determined vehicle characteristic.

[0031] In one or more embodiments the step of analysing the received image data may be carried out using a neural network to classify objects within a given image space. For example, the method may include applying a neural network onto an image generated on the basis of the image data obtained from the sensor, and analysing an image content within a limited area defined by a bounding box inside the image. A model database comprising 3D data of various vehicle series may be utilised to train a neural network to detect models and/or subclasses of cars corresponding to brand names.

[0032] In alternative embodiments, image analysis may be carried out by evaluating pixels inside a bounding box comprising the detected object of interest.One or more embodiments of the method comprise the steps of determining a position of the ego vehicle and/or another vehicle and determining a vehicle characteristic relating to the driving state of the other vehicle, such an embodiment may comprise the additional steps of determining guidance or safety information based on the determined vehicle characteristic and the position of the other vehicle and/or the position of the ego vehicle and outputting the determined guidance of safety information to the driver of the ego vehicle. The safety information may comprise an alert in cases where, for example, a braking process of a vehicle in front of the ego vehicle is ongoing as detected from the vehicle's brake light state, while the guidance information may be to follow a certain car model identified in the surrounding area of the ego vehicle, for example. The guidance information may be a combination of an audible instruction, such as through sound system in the ego vehicle, along with a 3D rendered image on the display unit, which may include a vehicle characteristic, of the car model identified in the surrounding area. Doing so may allow the driver to better understand the guidance information. As an example, the rendered image may improve the driver's confidence in his understanding of the guidance information. If the driver did not hear or did not understand the audible instruction, the driver may still comprehend the guidance information based on the rendered image. If the driver is uncertain about audible instruction, such as in a case where the driver is unfamiliar with model names and/or if there are similarly coloured vehicles, the driver may use the rendered image to improve his understanding. For example, if the audible instruction is to follow the yellow vehicle, and there are multiple yellow vehicles, differing in model type or vehicle characteristic, to choose from, then the driver may look to the rendered image to determine which one is the target vehicle to follow. This may be further enhanced when the rendered image is spatially arranged in the environmental model.

[0033] One or more embodiments further comprise the step of providing image data of a static object within a predefined surrounding range of the ego vehicle, analysing the provided data and outputting safety or guidance information to the driver of the ego vehicle based on the result of the analysis of provided image data and the position of the ego vehicle and/or the position of the other vehicle.

[0034] Examples for such embodiments include the determination of a meaning of a traffic sign or a traffic indicator identified in the surrounding area of the vehicle and outputting an alert to the driver, and, if useful, a driving instruction to avoid occurrence of a critical traffic situation, in cases where the system determines that the driving condition of the ego vehicle is not in accordance with the meaning of the traffic sign or the traffic indicator, for example.

BRIEF DESCRIPTION OF THE DRAWINGS



[0035] 

Fig. 1 schematically illustrates a driver assist system according to an embodiment.

Fig. 2 shows a process flow schematically illustrating an embodiment of a method for displaying traffic information to a driver of a vehicle.

Fig. 3 shows a process flow schematically illustrating another embodiment of a method for displaying traffic information to a driver of a vehicle.

Fig. 4 shows schematically an example of the information flow for generating an environmental model by an embodiment of the driver assist system and an embodiment of a method for displaying traffic information to a driver of a vehicle.

Fig. 5 A/B illustrates an example for car recognition using an embodiment of a driver assist system and a method for displaying traffic information to a driver of a vehicle.


DETAILED DESCRIPTION OF THE INVENTION



[0036] Figure 1 shows a schematic illustration of the driver assist system according to an embodiment. In the present embodiment, the driver assist system 10 is embodied as ADAS system 11 having a human machine interface (HMI) 12 comprising an augmented reality head-up display (AR-HUD) 13, a cluster instrument 14 and an infotainment screen 15.

[0037] Furthermore, the ADAS system 11 comprises an environmental model generation unit 16 and an image analysis unit 17 configured to determine a vehicle characteristic from an image displayed on AR-HUD 13, the cluster device 14 or an infotainment screen 15. In this embodiment, the image analysis unit 17 is configured to automatically analyse images of the environmental model rendered on one or more of the display devices 13-15, for example, every 50ms. In alternative embodiments, an interactive menu with relating guidance may be provided to allow the driver to select an object from a displayed image for image analysis for determining a relevant characteristic.

[0038] In the present embodiment, the driver assist system 10 additionally comprises a vehicle model database 18 for storing vehicle model type data of vehicle series of various brands and model types and a digital map database 19 for storing digital road map data. The data provided by the vehicle model database 18 and the digital map database 19 is used by the environmental model generation unit 16 to display an environmental model of the traffic situation within a predetermined distance around the ego vehicle depending on the sensor range and is additionally used by the image analysis unit for identifying a vehicle characteristic. In the present embodiment, after identification of a certain model type of a car in the area surrounding the ego vehicle, a corresponding graphical representation of the respective car model is included in the generated environmental model displayed on one or more of the display devices 13-15.

[0039] The driver assist system 10 further comprises a vision sensor 101 and a LiDAR sensor 102 for providing image data of an object detected in an area surrounding the ego vehicle, a sending and receiving unit 103 for receiving image data or a vehicle characteristic data from another vehicle by means of V2V-communication, and a GNSS sensor 104 for detecting the current position of the vehicle.

[0040] A process flow schematically illustrating an embodiment of a method for displaying traffic information to a driver of a vehicle is depicted in figure 2. The method for displaying traffic information to a driver of a vehicle comprises the steps of providing image data of at least one other vehicle in the surrounding area of the ego vehicle (step S21), analysing the received image data (step S22), determining a vehicle characteristic from the result of the analysis (step S23), and displaying information relating to the determined vehicle characteristic (step S24). The step of providing image data of another vehicle in the surrounding area of the ego vehicle may be replaced in cases where a specific vehicle characteristic data is received from an external station or by V2V communication from another traffic participant. It is also conceivable that image data is received from another vehicle or an external station which is then processed by the driver assist system provided in the ego vehicle.

[0041] A further embodiment of a method for displaying traffic information to a driver of a vehicle is depicted in figure 3. In this embodiment, the driver assist system is configured to not only detect and analyse image data of vehicles surrounding the ego vehicle, but in addition to detect and analyse image data of other objects in the environment of the ego vehicle which may possibly influence the driving behaviour of the driver and the vehicle's trajectory. For example, in the present embodiment, the state of a traffic light is determined by image analysis and object recognition of the traffic light, followed by analysis of the ON/OFF- state of the traffic light. Based on the result of the analysis, driving instruction is output by the driver assist system.

[0042] In step S31, image data of the surrounding of the vehicle is provided by a front camera mounted in the area below the windshield of the vehicle, for example. In step S32, the image data is continuously analysed using a convolutional neural network which has been trained to detect vehicles and traffic indicators. In step S33, a vehicle and a traffic sign are identified to be located on the road in front of the ego vehicle on the driving route of the ego vehicle.

[0043] In step S34, the position of the ego vehicle is detected by a GNSS sensor and based on the detection of the position of the ego vehicle and the camera image, the position of the traffic sign and of the other car in front of the ego vehicle are determined.

[0044] In step S35, the outer contour of the car image is compared to model data retrieved from a vehicle model database to determine the vehicle model of the other car driving in front of the ego vehicle. In step S36, based on the determination of the vehicle model in step S35, the shape of brake light of the respective car model, in the present embodiment a BMW 320 model which has been identified as a car driving in front of the ego vehicle, is retrieved from the database and compared to the camera image to determine whether or not the brake light is set ON.

[0045] In step S37, it is continuously analysed whether or not the brake light of the other car is turned ON for more than 5 seconds. If it is confirmed in step S37, that the brake light is in the ON-state for more than 5 seconds, an alert is issued graphically on the display unit of the driver assist system, "Attention vehicle in front is decelerating or standing still," in step S38. This way, a driver of the vehicle may be informed of a potential obstacle on his driving route even if the minimum distance to the potential obstacle at which a warning is conventionally output by the system due to an approaching obstacle has not yet been undershot.

[0046] In step S39, the condition state of the traffic light detected in step S33 is analysed and, if it is determined that the traffic light is in an ON-state in step 40, an information is output to the driver in step S41, "Attention, traffic light 300 m in front is RED." This way, the driver is informed and may, therefore, take action like decelerating the vehicle or setting the gear mechanism to a neutral position for reduction fuel consumption even if the traffic light is still remote with such a distance that braking has not yet become necessary.

[0047] While in the present embodiment alert messages and guidance indications have been output irrespectively of the distance to the detected objects, such messages may be output based on predefined conditions taking into account not only the detected state/condition of the brake light or of a traffic sign, for example, but also under consideration of the distance of the vehicle to the detected object e.g. the other car in front of the ego vehicle or the traffic sign. In such a manner, a more sophisticated guidance information can be determined and output to the driver based on the concept of object recognition and analysis in the surrounding area of the ego vehicle.

[0048] Fig. 4 shows schematically an example of the information flow realised by a software platform for generating an environmental model by an embodiment of the driver assist system and an embodiment of a method for displaying traffic information to a driver of a vehicle. Specifically, in this embodiment the ADAS middleware layer is realised by means of a Robert Operating System (ROS) software framework. Within the ROS software framework, different nodes are provided as executables, being subject to subscribe (s) or publish (p). For example, the process of collecting raw data of a vision sensor N1, raw data of a LiDAR sensor N2, GNSS-based position detection N3, the provision of an object list by a radar sensor N4, and the provision of a digital map data N5 each constitute (executable) nodes under the ROS framework. The same holds true for the relating object detection based on the respective data provided by the vision sensor and the LiDAR sensor, respectively, nodes N21, N22, N31 and N32.

[0049] The fusion of data of the detected objects together with localisation data as determined by the GNSS and road map data from the digital map, node N6, results in an environmental model of the surrounding of the ego vehicle. Components or parts or the entire environmental model may be displayed on an AR-HUD N7 or an instrument cluster N8 or may be utilised for planning purposes N9 such as navigation applications and the like.

[0050] In figure 5 A/B, an example for car model recognition is illustrated. As shown in figure 5A, in a first step an image from the perspective of the driver as taken by a stereo camera is analysed using a convolutional neural network to detect the presence of other cars in front of the ego vehicle. In a second step illustrated in figure 5B, the model type of the vehicle is analysed and determined by comparison of the outer circumference of the car shape with 3D car model data stored in the vehicle data storage.

[0051] While various embodiments have been discussed throughout the present specification, it should be understood that the invention is not limited to any one of these. Features described in the foregoing embodiments may be relevant to one or more embodiments and may be combined in a different manner to constitute additional embodiments as apparent to a person skilled in the art of the present application. It is therefore intended that the foregoing detailed description be regarded as illustrative rather than limiting whereas the scope of protection is being defined by the appending set of claims.


Claims

1. Driver assistance system for providing traffic information to a driver of a ego vehicle, the driver assistance system comprising:

- at least one sensor for detecting objects, including other vehicles, in a surrounding area of the ego vehicle and for providing image data of the detected objects,

- a digital map database providing road data,

- a position detection device for detecting a position of the ego vehicle on the road, and

- a display unit for rendering visual information relating to an environmental model of the surrounding area of the ego vehicle based on image data of the at least one sensor, the position from the position detection device, and the digital map data,

- wherein the system comprises an image analysis unit configured to determine a vehicle characteristic of at least one other vehicle in the surrounding area of the ego vehicle from a vehicle image of the at least one other vehicle generated on the basis of the image data obtained by the at least one sensor and to display information on the display unit based on the determined vehicle characteristic.


 
2. Driver assistance system for a vehicle according to claim 1 or 2, wherein the image analysis unit is configured to determine a vehicle characteristic relating to a current driving state of the other vehicle.
 
3. Driver assistance system for a vehicle according to claim 1, wherein the image analysis unit is configured to determine from the image data a vehicle characteristic depending on a vehicle brand, a model type and/or a colour of the other vehicle by comparison with 3D model data stored in a vehicle model database.
 
4. Driver assistance system for a vehicle according to claim 3, wherein the image analysis unit is configured to determine a vehicle characteristic depending on a brake light state and/or a turning light state and/or a driving light state of the other vehicle.
 
5. Driver assistance system for a vehicle according to one of claims 1-4, comprising a receiving unit for receiving, from another vehicle, communication data relating to the vehicle characteristic of one of the other cars in the surrounding area of the ego vehicle.
 
6. Driver assistance system for a vehicle according to one of claims 1-5, wherein the sensor is a camera, a vision sensor and/or a LiDAR sensor.
 
7. Driver assistance system for a vehicle according to one of claims 1-6, wherein the image analysis unit is configured to continuously analyze data relating to other vehicles in the surrounding area of the ego vehicle and to continuously determine one or more vehicle characteristic from the analysed data and to store the determined vehicle characteristic together with time and/or location information.
 
8. Driver assistance system for a vehicle according to one of claims 1-7, wherein the image analysis unit is configured to detect a state of a static object, in particular the state of a traffic light in the surrounding area of the ego vehicle as an object.
 
9. Driver assistance system for a vehicle according to claim 8, wherein the system is configured to analyse a relationship between a detected position of the other vehicle, a detected braking light state of the other vehicle and a detected state of a traffic light of the other vehicle and to output on the display unit an information relating to the result of the analysis.
 
10. Driver assistance system for a vehicle according to one of claims 1-9, wherein the display unit comprises an instrument cluster or head-up display of an ADAS system.
 
11. Method for displaying traffic information to a driver of a vehicle, comprising the steps of:

- providing image data of at least one other vehicle in the surrounding area of the ego vehicle,

- analysing the received image data,

- determining a vehicle characteristic from the result of the analysis, and

- displaying information relating to the determined vehicle characteristic.


 
12. Method according to claim 11, wherein the step of analysing the received image data comprises applying a neural network for classifying objects in the image provided on the display unit.
 
13. Method according to claim 11 or 12, comprising the steps of determining a position of the other vehicle and determining a vehicle characteristic relating to the driving state of the other vehicle.
 
14. Method according to one of claims 11 to 13, comprising the additional steps of determining guidance or safety information based on the determined vehicle characteristic and the position of the other vehicle and/or the position of the ego vehicle and outputting the determined guidance of safety information to the driver of the ego vehicle.
 
15. Method according to one of claims 13 or 14, further comprising the step of providing image data of a static object within a predefined surrounding range of the ego vehicle, analysing the provided data and outputting safety or guidance information to the driver of the ego vehicle in dependence on the result of the analysis of provided image data and the position of the ego vehicle and/or the position of the other vehicle.
 


Amended claims in accordance with Rule 137(2) EPC.


1. Driver assistance system for providing traffic information to a driver of a ego vehicle, the driver assistance system comprising:

- at least one sensor for detecting objects, including other vehicles, in a surrounding area of the ego vehicle and for providing image data of the detected objects,

- a digital map database providing road data,

- a position detection device for detecting a position of the ego vehicle on the road, and

- a display unit for rendering visual information relating to an environmental model of the surrounding area of the ego vehicle based on image data of the at least one sensor, the position from the position detection device, and the digital map data,

- characterized in that

- the system comprises an image analysis unit configured to determine, from a vehicle image of at least one other vehicle in the surrounding area of the ego vehicle generated on the basis of the image data obtained by the at least one sensor, a vehicle brand, a model type and a colour of the other vehicle by comparison with 3D model data stored in a vehicle model database and to display a specific graphical representation of the determined vehicle model and colour on the display unit based on the determined vehicle brand, model type and colour.


 
2. Driver assistance system for a vehicle according to claim 1 or 2, wherein the image analysis unit is configured to determine a vehicle characteristic relating to a current driving state of the other vehicle.
 
3. Driver assistance system for a vehicle according to claim 1 or 2, wherein the image analysis unit is configured to determine a vehicle characteristic depending on a brake light state and/or a turning light state and/or a driving light state of the other vehicle.
 
4. Driver assistance system for a vehicle according to one of claims 1-3, comprising a receiving unit for receiving, from another vehicle, communication data relating to the vehicle characteristic of one of the other cars in the surrounding area of the ego vehicle.
 
5. Driver assistance system for a vehicle according to one of claims 1-4, wherein the sensor is a camera, a vision sensor and/or a LiDAR sensor.
 
6. Driver assistance system for a vehicle according to one of claims 1-5, wherein the image analysis unit is configured to continuously analyze data relating to other vehicles in the surrounding area of the ego vehicle and to continuously determine one or more vehicle characteristic from the analysed data and to store the determined vehicle characteristic together with time and/or location information.
 
7. Driver assistance system for a vehicle according to one of claims 1-6, wherein the image analysis unit is configured to detect a state of a static object, in particular the state of a traffic light in the surrounding area of the ego vehicle as an object.
 
8. Driver assistance system for a vehicle according to claim 7, wherein the system is configured to analyse a relationship between a detected position of the other vehicle, a detected braking light state of the other vehicle and a detected state of a traffic light and to output on the display unit an information relating to the result of the analysis.
 
9. Driver assistance system for a vehicle according to one of claims 1-8, wherein the display unit comprises an instrument cluster or head-up display of an ADAS system.
 
10. Method for displaying traffic information to a driver of a vehicle, comprising the steps of:

- providing image data of at least one other vehicle in the surrounding area of the ego vehicle by a sensor,

- analyzing the received image data,

- determining a vehicle brand, a model type and a color of the other vehicle by comparison with 3D model data stored in a vehicle model database, and

- displaying a specific graphical representation of the determined vehicle model and color on a display unit.


 
11. Method according to claim 10, wherein the step of analysing the received image data comprises applying a neural network for classifying objects in the image provided on the display unit.
 
12. Method according to claim 10 or 11, comprising the steps of determining a position of the other vehicle and determining a vehicle characteristic relating to the driving state of the other vehicle.
 
13. Method according to one of claims 11 or 12, comprising the additional steps of determining guidance or safety information based on the determined vehicle characteristic and the position of the other vehicle and/or the position of the ego vehicle and outputting the determined guidance of safety information to the driver of the ego vehicle.
 
14. Method according to one of claims 12 or 13, further comprising the step of providing image data of a static object within a predefined surrounding range of the ego vehicle, analysing the provided data and outputting safety or guidance information to the driver of the ego vehicle in dependence on the result of the analysis of provided image data and the position of the ego vehicle and/or the position of the other vehicle.
 




Drawing



















Search report









Search report