Method for warning the driver of a motor vehicle depending on a determined time to collision, camera system and motor vehicle

Abstract

 The invention relates to a method for warning the driver of a motor vehicle of an imminent collision with a target object located in an environmental region of the motor vehicle and approaching the motor vehicle, wherein a temporal sequence of images of the environmental region is provided by means of a camera of the motor vehicle, wherein a measured value (14; TTC1 to TTCx) for a degree of risk (TTC) with respect to the collision is respectively determined from at least a subset of the images based on the respective image by means of an image processing device, and wherein the warning of the driver is effected depending on the measured values (14; TTC1 to TTCx), wherein based on the respectively current measured value (TTC1) as well as based on a plurality of previous measured values (TTC1 to TTCx), a fitting function (15) is determined, which satisfied a predetermined optimization criterion with respect to the measured values (TTC1 to TTCx), and wherein the current degree of risk (TTC) is determined based on the fitting function (15) for warning the driver.

Description

[0001] The invention relates to a method for warning the driver of a motor vehicle of an imminent collision with a target object located in an environmental region of the motor vehicle and approaching the motor vehicle, wherein a temporal sequence of images of the environmental region is provided by means of a camera of the motor vehicle, wherein a measured value for a degree of risk with respect to the collision is respectively determined from at least a subset of the images based on the respective image by means of an image processing device, and wherein the warning of the driver is effected depending on the measured values. In addition, the invention relates to a camera system for performing such a method as well as to a motor vehicle with such a camera system.

[0002] Camera-based collision warning systems are already prior art. Presently, the interest is in particular directed to a so-called "cross traffic alert" system, which serves for warning of possible cross traffic based on images provided by means of a camera. Such camera systems can for example be used in particularly advantageous manner at garage exits, in parking out of a parking space or else on intersections, i.e. overall in road situations, in which the driver has restricted sight to the cross traffic. Usually, therein, a camera is mounted on a front bumper of the motor vehicle and/or a camera in the rear area of the motor vehicle (for example on the rear bumper or on the tailgate). This camera has a relatively wide opening angle in a range of values from 160° to 200° and therefore is able to provide images, in which the cross traffic is depicted. These images are communicated to an electronic image processing device processing the images. The image processing device can for example identify target objects - for example other vehicles, pedestrians, cyclists and the like - and track them over the sequence of images. The images can also be displayed on a display.

[0003] In this context, it is already prior art to calculate the so-called time to collision, i.e. that time the target object requires for reaching the motor vehicle, based on the sequence of images of a camera. This time to collision presents a degree of risk, which is proportional to the existent risk of collision.

[0004] Therein, a measured value for the time to collision can respectively be determined to each image or to each n-th image (n>1). However, it has turned out that these measured values have a relatively great fluctuation over the time such that additional filtering of the measured values is required.

[0005] The determination of the time to collision based on images of a camera is for example already known from the document US 2009/0148986 A1.

[0006] As already explained, the measured values have to be subjected to additional filtering in the prior art in order to be able to provide usable results for the time to collision. Usually, therein, the so-called Kalman filter is used, as it is for example described in the document US 2010/0191391 A1. However, a Kalman filter is associated with relatively great disadvantages. Thus, a relatively high computational power is required for implementing a Kalman filter, but which is only available in restricted manner in particular in motor vehicles. The computational power is in particular not present in so-called embedded systems. Additionally, a Kalman filter presents an algorithm excessively complex for the present application, which optionally also can cause a delay in the calculation of the current time to collision.

[0007] One object of the invention is in providing a method, a camera system as well as a motor vehicle, in which measures are taken, which ensure that the current degree of risk with respect to the collision can be determined particularly precisely and with least computational effort.

[0008] According to the invention, this object is solved by a method, by a motor vehicle as well as by a camera system having the features according to the respective independent claims. Advantageous implementations of the invention are the subject matter of the dependent claims, of the description and of the figures.

[0009] A method according to the invention serves for warning the driver of a motor vehicle of an imminent (possible) collision with a target object located in an environmental region of the motor vehicle and approaching the motor vehicle. A temporal sequence of images of the environmental region is provided by means of a camera of the motor vehicle. To at least a subset of the images - for example to each image or to each n-th image (n>1) - a measured value for a degree of risk with respect to the collision is respectively determined based on the respective image. This determination is effected by means of an electronic image processing device. According to the invention, it is provided that a fitting function is determined based on the respectively current measured value as well as based on a plurality of previous measured values (for example a predetermined number of previous measured values), which satisfies a predetermined optimization criterion (for example "least squares") with respect to the measured values, and that the current degree of risk is determined based on the fitting function for warning the driver.

[0010] Without implementation of a Kalman filter, the degree of risk can therefore be determined particularly precisely and with least computational effort by using also the previous measured values and thus the measurement history in addition to the current measured value, in order to determine a fitting function or fitting curve. This fitting function thus considers a plurality of measured values and preferably is continuously adapted and updated with the respectively new current measured value. By implementing such a fitting function, the method according to the invention can also be implemented in an embedded system, which, as is known, only has restricted computational power. In particular in motor vehicles, this proves particularly advantageous.

[0011] As the degree of risk, for example the above mentioned time to collision (TTC, or time to crossing) and/or a distance to the possible collision can be determined based on the images. Both the TTC and the distance to the collision represent a reliable measure of the degree of risk. For determining the degree of risk, therein, a distance of the target object from the motor vehicle can be determined based on the respective image. However, additionally or alternatively, the degree of risk can also include a position of the target object in the image and/or in the three-dimensional world and/or a target angle relative to the motor vehicle.

[0012] With respect to the above mentioned optimization criterion, the fitting function can preferably be determined by means of a regression analysis method. In this context, it has proven advantageous if the linear or square regression is used. In this manner, the computational effort can further be reduced.

[0013] In an embodiment, for determining the fitting function, it can be assumed that the target object moves with a constant velocity relatively to the motor vehicle such that a linear function is determined as the fitting function. In particular in the above mentioned "cross traffic alert" system serving for detecting the cross traffic, this assumption proves advantageous. Namely, in this system, it can be assumed with high probability that the cross traffic moves substantially with constant velocity relatively to the motor vehicle. This assumption reduces the computational effort to a minimum.

[0014] However, alternatively, a non-linear fitting function can also be determined, in which variation of the relative velocity is also taken into account.

[0015] For determining the fitting function, the measured values are preferably each weighted with an associated weighting factor. Then, the fitting function is determined based on the weighted measured values. In this manner, the method according to the invention can each be optimally adapted to very different applications and different road situations by correspondingly adjusting the weighting factors. Thus, the method can be universally used for different driver assistance systems.

[0016] In an embodiment, it is provided that the weighting factors are determined depending on a position of the target object in the respective image. This embodiment is based on the realization that the accuracy of the measured values depends on the position of the target object in the image frame. The more precise measured values can therefore be weighted with greater weighting factors than the less precise measured values in order to be able to overall determine a very precise fitting function.

[0017] Preferably, the measured values constitute a measurement sequence of measured values, in which the measured values are ordered corresponding to an order of the images within the sequence. Thus, the measurement sequence begins with the respectively current measured value and terminates with an oldest measured value. Preferably, the weighting factors are determined depending on the position of the respective measured value within the measurement sequence. In other words, the weighting factors depend on the "age" of the respective measured value. This embodiment is based on the fact that the recent measured values are considerably more important than the older measured values and therefore are to be prioritized with respect to the older measured values. Thus, a fitting function can be provided, which allows precise estimation of the degree of risk in the respectively current road situation.

[0018] The weighting factors can each be calculated from at least one partial weighting factor, which is determined based on a weighting function having a (global) maximum for the current measured value and a (global) minimum for an oldest measured value. This weighting function can for example be an exponential function, for example the e^-t function. Thus, this embodiment ensures that the current measured values are weighted with greater weighting factors than the older measured values. As already explained, thereby, a fitting function can be provided, which allows very precise determination of the current degree of risk for the instantaneous road situation.

[0019] It can also be provided that the weighting factors, with which the respective measured values are weighted, are each calculated from at least a first and a second partial weighting factor, for example by multiplication of the first partial weighting factor by the second partial weighting factor. The first partial weighting factor can be determined depending on a position of the target object in the respective image. The second partial weighting factor in turn is preferably determined depending on the position of the respective measured value within the measurement sequence and thus depending on the "age" of the respective measured value. By providing two different partial weighting factors, thus, the position of the target object in the respective image on the one hand as well as also the position of the respective measured value within the measurement sequence on the other hand can be taken into account. The advantages of these two embodiments can therefore be combined in advantageous manner.

[0020] It has turned out that the accuracy of the fitting function can be adversely affected by so-called outliers. In order to prevent such adverse affectation of the accuracy of the fitting function, the following embodiments are proposed:

Before the current fitting function is updated based on a new measured value, it can be examined if a deviation of this new measured value from the current fitting function is less or greater than a preset threshold value. The update of the fitting function based on this new measured value can only be performed if the deviation is less than the threshold value. In other words, the respectively new measured value is ignored in determining the fitting function if the deviation thereof from the current fitting function is greater than the threshold value. Thus, the outliers can be early detected and filtered out such that the reliability and the accuracy in determining the degree of risk can be increased.

[0021] Additionally or alternatively, the following method can also be performed: After update of the fitting function based on a new measured value, the measured values used for the fitting function (for example all of the measured values, which have been considered in the current fitting function) can be examined to the effect if a deviation of the respective measured values from the current fitting function is less or greater than a preset threshold value. This threshold value can be the above mentioned threshold value or another threshold value. A new fitting function can then be determined exclusively based on those measured values, the deviation of which is less than the threshold value. In this embodiment, thus, former measured values can also be filtered out as outliers such that the accuracy of the fitting function can be further improved.

[0022] In addition, the invention relates to a camera system formed for performing a method according to the invention.

[0023] A motor vehicle according to the invention, in particular a passenger car, includes a camera system according to the invention.

[0024] The preferred embodiments presented with respect to the method according to the invention and the advantages thereof correspondingly apply to the camera system according to the invention as well as to the motor vehicle according to the invention.

[0025] Further features of the invention are apparent from the claims, the figures and the description of figures. All of the features and feature combinations mentioned above in the description as well as the features and feature combinations mentioned below in the description of figures and/or shown in the figures alone are usable not only in the respectively specified combination, but also in other combinations or else alone.

[0026] Now, the invention is explained in more detail based on a preferred embodiment as well as with reference to the attached drawings.

[0027] There show:

Fig. 1

in schematic Illustration a motor vehicle with a camera system according to an embodiment of the invention;

Fig. 2

an exemplary progression of a fitting function;

Fig. 3

an exemplary progression of a first weighting function;

Fig. 4

an exemplary progression of a second weighting function; and

Fig. 5

in schematic illustration a possible road situation for explaining the camera system.

[0028] A motor vehicle 1 illustrated in Fig. 1 is for example a passenger car. The motor vehicle 1 includes a camera system 2, which is formed as a cross traffic alert system and thus as a collision warning system in the embodiment, which warns the driver of the motor vehicle 1 of cross traffic.

[0029] The camera system 2 includes at least one camera 3, 4, which is attached to the motor vehicle 1. For example, two cameras 3, 4 can be provided, wherein the invention is not restricted to a certain number of cameras 3, 4 and the number of the cameras 3, 4 can be arbitrary. In the embodiment, a first camera 3 is disposed on the front bumper of the motor vehicle 1. A second camera 4 is disposed in the rear area, for example on the rear bumper or on a tailgate of the motor vehicle 1. The cameras 3, 4 each provide a temporal sequence of images and communicate these images to a central electronic image processing device 5 processing the received images and being able to provide very different functionalities in the motor vehicle 1 based on the images. In the collision warning system 2, it is provided that the driver is warned of the cross traffic based on the images, namely by means of a warning device 6, which can for example include a display and/or a speaker.

[0030] In Fig. 5, a possible road situation is shown, in which the driver can be assisted by the camera system 2 of the vehicle 1. The motor vehicle 1 is in a garage exit 7, which is bounded by respective walls 8, 9 on both sides. The garage exit 7 goes to a road 10, which extends perpendicularly to the motor vehicle 1. A further vehicle 11 (target object) travels on the road 10, which moves towards the motor vehicle 1 according to the arrow representation 12. The vehicle 11 then moves past the garage exit 7 and thus passes the garage exit 7.

[0031] Due to the walls 8, 9, the driver of the motor vehicle 1 has restricted sight to the road 10. Thus, the driver is not able to see himself the further vehicle 11 such that the camera system 2 has to intervene here. Based on the images of the camera 3, the image processing device 5 determines measured values for the time to collision (TTC), which represents a degree of risk. Therein, the TTC means a period of time, which the vehicle 11 requires for reaching the motor vehicle 1 or for passing the garage exit 7. Thus, the TTC describes a period of time until a possible collision, which would occur if the motor vehicle 1 would be on a collision trajectory of the vehicle 11. In still other words, the TTC describes a period of time that the vehicle 11 requires for reaching a line 13 presenting an extension of the lateral flank of the motor vehicle 1 and extending perpendicularly to the direction of travel 12 of the vehicle 11. Thus, the TTC can also be calculated if the vehicle 11 will prospectively pass the motor vehicle 1 and thus collision will prospectively not occur.

[0032] Thus, the image processing device 5 calculates a measured value for the TTC to each or to each n-th image (n>1). Thus, a measurement sequence of measured values is formed over the sequence of images:

TTC1, TTC2, TTC3,...TTCx,

wherein TTC1 denotes the respectively current measured value, which is determined based on the current image. Therein, the number x of the measured values TTC1 to TTCx can be a preset number such that the measurement sequence overall always includes a preset and fixed number of measured values TTC1 to TTCx. After acquiring a new measured value TTC1, thus, the respectively oldest measured value TTCx is removed from the measurement sequence - similar to a register. However, alternatively, it can also be provided that the number x of the measured values TTC1 to TTCx is constantly increased such that the measurement sequence includes all of the measured values TTC1 to TTCx, which were determined with respect to one and the same target object 11.

[0033] For determining the measured values TTC1 to TTCx, the target object 11 is identified in the respective image, and the position thereof in the respective image is determined. Based on the position and the relative velocity, then, the TTC can be determined.

[0034] In Fig. 2, an exemplary progression of the measured values TTC1 to TTCx (in seconds) is shown depending on the time t and thus over the sequence of images. The progression of the determined measured values TTC1 to TTCx is denoted by 14 in Fig. 2. As is apparent from Fig. 2, the measured values TTC1 to TTCx have a relatively great standard deviation and thus a relatively great fluctuation. In order to allow accurate estimation of the actual current TTC based on the measured values TTC1 to TTCx, a fitting function is determined, which is exemplarily shown in Fig. 2 and denoted by 15. In the embodiment, a linear fitting function 15 is used; however, the invention is not restricted to such a linear function.

[0035] For determining the fitting function 15, preferably, a regression analysis method is used, such as in particular the linear or square regression. For example, the "least squares" method can also be used.

[0036] In determining the fitting function 15, the measured values TTC1 to TTCx are each weighted with a weighting factor. Therein, each weighting factor is composed of two partial weighting factors, for example by a multiplication:

wherein W_total denotes the respective weighting factor, with which the measured values TTC1 to TTCx are weighted, W1 denotes a first partial weighting factor and W2 denotes a second partial weighting factor.

[0037] Therein, the first partial weighting factor W1 depends on the position of the target object 11 in the respective image. According to Fig. 3, here, a first heuristic weighting function 16 can be defined, which represents the dependency of the partial weighting factor W1 on a position P of the target object 11 in the image. Therein, only the dependency on the position P in a single direction of the image frame, for example in horizontal direction, is shown in Fig. 3. However, this first weighting function 16 can also be a three-dimensional function considering both image directions. Namely, the accuracy of the TTC is correlated with the position P such that the more precise measured values TTC1 to TTCx can be prioritized by corresponding weighting.

[0038] The respective second partial weighting factor W2 in turn depends on the position of the respective measured value within the measurement sequence TTC1 to TTCx. For determining the second partial weighting factor W2, a second weighting function 17 is defined, which is preferably a heuristic function and is exemplarily illustrated in Fig. 4. As is apparent from Fig. 4, the relation applies that the second partial weighting factor W2 is higher at the "recent" measured values than at the "older" measured values. The second weighting function 17 can be a linear, a polynomial or an exponential function. Therein, an exponential function is shown in Fig. 4, namely: W2=e^-t, Here, t denotes the time since the measurement of the respective value TTC1 to TTCx and thus the age of the measured value TTC1 to TTCx. However, in all of the embodiments, it is provided that the second weighting function 17 has a global maximum 18 for the respectively current measured value TTC1 and a global minimum 19 for the oldest measured value TTCx.

[0039] Generally speaking, the first and/or the second weighting function 16, 17 can be a heuristic function.

[0040] If the used regression analysis method requires it, the weighting factors W_total can optionally also be normalized, for example such that the sum of all of the weighting factors is equal to "1".

[0041] Optionally, filtering of outliers can also be performed to prevent adverse affectation of the fitting function 15. For example, the respectively new measured value TTC1 can be examined to the effect if the deviation thereof from the current (not yet adapted) fitting function 15 is less than a preset threshold value. If it is detected that the current measured value TTC1 is greater than the threshold value, thus, this measured value TTC1 can be ignored such that adaptation of the fitting function 15 based on this new measured value TTC1 is not effected. Additionally or alternatively, after acquisition of each new measured value TTC1, all of the previous measured values TTC1 to TTCx can also be examined to the effect if the deviation thereof from the newly adapted fitting function 15 is less or greater than a preset threshold value. If measured values TTC1 to TTCx are detected, the deviation of which is greater than the threshold value, thus, these measured values TTC1 to TTCx can be filtered out, and a new fitting function 15 can be determined exclusively based on those measured values TTC1 to TTCx, the deviation of which is less than the threshold value. This new fitting function 15 can then be used for the determination of the current TTC.

Claims

1. Method for warning the driver of a motor vehicle (1) of an imminent collision with a target object (11) located in an environmental region of the motor vehicle (1) and approaching the motor vehicle (1), wherein a temporal sequence of images of the environmental region is provided by means of a camera (3, 4) of the motor vehicle (1), wherein a measured value (TTC1 to TTCx) for a degree of risk (TTC) with respect to the collision is respectively determined from at least a subset of the images based on the respective image by means of an image processing device (5), and wherein the warning of the driver is effected depending on the measured values (TTC1 to TTCx),
characterized in that
based on the respectively current measured value (TTC1) as well as based on a plurality of previous measured values (TTC1 to TTCx), a fitting function (15) is determined, which satisfies a predetermined optimization criterion with respect to the measured values (TTC1 to TTCx), and that the current degree of risk (TTC) is determined based on the fitting function (15) for warning the driver.

2. Method according to claim 1,
characterized in that
a time to collision and/or a distance to collision is determined as the degree of risk (TTC) based on the images.

3. Method according to claim 1 or 2,
characterized in that
the fitting function (15) is determined by means of a regression analysis method.

4. Method according to any one of the preceding claims,
characterized in that
for determining the fitting function (15), it is assumed that the target object (11) moves with constant velocity relatively to the motor vehicle (1) such that a linear function is determined as the fitting function (15).

5. Method according to any one of the preceding claims,
characterized in that
for determining the fitting function (15), the measured values (TTC1 to TTCx) are each weighted with an associated weighting factor (W_total) and the fitting function (15) is determined based on the weighted measured values (TTC1 to TTCx).

6. Method according to claim 5,
characterized in that
the weighting factors (W_total) are determined depending on a position (P) of the target object (11) in the respective image.

7. Method according to claim 5 or 6,
characterized in that
the measured values (TTC1 to TTCx) constitute a measurement sequence of measured values (TTC1 to TTCx), in which the measured values (TTC1 to TTCx) are ordered corresponding to an order of the images within the sequence of images, wherein the weighting factors (W_total) are determined depending on the position of the respective measured value (TTC1 to TTCx) within the measurement sequence.

8. Method according to claim 7,
characterized in that
the weighting factors (W_total), with which the respective measured values (TTC1 to TTCx) are weighted, are each calculated from at least one partial weighting factor (W2), which is determined based on a weighting function (17), which has a maximum (18) for the current measured value (TTC1) and a minimum (19) for an oldest measured value (TTCx).

9. Method according to any one of claims 5 to 8,
characterized in that
the weighting factors (W_total), with which the respective measured values (TTC1 to TTCx) are weighted, are each calculated from at least a first and a second partial weighting factor (W1, W2), wherein the first partial weighting factor (W1) is determined depending on a position (P) of the target object (11) in the respective image and the second partial weighting factor (W2) is determined depending on the position of the respective measured value (TTC1 to TTCx) within the measurement sequence.

10. Method according to any one of the preceding claims,
characterized in that
before the current fitting function (15) is updated based on a new measured value (TTC1), it is examined if a deviation of this new measured value (TTC1) from the current fitting function (15) is less or greater than a preset threshold value, and the update of the fitting function (15) based on the new measured value (TTC1) is only performed if the deviation is less than the threshold value.

11. Method according to any one of the preceding claims,
characterized in that
after update of the fitting function (15) based on the new measured value (TTC1), the measured values (TTC1 to TTCx) used for the fitting function (15) are examined to the effect if a deviation of the respective measured values (TTC1 to TTCx) from the current fitting function (15) is less or greater than a preset threshold value, and a new fitting function (15) is determined exclusively based on those measured values (TTC1 to TTCx), the deviation of which is less than the threshold value.

12. Camera system (2) for warning the driver of a motor vehicle (1) of an imminent collision with a target object (11) located in an environmental region of the motor vehicle (1) and approaching the motor vehicle (1), including a camera (3, 4) for providing a temporal sequence of images of the environmental region, including an image processing device (5) adapted to respectively determine a measured value (TTC1 to TTCx) for a degree of risk (TTC) with respect to the collision from at least a subset of the images based on the respective image, and including a warning device (6) for warning the driver depending on the measured values (TTC1 to TTCx),
characterized in that
the camera system (2) is adapted to determine a fitting function (15) based on the respectively current measured value (TTC1) as well as based on a plurality of previous measured values (TTC1 to TTCx), which satisfies a predetermined optimization criterion with respect to the measured values (TTC1 to TTCx), and to determine the current degree of risk (TTC) based on the fitting function (15) for warning the driver.

13. Motor vehicle (1) with a camera system (2) according to claim 12.

Drawing