(19)
(11)EP 3 147 884 B1

(12)EUROPEAN PATENT SPECIFICATION

(45)Mention of the grant of the patent:
11.12.2019 Bulletin 2019/50

(21)Application number: 14892646.2

(22)Date of filing:  20.05.2014
(51)International Patent Classification (IPC): 
G08G 1/095(2006.01)
G06K 9/00(2006.01)
G01C 21/26(2006.01)
G06T 7/70(2017.01)
(86)International application number:
PCT/JP2014/063347
(87)International publication number:
WO 2015/177864 (26.11.2015 Gazette  2015/47)

(54)

TRAFFIC-LIGHT RECOGNITION DEVICE AND TRAFFIC-LIGHT RECOGNITION METHOD

VORRICHTUNG UND VERFAHREN ZUR ERKENNUNG EINER VERKEHRSAMPEL

DISPOSITIF DE RECONNAISSANCE DE FEUX DE CIRCULATION ET PROCÉDÉ DE RECONNAISSANCE DE FEUX DE CIRCULATION


(84)Designated Contracting States:
AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

(43)Date of publication of application:
29.03.2017 Bulletin 2017/13

(73)Proprietor: NISSAN MOTOR CO., LTD.
Yokohama-shi, Kanagawa 221-0023 (JP)

(72)Inventors:
  • YAMANOI, Daiki
    Atsugi-shi Kanagawa 243-0123 (JP)
  • ANDOU, Toshiyuki
    Atsugi-shi Kanagawa 243-0123 (JP)
  • NISHIUCHI, Hidekazu
    Atsugi-shi Kanagawa 243-0123 (JP)
  • MATSUO, Haruo
    Atsugi-shi Kanagawa 243-0123 (JP)
  • SATOH, Hiroshi
    Atsugi-shi Kanagawa 243-0123 (JP)

(74)Representative: Grünecker Patent- und Rechtsanwälte PartG mbB 
Leopoldstraße 4
80802 München
80802 München (DE)


(56)References cited: : 
WO-A1-2008/038370
JP-A- 2007 257 299
JP-A- 2008 149 786
JP-A- 2013 171 489
US-A1- 2016 343 253
JP-A- 2007 178 223
JP-A- 2007 320 458
JP-A- 2008 293 277
US-A1- 2010 302 361
  
  • LINDNER F ET AL: "Robust recognition of traffic signals", INTELLIGENT VEHICLES SYMPOSIUM, 2004 IEEE PARMA, ITALY JUNE 14-17, 2004, PISCATAWAY, NJ, USA,IEEE, 14 June 2004 (2004-06-14), pages 49-53, XP010727441, DOI: 10.1109/IVS.2004.1336354 ISBN: 978-0-7803-8310-4
  
Note: Within nine months from the publication of the mention of the grant of the European patent, any person may give notice to the European Patent Office of opposition to the European patent granted. Notice of opposition shall be filed in a written reasoned statement. It shall not be deemed to have been filed until the opposition fee has been paid. (Art. 99(1) European Patent Convention).


Description

TECHNICAL FIELD



[0001] The present invention relates to a traffic signal recognition apparatus and a traffic signal recognition method.

BACKGROUND ART



[0002] A conventionally known vehicle surroundings recognition apparatus recognizes a traffic signal or a stop sign in an image acquired by imaging a scene ahead of the vehicle in its travelling direction (Patent Literature 1). When there are multiple traffic signals or stop signs in the image, Patent Literature 1 determines the order in which the traffic signals or stop signs are arranged in the travelling direction of the vehicle, and controls the travelling conditions of the vehicle according to a traffic signal or a stop sign that is located closest to the vehicle.

CITATION LIST


PATENT LITERATURE



[0003] Patent Literature 1: Japanese Patent Application Publication No. 2007-257299

SUMMARY OF INVENTION


TECHNICAL PROBLEM



[0004] When selecting, from multiple traffic signals, a traffic signal based on which driving control is to be performed, Patent Literature 1 considers the distance from the vehicle to the traffic signal, but fails to consider the possibility of the traffic signal being blocked out of sight. A traffic signal is, in some cases, hidden and invisible from the vehicle due to a factor unknowable from map information, such as a preceding vehicle, a street tree, or road work.

[0005] The present invention has been made in consideration of the above problem, and has an objective to provide a traffic signal recognition apparatus and a traffic signal recognition method that can improve a probability for detection of a traffic signal when two or more traffic signals are predicted to be captured in an image.

SOLUTION TO PROBLEM



[0006] A traffic signal recognition apparatus according to an aspect of the present invention comprises an imager mounted in a vehicle, which is configured to acquire an image by imaging surroundings of a vehicle, a self-position detector configured to detect a self-position of the vehicle, a traffic signal detector configured to detect, in the image, a traffic signal around the vehicle, and a traffic signal position estimator configured to estimate a relative position of the traffic signal with respect to the vehicle, using the self-position and map information containing positional information on the traffic signals. A traffic signal identifier is configured to identify two or more of the traffic signals around the vehicle that are predicted to be captured in the image, using the estimated relative position, and a priority level assigner is configured to assign a priority level to each of the two or more traffic signals identified by the traffic signal identifier using a possibility of the traffic signal being blocked out of sight that is derived from the positional information on the traffic signal. When the traffic identifier identifies two or more traffic signals, the traffic signal detector detects in the image is a traffic signal with a highest priority level among the two or more traffic signals, wherein after succeeding in detecting a traffic signal with the highest priority level, the traffic signal detector is configured to continue detecting the traffic signal and the traffic signal detector is configured to perform image processing for traffic signal detection on the image within a detection area set for the traffic signal having the highest priority level.

BRIEF DESCRIPTION OF DRAWINGS



[0007] 

[Fig. 1] Fig. 1 is a block diagram showing information inputted to and outputted from a traffic signal recognition apparatus 100 according to an embodiment.

[Fig. 2] Fig. 2 is a block diagram showing the configuration of the traffic signal recognition apparatus 100 according to a first embodiment and a dataflow therein.

[Fig. 3] Fig. 3 is a block diagram showing the configuration of a detection area setter 14 in Fig. 2 and a dataflow therein.

[Fig. 4] Fig. 4 is a bird's eye view showing an example of priority level assignment by a priority level assigner 24 in Fig. 3.

[Fig. 5] Fig. 5 is a flowchart showing an example of a traffic signal recognition method using the traffic signal recognition apparatus 100.

[Fig. 6] Fig. 6 is a flowchart showing a detailed procedure of Step S13 in Fig. 5 in the first embodiment.

[Fig. 7] Fig. 7 is a flowchart showing a detailed procedure of Step S13 in Fig. 5 in a second embodiment.

[Fig. 8] Fig. 8 is a block diagram showing the configuration of a priority level assigner 24b according to a third embodiment.

[Fig. 9] Fig. 9 is a bird's eye view showing an example of priority level assignment by the priority level assigner 24b in Fig. 8.

[Fig. 10] Fig. 10 is a block diagram showing the configuration of a priority level assigner 24c according to a fourth embodiment.

[Fig. 11] Fig. 11 is a flowchart showing a detailed procedure of Step S07 in Fig. 5 in the fourth embodiment.

[Fig. 12] Fig. 12 is a bird's eye view showing an example of priority level assignment by the priority level assigner 24c in Fig. 10.


DESCRIPTION OF EMBODIMENTS



[0008] Hereinbelow, embodiments of the present invention are described based on the drawings. The same components are denoted by the same reference numerals to avoid repetitive description.

[First Embodiment]



[0009] With reference to Fig. 1, a description is given of information inputted to and outputted from a traffic signal recognition apparatus 100 according to an embodiment. The traffic signal recognition apparatus 100 recognizes a traffic signal, which is placed near a road on which a vehicle is travelling, in an image captured by an imager (camera) mounted in the vehicle.

[0010] The traffic signal recognition apparatus 100 receives input of map information D02, landmark information D01, and camera information D03. The map information D02 contains positional information on traffic signals, in which positions in an actual environment and positions on a map are associated with each other beforehand. The landmark information D01 is used to calculate the self-position of the vehicle in the actual environment. Landmarks include a characteristic object on the ground (terrestrial landmarks) and a Global Positioning System (GPS) satellite that transmits GPS signals receivable by the vehicle. In the embodiment, a terrestrial landmark is used as an example. The landmark information D01 contains, for example, positional information on terrestrial landmarks. The camera information D03 is used to obtain an image of the surroundings (e.g., the front) of the vehicle from the imager. Based on these pieces of information D01 to D03, the traffic signal recognition apparatus 100 outputs a result of recognition of a traffic signal as traffic signal information D04.

[0011] With reference to Fig. 2, the configuration of the traffic signal recognition apparatus 100 according to a first embodiment and a dataflow therein are described. The traffic signal recognition apparatus 100 includes an imager 11, a self-position detector 12, a traffic signal position estimator 13, a detection area setter 14, and a traffic signal detector 15.

[0012] The imager 11 is mounted in a vehicle and acquires an image by imaging the surroundings of the vehicle. The imager 11 is a camera including a solid-state image pickup element, for example, a CCD and a CMOS, and acquires an image on which image processing can be performed. The imager 11 sets the angle of view of its lens and vertical and horizontal camera angles based on the camera information D03, and outputs an acquired image as image data D08.

[0013] The self-position detector 12 detects the self-position of the vehicle based on the landmark information D01. The landmark information D01 indicates, for example, the relative position of a terrestrial landmark (a store, a sight, or a sightseeing spot), detected by a sensor such as an in-vehicle camera or a laser radar, with respect to the vehicle. In the map information D02, positional information on the terrestrial landmark is preregistered. The self-position detector 12 can detect the self-position of the vehicle by checking the landmark information D01 and the relative position information on the terrestrial landmark against each other. Herein, the "position" includes coordinates and an attitude. To be more specific, the position of a terrestrial landmark includes the coordinates and attitude of the terrestrial landmark, and the position of the vehicle includes the coordinates and attitude of the vehicle. As self-position information D05, the self-position detector 12 outputs the coordinates (x, y, z) represented in a reference coordinate system and the attitude components (pitch, yaw, and roll) in the rotational directions of the respective coordinate axes.

[0014] For example, the self-position detector 12 detects an initial position of the vehicle using the landmark information D01. The initial position is the position of the vehicle, i.e., the coordinates and attitude thereof, which can be directly obtained from the landmark information D01. Then, the self-position detector 12 adds the amount of movement of the vehicle to the initial position, to thereby calculate the self-position of the vehicle 51. The self-position detector 12 can estimate the amount of movement of the vehicle, i.e., an amount of change in coordinates and attitude, per unit time by using an odometer, a radar, a gyroscopic sensor, a yaw-rate sensor, and a steer-angle sensor.

[0015] The traffic signal position estimator 13 estimates the relative position of a traffic signal with respect to the vehicle based on the map information D02 and the self-position information D05. In the map information D02, the positional information (coordinates information) on the traffic signal is preregistered. The traffic signal position estimator 13 can obtain the relative coordinates of the traffic signal with respect to the vehicle from the coordinates of the traffic signal and the coordinates and attitude of the vehicle. The traffic signal position estimator 13 outputs the estimated relative coordinates of the traffic signal as relative position information D06. Note that the traffic signal estimated by the traffic signal position estimator 13 is a traffic signal that presents a signal to the vehicle.

[0016] The detection area setter 14 sets a detection area for the traffic signal within an image, based on the relative position of the traffic signal. Because the imager 11 is fixed to the vehicle, a position on the image at which the traffic signal is likely to be captured can be specified within the image when the angle of view and imaging direction of the imager 11 are determined. Based on this position on the image, the detection area setter 14 determines the detection area for the traffic signal within the image.

[0017] The traffic signal detector 15, when there are two or more traffic signals, detects a traffic signal having the highest priority level among the two or more traffic signals from the image. Specifically, the traffic signal detector 15 performs image processing for traffic signal detection on the image data D08 within a detection area set for the traffic signal having the highest priority level. The image processing method is not limited. For example, a traffic light in the traffic signal can be detected using synchronous detection processing based on the alternating cycle of commercial power supplies or using hue and shape similarity determination processing. Other known image processing may also be used for traffic signal detection. Performing the image processing not on the entire image data D08, but only on a part thereof (the detection area) reduces the information processing load for the traffic signal detection, allowing quick traffic signal detection. The traffic signal detector 15 outputs a result of the traffic signal detection as the traffic signal information D04.

[0018] The self-position detector 12, the traffic signal position estimator 13, the detection area setter 14, and the traffic signal detector 15 can be implemented by a microcontroller having a CPU, a memory, and an input and output unit. Specifically, the CPU executes preinstalled computer programs to implement the multiple information processors (12 to 15) in the microcontroller. Part of the memory in the microcontroller forms map database storing the map information D02. The microcontroller may be used also as an ECU used for other vehicle controls (e.g., autonomous driving control).

[0019] With reference to Fig. 3, the configuration of the detection area setter 14 in Fig. 2 and a dataflow therein are described. The detection area setter 14 includes a traffic signal identifier 23, a priority level assigner 24, and an area determiner 25.

[0020] The traffic signal identifier 23 identifies two or more traffic signals predicted to be captured in an image, based on the relative coordinates of the traffic signals (relative position information D06). Because the angle of view and vertical and horizontal directions for the image can be specified based on the camera information D03, the traffic signal identifier 23 can identify the traffic signals that fall within the frame of the image. However, the traffic signals identified by the traffic signal identifier 23 include a traffic signal that is within the frame of the image but is not captured in the image because the traffic signal is blocked out of sight due to a factor unknowable from the map information, as will be described later.

[0021] The priority level assigner 24 assigns priority levels to the two or more traffic signals contained in the map information D02 and identified by the traffic signal identifier 23, the assignment being based on the possibilities of the traffic signals being blocked out of sight. The traffic signals differ in their likeliness of being captured in an image, i.e., possibility of being blocked out of sight by an obstacle located around the vehicle, depending on the positions of the traffic signals relative to the road and the positions of the traffic signals relative to the vehicle. Details for this will be given later with reference to Fig. 4. For the reason above, when two or more traffic signals are predicted to be captured in an image, the priority level assigner 24 assigns priority levels to the two or more traffic signals.

[0022]  The area determiner 25 sets a detection area for each of the traffic signals within the image. When the angle of view and imaging direction of the imager 11 are determined, the position of a traffic signal predicted to be captured can be identified on an image based on the relative coordinates of the traffic signal (relative position information D06). The area determiner 25 transforms the relative coordinates of the traffic signal into the coordinates on the image in which the traffic signal has been captured. Specifically, the area determiner 25 receives input of the relative position information D06 and performs, based on the lens optical system in the imager 11, coordinate transformation from the coordinates of the traffic signal in the three-dimensional coordinate system (x, y, z) to the coordinates on the image in the two-dimensional coordinate system (xl, yl). The coordinate transformation method is not limited, and any known method may be used. The area determiner 25 sets a predetermined area containing the transformed coordinates on the image (xl, yl), as a detection area for the traffic signal on the image.

[0023] As described, the detection area setter 14 identifies two or more traffic signals that fall within the frame of the image, assigns priority levels to the traffic signals thus identified, and sets a detection area for each of the identified traffic signals.

[0024] With reference to Fig. 4, an example of priority level assignment by the priority level assigner 24 is described. In this example, at an intersection Crs, the traffic signal identifier 23 identifies three traffic signals (Ts1, Ts2, Ts3) to be obeyed by the vehicle 51 travelling on a road RD. In this example, the intersection Crs is the first intersection to be crossed by the vehicle 51. Needless to say, the traffic signal identifier 23 may identify traffic signals at the second, third, etc intersections to be crossed by the vehicle 51 after crossing the intersection Crs.

[0025] Based on the positional information on the traffic signals contained in the map information D02, the traffic signal identifier 23 identifies that the traffic signal Ts1 is located near the oncoming lane (PL), and the traffic signals Ts2 and Ts3 are located near the current lane (WL). In addition, the traffic signal identifier 23 identifies that the traffic signal Ts2 is located on the near side of the intersection Crs and that the traffic signal Ts3 is located on the far side of the intersection Crs.

[0026] The priority level assigner 24 assigns priority levels to the three traffic signals (Ts1, Ts2, Ts3) in conformity with basis rules (R1) and (R2) given below.

(R1) Assign a higher priority level to a traffic signal located near the oncoming lane (PL) than to a traffic signal located near the current lane (WL) of travel of the vehicle 51.

(R2) Assign a higher priority level to a traffic signal close to the vehicle 51 than to a traffic signal distant from the vehicle 51.



[0027] The basic rule (R1) is defined based on the following reason. A possibility of being blocked out of sight due to a factor unknowable from the map information, such as a preceding car, a street tree planted on the shoulder of the road, or road work, is higher for a traffic signal located on the current lane (WL) than for a traffic signal located on the oncoming lane (PL). Although an oncoming vehicle might block a traffic signal located on the oncoming lane (PL), the time period of such blocking is only a moment when the oncoming vehicle passes the traffic signal. On the other hand, when a traffic signal located on the current lane (WL) is blocked out of sight due to a factor unknowable from the map information, the time period of such blocking is longer than that for a traffic signal located on the oncoming lane (PL). For example, when the vehicle is following a large-sized vehicle, a traffic signal located on the current lane (WL) is blocked out of sight by the large-sized vehicle for a continuous period of time. For this reason, a higher priority level is assigned to a traffic signal located on the oncoming lane (PL) than to a traffic signal located on the current lane (WL) of travel of the vehicle. This allows preferential detection of a traffic signal which is less likely to be blocked out of sight or is likely to be blocked out of sight only for a short period of time.

[0028] The basic rule (R2) is defined based on the following reason. When multiple traffic signals located on the same lane (the current lane WL or the oncoming lane PL) are viewed from the vehicle, a traffic signal close to the vehicle can be recognized more stably than a traffic signal distant from the vehicle because the close traffic signal has a lower possibility of being blocked out of sight and is imaged in a larger size than the far traffic signal.

[0029] According to the basic rules (R1) and (R2), the priority levels of the three traffic signals (Ts1, Ts2, Ts3) in Fig. 4 are, from highest to lowest, the traffic signal Ts1, the traffic signal Ts2, and the traffic signal Ts3.

[0030] In some cases, the traffic signal identifier 23 identifies an advance warning traffic signal as any of the two or more traffic signals predicted to be captured in the image. Typically, an advance warning traffic signal is provided to give notification of a road condition or traffic-signal condition ahead of time. Thus, even at a position where a regular traffic signal may be blocked out of sight, an advance warning traffic signal is less likely to be blocked out of sight. For this reason, the priority level assigner 24 assigns a higher priority level to an advance warning traffic signal than to other traffic signals. This allows even earlier traffic signal detection.

[0031] Although three traffic signals are identified at the intersection Crs in the embodiment, the priority level assigner 24 can of course assign priority levels to two, four, or more traffic signals in conformity with the above rules (R1) and (R2). Additionally, it should be noted that the traffic signals are not limited to ones located at an intersection, but include all traffic signals to be obeyed on a road on which the vehicle is travelling, such as, for example, a traffic signal for a crosswalk on the road RD.

[0032]  With reference to Fig. 5, an example of a traffic signal recognition method using the traffic signal recognition apparatus 100 is described. The flowchart shown in Fig. 5 is repeated with a predetermined period.

[0033] First, in Step S01, the imager 11 acquires an image by imaging the surroundings of the vehicle 51 based on the camera information D03. In Step S03, the self-position detector 12 detects a self-position of the vehicle 51 using the landmark information D01, and outputs the detected self-position as the self-position information D05.

[0034] In Step S05, the traffic signal position estimator 13 estimates the relative positions of traffic signals with respect to the vehicle 51 based on the map information D02 and the self-position information D05.

[0035] In Step S07, the traffic signal identifier 23 identifies two or more traffic signals predicted to be captured in the image based on the relative coordinates of the traffic signals (the relative position information D06). Then, the priority level assigner 24 assigns priority levels to the two or more traffic signals identified by the traffic signal identifier 23, based on the possibilities of the traffic signals being blocked out of sight. These possibilities are derivable from the positional information on the traffic signals, which is contained in the map information D02. Specifically, the priority level assigner 24 assigns priority levels to the two or more traffic signals in conformity with the basic rules (R1) and (R2).

[0036] In Step S09, the area determiner 25 transforms the relative coordinates (x, y, z) of each traffic signal into coordinates on the image (xl, yl), and determines the center coordinates of a detection area based on the coordinates on the image. The detection area has predetermined size and shape. The detection area is thus determined for each of the two or more traffic signals (Step S11).

[0037]  In Step S13, the traffic signal detector 15 performs image processing for traffic signal detection on the image data D08 within the detection area, according to the assigned priority levels if there are two or more traffic signals.

[0038] With reference to Fig. 6, a detailed procedure of Step S13 in Fig. 5 is described. The following description assumes that N traffic signals have been identified in Step S07 in Fig. 5. First, in Step S21, image data within the detection areas for the respective N traffic signals are extracted. In Step S23, the priority levels (i) assigned to the traffic signals in Step S07 in Fig. 5 are mapped onto the extracted detection areas. Note that "i" is a natural number from 1 to N, and i=1 at the initial setting.

[0039] In Step S25, traffic signal detection is attempted, starting with the detection area corresponding to a traffic signal with the highest (i=1) priority level among the N traffic signals. When the traffic signal is successfully detected (YES in S25), the flowchart proceeds to Step S27 to output the traffic signal information D04. Thereafter, without changing the priority level (Step S29), the flowchart returns to Step S25. This allows the traffic signal to be continuously detected within the same detection area when the traffic signal is successfully detected (YES in S25).

[0040] When the traffic signal detection is unsuccessful (NO in S25), on the other hand, the flowchart proceeds to Step S31 to determine whether the current priority level (i) is N, i.e., whether the current priority level (i) is the lowest level. When the current priority level (i) is not N (NO in S31), there is still a detection area for which detection is yet to be attempted. Then, the current priority level (i) is incremented by one (Step S35), i.e., the current priority level is lowered by one, and the flowchart returns to Step S25. When the current priority level (i) is N (YES in S31), on the other hand, it means that detection has been attempted for all the detection areas. Then, after setting the highest priority level as the current priority level (i) again (Step S33), the flowchart returns to Step S25.

[0041]  The first embodiment described above offers the following advantageous effects.

[0042] A traffic signal is, in some cases, hidden and invisible from a vehicle due to a factor unknowable from map information, such as a preceding vehicle, a street tree, or road work. In addition, depending on the positions of the traffic signals relative to the road, traffic signals differ in their likeliness of being captured in an image, i.e., possibility of being blocked out of sight by an obstacle around the vehicle. Considering such circumstances, when it is predicted based on the map information D02 and the self-position information D05 that two or more traffic signals may be captured in an image, the two or more traffic signals are assigned priority levels (i). Then, detection in the image is started with a traffic signal having the highest priority level (i). This increases the probability of detecting a traffic signal when it is predicted that two or more traffic signals may be captured in the image. In addition, early detection of a traffic signal as well as decrease in the computation load can be achieved.

[0043] As described in regard to the basic rule (R1), the priority level assigner 24 assigns a higher priority level to a traffic signal located on the oncoming lane (PL) than to a traffic signal located on the current lane (WL) of travel of the vehicle. This allows preferential detection of a traffic signal which is less likely to be blocked out of sight or is likely to be blocked out of sight for a short period of time.

[0044] Typically, an advance warning traffic signal is provided to give notification of a road or traffic condition ahead of time. Thus, even at a position where a regular traffic signal may be blocked out of sight, an advance warning traffic signal is less likely to be blocked out of sight. For this reason, when the traffic signal identifier 23 identifies an advance warning traffic signal as any of the two or more traffic signals predicted to be captured in an image, the priority level assigner 24 gives a higher priority level to the advance warning traffic signal than to the other traffic signals. This allows even earlier detection of the traffic signals.

[0045] As shown in Fig. 6, after succeeding in detecting a traffic signal with the highest priority level (YES in S25), the traffic signal detector 15 continues to detect the traffic signal. This allows stable, continuous detection of the traffic signal.

[0046] It is sometimes faster to detect a traffic signal with the second highest priority level than to wait for detection of a traffic signal with the highest priority level. Thus, as shown in Fig. 6, the traffic signal detector 15 detects a traffic signal with the second highest priority level when unable to detect a traffic signal with the highest priority level (NO in S25). This enables early detection of a traffic signal. In addition, in the flowchart in Fig. 6, the traffic signal detector 15 continues to detect the traffic signal with the second highest priority level. This enables stable, continuous detection of the traffic signal.

(Second Embodiment)



[0047] In the flowchart in Fig. 6, as long as a traffic signal with a high priority level is being successfully detected, the traffic signal detector 15 continues to detect that traffic signal and does not detect other traffic signals with low priority levels. In a second embodiment described below, a traffic signal recognition apparatus and a traffic signal recognition method detect, irrespective of whether traffic signal detection is successful, all the two or more traffic signals identified by the traffic signal identifier 23, sequentially from a traffic signal with the highest priority level.

[0048] The second embodiment is different from the first embodiment in the procedure of Step S13 in Fig. 5 which is shown in detail in Fig. 6, but is the same as the first embodiment in the configuration of the traffic signal recognition apparatus 100 shown in Fig. 2 and the procedures of Steps S01 to S11 in Fig. 5. Description is omitted for the configuration and procedures which are the same as those of the first embodiment.

[0049]  With reference to Fig. 7, a detailed procedure of Step S13 in Fig. 5 is described. First, like in Fig. 6, image data within the detection areas for the respective N traffic signals are extracted in Step S21, and the priority levels (i) assigned to the traffic signals are mapped onto the extracted detection areas in Step S23.

[0050] In Step S25, like in Fig. 6, traffic signal detection is attempted, starting with a detection area corresponding to the highest priority level (i=1). When the traffic signal detection is unsuccessful (NO in S25), the flowchart proceeds to Step S31. When the traffic signal detection is successful (YES in S25), the flowchart proceeds to Step S27 to output the traffic signal information D04, and then, unlike in Fig. 6, proceeds to Step S31.

[0051] In Step S31, like in Fig. 6, it is determined whether the current priority level (i) is N. When the current priority level (i) is not N (NO in S31), the current priority level (i) is incremented by one (Step S35), and the flowchart returns to Step S25. When the current priority level (i) is N (YES in S31), on the other hand, the highest priority level is set as the current priority level (i) again (Step S33), and the flowchart returns to Step S25.

[0052] In this way, when a traffic signal is successfully detected (YES in S25), instead of continuing to detect that traffic signal from the same detection area, detection for the next traffic signal is attempted. In other words, the traffic signal detector 15 detects the two or more traffic signals sequentially from the one with the highest priority level, irrespective of whether the traffic signal detection is successful. Using detection results of all the identified traffic signals for signal recognition enables prevention of erroneous recognition of traffic signals. For this reason, the two or more traffic signals are detected sequentially from the one with the highest priority level, irrespective of whether traffic signal detection is successful. This increases accuracy of traffic signal recognition.

[Third Embodiment]



[0053] The possibility of a traffic signal being blocked out of sight is affected by geographic information including the shape of a road on which a vehicle is travelling (such as a straight road, a curved road, or a gradient), the presence of street trees, and the presence of an obstacle such as road work. In a third embodiment described below, a traffic signal recognition apparatus and a traffic signal recognition method take geographic information into account when assigning priority levels.

[0054] The third embodiment is different from the first embodiment in the configuration of a priority level assigner 24b and Step S07 in Fig. 5, but is the same as the first embodiment in other configurations of the traffic signal recognition apparatus 100 and other steps in Fig. 5. Description is omitted for the configurations and steps which are the same as those of the first embodiment.

[0055] With reference to Fig. 8, the configuration of the priority level assigner 24b according to the third embodiment is described. The priority level assigner 24b includes a basic level assigner 31 and a first level changer 32. The basic level assigner 31 performs the same arithmetic processing as the priority level assigner 24 in Fig. 3. Specifically, the basic level assigner 31 assigns priority levels to the two or more traffic signals identified by the traffic signal identifier 23, in conformity with the basic rules (R1) and (R2). The priority levels assigned by the basic level assigner 31 are called "basic priority levels".

[0056] The first level changer 32 changes the basic priority levels based on geographic information. The changed priority levels are outputted as priority levels assigned by the priority level assigner 24b. To be more specific, as shown in Fig. 9, the first level changer 32 determines the shape of the road RD extending from the current position (self-position) of the vehicle 51 to the intersection Crs at which a traffic signal to be detected is located. For example, the first level changer 32 refers to the map information D02 and determines whether the shape of the road RD on which the vehicle 51 is travelling is a curve curving toward the oncoming lane (PL) from the vehicle 51 in a travelling direction DD.

[0057] When determining that the road RD is a curve curving toward the oncoming lane (PL), the first level changer 32 changes the basic priority levels in conformity with an alteration rule (V1) described below.

[0058] (V1) Assign a higher priority level to a traffic signal located near the current lane (WL) of travel of the vehicle 51 than to a traffic signal located near the oncoming lane (PL).

[0059] Of the current lane (WL) and the oncoming lane (PL), a traffic signal located on the opposite side of the turning direction of the vehicle 51 can be detected from a position farther away from an intersection than a traffic signal located on the same side as the turning direction of the vehicle 51. Accordingly, when the road curves toward the oncoming lane (PL), a traffic signal near the current lane (WL) can be detected earlier than a traffic signal near the oncoming lane (PL). Thus, the alteration rule (VI) is applied to reverse the priority relation in the basic rule (R1).

[0060] According to the alteration rule (VI) described above, the priority levels of three traffic signals (Ts1, Ts2, Ts3) in Fig. 9 are, from highest to lowest, the traffic signal Ts2, the traffic signal Ts3, and the traffic signal Ts1. Specifically, the traffic signal Ts2 and the traffic signal Ts3 which are located near the current lane (WL) are assigned higher priority levels than the traffic signal Ts1 located near the oncoming lane (PL). The basic rule (R2) still applies even when the alteration rule (V1) is applied. Thus, the traffic signal Ts2 on the near side of the intersection is assigned a higher priority level than the traffic signal Ts3 located on the far side of the intersection Crs.

[0061] The geographic information taken into account by the first level changer 32 may be the shape (gradient) of a road other than the curve, or an obstacle such as a street tree or road work. In such cases, the traffic signal recognition apparatus 100 learns gradient information on the road and obstacle information from travelling data or the like, and adds that information as part of the map information D02. Based on the gradient information and the obstacle information, the first level changer 32 predicts which traffic signal is least likely to be blocked out of sight, and changes the basic priority levels. Although the example in Fig. 9 shows a right curve with the vehicle driving on the left, the third embodiment is of course applicable to a left curve with the vehicle driving on the right.

[0062] As described, when the shape of the road RD on which the vehicle 51 is travelling is a curve that curves toward the oncoming lane PL from the vehicle 51's view, the priority level assigner 24b assigns a higher priority level to the traffic signal Ts1 located near the oncoming lane PL than to the traffic signals (Ts2, Ts3) located near the current lane WL. This enables the priority levels to be assigned appropriately in accordance with the road shape.

[Fourth Embodiment]



[0063] In a fourth embodiment describe below, a traffic signal recognition apparatus and a traffic signal recognition method attempt to detect traffic signals before priority level assignment, and assign priority levels to the traffic signals that are detected.

[0064] The fourth embodiment is different from the first embodiment in the configuration of a priority level assigner 24c and Step S07 in Fig. 5, but is the same as the first embodiment in other configurations of the traffic signal recognition apparatus 100 and other steps in Fig. 5. Description is omitted for the configurations and steps which are the same as that of the first embodiment.

[0065] With reference to Fig. 8, the configuration of the priority level assigner 24c according to the fourth embodiment is described. The priority level assigner 24c includes the basic level assigner 31 and a second level changer 33. The basic level assigner 31 performs the same arithmetic processing as the priority level assigner 24 in Fig. 3. Specifically, the basic level assigner 31 assigns basic priority levels to the two or more traffic signals identified by the traffic signal identifier 23, in conformity with the basic rules (R1) and (R2).

[0066] The second level changer 33 changes the basic priority levels based on results of traffic signal detection obtained by the traffic signal detector 15. The changed priority levels are outputted as priority levels assigned by the priority level assigner 24c. It is difficult to predict a traffic signal that is blocked out of sight due to a factor unknowable from map information, such as a traffic environment and a road condition, based only on the positional information on the traffic signal. It is desirable that an attempt for detection be made for all the traffic signals identified by the traffic signal identifier 23 and that results of the detection be fed back into the priority levels. Thus, the traffic signal detector 15 makes an attempt for detection for all the traffic signals in advance, and then the second level changer 33 assigns priority levels to the traffic signals that are successfully detected.

[0067] With reference to Fig. 11, a detailed procedure in Step S07 in Fig. 5 is described. First, in Step S21, image data within the detection areas for the respective N traffic signals are extracted, and in Step S23, the priority levels (i) assigned to the traffic signals are mapped onto the extracted detection areas.

[0068] In Step S25, an attempt is made for traffic signal detection in a detection area that corresponds to a traffic signal with the highest priority level (i=1). When the traffic signal is successfully detected (YES in S25), the flowchart proceeds to Step S39 to give the traffic signal a flag indicative of successful detection. This enables the traffic signal recognition apparatus to learn that this traffic signal is detectable. When the traffic signal is not successfully detected (NO in S25), the flowchart proceeds to Step S37 not to give the successful detection flag. This enables the traffic signal recognition apparatus to learn that this traffic signal is undetectable.

[0069] Thereafter, the flowchart proceeds to Step S31 to determine whether the current priority level (i) is N. When the current priority level (i) is not N (NO in S31), the current priority level (i) is incremented by one (Step S35), and the flowchart returns to Step S25. When the current priority level (i) is N (YES in S31), it means that an attempt for detection has been made for all the traffic signals. Thus, the flowchart proceeds to Step S41 to assign priority levels to the flagged traffic signals. This way, priority levels can be reassigned to the detectable traffic signals extracted from the traffic signals assigned basic priority levels by the basic level assigner 31.

[0070] Fig. 12 shows an example of a road condition where, on the road RD which is a rightward curve, a street tree OBT is blocking a part of the traffic signal Ts1, making the part of the traffic signal Ts1 invisible from the vehicle 51. In this condition, the basic level assigner 31 assigns the basic priority levels to, from highest to lowest, the traffic signal Ts1, the traffic signal Ts2, and the traffic signal Ts3, in conformity with the basic rules (R1) and (R2). Then, the second level changer 33 receives detection results indicating that the traffic signal Ts2 and the traffic signal Ts3 are flagged and the traffic signal Ts1 is not flagged. Based on this detection results, the second level changer 33 assigns priority levels to the traffic signal Ts2 and the traffic signal Ts3, and not to the traffic signal Ts1. As the rules used for the assignment of the priority levels, the second level changer 33 may apply the basic rules (R1) and (R2), or additionally, the alteration rule (V1).

[0071] It is difficult to predict a traffic signal that is blocked out of sight due to a factor unknowable from map information, such as a traffic environment and a road condition, based only on the positional information on the traffic signal. It is desirable that an attempt for detection be made for all the traffic signals identified by the traffic signal identifier 23 and that results of the detection be fed back into the priority levels. Thus, the traffic signal detector makes an attempt for detection for all the traffic signals, and the priority level assigner assigns priority levels to the traffic signals that are successfully detected. This allows elimination of undetectable traffic signals from priority-level assignment targets, and therefore allows early detection of detectable traffic signals.

[0072] Although the present invention has been described above using the embodiments, the present invention is not limited to what is described hereinabove, and can be modified and improved variously, in the scope of the claims.

[0073] For example, the first embodiment describes the basic rule for assigning a higher priority level to an advance warning traffic signal than to other traffic signals. This basic rule is applied to the other embodiments, as well. Specifically, the first level changer 32 assigns a higher priority level to an advance warning traffic signal than to other traffic signals, irrespective of whether the advance warning traffic signal is located near the oncoming lane or the current lane. Alternatively, the second level changer 33 assigns a higher priority level to an advance warning traffic signal than to other traffic signals when the advance warning traffic signal is flagged.

REFERENCE SIGNS LIST



[0074] 
11
imager
12
self-position detector
13
traffic signal position estimator
23
traffic signal identifier
24
priority level assigner
15
traffic signal detector
100
traffic signal recognition apparatus
D02
map information
PL
oncoming lane
RD
road
Ts1, Ts2, Ts3
traffic signal
WL
current lane



Claims

1. A traffic signal recognition apparatus (100) comprising:

an imager (11) mounted in a vehicle and configured to acquire an image by imaging surroundings of the vehicle;

a self-position detector (12) configured to detect a self-position of the vehicle;

a traffic signal detector (15) configured to detect, in the image, a traffic signal (Ts1, Ts2, Ts3) around the vehicle;

a traffic signal position estimator (13) configured to estimate a relative position of the traffic signal (Ts1, Ts2, Ts3) with respect to the vehicle, using the self-position and map information (D02) containing positional information on the traffic signals (Ts1, Ts2, Ts3),

a traffic signal identifier (23) configured to identify two or more of the traffic signals (Ts1, Ts2, Ts3) around the vehicle that are predicted to be captured in the image, using the estimated relative position;

characterized by:

a priority level assigner (24) configured to assign a priority level to each of the two or more traffic signals (Ts1, Ts2, Ts3) identified by the traffic signal identifier (23), using a possibility of the traffic signal (Ts1, Ts2, Ts3) being blocked out of sight that is derived from the positional information on the traffic signal, wherein

the two or more traffic signals (Ts1, Ts2, Ts3) have different relative positions with respect to each other, and

when the traffic signal identifier (23) identifies the two or more traffic signals (Ts1, Ts2, Ts3), the traffic signal detector (15) detects, in the image, a traffic signal (Ts1) with a highest priority level among the two or more traffic signals (Ts1, Ts2, Ts3), wherein after succeeding in detecting a traffic signal with the highest priority level, the traffic signal detector (15) is configured to continue detecting the traffic signal (Ts1) and the traffic signal detector (15) is configured to perform image processing for traffic signal detection on the image within a detection area set for the traffic signal having the highest priority level.


 
2. The traffic signal recognition apparatus (100) according to claim 1, wherein
the priority level assigned by the priority level assigner (24) is higher for a traffic signal (Ts1) located near an oncoming lane (PL) than for a traffic signal (Ts2, Ts3) located near a current lane (WL) of travel of the vehicle.
 
3. The traffic signal recognition apparatus (100) according to claim 1, wherein
when a shape of a road (RD) on which the vehicle is travelling is a curve curving toward an oncoming lane (PL) from a viewpoint of the vehicle, the priority level assigned by the priority level assigner (24) is higher for a traffic signal (Ts2, Ts3) located near a current lane (WL) of travel of the vehicle than for a traffic signal (Ts1) located near the oncoming lane (PL).
 
4. The traffic signal recognition apparatus (100) according to any one of claims 1 to 3, wherein
when the traffic signal identifier (23) identifies an advance warning traffic signal as any of the two or more traffic signals predicted to be captured in the image, the priority level assigned by the priority level assigner (24) is higher for the advance warning traffic signal than for a rest of the traffic signals.
 
5. The traffic signal recognition apparatus (100) according to any one of claims 1 to 4, wherein
when successfully detecting the traffic signal (Ts1) with the highest priority level, the traffic signal detector (15) continues to detect the traffic signal (Ts1).
 
6. The traffic signal recognition apparatus (100) according to any one of claims 1 to 5, wherein
when not successfully detecting the traffic signal (Ts1) with the highest priority level, the traffic signal detector (15) detects a traffic signal (Ts2) with a second highest priority level.
 
7. The traffic signal recognition apparatus (100) according to any one of claims 1 to 4, wherein
the traffic signal detector (15) detects the two or more traffic signals (Ts1, Ts2, Ts3) sequentially from the traffic signal (Ts1) with the highest priority level, irrespective of whether the traffic signal (Ts1, Ts2, Ts3) is successfully detected or not.
 
8. The traffic signal recognition apparatus (100) according to any one of claims 1 to 4, wherein
the traffic signal detector (15) attempts to detect each of the traffic signals (Ts1, Ts2, Ts3) which are identified by the traffic signal identifier (23), before the priority level assigner (24) assigns a priority level,
the priority level assigner (24) assigns the priority level to each of the traffic signals (Ts1, Ts2, Ts3) which are identified by the traffic signal identifier (23) and successfully detected, and
the traffic signal detector (15) detects, in the image, the traffic signal (Ts1) with the highest priority level among the two or more traffic signals (Ts1, Ts2, Ts3) after the priority level is assigned.
 
9. A traffic signal recognition method comprising:

acquiring an image by imaging surroundings of a vehicle using an imager (11) mounted in the vehicle;

detecting a self-position of the vehicle;

estimating a relative position of the traffic signal (Ts1, Ts2, Ts3) with respect to the vehicle, using the self-position and map information (D02) containing positional information on the traffic signals (Ts1, Ts2, Ts3),

identifying two or more of the traffic signals (Ts1, Ts2, Ts3) around the vehicle that are predicted to be captured in the image, using the estimated relative positions; and

assigning a priority level to each of the two or more traffic signals (Ts1, Ts2, Ts3) thus identified, using a possibility of the traffic signal (Ts1, Ts2, Ts3) being blocked out of sight that is derived from the positional information on the traffic signals, wherein

the two or more traffic signals (Ts1, Ts2, Ts3) have different relative positions with respect to each other, and

when the two or more traffic signals (Ts1, Ts2, Ts3) are identified, the traffic signal detected in the image is a traffic signal (Ts1) with a highest priority level among the two or more traffic signals (Ts1, Ts2, Ts3), wherein after succeeding in detecting a traffic signal with the highest priority level, the traffic signal detector (15) continues to detect the traffic signal (Ts1) and the traffic signal detector (15) performs image processing for traffic signal detection on the image within a detection area set for the traffic signal having the highest priority level.


 


Ansprüche

1. Eine Vorrichtung zur Erkennung eines Verkehrssignals (100) umfassend:

einen Bildgeber (11), der in einem Fahrzeug montiert und konfiguriert ist, um ein Bild durch Abbilden der Umgebung des Fahrzeugs zu erfassen;

einen Selbstpositionsdetektor (12), der konfiguriert ist, um eine Selbstposition des Fahrzeugs zu erfassen;

einen Verkehrssignaldetektor (15), der konfiguriert ist, um in dem Bild ein Verkehrssignal (Ts1, Ts2, Ts3) um das Fahrzeug herum zu erfassen;

einen Verkehrssignalpositionsschätzer (13), der konfiguriert ist, um eine relative Position des Verkehrssignals (Ts1, Ts2, Ts3) in Bezug auf das Fahrzeug unter Verwendung der Selbstposition und Karteninformationen (D02), welche Positionsinformationen zu den Verkehrssignalen (Ts1, Ts2, Ts3) enthalten, zu schätzen,

ein Verkehrssignalidentifikator (23), die konfiguriert ist, um zwei oder mehr der Verkehrssignale (Ts1, Ts2, Ts3) um das Fahrzeug herum zu identifizieren, von denen unter Verwendung der geschätzten relativen Position, vorhergesagt wird, dass sie in dem Bild erfasst werden,;

gekennzeichnet durch:

einen Prioritätsniveauzuweiser (24), der konfiguriert ist, um jedem der zwei oder mehr Verkehrssignalen (Ts1, Ts2, Ts3), die durch die Verkehrssignalidentifikation (23) identifiziert werden, ein Prioritätsniveau zuzuweisen, unter Verwendung einer Wahrscheinlichkeit dass die Sicht auf das Verkehrssignal (Ts1, Ts2, Ts3), die aus der Positionsinformation der Verkehrssignale abgeleitet wird, gesperrt wird, wobei

die zwei oder mehr Verkehrssignale (Ts1, Ts2, Ts3) haben unterschiedliche relative Positionen zueinander und

wenn der Verkehrssignalidentifikator (23) die zwei oder mehr Verkehrssignale (Ts1, Ts2, Ts3) identifiziert, detektiert der Verkehrssignaldetektor (15) in dem Bild ein Verkehrssignal (Ts1) mit dem höchsten Prioritätsniveau unter den zwei oder mehr Verkehrssignalen (Ts1, Ts2, Ts3), wobei der Verkehrssignaldetektor (15) nach dem erfolgreichen Erfassen eines Verkehrssignals mit dem höchsten Prioritätsniveau so konfiguriert ist, dass er das Verkehrssignal (Ts1) weiterhin erfasst und der Verkehrssignaldetektor (15) konfiguriert ist, um eine Bildverarbeitung zur Verkehrssignalerfassung an dem Bild innerhalb eines Erfassungsbereichs durchzuführen, der für das Verkehrssignal mit dem höchsten Prioritätsniveau eingestellt ist.


 
2. Die Vorrichtung zur Erkennung eines Verkehrssignals (100) gemäß Anspruch 1, wobei
das vom Prioritätsniveauzuweiser (24) zugewiesene Prioritätsniveau für ein Verkehrssignal (Ts1) in der Nähe einer Gegenfahrspur (PL) höher ist als für ein Verkehrssignal (Ts2, Ts3) in der Nähe einer aktuellen Fahrspur (WL) des Fahrzeugs.
 
3. Die Vorrichtung zur Erkennung eines Verkehrssignals (100) gemäß Anspruch 1, wobei
wenn eine Form einer Straße (RD), auf der das Fahrzeug fährt, eine Kurve ist, die sich vom Standpunkt des Fahrzeugs aus in Richtung einer Gegenfahrspur (PL) krümmt, ist das vom Prioritätsniveauzuweiser (24) zugewiesene Prioritätsniveau für eine Verkehrssignal (Ts2, Ts3) in der Nähe einer aktuellen Fahrspur (WL) des Fahrzeugs höher als für ein Verkehrssignal (Ts1) in der Nähe der Gegenfahrspur (PL).
 
4. Die Vorrichtung zur Erkennung eines Verkehrssignals (100) gemäß einem der Ansprüche 1 bis 3, wobei
wenn der Verkehrssignalidentifikator (23) ein sich näherndes Verkehrssignal als eines der zwei oder mehr Verkehrssignale identifiziert, von denen vorhergesagt wird, dass sie in dem Bild erfasst werden, ist das vom Prioritätsniveauzuweiser (24) zugewiesene Prioritätsniveau für das sich nähernde Verkehrssignal höher als für einen Rest der Verkehrssignale.
 
5. Die Vorrichtung zur Erkennung eines Verkehrssignals (100) gemäß einem der Ansprüche 1 bis 4, wobei
der Verkehrssignaldetektor (15) bei erfolgreicher Erkennung des Verkehrssignals (Ts1) mit dem höchsten Prioritätsniveau die Erkennung des Verkehrssignals (Ts1) fortsetzt.
 
6. Die Vorrichtung zur Erkennung eines Verkehrssignals (100) gemäß einem der Ansprüche 1 bis 5, wobei
wenn das Verkehrssignal (Ts1) mit dem höchsten Prioritätsniveau nicht erfolgreich erfasst wird, erfasst der Verkehrssignaldetektor (15) ein Verkehrssignal (Ts2) mit dem zweithöchsten Prioritätsniveau.
 
7. Die Vorrichtung zur Erkennung eines Verkehrssignals (100) gemäß einem der Ansprüche 1 bis 4, wobei
der Verkehrssignaldetektor (15) die zwei oder mehr Verkehrssignale (Ts1, Ts2, Ts3) nacheinander aus dem Verkehrssignal (Ts1) mit dem höchsten Prioritätsniveau erkennt, unabhängig davon, ob das Verkehrssignal (Ts1, Ts2, Ts3) erfolgreich erkannt wurde oder nicht.
 
8. Die Vorrichtung zur Erkennung eines Verkehrssignals (100) gemäß einem der Ansprüche 1 bis 4, wobei
der Verkehrssignaldetektor (15) versucht, jedes der durch den Verkehrssignalidentifikator (23) identifizierten Verkehrssignale (Ts1, Ts2, Ts3) zu erfassen, bevor der Prioritätsniveauzuweiser (24) ein Prioritätsniveau zuweist,
der Prioritätsniveauzuweiser (24) das Prioritätsniveau jedem der Verkehrssignale (Ts1, Ts2, Ts3) zuordnet, die durch den Verkehrssignalidentifikator (23) identifiziert und erfolgreich erkannt werden, und
der Verkehrssignaldetektor (15) in dem Bild das Verkehrssignal (Ts1) mit dem höchsten Prioritätsniveau unter den zwei oder mehr Verkehrssignalen (Ts1, Ts2, Ts3) erkennt, nachdem das Prioritätsniveau zugewiesen wurde.
 
9. Eine Methode zur Erkennung eines Verkehrssignals umfassend:

Erfassen eines Bildes durch Abbilden der Umgebung eines Fahrzeugs unter Verwendung eines im Fahrzeug montierten Bildgebers (11);

Erfassen einer Selbstposition des Fahrzeugs;

Schätzen einer relativen Position des Verkehrssignals (Ts1, Ts2, Ts3) in Bezug auf das Fahrzeug unter Verwendung der Selbstposition und Karteninformationen (D02), die Positionsinformationen zu den Verkehrssignalen (Ts1, Ts2, Ts3) enthalten,

Identifizieren von zwei oder mehr der Verkehrssignale (Ts1, Ts2, Ts3) um das Fahrzeug, von denen vorhergesagt wird, dass sie in dem Bild erfasst werden, unter Verwendung der geschätzten relativen Positionen; und

Zuweisen eines Prioritätsniveaus zu jedem der zwei oder mehr so identifizierten Verkehrssignale (Ts1, Ts2, Ts3) unter Verwendung einer Wahrscheinlichkeit, dass die Sicht auf das Verkehrssignal (Ts1, Ts2, Ts3) gesperrt wird, welche aus den Positionsinformationen der Verkehrszeichen abgeleitet wird, wobei

die zwei oder mehr Verkehrssignale (Ts1, Ts2, Ts3) haben unterschiedliche relative Positionen zueinander und

wenn die zwei oder mehr Verkehrssignale (Ts1, Ts2, Ts3) identifiziert werden, ist das in dem Bild erfasste Verkehrssignal ein Verkehrssignal (Ts1) mit dem höchsten Prioritätsniveau unter den zwei oder mehr Verkehrssignalen (Ts1, Ts2, Ts3), wobei nach dem erfolgreichen Erfassen eines Verkehrssignals mit dem höchsten Prioritätsniveau der Verkehrssignaldetektor (15) das Verkehrssignal (Ts1) weiterhin erfasst wird und der Verkehrssignaldetektor (15) eine Bildverarbeitung zur Verkehrssignaldetektion an dem Bild innerhalb einem Erfassungsbereich durchführt, für den das Verkehrssignal mit dem höchsten Prioritätsniveau eingestellt ist.


 


Revendications

1. Appareil de reconnaissance de feux de circulation (100) comportant :

un dispositif d'imagerie (11) monté dans un véhicule et configuré pour acquérir une image en capturant une image de l'environnement du véhicule ;

un détecteur de position propre (12) configuré pour détecter une position propre au véhicule ;

un détecteur de feu de circulation (15) configuré pour détecter, dans l'image, un feu de circulation (Ts1, Ts2, Ts3) autour du véhicule ;

un dispositif d'estimation de la position du feu de circulation (13) configuré pour estimer une position relative du feu de circulation (Ts1, Ts2, Ts3) par rapport au véhicule en utilisant la position propre et des informations cartographiques (D02) contenant des informations de position des feux de circulation (Ts1, Ts2, Ts3),

un dispositif d'identification de feu de circulation (23) configuré pour identifier deux ou plusieurs des feux de circulation (Ts1, Ts2, Ts3) autour du véhicule qui devraient être capturés dans l'image en utilisant la position relative estimée ;

caractérisé par :

un dispositif d'attribution de niveau de priorité (24) configuré pour attribuer un niveau de priorité à chacun des deux ou plusieurs feux de circulation (Ts1, Ts2, Ts3) identifiés par le dispositif d'identification de feu de circulation (23) en prenant en compte la possibilité que le feu de circulation (Ts1, Ts2, Ts3) est caché, découlant de l'information de position du feu de circulation, où

les deux ou plusieurs feux de circulation (Ts1, Ts2, Ts3) ont des positions relatives différentes les uns par rapport aux autres, et

quand le dispositif d'identification de feu de circulation (23) identifie les deux ou plusieurs feux de circulation (Ts1, Ts2, Ts3), le détecteur de feu de circulation (15) détecte dans l'image un feu de circulation (Ts1) ayant le plus haut niveau de priorité parmi les deux ou plusieurs feux de circulation (Ts1, Ts2, Ts3), où après avoir réussi à détecter un feu de circulation ayant le plus haut niveau de priorité, le détecteur de feu de circulation (15) est configuré pour continuer à détecter le feu de circulation (Ts1) et le détecteur de feu de circulation (15) est configuré pour réaliser un traitement d'image pour la détection d'un feu de circulation dans l'image, à l'intérieur d'une zone de détection réglée pour le feu de circulation ayant le plus haut niveau de priorité.


 
2. Appareil de reconnaissance de feu de circulation (100) selon la revendication 1, où
le niveau de priorité attribué par le dispositif d'attribution de niveau de priorité (24) est plus élevé pour un feu de circulation (Ts1) situé près d'une voie en sens contraire (PL) que pour un feu de circulation (Ts2, Ts3) situé près d'une voie actuelle (WL) de déplacement du véhicule.
 
3. Appareil de reconnaissance de feu de circulation (100) selon la revendication 1, où
quand une route (RD) sur laquelle le véhicule se déplace forme une courbe vers une voie en sens contraire (PL) du point de vue du véhicule, le niveau de priorité attribué par le dispositif d'attribution de niveau de priorité (24) est plus élevé pour un feu de circulation (Ts2, Ts3) situé près d'une voie actuelle (WL) de circulation du véhicule que pour un feu de circulation (Ts1) situé près de la voie en sens contraire (PL).
 
4. Appareil de reconnaissance de feu de circulation (100) selon l'une quelconque des revendications 1 à 3, où
quand le dispositif d'identification de feu de circulation (23) identifie un feu de circulation d'avertissement avancé en tant que l'un des deux ou plusieurs feux de circulation qui devraient être capturés dans l'image, le niveau de priorité attribué par le dispositif d'attribution de niveau de priorité (24) est plus élevé pour le feu de circulation d'avertissement avancé que pour le reste des feux de circulation.
 
5. Appareil de reconnaissance de feu de circulation (100) selon l'une quelconque des revendications 1 à 4, où
quand le feu de circulation (Ts1) avec le plus haut niveau de priorité est détecté avec succès, le détecteur de feu de circulation (15) continue de détecter le feu de circulation (Ts1).
 
6. Appareil de reconnaissance de feu de circulation (100) selon l'une quelconque des revendications 1 à 5, où
quand le feu de circulation (Ts1) avec le plus haut niveau de priorité n'est pas détecté avec succès, le détecteur de feu de circulation (15) détecte un feu de circulation (Ts2) ayant le deuxième niveau de priorité le plus élevé.
 
7. Appareil de reconnaissance de feu de circulation (100) selon l'une quelconque des revendications 1 à 4, où
le détecteur de feu de circulation (15) détecte séquentiellement les deux ou plusieurs feux de circulation (Ts1, Ts2, Ts3) à partir du feu de circulation (Ts1) ayant le plus haut niveau de priorité, que le feu de signalisation (Ts1, Ts2, Ts3) soit détecté avec succès ou non.
 
8. Appareil de reconnaissance de feu de circulation (100) selon l'une quelconque des revendications 1 à 4, où
le détecteur de feu de circulation (15) tente de détecter chacun des feux de circulation (Ts1, Ts2, Ts3) qui sont identifiés par le dispositif d'identification de feux de circulation (23) avant que le dispositif d'attribution de niveau de priorité (24) n'attribue un niveau de priorité,
le dispositif d'attribution de niveau de priorité (24) attribue le niveau de priorité à chacun des feux de circulation (Ts1, Ts2, Ts3) qui sont identifiés par le dispositif d'identification de feu de circulation (23) et détectés avec succès, et
le détecteur de feu de circulation (15) détecte, dans l'image, le feu de circulation (Ts1) avec le niveau de priorité le plus élevé parmi les deux ou plusieurs feux de circulation (Ts1, Ts2, Ts3), après que le niveau de priorité a été attribué.
 
9. Procédé de reconnaissance de feu de circulation comportant les étapes consistant à :

acquérir une image en capturant une image de l'environnement d'un véhicule en utilisant un dispositif d'imagerie (11) monté dans le véhicule ;

détecter une position propre au véhicule ;

estimer une position relative du feu de circulation (Ts1, Ts2, Ts3) par rapport au véhicule en utilisant la position propre et des informations cartographiques (D02) contenant des informations de position des feux de circulation (Ts1, Ts2, Ts3),

identifier deux ou plusieurs des feux de circulation (Ts1, Ts2, Ts3) autour du véhicule qui devraient être capturés dans l'image en utilisant les positions relatives estimées ; et

attribuer un niveau de priorité à chacun des deux ou plusieurs feux de circulation (Ts1, Ts2, Ts3) ainsi identifiés, en prenant en compte la possibilité que le feu de circulation (Ts1, Ts2, Ts3) est caché, découlant de l'information de position du feu de circulation, où

les deux ou plusieurs feux de circulation (Ts1, Ts2, Ts3) ont des positions relatives différentes les uns par rapport aux autres, et

quand les deux ou plusieurs feux de circulation (Ts1, Ts2, Ts3) sont identifiés, le feu de circulation détecté dans l'image est un feu de circulation (Ts1) ayant le plus haut niveau de priorité parmi les deux ou plusieurs feux de circulation (Ts1, Ts2, Ts3), où après avoir réussi à détecter un feu de circulation ayant le plus haut niveau de priorité, le détecteur de feu de circulation (15) continue à détecter le feu de circulation (Ts1) et le détecteur de feu de circulation (15) réalise un traitement d'image pour la détection d'un feu de circulation dans l'image, à l'intérieur d'une zone de détection réglée pour le feu de circulation ayant le plus haut niveau de priorité.


 




Drawing





























Cited references

REFERENCES CITED IN THE DESCRIPTION



This list of references cited by the applicant is for the reader's convenience only. It does not form part of the European patent document. Even though great care has been taken in compiling the references, errors or omissions cannot be excluded and the EPO disclaims all liability in this regard.

Patent documents cited in the description