|
(11) | EP 4 350 656 A1 |
| (12) | EUROPEAN PATENT APPLICATION |
|
|
|
|
||||||||||||||||||||||||
|
|||||||||||||||||||||||||
| (54) | METHOD FOR DETECTING AN INATTENTIVE PEDESTRIAN CROSSING A ROADWAY, METHOD FOR OPERATING A FULLY OR PARTIALLY AUTONOMOUS VEHICLE, METHOD FOR INFORMING A CENTRAL TRAFFIC CONTROL ENTITY ABOUT AN INATTENTIVE PEDESTRIAN, METHOD FOR CONTROLLING A TRAFFIC SYSTEM, DATA PROCESSING APPARATUS AND TRAFFIC CONTROL SYSTEM |
| (57) The disclosure relates to a method for detecting an inattentive pedestrian (12) crossing
a roadway (R). According to the method, a pedestrian crossing (P) on the roadway (R)
is detected or a pedestrian crossing location information (PCL) is received. Additionally,
a head pose of a pedestrian (12) is detected. It is inferred that the pedestrian (12)
is inattentive if the head pose describes that the pedestrian (12) is looking downwards.
Moreover, a method for operating a vehicle (12) is described. The method comprises
performing the above method for detecting an inattentive pedestrian (12). The method
additionally comprises providing an inattentive pedestrian status information (IPS)
to a central traffic control entity (16). Furthermore, a method for informing a central
traffic control entity (16) about an inattentive pedestrian (12) is described. The
pedestrian (12) is crossing a roadway (R) and carries a mobile electronic device (14).
Beyond that, a method for controlling a traffic system is presented. Moreover, a data
processing apparatus (18, 34, 46) and a traffic control system (58) are described.
|
[0001] The present disclosure relates to a method for detecting an inattentive pedestrian crossing a roadway.
[0002] Additionally, the present disclosure is directed to a method for operating a fully or partially autonomous vehicle.
[0003] Moreover, the present disclosure relates to a method for informing a central traffic control entity about an inattentive pedestrian.
[0004] The present disclosure is also directed to a method for controlling a traffic system. The traffic system comprises at least one fully or partially autonomous vehicle and at least one pedestrian.
[0005] The present disclosure is additionally directed to a data processing apparatus comprising means for carrying out at least one of the above methods and a traffic control system.
[0006] Inattentive pedestrians are a frequent cause of accidents. In this context, pedestrians are for example distracted by interacting with a mobile electronic device such as a smart phone or a smart watch. While human drivers usually have a certain ability to distinguish between attentive pedestrians and inattentive pedestrians, this discrimination is not possible for autonomous driving system of fully or partially autonomous vehicles. Consequently, the fully or partially autonomous vehicle reacts in the same way to an attentive pedestrian and an inattentive pedestrian.
[0007] It is, thus, an objective of the present disclosure to improve the ability of fully or partially autonomous vehicles to react appropriately to inattentive pedestrians.
[0008] The problem is at least partially solved or alleviated by the subject matter of the independent claims of the present disclosure, wherein further examples are incorporated in the dependent claims.
[0009] According to a first aspect, there is provided a method for detecting an inattentive pedestrian crossing a roadway. The method comprises:
- detecting a pedestrian crossing on the roadway or receiving a pedestrian crossing location information describing a location of a pedestrian crossing on the roadway,
- detecting a head pose of a pedestrian being located on the pedestrian crossing, and
- inferring that the pedestrian is inattentive if the head pose describes that the pedestrian is looking downwards.
[0010] Thus, in a first alternative of the first step, a pedestrian crossing is detected. In this context, a pedestrian crossing is a portion of a roadway being marked or otherwise indicated as a path for pedestrians wishing to cross the roadway. An example of a pedestrian crossing is a so-called zebra crossing. Other examples of pedestrian crossings are marked by simple lines indicating the path for pedestrians. This detection may be performed by a generally known system of a fully or partially autonomous vehicle for detecting objects in an environment of the vehicle. Such a system can comprise a camera unit, a lidar unit or a radar unit. The system may form part of a vehicle, e.g. a fully or partially autonomous vehicle. Alternatively, the system may form part of the roadway infrastructure. In simplified words, such a system may form part of a traffic camera system. In a second alternative of the first step, a pedestrian crossing location information is received. This information describes a location of a pedestrian crossing on the roadway. As a result of the first step, a location of a pedestrian crossing is known. Subsequently, a pedestrian is detected on the pedestrian crossing. More specifically, a head pose of the pedestrian is detected. The pedestrian is determined to be inattentive, if the detected head pose describes that the pedestrian is looking downwards, i.e. towards the roadway. Consequently, an inattentive pedestrian may be determined in a simple and reliable manner. Based thereon, actions may be initiated or taken to improve road safety.
[0011] The basic idea underlying the present disclosure starts from the observation that many pedestrians use a mobile electronic device such as a smart phone or a smart watch while walking and even while crossing a roadway on a pedestrian crossing. This has the effect that the pedestrians are less aware of traffic participants around them and possible risky situations such as a car approaching. However, it has been found that the fact that the pedestrian is inattentive may be derived from the head pose of the pedestrian. In other words, inattentiveness of a pedestrian may be reliably determined once the head pose is known. Based on this information, necessary precautions may be initiated.
[0012] In an example, the pedestrian crossing location information may be received from database of pedestrian crossings. In such a database, locations of at least some pedestrian crossings in a predefined area are stored.
[0013] The database may form part of a central traffic control entity. Alternatively, the database may form part of a navigation system of a vehicle. In the latter case, the pedestrian crossing location information may form part of at least one map being provided by the navigation system.
[0015] In an example, determining a head pose of the pedestrian being located on the pedestrian crossing comprises:
- detecting the pedestrian,
- detecting a head of the pedestrian, and
- detecting the head pose of the head of the pedestrian.
[0016] All these steps may be performed by a system as described above which for example comprises a camera unit, a lidar unit or a radar unit. Thus, the head pose of the pedestrian is detected in several steps. First, the pedestrian as such is detected. Then, the pedestrian's head is detected. Subsequently, the head pose is detected. This procedure further enhances the detection reliability of the head pose of the pedestrian.
[0018] In an example, the method further comprises detecting an absence of traffic lights. In a case in which a traffic light is detected, the method is abandoned. In other words, the method is only performed for pedestrian crossings which are not equipped with a traffic light, e.g. a zebra crossing.
[0019] In an example, the method further comprises:
- detecting a height of the pedestrian being located on the pedestrian crossing, and
- determining whether the height is inferior to a pre-defined height threshold.
[0020] The pre-defined height threshold may be chosen such that pedestrians having a height inferior to the pre-defined height threshold are considered to be kids. Pedestrians having a height equaling the pre-defined height threshold or exceeding the height threshold may be considered to be adults. Thus, based on the detected height of the pedestrian, it is possible to initiate different measures for kids and for adults.
[0021] In an example, the method further comprises providing an inattentive pedestrian status information. The inattentive pedestrian status information comprises an identifier of the pedestrian crossing on which the inattentive pedestrian has been determined and an information that an inattentive pedestrian has been detected. In this context, providing may comprise sending or pushing the inattentive pedestrian status information to another entity. Additionally or alternatively, providing may comprise making available the inattentive pedestrian status information for being pulled or requested by another entity. In short, another entity may be informed about the fact that there is an inattentive pedestrian and about the pedestrian crossing on which the inattentive pedestrian has been detected. The identifier of the pedestrian crossing may be a name or identification code of the pedestrian crossing. Additionally or alternatively, the identifier comprises a pedestrian crossing location information describing a location of a pedestrian crossing on the roadway.
[0022] In an example, the method further comprises:
- detecting a face of the pedestrian, and
- ending the method if a face of the pedestrian has been detected.
[0023] The reasoning behind this example is that in a case in which the face of the pedestrian has been detected, the pedestrian has been looking up such that he or she has been able to see his or her surroundings which may for example comprise other traffic participants. Then, the pedestrian is not considered to be inattentive anymore. Furthermore, in this example, the probability of erroneously determining that a pedestrian is inattentive is reduced. This may happen in a situation in which the pedestrian looks downwards just for a comparatively short time.
[0024] In an example, the method further comprises:
- tracking a position of the pedestrian, and
- ending the method if the tracked position of the pedestrian lies outside the roadway.
[0025] If the pedestrian is not on the roadway, it may be irrelevant whether the pedestrian is attentive or not. In other words, the method is only performed while the pedestrian is on the roadway or street.
[0026] According to a second aspect, there is provided a method for operating a fully or partially autonomous vehicle, comprising:
- performing the method for detecting an inattentive pedestrian crossing a roadway according to the present disclosure, and
- providing an inattentive pedestrian status information to a central traffic control entity, wherein the inattentive pedestrian status information comprises an identifier of the pedestrian crossing on which the inattentive pedestrian has been determined and an information that an inattentive pedestrian has been detected.
[0027] Thus, the fully or partially autonomous vehicle is in a position to not only detect a pedestrian but also to distinguish between attentive and inattentive pedestrians. The method may be performed by a generally known system of a fully or partially autonomous vehicle for detecting objects in an environment of the vehicle. Such a system can comprise a camera unit, a lidar unit or a radar unit. The system may form part of the vehicle. As a consequence thereof, the fully or partially autonomous vehicle may initiate a reaction to the inattentive pedestrian. Moreover, due to providing the inattentive pedestrian status information to the central traffic control entity, also other traffic participants may be informed about the inattentive pedestrian and initiate a reaction to the inattentive pedestrian. Overall, road safety is increased.
[0028] In an example, a vehicle location may be additionally provided to the central traffic control entity. Thus, the central traffic control entity also disposes of a location information describing the location at which the inattentive pedestrian has been detected.
[0029] In an example, the method further comprises:
- receiving a trigger signal from a central traffic control entity triggering a collision mitigation action of the fully or partially autonomous vehicle, and
- requesting the performance of the collision mitigation action.
[0030] The trigger signal from the central traffic control entity may be received as a consequence of an inattentive pedestrian status information which has been provided to the central control entity. The traffic participant, e.g. vehicle, providing the inattentive pedestrian status information, does not necessarily need to be the same as the traffic participant, e.g. vehicle, receiving the trigger signal. Consequently, one or more vehicles can perform a collision mitigation action. Thereby, a probability of collision between the inattentive pedestrian and the vehicle is reduced.
[0031] In an example, a collision mitigation action may comprise a warning activity of the fully or partially autonomous vehicle and/or a collision avoidance maneuver of the fully or partially autonomous vehicle. The collision avoidance maneuver may comprise a breaking maneuver. In this context, a traveling speed of the vehicle may be reduced. In a special case, the vehicle may be stopped. Additionally or alternatively, the collision avoidance maneuver may comprise a steering maneuver. Thus, a travel path of the vehicle is amended such that it does not cross the travel path of the inattentive pedestrian anymore. An example collision avoidance maneuver is a lane change maneuver.
[0032] According to a third aspect, there is provided a method for informing a central traffic control entity about an inattentive pedestrian, wherein the pedestrian is crossing a roadway and wherein the pedestrian carries a mobile electronic device. The method comprises:
- detecting that the pedestrian is walking on a known pedestrian crossing by comparing a pedestrian location to a known pedestrian crossing location information,
- detecting that a screen of the mobile electronic device is active,
- detecting that the mobile electronic device is receiving a user input,
- inferring that the pedestrian is crossing the roadway and is inattentive, and
- providing a pedestrian inattentiveness information to a central traffic control entity, wherein the pedestrian inattentiveness information comprises an information that the pedestrian is inattentive and a location of the pedestrian or an identifier of the pedestrian crossing which the pedestrian is using.
[0033] This method may be performed by a mobile electronic device being carried by the pedestrian. The mobile electronic device may be smart phone, a smart watch, a tablet or any other handheld or wearable device. The pedestrian location may be determined using a location sensor, e.g. a GPS module, of the mobile electronic device. The pedestrian crossing location information may be provided by a database. In the present context, an active screen of a mobile electronic devices to be understood as a screen that is neither locked nor in standby mode. A user input may be performed by touching the screen. In such a situation, the fact that the pedestrian is inattentive is reported to the central traffic control entity. Thus, in the central traffic control entity and information is available that there is an inattentive pedestrian and where the inattentive pedestrian is. Based on this information, appropriate reactions of other traffic participants may be triggered. This enhances road safety.
[0034] In an example, also a moving direction of the pedestrian may be detected. This may also be done using a location sensor of the mobile electronic device being carried by the pedestrian. In this case, the pedestrian inattentiveness information may also comprise the moving direction. The moving direction may be determined based on a stream of positions of the mobile electronic device. Again, the location sensor may be used. Consequently, other traffic participants may be in a position to further adapt reactions to the inattentive pedestrian.
[0035] In an example, the method further comprises:
- receiving a trigger signal from a central traffic control entity triggering a collision mitigation action of the mobile electronic device, and
- requesting the performance of the collision mitigation action.
[0036] The collision mitigation action may comprise a warning message being provided to the pedestrian by the mobile electronic device carried by the pedestrian. The warning message may be audible or visible. Consequently, the pedestrian is incited to be attentive, e.g. by looking up from the mobile electronic device.
[0037] According to a fourth aspect, there is provided a method for controlling a traffic system, wherein the traffic system comprises at least one fully or partially autonomous vehicle and at least one pedestrian. The method comprises:
- receiving an inattentive pedestrian status information from the fully or partially autonomous vehicle, wherein the inattentive pedestrian status information comprises an identifier of the pedestrian crossing on which the inattentive pedestrian has been determined and an information that an inattentive pedestrian has been determined,
- receiving a pedestrian inattentiveness information from a mobile electronic device of the pedestrian, wherein the pedestrian inattentiveness information comprises an information that the pedestrian is inattentive and a location of the pedestrian or an identifier of the pedestrian crossing which the pedestrian is using,
- triggering a collision mitigation action of the fully or partially autonomous vehicle and/or the pedestrian.
[0038] The inattentive pedestrian status information may be provided by the performance of a method for operating a fully or partially autonomous vehicle according to the present disclosure. The pedestrian inattentiveness information may be provided by the performance of a method for informing a central traffic control entity about an inattentive pedestrian. Based thereon, a collision mitigation action may be triggered for the fully or partially autonomous vehicle, the pedestrian or both. Consequently, the vehicle, the pedestrian or both can react to the fact that the pedestrian is inattentive.
[0039] In an example, the collision mitigation action is only triggered, if the inattentive pedestrian status information and the pedestrian inattentiveness information relate to the same pedestrian crossing.
[0040] In an example, triggering a collision mitigation action comprises triggering at least one of a warning message being provided to the pedestrian by the mobile electronic device carried by the pedestrian, a warning activity of the fully or partially autonomous vehicle and a collision avoidance maneuver of the fully or partially autonomous vehicle.
[0041] As has been described before, the warning message provided to the pedestrian can be audible or visible. In the latter case, the screen may be fully or partly covered by the warning message. In a case in which it has been determined that the pedestrian is a kid, the warning message may fully covered the screen. Optionally, providing a warning message may comprise triggering current actions of the mobile electronic device to stop. For example if the mobile electronic device is playing music, the music may be stopped. The warning message may at least indicate the fact that a vehicle is approaching. Optionally, the warning message may also indicate a direction from which the vehicle is approaching. A warning activity of the fully or partially autonomous vehicle may comprise at least one of honking, activating head lights, and activating turn lights. As has already been explained before, a collision avoidance maneuver may comprise a breaking maneuver and/or a steering maneuver.
[0042] In an example, the collision avoidance maneuver imposes a safety distance between the fully or partially autonomous vehicle and the detected pedestrian. The speed of the vehicle is limited based on the impose safety distance.
[0043] According to a fifth aspect, there is provided a data processing apparatus comprising means for carrying out at least one of the methods of the present disclosure. Consequently, by using such a data processing apparatus, an inattentive pedestrian may be determined in a simple and reliable manner. This detection result may be used by traffic participants such as fully or partially autonomous vehicles or pedestrians. The traffic participants can react appropriately to a detected, inattentive pedestrian. Overall, road safety is increased.
[0044] In an example, the data processing apparatus may form part of a fully or partially autonomous vehicle. In this context, the data processing apparatus may comprise means for carrying out the method for detecting an inattentive pedestrian crossing a roadway according to the present disclosure and/or comprise means for carrying out the method for operating a fully or partially autonomous vehicle according to the present disclosure.
[0045] In an example, the data processing apparatus may form part of a pedestrian's mobile electronic device. In this context, the data processing apparatus may comprise means for carrying out the method for informing a central traffic control entity about an inattentive pedestrian according to the present disclosure.
[0046] In an example, the data processing apparatus may form part of a central traffic control entity. In this context, the data processing apparatus may comprise means for carrying out the method for controlling a traffic system according to the present disclosure.
[0047] According to a sixth aspect, there is provided a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out at least one of the following methods:
- the method for detecting an inattentive pedestrian crossing a roadway according to the present disclosure,
- the method for operating a fully or partially autonomous vehicle according to the present disclosure,
- the method for informing a central traffic control entity about an inattentive pedestrian according to the present disclosure, and
- the method for controlling a traffic system according to the present disclosure.
[0048] According to a seventh aspect, there is provided a computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out at least one of the following methods:
- the method for detecting an inattentive pedestrian crossing a roadway according to the present disclosure,
- the method for operating a fully or partially autonomous vehicle according to the present disclosure,
- the method for informing a central traffic control entity about an inattentive pedestrian according to the present disclosure, and
- the method for controlling a traffic system according to the present disclosure.
[0049] According to an eighth aspect, there is provided a traffic control system comprising a first data processing apparatus according to the present disclosure comprising means for carrying out the method for operating a fully or partially autonomous vehicle according to the present disclosure, a second data processing apparatus according to the present disclosure comprising means for carrying out the methods for informing a central traffic control entity about an inattentive pedestrian according to the present disclosure, and a third data processing apparatus according to the present disclosure comprising means for carrying out the method for controlling a traffic system according to the present disclosure. Consequently, by using such a traffic control system, an inattentive pedestrian may be determined in a simple and reliable manner. This detection result may be used such that traffic participants can react appropriately to a detected, inattentive pedestrian. Overall, road safety is increased.
[0050] The methods of the present disclosure may be at least partly computer-implemented, and may be implemented in software or in hardware, or in software and hardware. Further, the methods may be carried out by computer program instructions running on means that provide data processing functions. The data processing means may be a suitable computing means, such as an electronic control module etc., which may also be a distributed computer system. The data processing means or the computer, respectively, may comprise one or more of a processor, a memory, a data interface, or the like.
[0051] It should be noted that the above examples may be combined with each other irrespective of the aspect involved.
[0052] These and other aspects of the present disclosure will become apparent from and elucidated with reference to the examples described hereinafter.
[0053] Examples of the disclosure will be described in the following with reference to the following drawings.
- Figure 1
- illustrates a traffic situation comprising a fully autonomous vehicle having a first data processing apparatus according to the present disclosure comprising means for carrying out a method for operating a fully or partially autonomous vehicle according to the present disclosure and a method for detecting an inattentive pedestrian according to the present disclosure, a pedestrian carrying a mobile electronic device having a second data processing apparatus according to the present disclosure comprising means for carrying out a method for informing a central traffic control entity about an inattentive pedestrian according to the present disclosure, and a central traffic control entity comprising a third data processing apparatus according to the present disclosure comprising means for carrying out a method for controlling a traffic system according to the present disclosure, and
- Figure 2
- illustrates the pedestrian from a different perspective.
[0054] The Figures are merely schematic representations and serve only to illustrate examples of the disclosure. Identical or equivalent elements are in principle provided with the same reference signs.
[0057] Moreover, the roadway R comprises a pedestrian crossing P which is a so-called zebra crossing in the present example.
[0059] Additionally, a pedestrian 12 is crossing the roadway R via the pedestrian crossing P. The pedestrian 12 is moving along traveling direction D2.
[0061] The pedestrian 12 is inattentive since he or she is looking on a screen of the mobile electronic device 14 (see also Figure 2).
[0062] Furthermore, a central traffic control entity 16 is provided. The central traffic control entity 16 is configured to control a traffic system comprising the vehicle 10 and the pedestrian 12.
[0064] The first data processing apparatus 18 comprises a first data processing unit 20 and a first data storage unit 22.
[0065] A first computer readable storage medium 24 forms part of the first data storage unit 22. On the first computer readable storage medium 24, there is provided a first computer program 26.
[0066] The computer program 26 comprises instructions which, when the computer program 26 is executed by the first data processing unit 20 or, more generally, a computer, cause the first data processing unit 20 or the computer to carry out a method for operating a fully or partially autonomous vehicle.
[0067] Consequently, the first data processing unit 20 and the first data storage unit 22 form means 28 for carrying out the method for operating a fully or partially autonomous vehicle.
[0069] In a first step S11, a method for detecting an inattentive pedestrian crossing a roadway is performed.
[0070] This means that the computer program 26 also comprises instructions which, when the computer program 26 is executed by the first data processing unit 20 or, more generally, a computer, cause the first data processing unit 20 or the computer to carry out a method for detecting an inattentive pedestrian crossing a roadway.
[0071] Consequently, the first data processing unit 20 and the first data storage unit 22 also form means 30 for carrying out the method for detecting an inattentive pedestrian crossing a roadway.
[0072] It is understood that in other examples, the method for detecting an inattentive pedestrian crossing the roadway may as well be performed by other means, e.g. a data processing apparatus separate from the first data processing apparatus 18.
[0074] This may be done by use of an environmental sensing system of the vehicle 10. In Figure 1, the environmental sensing system is symbolized by a camera unit 32.
[0075] Alternative to the detection of the pedestrian crossing P, a pedestrian crossing location information PCL may be received. The pedestrian crossing location information PCL describes a location of the pedestrian crossing P on the roadway R. The pedestrian crossing location information PCL may be provided by the central traffic control entity 16.
[0077] Thereafter, in an optional step S11b, an absence of a traffic light is detected. Only then, the method is continued. Otherwise, i.e. if a traffic light is detected, the method is abandoned.
[0078] Subsequently, in a step S11c, a head pose of the pedestrian 12 being located on the pedestrian crossing P is detected.
[0079] To this end, in a first sub- step, the pedestrian 12 is detected. Then, in a second sub-step, the head of the pedestrian 12 is detected. After that, in a third sub-step, the head pose of the pedestrian is detected, i.e. the orientation of the head in space. In other words, a direction G of gaze of the pedestrian 12 is determined. The direction G of gaze, and thus also the head pose, is considered to describe that the pedestrian is looking downwards if it is oriented downwards with respect to a horizontal direction. This is done based on pictures that are captured by the camera unit 32.
[0080] In the following, in a step SlId, it is inferred that the pedestrian 12 is inattentive if the head pose describes that the pedestrian 12 is looking downwards. In the present context, looking downwards means looking towards the roadway R or the pedestrian crossing P. This is illustrated in Figure 2.
[0081] In an optional step S11e, a height of the pedestrian 12 is detected and compared to a pre-defined height threshold.
[0082] In a case in which the pedestrian height is determined to be inferior to the pre-defined height threshold, the pedestrian 12 is regarded as a kid. If the pedestrian height equals or exceeds the pre-defined height threshold, the pedestrian 12 is regarded as an adult.
[0083] Subsequently, in a step S11f, an inattentive pedestrian status information IPS is provided to the central traffic control entity 16. This step may also be considered as a step S12 of the method for operating a fully or partially autonomous vehicle.
[0084] The inattentive pedestrian status information IPS comprises an identifier of the pedestrian crossing P on which the inattentive pedestrian 12 has been determined and an information that an inattentive pedestrian has been detected. The identifier may comprise a location of the pedestrian crossing P. The inattentive pedestrian status information IPS may optionally also comprise the information whether the pedestrian 12 is regarded as a kid or as an adult.
[0085] In an optional step S11g which may be performed in parallel to the steps as mentioned above, a face of the pedestrian 12 may be detected. In a case in which the face of the pedestrian 12 has been detected, the method is abandoned. In this case it is considered that the pedestrian 12 has seen the vehicle 10 and, thus, is not considered to be inattentive anymore. Otherwise, i.e. if the face of the pedestrian 12 is not detected, the method is continued.
[0086] In a further optional step S11h, which may also be performed in parallel to the steps as mentioned above, a position of the pedestrian 12 is tracked. Consequently, a situation in which the pedestrian 12 is located on the roadway R may be distinguished over a situation in which the pedestrian 12 is located outside the roadway R. If the pedestrian 12 is found to be outside the roadway R, the method is abandoned.
[0087] Additionally, in a step S13 of the method for operating a fully or partially autonomous vehicle, the first data processing apparatus 18 may receive a first trigger signal TS1 from the central traffic control entity 16 triggering a collision mitigation action of the vehicle 10.
[0088] Subsequently, in a step S14, the first data processing apparatus 18 requests the performance of the collision mitigation action.
[0089] The collision mitigation action for example comprises a breaking action such as an emergency braking. Alternatively, the collision mitigation action may comprise a lane change maneuver. The lane change maneuver has the consequence, that the vehicle's 10 trajectory and the pedestrian's 12 trajectory do not interfere with one another.
[0090] Furthermore, the mobile electronic device 14 comprises a second data processing apparatus 34.
[0091] The second data processing apparatus 34 comprises a second data processing unit 36 and a second data storage unit 38.
[0092] A second computer readable storage medium 40 forms part of the second data storage unit 38. On the second computer readable storage medium 40, there is provided a second computer program 42.
[0093] The second computer program 42 comprises instructions which, when the computer program 42 is executed by the second data processing unit 36 or more generally a computer, cause the second data processing unit 36 or the computer to carry out a method for informing a central traffic control entity about an inattentive pedestrian.
[0094] Consequently, the second data processing unit 36 and the second data storage unit 38 form means 44 for carrying out the method for informing a central traffic control entity about an inattentive pedestrian.
[0096] As has already been mentioned before, the pedestrian 12 is crossing the roadway R and the pedestrian 12 carries the mobile electronic device 14.
[0097] In a first step S21, it is detected that the pedestrian 12 is walking on a known pedestrian crossing P. To this end, a pedestrian location may be determined by a location sensor of the mobile electronic device 14 and may be compared to a known pedestrian crossing location information PCL. The known pedestrian crossing location information PCL may be provided to the mobile electronic device 14 by the central traffic control entity 16.
[0098] In a step S22, it is additionally detected that a screen of the mobile electronic device 14 is active. This means that the screen is unlocked and not in standby mode.
[0099] Moreover, in a step S23, it is detected that the mobile electronic device 14 is receiving a user input. In this context, the reception of a user input may be detected, if a user input is detected within a pre-defined time span.
[0100] In the following, in step S24, it is inferred that the pedestrian 12 is crossing the roadway R and is inattentive. The inattentiveness is due to the fact that the pedestrian 12 is manipulating the mobile electronic device 14.
[0101] Subsequently, in a step S25, a pedestrian inattentiveness information PII is provided to the central traffic control entity 16. The pedestrian inattentiveness information PII comprises an information that the pedestrian 12 is inattentive and a location of the pedestrian 12 or an identifier of the pedestrian crossing P which the pedestrian 12 is using. As before, the identifier of the pedestrian crossing P may comprise a location of the pedestrian crossing P.
[0102] Additionally, in a step S26, the second data processing apparatus 34 may receive a second trigger signal TS2 from a central traffic control entity 16 triggering a collision mitigation action of the mobile electronic device 14.
[0103] Thereafter, in a step S27, the performance of the collision mitigation action may be requested.
[0104] In the present context, the collision mitigation action of the mobile electronic device 14 may be visible and/or audible.
[0105] In a first example, a warning message may be displayed on the screen of the mobile electronic device 14. In a second example, a warning sound may be issued by the mobile electronic device 14.
[0107] It is noted that steps S26 and S27 may be performed following the provision of an inattentive pedestrian status information IPS received by the central traffic control entity 16 or based on a pedestrian inattentiveness information PII received by the central traffic control entity 16. In simplified words, either the vehicle 10 or the pedestrian 12 may cause the second trigger signal TS2 leading to a collision mitigation action of the mobile electronic device 14.
[0108] In a special case, the collision mitigation action of the mobile electronic device 14 may only be triggered based on both an inattentive pedestrian status information IPS and a pedestrian inattentiveness information PII relating to the same pedestrian crossing P.
[0109] The central traffic control entity 16, which has already been mentioned several times, comprises a third data processing apparatus 46.
[0110] The third data processing apparatus 46 comprises a third data processing unit 48 and a third data storage unit 50.
[0111] A third computer readable storage medium 52 forms part of the third data storage unit 50. On the third computer readable storage medium 52, there is provided a third computer program 54.
[0112] The third computer program 54 comprises instructions which, when the third computer program 54 is executed by the third data processing unit 48 or more generally a computer, cause the third data processing unit 48 or the computer to carry out a method for controlling a traffic system.
[0113] Consequently, the third data processing unit 48 and the third data storage unit 50 form means 56 for carrying out the method for controlling a traffic system.
[0115] As has been mentioned before, the traffic system comprises at least the vehicle 10 and the pedestrian 12.
[0116] In a first step S31, an inattentive pedestrian status information IPS is received from the vehicle 10, more precisely the first data processing apparatus 18. As has been mentioned before, the inattentive pedestrian status information IPS comprises an identifier of the pedestrian crossing P on which the inattentive pedestrian has been determined and an information that an inattentive pedestrian has been determined.
[0117] Additionally, in a step S32, a pedestrian inattentiveness information PII is received from the mobile electronic device 14 of the pedestrian 12, more precisely from the second data processing apparatus 34. The pedestrian inattentiveness information PII comprises an information that the pedestrian is inattentive and a location of the pedestrian 12 or an identifier of the pedestrian crossing P which the pedestrian 12 is using.
[0118] In a case, in which both the inattentive pedestrian status information IPS and the pedestrian inattentiveness information PII relate to the same pedestrian crossing P, in a step S33, a collision mitigation action may be triggered.
[0119] The collision mitigation action may be triggered for the vehicle 10 and/or for the pedestrian 12, more precisely for the mobile electronic device 14.
[0120] Triggering a collision mitigation action may comprise triggering a warning message being provided to the pedestrian 12 by the mobile electronic device 14 carried by the pedestrian. For example, the warning message is displayed on a screen of the mobile electronic device 14.
[0121] Additionally or alternatively, a warning activity of the vehicle 10 may be triggered. The warning activity of the vehicle 10 may comprise at least one of honking, activating the headlights and activating the turn lights.
[0122] Further additionally or alternatively, a collision avoidance maneuver of the vehicle 10 may be triggered. As has already been explained before, the collision avoidance maneuver may comprise a breaking activity or a steering activity.
[0123] In simplified words, the vehicle 10 or the pedestrian 12 or both are triggered to react to the situation in a way to avoid a collision between the vehicle 10 and the pedestrian 12.
[0124] It is noted that the first data processing apparatus 14, the second data processing apparatus 34, and the third data processing apparatus 46 together may be designated as a traffic control system 58.
[0125] Other variations to the disclosed examples can be understood and effected by those skilled in the art in practicing the claimed disclosure, from the study of the drawings, the disclosure, and the appended claims. In the claims the word "comprising" does not exclude other elements or steps and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items or steps recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. A computer program may be stored/distributed on a suitable medium such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems. Any reference signs in the claims should not be construed as limiting the scope of the claims.
LIST OF REFERENCE SIGNS
[0126]
- 10
- vehicle
- 12
- pedestrian
- 14
- mobile electronic device
- 16
- central traffic control entity
- 18
- first data processing apparatus
- 20
- first data processing unit
- 22
- first data storage unit
- 24
- first computer readable storage medium
- 26
- first computer program
- 28
- means for carrying out the method for operating a fully or partially autonomous vehicle
- 30
- means for carrying out the method for detecting an inattentive pedestrian crossing a roadway
- 32
- camera unit
- 34
- second data processing apparatus
- 36
- second data processing unit
- 38
- second data storage unit
- 40
- second computer readable storage medium
- 42
- second computer program
- 44
- means for carrying out a method for informing a central traffic control entity about an inattentive pedestrian
- 46
- third data processing apparatus
- 48
- third data processing unit
- 50
- third data storage unit
- 52
- third computer readable storage medium
- 54
- third computer program
- 56
- means for carrying out the method for controlling a traffic system
- 58
- traffic control system
- D1
- travelling direction of the vehicle
- D2
- travelling direction of the pedestrian
- G
- direction of gaze
- IPS
- inattentive pedestrian status information
- L1
- lane
- L2
- lane
- P
- pedestrian crossing
- PCL
- pedestrian crossing location information
- PII
- pedestrian inattentiveness information
- R
- roadway
- TS1
- first trigger signal
- TS2
- second trigger signal
- S11a to Sllh
- steps of the method for detecting an inattentive pedestrian crossing a roadway
- S1 1 to S14
- steps of the method for operating a fully or partially autonomous vehicle
- S21 to S27
- steps of the method for informing a central traffic control entity about an inattentive pedestrian
- S31 to S33
- steps of the method for controlling a traffic system
1. A method for detecting an inattentive pedestrian (12) crossing a roadway (R), comprising:
- detecting a pedestrian crossing (P) on the roadway (R) or receiving a pedestrian crossing location information (PCL) describing a location of a pedestrian crossing (P) on the roadway (R) (S11a),
- detecting a head pose of a pedestrian (12) being located on the pedestrian crossing (P) (S11c), and
- inferring that the pedestrian (12) is inattentive if the head pose describes that the pedestrian (12) is looking downwards (S11d).
2. The method of claim 1, wherein determining a head pose of the pedestrian (12) being
located on the pedestrian crossing (P) (S11c) comprises:
- detecting the pedestrian (12),
- detecting a head of the pedestrian (12), and
- detecting the head pose of the head of the pedestrian (12).
3. The method of claim 1 or 2, further comprising detecting an absence of traffic lights
(S11b).
4. The method of any one of the preceding claims, further comprising:
- detecting a height of the pedestrian (12) being located on the pedestrian crossing (P), and
- determining whether the height is inferior to a pre-defined height threshold (S11e).
5. The method of any one of the preceding claims, further comprising: providing an inattentive
pedestrian status information (IPS), wherein the inattentive pedestrian status information
(IPS) comprises an identifier of the pedestrian crossing (P) on which the inattentive
pedestrian (12) has been determined and an information that an inattentive pedestrian
(12) has been detected (S11f).
6. The method of any one of the preceding claims, further comprising:
- detecting a face of the pedestrian (12), and
- ending the method if a face of the pedestrian (12) has been detected (S11g).
7. The method of any one of the preceding claims, further comprising:
- tracking a position of the pedestrian (12), and
- ending the method if the tracked position of the pedestrian (12) lies outside the roadway (R) (S11h).
8. A method for operating a fully or partially autonomous vehicle (10), comprising:
- performing the method according to any one of the preceding claims (S11), and
- providing an inattentive pedestrian status information (IPS) to a central traffic control entity (16), wherein the inattentive pedestrian status information (IPS) comprises an identifier of the pedestrian crossing (P) on which the inattentive pedestrian (12) has been determined and an information that an inattentive pedestrian (12) has been detected (S12).
9. The method of claim 8, further comprising:
- receiving a trigger signal (TS1) from a central traffic control entity (16) triggering a collision mitigation action of the fully or partially autonomous vehicle (12) (S13), and
- requesting the performance of the collision mitigation action (S14).
10. A method for informing a central traffic control entity (16) about an inattentive
pedestrian (12), wherein the pedestrian (12) is crossing a roadway (R) and wherein
the pedestrian (12) carries a mobile electronic device (14), comprising:
- detecting that the pedestrian (12) is walking on a known pedestrian crossing (P) by comparing a pedestrian location to a known pedestrian crossing location information (PCL) (S21),
- detecting that a screen of the mobile electronic device (14) is active (S22),
- detecting that the mobile electronic device (14) is receiving a user input (S23),
- inferring that the pedestrian (12) is crossing the roadway (R) and is inattentive (S24), and
- providing a pedestrian inattentiveness information (PII) to a central traffic control entity (16), wherein the pedestrian inattentiveness information (PII) comprises an information that the pedestrian (12) is inattentive and a location of the pedestrian (12) or an identifier of the pedestrian crossing (P) which the pedestrian (12) is using (S25).
11. The method of claim 10, further comprising:
- receiving a trigger signal (TS2) from a central traffic control entity (16) triggering a collision mitigation action of the mobile electronic device (14) (S26), and
- requesting the performance of the collision mitigation action (S27).
12. A method for controlling a traffic system, wherein the traffic system comprises at
least one fully or partially autonomous vehicle (10) and at least one pedestrian (12),
the method comprising:
- receiving an inattentive pedestrian status information (IPS) from the fully or partially autonomous vehicle (10), wherein the inattentive pedestrian status information (IPS) comprises an identifier of the pedestrian crossing (P) on which the inattentive pedestrian (12) has been determined and an information that an inattentive pedestrian (12) has been determined (S31),
- receiving a pedestrian inattentiveness information (PII) from a mobile electronic device (14) of the pedestrian (12), wherein the pedestrian inattentiveness information (PII) comprises an information that the pedestrian (12) is inattentive and a location of the pedestrian (12) or an identifier of the pedestrian crossing (P) which the pedestrian (12) is using (S32),
- triggering a collision mitigation action of the fully or partially autonomous vehicle (10) and/or the pedestrian (12) (S33).
13. The method of claim 12, wherein triggering a collision mitigation action (S33) comprises
triggering at least one of
a warning message being provided to the pedestrian (12) by the mobile electronic device (14) carried by the pedestrian (12),
a warning activity of the fully or partially autonomous vehicle (10) and
a collision avoidance maneuver of the fully or partially autonomous vehicle (10).
14. A data processing apparatus (18, 34, 46) comprising means (28, 30, 44, 56) for carrying
out at least one of the methods of the preceding claims.
15. A traffic control system (58) comprising:
a first data processing apparatus (18) according to claim 14 comprising means for carrying out the methods of claim 8 or 9,
a second data processing apparatus (34) according to claim 14 comprising means for carrying out the methods of claim 10 or 11, and
a third data processing apparatus (46) according to claim 14 comprising means for carrying out the methods of claim 12 or 13.
Amended claims in accordance with Rule 137(2) EPC.
1. A method for detecting an inattentive pedestrian (12) crossing a roadway (R), comprising:
- detecting a pedestrian crossing (P) on the roadway (R) or receiving a pedestrian crossing location information (PCL) describing a location of a pedestrian crossing (P) on the roadway (R) (S11a),
- detecting a head pose of a pedestrian (12) being located on the pedestrian crossing (P) (S11c),
- inferring that the pedestrian (12) is inattentive if the head pose describes that the pedestrian (12) is looking downwards (S11d), and
- providing an inattentive pedestrian status information (IPS), wherein the inattentive pedestrian status information (IPS) comprises an identifier of the pedestrian crossing (P) on which the inattentive pedestrian (12) has been determined and an information that an inattentive pedestrian (12) has been detected (S11f).
2. The method of claim 1, wherein determining a head pose of the pedestrian (12) being
located on the pedestrian crossing (P) (S11c) comprises:
- detecting the pedestrian (12),
- detecting a head of the pedestrian (12), and
- detecting the head pose of the head of the pedestrian (12).
3. The method of claim 1 or 2, further comprising detecting an absence of traffic lights
(S11b).
4. The method of any one of the preceding claims, further comprising:
- detecting a height of the pedestrian (12) being located on the pedestrian crossing (P), and
- determining whether the height is inferior to a pre-defined height threshold (S11e).
5. The method of any one of the preceding claims, further comprising:
- detecting a face of the pedestrian (12), and
- ending the method if a face of the pedestrian (12) has been detected (S 11g).
6. The method of any one of the preceding claims, further comprising:
- tracking a position of the pedestrian (12), and
- ending the method if the tracked position of the pedestrian (12) lies outside the roadway (R) (S11h).
7. A method for operating a fully or partially autonomous vehicle (10), comprising:
- performing the method according to any one of the preceding claims (S11), and
- providing the inattentive pedestrian status information (IPS) to a central traffic control entity (16).
8. The method of claim 7, further comprising:
- receiving a trigger signal (TS1) from a central traffic control entity (16) triggering a collision mitigation action of the fully or partially autonomous vehicle (12) (S13), and
- requesting the performance of the collision mitigation action (S14).
9. A method for informing a central traffic control entity (16) about an inattentive
pedestrian (12), wherein the pedestrian (12) is crossing a roadway (R) and wherein
the pedestrian (12) carries a mobile electronic device (14), comprising:
- detecting that the pedestrian (12) is walking on a known pedestrian crossing (P) by comparing a pedestrian location to a known pedestrian crossing location information (PCL) (S21),
- detecting that a screen of the mobile electronic device (14) is active (S22),
- detecting that the mobile electronic device (14) is receiving a user input (S23),
- inferring that the pedestrian (12) is crossing the roadway (R) and is inattentive (S24), and
- providing a pedestrian inattentiveness information (PII) to a central traffic control entity (16), wherein the pedestrian inattentiveness information (PII) comprises an information that the pedestrian (12) is inattentive and a location of the pedestrian (12) or an identifier of the pedestrian crossing (P) which the pedestrian (12) is using (S25).
10. The method of claim 9, further comprising:
- receiving a trigger signal (TS2) from a central traffic control entity (16) triggering a collision mitigation action of the mobile electronic device (14) (S26), and
- requesting the performance of the collision mitigation action (S27).
11. A method for controlling a traffic system, wherein the traffic system comprises at
least one fully or partially autonomous vehicle (10) and at least one pedestrian (12),
the method comprising:
- receiving an inattentive pedestrian status information (IPS) from the fully or partially autonomous vehicle (10), wherein the inattentive pedestrian status information (IPS) comprises an identifier of the pedestrian crossing (P) on which the inattentive pedestrian (12) has been determined and an information that an inattentive pedestrian (12) has been determined (S31),
- receiving a pedestrian inattentiveness information (PII) from a mobile electronic device (14) of the pedestrian (12), wherein the pedestrian inattentiveness information (PII) comprises an information that the pedestrian (12) is inattentive and a location of the pedestrian (12) or an identifier of the pedestrian crossing (P) which the pedestrian (12) is using (S32),
- triggering a collision mitigation action of the fully or partially autonomous vehicle (10) and/or the pedestrian (12) (S33).
12. The method of claim 11, wherein triggering a collision mitigation action (S33) comprises
triggering at least one of
a warning message being provided to the pedestrian (12) by the mobile electronic device (14) carried by the pedestrian (12),
a warning activity of the fully or partially autonomous vehicle (10) and
a collision avoidance maneuver of the fully or partially autonomous vehicle (10).
13. A data processing apparatus (18, 34, 46) comprising means (28, 30, 44, 56) for carrying
out at least one of the methods of the preceding claims.
14. A traffic control system (58) comprising:
a first data processing apparatus (18) according to claim 13 comprising means for carrying out the methods of claim 7 or 8,
a second data processing apparatus (34) according to claim 13 comprising means for carrying out the methods of claim 9 or 10, and
a third data processing apparatus (46) according to claim 13 comprising means for carrying out the methods of claim 11 or 12.