A method, a device and a receiver for processing video

Abstract

 The invention concerns a device (103, 110) and method for creating a video in a vehicle (100) in particular for sending to a receiver, comprising an analyzer (110) adapted to assess a traffic situation from a captured image of the outside of the vehicle (100), and a multiplexer (103) adapted to join the captured image with recorded audio from a passenger compartment of the vehicle (100) and information about the assessed traffic situation as the video. Furthermore the invention concerns a receiver (107, 109) and method for playing a video comprising an output (109) adapted to output a video, wherein the receiver comprises a de-multiplexer (107) adapted to separate information about an assessed traffic situation from the video, and to determine whether to send an assistive message to the output (109) or not depending on the information about the assessed traffic situation.

Description

Field of the invention

[0001] The invention relates to a method, a device and a receiver for processing video.

Background

[0002] Modern cars are equipped with cameras, mainly for viewing the car environment while parking or while entering a crossing or for reading traffic signs.

[0003] Modern cars comprise communication assistance systems for the driver or passenger to make voice calls through the communication network.

[0004] In many situations, driving and communicating at the same time is not an issue. However critical situations exist, in which for safety sake the focus should be on driving rather than communicating.

Summary

[0005] The object of the invention is thus to automatically improve safety while driving.

[0006] The main idea of the invention is a device comprising an analyzer adapted to assess a traffic situation from a captured image of the outside of a vehicle, and a sender adapted to send information about the assessed traffic situation.

[0007] Advantageously the device comprises a multiplexer adapted to join the captured image with recorded audio from a passenger compartment of the vehicle.

[0008] Advantageously the device comprises a microphone for recording audio of the passenger compartment of the vehicle.

[0009] Advantageously the device comprises a camera for capturing the image of the outside of the vehicle.

[0010] The invention also concerns a vehicle comprising such device and a second device comprising a receiver and an output, wherein the receiver is adapted to receive information about an assessed traffic situation and adapted to determine whether to send an assistive message to the output or not depending on the information about the assessed traffic situation.

[0011] In the second device preferably the assistive message is determined from the information about the assessed traffic situation.

[0012] Further developments of the invention can be gathered from dependent claims and the following description.

Brief description of the figures

[0013] In the following the invention will be explained further making reference to the attached drawings.

Fig. 1 schematically shows a first part of a system according to a first embodiment.

Fig. 2 schematically shows a flow chart of a first method.

Fig. 3 schematically shows a flow chart of a second method.

Description of the embodiments

[0014] 

Figure 1 shows part of a system comprising an embodiment of the invention.

Figure 1 depicts a vehicle 100 and a device for creating a video in the vehicle in particular for sending to a receiver. The device comprises an analyzer 110 adapted to assess a traffic situation from a captured image of the outside of the vehicle, and a multiplexer 103 adapted to code or join the captured image with recorded audio from a passenger compartment of the vehicle and information about the assessed traffic situation as the video.

[0015] Passenger compartment in case of the vehicle is the inside of the vehicle. More generally the area where the driver or the passengers sit is the passenger compartment, even when the vehicle has no closed interior room.

[0016] Furthermore the device may comprise a sender 104 adapted to send the video. Alternatively the device is connectable to the sender 104.

[0017] The analyzer 110 and the multiplexer 103 are adapted to interact for joining information about the assessed traffic situation to the video, e.g. via a data link or data bus.

[0018] Figure 1 depicts also the receiver comprising an output 109 adapted to output the video received from aforementioned device and a de-multiplexer 107 adapted to decode the video or separate the information about the assessed traffic situation from the video, to determine an assistive message from the information, and to interact with the output 109 to output the assistive message on the output 109. The output 109 for example comprises a display and speaker to output images and audio of the video respectively.

[0019] The sender 104 is adapted to connect to the receiver via a data link including in the example of figure 1 an antenna 105 and a computer network 108. The data link for example includes a local area network (LAN), wireless local area network (WLAN, 802.11n) or wide area network (WAN). The connection is for example according to the well known internet protocol (IP), user datagram protocol (UDP), transmission control protocol (TCP) or stream control transmission protocol (SCTP). Signaling is for example according to the well known session initiation protocol (SIP) or Internet Protocol Multimedia Subsystem (IMS). Addressing may be done using the well known uniform resource locators (URL), uniform resource identifiers (URI).

[0020] Furthermore the data link between the sender 104 and the antenna 105 is in the example according to the Long Term Evolution standard, well known as LTE.

[0021] Any other type of data link suitable to connect the sender 104 of the vehicle 100 to the receiver may be used. For example a Global System for Mobile Communications link may be used.

[0022] Furthermore the multiplexer 103 and the de-multiplexer 107 operate in the example according to the well known H.263 Codec. Any other type of video coding/decoding may be used. For example any of the well known MPEG-4 compatible video-codec may be used.

[0023] The multiplexer 103 is adapted to receive the audio from the microphone 102, the image from the camera 101 and join audio and image into the video using the codec.

[0024] To that end the camera 101 and the microphone 102 are adapted to send audio and images to the multiplexer 103 continuously while the multiplexer 103 creates the video stream.

[0025] Furthermore the multiplexer 103 is adapted to receive information about the assessed traffic situation from the analyzer 110 and add it to the video.

[0026] The analyzer 110 for example is adapted to determine the distance or the change of the distance between the vehicle 100 and other objects captured by the camera 101 in the image. For example two consecutive images from the camera 101 are compared to decide from the size difference between the same object in both images the approximate change in distance. In an example in case the distance changes too quickly, the sentence "Heavy traffic Warning" is determined by the analyzer 110. Likewise in case an object moves from a position outside of the lane closer to a position in the lane of the vehicle 100, the sentence "Proximity Warning" is determined. The information about the assessed traffic situation is for example a text image indicating the warning as the word "Warning" or a situation dependent text explaining the reason, such as "Heavy traffic Warning" or "Proximity Warning".

[0027] Additionally or alternatively to text an audio signal playing the word "Warning" or the sentence "Heavy traffic Warning" or "Proximity Warning" may be generated as information about the assessed traffic situation by the analyzer 110.

[0028] The analyzer 110 is adapted to output the information about the assessed traffic situation to the multiplexer 103.

[0029] This way the multiplexer 103 determines a video stream from the input by the camera 101, the microphone 102 and the analyzer 110.

[0030] The recording of the audio, the capturing of the images, the assessment of the traffic situation and the joining to the video is in the example started upon detection of a request to establish a connection between the sender 104 and the receiver. For example a person in the passenger compartment of the vehicle 110 speaks a number as user input requesting a connection to the receiver. The number may be a telephone number or a short cut, like a name, mapped to a telephone number.

[0031] Likewise a Session Initiation Protocol connection request message to the sender 104 from the receiver may trigger the start.

[0032] The sender 104 in the example is adapted to establish the data link according the Long Term Evolution standard from the sender 104 to the antenna 105 and further to the receiver upon detection of the number by the analyzer 110.

[0033] In the example this is detected by the analyzer 110 by monitoring input from the microphone 102 via speech recognition. Alternatively a keyboard may be used to dial the number as indication of the request. In the example an internet protocol connection to the receiver is set up via the Long Term Evolution link, the analyzer 110 is adapted to determine the address of the receiver according to the used data link.

[0034] Furthermore the multiplexer 103 is connected to the sender 104 and adapted to output the video to the sender 104 via the connection. The video in the example is coded using the H.263 standard. The sender 104 is adapted to send the coded video to the receiver upon receipt of the coded video, e.g. using the internet protocol address of the receiver.

[0035] The antenna 105 is connected to the receiver via the computer network 105 and adapted to forward the coded video.

[0036] The receiver is adapted to receive the coded video via an interface. In the example the interface is according to the part of the de-multiplexer 107.

[0037] Upon receipt of the video, the de-multiplexer 107 is adapted to separate the information about the assessed traffic situation from the coded video. In the example the decoder uses the H.263 standard to decode the video. For example the warning information is separated from the video. Furthermore the warning information is used to determine an assistive message.

[0038] The assistive message may comprise of text instructions to be displayed on the output 109 via the display or audio instructions output by the speaker.

[0039] For example from the sentence "Heavy traffic Warning" it is determined that the conversation with the user at the sender 104 is to be interrupted. In this case, for example the multiplexer 107 interacts with the output 109 to display only the sentence "Heavy traffic Warning" rather than the images of the video.

[0040] Likewise the audio may be muted.

[0041] A first method for communicating is explained below making reference to figure 2.

[0042] The first method for example starts whenever the request to establish the connection between the sender 104 and the receiver is detected.

[0043] After the start a step 201 is executed.

[0044] In step 201 the image of the outside of the vehicle 100 is captured. Afterwards a step 202 is executed.

[0045] In step 202 the audio of the passenger compartment of the vehicle 100 is recorded.

[0046] Steps 201 and 202 may be performed simultaneously.

[0047] Afterwards a step 203 is executed.

[0048] In step 203 the traffic situation is assessed from the captured image. For example the change of the distance between the vehicle 100 and an object is determined from two consecutively captured images. This means that the traffic situation is assessed from a captured image of the outside of the vehicle. The assessment is for example stored as text "Heavy traffic Warning" in a variable. Afterwards a step 204 is executed.

[0049] In step 204 the information on about the assessed traffic situation is determined for example from the variable. The information is determined for example from the variable containing the sentence "Heavy traffic Warning" as the equivalent text image or audio. In case no warning is determined, the information about the assessed traffic situation is for example "normal". Afterwards a step 205 is executed.

[0050] In step 205 the captured image, the recorded audio and the information about the assessed traffic situation are joined to the video. This means that the captured image is joined with recorded audio from the passenger compartment of the vehicle and information about the assessed traffic situation as the video. The video may be compressed or coded as well. For example the video is compressed or coded according to the H.263 standard. This means that several captured images, recorded audio are joined with the information about the assessed traffic situation.

[0051] Afterwards a step 206 is executed.

[0052] In step 206 the video signal or the coded video signal is sent.

[0053] Afterwards a step 207 is executed.

[0054] In the step 207 a test is performed to determine whether the connection between the sender 104 and the receiver is active, or whether at least one of the devices is switched off. In case the connection is active, the step 201 is executed. Otherwise the first method ends.

[0055] The vehicle 100 for example has the aforementioned device integrated permanently. Permanently means that once mounted, the individual parts are fixed as part of the vehicle. Instead of permanently mounting the parts to the vehicle, the device may be integrated into a single housing, e.g. a smart phone. In this case the mounting may be via a special bracket or clamp that guarantees the mounting position while the smart phone is mounted. The system according to another embodiment comprises of the same devices. However in this other embodiment, the analyzer 110 or the multiplexer 107 are not in the vehicle 100 but in the network, e.g. as part of a computing cloud.

[0056] In this case the steps of the first method are for example executed in the same order however on the respective device. To that end additional steps of sending the audio, the images or the information are performed before the step of joining (205).

[0057] A second method for playing the video is explained below making reference to figure 3.

[0058] The second method for example starts whenever reception of the video starts at the receiver.

[0059] After the start a step 301 is executed.

[0060] In the step 301 the information about the assessed traffic situation is separated from the video. In the example the received video is decoded according to the H.263 standard and the text or audio containing the information about the assessed traffic situation is separated from the image or audio. For example a variable is set to the text "Heavy traffic Warning". In case the information about the assessed traffic situation is indicating "normal", in the video, the variable is set to "normal".

[0061] Afterwards a step 302 is executed.

[0062] In step 302 a test is performed to determine whether the information about the assessed traffic situation indicates "normal". In case the information indicates "normal" a step 305 is executed. Otherwise a step 303 is executed. For example in case the variable contains the text "normal" the step 305 is executed. This means that it is determined whether to send an assistive message to the output 109 or not depending on the information about the assessed traffic situation.

[0063] In step 303 an assistive message is determined from the information. In the example of the message "Heavy traffic Warning" for example an assistive message corresponding to a request to mute is created. This is for example a text "please mute audio" or a respective symbol is used This means that the assistive message is determined from the information about the assessed traffic situation. Afterwards a step 304 is executed.

[0064] In step 304 the assistive message is output. For example the text "please mute audio" or the symbol is displayed on the display. Optionally or alternatively a corresponding audio signal may be played. Afterwards the method ends.

[0065] Step 304 may be repeated before the end of the second method for one or several times until the connection is ended.

[0066] In step 305 the video is output without the information about the assessed traffic situation. The image is for example displayed on the display and the audio played via the speaker.

[0067] Afterwards step 301 is executed.

[0068] Preferably in the implementation of the device the vehicle receiver or the methods described above, the information about the traffic situation, the image of the video or the audio of the video coincide with one or more of the other.

[0069] Preferably in the implementation of the device the vehicle receiver or the methods described above the image of the video, the audio of the video or the video are live image, live audio or live video respectively.

[0070] The description and drawings merely illustrate the principles of the invention. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody the principles of the invention and are included within its spirit and scope. Furthermore, all examples recited herein are principally intended expressly to be only for pedagogical purposes to aid the reader in understanding the principles of the invention and the concepts contributed by the inventors to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions. Moreover, all statements herein reciting principles, aspects, and embodiments of the invention, as well as specific examples thereof, are intended to encompass equivalents thereof.

[0071] The functions of the various elements shown in the figures, including any functional blocks labeled as multiplexer, de-multiplexer, sender, receiver or analyzer, may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared. Moreover, explicit use of the term 'processor' or 'controller' should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (DSP) hardware, network processor, application specific integrated circuit (ASIC), field programmable gate array (FPGA), read only memory (ROM) for storing software, random access memory (RAM), and non volatile storage. Other hardware, conventional and/or custom, may also be included. Similarly, any switches shown in the figures are conceptual only. Their function may be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the implementer as more specifically understood from the context.

[0072] It should be appreciated by those skilled in the art that any block diagrams herein represent conceptual views of illustrative circuitry embodying the principles of the invention. Similarly, it will be appreciated that the flow charts represent various processes which may be substantially represented in computer readable medium and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.

[0073] A person of skill in the art would readily recognize that steps of various above-described methods can be performed by programmed computers. Herein, some embodiments are also intended to cover program storage devices, e.g., digital data storage media, which are machine or computer readable and encode machine-executable or computer-executable programs of instructions, wherein said instructions perform some or all of the steps of said above-described methods. The program storage devices may be, e.g., digital memories, magnetic storage media such as a magnetic disks and magnetic tapes, hard drives, or optically readable digital data storage media. The embodiments are also intended to cover computers programmed to perform said steps of the above-described methods.

Claims

1. A first device (110, 103) comprising an analyzer (110) adapted to assess (203) a traffic situation from a captured image of the outside of a vehicle (100) and a sender (104) adapted to send (206) information about the assessed traffic situation.

2. The device (110, 103) according to claim 1, comprising a multiplexer (103) adapted to join (205) the captured image with recorded audio from a passenger compartment of the vehicle (100).

3. The device (110, 103) according to claim 1 or 2, a microphone (102) for recording (202) audio of the passenger compartment of the vehicle (100).

4. The device (110, 103) according to claim 1 to 3, comprising a camera (101) for capturing (201) the image of the outside of the vehicle (100).

5. A vehicle (100) comprising a device (101, 102, 103) according to any of the aforementioned claims.

6. A second device comprising a receiver (105) and an output (109), wherein the receiver (105) is adapted to receive information about an assessed traffic situation and adapted to determine whether to send an assistive message to the output (109) or not depending on the information about the assessed traffic situation.

7. The device according to claim 6, wherein the assistive message is determined from the information about the assessed traffic situation.

8. A method comprising assessing (203) a traffic situation from a captured image of the outside of a vehicle (100), and sending (206)the information about the assessed traffic situation.

9. The method according to claim 8, comprising joining (205) the captured image with recorded audio from a passenger compartment of the vehicle (100) and information about the assessed traffic situation.

10. The method according to claim 8 or 9, comprising capturing (201) the image of the outside of the vehicle (100).

11. The method to claims 8 to 9, comprising recording (202) the audio of the passenger compartment of the vehicle (100).

12. A method comprising receiving information about a assessed traffic situation, and determining whether to send an assistive message or not depending on the information about the assessed traffic situation.

13. The method according to claim 12, wherein the assistive message is determined depending on the received assessed traffic situation.

14. The device (110, 103) according to claims 1 to 4, the vehicle (100) according to claim 5, the receiver (107, 109) according to claims 6 and 7, or the method according to claims 8 to 13, wherein the information about the traffic situation, the image or the audio of coincide with one or more of the other.

15. The device (110, 103) according to claims 1 to 4, the vehicle (100) according to claim 5, the receiver (107, 109) according to claims 6 and 7, or the method according to claims 8 to 13, wherein the image or the audio are live image or live audio respectively.

Drawing