1. Technical Field
[0001] The present invention relates generally to the field of the user interaction with
devices connected together. More specifically, the present invention concerns a system
for controlling an electronic device through a user interface displayed on a display
device of a head mounted unit. It also concerns a head mounted unit for such a system.
2. Background Art
[0002] The present invention aims at improving the communication with any electronic devices
present in the home. It is still a problem for people in their living room to be able
to find the right remote control to communicate with a given electronic device. This
problem becomes a nightmare when the number of electronic devices is high and when
each of them is equipped with its own remote control. So there is a clear need of
a universal remote control.
[0003] But to be universal, a remote control should be able to address any electronic device
whatever the type of command to be applied. This is generally not feasible to have
a simplified set of buttons so as to address every kind of electronic devices for
any command.
[0004] Some universal remote control units exist. They are able to address different electronic
devices thanks to a pre-setting of each command to a given button of the remote control
unit. So the electronic devices controlled by the remote control unit belong to a
predefined list. The addition of a device to this list requires pre-setting other
buttons of the remote control. As buttons should be associated with a command in a
comprehensive way, their number can be huge.
[0005] On the other hand, there are user interfaces available for any electronic device
having a display (e.g. a TV) or any electronic devices directly connected to a display
(e.g. a set-top-box). In these cases the remote control unit associated with the electronic
device can be simpler. One can navigate inside a menu of said user interface displayed
on this display thanks to a set of arrows representing the different directions and
a validation button. It allows reducing significantly the number of buttons of the
remote control. But this is not adapted to electronic devices without display, like
for example an audio amplifier. There are a lot of electronic devices for which no
universal remote control and no display can be associated with, although they could
be connected to a home communication framework or, more generally, to a local network,
with which the electronic devices can communicate.
[0006] A purpose of the present invention is to propose a remote control for any electronic
devices connected to at least a local network.
[0007] There are also some remote control systems using see-through glasses for providing
a user interface. In such a system, the see-through glasses display the user interface
associated with the electronic device to be controlled. The selection of a command
in the user interface can be performed by moving the head in such a way that the command
to be performed is aligned with the device to be controlled. Such a system is for
example disclosed in the patent application
US 2003/0020707. This system does not require any dedicated remote control unit but the action of
alignment to be made for selecting a command is not so easy to be executed, especially
if there are lots of electronic devices in the field of view of the user. So the selection
of commands by head moving is not easy, especially if the user interface comprises
a lot of buttons.
[0008] In other systems using see-through glasses, it is proposed to the user to make the
selection of a command in the user interface displayed by the see-through glasses
by using his fingers. Such a system is for example disclosed in the patent application
US 2011/0234386. A command in the user interface displayed on the see-through glasses is selected
by placing a finger on an icon associated with said command. This system does not
also require any dedicated remote control unit but the placement of the finger on
the icon of the command to be performed without going through other icons and without
risking selecting wrong commands is not so easy to execute. So this type of system
is not reliable.
[0009] A purpose of the present invention is to propose a system able to reliably select
a command of a user interface displayed through a head mounted display.
3. Summary of Invention
[0010] The present invention concerns a system for controlling an electronic device through
a user interface associated with said electronic device, said user interface comprising
a plurality of commands. The system comprises a head mounted unit comprising a display
device for displaying said dedicated user interface and a camera for capturing an
image of the electronic device to be controlled, a video processing unit, and a remote
control unit. The head mounted unit, the video processing unit, the remote control
unit and the electronic device are configured to communicate with each other. In this
system, the video processing unit is configured to identify the electronic device
to be controlled in the image captured by the camera, to transmit the user interface
associated with the identified electronic device to the display device, and to transmit
to the electronic device a command instruction associated with the command selected
by the remote control unit in said user interface displayed by the display device.
[0011] In one particular embodiment, he head mounted unit, the video processing unit, the
remote control unit and the electronic device are configured to communicate via a
communication framework. The communication framework designates a framework by which
devices connected to this framework can exchange data. The data are exchanged on one
or more communication networks. These communication networks may be wireless or not.
And the data are exchanged according to one or more communication protocols. The communication
framework is for example a home communication framework like QeO framework developed
by Technicolor.
[0012] The invention exploits the connection of electronic devices of any type to a communication
framework to allow exchanges between a remote control combined with a display for
controlling a connected electronic device.
[0013] The remote control unit is any electronic device connected to the communication framework
having means to navigate in the menus of the user interface.
[0014] In one particular embodiment, the remote control unit is a hand-held device. It can
be a smartphone or any hand held electronic device like a portable media player.
[0015] In one particular embodiment, the head mounted device comprises see-through glasses,
said see-through glasses being equipped with said camera.
[0016] In one particular embodiment, the video processing unit is carried by the head mounted
unit and is connected directly to the display device and/or the camera.
[0017] In a variant, the video processing unit is included in the remote control unit.
[0018] In one particular embodiment, the video processing unit is configured to identify,
in the image captured by the camera, the electronic device to be controlled by detecting,
in said image, a visual code associated with said electronic device.
[0019] In one particular embodiment, the visual code is a barcode or a QR code.
[0020] In one particular embodiment, the video processing unit is configured to identify,
in the image captured by the camera, the electronic device to be controlled by detecting,
in said image, the picture of the front end of said electronic device.
[0021] The invention concerns also a head mounted unit comprising a display device for displaying
a user interface, a camera for capturing an image of the electronic device to be controlled,
and a video processing unit configured to identify the electronic device to be controlled
in the image captured by the camera, to transmit the user interface associated with
the identified electronic device to the display device, to receive a command from
a remote control, and to transmit a command instruction associated with the selected
command to the electronic device.
[0022] In one particular embodiment, the video processing unit receives the command from
the remote control via a communication framework and transmits the command instruction
associated with the selected command to the electronic device via said communication
framework.
[0023] In one particular embodiment, the video processing unit is configured to identify,
in the image captured by the camera, the electronic device to be controlled by detecting,
in said image, a visual code associated with said electronic device.
[0024] In one particular embodiment, the visual code is a barcode or a QR code.
[0025] In one particular embodiment, the video processing unit is configured to identify,
in the image captured by the camera, the electronic device to be controlled by detecting,
in said image, the front end of said electronic device.
4. Brief description of the drawings
[0026] The invention can be better understood with reference to the following description
and drawings, given by way of example and not limiting the scope of protection, and
in which:
- Figure 1 illustrates a system for controlling an electronic device according the principles
of the invention;
- Figure 2 is a schematic view of the global architecture of the system of Figure 1.
- Figure 3 is a flow chart of the successive steps implemented by the system according
to the invention; and
- Figure 4 illustrates the capture of an image of an electronic device to be controlled
according to the invention.
5. Description of embodiments
[0027] In reference to Figures 1 and 2, the system according to the invention comprises
a see through glasses 10 having a display for displaying user interface of electronic
devices 20 to be controlled, a camera 11 for capturing an image of the electronic
devices 20 to be controlled, a video processing unit 12, and a remote control unit
13. The see-through glasses 10 are worn by a user. All these components are connected
together and are able to communicate with each other.
[0028] In Figures 1 and 2, the see through glasses 10, the camera 11, the video processing
unit 12, the remote control unit 13 and the electronic devices 20 communicate with
each other via a communication framework 14.
[0029] The communication framework 14 is preferably a local framework like a home communication
framework. For example, the components of the system are all connected to a local
network, for example a wireless network (WiFi), and communicate together via a communication
protocol like QeO framework developed by Technicolor.
[0030] The communication framework 14 can also be used through a global network like internet.
[0031] The remote control unit 13 is an electronic device connected to the network and using
the communication framework and having means to navigate in the menus of a user interface.
It can be a smartphone or any hand held electronic device like a portable media player.
This device may comprise keys (tactile or not), like arrow keys and enter key, for
navigating in menus of user interfaces.
[0032] The video processing unit 12 may be a separate unit as illustrated by Figure 1, may
be integrated to the see-through glasses 10 or may be the processing means of a set
top box. The video processing unit may also be processing means of the remote control
unit, for example processing means of a smart phone.
[0033] If the video processing unit 12 is integrated to the see-through glasses 10, it is
directly connected to the camera 11 and the display of the see-through glasses without
using the communication framework 14.
[0034] The video processing unit 12 is configured to identify the electronic device 20 to
be controlled in the image captured by the camera 11 and to transmit to the display
device the user interface associated with the identified electronic device via the
communication framework 14. This user interface is displayed by the see-through glasses
10 and the remote control unit 13 is used to select commands in this user interface.
[0035] Figure 3 shows the successive steps implemented by the system of Figure 1 and 2.
[0036] In a step S1, the camera 11 captures an image of the device 20 to be controlled.
The camera is directed towards the device 20 by the user by looking at the device.
An image of the front end of the device is captured by the camera 11 as shown in Figure
4.
[0037] In a step S2, the captured image is sent to the video processing unit 12 via the
communication framework 14.
[0038] In a step S3, the image is processed by the video processing unit 12 to detect the
device 20 to be controlled and to identify it. As mentioned before, the device 20
is detected and identified by detecting a visual code present in its front end, for
example a QR code 21, or by comparing a picture of its front end to pre-stored front
ends pictures. A user interface associated with the identified device 20 is selected.
[0039] In a step S4, the video processing unit 14 sends the user interface software associated
with the identified device 20 to the display 10 of the see through glasses.
[0040] In a step S5, the display device 10 of the see through glasses displays the received
user interface software and displays the user interface on its screen.
[0041] In a step S6, the user selects by the remote control unit 13 a command in the displayed
user interface for controlling the electronic device 20. This selection is displayed
on the see-through glasses. It can correspond to different menus that are reached
using the remote control unit. So the video processing unit 12 receives a command
representative of the command selected by the user in the user interface.
[0042] In step S7, the video processing unit 12 transmits a command instruction associated
with the command selected by the user to the electronic device 20.
[0043] If several electronic devices (2 devices stacked together) are present in the captured
image and detected by the video processing unit 12, the video processing unit displays
a menu for proposing to the user to select the electronic device to be controlled.
This menu is displayed by the display device 10 and the selection of the device to
be controlled is made by the remote control unit 13. In a variant, if several electronic
devices are detected in the captured image, one of the detected device's user interfaces
is proposed by default to the user on the display of see-through glasses. If the displayed
user interface is not the right one, the user can change the user interface by selecting
a dedicated icon in the user interface or by moving his head in the direction of the
other device he wants to select. In a more elaborated embodiment, the video processing
unit 12 sends through the communication framework 14 a message to the selected electronic
device 20 asking it to show that it has been selected by lighting a LED of the front
end of the electronic device 20.
[0044] Thus, according to the invention, any electronic device 20 connected to the communication
framework 14 and identifiable by a specific visual code or by identifying the front
end picture can be controlled by the system. It is then easy to drive these electronic
devices with a simple universal remote control 13 that can be a dedicated device or
even a smartphone.
[0045] The great advantage is to simplify the way you drive your electronic devices at home,
avoiding spending time to find the right remote control in your living room. The way
to control the electronic device 20 is then as simple as with any electronic device
having a display. At last, it should be noted that in one embodiment of the invention,
the previously mentioned processing means and/or means to navigate can be implemented
in hardware in a programmable FPGA
("Field Programmable Gate Array") component or ASIC (
"Application-Specific Integrated Circuit") component.
[0046] In one embodiment of the invention, a device such as a head mounted unit or the video
processing unit or a remote control unit, comprises a computing unit (for example
a CPU, for "
Central Processing Unit")
, and one or several memory units (for example a RAM (for "
Random Access Memory") block in which intermediate results can be stored temporarily during the execution
of instructions a computer program, or a ROM block in which, among other things, computer
programs are stored, or an EEPROM ("
Electrically-Erasable Programmable Read-Only Memory") block, or a flash block). Computer programs are made of instructions that can be
executed by the computing unit. Such device can also comprise a dedicated unit constituting
an input-output interface to allow the device to communicate with other devices. In
particular, this dedicated unit can be connected with an antenna (in order to perform
communication without contacts), or with serial ports (to carry communications via
physical contacts). These units can exchange data through buses for example together.
1. System for controlling an electronic device (20) through a user interface associated
with said electronic device, said user interface comprising a plurality of commands,
characterized in that the system comprises:
a head mounted unit comprising a display device (10) for displaying said dedicated
user interface and a camera (11) for capturing an image of the electronic device to
be controlled,
a video processing unit (12), and
a remote control unit (13),
and
in that:
- the head mounted unit, the video processing unit, the remote control unit and the
electronic device are configured to communicate with each other, and
- the video processing unit (12) is configured to identify the electronic device to
be controlled in the image captured by the camera, to transmit the user interface
associated with the identified electronic device to the display device, and to transmit
to the electronic device a command instruction associated with command selected by
the remote control unit (13) in said user interface displayed by the display device.
2. System according to claim 1, wherein the remote control unit (13) is a hand-held device.
3. System according to claim 1 or 2, wherein the head mounted device comprises see-through
glasses, said see-through glasses being equipped with said camera (11).
4. System according to any one of claims 1 to 3, wherein the video processing unit (12)
is carried by the head mounted unit and is connected to the display device (10) and/or
the camera (11).
5. System according to any one of claims 1 to 4, wherein the video processing unit (12)
is configured to identify, in the image captured by the camera, the electronic device
to be controlled by detecting, in said image, a visual code associated with said electronic
device.
6. System according to claim 5, wherein the visual code is a barcode or a QR code (21).
7. System according to any one of claims 1 to 4, wherein the video processing unit (12)
is configured to identify, in the image captured by the camera, the electronic device
to be controlled by detecting, in said image, the picture of the front end of said
electronic device.
8. Head mounted unit comprising a display device (10) for displaying a user interface
and a camera (11) for capturing an image of the electronic device to be controlled,
characterized in that it further comprises a video processing unit (12) configured to identify the electronic
device to be controlled in the image captured by the camera, to transmit the user
interface associated with the identified electronic device to the display device,
to receive a command from a remote control unit (13), and to transmit a command instruction
associated with the selected command to the electronic device.
9. Head mounted unit according to claim 8, wherein the video processing unit (12) is
configured to identify, in the image captured by the camera, the electronic device
to be controlled by detecting, in said image, a visual code associated with said electronic
device.
10. Head mounted unit according to claim 9, wherein the visual code is a barcode or a
QR code (21).
11. Head mounted unit according to claim 8, wherein the video processing unit (12) is
configured to identify, in the image captured by the camera, the electronic device
to be controlled by detecting, in said image, the picture of the front end of said
electronic device.