FIELD OF THE INVENTION
[0001] This invention relates to a food preparation assembly having a food preparation chamber
and a camera assembly for obtaining one or more images inside the food preparation
chamber.
[0002] The invention further relates to a method of controlling a camera assembly suitable
for obtaining one or more images inside a food preparation chamber of a food preparation
assembly, and a computer program for implementing the method.
[0003] The invention yet further relates to a method of determining a relationship for adjusting
an image characteristic of one or more images inside a food preparation chamber of
a food preparation assembly.
BACKGROUND OF THE INVENTION
[0004] Due to busy daily life, people usually have very limited time for home cooking. Moreover,
many people lack the cooking experience necessary to obtain satisfactory results.
There is accordingly growing demand for smart food preparation assemblies capable
of delivering food prepared according to the user's preference. Benefits of such smart
food preparation assemblies include greater user convenience, better taste and nutritional
value of the food prepared using such food preparation assemblies, and minimizing
or avoidance of harmful substances being generated in a cooking process using such
food preparation assemblies.
[0005] Including a camera in the food preparation assembly, for example by integrating a
camera into a kitchen appliance, can play an important role in smart assisted food
preparation, e.g. cooking. For example, a food type can be recognized by food image
recognition and a cooking setting can be determined based on food image analysis results.
Other applications of cameras in food preparation assemblies include cooking process
control, browning control, remote monitoring, burning prevention, and content for
social media sharing.
[0006] Due to the above-mentioned benefits, camera-integrated kitchen appliances have recently
been introduced onto the market. However, various challenges remain in terms of obtaining
images which can be reliably used for, in particular, food recognition, cooking process
control, browning control, remote monitoring, and burning prevention.
SUMMARY OF THE INVENTION
[0007] The invention is defined by the claims.
[0008] According to examples in accordance with an aspect of the invention, there is provided
a food preparation assembly comprising: a food preparation chamber; a camera assembly
for obtaining one or more images inside the food preparation chamber, the camera assembly
being controllable to adjust an image characteristic of the one or more images; a
radiation assembly controllable to adopt a radiation status in which the radiation
assembly emits radiation comprising visible light inside the food preparation chamber;
and one or more processors configured to: identify the radiation status; and control
the camera assembly to adjust the image characteristic using a predetermined relationship
for adjusting the image characteristic according to the identified radiation status.
[0009] Using such a predetermined relationship to adjust the image characteristic, such
as exposure and/or white balance, according to the identified radiation status of
the radiation assembly can assist to provide faster and/or more repeatable image characteristic
adjustment relative to, for instance, adjustment based only on an optical condition
detected inside the food preparation chamber when obtaining the image(s).
[0010] In some embodiments, the camera assembly may be configured to obtain a sequence of
images of the inside of the food preparation chamber, e.g. a sequence of images constituting
a video of the inside of the food preparation chamber.
[0011] Alternatively or additionally, the camera assembly may be configured to obtain a
still image of the inside of the food preparation chamber.
[0012] By identifying the radiation status(es) in advance and determining the predetermined
relationship for adjusting the image characteristic, e.g. camera assembly capture
setting(s) and/or image processing setting(s), according to the radiation status,
the image(s) can, for example, be brought into closer conformity with a "real" view
of the food ingredients perceptible with the naked eye, etc.
[0013] By bringing the image(s) into closer conformity with such a "real" view of the food
ingredients by controlling the camera assembly to, for instance, adjust the exposure
and/or white balance using the predetermined relationship, more precise/reliable food
recognition, cooking process control, browning control, doneness status determination,
remote monitoring, and/or burning prevention may be achieved.
[0014] In some embodiments, the radiation assembly is controllable to select the radiation
status from a plurality of radiation statuses, and the one or more processors is or
are configured to identify the selected radiation status, with the predetermined relationship
permitting adjusting of the image characteristic according to the identified selected
radiation status.
[0015] The image characteristic can accordingly be adjusted according to which of the plurality
of radiation statuses is selected.
[0016] In embodiments in which the radiation assembly is controllable to select the radiation
status from the plurality of radiation statuses, the plurality of radiation statuses
can comprise: a first radiation status; and a second radiation status, with the first
radiation status causing a first optical condition in the food preparation chamber,
and the second radiation status causing a second optical condition in the food preparation
chamber which is different from the first optical condition.
[0017] In such embodiments, the predetermined relationship may be determined such that the
image characteristic is adjusted to compensate for the difference between the first
optical condition and the second optical condition.
[0018] By compensating for the difference between the first and second optical conditions,
the image(s) obtained when the first radiation status is adopted can be comparable
to the image(s) obtained when the second radiation status is adopted.
[0019] For example, should the optical condition include an intensity of visible light inside
the food preparation chamber, adjustment of the image characteristic includes adjustment
of the exposure of the image(s) to compensate for the difference in intensity. Alternatively
or additionally, should the optical condition include a spectral composition of the
light inside the food preparation chamber, adjustment of the image characteristic
includes adjustment of the colour, e.g. white, balance of the image(s) to compensate
for the difference in spectral composition.
[0020] It is noted, for the avoidance of doubt, that the plurality of radiation statuses
can include further radiation statuses in addition to the above-described first and
second radiation statuses, such as a third radiation status, a fourth radiation status,
a fifth radiation status, and so on. In such embodiments, the predetermined relationship
may be determined such that the image characteristic is adjusted to compensate for
differences between the respective optical conditions caused by selection of each
of the radiation statuses.
[0021] In some embodiments, the radiation assembly comprises at least one lamp for illuminating
the inside of the food preparation chamber. In such embodiments, the radiation status
comprises an illumination status of the, e.g. each of the, at least one lamp.
[0022] The illumination status can, for instance, be an on/off status of the, e.g. each
of the, at least one lamp.
[0023] Alternatively or additionally, the illumination status can be an intensity of the
illumination provided by the, e.g. each of the, at least one lamp and/or the illumination
status can be a measure of the spectral composition of the illumination provided by
the, e.g. each of the, at least one lamp.
[0024] In some embodiments, the radiation assembly comprises at least one heating element
arranged to radiate inside the food preparation chamber. In such embodiments, the
radiation status comprises a heating status of the, e.g. each of the, at least one
heating element.
[0025] The heating status can, for instance, be an on/off status of the, e.g. each of the,
at least one heating element.
[0026] In embodiments in which the radiation status comprises the illumination status of
the at least one lamp and the heating status of the at least one heating element,
the illumination status can be an on/off status of the at least one lamp, with the
heating status being an on/off status of the at least one heating element.
[0027] Alternatively or additionally, the heating status can be a degree of heating provided
by the, e.g. each of the, at least one heating element and/or the heating status can
be a measure of the spectral composition of the radiation provided by the, e.g. each
of the, at least one heating element.
[0028] The food preparation assembly may comprise a control unit having a control configuration
arranged to control the radiation assembly to adopt the radiation status. In such
embodiments, the one or more processors is or are configured to identify the radiation
status based on said control configuration.
[0029] Thus, the radiation status can be obtained, in other words directly obtained, from
the control unit controlling the radiation assembly. This may advantageously obviate
sensing of an optical condition inside the food preparation chamber in order to adjust
the image characteristic.
[0030] In embodiments in which the control configuration comprises a control sequence which
controls the radiation status of the radiation assembly to change over time, the one
or more processors may be configured to control the camera assembly to adjust the
image characteristic (using the predetermined relationship) to coincide with the change
of the radiation status. This may enable particularly responsive adjustment of the
image characteristic which may be especially useful, for example, when the camera
assembly is configured to obtain a sequence of images, e.g. video, during the change
of the radiation status.
[0031] In some embodiments, the radiation status, e.g. each of the radiation statuses when
there the radiation status is selectable from a plurality of radiation statuses, is
associated with a power consumption of the radiation assembly, and the one or more
processors is or are configured to identify the radiation status based on the (respective)
power consumption of the radiation assembly.
[0032] The food preparation assembly may comprise a sensor for sensing radiation of the
radiation assembly. In such embodiments, the one or more processors is or are configured
to identify the radiation status based on the sensed radiation.
[0033] In this way, the radiation status can be identified according to the radiation detected
via the sensor.
[0034] In some embodiments, the camera assembly comprises an image capture module, and the
one or more processors may be configured to control at least one image capture setting
of the image capture module according to the identified radiation status to adjust
the image characteristic.
[0035] The identified radiation status may thus be used to adjust the acquisition of the
image(s).
[0036] The at least one image capture setting may, for instance, comprise an aperture size,
a shutter speed and/or a sensitivity setting.
[0037] The food preparation assembly may comprise an image processing module, and the one
or more processors may be configured to control at least one image processing setting
of the image processing module according to the identified radiation status to adjust
the image characteristic.
[0038] The image processing module can, for example, be included in the camera assembly,
a user device, such as a smart phone or tablet computer, and/or a cloud-based server.
[0039] In such embodiments, the identified radiation status is used to adjust the processing
of the image(s) by the image processing module.
[0040] The at least one processing setting may, for instance, comprise a colour balance
setting. Such a colour balance setting can, for example, be a white balance setting.
[0041] In at least some embodiments, the food preparation assembly comprises or is a domestic
food preparation assembly. For example, the food preparation assembly comprises a
domestic kitchen appliance.
[0042] Examples of such a domestic kitchen appliance include an air fryer, oven, steamer,
pressure cooker and rice cooker.
[0043] Particular mention is made of embodiments in which the camera assembly is integral,
in other words built-in, to such a domestic kitchen appliance.
[0044] In alternative embodiments, the camera assembly is a removable or detachable component
of the food preparation assembly, e.g. domestic food preparation assembly. A window
into the food preparation chamber may be provided, and the camera assembly may be
arranged such as to obtain the image(s) inside the food preparation chamber via the
window in order to assemble the food preparation assembly, e.g. the domestic food
preparation assembly.
[0045] For instance, the camera assembly, and optionally at least one of the one or more
processors, may be included in user device separate from a kitchen appliance comprising
the food preparation chamber and the radiation assembly, such as a smart phone or
tablet computer. The processor(s) may alternatively or additionally be included in
a cloud-based server. The food preparation assembly, e.g. domestic food preparation
assembly, may be assembled by arranging the camera assembly such as to obtain the
image(s) inside the food preparation chamber via the window.
[0046] In some embodiments, the food preparation chamber and the radiation assembly are
included in a kitchen appliance, e.g. domestic kitchen appliance, and the processor(s)
included in the user device and/or the cloud-based server is or are in communication,
e.g. wireless communication, with the kitchen appliance, e.g. with the above-described
control unit, in order to enable the processor(s) to identify the radiation status
and control the camera assembly to adjust the image characteristic of the one or more
images using the predetermined relationship.
[0047] According to another aspect there is provided a method of controlling a camera assembly
suitable for obtaining one or more images inside a food preparation chamber of a food
preparation assembly, the food preparation assembly further comprising a radiation
assembly controllable to adopt a radiation status in which the radiation assembly
emits radiation comprising visible light inside the food preparation chamber, the
method comprising: identifying the radiation status; and controlling the camera assembly
to adjust an image characteristic of the one or more images using a predetermined
relationship for adjusting the image characteristic according to the identified radiation
status.
[0048] The method may further comprise operating the camera assembly to obtain one or more
images inside the food preparation chamber.
[0049] The image characteristic may be adjusted according to the identified radiation status
in the obtained one or more images.
[0050] In some embodiments, the radiation assembly is controllable to select the radiation
status from a plurality of radiation statuses, and the identifying comprises identifying
the selected radiation status, with the predetermined relationship permitting adjusting
of the image characteristic according to the identified selected radiation status.
[0051] The image characteristic can accordingly be adjusted according to which of the plurality
of radiation statuses is selected.
[0052] In embodiments in which the radiation assembly is controllable to select the radiation
status from the plurality of radiation statuses, the plurality of radiation statuses
can comprise: a first radiation status; and a second radiation status, with the first
radiation status causing a first optical condition in the food preparation chamber,
and the second radiation status causing a second optical condition in the food preparation
chamber which is different from the first optical condition.
[0053] In such embodiments, the predetermined relationship may be determined such that the
image characteristic is adjusted to compensate for the difference between the first
optical condition and the second optical condition.
[0054] In some embodiments, the radiation assembly comprises at least one lamp for illuminating
the inside of the food preparation chamber. In such embodiments, the identifying comprises
identifying an illumination status of the, e.g. each of the, at least one lamp.
[0055] In some embodiments, the radiation assembly comprises at least one heating element
arranged to radiate inside the food preparation chamber. In such embodiments, the
identifying comprises identifying a heating status of the, e.g. each of the, at least
one heating element.
[0056] In embodiments in which the radiation assembly comprises the at least one lamp and
the at least one heating element, the identifying may comprise identifying the illumination
status of the at least one lamp and identifying the heating status of the at least
one heating element. In such embodiments, the illumination status can, for example,
be an on/off status of the at least one lamp, with the heating status being an on/off
status of the at least one heating element.
[0057] The food preparation assembly may comprise a control unit having a control configuration
arranged to control the radiation assembly to adopt the radiation status. In such
embodiments, the identifying may comprise identifying the radiation status based on
the control configuration.
[0058] In some embodiments, the method comprises determining a power consumption of the
radiation assembly, with the identifying comprising identifying the radiation status
based on the determined power consumption.
[0059] The food preparation assembly may comprise a sensor for sensing radiation of the
radiation assembly. In such embodiments, the identifying may comprise identifying
the radiation status based on the sensed radiation.
[0060] In some embodiments, the camera assembly comprises an image capture module, and the
controlling comprises controlling at least one image capture setting of the image
capture module according to the identified radiation status to adjust the image characteristic.
The at least one image capture setting may, for instance, comprise an aperture size,
a shutter speed and/or a sensitivity setting.
[0061] Alternatively or additionally, the food preparation assembly may comprise an image
processing module, and the controlling comprises controlling at least one image processing
setting of the image processing module according to the identified radiation status
to adjust the image characteristic. The at least one processing setting may, for instance,
comprise a colour balance setting. Such a colour balance setting can, for example,
be a white balance setting.
[0062] According to yet another aspect there is provided a computer program comprising computer
program code which is configured, when said computer program is run on one or more
processors included in a food preparation assembly further comprising a food preparation
chamber, a camera assembly for obtaining one or more images inside the food preparation
chamber, and a radiation assembly controllable to adopt a radiation status in which
the radiation assembly emits radiation comprising visible light inside the food preparation
chamber, to cause said one or more processors to implement the method according to
any of the embodiments described herein.
[0063] One or more non-transitory computer readable media may be provided, which non-transitory
computer readable media have a computer program stored thereon, with the computer
program comprises computer program code which is configured, when the computer program
is run on the one or more processors, to cause the one or more processors to implement
the method according to any of the embodiments described herein.
[0064] The one or more processors can be included in a kitchen appliance included in the
food preparation assembly, in a user device, for example a smart phone or tablet computer,
separate from such a kitchen appliance, and/or in a cloud-based server.
[0065] According to a further aspect there is provided a method of determining a relationship
for adjusting an image characteristic of one or more images inside a food preparation
chamber of a food preparation assembly comprising a radiation assembly, the one or
more images being obtained using a camera assembly, the method comprising: identifying
at least one radiation status of the radiation assembly in which radiation comprising
visible light is emitted inside the food preparation chamber; measuring, for each
of the at least one radiation status, an optical condition; and determining, using
said optical condition measured for each of the identified at least one radiation
status, said relationship for adjusting the image characteristic according to the
identified at least one radiation status.
[0066] This method may be used to establish/determine the above-described predetermined
relationship for adjusting the image characteristic according to the identified radiation
status.
[0067] More generally, embodiments described herein in relation to the food preparation
assembly and the method of controlling the camera assembly may be applicable to the
method of determining the relationship, and embodiments described herein in relation
to the method of determining the relationship may be applicable to the food preparation
assembly and the method of controlling the camera assembly.
BRIEF DESCRIPTION OF THE DRAWINGS
[0068] For a better understanding of the invention, and to show more clearly how it may
be carried into effect, reference will now be made, by way of example only, to the
accompanying drawings, in which:
FIG. 1 schematically depicts a food preparation assembly according to an example;
FIG. 2 provides a block diagram of a camera assembly according to an example;
FIG. 3 provides a flowchart of a method of operating a camera assembly according to
an example;
FIG. 4 graphically represents adjustment of shutter speed and white balance according
to radiation statuses of a radiation assembly as cooking progresses;
FIG. 5 provides a flowchart of an exemplary method of determining a relationship for
adjusting an image characteristic of one or more images; and
FIG. 6 provides a flowchart for illustrating how the relationship determined via the
method shown in FIG. 5 can be applied to an exemplary method of operating a camera
assembly.
DETAILED DESCRIPTION OF THE EMBODIMENTS
[0069] The invention will be described with reference to the Figures.
[0070] It should be understood that the detailed description and specific examples, while
indicating exemplary embodiments of the apparatus, systems and methods, are intended
for purposes of illustration only and are not intended to limit the scope of the invention.
These and other features, aspects, and advantages of the apparatus, systems and methods
of the present invention will become better understood from the following description,
appended claims, and accompanying drawings. It should be understood that the Figures
are merely schematic and are not drawn to scale. It should also be understood that
the same reference numerals are used throughout the Figures to indicate the same or
similar parts.
[0071] Provided is a food preparation assembly, which can also be referred to as a food
preparation system, comprising a food preparation chamber, and a camera assembly for
obtaining one or more images inside the food preparation chamber. The camera assembly
is controllable to adjust an image characteristic of the one or more images. The food
preparation assembly comprises a radiation assembly controllable to adopt a radiation
status in which the radiation assembly emits radiation comprising visible light inside
the food preparation chamber. One or more processors is or are configured to: identify
the radiation status; and control the camera assembly to adjust the image characteristic
using a predetermined relationship for adjusting the image characteristic according
to the identified radiation status. Further provided is a method of controlling a
camera assembly suitable for obtaining one or more images inside a food preparation
chamber of a food preparation assembly, a computer program for implementing the method,
and a method of determining a relationship for adjusting an image characteristic of
one or more images inside a food preparation chamber of a food preparation assembly.
[0072] Obtaining one or more images inside the food preparation chamber is desirable for
a number of reasons. In particular, the one or more images can be used for recognizing
food present in the food preparation chamber, cooking process monitoring, and/or cooking
process control based on analysis of the image(s).
[0073] Consistent image quality is particularly important for such uses. However, ensuring
sufficiently consistent quality of the image(s) can be challenging, particularly due
to changing optical conditions within the food preparation chamber resulting from
adjustments in operation of the radiation assembly also included in the food preparation
assembly, which radiation assembly emits radiation comprising visible light inside
the food preparation chamber.
[0074] For example, heating element(s) included in the food preparation assembly may shift
between on and off statuses during cooking. The illumination intensity and colour
temperature in the food preparation chamber may therefore vary significantly due to
the heating element(s) shifting between such statuses. If, for instance, the white
balance and exposure are not adjusted, both the colour and brightness of the image(s)
may be correspondingly significantly affected. As a result, the capability of the
image(s) to represent the appearance of the food may be adversely affected. This may
present difficulty in, in particular, food recognition, cooking process control, browning
control, doneness status determination, remote monitoring, and/or burning prevention,
etc. using the image(s).
[0075] Whilst the camera assembly can, for instance, itself detect an optical condition
within the food preparation chamber and adjust an image characteristic of the image(s)
based on such a detected optical condition, there may be a lag to such adjustment
and/or the adjustment may not be sufficiently repeatable to ensure sufficiently consistent
quality of the image(s).
[0076] The present disclosure is based on the insight that identifying the radiation status
of the radiation assembly, and using the predetermined relationship to adjust the
image characteristic, such as exposure and/or white balance, according to the identified
radiation status can assist to provide faster and/or more repeatable image characteristic
adjustment relative to, for instance, adjustment based only on an optical condition
detected inside the food preparation chamber when obtaining the image(s).
[0077] The thus obtained image(s) can therefore be usefully employed in, for example, food
recognition, cooking process control, browning control, doneness status determination,
remote monitoring, and/or burning prevention based on analysis of the image(s).
[0078] In at least some embodiments, the present disclosure provides a method of compensating
for change(s) in illuminance and/or colour temperature caused by the radiation assembly,
e.g. heating elements and/or illumination system/lamps, shifting between statuses.
[0079] Such a method may, for example, adjust exposure and/or white balance during cooking
for a camera assembly-integrated kitchen appliance, e.g. cooking device.
[0080] FIG. 1 schematically depicts a food preparation assembly 100 according to an example.
The food preparation assembly 100 comprises a food preparation chamber 102 in which
food ingredients 103 are receivable.
[0081] The food preparation assembly 100 comprises a camera assembly 104 for obtaining one
or more images inside the food preparation chamber 102. Depending on how the image(s)
is or are being used, e.g. for food recognition, cooking process monitoring, and/or
cooking process control, the camera assembly 104 may be configured to obtain a sequence
of images, e.g. a sequence of images constituting a video. Alternatively or additionally,
the camera assembly 104 may be configured to obtain a still image of the inside of
the food preparation chamber 102.
[0082] The camera assembly 104 may be arranged such as to enable obtaining of the one or
more images of the food ingredients 103 received in the food preparation chamber 102.
[0083] In the non-limiting example shown in FIG. 1, the camera assembly 104, or at least
an optical sensing element of the camera assembly 104, is arranged at or proximal
to the ceiling of the food preparation chamber 102 in order to obtain image(s) of
the food ingredients from above. Alternatively or additionally, the camera assembly
104, or at least the optical sensing element thereof, is arranged, for instance, at
a side of the food preparation chamber 102 between the base and the ceiling of the
food preparation chamber 102.
[0084] The food preparation assembly 100 comprises a radiation assembly 106A, 106B, 108A,
108B configured to emit radiation comprising visible light inside the food preparation
chamber 102. In particular, the radiation assembly 106A, 106B, 108A, 108B is controllable,
e.g. by a user of the food preparation assembly 100, to adopt a radiation status in
which the radiation assembly 106A, 106B, 108A, 108B emits radiation comprising visible
light inside the food preparation chamber 102.
[0085] In some embodiments, such as that shown in FIG. 1, the radiation assembly 106A, 106B,
108A, 108B comprises at least one lamp 106A, 106B for illuminating the inside of the
food preparation chamber 102. In such embodiments, the radiation status can comprise
an illumination status of the, e.g. each of the, at least one lamp 106A, 106B.
[0086] The lamp(s) 106A, 106B can, for example, be controlled by the user, e.g. by operating
a suitable user interface included in the food preparation assembly 100 for enabling
the user to control the lamp(s) 106A, 106B. This may, for example, assist the user
to visually monitor the process taking place in the food preparation chamber 102.
The light from the lamp(s) 106A, 106B may thus alter an optical condition within the
food preparation chamber 102.
[0087] The lamp(s) 106A, 106B may, for example, be or comprise light emitting diode(s),
although any suitable lamp type, such as filament lamps, can be contemplated.
[0088] In the non-limiting example shown in FIG. 1, the lamps 106A, 106B are arranged at
or proximal to the ceiling of the food preparation chamber 102, although other position(s)
for such lamp(s) 106A, 106B can be contemplated provided the lamp(s) 106A, 106B can
illuminate at least part of the inside of the food preparation chamber 102.
[0089] The illumination status can, for instance, comprise an on/off status of the, e.g.
each of the, at least one lamp 106A, 106B.
[0090] Alternatively or additionally, the illumination status can comprise an intensity
of the illumination provided by the, e.g. each of the, at least one lamp 106A, 106B.
[0091] Alternatively or additionally, the illumination status can comprise a measure of
the spectral composition of the illumination provided by the, e.g. each of the, at
least one lamp 106A, 106B.
[0092] When the at least one lamp 106A, 106B comprises a plurality of lamps 106A, 106B,
such as two (as shown in FIG. 1), three, four, five, six or more lamps 106A, 106B,
one or more of these lamps 106A, 106B may be operable independently of the other(s).
In such an example, the illumination status may comprise a combination of the individual
illumination statuses of each of the independently operable lamp(s) 106A, 106B.
[0093] In some embodiments, such as that shown in FIG. 1, the radiation assembly 106A, 106B,
108A, 108B comprises at least one heating element 108A, 108B arranged to radiate inside
the food preparation chamber 102. In such embodiments, the radiation status can comprise
a heating status of the, e.g. each of the, at least one heating element 108A, 108B.
[0094] The at least one heating element 108A, 108B may be used to warm or cook the food
ingredients 103 received in the food preparation chamber 102. However, the radiation
emitted by the heating element(s) 108A, 108B may include visible light. Such visible
light may alter an optical condition within the food preparation chamber 102.
[0095] The heating element(s) 108A, 108B may, for example, be or comprise resistive heating
element(s), although any suitable heating element type can be contemplated.
[0096] In the non-limiting example shown in FIG. 1, the heating elements 108A, 108B are
arranged at or proximal to the ceiling of the food preparation chamber 102, although
other position(s) for such heating element(s) 108A, 108B can be contemplated provided
the heating element(s) 108A, 108B can emit heat radiation into at least part of the
inside of the food preparation chamber 102.
[0097] The heating status can, for instance, be an on/off status of the, e.g. each of the,
at least one heating element 108A, 108B.
[0098] Alternatively or additionally, the heating status can be a degree of heating provided
by the, e.g. each of the, at least one heating element 108A, 108B and/or the heating
status can be a measure of the spectral composition of the radiation provided by the,
e.g. each of the, at least one heating element 108A, 108B.
[0099] When the at least one heating element 108A, 108B comprises a plurality of heating
elements 108A, 108B, such as two (as shown in FIG. 1), three, four, five, six or more
heating elements 108A, 108B, one or more of these heating elements 108A, 108B may
be operable independently of the other(s). In such an example, the heating status
may comprise a combination of the individual heating statuses of each of the independently
operable heating element(s) 108A, 108B.
[0100] In embodiments in which the radiation status comprises the illumination status of
the at least one lamp 106A, 106B and the heating status of the at least one heating
element 108A, 108B, the illumination status can, for example, be an on/off status
of the at least one lamp 106A, 106B, with the heating status being an on/off status
of the at least one heating element 108A, 108B.
[0101] In some embodiments, such as that shown in FIG. 1, the food preparation assembly
100 comprises a control unit 110 having a control configuration arranged to control
the radiation assembly 106A, 106B, 108A, 108B to adopt the radiation status.
[0102] Thus, the radiation status can be obtained, in other words directly obtained, from
the control unit 110 controlling the radiation assembly. This may advantageously obviate
sensing of an optical condition inside the food preparation chamber 102 for the purpose
of adjusting the image characteristic.
[0103] In some embodiments, the food preparation assembly 100 comprises a user interface
(not visible in FIG. 1) configured to permit user selection of the radiation status.
In such embodiments, the control configuration can, for example, correspond to or
be generated from the user selection.
[0104] In a non-limiting example, the control unit 110 is configured to receive a recipe
input, e.g. a recipe instruction input, and generate the control configuration based
on the received recipe input. In other words, the control configuration can be automatically
generated by the control unit 110 from the recipe input.
[0105] In some embodiments, the control configuration comprises a control sequence which
controls the radiation status of the radiation assembly 106A, 106B, 108A, 108B overtime.
[0106] The recipe input can be inputted in any suitable manner. For example, the recipe
input can be received by the control unit 110 from a recipe database in response to
a user selection from the recipe database.
[0107] More generally, due to the above-mentioned influence of the radiation assembly 106A,
106B, 108A, 108B, e.g. comprising the lamp(s) 106A, 106B and/or the heating element(s)
108A, 108B, an optical condition, e.g. an intensity and/or spectral composition of
visible light, inside the food preparation chamber 102 may change according to the
radiation status of the radiation assembly 106A, 106B, 108A, 108B.
[0108] For this reason, the food preparation assembly 100 comprises one or more processors
112 configured to identify the radiation status, and control the camera assembly 104
to adjust the image characteristic using a predetermined relationship for adjusting
the image characteristic according to the identified radiation status.
[0109] Using such a predetermined relationship to adjust the image characteristic, such
as exposure and/or white balance, according to the identified radiation status of
the radiation assembly 106A, 106B, 108A, 108B can assist to provide faster and/or
more repeatable image characteristic adjustment relative to, for instance, adjustment
based only on an optical condition detected inside the food preparation chamber 102
when obtaining the image(s).
[0110] In some embodiments, the radiation assembly 106A, 106B, 108A, 108B is controllable
to select the radiation status from a plurality of radiation statuses, and the one
or more processors 112 is or are configured to identify the selected radiation status,
with the predetermined relationship permitting adjusting of the image characteristic
according to the identified selected radiation status.
[0111] The image characteristic can accordingly be adjusted according to which of the plurality
of radiation statuses is selected.
[0112] In embodiments in which the radiation assembly is controllable to select the radiation
status from the plurality of radiation statuses, the plurality of radiation statuses
can comprise: a first radiation status; and a second radiation status, with the first
radiation status causing a first optical condition in the food preparation chamber
102, and the second radiation status causing a second optical condition in the food
preparation chamber 102 which is different from the first optical condition.
[0113] In such embodiments, the predetermined relationship may be determined such that the
image characteristic is adjusted to compensate for the difference between the first
optical condition and the second optical condition.
[0114] By compensating for the difference between the first and second optical conditions,
the image(s) obtained when the first radiation status is adopted can be comparable
to the image(s) obtained when the second radiation status is adopted.
[0115] For example, should the optical condition include an intensity of visible light inside
the food preparation chamber, adjustment of the image characteristic includes adjustment
of the exposure of the image(s) to compensate for the difference in intensity. Alternatively
or additionally, should the optical condition include a spectral composition of the
light inside the food preparation chamber, adjustment of the image characteristic
includes adjustment of the colour, e.g. white, balance of the image(s) to compensate
for the difference in spectral composition.
[0116] In a non-limiting example, the food preparation assembly 100, for example the above-described
control unit 110 included in the food preparation assembly 100, is configured to control
the heating element(s) 108A, 108B according to a cooking program in which the heating
element(s) 108A, 108B is or are controlled such that the food preparation chamber
102 is at a first temperature for a first predetermined duration, and then, e.g. when
a doneness level of the food ingredients 103 is approaching, is at as second temperature
lower than the first temperature.
[0117] In this example, the radiation status of the radiation assembly 106A, 106B, 108A,
108B, and in particular the heating status of the heating element(s) 108A, 108B, changes
from a first radiation status to a second radiation status in order to implement the
temperature reduction from the first temperature to the second temperature.
[0118] The first radiation status may, for instance, correspond to both of the heating elements
108A, 108B being on, with the second radiation status corresponding to one of the
heating elements 108A, 108B being on and the other of the heating elements 108B, 108A
being off.
[0119] The alteration in the optical condition, e.g. intensity and/or spectral composition
of visible light, inside the food preparation chamber 102 resulting from the radiation
status changing from the first radiation status to the second radiation status may
nonetheless be compensated for by the one or more processors 112 controlling the camera
assembly 104 to adjust the image characteristic. In particular, the predetermined
relationship used by the processor(s) 112 may be determined such that the image characteristic
is adjusted to compensate for the difference between the first optical condition and
the second optical condition.
[0120] This compensating may, in turn, improve the image-based control which can be exerted
over the cooking process, e.g. selection/estimation of the time required at the second
predetermined temperature. This is because the image(s) obtained when the radiation
assembly 106A, 106B, 108A, 108B adopts the first radiation status may be more reliably
compared with the image(s) obtained when the radiation assembly 106A, 106B, 108A,
108B adopts the second radiation status.
[0121] In embodiments in which the control configuration comprises the above-mentioned control
sequence which controls the radiation status of the radiation assembly 106A, 106B,
108A, 108B to change over time, the one or more processors 112 may be configured to
control the camera assembly 104 to adjust the image characteristic to coincide with
the change of the radiation status. This may enable particularly responsive adjustment
of the image characteristic which may be especially useful, for example, when the
camera assembly 104 is configured to obtain a sequence of images, e.g. constituting
a video, during the change of the radiation status.
[0122] Whilst the processor(s) 112 is or are shown in FIG. 1 receiving data, e.g. the control
configuration, from the control unit 110, this is not intended to be limiting. In
other embodiments, which may be an alternative or in addition to the control configuration
being used by the processor(s) 112 to identify the radiation status, the food preparation
assembly 100 may include a sensor (not visible) for sensing radiation of the radiation
assembly 106A, 106B, 108A, 108B. In such embodiments, the one or more processors 112
is or are configured to identify the radiation status based on the radiation sensed
by the sensor.
[0123] Alternatively or additionally, the one or more processors 112 may be configured to
identify the radiation status based on a power consumption of the radiation assembly
106A, 106B, 108A, 108B.
[0124] Referring to FIG. 2, in some embodiments, the camera assembly 104 comprises an image
capture module 114, and the one or more processors 112 is or are configured to control
at least one image capture setting of the image capture module 114 according to the
identified radiation status to adjust the image characteristic. The identified radiation
status may thus be used to adjust the acquisition of the image(s).
[0125] The at least one image capture setting can, for example, comprise or be an aperture
size, a shutter speed and/or a sensitivity setting.
[0126] The camera assembly 104 may, in at least some embodiments, further comprise an image
processing module 116. In such embodiments, the image processing module 116 may process
the image(s) acquired by the image capture module 114, thereby to obtain the image(s).
The image processing module 116 may alternatively or additionally be included in a
user device, such as a smart phone or tablet computer, separate from a kitchen appliance
in which the food preparation chamber 102 is included, and/or in a cloud-based server.
[0127] As an alternative or in addition to the above-described control over the at least
one capture setting of the image capture module 114 included in the camera assembly
104, the one or more processors 112 may be configured to control at least one image
processing setting of the image processing module 116 according to the identified
radiation status to adjust the image characteristic.
[0128] Thus, the identified radiation status can be used to adjust the processing of the
image(s) by the image processing module 116 included in the camera assembly 104.
[0129] The at least one processing setting may, for instance, comprise a colour balance
setting. Such a colour balance setting can, for example, be a white balance setting.
[0130] For example, in order to implement the above-described compensation in the case of
adoption of the second radiation status resulting in a higher visible light intensity
in the food preparation chamber 102 compared to that resulting from adoption of the
first radiation status, the aperture size may be reduced and/or the shutter speed
increased and/or the sensitivity decreased to compensate for this higher intensity
of visible light.
[0131] Alternatively or additionally, the colour balance, e.g. white balance, may be adjusted
to compensate for a different spectral composition of visible light resulting from
adoption of the second radiation status after adoption of the first radiation status.
[0132] More generally, the one or more processors 112, the image processing module 116 and/or
the control unit 110 can be implemented in numerous ways, with software and/or hardware,
to perform the various functions required. The processor(s) 112, the image processing
module 116 and/or the control unit 110 may, for example, employ one or more microprocessors
programmed using software (e.g., microcode) to perform the required functions. Examples
of processor components that may be employed in various embodiments of the present
disclosure include, but are not limited to, conventional microprocessors, application
specific integrated circuits (ASICs), and field-programmable gate arrays (FPGAs).
[0133] In various implementations, the one or more processors 112, the image processing
module 116 and/or the control unit 110 may be associated with one or more storage
media such as volatile and non-volatile computer memory such as RAM, PROM, EPROM,
and EEPROM. The storage media may be encoded with one or more programs that, when
executed on one or more processors and/or controllers, perform the required functions.
Various storage media may be fixed within a processor or controller or may be transportable,
such that the one or more programs stored thereon can be loaded into the one or more
processors 112, the image processing module 116 and/or the control unit 110.
[0134] Whilst the image processing can be implemented on the image processing module 116
included in the camera module 104, this is not intended to be limiting, and in other
examples the image processing module 116 may be alternatively or additionally included
in a cloud-based server and/or in a separate user device, such as a smart phone or
tablet computer.
[0135] In at least some embodiments, the food preparation assembly 100 is a domestic food
preparation assembly 100, e.g. a domestic food preparation assembly comprising a domestic
kitchen appliance.
[0136] Examples of such a domestic kitchen appliance include an air fryer, oven, steamer,
pressure cooker or rice cooker.
[0137] In some embodiments, the camera assembly 104 is integral to such a domestic kitchen
appliance.
[0138] In alternative embodiments, the camera assembly 104 is a removable or detachable
component of the food preparation assembly 100, e.g. domestic food preparation assembly.
A window (not visible) into the food preparation chamber 102 may be provided, and
the camera assembly 104 may be arranged such as to obtain the image(s) inside the
food preparation chamber 102 via the window in order to assemble the food preparation
assembly 100, e.g. the domestic food preparation assembly 100.
[0139] For instance, the camera assembly 104, and optionally at least one of the one or
more processors 112, may be included in a separate user device, such as a smart phone
or tablet computer. The processor(s) 112 may alternatively or additionally be included
in a cloud-based server. The food preparation assembly 100, e.g. domestic food preparation
assembly 100, may be assembled by arranging the camera assembly 104 such as to obtain
the image(s) inside the food preparation chamber 102 via the window.
[0140] In some embodiments, the food preparation chamber 102 and the radiation assembly
106A, 106B, 108A, 108B are included in a kitchen appliance, e.g. domestic kitchen
appliance, and the processor(s) 112 included in the user device and/or the cloud-based
server is or are in communication, e.g. wireless communication, with the kitchen appliance,
e.g. with the above-described control unit 110, in order to enable the processor(s)
112 to identify the radiation status and control the camera assembly 104 to adjust
the image characteristic of the one or more images.
[0141] In a non-limiting example, the food preparation assembly 100 is a kitchen appliance,
e.g. cooking appliance, in which the camera assembly 104, the radiation assembly 106A,
106B, 108A, 108B, e.g. comprising heating element(s) 108A, 108B and/or illumination
system/lamps 106A, 106B, processor(s) 112 and control unit 110 are integral components,
as shown in FIG. 1. In such an example the kitchen appliance can also include a data
processing unit (e.g. comprising the above-described image processing module 116,
and a data analysis module), and a storage unit for storing data.
[0142] FIG. 3 provides a flowchart of a method 200 of operating a camera assembly according
to an example. The camera assembly may be suitable for obtaining one or more images
inside a food preparation chamber of a food preparation assembly further comprising
a radiation assembly controllable to adopt a radiation status in which the radiation
assembly emits radiation comprising visible light inside the food preparation chamber.
The food preparation assembly may, for example, be the food preparation assembly 100
according to any of the embodiments described above.
[0143] The method 200 comprises identifying 202 the radiation status, and controlling 204
the camera assembly to adjust an image characteristic of the one or more images using
a predetermined relationship for adjusting the image characteristic according to the
identified radiation status.
[0144] The method 200 may further comprise operating 206 the camera assembly to obtain one
or more images inside the food preparation chamber.
[0145] The image characteristic may be adjusted according to the identified radiation status
in the obtained one or more images, as previously described.
[0146] Whilst the operating 206 is shown in FIG. 3 taking place after the controlling 204
the camera assembly to adjust an image characteristic of the one or more images using
the predetermined relationship, these operations can be implemented in any suitable
order.
[0147] For example, the camera assembly may be operated to acquire the image(s) and the
image processing setting(s) may be subsequently adjusted such that at least part of
204 can take place after 206.
[0148] In some embodiments, the radiation assembly is controllable to select the radiation
status from a plurality of radiation statuses, and the identifying 202 comprises identifying
the selected radiation status, with the predetermined relationship permitting adjusting
of the image characteristic according to the identified selected radiation status.
[0149] The image characteristic can accordingly be adjusted according to which of the plurality
of radiation statuses is selected.
[0150] In embodiments in which the radiation assembly is controllable to select the radiation
status from the plurality of radiation statuses, the plurality of radiation statuses
can comprise: a first radiation status; and a second radiation status, with the first
radiation status causing a first optical condition in the food preparation chamber,
and the second radiation status causing a second optical condition in the food preparation
chamber which is different from the first optical condition. In such embodiments,
the predetermined relationship may be determined such that the image characteristic
is adjusted to compensate for the difference between the first optical condition and
the second optical condition, as previously described.
[0151] FIG. 4 graphically represents adjustment of shutter speed and white balance according
to statuses of a radiation assembly as cooking progresses. In this non-limiting example,
the two lamps 106A, 106B are continuously illuminating the inside of the food preparation
chamber 102 during the entire cooking process, but the heating element 108A and the
heating element 108B simultaneously shift between on and off radiation statuses in
order to control the temperature within the food preparation chamber 102 during the
cooking process.
[0152] When the first radiation status ("Status 1") is adopted by both of the heating elements
108A, 108B being on, the illumination intensity in the food preparation chamber 102
may be higher and the colour temperature may be lower than when the second radiation
status ("Status 2") is adopted by both of the heating elements 108A, 108B being off.
Accordingly, the shutter speed may be increased, in this case such that the exposure
time decreases from 1/100 s to 1/150 s, and the white balance may be adjusted from
5500 K to 7500 K in response to the radiation status changing from the second radiation
status ("Status 2") to the first radiation status ("Status 1"). The adjustment in
these settings may assist to compensate for the change in the illumination intensity
and colour temperature caused by the change in radiation status.
[0153] These first and second radiation statuses are shown in Table 1. The optical condition,
in this case illumination intensity and color temperature, in the food preparation
chamber 102 resulting from each of the radiation statuses is measured for each of
the first and second radiation statuses before the cooking process. Suitable image
capture setting(s) and/or image processing setting(s) are then determined, as also
shown in the Table 1.
Table 1
|
First radiation status |
Second radiation status |
Radiation assembly 106A, 106B, 108A, 108B |
Lamp 106A |
On |
On |
Lamp 106B |
On |
On |
Heating element 108A |
On |
Off |
Heating element 108B |
On |
Off |
Optical condition inside the food preparation chamber 102 |
Illumination intensity |
600 Lux |
400 Lux |
Colour temperature |
3500 K |
5500 K |
Image capture settings and image processing settings of the camera assembly 104 |
Aperture |
F 2.8 |
F 2.8 |
Exposure time (s) |
1/150 |
1/100 |
ISO |
100 |
100 |
White balance |
7500 K |
5500 K |
[0154] During the cooking process, as also illustrated in FIG. 4, the radiation status can
be identified from the power consumption of the radiation assembly 106A, 106B, 108A,
108B and/or obtained from the control unit 110, then the image capture setting(s)
and/or the image processing setting(s) is or are selected according to Table 1.
[0155] A computer program comprising computer program code may be configured, when said
computer program is run on one or more processors 112 included in a food preparation
assembly 100 further comprising a food preparation chamber 102, a camera assembly
104 for obtaining one or more images inside the food preparation chamber, and a radiation
assembly 106A, 106B, 108A, 108B controllable to adopt a radiation status in which
the radiation assembly emits radiation comprising visible light inside the food preparation
chamber 102, to cause said one or more processors 112 to implement the method 200
according to any of the embodiments described herein.
[0156] The one or more processors 112 can be included in a kitchen appliance, e.g. a kitchen
appliance comprising the food preparation chamber 102 and the radiation assembly 106A,
106B, 108A, 108B, which kitchen appliance is included in the food preparation assembly.
Alternatively or additionally, the one or more processors 112 can be included in a
user device, for example a smart phone or tablet computer, separate from such a kitchen
appliance, and/or in a cloud-based server.
[0157] FIG. 5 provides a flowchart of a method 300 of determining a relationship for adjusting
an image characteristic of one or more images inside a food preparation chamber of
a food preparation assembly comprising a radiation assembly; the one or more images
being obtained using a camera assembly.
[0158] The food preparation assembly may, for example, be the food preparation assembly
100 according to any of the embodiments described above.
[0159] The method 300 comprises identifying 302 at least one radiation status of the radiation
assembly in which radiation comprising visible light is emitted inside the food preparation
chamber, and measuring 304, for each of the at least one radiation status, an optical
condition. The optical condition can, for instance, be measured inside the food preparation
chamber 102, e.g. using one or more suitable optical sensors.
[0160] The method 300 further comprises determining 306, using the optical condition measured
for each of the identified at least one radiation status, said relationship for adjusting
the image characteristic according to the identified at least one radiation status.
[0161] This method 300 may be used to determine the above-described predetermined relationship
for adjusting the image characteristic according to the identified radiation status.
[0162] FIG. 6 provides a flowchart for illustrating how the method 300 shown in FIG. 5 can
be applied to an exemplary method 200 of operating a camera assembly. FIG. 6 may,
for example, be regarded as illustrating a workflow of the development of embodiments
of the present disclosure.
[0163] The method 300 may be regarded as being included in a product development phase,
and the method 200 can be regarded as being included in a cooking phase, e.g. during
use, of the food preparation assembly 100.
[0164] At least one radiation status of the radiation assembly in which radiation comprising
visible light is emitted inside the food preparation chamber is identified in 302.
This identifying 302 may comprise identifying all potential combinations of components
capable of forming part of the radiation assembly, in other words the heating element(s)
and/or the illumination system/lamp(s), and the possible radiation statuses of the
radiation assembly which can be selected by controlling these components.
[0165] In 304, the optical condition is measured for each of the possible radiation statuses
identified in step 302. The optical condition can, for instance, be measured inside
the food preparation chamber 102, e.g. using one or more suitable optical sensors,
as previously described.
[0166] The determining 306, using the optical condition measured for each of the identified
at least one radiation status, the relationship for adjusting the image characteristic
according to the identified at least one radiation status is then implemented.
[0167] Turning to the method 200 included in the cooking phase in the example shown in FIG.
6, the identifying 202 the radiation status comprises obtaining 202A the individual
radiation statuses of all components capable of forming part of the radiation assembly,
in other words the heating element(s) and/or the illumination system/lamp(s). This
can be achieved by obtaining the individual statuses from the above-described control
unit and/or via sensing the radiation provided by the component(s). The radiation
status, in other words the overall or "combination" status, of the radiation assembly
can then be identified in 202B.
[0168] The camera assembly is then controlled in 204 to adjust an image characteristic of
the one or more images using the above-described predetermined relationship determined
in 306.
[0169] In summary, the present disclosure proposes to control a camera assembly to adjust
an image characteristic, e.g. exposure and/or white balance, for image acquisition
and processing to obtain image(s)/video that can reliably reflect food appearance.
[0170] Variations to the disclosed embodiments can be understood and effected by those skilled
in the art in practicing the claimed invention, from a study of the drawings, the
disclosure and the appended claims. In the claims, the word "comprising" does not
exclude other elements or steps, and the indefinite article "a" or "an" does not exclude
a plurality.
[0171] The mere fact that certain measures are recited in mutually different dependent claims
does not indicate that a combination of these measures cannot be used to advantage.
[0172] If the term "adapted to" is used in the claims or description, it is noted the term
"adapted to" is intended to be equivalent to the term "configured to".
[0173] Any reference signs in the claims should not be construed as limiting the scope.