Field of the Invention
[0001] The present invention relates to image processing apparatus and methods for enhancing
colour images.
Background of the Invention
[0002] For most applications, an amount of data required to represent colour images demands
that some form of compression or bandwidth reduction is performed in order to reduce
the amount of data, which must be communicated or stored. This is because colour images
require at least a representation of the red, green and blue (RGB) components of the
image to be communicated or stored in order to provide a faithful representation of
the original colour of the image. Next generation mobile radiotelephones may require
a facility to receive and display colour images over a wireless link for display on
a screen provided as part of the telephone. Similarly, for applications such as Personal
Digital Assistants (PDA) there is a requirement to communicate and store colour images
for display. For such applications, the colour images may be either received in compressed
form or compressed when received in order to reduce an amount of data, which is required
in order to communicate or store the image.
[0003] One technique for reducing an amount of information, which is required in order to
store a representation of a colour image, is to reduce a dynamic range of the RGB
signal samples by reducing a number of bits that are used to represent the RGB components.
For example, if each pixel is represented as 18-bits, each RGB component will be allocated
a 6-bit value providing a range of 64 possible colour values for each component, producing
262144 different colours. If the number of bits used to represent each component is
reduced to 4-bits, each pixel can be represented as 12-bits providing a 33% reduction
in the size of a memory required to store the colour image. However, a result of this
reduction is to reduce the number of possible colours to 4096, which can be represented
by each pixel, thereby reducing the so-called "colour palette" available to represent
the image. As a result, the quality of the stored image when reproduced may be affected.
For example, certain images such as faces or sun sets may suffer from contour effects,
which appear on certain areas of the image due to a limitation on the number of available
colours, resulting from the reduced colour palette. Preferably, therefore a reduction
in the number of bits used to represent the signal samples should be provided without
reducing the quality of the reproduced image.
Summary of Invention
[0004] According to the present invention there is provided an image processing apparatus
operable to process input colour component signal samples representative of an input
colour image to produce output colour component signal samples representing an enhanced
version of the input colour image. The apparatus comprises an enhancing processor
operable to increase the dynamic range of the input colour component samples, a low
pass filter operable to filter the increased dynamic range colour samples to form
the output colour component samples, and a detection processor. The detection processor
is operable to detect input colour component samples associated with a representation
of detail appearing in the input colour image. Consequent upon detection of the detail
in the image, the detection processor is operable to adjust the bandwidth of the low
pass filter in correspondence with the detected detail.
[0005] Embodiments of the present invention are arranged to provide a facility for reducing
an amount of data which is required to represent a colour image, whilst improving
or at least maintaining the quality of the image when reproduced. To this end, an
image processor embodying the present invention is provided with a colour enhancing
processor which is arranged to improve a dynamic range with which the colour components
of the image are represented by increasing a number of bits used to represent the
RGB colour component samples. The increased dynamic range samples are then filtered
with a low pass filter in order to provide a smooth transition between the colours
of the reproduced image. However, in order to reduce a likelihood of losing detail
within the reproduced image a detection processor is arranged to detect signal samples
representative of the detail within the image and to adjust the band width of the
low pass filter in correspondence with the detected detail. In one embodiment the
samples representing the detail are arranged to bypass the low pass filter. As a result,
a likelihood of losing detail in the image is reduced, whilst reducing a likelihood
of contour effects which may appear in the image as a result of a reduced colour palette
caused by a low dynamic range of the colour component signal samples.
[0006] Further advantageous features of the present invention will become apparent from
a description of embodiments. In particular, the image processor providing the enhanced
dynamic range colour samples may be implemented with a small number of components,
providing a cost effective implementation.
[0007] Various further aspects and features of the present invention are defined in the
appended claims.
Brief Description of Drawings
[0008] Embodiments of the present invention will now be described by way of example only
with reference to the accompanying drawings, where like parts are provided with corresponding
reference numerals, and in which:
Figure 1 is a schematic block diagram of an LCD display module;
Figure 2 is a schematic block diagram of an LCD display module including an image
processor according to an embodiment of the invention;
Figure 3 is a representation of an example colour image which suffers from contour
effects resulting from a limited colour palette;
Figure 4 is a schematic block diagram of the image processor appearing in Figure 2;
Figure 5 is a schematic block diagram of a dynamic range converter appearing in Figure
4;
Figure 6 is a table providing an illustration of values required to increase signal
samples in accordance with an increased dynamic range with respect to a corresponding
increase produced by copying most significant bits into least significant bits;
Figure 7 is a schematic illustration of an impulse response of a moving averaging
filter of eight taps duration;
Figure 8 is a schematic block diagram of a low pass filter appearing in Figure 4;
Figure 9 is a schematic block diagram of a detection processor appearing in Figure
4;
Figure 10 is a flow diagram representing the operation of the image processor shown
in Figure 4; and
Figure 11 is a representation of the example colour image after being processed by
the image processor shown in Figure 3.
Description of Preferred Embodiments
[0009] An embodiment of the present invention will now be described with reference to an
LCD display module. Figure 1 provides an illustration of a typical LCD display module
such as that which would be used within a mobile radiotelephone or a personal digital
assistant. As mobile radiotelephones move towards the next generation, the so-called
third generation (3G), there will be an increasing demand to display digital images
on a display module of mobile radiotelephones. A typical display module for this application
is an LCD display panel and as shown in Figure 1, such a module comprises a display
memory 1 which is arranged to store signal samples representative of a digital image
which is to be displayed on an LCD display panel 2. Pixels which are representative
of the colour image to be displayed are received on a connecting channel 4 and written
into the display memory 1 using write address logic 8. When the image is to be displayed,
the signal samples, stored in the memory 1, are read by read address logic 10 and
converted into the analogue domain by a digital to analogue converter 12 before being
displayed on the LCD panel 2.
[0010] As already explained, the present invention provides a facility for reducing the
amount of data storage, that is the size of the display memory 1 for the present example,
whilst maintaining the quality of the image to be displayed on, for example the LCD
panel 2. The reduction in the amount of data storage is achieved by reducing the dynamic
range of the signal samples representing the colour components. Alternatively, the
colour image may be received with component signal samples having a reduced dynamic
range as a result of being truncated, which may be required to communicate the image.
As will be appreciated the RGB components will be truncated by the same amount. For
both these examples, an image processor embodying the present invention is arranged
to improve a representation of the received or stored colour image. As will be explained,
the improvement can facilitate a faithful or at least improved reproduction of the
original image with a reduced likelihood of contour effects, which may appear in the
image.
[0011] An embodiment of the present invention is illustrated in Figure 2. Figure 2 shows
the LCD display module of Figure 1 in combination with an image processing circuit
20. As shown in Figure 2, the pixels of the colour image as received on channel 4
are represented as only 12 bits rather than 18 bits as shown in Figure 1. The reduction
in the resolution of the signal samples may be provided by a colour truncating processor
15. The colour truncating processor 15 may reduce the dynamic range of the signal
samples by truncating the signal samples from 18-bits to 12-bits by discarding the
two least significant bits of each RGB component. As such, the RGB colour components
are represented as 4 bits rather than 6 bits, resulting in a 33% reduction in a storage
requirement of the display memory 1.
[0012] A result of reducing the dynamic range of the colour signal samples by truncating
from 6 bits to 4 bits is to reduce the colour palette available for displaying the
reproduced image. As a result a colour image may suffer a reduction in quality which
can cause contours or apparent flat sections to appear in the reproduced image. An
example of such a colour image suffering from contour effects as a result of a reduced
colour palette is shown in Figure 3.
[0013] In order to reduce the likelihood of contour effects appearing in the reproduced
image, the display module according to the present invention is provided with an image
processor 20. The image processor 20 is arranged to increase the dynamic range of
the colour component signal samples, which are read out of the display memory 1. As
shown in Figure 2, for the example embodiment illustrated, the number of bits, which
are used to represent each pixel, is increased from 4-bits to 6-bits. The image processor
20 is shown in more detail in Figure 4.
[0014] In Figure 4 the image processor 20 receives the 4 bit RGB colour component signal
samples on a connecting channel 22 from the read address logic 10. The colour component
signal samples are fed to a colour dynamic range converter 30 and a detection processor
34 in parallel. The dynamic range converter 30 is arranged to increase the number
of bits, which are used to represent the colour component signal samples from 4 bits
to 6 bits. The enhanced colour component signal samples are then fed to a low pass
filter 36 after which the enhanced signal samples are fed to an output channel 24.
As shown in Figure 2, the enhanced signal samples are fed to a digital to analogue
converter 12 as shown in Figure 4.
[0015] The detection processor 34 is arranged to generate a filter selection control signal,
which is fed on a channel 38 to the low pass filter 36.
[0016] In accordance with the increased dynamic range provided by the dynamic range converter
30, the values of the 4-bit signal samples can be scaled to reflect the enhanced dynamic
range provided by increasing the number of bits to six. In order to adjust the colour
component sample values in accordance with the increased dynamic range, the values
of the input signal samples are multiplied by again factor G. The gain factor G is
determined in accordance with the original resolution of
n-bits and the increase in the resolution to q-bits as represented by the following
equation:
![](https://data.epo.org/publication-server/image?imagePath=2004/06/DOC/EPNWA2/EP03250945NWA2/imgb0001)
[0017] To reduce a number of components required to implement the gain factor
G, the gain factor
G is applied by copying the two most significant bits from the 4 bit input sample into
the two least significant bits appended to the 6-bit output sample. Such an arrangement
for applying the gain to the input signal samples is illustrated in Figure 5. In Figure
5, the 4 bit input signal sample is represented as the letters A B C D which are stored
in a four stage register 40. A six stage register 42 is provided for forming the output
signal sample. Connecting the four stage register to the six stage register are connecting
channels 44. As shown in Figure 5 the connecting channels arrange for the 2 bit values
A, B in the first two stages of the shift register 40 to be copied into the last two
stages of the register 42. Thus, the values of the two least significant bits of the
output signal sample are formed from the most significant bits of the input signal
samples.
[0018] Figure 6 provides a table illustrating all possible values of the 4-bit input signal
samples with respect to the gain required to adjust the signal sample values from
4-bits to 6-bits, to take advantage of the increased dynamic range. The gain, as calculated
by the above formula, is shown with respect to the gain, which is provided by bit
replication in accordance with the arrangement of registers shown in Figure 5. As
can be seen in Figure 6 the bit replication arrangement provides a good approximation
to the calculated gain factor
G but has an advantage of not requiring any multiplier components, thus reducing implementation
costs.
[0019] The enhanced colour component signal samples are filtered by a low pass filter 36
in order to provide a smooth transition between colour values. Filtering the signal
sample values has the effect of reducing the likelihood of contour effects, which
may appear in the reproduced image. In preferred embodiments, the low pass filter
36 is implemented as a moving averaging filter having an impulse response as shown
in Figure 7. Figure 7 illustrates an impulse response of an eight tap moving averaging
filter providing an effective width of T
W.
[0020] A moving averaging filter effectively forms an output signal sample from the average
values of the input signal samples within a window formed by the impulse response
of the moving averaging filter. The filter is arranged to form a smoothing effect
on the contours of the image caused by the limited dynamic range of the resolution
of each signal sample.
[0021] For the present example shown in Figure 7 an eight tap moving averaging filter is
used so that the output sample is formed from an average of the values within the
eight tap window. As will be explained shortly, in other embodiments, as well as the
eight tap moving averaging filter, the low pass filter 36 includes a four tap moving
averaging filter and a two tap moving averaging filter. These are used to provide
an increase in the pass bandwidth of the low pass filter as the samples representing
detail move towards a centre tap of the eight-tap filter. This provides a more gradual
amount of filtering to the enhanced signal samples, where the detection processor
determines that these signal samples represent detail.
[0022] Figure 8 provides an illustration of the low pass filter 36 according to one example
embodiment. In Figure 8 the low pass filter 36 has three moving averaging filters,
which include the eight-tap filter having an impulse response shown in Figure 7. As
shown in Figure 8 an eight-tap delay 50 is used to delay the input signal samples
by eight samples. At the output of the eighth tap, adders 52, 54 are used to form
the output sample in combination with a one-stage register 56. Effectively, the filter
36 shown in Figure 8 implements a moving averaging filter by accumulating the last
eight signal sample values as performed by the adder 54 whilst subtracting the last
value to pass thorough and fall outside the eight tap window as implemented by the
adder 52. The adders 52, 54 and the register 56 therefore form an accumulation unit
58 to provide the calculation of the moving average output.
[0023] Effectively therefore, the accumulation unit 58 shown in Figure 8 implements a moving
averaging filter without a requirement for summing and dividing all the signal sample
components within an eight tap window to calculate every output signal sample. The
circuit shown in Figure 8 therefore provides an advantage in reducing the number of
components required to implement the moving averaging filter and for which no multipliers
are necessary. Correspondingly, second and third accumulation units 58', 58" form
output signal samples representing the four-tap and two-tap moving averaging filters.
To this end, the second accumulator 58' is arranged to receive samples from the second
stage and the sixth stage to form the four-tap moving averaging filter, whereas the
third accumulator 58" is arranged to receive samples from the third stage and fifth
stage. Thus the eight, four and the two tap filters are centred with respect to a
common temporal reference.
[0024] As shown in Figure 8, the control signal is received from the channel 38 by a selection
processor 59. The selection processor 59 has four input channels 51, 53, 55, 57. The
four input channels 51, 53, 55, 57 are arranged to receive the output sample from
the eight tap moving averaging filter 51, the output sample from the four tap moving
averaging filter 53, the output sample from the two tap moving averaging filter 55,
and the signal sample from the fourth stage of the shift register 50, respectively.
The selection of the output from either the two tap, the four tap, or the eight tap
output, or the bypass sample from the fourth stage of the shift register is determined
in accordance with the control signal received from the detection processor 34.
[0025] In order to avoid a likelihood of reducing the clarity of detail such as edges or
single pixel objects in the reproduced image, the detection processor is used to detect
such detail. The detection processor then generates a control signal in order to select
output samples from either the eight-tap, four-tap or two-tap moving averaging filter,
or the bypass sample. Effectively, the selection signal adjusts the bandwidth of the
low pass filter in accordance with the relative position of pixels representing detail
with respect to the centre tap. Figure 9 provides an example implementation of the
detection processor 34. As shown in Figure 9 the detection processor 34 comprises
a shift register 60 having eight stages which correspond to the stages of the shift
register 50 used in the low pass filter 36. As shown in Figure 9, an adder 62 is arranged
to receive an input signal sample and a signal sample of the output of the second
stage of the shift register 60. The adder 62 is arranged to subtract the newly received
signal sample from the signal sample present in the second stage to form a difference
at an output 64. A threshold processor 66 then compares the difference between the
signal samples with a predetermined threshold and if the difference is greater than
the threshold, the processor 66 produces a control signal on the control channel 38.
The threshold is determined so that detail is detected if all of the three RGB signal
samples are greater than the corresponding values of the RGB samples of a pixel from
two sample periods earlier. However, in addition to the adder 62 a further adder 68
is arranged to calculate a difference between the output of the fourth stage and the
second stage, which is also fed to the threshold processor 66.
[0026] The operation of the detection processor is represented by the flow chart in Figure
10. As shown in Figure 9 the latest signal sample is compared with the contents of
the second previous signal sample as performed by the adder 62. The threshold processor
66 determines whether detail has been detected between these two compared signal samples,
which is represented in Figure 10 as step S2.
[0027] As mentioned above, the threshold processor 66 also receives a comparison between
the output of the second stage of the shift register and an output of the fourth stage
of the shift register as performed by an adder 68. Therefore, at step S4 in the flow
diagram the threshold processor determines whether the detail has been detected between
effectively a latest signal sample and the second previous signal sample within a
four tap window provided by sample values at the outputs of the third, fourth, fifth
and sixth register stages.
[0028] If the threshold processor determines that detail has not been detected within the
four-tap filter, but is present in the eight-tap filter, from a comparison of the
contents of the second and fourth stage of the eight tap delay, then at step S4 the
threshold processor determines that the four tap filter should be selected to produce
the output pixel samples, at step S5. If the threshold detection processor determines
that the detail samples is within the four tap filter but just before the two centre
taps forming the two-tap filter the processing proceeds to step S8.
[0029] At step S8 the value of the signal sample in the fourth stage is also compared with
a second threshold formed between the threshold value for detecting detail and a predetermined
offset value, as expressed below:
![](https://data.epo.org/publication-server/image?imagePath=2004/06/DOC/EPNWA2/EP03250945NWA2/imgb0002)
[0030] If the signal sample value is greater than the threshold, but less than the threshold
plus the offset value, then the control signal is generated on the channel 38 to select
the output of the two-tap moving averaging filter 55, as performed by the step S10.
If however the value of the signal sample is greater than the threshold plus the offset
value, then the bypass sample is selected from the output of the fourth-tap of the
shift register 50.
[0031] If detail is not detected at either step S2 or S4 then the processing proceeds via
the loop S6 to the beginning of step S2 and the control signal is generated to select
the output from the eight tap filter.
[0032] A further advantage is provided by selecting either the four-tap output or the two
tap output independence upon the relative position of a sample representing detail
in the colour image and the value of the samples representing the colour image. Effectively,
the threshold processor is adjusting the bandwidth of the low pass filter in accordance
with the relative frequency bandwidth of the pixels samples passing through the filter.
Accordingly, as the detail pixels sample are detected within the eight tap window,
but just before the four tap window, then the four tap output is selected. If the
detail pixel samples are detected within the four taps window but just before the
two-tap window, then the output from the two-tap window is selected. Otherwise the
output samples from the eight tap moving averaging filter are selected. By selecting
a degree of filtering in accordance with a relative position in the eight-tap window,
an improved representation of the colour image is provided in which the likelihood
of contour effects is reduced.
[0033] Furthermore, it has been found to improve the colour image quality if the two tap
moving averaging filter is used to produce output samples, when samples representing
detail in the colour image are determined to fall between the threshold and the threshold
value plus the predetermined offset. The predetermined offset value is set to detect
sample values which exceed the threshold value by a relatively small amount. It has
been discovered that filtering such sample values with a two tap moving averaging
filter provides an improvement in the reproduced colour image. This is because these
sample values which just exceed the threshold but not the offset represent intermediate
detail in the reproduced image which requires some limited degree of filtering with
respect to the two immediately surrounding samples. Therefore as illustrated by step
S7, S8 and S10, if the difference between signal samples is greater than the threshold
plus the offset, then the filter is bypassed and the centre sample that is that contained
in the fourth stage of the register 60 is passed to the output of the filter.
[0034] A result of processing an image represented as 12 bit signal samples shown in Figure
3 with the image processor 20 is shown in Figure 11. As can be seen in Figure 12 the
contour effects which appear in the face of the women in Figure 3 have been removed.
However the fine detail information has been preserved in this image.
[0035] Although in the example embodiment described above, the low pass filter is arranged
to filter only one-dimensional signal samples, in alternative embodiments a two-dimensional
filter may be used to provide a further improvement in the reproduced image. It will
be appreciated therefore that the filter could be arranged to filter vertical or horizontal
component samples of the image.
[0036] Various modifications may be made to the embodiments of the invention as herein before
described without departing from the scope of the present invention. In particular,
as will be appreciated a colour image may be compressed by representing the colour
samples with an even smaller number of bits. The colour image may then be recovered
by increasing the resolution of the bits as the colour signal samples are read out
from the memory, to the original resolution in accordance with the embodiments described
above. Thus it will be appreciated that resolutions other than 12 bits for storing
the colour image, and the enhanced resolution samples of 18 bits are envisaged.
1. An image processing apparatus operable to process input colour component signal samples
representative of an input colour image to produce output colour component signal
samples representing an enhanced version of said input colour image, said apparatus
comprising
an enhancing processor operable to increase the dynamic range of the input colour
component samples,
a low pass filter operable to filter the increased dynamic range colour signal
samples to form said output colour component samples, and
a detection processor operable to detect input colour component samples associated
with a representation of detail appearing in said input colour image, and consequent
upon detection of said detail said detection processor is operable to adjust the band
width of the low pass filter in correspondence with the detected detail.
2. An image processing apparatus as claimed in Claim 1, wherein said detection processor
is operable to adjust the bandwidth of low pass filter by selecting said increased
dynamic range colour samples representing said detail as the output colour component
signal samples, without passing through said low pass filter.
3. An image processing apparatus as claimed in Claim 1 or 2, wherein said input colour
signal samples are represented as a predetermined number of bits n and said enhancing processor is operable to increase the resolution of said input
colour component samples by adding a predetermined number of additional least significant
bits l to said input colour signal samples and copying the values of the corresponding number
l of most significant bits of said input component signal samples into the l least significant bits.
4. An image processing apparatus as claimed in Claim 1, 2 or 3, wherein said low pass
filter includes a moving averaging filter having a predetermined number of taps d.
5. An image processing apparatus as claimed in Claim 4, wherein said input colour component
samples are passed through said moving average filter to form said output colour component
samples, and consequent upon detection of detail the samples of said input colour
component samples associated with said detail are passed to said output when said
associated samples reach a centre tap of said moving averaging filter.
6. An image processing apparatus as claimed in any preceding Claim, wherein said low
pass filter comprises a plurality of low pass filters, each having a different pass
bandwidth, and the detection processor is operable to adjust the bandwidth of low
pass filter by selecting the output of one of said low pass filters.
7. An image processing apparatus as claimed in Claim 6, wherein each of said low pass
filters comprises a moving averaging filter, each having a different number of taps,
each of said moving averaging filters providing a different length window of input
signal samples centred with respect to a common temporal sample position, and said
detection processor is operable to detect samples representative of said detail with
respect to said common temporal reference and consequent upon said detection, to select
the output signal sample from a corresponding one of said moving averaging filters,
before said detail samples have entered the window of the filter.
8. An image processing apparatus as claimed in Claim 6 or 7, wherein said detection processor
is operable to detect the samples associated with said detail by comparing successive
samples present in the taps of said moving averaging filters.
9. An image processing apparatus as claimed in Claim 8, wherein said detection processor
is operable to detect samples associated with detail by comparing each of the colour
component samples of one pixel with the corresponding colour component sample of a
successive pixel, and if each of the colour component samples of the successive pixel
has changed, detecting detail represented by said successive pixel.
10. An image processing apparatus as claimed in Claim 8 or 9, wherein said compared samples
are separated by two sample periods.
11. A display device comprising a display, a display memory and an image processing apparatus
as claimed in any of Claims 1 to 10, wherein input colour component samples representative
of a colour image are stored in said display memory prior to being displayed and said
input colour component samples are read out from said display memory and processed
by said image processing apparatus before being displayed by said display device.
12. A display device as claimed in Claim 11, comprising
a compressing processor operable to receive said input colour component samples
and to reduce a resolution of each of said samples by discarding at least one least
significant bit of each said sample, said reduced resolution samples being stored
in said memory, wherein a capacity of said memory required to store said colour image
is reduced in accordance a number of said least significant bits of each said sample
discarded.
13. A display device as claimed in Claim 11 or 12, wherein said display is a Liquid Crystal
Display (LCD).
14. A display device as claimed in Claim 11, 12 or 13, wherein enhanced colour component
signal samples produced by said image processing apparatus are converted to analogue
form by an analogue-to-digital converter for display on said LCD display.
15. A computing or communicating device having a display device according to any of Claims
11 to 14.
16. A mobile radiotelephone having a display device as claimed in any of Claims 11 to
14.
17. A method of processing input colour component signal samples representative of an
input colour image to produce output colour component signal samples representative
of an enhanced version of said input colour image, said method comprising
increasing the resolution of the input colour component samples,
low pass filtering the increased resolution colour component signal samples to
form said output colour component samples, and
detecting input colour component samples associated with a representation of detail
appearing in said input colour image, and consequent upon detection of said detail,
adjusting the band width of the low pass filter in correspondence with the detected
detail.