(19)
(11)EP 3 624 454 A1

(12)EUROPEAN PATENT APPLICATION

(43)Date of publication:
18.03.2020 Bulletin 2020/12

(21)Application number: 19191126.2

(22)Date of filing:  09.08.2019
(51)Int. Cl.: 
H04N 19/48  (2014.01)
H04N 5/357  (2011.01)
G06T 9/00  (2006.01)
H04N 19/60  (2014.01)
H04N 5/361  (2011.01)
(84)Designated Contracting States:
AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR
Designated Extension States:
BA ME
Designated Validation States:
KH MA MD TN

(30)Priority: 13.09.2018 AU 2018903448

(71)Applicant: Blackmagic Design Pty Ltd
Port Melbourne, Victoria 3207 (AU)

(72)Inventors:
  • BUETTNER, Carsten
    Port Melbourne, Victoria 3207 (AU)
  • HABEL, Rowan Douglas
    Port Melbourne, Victoria 3207 (AU)

(74)Representative: Kramer, Dani et al
Mathys & Squire LLP The Shard 32 London Bridge Street
London SE1 9SG
London SE1 9SG (GB)

  


(54)IMAGE PROCESSING METHOD AND SYSTEM


(57) The present invention relates to a method of processing image data that is represented in a frequency domain. The method includes the step of applying a noise signal (140) to the image data to yield noise-applied image data.




Description

Field of the invention



[0001] The present invention relates to methods and systems for processing image data. More specifically, the present invention relates to methods and systems for compression of image data that includes noise and its decompression.

Background of the invention



[0002] Digital colour cameras capture colour images of real-life scenes electronically using an image sensor, usually a charge-coupled device (CCD) or a complementary metal oxide semiconductor sensor (CMOS). The data comprising the image is often processed in the camera and encoded, either for internal storage on the camera or for transmission from the camera for storage on external media and/or downstream processing.

[0003] Digital image capture involves a process of opto-electronic conversion which introduces noise into the image data. Because noise is included in an image almost from the moment of its creation, the noise necessarily undergoes the same image processing and encoding routines as the non-noise component of the image. For encoding techniques that involve discrete cosine transformations (DCT) (and especially those operating at higher compression ratios), higher frequency image components tend to be removed, which manifests as block-type structures in the encoded image. Noise components removed through DCT encoding can impart a 'plastic' appearance to surfaces depicted in the encoded image. Consequently, images areas that are low in structural detail may appear to look 'unnatural' due to these removed higher frequencies and distorted spatial noise characteristics.

[0004] There are myriad techniques for dealing with the inevitable noise that is present in a typical image. These include signal processing approaches, as well as various categories of filtering-such as deblocking filters applied after the image is decoded, which aim to attenuate certain compression artefacts. Such filtering generally attempts to reduce the visibility of block-type artefacts.

[0005] The present invention aims to provide an approach to dealing with such compression artefacts.

[0006] Reference to any prior art in the specification is not an acknowledgment or suggestion that this prior art forms part of the common general knowledge in any jurisdiction or that this prior art could reasonably be expected to be understood, regarded as relevant, and/or combined with other pieces of prior art by a skilled person in the art.

Summary of the invention



[0007] Aspects and embodiments of the present invention are set out in the appended claims. These and other aspects and embodiments of the invention are also described herein. According to a first aspect of the present invention there is provided a method of processing image data represented in a frequency domain, the method including the step of applying a noise signal to the image data to yield noise-applied image data.

[0008] The present invention takes what may be seen as a counterintuitive approach to image processing by applying a noise signal to an image rather than attempting to attenuate or wholly remove the noise.

[0009] The invention arises from observations that certain components and intensities of image noise can actually be experienced by a viewer as a perceptually pleasant or more realistic attribute of the image being viewed. This is especially so when it is a moving image being viewed. More specifically, image noise that is dominated by particular distributions of photon shot noise can be experienced by viewers as being perceptually pleasant. As some of this perceptually pleasant image noise is typically being lost or distorted in the compression process, the invention aims to reproduce certain spatial and temporal noise distributions so they are better represented in the decoded image.

[0010] Typically, the image data represented in the frequency domain includes one or more sets of coefficients arising from application of a discrete cosine transform, with each set of coefficients comprising a DC coefficient and one or more AC coefficients. According to this embodiment, the noise signal can be applied to the image data by modifying one or more of the coefficients in one or more of the sets.

[0011] Such modification may in some cases comprise replacing one or more of the AC coefficients with random or pseudo random numbers having a distribution. Optionally, the random or pseudo random numbers are generated using a seed value. According to this embodiment, the replaced AC coefficients can be those having zero value.

[0012] According to preferred embodiments, the method further includes the step of equalising noise present in the image data, the equalising step being performed prior to the step of applying the noise signal. In this embodiment, the equalising step is performed on an uncompressed, spatial representation of the image data. Typically, the equalising step includes equalising the noise such that its amplitude or standard deviation is substantially constant across the entirety of the image data.

[0013] The noise may be equalised by reference to a noise model, which can be a model of the opto-electronic conversion performed by the specific image sensors that were used to capture the image data. For example, the noise model may characterise noise by reference to its spatio-temporal distribution within the image data. The noise model may also account for photon shot noise and/or read noise associated with the opto-electronic conversion.

[0014] According to some embodiments, the noise model substantially follows a normal distribution.

[0015] According to some embodiments, the noise signal applied to the image data is scaled by reference to the noise model.

[0016] Optionally, the method includes the steps of applying an inverse quantisation to the image file prior to application of the noise signal, as well as entropy-decoding the compressed image (optionally prior to the step of applying the inverse quantisation).

[0017] The method may also include the step of applying an inverse discrete cosine transform, optionally after the step of applying the noise signal to the image data.

[0018] According to another aspect, the present invention provides a method for processing an image signal obtained from an image sensor, the image signal having a noise component with an amplitude and an image component with an amplitude, the method including:

determining a conversion function from characteristics of the image sensor; and

applying the conversion function to the image signal to yield a signal with a noise component of substantially constant amplitude independent of the amplitude of the image component.



[0019] The "amplitude" of the noise component does not refer to noise component's value, but rather to the amplitude of a statistical measure of the noise component.

[0020] The method may further include the step of further recording details of the conversion function in association with the image signal. The method may further include the step of converting the image data into a frequency domain representation. According to some embodiments, the conversion comprises applying a discrete cosine transform to the image data.

[0021] The method may further including the step of quantising the converted image data, as well as entropy encoding the quantised image file.

[0022] The invention extends to methods, system and apparatus substantially as herein described and/or as illustrated with reference to the accompanying figures.

[0023] The invention also provides a computer program or a computer program product for carrying out any of the methods described herein, and/or for embodying any of the apparatus features described herein, and a computer readable medium having stored thereon a program for carrying out any of the methods described herein and/or for embodying any of the apparatus features described herein.

[0024] The invention also provides a signal embodying a computer program or a computer program product for carrying out any of the methods described herein, and/or for embodying any of the apparatus features described herein, a method of transmitting such a signal, and a computer product having an operating system which supports a computer program for carrying out the methods described herein and/or for embodying any of the apparatus features described herein.

[0025] Any feature in one aspect of the invention may be applied to other aspects of the invention, in any appropriate combination. In particular, method aspects may be applied to apparatus aspects, and vice versa. As used herein, means plus function features may be expressed alternatively in terms of their corresponding structure, such as a suitably programmed processor and associated memory.

[0026] Furthermore, features implanted in hardware may generally be implemented in software, and vice versa. Any reference to software and hardware features herein should be construed accordingly.

Brief description of the drawings



[0027] An embodiment of the invention will now be described with reference to the accompanying drawings, in which:

Figure 1 is a schematic illustration of a digital camera;

Figures 2 and 2A are flow charts of the encoding phase of a method in accordance with embodiments of the present invention;

Figure 3 is a flow chart of the decoding phase of a method in accordance with an embodiment of the present invention;

Figure 4 is an example plot that depicts a representative relationship between certain image capture parameters;

Figure 5 is an example plot of total noise versus generated electrons converted into DN in both axes;

Figure 6 is an example transform that converts a linear signal into a noise equalised response (NER) such that the plot of Figure 5 is a flat line;

Figure 7 is an example surface plot of a certain white noise standard deviation at each frequency domain coefficient of an 8x8 DCT;

Figures 8 and 9 are example surface plots of certain shaped noise standard deviations at each frequency domain coefficient of an 8x8 DCT.


Detailed description of the embodiments



[0028] Embodiments of the present invention will now be described with reference to specific examples, but the scope of the invention should not be considered as being limited to such examples.

[0029] Figure 1 is a schematic representation of a camera 10 capable of implementing methods according to the present disclosure. The camera 10 includes an image capture system 12 configured to convert received light into image data. In this example the image capture system 12 includes an image sensor 12A (e.g. a CCD or CMOS image sensing chip or similar) and associated optical filter 12B (e.g. an IR cut-off filter or an optical low pass filter). An optical system 14, e.g. lens is also provided to form images on the image sensor. The image sensor includes a colour filter array (CFA), in the form of a spatial array of colour filters, placed in front of the image sensor chip to enable capture of colour images. The filters of the CFA are spatially correlated with photosites of the single image sensor, so that each photosite has a corresponding spectrally selective filter. In this example the CFA used is a Bayer CFA with alternating red, green and blue colour filters, although the techniques of the present invention are not limited to use with any specific CFA.

[0030] The video data that is generated by the image capture system 12 is passed to the image processing subsystem 18. The image processing subsystem 18 may comprise one or more data processors, such as an ASIC or FPGA or microprocessor with associated software, and is configured to perform a range of image processing tasks. Working memory 20 is provided to enable temporary storage of data or software or the like during image processing and other tasks.

[0031] The image processing subsystem 18 also includes a video encoding system 22. The video encoding system 22 will typically be implemented by providing software configured to enable a processor to implement one or more video codecs. This system can be used to encode and compress the video data into a desired format as well as executing the video processing methods in accordance with the current disclosure. The disclosed methods can also be executed on processing platforms separate from the camera, such as dedicated external video recorders and general purpose computer systems.

[0032] The image processing subsystem 18 may also include a format conversion system 24 which processes video output data into a format (such as Y'CbCr) that is better able to be transmitted over a video transmission system 26 or that is more convenient for downstream processing. The format conversion system 24 is provided to format the video data into one of said video transmission formats to enable transmission, before being passed to the video transmission system 26. This can include transcoding video data from its original format into (one of) an appropriate video transmission format of the video transmission system 26.

[0033] The video transmission system is operable to transmit (and optionally receive) video output data (such as to an external video recorder or recorder/monitor) via a video interface having at least a video output port. The video interface can be bi-directional and thus also include a video input port. As an example the video interface could be an SDI interface or other like interface.

[0034] The camera also includes a data storage system in the form of a memory control subsystem 28 which is configured to control persistent storage of video data (and any other data) on a local non-volatile memory 30. The local memory 30 may use a removable memory such as a memory card or removable hard drive. However in the general case the memory control subsystem 28 is arranged to transmit and/or receive control signals to/from the local memory 30 to control storage and retrieval of video data on the memory 30 and also to perform any encoding or formatting of data for storage. The memory 30 could be a solid state drive operating according to the Serial ATA protocol, in which case the memory control subsystem will operate to control operation of the SATA drive and manage reading and writing of data to it.

[0035] The encoding phase 35 of the method according to an embodiment of the current disclosure is illustrated by the flowchart of Figure 2. The method commences at step 40 with the capturing of an image. As noted above, digital image capture involves the conversion of light received at image sensor 12A first into an electric charge, followed by a voltage and then finally into digital numbers (DN). The meaning of the DNs is dependent on the particular image representation format being used and the techniques of the present invention are not limited to any particular format. For example, each pixel in an image in RGB format comprises a vector of three DNs; one for each of three colour channels (being red, green and blue), with the entire image comprising a two dimensional array of pixels. The two dimensional pixel array can also be analysed from the perspective of being a digital image signal.

[0036] The present invention can equally be performed on raw image data that has not undergone demosaicing as well as on image data that includes 4 or more spectral bands.

[0037] Subsequent to image capture, a conversion function (step 50) is applied to each pixel comprising the image signal. In this regard, it is a fact that the process of opto-electronic conversion inevitably introduces noise into image signals during their construction. This introduced noise is generated from both the opto-electronic and electronic componentry of the image capture system 12, and because noise begins to be introduced to an image signal from the moment of its creation, the noise component undergoes the same image processing routines as the image component of the signal.

[0038] However, the inventors have discerned that certain components and intensities of image noise can actually be experienced by a viewer as a perceptually pleasant, or more realistic attribute of the image being viewed.

[0039] In particular, the inventors have discerned that image noise which is dominated by particular distributions of photon shot noise can be experienced by viewers as a perceptually pleasant attribute. Photon shot noise arises from the fundamental physical processes governing the behaviour of discrete quantum particles and their interaction with image sensors. As noted above, the array of photosites on image sensor 12A produces a signal in proportion to the amount of incident light striking each photosite during a defined time period (or 'integration time'). However, even with constant illumination and integration time, the signal captured by the image sensor always subtly varies from one capture to another, producing the phenomenon of photon shot nose.

[0040] The random statistical fluctuations that characterise photon shot noise can be modelled as a Poisson process, with the standard deviation thereof (s_e) being described by the formula:

with N_e being the number of electrons generated in the pixel's photodiode.

[0041] This Poisson process can also be described as a normal distribution.

[0042] Another important temporal noise component is the noise floor, also known as the 'read noise' or 's_r' of the image sensor 12A. The noise floor usually incorporates a combination of individual noise sources including optical and and electronic components such as amplifiers and ADCs. The noise floor can also be modelled as a normal distribution that is constant for specific componentry settings, such as a fixed amplifier gain. The noise floor is also independent of the integration time and of the amount of light received at the image sensor.

[0043] Modelling these temporal noise sources leads to a total noise of:



[0044] This quantity can be transferred into the digital domain by a system conversion factor.

[0045] Knowing the noise model for a specific image sensor and its settings allows the conversion function to be computed and applied to the image sensor's usually linear signal output in a way that the standard deviation of the temporal noise is substantially constant across a spectrum of signal levels.

[0046] This converted signal is described herein as a 'noise equalised response'.

[0047] As described below, using this concept of noise-equalised response also allows noise having a substantially constant distribution to be added to an image area-or indeed to the whole image- independent of the local brightness.

[0048] Subsequent to conversion, typical image processing can be performed (step 60) on the noise-equalised response signal, such as demosaicing, application of filter kernels and/or transformation into other colour spaces (such as YCbCr). Such processing may necessarily have the effect of changing the noise distribution.

[0049] Following image processing, the noise equalised response signal is encoded (step 70) using a discrete cosine transform (DCT). Such encoding involves splitting the image data array into suitable sized blocks (such as 8x8) and converting each block to a frequency-domain representation, using a DCT. Before computing the DCT of each block, its values (that incorporate the noise-equalised response) may be shifted to ensure a range centered on zero. A two-dimensional DCT is then taken of the block. This yields a block having a top-left corner entry (or DC coefficient) that is often of relatively larger magnitude that defines the block's basic intensity level. The remaining coefficients are known as the AC coefficients.

[0050] DCT encoding in this way allows the noise distribution to be analysed in the DCT (namely frequency) domain. In addition (and as described below), as the amplitude of the noise signal is known and (at least in some cases) substantially constant, each DCT block can be individually analysed during a decoding operation at a time after inverse quantisation has been performed. This allows a noise distribution to be regenerated in the frequency domain that closely matches (in the sense of being perceptually similar) the distribution that was present in the signal before undergoing quantisation and other rounding performed during encoding.

[0051] Furthermore, knowing the standard deviation of the noise embedded in the noise-equalised response allows data storage for certain image areas having more constant brightness, to be limited to a DC offset and thus minimised. The spatial and temporal noise for these areas can be readily regenerated and added to the signal at a later time. In this regard, details of the noise distribution can be stored as metadata in selected frame headers or in other parts of the image data including tiles, slices or at block level. This allows a decoder (discussed below) to apply an appropriate noise signal for each specific frame.

[0052] The DCT blocks are next quantised (step 80) by reference to a suitable quantization matrix. Quantization yields DCT blocks with coefficient values that are smaller in magnitude than the input. Input frequency coefficients that are sufficiently small (low magnitude) with respect to a given degree of quantisation will be compressed down to zero. When this occurs, any noise component present in the signal at such coefficients is completely removed.

[0053] Following quantization, the DCT blocks are further compressed, this time losslessly, through a process of entropy encoding (step 90). This involves the application of a run-length encoding algorithm to the frequency data, and a Huffman-type encoding of the resultant data. This yields an encoded compressed image which can be written into an encoded image file 100.

[0054] An alternative encoding phase is illustrated by reference to the flowchart in Figure 2A. The illustrated encoding methodology is as per. Figure 2, but with the conversion function (step 65 in Figure 2A) being applied to image data that has already undergone processing (step 55 in Figure 2A). The image processing performed in step 55 is equivalent to that performed in step 60, namely demosaicing, application of filter kernels and/or transformation into other colour spaces (such as YCbCr).

[0055] The decoding phase 110 of the method according to the current disclosure is illustrated by the flowchart of Figure 3. The decoding phase 110 is used to play back previously recorded image data, which can occur on camera hardware or on a separate processor as part of a video player or editing system.

[0056] Method 110 is performed on the encoded image file 100 generated by performing method 35. Performing method 110 first involves entropy-decoding (step 120) of encoded image file 100. Entropy decoding involves perfectly reversing the entropy encoding (step 90) of the encoding phase 35 to yield a collection of DCT blocks, each comprising a matrix of quantized DCT coefficients.

[0057] Inverse quantisation (step 130) is then performed on each block of quantised DCT coefficients. Inverse quantisation involves reversing, as far as is possible, the quantisation step (step 80) of the encoding phase 35. It is performed with reference to the same quantisation matrix selected for use in quantisation (step 80). The de-quantised DCT blocks thus produced, closely (but not exactly) resemble the original DCT coefficient matrix-namely that produced during the DCT step (step 70). In turn, the DCT coefficient matrix is a representation of the original image file in the frequency domain.

[0058] The imperfect reproduction achieved from de-quantising a quantised DCT block is largely responsible for the appearance of compression-related artefacts seen in a fully decoded image or sequence of images. These artefacts are due to the original frequency coefficients being rounded to a smaller number of quantised levels, including to zero. After inverse quantisation, any coefficient with a value of zero had potentially originally been a small noise or signal component that was removed. Whilst the precise original values of the noise components can never be recovered, knowledge of the noise's random distribution allows generation of new random or pseudo-random data values that follow that distribution (or an approximation thereof).

[0059] In this regard, noise reproduction is performed (step 140) on each DCT coefficient matrix output from the inverse quantisation step. Noise reproduction involves modifying the AC coefficients of each DCT block. A preferred modification involves substituting normally distributed random or pseudo-random numbers for any AC coefficients of the DCT coefficient matrix having a zero value. The pseudo random number generator can have reference to a seed value present in the header of each frame. This is to ensure consistency in the reproduced noise signal across each frame irrespective of how and when the frame is decoded.

[0060] Noise reproduction has the result of encoding a normal noise distribution into the DCT coefficient matrix, thereby subtly modifying one or more of its components.

[0061] In some cases, this is achieved in such a way that the noise characteristics of the fully decoded image after inverse DCT (and after other optional operations) appear to be the same as the noise present in the original image before encoding.

[0062] As the original noise component of the signal has a random distribution both temporally and spatially, a new random distribution can be created that may not have exactly the same values in each specific block as existed before quantisation. However, as the noise signals share the same statistical distribution, an exact reproduction of coefficient values is not required.

[0063] After noise reproduction, the DCT coefficient matrix (now including the modified coefficients modulating the desired noise signal) undergoes Inverse DCT (step 150). The process proceeds by initially taking the two-dimensional inverse DCT of the matrix, rounding the results to integer values and shifting the integer values to restore the original data range.

[0064] Subsequent to Inverse DCT, optional deblocking processes (step 160) can be performed on the resulting matrix such as to account for non-white-noise distributions where higher frequencies are attenuated in comparison to lower frequency noise components. Other optional processes include inverse conversion from any applied noise-equalised response, colour processing or other image processing.

[0065] A decoded image 170 is produced from performing method 110.

[0066] The following discussion is an example of performing the methods described above on a typical image with typical noise characteristics, in which:

N_e - is the number of electrons generated in a pixel's photodiode

s_e - is the standard deviation of the photon shot noise component of the temporal noise (using the unit: electrons)

s_r - is the standard deviation of the noise floor (or the "read noise") component of the temporal noise (using the unit: electrons); and

s_total - is the total noise standard deviation (using the unit: electrons)



[0067] As noted above, most of the signal range is ideally dominated by photon shot noise which is modelled as a Poisson process and the standard deviation s_e can be described by:



[0068] Modelling only these 2 temporal noise sources leads to a total noise (summed in quadrature) of



[0069] The maximum value taken by this model occurs when N_e is maximal. This represents the number of electrons generated in a photodiode that is saturated with light - namely the "full well capacity" of the photodiode. In the discussion below:

fwc - is the Full Well Capacity (electrons); and

s_max - is the Maximum standard deviation of the total noise



[0070] Typical example values of these quantities for an image sensor are:

s_r = 3

fwc = 30000



[0071] The graph shown in Figure 4 depicts the representative relationship between N_e and s_total.

[0072] This can be transferred into the digital domain by a system conversion factor, which is a linear scaling from units of electrons to units of digital-numbers (DN).

[0073] The system conversion factor is in units of DN per electron, with:

dn_max - being the DN value to which full well capacity is mapped;

dn_off - being the DN "offset" value to which a zero electrons (black) input signal is mapped (note: this is not the mapping of the read_noise level); and

A_cf - being the Conversion Factor (DN/e-).

dn_off is typically set as a pedestal level greater than zero in order to avoid clamping the read-noise that is present even in a zero electron input scenario.



[0074] Example values for a camera's image acquisition are:

dn_max = 65535;

dn_min, dn_off = 160;



[0075] As shown in Figure 5, the representation of total noise versus generated electrons can be converted to DN in both axes.

[0076] For an input DN value, conversion back to the originating electron count is



[0077] For inputs that, through subjection to noise, have a value lower than dn_min, the noise is modelled as being just the read-noise.

[0078] This is equivalent to clamping the input electrons to non-negative values.
input_e = max{0, (input_DN - dn_off)}/A_cf

[0079] The associated total noise is thus modelled as



[0080] The noise can be re-expressed in the digital domain as



[0081] It is convenient to derive a transform that converts the linear signal into the noise equalised response (NER) so that the plot of Figure 5 becomes a flat line, with a substantially constant noise standard deviation across the entire signal range. An example of such a transform is shown in Figure 6.

[0082] The noise of such a non-linearly mapped resultant signal, y = f(x), where x is the linear input DN signal, is computed as the product of the linear signal's noise and the gradient function dy/dx = f'(x).
As such, the mapping must satisfy:



For x >= dn_off, this is solved as:

For x < dn_off, this is solved as:



[0083] As such, so long as the 'c' constants are chosen in such a way as to ensure smooth continuity between the two sub-ranges, a constant signal noise response results across the signal input range.

[0084] A non-zero pedestal is also used in the transformed signal to ensure a reasonable number of STD-deviations of dark noise are not clamped, namely:

bps - the DN bits per sample of the output NER signal

offset- a non-zero offset (pedestal) used for the NER signal

b - a fractional offset

Let b = offset/2bpp-1



[0085] The transform from linear signal to NER signal is defined as:

with further clamping to ensure y is non-negative everywhere.

[0086] For which the constant noise STD-deviation value (DN) is everywhere (except where clamped)


Frequency-domain Noise Characterisation



[0087] If no pre-compression image processing is performed, the NER image is modelled as having a spatial noise distribution of white noise. This is illustrated in Figure 7.

[0088] If compression were to completely remove (quantise/round) all noise in such a DCT 'block', the reproduced noise applied (step 140) during the decompression step (prior to the Inverse DCT operation), would have a similar profile (though only applied at the AC coefficients having zero value).

[0089] If pre-compression image processing is performed, and includes spatial filtering operations that are not band-pass (typically low-pass in practice), the spatial covariances created as a result lead to a non-white noise distribution that presents in the frequency domain as non-uniform noise level across the different DCT frequency coefficients:
This is illustrated in Figures 8 and 9 through consideration of an 8x8 DCT, but any other size frequency decomposition should behave similarly.

[0090] If compression were to completely remove (quantise/round) all noise in such a DCT 'block', and the desire is to recreate the shaped noise profile (i.e rather than white-noise), then the reproduced noise applied (step 140) during the decompression process (prior to the IDCT operation), would have a similar profile (though only applied at the AC coefficients having zero value). Noise injected would not have the same STD-deviation for all frequency coefficients.

[0091] It will be understood that the invention disclosed and defined in this specification extends to all alternative combinations of two or more of the individual features mentioned or evident from the text or drawings. All of these different combinations constitute various alternative aspects of the invention.

[0092] It will be understood that the invention has been described above purely by way of example, and modifications of detail can be made within the scope of the invention.

[0093] Each feature disclosed in the description, and (where appropriate) the claims and drawings may be provided independently or in any appropriate combination.

[0094] Reference numerals appearing in the claims are by way of illustration only and shall have no limiting effect on the scope of the claims.


Claims

1. A method of processing image data represented in a frequency domain, the method including the step of applying a noise signal (140) to the image data to yield noise-applied image data.
 
2. A method according to claim 1, wherein the image data includes one or more sets of coefficients arising from application of a discrete cosine transform (70), each set of coefficients comprising a DC coefficient and one or more AC coefficients.
 
3. A method according to claim 2, wherein the noise signal (140) is applied to the image data by modifying one or more of the coefficients in one or more of the sets.
 
4. A method according to claim 3, wherein the modifying comprises replacing one or more of the AC coefficients with random or pseudo random numbers having a distribution.
 
5. A method according to claim 4, wherein the random or pseudo random numbers are generated using a seed value.
 
6. A method according to claim 4, wherein the replaced AC coefficients are those having zero value.
 
7. A method according to any preceding claim, further including the step of equalising noise present in the image data, the equalising step being performed prior to the step of applying the noise signal (140).
 
8. A method according to claim 7, wherein the equalising step includes equalising the noise such that its amplitude or standard deviation is substantially constant across the entirety of the image data.
 
9. A method according to claim 7, wherein the noise is equalised by reference to a noise model.
 
10. A method according to claim 9, wherein the noise model is a model of an opto-electronic conversion performed by one or more image sensors (12A) on which the image data was captured.
 
11. A method according to claim 10, wherein the noise model accounts for photon shot noise and/or read noise associated with the opto-electronic conversion.
 
12. A method according to claim 9, wherein the noise signal (140) applied to the image data is scaled by reference to the noise model.
 
13. A method according to any preceding claim, further including the step of applying an inverse quantisation (130) to the image data prior to the step of applying the noise signal (140) to the image data.
 
14. A method according to claim 13, further including the step of applying an inverse discrete cosine transform (150) after the step of applying the noise signal (140) to the image data.
 
15. A method according to claim 13, further including the step of entropy-decoding (120) the image data prior to the step of applying the inverse quantisation (130).
 
16. A method for processing an image signal obtained from an image sensor (12A), the image signal having a noise component with an amplitude and an image component with an amplitude, the method including:

determining a conversion function (50) from characteristics of the image sensor; and

applying the conversion function to the image signal to yield a signal with a noise component of substantially constant amplitude independent of the amplitude of the image component.


 
17. A method according to claim 16, further including the step of recording details of the conversion function in association with the image signal.
 
18. A method according to claim 16, further including the step of converting the image signal into a frequency domain representation.
 
19. A method according to claim 18, wherein the conversion comprises applying a discrete cosine transform (70) to the image signal.
 
20. A method according to claim 18, further including the step of quantising (80) the converted image signal.
 
21. A method according to claim 20, further including the step of entropy encoding (90) the quantised image signal.
 




Drawing