|
(11) | EP 2 382 598 B1 |
| (12) | EUROPEAN PATENT SPECIFICATION |
|
|
| (54) |
PROCESSING OF REMOTELY ACQUIRED IMAGING DATA INCLUDING MOVING OBJECTS VERARBEITUNG VON FERNERFASSTEN BILDGEBUNGSDATEN MIT BEWEGLICHEN OBJEKTEN TRAITEMENT DE DONNÉES D'IMAGERIE ACQUISES À DISTANCE COMPRENANT DES OBJETS MOUVANTS |
|
|
|||||||||||||||||||||||||||||||
| Note: Within nine months from the publication of the mention of the grant of the European patent, any person may give notice to the European Patent Office of opposition to the European patent granted. Notice of opposition shall be filed in a written reasoned statement. It shall not be deemed to have been filed until the opposition fee has been paid. (Art. 99(1) European Patent Convention). |
[0001] The invention concerns image processing, and more particularly, an image processing method for images having different spatial and spectral resolutions and including moving objects.
[0002] In the field of remote image sensing, two common types of images include panchromatic imagery and multi-spectral imagery. Panchromatic imagery is imagery that is obtained by a remote sensing device with a sensor designed to detect electromagnetic energy in only one very broad band. This one very broad band typically includes most of the wavelengths of visible light. Panchromatic imagery has the advantage of offering very high spatial resolution. In contrast, multi-spectral imagery is typically created from several narrow spectral bands within the visible light region and the near infrared region. Consequently, a multi-spectral image is generally comprised of two or more image data sets, each created by sensors responsive to different portions of the optical spectrum (e.g., blue, green, red, infrared). Multi-spectral images are advantageous because they contain spectral information which is not available from a similar panchromatic image. However, multi-spectral images typically have a lower spatial resolution as compared to panchromatic images.
[0003] It is often desirable to enhance a multi-spectral image with the high resolution of a panchromatic image and vice versa. Typically this process is referred to as "fusing" of the image pair. In general, there are several requirements for successfully accomplishing the fusing process, such as the registration of the image pair. The registration process involves a determination of where each pixel in the panchromatic image maps to a location in the multi-spectral image. This process must generally be accomplished with great accuracy for best results. For example, it is desirable for each pixel in the panchromatic image to be mapped to the multi-spectral image with an accuracy of less than 0.1 panchromatic pixel radius. Registration is typically accomplished via the use of metadata associated with the images that specifies the geographic location being imaged. In addition, other registration processes are typically used to account for differences in the imaged geographic location due to variations in sensor position and acquisition time. However, such registration methods generally fail to account for moving objects in the image pair, such as clouds, aircraft, watercraft, and ground vehicles, resulting in errors in the final fused image. Therefore, what is needed are systems and methods for registration and fusion of image pairs that take into account moving objects.
[0004] United States Patent 7,298,922 discloses globally registering a synthetic panoramic image and an acquired panchromatic image. The SPIE article Geopositional accuracy evaluation of QuickBird Ortho-Ready Standard 2A multispectral imagery, by Jamison, et. al., discloses a process of co-registering panchromatic and multispectral bands. United States Patent Application Publication US 2008/0037869 discloses generating global motion compensated images to compensate for background motion in the images, and further discloses local motion vectors for regions.
[0005] Embodiments of the present invention provide systems and methods for processing remotely acquired imagery including moving objects, according to the appended claims.
FIG. 1 is a conceptual illustration showing mapping functions are adjusted for an image pair including a moving object according to the various embodiments of the present invention.
FIG. 2 is a conceptual illustration a fused image generated from an image pair in FIG. 1 including a moving object according to the various embodiments of the present invention.
FIG. 3 is a schematic view of a computer system within which a set of instructions operate according to an embodiment of the invention.
FIG. 4 is a flowchart of steps in an exemplary method for processing remotely acquired imagery with moving objects according to an embodiment of the present invention.
FIG. 5 is a plot showing an example of a spectral response of sensors used to create a panchromatic image and a multi-spectral image that is useful for describing the present invention.
FIG. 6 is a conceptual illustration showing how spectral weights are used in a downsample processing of a multi-spectral image for decreasing the spectral resolution of the multi-spectral image.
FIG. 7 is a flowchart of steps in an exemplary method for processing remotely acquired imagery with moving objects according to an embodiment of the present invention.
[0006] Embodiments of the present invention provide systems and methods for adjusting remotely acquired imagery including moving objects. In particular, the various embodiments of the present invention provide systems and methods for improving registration and subsequent fusion of remotely acquired image pairs including moving objects. In general, moving objects in image pairs can create registration problems when there is non-simultaneous acquisition of the image pair. That is, even though conventional remote imagery systems acquire image pair data contemporaneously, slight delays are typical in a conventional acquisition processes. For example, differences in focal plane geometry can result in delays of ~0.5 seconds. As a result, the positions of fast-moving objects, such as clouds, aircraft, watercraft, and ground vehicles, can vary with respect to the geographic location being imaged.
[0007] As previously discussed, using conventional registration and fusion methods results in fused images that fail to take into account such moving objects. That is, such methods provide for associating pixels in each of the remotely acquired images based on primarily geographic information, but generally fail to account for pixels associated a moving object appearing in first and second areas of first and second images, respectively. As a result, the areas of the fused image corresponding to these first and second areas generally introduce errors into the fused image, obscuring details of the object. Accordingly, as the number of moving objects in the image pair increases, the number of errors in the fused image also increases.
[0008] To overcome these limitations, embodiments of the present invention provide systems and methods for registering an image pair that includes (1) generating a mapping function based on metadata and/or the image data aggregately, (2) locating areas in the image pair corresponding to moving objects, and (3) providing a registration correction in the mapping function for such areas of the image pair. This is conceptually illustrated with respect to FIG. 1.
[0009] FIG. 1 shows an exemplary image pair 100 including remote imagery data. As used herein, "remote imagery data" refers to any set of data defining an image pair. That is, the remote imagery data includes image data and any type of meta-data associated with a first and at least a second image to be combined. The image data is acquired from any remotely positioned sensor or imaging device. For example, the remote sensor can be positioned to operate on, by way of example and not limitation, an elevated viewing structure, an aircraft, a spacecraft, or man-made satellite. That is, the remote data is acquired from any position, fixed or mobile, that is elevated with respect to the imaged location. The image data can include light intensity data for an image acquired various sensors, each associated with a particular range of wavelengths (i.e., a spectral band). Therefore, in the various embodiments of the present invention, the remote imagery data can include multi-spectral (~4 bands), hyper-spectral (>100 bands), and/or panchromatic (visible band) image data.
[0010] In general, remote imagery data is assumed to have certain other characteristics. For example, the different images are typically acquired within a very short time of each other and from nearly the same position to reduce global registration errors. Still, it will be understood by those skilled in the art that the present invention can be utilized in connection with different images that do not necessarily satisfy this criteria, possibly with degraded results. Accordingly, image pairs can be obtained using differently positioned sensors, but can result in more complex registration processes, including more complex or multiple shifting, scaling, and rotation processes which take into account the three-dimensional (3D) shape of the scene.
[0011] For exemplary image pair 100, the associated remote imagery data comprises a panchromatic 102 image, a multi-spectral image 104, and associated meta-data. By way of example and not limitation, the meta-data includes information identifying a date, a time, and the geographic location for the images. For example, geographic coordinates for the four corners of a rectangular image can be provided in the meta-data. Other information suitable for facilitating the registration process can also be included in the meta-data, including any additional information regarding the sensor or the location being imaged.
[0012] In the various embodiments of the present invention, a global mapping function is provided that maps each pixel of MSI image 104 to a pixel of panchromatic image 102 and vice versa. For example pixels in image 102 can be mapped to pixels in image 104 using xp = M2P(xm). Similarly, pixels in image 104 can be mapped to pixels in image 102 using xm = P2M(xp). Such mapping functions can be generated based on meta-data or other methods, as described below. One of ordinary skill in the art will readily recognize that MSI images typically have a reduced spatial resolution as compared to panchromatic images. Accordingly, when mapping panchromatic image 102 to MSI image 104, multiple pixels of the panchromatic image 102 are mapped to at least one common pixel in the MSI image 104. Similarly, when mapping MSI image 104 to panchromatic image 102, one pixel of the MSI image 102 is mapped to multiple pixels of the panchromatic image 102.
[0013] As shown in FIG. 1, the images 102, 104 can include areas 106, 108 associated with a same object in motion. That is, if image 102 is acquired first, area 106 in image 102 would be associated with a first location of the object and area 108 would be associated with a second location of the object. Conventional registration and fusion processes generally assume that the areas 106, 108 are associated with non-moving objects and the global mapping functions used are configured to associate areas 106 and 108 with areas 110 and 112, respectively. However, if area 106 is associated with area 110 the resulting fused image associated with area 106 would be a combination of the object in area 106 and the surface obscured by the object in area 106 (area 110). Similarly, if area 108 is associated with area 112 using a global mapping function, the resulting fused image associated with area 108 would be a combination of the object in area 108 and the surface obscured by the object in area 108 (area 112). In either case, the resulting fused image in such areas is incorrect and can result in blurring and obscuring of details of the object in motion.
[0014] Accordingly, in the various embodiments of the present invention, rather than associating areas 106 and 108 with areas 110 and 112, respectively, area 106 is instead associated with area 108, or vice versa, using an alternate mapping function for the areas. As a result, when adding the high spectral resolution information in the MSI image 104 to the panchromatic image 102, the high spectral resolution information for the object in area 106 is instead obtained from the high spectral resolution information for the object in area 108. For example, as shown in FIG. 1, the pixels in area 106 can be mapped to pixels in area 108 using xp = M2P(xm). Alternatively, when adding the high spatial resolution information in the panchromatic image 102 to the MSI image 104, the high spatial resolution information for the object in area 108 is instead obtained from the high spectral resolution information from the object in area 106. For example, pixels in area 108 can be mapped to pixels in are a 106 using xm = P2M(xp), as shown in FIG. 1. The details of the generation of alternate mapping function with be described in greater detail below. The fusion results are conceptually illustrated in FIG. 2.
[0015] FIG. 2 shows an exemplary fused image 200 resulting from the fusion of image pair 100 in FIG. 1 according to an embodiment of the present invention. As shown in FIG. 2, fused image 200 includes an area 202 associated with non-moving objects in image pair 100, an area 204 associated with area 106 in FIG. 1, and an area 206 associated with are 108 in FIG. 1. For ease of illustration, fused image 200 is the panchromatic image 102 in FIG. 1 enhanced with the high spectral resolution information of the MSI image 104 in FIG. 1. Accordingly, the mapping function M2P, as shown in FIG. 1, permits fusion of image pair 100 to provide area 202 of the fused image, i.e., the areas of images 102, 104 associated with non-moving objects. Area 204 is provided by fusion of areas 106 and 108 using the alternate mapping function xm = P2M(xp), as described above. Finally area 206 can be provided by a fusion smoothing function. That is, although area 206 would be normally be the result of the fusion of areas 108 and 112 in FIG. 1, such a fusion would introduce errors into the fused image, as previously described. However, since the area 112 in the panchromatic image is associated with the surface obscured by the object in motion, an assumption can be made that the high spectral resolution in area 108 can be estimated from the areas of the MSI image 104 surrounding area 108. Accordingly, a smoothing function can be applied to cause the fusion process to calculate alternate high spectral resolution information for area 108 for use with area 112 to generate the fused area 206. Similarly, a fused image comprising the MSI image 104 enhanced with the high spatial resolution of the panchromatic image 102 can be provided. The details of a fusion process using a modified mapping function and a smoothing function will be described in greater detail below.
[0016] Although the various exemplary embodiments are primarily described herein in terms of utilizing a panchromatic image to enhance the spatial resolution of a MSI image, these embodiments are provided for ease of illustration only and the present invention is not limited in this regard. The methods and systems described herein are equally applicable to enhancement of the spectral resolution of a panchromatic image using information from the MSI image. Furthermore, the present invention is not limited to processing of exclusively MSI-panchromatic image pairs. The methods and system described here are equally applicable to image pairs comprising any types of images having different spatial and/or spectral resolutions.
[0017] The various embodiments of present invention are specifically embodied as a method, a data processing system, and a computer program product for generating mapping functions for image pairs. Accordingly, the present invention can take the form as an entirely hardware embodiment, an entirely software embodiment, or any combination thereof. However, the invention is not limited in this regard and can be implemented in many other forms not described herein. For example, FIG. 3 is a schematic diagram of an embodiment of a computer system 300 for executing a set of instructions that, when executed, causes the computer system 300 to perform one or more of the methodologies and procedures described herein. In some embodiments, the computer system 300 operates as a standalone device. In other embodiments, the computer system 300 is connected (e.g., using a network) to other computing devices. In a networked deployment, the computer system 300 operates in the capacity of a server or a client developer machine in server-client developer network environment, or as a peer machine in a peer-to-peer (or distributed) network environment.
[0018] In the some embodiments, the computer system 300 can comprise various types of computing systems and devices, including a server computer, a client user computer, a personal computer (PC), a tablet PC, a laptop computer, a desktop computer, a control system, a network router, switch or bridge, or any other device capable of executing a set of instructions (sequential or otherwise) that specifies actions to be taken by that device. It is to be understood that a device of the present disclosure also includes any electronic device that provides voice, video or data communication. Further, while a single computer is illustrated, the phrase "computer system" shall be understood to include any collection of computing devices that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.
[0019] The computer system 300 includes a processor 302 (such as a central processing unit (CPU), a graphics processing unit (GPU, or both), a main memory 304 and a static memory 306, which communicate with each other via a bus 308. The computer system 300 further includes a display unit 310, such as a video display (e.g., a liquid crystal display or LCD), a flat panel, a solid state display, or a cathode ray tube (CRT)). The computer system also includes an input device 312 (e.g., a keyboard), a cursor control device 314 (e.g., a mouse), a disk drive unit 316, a signal generation device 318 (e.g., a speaker or remote control) and a network interface device 320.
[0020] The disk drive unit 316 includes a computer-readable storage medium 322 on which is stored one or more sets of instructions 324 (e.g., software code) configured to implement one or more of the methodologies, procedures, or functions described herein. The instructions 324 reside, completely or at least partially, within the main memory 304, the static memory 306, and/or within the processor 302 during execution thereof by the computer system 300. The main memory 304 and the processor 302 also can constitute machine-readable media.
[0021] Those skilled in the art will appreciate that the computer system architecture illustrated in FIG. 3 is one possible example of a computer system. However, the invention is not limited in this regard and any other suitable computer system architecture can also be used without limitation.
[0022] For example, dedicated hardware implementations including, but not limited to, application-specific integrated circuits, programmable logic arrays, and other hardware devices can likewise be constructed to implement the methods described herein. Applications that can include the apparatus and systems of various embodiments broadly include a variety of electronic and computer systems. Some embodiments implement functions in two or more specific interconnected hardware modules or devices with related control and data signals communicated between and through the modules, or as portions of an application-specific integrated circuit. Thus, the exemplary system is applicable to software, firmware, and hardware implementations.
[0023] In accordance with various examples of the present invention, the methods described below can be stored as software programs in a computer-readable storage medium and can be configured for running on a computer processor. Furthermore, software implementations can include, but are not limited to, distributed processing, component/object distributed processing, parallel processing, virtual machine processing, which can also be constructed to implement the methods described herein.
[0024] Therefore, in some examples of the present invention, the present invention is embodied as a computer-readable storage medium containing instructions 324 or that receives and executes instructions 324 from a propagated signal so that a device connected to a network environment 326 sends or receive voice and/or video data and that communicate over the network 326 using the instructions 324. The instructions 324 are further transmitted or received over a network 326 via the network interface device 320.
[0025] While the computer-readable storage medium 322 is shown in an example to be a single storage medium, the term "computer-readable storage medium" should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions. The term "computer-readable storage medium" shall also be taken to include any medium that is capable of storing, encoding or carrying a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the present disclosure.
[0026] The term "computer-readable medium" shall accordingly be taken to include, but not be limited to, solid-state memories such as a memory card or other package that houses one or more read-only (non-volatile) memories, random access memories, or other re-writable (volatile) memories; magneto-optical or optical medium such as a disk or tape; as well as carrier wave signals such as a signal embodying computer instructions in a transmission medium; and/or a digital file attachment to e-mail or other self-contained information archive or set of archives considered to be a distribution medium equivalent to a tangible storage medium. Accordingly, the disclosure is considered to include any one or more of a computer-readable medium or a distribution medium, as listed herein and to include recognized equivalents and successor media, in which the software implementations herein are stored.
[0027] Although the present specification describes components and functions implemented in the embodiments with reference to particular standards and protocols, the disclosure is not limited to such standards and protocols. Each of the standards for Internet and other packet switched network transmission (e.g., TCP/IP, UDP/IP, HTML, and HTTP) represent examples of the state of the art. Such standards are periodically superseded by faster or more efficient equivalents having essentially the same functions. Accordingly, replacement standards and protocols having the same functions are considered equivalents.
[0028] The present invention will now be described in greater detail in relation to the flowchart in FIG. 4, illustrating steps in an exemplary method 400 for processing remote imagery data including moving objects according to an embodiment of the present invention.
[0029] As shown in FIG. 4, the method 400 can begin with step 402 and continue on to step 404. In step 404, an image pair is received comprising meta-data and imagery data of a geographic location. In exemplary method 400, the image pair defines a first image of a panchromatic type and a second image of an MSI type. Once the image pair is obtained in step 404, a mapping function for registering, i.e., aligning, the pixels in the image pair is created in step 406. In general, step 406 involves generating a mathematical function based on determining where each point in the panchromatic image maps to coordinates in the MSI image. A number of conventional methods exist for generating the mapping function. Typically, they involve selecting a number of points in one image, finding where they map to in the other image, and then optimizing the coefficients of a transform. This is usually a least squares error solution that permits one to obtain a set of coefficients that minimize the squared error of mapping points from one image to another. For best results in the fusing process, the panchromatic image is preferably mapped to the multi-spectral image with an accuracy defined by an error distance which is less than a dimension defined by 0.1 panchromatic pixel. The mapping function created in step 406 determines the mapping of points from the coordinates of one image to the coordinates of the other image. This mapping function can be as simple as a linear transformation of the form x1 = ax2 + by2 + x0, or a complex transformation modeling the geometry of both sensors and the surface imaged. The mapping function can be based on coordinates included within the imagery data as meta-data. For example, the meta-data can include latitude and longitude coordinates of the four corners of the acquired image pairs and the initial mapping function can be based on these coordinates.
[0030] Once the initial mapping function is created in step 406, the multispectral image data is combined to generate an approximated panchromatic image in step 408. In particular, the approximated panchromatic image is generated from multi-spectral image data calibrated to accurately correspond to the radiance values of pixels of the panchromatic image. Therefore, prior to generating the approximated panchromatic image, the spectral weights of the radiance values for the spectral bands comprising the multi-spectral image need to be determined. As used herein, the term "radiance value" generally refers to a digital value assigned to a pixel which is intended to represent the intensity of light energy received by a sensor at the location defined by that pixel. In this regard, it should be understood that these radiance values may be scaled differently in two different sensors. Accordingly, it will be appreciated that the radiance values from the two different sensors must somehow be adjusted or scaled by using suitable weighting factors before the radiance values from the two different sensors can be combined together in a meaningful way. This process is referred to as calibration.
[0031] One of ordinary skill in the art will recognize that a complete multispectral image of a particular geographic is typically comprised of several optical or spectral image bands. In each of these bands, the sensor is responsive to a very limited range of optical wavelengths. This concept is illustrated in FIG. 5 which shows curves 501, 502, 503, 504 which represent a sensor's response to four different bands of optical wavelengths. The sensor essentially creates one image for each optical band represented by the response curves 501, 502, 503, 504. In this example, a single multi-spectral image would be comprised of images obtained by the sensor using these four spectral bands. Those skilled in the art will appreciate that different sensor systems can have more or fewer optical bands. In contrast, the panchromatic image is a single image which is obtained by a sensor that is responsive to a much wider range of optical wavelengths. In FIG. 5, the response of the panchromatic sensor is illustrated by curve 500.
[0032] In FIG. 5, it can be seen that the response curves 501, 502, 503, 504 of the sensor for the various multi-spectral bands can be very different as compared to the response curve 500 of the panchromatic sensor for the same wavelengths. These differences in the responsiveness of the sensor to the various optical bands will result in scaling differences as between the radiance values associated with each pixel for the multi-spectral image as compared to the panchromatic image. Therefore a calibration function is needed to scale the radiance values for each pixel as measured by the multi-spectral sensor to correspond to the scaling of radiance values resulting from the panchromatic sensor. For example, consider the spectral response represented by curves 500 and 501 at ~500nm. The curve 501 has a spectral response of approximately 1.0 whereas the spectral response of the panchromatic sensor shows an average spectral response in the range of about 0.35. Ignoring for the moment the response of curve 502 in the wavelength range defined by curve 501, the radiance values for pixels in a multi-spectral image using a sensor having the characteristics of response curve 501 would likely need to be scaled by a weighting value of about 0.35 in order for such radiance values to be properly calibrated to those values measured by a sensor having the response indicated by curve 500. In general, proper spectral calibration would require that the pixel radiance values associated with each spectral band in FIG. 5 would need to be added together to obtain a total radiance value that is properly scaled to the radiance values obtained using a sensor having the response defined by curve 500. This process is conceptually illustrated in FIG. 6.
[0033] Mathematically, the process in FIG. 6 can be expressed as follows in equation (1):
where:
PMSI(i,j) is the approximated panchromatic radiance of each spectrally down-sampled pixel;
Wλ are the spectral weights for each of the spectral bands, λ;
M,λ(i,j) is the radiance value for each pixel for each spectral band comprising the multi-spectral image; and
P0 is a constant offset value.
[0034] Once these spectral weights are calibrated, the approximated panchromatic image can be generated in step 408. Afterwards, the panchromatic image received in step 404 and the approximated panchromatic image obtained in step 410 can be compared to determine spectral differences for the image pairs. That is, differences in radiance values. Such a comparison can be performed by aligning the received and generated panchromatic images using the mapping function obtained in step 408, scaling one or both of the images to the same spatial resolution, and subtracting the mapped values for the pixels in the images to determine areas associated with differences in radiance values. Although the approximated panchromatic image, being derived from a combination of multi-spectral images, is based on the lower spatial resolution as the multi-spectral images, the differences in radiance values between the received and approximated panchromatic images should only be minor for non-moving objects. In contrast, the localized differences in radiance values between the received and approximated panchromatic images, due to moving objects, can be significant. Accordingly, one aspect of the present invention provides for identifying the locations of moving objects based on these localized differences in radiance values between the received images.
[0035] Once the differences between the received and approximated panchromatic images are identified in step 410, the method 400 can determine in step 412 if any areas of the first image pair are exhibiting some degree of spectral difference and if the amount of spectral difference obtained in step 410 for any area of image pair is greater than a first threshold value. That is, whether any of the differences are greater than an amount expected by normal variation in intensity between the received and approximated panchromatic images. In some embodiments, the intensities in the received and approximated images can be normalized prior to the comparison in step 410 to reduce variations in radiance values due to overall differences in radiance values.
[0036] In some examples identifying an area in step 412 can require that an area with a spectral difference greater than the first threshold include at least a minimum amount of pixels. Such a configuration limits the amount of subsequent remapping to only objects that would affect the fused image significantly, such as moving objects of a sufficiently large size and/or objects having motion of a sufficient large distance. For example, in such examples, the difference can be required to occur over at least an area of 3 by 3 pixels. Accordingly, in such examples of the present invention, smaller areas are ignored and are assumed to be associated with motion on a smaller scale or motion of smaller objects. Generally, such small scale changes are generally not expected to significantly affect the resulting fused image.
[0037] In general, assuming that moving objects account for a small fraction of the scene being imaged, the root-mean squared (RMS) error of fit between the panchromatic image radiance and the spectrally down-sampled MSI image radiance can be used in some embodiments to set a constant false alarm rate or value for the first threshold corresponding to - 2.0 standard deviations. As a result, because of the nearly identical collection geometry and scene illumination of the panchromatic and MSI images, value of the first threshold can be significantly low, such as -30 digital numbers for 11-bit panchromatic images. That is, the threshold is typically corresponds to a difference in image radiance of between 1% and 2%.
[0038] If the result of step 410 is that no areas of the images are associated with spectral differences above the first threshold limit in step 412 (i.e., all areas of the images are spectrally equivalent), the image pair is fused in step 414 according to conventional methods with no additional processing of the mapping function. The method 400 then ends in step 416 and resumes previous processing. However, if at least one area of the image pair associated with a spectral difference above the first threshold limit is found in step 412, the method 400 can proceed to step 418. In step 418, a first area of one of the images obtained in step 404 and associated with a spectral difference greater than the first threshold value is selected. For ease of illustration, the first image is the panchromatic image obtained in step 404. However, in the various examples of the present invention, either of the images used in step 410 can be used.
[0039] Once the first area from the first image (panchromatic image) is selected in step 418, a candidate area of the second image (the approximated panchromatic image) can be selected in step 420. That is, an area of the second image also associated with a spectral difference greater than the first threshold value, but that is not currently associated with an alternate mapping. The candidate area can be selected in several ways. For example, in some examples the candidate area can be an area of the second image associated with a difference greater than the first threshold value and having a size and shape substantially the same as the first area from the first image. That is, the number and arrangement of pixels (once mapped) in the first and candidate areas are approximately the same.
[0040] In the various embodiments of the present invention, how well the regions in the panchromatic and MSI images match can be evaluated based on finding a mapping of radiances of the regions between images that minimizes differences in radiances between the images. That is, a mapping of pixels within the first region (of the panchromatic image) to pixels in the second region (of the MSI image) that minimizes the RMS radiance error between the panchromatic images and approximated panchromatic image. In general, this mapping can be limited to rigid translation and limited rotation of the object region between the two images. As a result, during this evaluation the similarity of shapes of detected regions of change has little impact. That is, a good match can be obtain when the shape of the object cannot be completely observed, such as a moving object partially obscured in one image as it moves under a tree of bridge. In other words, the value of the match will be determined by how well the visible portions of the object can be mapped between the two images, not necessarily the object as a whole.
[0041] Additionally, the candidate area can be selected based on distance. In general, the moving objects that will be captured in the image data will have velocities between 5 and 150 m/s. Accordingly, for a 0.5 s delay in acquisition, such a moving object would traverse between 2.5 and 75m. In a 1-m resolution image with 8000 by 8000 pixels, such motion translates to less than 15 pixels of displacement. Accordingly, to increase computational efficiency, an assumption that the object in the first and second images will be in relative proximity to each other can be made. Therefore, if several candidate areas of approximately the size and shape are present in the second image, as described above, the relatively closest candidate area can be selected.
[0042] In some examples the candidate area can also be estimated based on an identification of an object and an estimate of the object's motion. For example, if shape recognition software is provided, the object can be classified, for example, as a cloud, an aircraft, a watercraft, or a ground vehicle. Accordingly, more accurate estimates of the velocity of these identified objects can be provided. For example, in the case of an aircraft, aircraft velocities are typically on the order of 75 to 100 m/s. Therefore, for a 0.5-s delay only candidate areas within 35 to 50 m of the first area can be selected. In contrast, ground vehicles typically travel at velocities of 5 to 15 m/s. Accordingly, for a 0.5-s delay the candidate areas are selected from those at 10 m or less. In the various examples of the present invention, the utility of such an approach can be limited based on the image resolution and the amount of delay between images.
[0043] After the candidate area from the second image is selected in step 420, the differences between the first area selected in step 418 and the candidate area can be evaluated in step 422. That is, similar to step 410, the area selected in step 418 and 420 can be scaled to the same resolution, if necessary, and a difference between the areas can be computed. Afterwards, if the overall difference is less than a second threshold value at step 424 (i.e., spectrally equivalent), which can be the same or different that the first threshold value, the method can proceed to step 426. However, if the overall difference is greater than the second threshold value, the method 400 proceeds to step 428. As used herein, the "overall difference" obtained in step 422 can be the aggregate difference between the areas selected in steps 418 and 420. For example, the overall difference can be an average of the difference in radiance values between corresponding pixels in the different areas or a difference between the average radiance values for the different areas.
[0044] In general, there will be a number of areas of change detected between the panchromatic and approximated panchromatic images and a variety of ways that the areas of change can be paired. However, only the pairing of areas between images that can be explained as moving objects in the scene is sought. Typically, a better pairing will minimize the RMS error between the panchromatic and approximated panchromatic images after the mapping between the images is updated to reflect the presumed moving objects. Some areas of change should not be paired because they are not visible in both images, such as the ground under the moving vehicle. Furthermore, a given area of change in one image is generally mapped to only a single object in the other image. However, in some circumstances, some areas in one image can match multiple candidate areas in the other image. Given a choice between such multiple candidate areas, they will be matched with the one that provides the greatest reduction in RMS error between the two images. In at least some examples if no candidate area in the other image for a selected area in the first image provides a reduction in RMS between the two images, the selected area will not be matched or paired with any of the candidate areas in the other image.
[0045] If the overall difference is greater than the second threshold limit in step 424, a different candidate area can be selected in step 428. The selection of the candidate area in step 428 is similar to the selection in step 420. However, in step 428, only candidate areas that have not yet been compared to the first area are selected. After the next candidate area is selected in step 428, steps 422 and 428 can be repeated until a suitable candidate area is found at step 424. Afterwards, the method 400 can proceed to step 426.
[0046] In step 426, the mapping function(s) obtained in step 406 can be modified. In particular, the mapping functions are modified to provide an alternate mapping for the pixels in the first area or the pixels in the selected candidate area and vice versa. Accordingly the modified mapping functions result in pixels from the first area in the first image to map to the pixels in the selected candidate area in the second image and vice versa. Additionally in step 428, a smoothing function is configured for the areas originally corresponding to the candidate area or the first area in the mapping functions. That is, as previously described, based on the assumption that if the object in an image is removed, the obscured surface below the object will have a radiance profile similar to that of the areas surrounding the area containing the object. Accordingly, for areas in a first image which are not associated with a moving object but in which the corresponding area in the second image is associated with a moving object, radiance values are interpolated from the areas of the second image surrounding the moving object rather than the radiance values associated with the moving object. Therefore, the smoothing function is used to identify these areas area associated with non-moving objects and requiring alternate radiance values. For example, in the case of an MSI image enhanced using spatial information from a panchromatic image, the smoothing function would identify areas of the MSI image for which these alternate radiance values need to be calculated. A similar smoothing function can be configured for the panchromatic image being enhanced by the MSI image.
[0047] Once the mapping and smoothing functions are configured in steps 426 and 430, in step 432, if at least a second area of the first image is associated with a difference meeting the criteria in step 412, a next area of the first image can be selected in step 418 and steps 420-430 can be repeated. Consequently, steps 418-432 can be repeated until all areas of the first image are properly mapped. Next, in step 434, the first and second images are fused based on the modified mapping function and smoothing function. Although various fusion processes and methods can be used with the various embodiments of the present invention, an exemplary fusion process is described below with respect to FIG. 7 to further illustrate the use of the modified mapping function and the smoothing function. Method 400 can end in step 416 and resume previous processing.
[0048] FIG. 7 shows flow chart of an exemplary method 700 for fusing a panchromatic and MSI image pair according an embodiment of the present invention. In particular, method 700 illustrated enhancing a MSI image with the higher spatial information of a corresponding panchromatic image. The method 700 can begin in step 702 and continue to step 704. In step 704, a modified mapping function and a smoothing function are received. The modified mapping function and the smoothing function can be generated, for example, as described above with respect to FIG. 4. However, the invention is not limited in this regard and the modified mapping function and the smoothing function can be generated in accordance with the various embodiments of the present invention.
[0049] After the functions are received in step 704, the one or more pixels of the MSI image to be enhanced using the spatial information of the panchromatic image can be selected in step 706. The number of pixels selected can be based on the type of fusion process to be used. In some examples the pixels of the MSI image can be enhanced individually. In other examples, the values for the pixels of the MSI image can be considered interdependent and therefore the fusion process can provide for enhancing the pixels collectively. Once the pixel(s) of the MSI image are selected in step 706, the corresponding pixels of the panchromatic image can be selected in step 708 based on the modified mapping function received in step 704. As previously described, the mapping function comprises a mapping function modified to account for moving objects, such as that generated in FIG. 4.
[0050] Alternatively or in conjunction with step 708, the method 700 also determines in step 710 the pixel(s) selected in step 706 are associated with an area for which alternate radiance values need to be calculated. As previously described, the smoothing function for the MSI image specifies that the actual values of the panchromatic image are not to be used during the fusion process. Rather, such values are estimated or calculated, as described herein. Therefore, if the selected pixels for the MSI image are associated with one of the areas specified in the smoothing function at step 712, the radiance values from the panchromatic image to be using for enhancing the selected pixels of the MSI image can be calculated at step 714.
[0051] In step 714, the assumption is made, as previously described, that the surface obscured by a moving object in the MSI image will have approximately the same radiance values as pixels surrounding the object. Accordingly, at step 714, the radiance values corresponding to the pixels of the MSI can be estimated from the radiance values along a border region for the area specified by the smoothing function. That is, in the fused image, these border regions will be handled as if these pixels were voided out in the source imagery. For example, the pixels in the background obscured by a moving vehicle in the MSI image will generally not be available to provide accurate colorization of the panchromatic pixels in the same region. In such cases, the color can be interpolated from the adjacent pixels of the background not obscured by the vehicle in the MSI image. Similarly, the background obscured by the vehicle in the panchromatic image will generally not have the high spatial resolution of the panchromatic image to properly texture the background of the MSI image, resulting in this patch of the fused product being limited to the MSI resolution instead. Once the radiance values are calculated in step 714, the radiance values can be used in step 716 to enhance the selected pixels of the MSI image.
[0052] Alternatively, if the selected pixels for the MSI image are not associated with one of the areas specified in the smoothing function at step 712, the values from the panchromatic image to be used for enhancing the selected pixels of the panchromatic image can calculated in step 718 from the pixels selected in step 708. In particular, the radiance value for each of the MSI pixels for each of the MSI bands can be interpolated from the corresponding pixels selected in step 708 according to the modified mapping function. Once the radiance values are calculated in step 718, the radiance values can be used in step 716 to enhance the selected pixels of the MSI image.
[0053] Regardless of how the radiance values for step 716 are calculated, the method 700 can continue to step 720 and determine if other pixels of the MSI image still need to be enhanced. If additional pixels of the MSI image still require enhancement at step 720, the method can be repeated for these additional pixel(s) starting at step 706. Otherwise, the method 700 can end at step 722 and resume previous processing.
[0054] The illustrations of embodiments described herein are intended to provide a general understanding of the structure of various embodiments, and they are not intended to serve as a complete description of all the elements and features of apparatus and systems that might make use of the structures described herein. Many other embodiments will be apparent to those of skill in the art upon reviewing the above description. Other embodiments can be utilized and derived therefrom, such that structural and logical substitutions and changes can be made without departing from the scope of the appended claims. Figures are also merely representational and can not be drawn to scale. Certain proportions thereof may be exaggerated, while others may be minimized. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.
receiving a first set of imagery data and a first set of associated metadata identifying geo-spatial information for the first set of imagery data (404), said first set of imagery data defining a first image of a panchromatic image type, said first image having a first spatial resolution and a first spectral resolution;
receiving a second set of imagery data and a second set of associated metadata identifying geo-spatial information for the second set of imagery data (404), said second set of imagery data defining a second image of a multi-spectral image type, said second image having a second spatial resolution less than said first spatial resolution and a second spectral resolution greater than said first spectral resolution;
obtaining at least a first mapping function specifying a mapping between pixels in said first image and pixels in said second image based on at least said first and said second sets of associated metadata (406);
generating a third set of imagery data defining a third image of a panchromatic type based on said second set of imagery data, said third image having said first spatial resolution and said first spectral resolution (408);
identifying_at least a first area of said pixels in said first image and at least a first area of said pixels in said third image that are non-corresponding according to said first mapping function (410, 412, 418, 420); characterized by
generating an alternate mapping function for said first and second sets of imagery data specifying a mapping exclusively for pixels of said first area that are non-corresponding according to said first mapping function;
obtaining at least a second mapping function based on said first mapping function and said alternate mapping function; and
combining said first and said second sets of imagery data according to said second mapping function to generate a fourth set of imagery data, said fourth set of imagery data having said first spatial resolution and said second spectral resolution (434), wherein said generating said fourth set of imagery data further comprises:
computing radiance values for a first area of said fourth image corresponding to said first area of said first image based on radiance values for said first area of said second image specified in said second mapping function; and
using a smoothing function to compute radiance values for a second area of said fourth image corresponding to said first area of said second image based on radiance values for a plurality of pixels in said second image surrounding said first area of said second image specified in said second mapping function.
selecting said first area of said first image based on a difference between said first and said third images;
selecting at least one candidate area of said third image, said candidate area of said third image spectrally different than a corresponding area of said first image according to said first mapping function;
comparing said first area of said first image to said candidate area of said third image; and
providing said alternate mapping if said first area of said first image and said candidate area of said third image are spectrally equivalent, said alternate mapping associating said first area of said first image and an area of said second image corresponding to said candidate area of said third image.
identifying a plurality of candidate areas in said third image;
selecting one of said plurality of candidate areas in said third image said first area of said first image resulting a minimum root-mean square (RMS) difference.
a storage element for receiving a first set of imagery data and a first set of associated metadata identifying geo-spatial information for the first set of imagery data and a second set of imagery data and a second set of associated metadata identifying geo-spatial information for the second set of imagery data, said first set of imagery data defining a first image of a panchromatic image type and having a first spatial resolution and a first spectral resolution, and said second set of imagery data defining a second image of a multi-spectral image type and having a second spatial resolution less than said first spatial resolution and a second spectral resolution greater than said first spectral resolution; and
a processing element communicatively coupled to said storage element, said processing element configured for:
obtaining at least a first mapping function specifying a mapping between pixels in said first image and pixels in said second image based on at least said first and said second sets of associated metadata (406),
generating a third set of imagery data defining a third image of a panchromatic type based on said second set of imagery data, said third image having said first spatial resolution and said first spectral resolution (408), and
identifying at least a first area of said pixels in said first image and at least a first area of said pixels in said third image that are non-corresponding according to said first mapping function (410, 412, 418, 420); characterized by
generating an alternate mapping function for said first and second sets of imagery data specifying a mapping exclusively for said pixels of said first area that are non-corresponding according to said first mapping function;
obtaining at least a second mapping function based on said first mapping function and said alternate mapping function; and
combining said first and said second sets of imagery data according to said second mapping function to generate a fourth set of imagery data, said fourth set of imagery data having said first spatial resolution and said second spectral resolution, wherein said processing element is further configured during said generating said fourth set of imagery data for:
computing radiance values for a first area of said fourth image corresponding to said first area of said second image based on radiance values for said first area of said first image specified in said second mapping function; and
using a smoothing function to compute radiance values for a second area of said fourth image corresponding to said first area of said first image based on radiance values for a plurality of pixels in said first image surrounding said first area of said first image specified in said second mapping function.
selecting said first area of said first image based on a difference between said first and said third images;
selecting at least one candidate area of said third image, said candidate area of said third image spectrally different than a corresponding area of said first image according to said first mapping function;
comparing said first area of said first image to said candidate area of said third image; and
providing said alternate mapping function if said first area of said first image and said candidate area of said third image are spectrally equivalent, said alternate mapping function associating said first area of said first image and an area of said second image corresponding to said candidate area of said third image.
identifying a plurality of candidate areas in said third image;
selecting at least one of said plurality of candidate areas in said third image having a size and shape substantially equal to a size and shape of said first area of said first image.
Empfangen einer ersten Gruppe von Bilddaten und einer ersten Gruppe zugehöriger Metadaten, die geo-räumliche Informationen für die erste Gruppe von Bilddaten (404) bestimmen, wobei die erste Gruppe von Bilddaten ein erstes Bild eines farbempfindlichen Bildtyps definiert, wobei das erste Bild eine erste räumliche Auflösung und eine erste Spektralauflösung aufweist;
Empfangen einer zweiten Gruppe von Bilddaten und einer zweiten Gruppe zugehöriger Metadaten, die geo-räumliche Informationen für die zweite Gruppe von Bilddaten (404) bestimmen, wobei die zweite Gruppe von Bilddaten ein zweites Bild eines multispektralen Bildtyps definiert, wobei das zweite Bild eine zweite räumliche Auflösung kleiner als die erste räumliche Auflösung und eine zweite Spektralauflösung höher als die erste Spektralauflösung aufweist;
Erhalten wenigstens einer ersten Abgleichungsfunktion, die eine Abgleichung zwischen Bildpunkten im ersten Bild und Bildpunkten im zweiten Bild basierend auf wenigstens der ersten und zweiten Gruppe zugehöriger Metadaten (406) angibt;
Generieren einer dritten Gruppe von Bilddaten, die ein drittes Bild eines farbempfindlichen Typs basierend auf der zweiten Gruppe von Bilddaten definiert, wobei das dritte Bild über eine erste räumliche Auflösung und eine erste Spektralauflösung (408) verfügt;
Bestimmen von wenigstens einem ersten Bereich der Bildpunkte im ersten Bild und von wenigstens einem ersten Bereich der Bildpunkte im dritten Bild, die gemäß der ersten Abgleichungsfunktion (410, 412, 418, 420) nicht übereinstimmend sind; gekennzeichnet durch
Generieren einer alternativen Abgleichungsfunktion für die erste und zweite Gruppe der Bilddaten, die eine Abgleichung ausschließlich für Bildpunkte des ersten Bereichs angibt, die gemäß der ersten Abgleichungsfunktion nicht übereinstimmend sind;
Erhalten von wenigstens einer zweiten Abgleichungsfunktion basierend auf der ersten Abgleichungsfunktion und der alternativen Abgleichungsfunktion; und
Kombinieren der ersten und zweiten Gruppe von Bilddaten gemäß der zweiten Abgleichungsfunktion, um eine vierte Gruppe von Bilddaten zu generieren, wobei die vierte Gruppe von Bilddaten die erste räumliche Auflösung und die zweite Spektralauflösung (434) aufweist, wobei das Generieren der vierten Gruppe von Bilddaten weiterhin umfasst:
Berechnen von Strahldichtewerten für einen ersten Bereich des vierten Bildes, der einem ersten Bereich des ersten Bildes entspricht, basierend auf Strahldichtewerten für den in der zweiten Abgleichungsfunktion angegebenen ersten Bereich des zweiten Bildes; und
Verwenden einer Glättungsfunktion zum Berechnen der Strahldichtewerte für einen zweiten Bereich des vierten Bildes, der dem ersten Bereich des zweiten Bildes entspricht, basierend auf Strahldichtewerten für eine Vielzahl an Bildpunkten im zweiten Bild, die den in der zweiten Abgleichungsfunktion angegebenen ersten Bereich des zweiten Bildes umgeben.
Auswählen des ersten Bereichs des ersten Bildes basierend auf einem Unterschied zwischen dem ersten und dem dritten Bild;
Auswählen von wenigstens einem Kandidatenbereich des dritten Bildes, wobei der Kandidatenbereich des dritten Bildes sich spektral von einem entsprechenden Bereich des ersten Bildes gemäß der ersten Abgleichungsfunktion unterscheidet;
Vergleichen des ersten Bereichs des ersten Bildes mit dem Kandidatenbereich des dritten Bildes; und
Vorsehen einer alternativen Abgleichung, wenn der erste Bereich des ersten Bildes und der Kandidatenbereich des dritten Bildes spektral äquivalent sind, wobei die alternative Abgleichung den ersten Bereich des ersten Bildes und einen Bereich des zweiten Bildes verknüpft, der dem Kandidatenbereich des dritten Bildes entspricht.
Bestimmen einer Vielzahl von Kandidatenbereichen im dritten Bild;
Auswählen von einem der Vielzahl der Kandidatenbereiche im dritten Bild, wobei der erste Bereich des ersten Bildes zu einer kleinsten quadratischen Mittelwert (RMS, Root-Mean Square)-Differenz führt.
ein Speicherelement zum Empfangen einer ersten Gruppe von Bilddaten und einer ersten Gruppe von zugehörigen Metadaten, die geo-räumliche Informationen für die erste Gruppe von Bilddaten bestimmen, und einer zweiten Gruppe von Bilddaten und einer zweiten Gruppe von zugehörigen Metadaten, die geo-räumliche Informationen für die zweite Gruppe von Bilddaten bestimmen, wobei die erste Gruppe von Bilddaten ein erstes Bild eines farbempfindlichen Bildtyps definiert und eine erste räumliche Auflösung und eine erste Spektralauflösung aufweist, und die zweite Gruppe an Bilddaten ein zweites Bild eines multispektralen Bildtyps definiert und eine zweite räumliche Auflösung kleiner als die erste räumliche Auflösung und eine zweite spektrale Auflösung höher als die erste spektrale Auflösung aufweist; und
ein Verarbeitungselement, das austauschfähig mit dem Speicherelement gekoppelt ist, wobei das Verarbeitungselement ausgebildet ist zum:
Erhalten von wenigstens einer ersten Abgleichungsfunktion, die eine Abgleichung zwischen Bildpunkten im ersten Bild und Bildpunkten im zweiten Bild basierend auf wenigstens der ersten und zweiten Gruppe zugehöriger Metadaten (406) angibt;
Generieren einer dritten Gruppe von Bilddaten, die ein drittes Bild eines farbempfindlichen Typs basierend auf der zweiten Gruppe von Bilddaten definiert, wobei das dritte Bild über eine erste räumliche Auflösung und eine erste Spektralauflösung (408) verfügt; und
Bestimmen von wenigstens einem ersten Bereich der Bildpunkte im ersten Bild und von wenigstens einem ersten Bereich der Bildpunkte im dritten Bild, die gemäß der ersten Abgleichungsfunktion (410, 412, 418, 420) nicht übereinstimmend sind; gekennzeichnet durch
Generieren einer alternativen Abgleichungsfunktion für die erste und zweite Gruppe von Bilddaten, die eine Abgleichung ausschließlich für Bildpunkte des ersten Bereichs angeben, die gemäß der ersten Abgleichungsfunktion nicht übereinstimmend sind;
Erhalten von wenigstens einer zweiten Abgleichungsfunktion basierend auf der ersten Abgleichungsfunktion und der alternativen Abgleichungsfunktion; und
Kombinieren der ersten und zweiten Gruppe von Bilddaten gemäß der zweiten Abgleichungsfunktion, um eine vierten Gruppe von Bilddaten zu generieren, wobei die vierten Gruppe von Bilddaten eine erste räumliche Auflösung und eine zweite Spektralauflösung aufweist, wobei das Verarbeitungselement beim Generieren der vierten Gruppe an Bilddaten weiterhin ausgebildet ist zum:
Berechnen von Strahldichtewerten für einen ersten Bereich des vierten Bildes, der dem ersten Bereich des zweiten Bildes basierend auf Strahldichtewerten für den in der zweiten Abgleichungsfunktion angegebenen ersten Bereich des ersten Bildes entspricht; und
Verwenden einer Glättungsfunktion zum Berechnen der Strahldichtewerte für einen zweiten Bereich des vierten Bildes, der dem ersten Bereich des ersten Bildes entspricht, basierend auf Strahldichtewerten für eine Vielzahl an Bildpunkten in dem ersten Bild, die den in der zweiten Abgleichungsfunktion angegebenen ersten Bereich des ersten Bildes umgeben.
Auswählen des ersten Bereichs des ersten Bildes basierend auf einer Differenz zwischen dem ersten und dem dritten Bild;
Auswählen von wenigstens einem Kandidatenbereich des dritten Bildes, wobei der Kandidatenbereich des dritten Bildes sich spektral von einem entsprechenden Bereich des ersten Bildes gemäß der ersten Abgleichungsfunktion unterscheidet;
Vergleichen des ersten Bereichs des ersten Bildes mit dem Kandidatenbereich des dritten Bildes; und
Vorsehen einer alternativen Abgleichungsfunktion, wenn der erste Bereich des ersten Bildes und der Kandidatenbereich des dritten Bildes spektral äquivalent sind, wobei die alternative Abgleichungsfunktion den ersten Bereich des ersten Bildes und einen Bereich des zweiten Bildes, der dem Kandidatenbereich des dritten Bildes entspricht, verknüpft.
Bestimmen einer Vielzahl von Kandidatenbereichen im dritten Bild;
Auswählen von wenigstens einem der Vielzahl der Kandidatenbereiche im dritten Bild, der eine Größe und Form aufweist, die im Wesentlichen gleich der Größe und Form des ersten Bereichs des ersten Bildes ist.
recevoir un premier ensemble de données d'imagerie et un premier ensemble de métadonnées associées identifiant des informations géospatiales pour le premier ensemble de données d'imagerie (404), ledit premier ensemble de données d'imagerie définissant une première image d'un type d'image panchromatique, ladite première image présentant une première résolution spatiale et une première résolution spectrale ;
recevoir un deuxième ensemble de données d'imagerie et un deuxième ensemble de métadonnées associées identifiant des informations géospatiales pour le deuxième ensemble de données d'imagerie (404), ledit deuxième ensemble de données d'imagerie définissant une deuxième image d'un type d'image multispectrale, ladite deuxième image présentant une deuxième résolution spatiale inférieure à ladite première résolution spatiale et une deuxième résolution spectrale supérieure à ladite première résolution spectrale ;
obtenir au moins une première fonction de mise en correspondance spécifiant une mise en correspondance entre les pixels dans ladite première image et les pixels dans ladite deuxième image sur la base au moins dudit premier et dudit deuxième ensembles de métadonnées associées (406) ;
générer un troisième ensemble de données d'imagerie définissant une troisième image d'un type panchromatique sur la base dudit deuxième ensemble de données d'imagerie, ladite troisième image présentant ladite première résolution spatiale et ladite première résolution spectrale (408) ;
identifier au moins une première zone desdits pixels dans ladite première image et au moins une première zone desdits pixels dans ladite troisième image qui ne sont pas en correspondance conformément à ladite première fonction de mise en correspondance (410, 412, 418, 420) ; caractérisé par les étapes consistant à
générer une fonction de mise en correspondance alternative pour lesdits premier et deuxième ensembles de données d'imagerie spécifiant une mise en correspondance exclusivement pour les pixels de ladite première zone qui ne sont pas en correspondance conformément à ladite première fonction de mise en correspondance ;
obtenir au moins une deuxième fonction de mise en correspondance sur la base de ladite première fonction de mise en correspondance et de ladite fonction de mise en correspondance alternative ; et
combiner ledit premier et ledit deuxième ensembles de données d'imagerie conformément à ladite deuxième fonction de mise en correspondance pour générer un quatrième ensemble de données d'imagerie, ledit quatrième ensemble de données d'imagerie présentant ladite première résolution spatiale et ladite deuxième résolution spectrale (434), dans lequel ladite étape consistant à générer ledit quatrième ensemble de données d'imagerie comprend en outre les étapes consistant à:
calculer des valeurs de luminance énergétique pour une première zone de ladite quatrième image correspondant à ladite première zone de ladite première image sur la base des valeurs de luminance énergétique pour ladite première zone de ladite deuxième image spécifiée dans ladite deuxième fonction de mise en correspondance ; et
utiliser une fonction de lissage pour calculer les valeurs de luminance énergétique pour une deuxième zone de ladite quatrième image correspondant à ladite première zone de ladite deuxième image sur la base des valeurs de luminance énergétique pour une pluralité de pixels dans ladite deuxième image entourant ladite première zone de ladite deuxième image spécifiée dans ladite deuxième fonction de mise en correspondance.
sélectionner ladite première zone de ladite première image sur la base d'une différence entre ladite première et ladite troisième images ;
sélectionner au moins une zone candidate de ladite troisième image, ladite zone candidate de ladite troisième image étant spectralement différente d'une zone correspondante de ladite première image conformément à ladite première fonction de mise en correspondance ;
comparer ladite première zone de ladite première image avec ladite zone candidate de ladite troisième image ; et
prévoir ladite mise en correspondance alternative si ladite première zone de ladite première image et ladite zone candidate de ladite troisième image sont spectralement équivalentes, ladite mise en correspondance alternative associant ladite première zone de ladite première image et une zone de ladite deuxième image correspondant à ladite zone candidate de ladite troisième image.
identifier une pluralité de zones candidates dans ladite troisième image ;
sélectionner l'une parmi ladite pluralité de zones candidates dans ladite troisième image, ladite première zone de ladite première image résultant d'une différence de moyenne quadratique (RMS) minimale.
un élément de mémorisation destiné à recevoir un premier ensemble de données d'imagerie et un premier ensemble de métadonnées associées identifiant des informations géospatiales pour le premier ensemble de données d'imagerie et un deuxième ensemble de données d'imagerie et un deuxième ensemble de métadonnées associées identifiant des informations géospatiales pour le deuxième ensemble de données d'imagerie, ledit premier ensemble de données d'imagerie définissant une première image d'un type d'image panchromatique et présentant une première résolution spatiale et une première résolution spectrale, et ledit deuxième ensemble de données d'imagerie définissant une deuxième image d'un type d'image multispectrale et présentant une deuxième résolution spatiale inférieure à ladite première résolution spatiale et une deuxième résolution spectrale supérieure à ladite première résolution spectrale ; et
un élément de traitement couplé en communication audit élément de mémorisation, ledit élément de traitement étant configuré pour :
obtenir au moins une première fonction de mise en correspondance spécifiant une mise en correspondance entre les pixels dans ladite première image et les pixels dans ladite deuxième image sur la base au moins dudit premier et dudit deuxième ensembles de métadonnées associées (406),
générer un troisième ensemble de données d'imagerie définissant une troisième image d'un type panchromatique sur la base du deuxième ensemble de données d'imagerie, ladite troisième image présentant ladite première résolution spatiale et ladite première résolution spectrale (408), et
identifier au moins une première zone desdits pixels dans ladite première image et au moins une première zone desdits pixels dans ladite troisième image qui ne sont pas en correspondance conformément à ladite première fonction de mise en correspondance (410, 412, 418, 420) ; caractérisé par les étapes consistant à
générer une fonction de mise en correspondance alternative pour lesdits premier et deuxième ensembles de données d'imagerie spécifiant une mise en correspondance exclusivement pour lesdits pixels de ladite première zone qui ne sont pas en correspondance conformément à ladite première fonction de mise en correspondance ;
obtenir au moins une deuxième fonction de mise en correspondance sur la base de ladite première fonction de mise en correspondance et de ladite fonction de mise en correspondance alternative ; et
combiner ledit premier et ledit deuxième ensembles de données d'imagerie conformément à ladite deuxième fonction de mise en correspondance pour générer un quatrième ensemble de données d'imagerie, ledit quatrième ensemble de données d'imagerie présentant ladite première résolution spatiale et ladite deuxième résolution spectrale, dans lequel ledit élément de traitement est en outre configuré pendant ladite étape consistant à générer ledit quatrième ensemble de données d'imagerie pour :
calculer les valeurs de luminance énergétique pour une première zone de ladite quatrième image correspondant à ladite première zone de ladite deuxième image sur la base des valeurs de luminance énergétique pour ladite première zone de ladite première image spécifiée dans ladite deuxième fonction de mise en correspondance ; et
utiliser une fonction de lissage pour calculer les valeurs de luminance énergétique pour une deuxième zone de ladite quatrième image correspondant à ladite première zone de ladite première image sur la base des valeurs de luminance énergétique pour une pluralité de pixels dans ladite première image entourant ladite première zone de ladite première image spécifiée dans ladite deuxième fonction de mise en correspondance.
sélectionner ladite première zone de ladite première image sur la base d'une différence entre ladite première et ladite troisième images ;
sélectionner au moins une zone candidate de ladite troisième image, ladite zone candidate de ladite troisième image étant spectralement différente d'une zone correspondante de ladite première image conformément à ladite première fonction de mise en correspondance ;
comparer ladite première zone de ladite première image à ladite zone candidate de ladite troisième image ; et
prévoir ladite fonction de mise en correspondance alternative si ladite première zone de ladite première image et ladite zone candidate de ladite troisième image sont spectralement équivalentes, ladite fonction de mise en correspondance alternative associant ladite première zone de ladite première image et une zone de ladite deuxième image correspondant à ladite zone candidate de ladite troisième image.
identifier une pluralité de zones candidates dans ladite troisième image ;
sélectionner au moins l'une parmi ladite pluralité de zones candidates dans ladite troisième image présentant une taille et une forme sensiblement égales à une taille et une forme de ladite première zone de ladite première image.
REFERENCES CITED IN THE DESCRIPTION
This list of references cited by the applicant is for the reader's convenience only. It does not form part of the European patent document. Even though great care has been taken in compiling the references, errors or omissions cannot be excluded and the EPO disclaims all liability in this regard.
Patent documents cited in the description