(19)
(11)EP 2 180 695 B1

(12)EUROPEAN PATENT SPECIFICATION

(45)Mention of the grant of the patent:
09.09.2020 Bulletin 2020/37

(21)Application number: 09173385.7

(22)Date of filing:  19.10.2009
(51)International Patent Classification (IPC): 
H04N 5/44(2011.01)
H04N 7/01(2006.01)

(54)

Apparatus and method for improving frame rate using motion trajectory

Vorrichtung und Verfahren zur Verbesserung der Bildfrequenz mittels der Bewegungsbahn

Appareil et procédé pour améliorer la fréquence d'image en utilisant la trajectoire du mouvement


(84)Designated Contracting States:
AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK SM TR

(30)Priority: 23.10.2008 KR 20080103994

(43)Date of publication of application:
28.04.2010 Bulletin 2010/17

(73)Proprietor: Samsung Electronics Co., Ltd.
Suwon-si, Gyeonggi-do, 443-742 (KR)

(72)Inventors:
  • Cho, Yang Ho
    Gyeonggi-do 445-982 (KR)
  • Lee, Ho Young
    Suwon-si Gyeonggi-do 443-768 (KR)
  • Park, Du-Sik
    Suwon-si Gyeonggi-do 443-470 (KR)

(74)Representative: Grootscholten, Johannes A.M. et al
Arnold & Siedsma Bezuidenhoutseweg 57
2594 AC The Hague
2594 AC The Hague (NL)


(56)References cited: : 
JP-A- 2005 318 548
JP-A- 2007 334 625
JP-A- 2007 282 155
  
      
    Note: Within nine months from the publication of the mention of the grant of the European patent, any person may give notice to the European Patent Office of opposition to the European patent granted. Notice of opposition shall be filed in a written reasoned statement. It shall not be deemed to have been filed until the opposition fee has been paid. (Art. 99(1) European Patent Convention).


    Description


    [0001] The invention relates to an apparatus and a method for improving a frame rate of an image, and more particularly, to a frame improving apparatus and method that may effectively generate an intermediate frame using a motion trajectory according to a feature point in a frame.

    [0002] When an image with a low frame rate is replayed by a device with a high frame rate, motion between frames is abnormally perceived. Also, in the case of an image device using an LCD, deterioration of image quality occurs in a video clip having high-speed motion, caused by a limited response speed due to the physical limitations of the LCD.

    [0003] Accordingly, a method that generates an intermediate frame between frames using a motion vector determined by estimating motion to improve a frame rate is required. However, in a conventional motion vector estimated in frame units, there is a low probability that the motion vector is accurately estimated in all frames due to limitations of a motion estimating algorithm, lack of used frame information, and lack of information for determining an accurate motion vector, and thus deterioration of the image quality may occur.

    [0004] Therefore, a frame rate improving method for preventing deterioration of image quality from an image and for increasing confidence with respect to the motion information is required.

    [0005] JP 2007 282155 A discloses a method for frame rate conversion, by correlating motion vectors to as many pixels of an interpolated frame as possible in order to interpolate regions for which motion vectors of a forward direction cannot be interpolated.

    [0006] JP 2007 334625 A discloses a method for detecting moving vectors from an image with high accuracy by applying camera shake correction processing by correcting deviations in the distribution of feature points that are traced to detect a moving vector.

    [0007] The apparatus of the invention has the features of claim 1. The apparatus includes a feature point trajectory information determiner to determine forward feature point trajectory information by tracing a feature point of a frame, a motion estimator to perform a block-based backward motion estimation with respect to a current frame based on whether a feature point traced for each block of the current frame exists, and a frame generator to generate a new intermediate frame between the current frame and a previous frame using a motion vector determined according to the backward motion estimation.

    [0008] The feature point trajectory information determiner may additionally extract a feature point when any one of a case when a scene is changed between frames, a case when a number of feature points, each having a trajectory being traced, is less than a predetermined number of feature points, and a case when a number of feature points being extracted from a predetermined area of divided areas constituting a frame is less than another predetermined number of feature points occurs.

    [0009] An occlusion area detector may detect an occlusion area between frames using a Sum of Absolute Differences (SAD) of a block and the forward feature point trajectory information when the feature point is traced forward with respect to the plurality of frames.

    [0010] Also, the motion estimator performs a backward motion estimation for a current block using the feature point trajectory information of the feature point when the traced feature point exists in the current block for which the motion estimation is performed in the current frame.

    [0011] Also, the motion estimator performs a backward motion estimation for a current block using a motion vector of a block adjacent to the current block when the traced feature point does not exist in the current block for which the motion estimation is performed in the current frame.

    [0012] The method of the invention has the features of claim 9. The method includes determining forward feature point trajectory information by tracing a feature point of a frame, performing a block-based backward motion estimation with respect to the current frame based on whether a feature point traced for each block of the current frame exists, and generating a new intermediate frame between a current frame and a previous frame using a motion vector determined according to the backward motion estimation.

    [0013] Additional aspects, features, and/or advantages will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the disclosure.

    [0014] These and/or other aspects, features, and advantages will become apparent and more readily appreciated from the following description, taken in conjunction with the accompanying drawings of which:

    FIG. 1 is a block diagram illustrating a frame rate improving apparatus using a motion trajectory according to the invention;

    FIG. 2 illustrates an example of tracing a forward feature point with respect to a plurality of frames according to the invention;

    FIG. 3 illustrates an example of additionally extracting a feature point according to the invention;

    FIGS. 4A and 4B illustrate a process of determining of a motion vector of a current block by performing a backward motion estimation according to the invention;

    FIG. 5 illustrates a process of generating of an intermediate frame using a motion vector of a current block according to the invention;

    FIGS. 6A and 6B illustrates a process of detecting of a covered area of an occlusion area between frames according to the invention;

    FIGS. 7A and 7B illustrate a process of detecting of an uncovered area of an occlusion area between frames according to the invention;

    FIG. 8 is a flowchart illustrating a frame rate improving method using a motion trajectory according to an example embodiment; and

    FIG. 9 is a flowchart illustrating a process of additionally extracting of a feature point according to an example embodiment.



    [0015] Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. Example embodiments are described below to explain the present disclosure by referring to the figures.

    [0016] Note that the example embodiments of the figures relate to features which may be used in embodiments falling within the scope of the invention.

    [0017] A frame rate improving method according to an example embodiment may be implemented by a frame rate improving apparatus.

    [0018] FIG. 1 is a block diagram illustrating a frame rate improving apparatus using a motion trajectory according to an example embodiment.

    [0019] Referring to FIG. 1, the frame rate improving apparatus 100 includes a feature point trajectory information determiner 101, a motion estimator 102, a frame generator 103, and an occlusion area detector 104.

    [0020] The feature point trajectory information determiner 101 determines forward feature point trajectory information by tracing a feature point of a frame. The feature point trajectory information determiner (101) successively traces a feature point extracted from an initial frame with respect to a plurality of forward frames and determine feature point trajectory information of the corresponding feature point. The initial frame may not indicate a first frame among frames constituting an image, but may indicate a frame where a feature point initially appears.

    [0021] For example, the feature point trajectory information determiner 101 may extract a feature point from a frame 0, which is an initial frame, and may successively ascertain a location of a feature point for each frame 1, frame 2, and frame 3 appearing later. Then, the feature point trajectory information determiner 101 may determine feature trajectory information by connecting the ascertained feature points between frames. In detail, the feature point trajectory information is about a motion trajectory representing a direction in which a specific area including a feature point moves.

    [0022] In this instance, the feature point may include a point which is extracted from a boundary, a corner, or a texture showing a high frequency feature within a local area of an image. Also, at least one feature point may exist in a frame.

    [0023] When a predetermined circumstance occurs, the feature point trajectory information determiner 101 may additionally extract a feature point to maintain confidence with respect to the feature point trajectory information. Subsequently, the feature point trajectory information determiner 101 may determine feature point trajectory information with respect to the additionally extracted feature point.

    [0024] As an example, when a scene is changed between frames, the feature point information determiner 101 additionally extracts a feature point with respect to a frame where the scene is changed. Subsequently, when a number of feature points, each having a trajectory being traced, is less than a predetermined number of feature points, the feature point trajectory information determiner 101 additionally extracts the feature point. Also, when a number of feature points being extracted from a predetermined area of divided areas constituting of a frame is less than another predetermined number of feature points, the feature point trajectory information determiner 101 additionally extracts the feature point.

    [0025] Detailed examples of additional extracting of the feature point will be described with reference to FIG. 3 and FIG. 9.

    [0026] The motion estimator 102 performs a block-based backward motion estimation with respect to a current frame based on whether a feature point traced for each block of the current frame exists. In this instance, the current frame indicates a frame for which the motion is traced from among a plurality of frames constituting an image. The motion estimator 102 may estimate the motion based on a block as opposed to all pixels constituting the frame, considering an amount of operation and an efficiency of an algorithm.

    [0027] As an example, when the traced feature point exists in a current block of the current frame, the motion estimator 102 performs the backward motion estimation with respect to the current block using feature point trajectory information of the feature point. That is, the feature point trajectory information may be used for estimating a motion as a reference with high confidence.

    [0028] Also, when the traced feature point does not exist in the current block of the current frame, the motion estimator 102 performs the backward motion estimation with respect to the current block using a motion vector of a block adjacent to the current block. The motion estimator 102 may guarantee an accuracy of motion estimation by using the motion vector of the adjacent block having a high correlation with the current block.

    [0029] A process of estimating the motion with respect to the current block will be described with reference to FIGS. 4A and 4B in detail.

    [0030] The frame generator 103 generates a new intermediate frame between the current frame and a previous frame using a motion vector determined according to the motion estimation. As a number of the intermediate frames increase, a frame rate may increase. As an example, the frame generator103 may generate the intermediate frame by averaging blocks located in the previous frame and the current frame according to a direction of the motion vector.

    [0031] The occlusion area detector 104 may detect an occlusion area between frames using a Sum of Absolute Differences (SAD) and forward feature point trajectory information when the feature point is traced forward with respect to the plurality of frames. In this instance, the SAD indicates a block matching error, and small SAD indicates small change in the motion of the block.

    [0032] The occlusion area detector 104 determines an area in which a feature point extracted from the previous frame is not extracted from the current frame and the SAD of the block is equal to or greater than a predetermined SAD as a covered area of the occlusion area. Also, the occlusion area detector 104 determines an area in which a feature point that is not extracted from the previous frame is extracted from the current frame and the SAD of the block is equal to or greater than the predetermined SAD as an uncovered area of the occlusion area.

    [0033] Therefore, the occlusion area detector 104 determines an area where feature point trajectory information may not be determined since a corresponding feature point between the frames does not exist, as the occlusion area. A process of determining of the occlusion area will be described with reference to FIGS. 6A and 6B, FIGS 7A and 7B.

    [0034] FIG. 2 illustrates an example of tracing a forward feature point with respect to a plurality of frames according to an example embodiment.

    [0035] Referring to FIG. 2, four frames are illustrated, and an image sequentially includes a frame (0) 201, a frame (1) 202, a frame (2) 203, and a frame (3) 204.

    [0036] As illustrated in FIG. 2, a feature point trajectory information determiner 101 may extract a feature point from the frame (0) 201. The frame (0) 201 indicates an initial frame. In this instance, at least one feature point may be extracted. That is, there is at least one feature point trajectory information with respect to the feature point. As an example, the feature point may include a point which is extracted from a boundary, a corner, or a texture showing a high frequency within a local area of the image.

    [0037] The feature point trajectory information determiner 101 may determine the feature point trajectory information of the feature point by extracting a feature point from a temporally subsequent frame, a frame (1) 202, matched with the feature point extracted from the frame (0) 201. In the same manner, the feature point trajectory information determiner 101 may extract the same feature point from the frame (2) 203 and frame (3) 204, and successively connect the extracted feature point to determine feature point trajectory information.

    [0038] In this instance, to guarantee confidence of the feature point trajectory information, the feature point trajectory information determiner 101 may determine the feature trajectory information based on a feature point successively traced with respect to at least three frames. A number of the frames may not be limited.

    [0039] Therefore, according to an example embodiment, the feature point trajectory information determiner 101 may accurately estimate a motion vector based on tracing of a motion trajectory of a block with respect to the plurality of frames, compared with when estimating the motion vector based on tracing of the motion trajectory of the block with respect to two frames.

    [0040] FIG. 3 illustrates an example of additionally extracting a feature point according to an example embodiment.

    [0041] When a plurality of frames pass after extracting a feature point from a frame (0) 201, the initial frame illustrated in FIG. 2, feature points corresponding to the frames may not be extracted due to a change in a pixel brightness, a sudden motion, and an occlusion area, and thus feature point trajectory information may decrease. Also, the feature point extracted from the initial frame may need to be updated during trajectory tracing, due to a scene change between frames and a new object. Also, to generate an accurate intermediate frame, various feature point trajectory information of the feature point may be required.

    [0042] As an example, the feature point trajectory information determiner 101 additionally extracts a feature point when any one of following cases occurs.
    1. (1) When a scene is changed between frames:
      In this instance, the scene change may be recognized when there is a change in a brightness distribution chart (histogram) of an image or when an SAD of a block excessively increases. When the scene is changed, a corresponding feature point between the frames may not exist, and thus, the feature point trajectory information determiner 101 may additionally extract an additional feature point.
    2. (2) When a number of feature points, each having a trajectory being traced, is less than a predetermined number of feature points:
      When the number of the estimated feature points is less than the predetermined number of feature points (TH1), the feature point trajectory information determiner 101 may additionally extract an additional feature point to use feature point trajectory information with high confidence.
      In a graph illustrated in FIG. 3, an X-axis indicates a temporal axis of an image (frame number) and Y-axis indicates spatial coordinates within a corresponding frame. When five feature points are extracted from the frame 0, the feature point trajectory information determiner 101 may trace five feature point trajectories for each frame. As illustrated in FIG. 3, when the number of the predetermined number of feature points (TH1) is five and the number of the feature points traced in a frame (2) 203 is four, the additional feature point may need to be additionally extracted from the frame (2) 203.
    3. (3) When a number of feature points extracted from a predetermined area of divided areas is less than a predetermined number of feature points (TH2):
      In the case that the frame consists of a plurality of the divided areas, when the number of the feature points extracted from the predetermined area of the divided areas is less than the predetermined number of feature points (TH2), the feature point trajectory information determiner 101 may additionally extract an additional feature point to use feature point trajectory information with high confidence. That is, the feature point trajectory information may evenly exist in an entire area to perform accurate motion estimate and to cope with a new object.


    [0043] As illustrated in FIG. 3, since a number of feature point trajectory information is one in a lower area of a frame (3) 204 when the predetermined number of feature points (TH2) is two, the feature point trajectory information determiner 101 may additionally extract the additional feature point.

    [0044] FIGS. 4A and 4B illustrate a process of determining of a motion vector of a current block by performing a backward motion estimation according to an example embodiment.

    [0045] A motion estimator 102 performs a block-based backward motion estimation with respect to the current frame based on whether a feature point is traced for each block of the current frame. In this instance, the current block indicates a block for which a motion vector is determined through motion estimation from among blocks included in the current frame. Also, an adjacent block indicates a block directly adjacent to the current block, the adjacent block already having a motion vector being determined through motion estimation.

    [0046] As an example, the motion estimator 102 may perform the backward motion estimation with respect to the current block using feature point trajectory information of the feature point when the traced feature point exists in the current block of the current frame.

    [0047] FIG. 4A illustrates a process of the performing of the backward motion estimation with respect to the current block using the feature point trajectory information.

    [0048] Referring to FIG. 4A, a feature point exist in a block 406, the current block, of a frame (3) 404, the current frame. Subsequently, the motion estimator 102 may perform a backward motion estimation considering feature point trajectory information of the feature point. That is, the motion estimator 102 may determine a motion vector (MV) of the current block 406 using the feature point trajectory information, the information indicating trajectory information from a feature point corresponding to a frame (3) 404 to a feature point corresponding to a previous frame (2) 403. In other words, the motion estimator102 may determine the motion vector as a vector from the current block 405 of the frame (2) 403 to a dotted line block to be corresponding to the feature point trajectory information.

    [0049] When the traced feature point exists in the current block of the current frame, the motion estimator 102 performs the backward motion estimation with respect to the current block using the motion vector of a block adjacent to the current block.

    [0050] FIG. 4B illustrates a process of the performing of the backward motion estimation with respect to the current block using the motion vector of the adjacent block. The motion vector 102 may determine a motion vector with respect to a block 407, the current block, of a frame 3 402, the current frame, using motion vectors (MV1, MV2, MV3) of adjacent blocks 408, 409, and 410 for each of which motion estimation is already performed. In this instance, the motion estimator 102 may determine a motion vector having a highest correlation with the current block from among the motion vectors of the adjacent blocks 408, 409, and 410 as the motion vector of the current block 407.

    [0051] That is, when the feature point exists in the current block for which the motion vector is determined, the motion estimator 102 may determine the motion vector of the current block using feature trajectory information with high confidence, thereby improving an accuracy of the motion estimation. However, when the feature point does not exist in the current block, the motion estimator 102 may improve the accuracy of the motion estimation by determining the motion vector of the current block using the motion vector of the adjacent block having the high correlation with the current block.

    [0052] FIG. 5 illustrates a process of generating of an intermediate frame using a motion vector of a current block according to an example embodiment.

    [0053] In this instance, a frame generator 103 generates a new intermediate frame by interpolating between a previous frame 501 and a current frame 503. Referring to FIG. 5, when a motion vector (MV) with respect to a current block 505 of the current frame 503 is determined by tracing a motion, the frame generator 103 may generate the intermediate frame 502 by averaging blocks located between the previous frame 501 and the current frame 503 considering a direction of the motion vector. A process of generating of the intermediate frame 502 is performed with respect to all blocks included in the current frame 503.

    [0054] As an example, when an occlusion area exists between the previous frame 501 and the current frame 503, the frame generator 103 generates the intermediate frame 502 using one frame from among the previous frame 501 or the current frame 503 depending on a type of the occlusion area. Specifically, when the occlusion area is a covered area, the frame generator 103 may generate the intermediate frame 502 using the previous frame 501. Also, when the occlusion area is an uncovered area, the frame generator 1 generates the intermediate frame 502 using the current frame 503.

    [0055] A process of detecting the occlusion area and generating the intermediate frame with respect to the occlusion area will be described with reference to FIGS. 6A and. 6B and FIGS. 7A and 7B.

    [0056] FIGS. 6A and 6B illustrates a process of detecting of a covered area of an occlusion area between frames according to an example embodiment.

    [0057] An occlusion area detector 104 may detect an occlusion area between frames using an SAD of a block and the forward feature point trajectory information when a feature point is traced forward with respect to a plurality of frames. The occlusion area detector 104 determines an area in which a feature point extracted from a previous frame is not extracted from a current frame and the SAD of the block is equal to or greater than a predetermined SAD as a covered area of the occlusion area. In the case of the occlusion area, generally, the SAD value may be great since corresponding points between frames do not exist.

    [0058] Referring to FIG. 6A, a trajectory of a feature point is traced as a center of a "circle" existing in a previous frame 601. However, in a current frame 602, the feature point is no longer traced since the "circle" moves behind a "square". That is, the occlusion area exists between the previous frame 601 and the current frame 602. In this instance, the occlusion area detector 104 may determine the "circle" as the covered area of the occlusion area. For reference, the "square" may be determined as the uncovered area.

    [0059] In this instance, since deterioration of image quality may occur when a frame with respect to an area determined as the covered area is generated using the previous frame and the current frame, the frame generator 103 may generate the frame with respect to the area determined as the covered area only referring to the previous frame.

    [0060] That is, referring to a FIG. 6B, when a trajectory of a soccer ball appearing in a t frame corresponding to the previous frame 601, is not traced in a t+1 frame corresponding to the current frame 602, the occlusion area detector 104 may determine the soccer ball as the covered area. Subsequently, the frame generator 103 may generate the soccer ball in a t + 0.5 frame corresponding to the intermediate frame, only referring to a motion vector of the soccer ball that appears in the t frame, the previous frame.

    [0061] FIGS. 7A and 7B illustrate a process of detecting of an uncovered area of an occlusion area between frames according to an example embodiment

    [0062] An occlusion area detector 104 determines an area in which a feature point that is not extracted from a previous frame is extracted from a current frame and the SAD of the block is equal to or greater than a predetermined SAD as an uncovered area of the occlusion area.

    [0063] Referring to FIG. 7A, a "square" does not exist in a previous frame 701, however, appears in a current frame 702, and thus, a new feature point is extracted. That is, the current frame 702 where the "square" appears may be an initial frame where a feature point is extracted. That is, the occlusion area exists between the previous frame 701 and the current frame 702. In this instance, the occlusion area detector 104 may determine the "square" of the occlusion area as the uncovered area.

    [0064] In this instance, since deterioration of image quality may occur when a frame with respect to an area determined as the uncovered area is generated using the previous frame and the current frame, the frame generator 103 may generate the frame with respect to the area determined as the uncovered area only referring to the current frame.

    [0065] That is, referring to FIG. 7B, when a soccer ball that does not appear in a t frame corresponding to the previous frame 701, suddenly appears in a t + 1 frame corresponding to the current frame 702, and a trajectory of the soccer ball is traced, the occlusion area detector 104 may determine the soccer ball as the uncovered area. Subsequently, the frame generator 103 may generate the soccer ball in a t + 0.5 frame corresponding to the intermediate frame, only referring to a motion vector of the soccer ball that appears in the t + 1 frame corresponding to the current frame 702.

    [0066] FIG. 8 is a flowchart illustrating a frame rate improving method using a motion trajectory according to an example embodiment.

    [0067] In operation S801, a frame rate improving apparatus may extract a feature point from an initial frame. In this instance, the feature point may include a point which is extracted from a boundary, a corner, or a texture showing high frequency within a local area of an image.

    [0068] In operation S802, the frame rate improving apparatus may determine feature point trajectory information of the feature point by successively tracing the feature point extracted from the initial frame with respect to a plurality of forward frames. That is, the frame rate improving apparatus may determine forward feature point trajectory information by tracing a feature point of a frame.

    [0069] As an example, when the successively traced trajectory of the feature point is discontinued in a specific frame, the frame rate improving apparatus may detect a corresponding area as a covered area of an occlusion area. Also, when a new feature point is determined due to a sudden appearance of a new object in the specific frame, the frame rate improving apparatus may detect an area including the determined feature point as an uncovered area of the occlusion area.

    [0070] In operation S803, the frame rate improving apparatus may perform motion estimation based on a result of verifying whether the traced feature point exists in the present block. In this instance, the present block may indicate a block included in the present frame for which motion estimation is performed.

    [0071] When the traced feature point does not exist in the present block, the frame rate improving apparatus may perform backward motion estimation with respect to the present block using a motion vector of a block adjacent to the present block in operation S804. In this instance, the adjacent block may indicate a block already having a motion vector being determined through motion estimation. Also, the backward motion estimation may indicate a motion vector with respect to the present block from the present frame to the previous frame.

    [0072] When the traced feature point exists in the present block, the frame rate improving apparatus may perform backward motion estimation with respect to the present block using feature point trajectory information of the feature point in operation S805. That is, since the feature point trajectory information determined through a plurality of frames may indicate information with high confidence used for determining the motion vector of the present block, accuracy of the motion estimation may increase.

    [0073] When the motion vector with respect to the present block is determined, the frame rate improving apparatus may generate a new intermediate frame between the present frame and the previous frame using the motion vector determined according to the motion estimation.

    [0074] In this instance, the frame rate improver may generate the intermediate frame with respect to the covered area of the occlusion area using the motion vector of the previous frame. Also, the frame rate improving apparatus may generate the intermediate frame with respect to the uncovered area of the occlusion area using the motion vector of the present frame.

    [0075] Omitted descriptions related to FIG. 8 may be described with reference to FIGS. 1 to 7B.

    [0076] FIG. 9 is a flowchart illustrating a process of additionally extracting a feature point according to an example embodiment.

    [0077] The frame rate improving apparatus may extract a feature point from a feature point, and determine feature point trajectory information by successively tracing the feature point with respect to a plurality of forward frames. In this instance, the frame rate improving apparatus may additionally extract a feature point through a process illustrated in FIG. 9. The frame rate improving apparatus may determine feature point trajectory information by successively tracing the additionally extracted feature point with respect to the plurality of frames.

    [0078] In operation S901, the frame rate improving apparatus may determine whether a frame where the feature point is extracted is an initial frame (frame (0)). When the frame where the feature point is extracted is the initial frame, the frame rate improving apparatus may extract the feature point and proceed to a subsequent frame in operation S903.

    [0079] When the frame where the feature point is extracted is not the initial frame, the frame rate improving apparatus may determine whether a scene change between frames is sensed in operation S902. When the scene change is sensed, the frame rate improving apparatus may extract the feature point and proceed to a subsequent frame in operation S903.

    [0080] Also, when the frame where the feature point is extracted is not the initial frame and also a scene change is not sensed, the frame rate improving apparatus may determine feature point trajectory information using an extracted feature point, and proceed to a subsequent frame in operation S904.

    [0081] In operation S905, the frame rate improving apparatus may determine whether a number of feature points extracted from a specific frame is less than a predetermined number of feature points (TH1). When the number of the extracted feature points is less than the predetermined number of feature points (TH1), the frame rate improving apparatus may additionally extract a feature point in operation S907.

    [0082] When the number of extracted feature points is equal to or greater than the predetermined number of the feature points (TH1), the frame rate improving apparatus may determine whether a number of feature points extracted from a predetermined area is less than a predetermined number of feature points (TH2). When the number of the feature points extracted from the predetermined area is less than the predetermined number of the feature points (TH2), the frame rate improving apparatus may additionally extract a feature point in operation S907.

    [0083] Also, when the number of feature points extracted from the predetermined area is equal to or greater than the predetermined number of the feature points (TH2), the frame rate improving apparatus may determine whether the frame where the feature point is extracted is a last frame in operation S908. When the frame is the last frame, the frame rate improving apparatus may end the process of determining of the feature point trajectory information. Also, when the frame is not the last frame, the frame rate improving apparatus may determine the feature point trajectory information by extracting a feature point with respect to a frame from operation S901.

    [0084] According to an example embodiment, there may be provided a frame rate improving apparatus and method that may successively trace a trajectory forward, and perform backward motion estimation through feature point trajectory information, thereby increasing an accuracy of motion estimation.

    [0085] According to an example embodiment, there may be provided a frame rate improving apparatus and method that may perform backward motion estimation through a motion vector of an adjacent block having a high correlation with a present block when the feature point does not exist in the present block, thereby increasing an accuracy of motion estimation.

    [0086] According to an example embodiment, there may be provided a frame rate improving apparatus and method that may additionally extract a feature point to use a feature point even with respect to entire area of frame, thereby increasing an accuracy of motion estimation.

    [0087] According to an example embodiment, there may be provided a frame rate improving apparatus and method that may detect an occlusion area where a trajectory of the feature point is not traced and may use a different frame generating method according to a kind of the occlusion area, thereby preventing deterioration of image quality from a frame to be generated.

    [0088] Also, the frame rate improving method according to the above-described example embodiments may be recorded in computer-readable media including program instructions to implement various operations embodied by a processor or a computer. The media may also include, alone or in combination with the program instructions, data files, data structures, and the like. Examples of computer-readable media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD ROM disks and DVDs; magnetooptical media such as optical disks; and hardware devices that are specially configured to store and perform program instructions, such as read-only memory (ROM), random access memory (RAM), flash memory, and the like. Examples of program instructions include both machine code, such as produced by a compiler, and files containing higher level code that may be executed by the computer using an interpreter. The described hardware devices may be configured to act as one or more software modules in order to perform the operations of the above-described example embodiments, or vice versa.

    [0089] Although a few example embodiments have been shown and described, the present disclosure is not limited to the described example embodiments. Instead, it would be appreciated by those skilled in the art that changes may be made to these example embodiments without departing from the scope of the invention which is defined by the claims and their equivalents.


    Claims

    1. An apparatus, including a processor (100), for improving a frame rate using a motion trajectory comprising:

    a feature point trajectory information determiner (101) adapted to determine forward feature point trajectory information by tracing a feature point of a frame;

    a motion estimator (102) adapted to perform a block-based backward motion estimation with respect to a current frame based on whether a feature point traced for each block of the current frame exists;

    a frame generator (103) adapted to generate a new intermediate frame between the current frame and a previous frame using a motion vector determined according to the backward motion estimation; and

    an occlusion area detector (104) adapted to detect an occlusion area between frames using the forward feature point trajectory information when the feature point is traced forward with respect to the plurality of frames,

    wherein the occlusion area is an area where feature point trajectory information may not be determined since a corresponding feature point between the frames does not exist and a type of the occlusion area includes a covered area and an uncovered area,

    wherein the frame generator (103) generates the new intermediate frame using one frame from among the previous frame or the current frame depending on the type of occlusion area.


     
    2. The apparatus of claim 1, wherein the feature point trajectory information determiner (101) is adapted to successively trace a feature point extracted from an initial frame with respect to a plurality of forward frames and to determine feature point trajectory information of the feature point.
     
    3. The apparatus of claim 1 or 2, wherein the feature point trajectory information determiner (101) is adapted to additionally extract a feature point when any one of: a case when a scene is changed between frames, a case when a number of feature points, each having a trajectory being traced, is less than a predetermined number of feature points, and a case when a number of feature points being extracted from a predetermined area of divided areas constituting a frame is less than another predetermined number of feature points occurs.
     
    4. The apparatus of claim 1, wherein the occlusion area detector is adapted to:

    determine an area in which a feature point extracted from the previous frame is not extracted from the current frame and a Sum of Absolute Differences (SAD) of the block is equal to or greater than a predetermined SAD as a covered area of the occlusion area, and

    determine an area in which a feature point that is not extracted from the previous frame is extracted from the current frame and the SAD of the block is equal to or greater than the predetermined SAD as an uncovered area of the occlusion area.


     
    5. The apparatus of claim 4, wherein the frame generator (103) is adapted to:
    generate the intermediate frame with respect to the covered area using a motion vector of the previous frame, and generate the intermediate frame with respect to the uncovered area using a motion vector of the current frame.
     
    6. The apparatus of any of the previous claims, wherein the motion estimator (102) is adapted to perform a backward motion estimation for a current block using the feature point trajectory information of the feature point when the traced feature point exists in the current block for which the motion estimation is performed in the current frame.
     
    7. The apparatus of any of the previous claims, wherein the motion estimator (102) is adapted to perform a backward motion estimation for a current block using a motion vector of a block adjacent to the current block when the traced feature point does not exist in the current block for which the motion estimation is performed in the current frame.
     
    8. The apparatus of any of the previous claims, wherein the frame generator (103) is adapted to generate the intermediate frame by averaging blocks located in the previous frame and current frame according to a direction of the motion vector.
     
    9. A method for improving a frame rate using a motion trajectory, the method comprising:

    determining forward feature point trajectory information (S802) by tracing a feature point of a frame;

    detecting an occlusion area between frames (S803) using the forward feature point trajectory information when the feature point is traced forward with respect to the plurality of frames;

    performing a block-based backward motion estimation (S805) with respect to the current frame based on whether a feature point traced for each block of the current frame exists; and

    generating a new intermediate frame (S806) between a current frame and a previous frame using a motion vector determined according to the backward motion estimation,

    wherein the occlusion area is an area where feature point trajectory information may not be determined since a corresponding feature point between the frames does not exist and a type of the occlusion area includes a covered area and an uncovered area,

    wherein the generating of a new intermediate frame (S806) includes generating the new intermediate frame using one frame from among the previous frame or the current frame depending on the type of the occlusion area.


     
    10. The method of claim 9, wherein the determining of the forward feature point trajectory information additionally extracts the feature point (S903, S907) when any one of: a case when a scene is change between frames, a case when a number of feature points, each having a trajectory being traced, is less than a predetermined number of feature points, and a case when a number of feature points being extracted from a predetermined area of divided areas constituting a frame is less than another predetermined number of feature points occurs.
     
    11. The method of claim 9, wherein the detecting of the occlusion area between the frame comprises:

    determining an area in which a feature point extracted from the previous frame is not extracted from the current frame and a Sum of Absolute Differences (SAD) of a block is equal to or greater than a predetermined SAD as a covered area of the occlusion area, and

    determining an area in which a feature point that is not extracted from the previous frame is extracted from the current frame and the SAD of the block is equal to or greater than the predetermined SAD as an uncovered area of the occlusion area;

    wherein the generating of the intermediate frame preferably comprises:

    generating the intermediate frame with respect to the covered area using a motion vector of the previous frame, and

    generating the intermediate frame with respect to the uncovered area using a motion vector of the current frame.


     
    12. The method of any of the claims 9-11, wherein the performing of the block-based backward motion estimation performs a backward motion estimation for a current block using the feature point trajectory information of the feature point when the traced feature point exists in the current block for which the motion estimation is performed in the current frame, and performs a backward motion estimation for a current block using a motion vector of a block adjacent to the current block when the traced feature point does not exist in the current block for which the motion estimation is performed in the current frame.
     
    13. Computer program product with processing instructions for performing the method of any of the claims 9-12.
     


    Ansprüche

    1. Einen Prozessor (100) enthaltende Vorrichtung zur Verbesserung einer Bildfrequenz mittels einer Bewegungsbahn, wobei die Vorrichtung Folgendes umfasst:

    einen Merkmalspunkt-Bewegungsbahninformationenbestimmer (101), der dazu angepasst ist, Vorwärtsmerkmalspunkt-Bewegungsbahninformationen durch Verfolgen eines Merkmalspunktes eines Einzelbildes zu bestimmen;

    einen Bewegungsschätzer (102), der dazu angepasst ist, eine blockbasierte Rückwärtsbewegungsschätzung in Bezug auf ein aktuelles Einzelbild durchzuführen, und zwar basierend darauf, ob ein für jeden Block des aktuellen Einzelbildes verfolgter Merkmalspunkt existiert;

    einen Einzelbildgenerator (103), der dazu angepasst ist, ein neues Zwischenbild zwischen dem aktuellen Einzelbild und einem vorherigen Einzelbild zu erzeugen, und zwar unter Verwendung eines Bewegungsvektors, der gemäß der Rückwärtsbewegungsschätzung bestimmt wird; und einen Okklusionsbereichsdetektor (104), der dazu angepasst ist, einen Okklusionsbereich zwischen Einzelbildern unter Verwendung der Vorwärtsmerkmalspunkt-Bewegungsbahninformationen zu erfassen, wenn der Merkmalspunkt in Bezug auf die Vielzahl von Einzelbildern vorwärts verfolgt wird,

    wobei es sich bei dem Okklusionsbereich um einen Bereich handelt, in dem Merkmalspunkt-Bewegungsbahninformationen nicht bestimmt werden dürfen, da ein entsprechender Merkmalspunkt zwischen den Einzelbildern nicht existiert und eine Art des Okklusionsbereichs einen abgedeckten Bereich und einen nicht abgedeckten Bereich umfasst,

    wobei der Einzelbildgenerator (103) in Abhängigkeit von der Art des Okklusionsbereichs das neue Zwischenbild unter Verwendung eines Einzelbildes aus dem vorherigen Einzelbild oder dem aktuellen Einzelbild erzeugt.


     
    2. Vorrichtung nach Anspruch 1, wobei der Merkmalspunkt-Bewegungsbahninformationenbestimmer (101) dazu angepasst ist, fortlaufend einen Merkmalspunkt zu verfolgen, der aus einem ursprünglichen Einzelbild in Bezug auf eine Vielzahl von Vorwärtseinzelbildern extrahiert wurde, und Merkmalspunkt-Bewegungsbahninformationen des Merkmalspunkts zu bestimmen.
     
    3. Vorrichtung nach Anspruch 1 oder 2, wobei der Merkmalspunkt-Bewegungsbahninformationenbestimmer (101) dazu angepasst ist, in einem der folgenden Fälle zusätzlich einen Merkmalspunkt zu extrahieren: in einem Fall, in dem eine Szene sich zwischen den Einzelbildern ändert, in einem Fall, in dem eine Anzahl von Merkmalspunkten, deren Bewegungsbahn jeweils verfolgt wird, geringer als eine vorbestimmte Anzahl von Merkmalspunkten ist, und in einem Fall, in dem eine Anzahl von Merkmalspunkten, die aus einem vorbestimmten Bereich von unterteilten Bereichen extrahiert werden, die ein Einzelbild bilden, geringer als eine weitere vorbestimmte Anzahl von Merkmalspunkten ist.
     
    4. Vorrichtung nach Anspruch 1, wobei der Okklusionsbereichsdetektor zu Folgendem angepasst ist:

    Bestimmen eines Bereichs, in dem ein aus dem vorherigen Einzelbild extrahierter Merkmalspunkt nicht aus dem aktuellen Einzelbild extrahiert wird und eine Summe der absoluten Differenzen (SAD) des Blocks größer oder gleich einer vorbestimmten SAD ist, als abgedeckten Bereich des Okklusionsbereichs, und

    Bestimmen eines Bereichs, in dem ein nicht aus dem vorherigen Einzelbild extrahierter Merkmalspunkt aus dem aktuellen Einzelbild extrahiert wird und die SAD des Blocks größer oder gleich der vorbestimmten SAD ist, als einen nicht abgedeckten Bereich des Okklusionsbereichs.


     
    5. Vorrichtung nach Anspruch 4, wobei der Einzelbildgenerator (103) zu Folgendem angepasst ist: Erzeugen des Zwischenbildes in Bezug auf den abgedeckten Bereich unter Verwendung eines Bewegungsvektors des vorherigen Einzelbildes, und Erzeugen des Zwischenbildes in Bezug auf den nicht abgedeckten Bereich unter Verwendung eines Bewegungsvektors des aktuellen Einzelbildes.
     
    6. Vorrichtung nach einem der vorhergehenden Ansprüche, wobei der Bewegungsschätzer (102) dazu angepasst ist, eine Rückwärtsbewegungsschätzung für einen aktuellen Block unter Verwendung der Merkmalspunkt-Bewegungsbahninformationen des Merkmalspunkts durchzuführen, wenn der verfolgte Merkmalspunkt in dem aktuellen Block existiert, für den die Bewegungsschätzung in dem aktuellen Einzelbild durchgeführt wird.
     
    7. Vorrichtung nach einem der vorhergehenden Ansprüche, wobei der Bewegungsschätzer (102) dazu angepasst ist, eine Rückwärtsbewegungsschätzung für einen aktuellen Block unter Verwendung eines Bewegungsvektors eines Blocks, der an den aktuellen Block angrenzt, durchzuführen, wenn der verfolgte Merkmalspunkt nicht in dem aktuellen Block existiert, für den die Bewegungsschätzung in dem aktuellen Einzelbild durchgeführt wird.
     
    8. Vorrichtung nach einem der vorhergehenden Ansprüche, wobei der Einzelbildgenerator (103) dazu angepasst ist, durch Mitteln von Blöcken, die sich in dem vorherigen Einzelbild und aktuellen Einzelbild befinden, gemäß einer Richtung des Bewegungsvektors das Zwischenbild zu erzeugen.
     
    9. Verfahren zur Verbesserung einer Bildfrequenz unter Verwendung einer Bewegungsbahn, wobei das Verfahren Folgendes umfasst:

    Bestimmen von Vorwärtsmerkmalspunkt-Bewegungsbahninformationen (S802) durch Verfolgen eines Merkmalspunkts eines Einzelbildes;

    Erfassen eines Okklusionsbereichs zwischen Einzelbildern (S803) unter Verwendung der Vorwärtsmerkmalspunkt-Bewegungsbahninformationen, wenn der Merkmalspunkt in Bezug auf die Vielzahl von Einzelbildern vorwärts verfolgt wird,

    Durchführen einer blockbasierten Rückwärtsbewegungsschätzung (S805) in Bezug auf das aktuelle Einzelbild basierend darauf, ob ein für jeden Block des aktuellen Einzelbildes verfolgter Merkmalspunkt existiert; und

    Erzeugen eines neuen Zwischenbildes (S806) zwischen einem aktuellen Einzelbild und einem vorherigen Einzelbild unter Verwendung eines Bewegungsvektors, der gemäß der Rückwärtsbewegungsschätzung bestimmt wird,

    wobei es sich bei dem Okklusionsbereich um einen Bereich handelt, in dem Merkmalspunkt-Bewegungsbahninformationen nicht bestimmt werden dürfen, da ein entsprechender Merkmalspunkt zwischen den Einzelbildern nicht existiert und eine Art des Okklusionsbereichs einen abgedeckten Bereich und einen nicht abgedeckten Bereich umfasst,

    wobei das Erzeugen eines neuen Zwischenbildes (S806) das Erzeugen des neuen Zwischenbildes unter Verwendung eines Einzelbildes aus dem vorherigen Einzelbild oder dem aktuellen Einzelbild in Abhängigkeit von der Art des Okklusionsbereichs umfasst.


     
    10. Verfahren nach Anspruch 9, wobei in einem der folgenden Fälle beim Bestimmen der Vorwärtsmerkmalspunkt-Bewegungsbahninformationen zusätzlich der Merkmalspunkt (S903, S907) extrahiert wird: in einem Fall, in dem eine Szene sich zwischen den Einzelbildern ändert, in einem Fall, in dem eine Anzahl von Merkmalspunkten, deren Bewegungsbahn jeweils verfolgt wird, geringer als eine vorbestimmte Anzahl von Merkmalspunkten ist, und in einem Fall, in dem eine Anzahl von Merkmalspunkten, die aus einem vorbestimmten Bereich von unterteilten Bereichen, die ein Einzelbild bilden, geringer als eine weitere vorbestimmte Anzahl von Merkmalspunkten ist.
     
    11. Verfahren nach Anspruch 9, wobei das Erkennen des Okklusionsbereichs zwischen den Einzelbildern Folgendes umfasst:

    Bestimmen eines Bereichs, in dem ein aus dem vorherigen Einzelbild extrahierter Merkmalspunkt nicht aus dem aktuellen Einzelbild extrahiert wird und eine Summe der absoluten Differenzen (SAD) eines Blocks größer oder gleich einer vorbestimmten SAD ist, als einen abgedeckten Bereich des Okklusionsbereichs, und

    Bestimmen eines Bereichs, in dem ein nicht aus dem vorherigen Einzelbild extrahierter Merkmalspunkt aus dem aktuellen Einzelbild extrahiert wird und die SAD des Blocks größer oder gleich der vorbestimmten SAD ist, als einen nicht abgedeckten Bereich des Okklusionsbereichs;

    wobei das Erzeugen des Zwischenbildes vorzugsweise Folgendes umfasst:

    Erzeugen des Zwischenbildes in Bezug auf den abgedeckten Bereich unter Verwendung eines Bewegungsvektors des vorherigen Einzelbildes, und

    Erzeugen des Zwischenbildes in Bezug auf den nicht abgedeckten Bereich unter Verwendung eines Bewegungsvektors des aktuellen Einzelbildes.


     
    12. Verfahren nach einem der Ansprüche 9 bis 11, wobei beim Durchführen der blockbasierten Rückwärtsbewegungsschätzung für einen aktuellen Block eine Rückwärtsbewegungsschätzung für einen aktuellen Block unter Verwendung der Merkmalspunkt-Bewegungsbahninformationen des Merkmalspunkts durchgeführt wird, wenn der verfolgte Merkmalspunkt in dem aktuellen Block existiert, für den die Bewegungsschätzung in dem aktuellen Einzelbild durchgeführt wird, und eine Rückwärtsbewegungsschätzung für einen aktuellen Block unter Verwendung eines Bewegungsvektors eines Blocks, der an den aktuellen Block angrenzt, durchgeführt wird, wenn der verfolgte Merkmalspunkt nicht in dem aktuellen Block existiert, für den die Bewegungsschätzung in dem aktuellen Einzelbild durchgeführt wird.
     
    13. Computerprogrammprodukt mit Verarbeitungsanweisungen zur Durchführung des Verfahrens nach einem der Ansprüche 9 bis 12.
     


    Revendications

    1. Dispositif, comportant un processeur (100), visant à améliorer la fréquence d'images au moyen d'une trajectoire de mouvement comprenant :

    un déterminateur d'informations de trajectoire de point caractéristique (101) apte à déterminer des informations de trajectoire de point caractéristique avant par suivi d'un point caractéristique d'une trame ;

    un estimateur de mouvement (102) apte à réaliser une estimation de mouvement arrière bloc par bloc eu égard à une trame actuelle compte tenu de l'existence ou non d'un point caractéristique suivi pour chaque bloc de la trame actuelle ;

    un générateur de trame (103) apte à générer une nouvelle trame intermédiaire entre la trame actuelle et une trame précédente au moyen d'un vecteur de mouvement déterminé conformément à l'estimation de mouvement arrière ; et

    un détecteur de zone d'occlusion (104) apte à détecter une zone d'occlusion entre trames au moyen des informations de trajectoire de point caractéristique avant lorsque le point caractéristique est suivi en avant eu égard à la pluralité de trames ;

    ladite zone d'occlusion étant une zone dans laquelle des informations de trajectoire de point caractéristique peuvent ne pas être déterminées du fait de la non-existence d'un point caractéristique correspondant entre les trames, et le type de la zone d'occlusion comprenant une zone couverte et une zone non couverte,

    ledit générateur de trame (103) générant la nouvelle trame intermédiaire au moyen d'une trame parmi la trame précédente et la trame actuelle en fonction du type de la zone d'occlusion.


     
    2. Dispositif selon la revendication 1, dans lequel le déterminateur d'informations de trajectoire de point caractéristique (101) est apte à suivre de façon successive un point caractéristique extrait d'une trame initiale eu égard à une pluralité de trames avant et à déterminer des informations de trajectoire de point caractéristique propres au point caractéristique.
     
    3. Dispositif selon la revendication 1 ou 2, dans lequel le déterminateur d'informations de trajectoire de point caractéristique (101) est apte à extraire en outre un point caractéristique lorsqu'il se produit l'un des cas suivants : un cas dans lequel une scène change entre les trames, un cas dans lequel le nombre de points caractéristiques, ayant chacun une trajectoire suivie, est inférieur à un nombre prédéterminé de points caractéristiques, et un cas dans lequel le nombre de points caractéristiques extraits d'une zone prédéterminée de zones divisées constituant une trame est inférieur à un autre nombre prédéterminé de points caractéristiques.
     
    4. Dispositif selon la revendication 1, dans lequel le détecteur de zone d'occlusion est apte à :

    déterminer une zone dans laquelle un point caractéristique extrait de la trame précédente n'est pas extrait de la trame actuelle et dans laquelle la somme des différences absolues (SAD, sum of absolute différences) du bloc est supérieure ou égale à une SAD prédéterminée, comme zone couverte de la zone d'occlusion, et

    déterminer une zone dans laquelle un point caractéristique qui n'est pas extrait de la trame précédente est extrait de la trame actuelle et dans laquelle la SAD du bloc est supérieure ou égale à la SAD prédéterminée, comme zone non couverte de la zone d'occlusion.


     
    5. Dispositif selon la revendication 4, dans lequel le générateur de trame (103) est apte à :
    générer la trame intermédiaire eu égard à la zone couverte au moyen d'un vecteur de mouvement de la trame précédente, et générer la trame intermédiaire eu égard à la zone non couverte au moyen d'un vecteur de mouvement de la trame actuelle.
     
    6. Dispositif selon l'une quelconque des revendications précédentes, dans lequel l'estimateur de mouvement (102) est apte à réaliser une estimation de mouvement arrière relativement à un bloc actuel au moyen des informations de trajectoire de point caractéristique propres au point caractéristique lorsque le point caractéristique suivi existe dans le bloc actuel relativement auquel l'estimation de mouvement est réalisée dans la trame actuelle.
     
    7. Dispositif selon l'une quelconque des revendications précédentes, dans lequel l'estimateur de mouvement (102) est apte à réaliser une estimation de mouvement arrière relativement à un bloc actuel au moyen d'un vecteur de mouvement propre à un bloc adjacent au bloc actuel lorsque le point caractéristique suivi n'existe pas dans le bloc actuel relativement auquel l'estimation de mouvement est réalisée dans la trame actuelle.
     
    8. Dispositif selon l'une quelconque des revendications précédentes, dans lequel le générateur de trame (103) est apte à générer la trame intermédiaire en moyennant les blocs situés dans la trame précédente et dans la trame actuelle conformément à la direction du vecteur de mouvement.
     
    9. Procédé visant à améliorer la fréquence d'images au moyen d'une trajectoire de mouvement, le procédé comprenant :

    la détermination d'informations de trajectoire de point caractéristique avant (S802) par suivi d'un point caractéristique d'une trame ;

    la détection d'une zone d'occlusion entre trames (S803) au moyen des informations de trajectoire de point caractéristique avant lorsque le point caractéristique est suivi en avant eu égard à la pluralité de trames ;

    la réalisation d'une estimation de mouvement arrière bloc par bloc (S805) eu égard à la trame actuelle compte tenu de l'existence ou non d'un point caractéristique suivi pour chaque bloc de la trame actuelle ; et

    la génération d'une nouvelle trame intermédiaire (S806) entre une trame actuelle et une trame précédente au moyen d'un vecteur de mouvement déterminé conformément à l'estimation de mouvement arrière ;

    ladite zone d'occlusion étant une zone dans laquelle des informations de trajectoire de point caractéristique peuvent ne pas être déterminées du fait de la non-existence d'un point caractéristique correspondant entre les trames, et le type de la zone d'occlusion comprenant une zone couverte et une zone non couverte,

    ladite génération d'une nouvelle trame intermédiaire (S806) comprenant la génération de la nouvelle trame intermédiaire au moyen d'une trame parmi la trame précédente et la trame actuelle en fonction du type de la zone d'occlusion.


     
    10. Procédé selon la revendication 9, dans lequel la détermination des informations de trajectoire de point caractéristique avant entraîne en outre l'extraction du point caractéristique (S903, S907) lorsqu'il se produit l'un des cas suivants : un cas dans lequel une scène change entre les trames, un cas dans lequel le nombre de points caractéristiques, ayant chacun une trajectoire suivie, est inférieur à un nombre prédéterminé de points caractéristiques, et un cas dans lequel le nombre de points caractéristiques extraits d'une zone prédéterminée de zones divisées constituant une trame est inférieur à un autre nombre prédéterminé de points caractéristiques.
     
    11. Procédé selon la revendication 9, dans lequel la détection de la zone d'occlusion entre les trames comprend :

    la détermination d'une zone, dans laquelle un point caractéristique extrait de la trame précédente n'est pas extrait de la trame actuelle et dans laquelle la somme des différences absolues (SAD, sum of absolute différences) d'un bloc est supérieure ou égale à une SAD prédéterminée, comme zone couverte de la zone d'occlusion, et

    la détermination d'une zone, dans laquelle un point caractéristique qui n'est pas extrait de la trame précédente est extrait de la trame actuelle et dans laquelle la SAD du bloc est supérieure ou égale à la SAD prédéterminée, comme une zone non couverte de la zone d'occlusion ;

    ladite génération de la trame intermédiaire comprenant de préférence :

    la génération de la trame intermédiaire eu égard à la zone couverte au moyen d'un vecteur de mouvement de la trame précédente, et

    la génération de la trame intermédiaire eu égard à la zone non couverte au moyen d'un vecteur de mouvement de la trame actuelle.


     
    12. Procédé selon l'une quelconque des revendications 9 à 11, dans lequel la réalisation de l'estimation de mouvement arrière bloc par bloc réalise une estimation de mouvement arrière relativement à un bloc actuel au moyen des informations de trajectoire de point caractéristique propres au point caractéristique lorsque le point caractéristique suivi existe dans le bloc actuel relativement auquel l'estimation de mouvement est réalisée dans la trame actuelle, et réalise une estimation de mouvement arrière relativement à un bloc actuel au moyen d'un vecteur de mouvement propre à un bloc adjacent au bloc actuel lorsque le point caractéristique suivi n'existe pas dans le bloc actuel relativement auquel l'estimation de mouvement est réalisée dans la trame actuelle.
     
    13. Produit-programme informatique contenant des instructions de traitement visant à réaliser le procédé selon l'une quelconque des revendications 9 à 12.
     




    Drawing
































    Cited references

    REFERENCES CITED IN THE DESCRIPTION



    This list of references cited by the applicant is for the reader's convenience only. It does not form part of the European patent document. Even though great care has been taken in compiling the references, errors or omissions cannot be excluded and the EPO disclaims all liability in this regard.

    Patent documents cited in the description