|
(11) | EP 3 654 284 A1 |
| (12) | EUROPEAN PATENT APPLICATION |
| published in accordance with Art. 153(4) EPC |
|
|
|
|
|||||||||||||||||||||||||||
| (54) | TARGET TRACKING METHOD AND APPARATUS, ELECTRONIC DEVICE AND STORAGE MEDIUM |
| (57) This application relates to a target tracking method and apparatus, an electronic
device, and a storage medium. The method includes: obtaining, according to a target
detection on image data, a plurality of target instances, each of the plurality of
target instances corresponding to one of a plurality of tracked targets (310); determining,
a plurality of trajectory segments, each of the plurality of trajectory segments is
formed by connecting a subset of the target instances corresponding to a same tracked
target of the plurality of tracked targets (330); determining, feature information
of the plurality of trajectory segments by performing feature construction on the
trajectory segment according to the target instances in the trajectory segment (350);
performing clustering on specified trajectory segments according to the feature information,
to obtain a type distribution of the specified trajectory segments (370); and determining,
according to the type distribution of the specified trajectory segments, a target
tracking result including a same type of the specified trajectory segments(390). This
application can effectively improve target tracking accuracy. |
RELATED APPLICATION
[0001] This application claims priority to Chinese Patent Application No. 201710573025.4, entitled "TARGET TRACKING METHOD AND APPARATUS, AND ELECTRONIC DEVICE" filed with the Chinese Patent Office on July 14, 2017, which is incorporated by reference in its entirety.
TECHNICAL FIELD
[0002] This application relates to the field of computer technologies, and in particular, to a target tracking method and apparatus, an electronic device, and a storage medium.
BACKGROUND
[0003] As image capturing devices are becoming more abundant, a large quantity of image capturing devices, such as cameras, can be deployed both indoors and outdoors to perform target detection (tracking by detection) on tracked targets by using image data collected by the image capturing devices at any time, thereby tracking a target.
[0004] However, in a process of tracking the target, two different tracked targets may be obtained through target detection because of an attitude change of the tracked targets (for example, people). Alternatively, a tracked target may disappear or reappear due to discontinuous collection of image data, which still leads to two different tracked targets obtained through target detection.
[0005] The foregoing shows that the related target tracking still has a disadvantage of low accuracy.
SUMMARY
[0006] To resolve the foregoing technical problem, this application provides a target tracking method and apparatus, an electronic device, and a storage medium.
[0007] The technical solutions used in this application are as follows:
A target tracking method includes: obtaining image data, and performing target detection on the image data to obtain at least one target instance, each target instance corresponding to one tracked target; searching the at least one target instance for target instances including a same tracked target, and connecting the target instances including the same tracked target to form a trajectory segment; performing feature construction on the trajectory segment by using the target instances in the trajectory segment, to obtain trajectory segment feature information; performing clustering on specified trajectory segments according to the trajectory segment feature information, to obtain a trajectory segment type distribution; and connecting trajectory segments of a same type in the trajectory segment type distribution to form a target tracking result.
[0008] A target tracking apparatus includes: a target instance obtaining module, configured to: obtain image data, and perform target detection on the image data to obtain at least one target instance, each target instance corresponding to one tracked target; a trajectory segment obtaining module, configured to: search the at least one target instance for target instances including a same tracked target, and connect the target instances including the same tracked target to form a trajectory segment; a feature information obtaining module, configured to perform feature construction on the trajectory segment by using the target instances in the trajectory segment, to obtain trajectory segment feature information; a type distribution obtaining module, configured to perform clustering on specified trajectory segments according to the trajectory segment feature information, to obtain a trajectory segment type distribution; and a tracking result obtaining module, configured to connect trajectory segments of a same type in the trajectory segment type distribution to form a target tracking result.
[0009] An electronic device includes one or more processors and one or more memories, the memory storing computer-readable instructions, and the computer-readable instructions, when executed by the processor, implementing the foregoing target tracking method.
[0010] A computer-readable storage medium, storing a computer program, the computer program, when executed by a processor, implementing the foregoing target tracking method.
[0011] Compared with the existing technology, this application has the following beneficial effects:
Target detection is performed on an obtained image data, to obtain at least one target instance including at least one tracked target. The at least one target instance is searched for target instances including a same tracked target, to connect the target instances including the same tracked target to form a trajectory segment. Then, feature construction is performed on the trajectory segment by using the target instances in the trajectory segment, to obtain trajectory segment feature information, and clustering is performed on a trajectory segment according to the trajectory segment feature information, so that trajectory segments of a same type in an trajectory segment type distribution obtained through clustering are connected to form a target tracking result. Therefore, all target instances including a same tracked target are connected to form a trajectory segment, and clustering is performed on the trajectory segment according to the trajectory segment feature information, so that all connected trajectory segments in a target tracking result also include the same tracked target, thereby implementing accurate target tracking, and resolving a problem of low target tracking accuracy in the existing technology.
[0012] It is understood that the above general descriptions and the following detailed descriptions are merely for exemplary and explanatory purposes, and cannot limit this application.
BRIEF DESCRIPTION OF THE DRAWINGS
[0013] The accompanying drawings, which are incorporated herein and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
FIG. 1 is a schematic diagram of an implementation environment according to this application;
FIG. 2 is a schematic block diagram of hardware of a server according to an exemplary embodiment;
FIG. 3 is a flowchart of a target tracking method according to an exemplary embodiment;
FIG. 4 is a flowchart of an embodiment of step 350 in the embodiment corresponding to FIG. 3;
FIG. 5 is a schematic diagram of a target instance when a tracked target is a person according to this application;
FIG. 6 is a flowchart of an embodiment of step 351 in the embodiment corresponding to FIG. 4;
FIG. 7 is a flowchart of an embodiment of step 353 in the embodiment corresponding to FIG. 4;
FIG. 8 is a flowchart of an embodiment of step 370 in the embodiment corresponding to FIG. 3;
FIG. 9 is a flowchart of an embodiment of step 371 in the embodiment corresponding to FIG. 8;
FIG. 10 is a block diagram of a target tracking apparatus according to an exemplary embodiment;
FIG. 11 is a block diagram of an embodiment of a feature information obtaining module 750 in the embodiment corresponding to FIG. 10;
FIG. 12 is a block diagram of an embodiment of a target feature construction unit 751 in the embodiment corresponding to FIG. 11;
FIG. 13 is a block diagram of an embodiment of a trajectory segment feature construction unit 753 in the embodiment corresponding to FIG. 11;
FIG. 14 is a block diagram of an embodiment of a type distribution obtaining module 770 in the embodiment corresponding to FIG. 10;
FIG. 15 is a block diagram of another embodiment of a type distribution obtaining module 770 in the embodiment corresponding to FIG. 10;
FIG. 16 is a block diagram of an embodiment of a first likelihood calculation unit 771 in the embodiment corresponding to FIG. 15; and
FIG. 17 is a block diagram of another embodiment of a type distribution obtaining module 770 in the embodiment corresponding to FIG. 10.
DESCRIPTION OF EMBODIMENTS
[0014] Exemplary embodiments are described in detail herein, and examples of the exemplary embodiments are shown in the accompanying drawings. When the following descriptions relate to the accompanying drawings, unless indicated otherwise, same numbers in different accompanying drawings represent same or similar elements. Implementations described in the following exemplary embodiments are examples of the apparatus and method that are detailed in claims and that are consistent with some aspects in this application.
[0015] FIG. 1 is a schematic diagram of an implementation environment of a target tracking method. The implementation environment includes a server 100 and several image capturing devices 200 disposed everywhere.
[0016] The image capturing device 200 may be an electronic device for collecting image data, such as cameras, video recorders, video cameras, and the like. Correspondingly, the image data includes but is not limited to a video, a photo, and the like.
[0017] In this implementation environment, the server 100 obtains, by interacting with the image capturing devices 200, the image data collected by the image capturing devices 200, and performs target detection on tracked targets by using the image data to track a target.
[0018] FIG. 2 is a schematic block diagram of hardware of a server 100 according to an exemplary embodiment. It is noted that the server 100 is only an example adapted to this application, and is not intended to suggest any limitation as to the scope of use of this application. The server 100 may not be explained as being dependent on or needing to have one or more components of the exemplary server 100 shown in FIG 2.
[0019] A hardware structure of the server 100 can vary greatly due to different configurations or performance. As shown in FIG. 2, the server 100 includes a power supply 110, an interface 130, at least one storage medium 150, and at least one central processing unit (CPU) 170.
[0020] The power supply 110 is configured to provide an operating voltage for each hardware device on the server 100.
[0021] The interface 130 includes at least one wired or wireless network interface 131, at least one serial-parallel conversion interface 133, at least one input/output interface 135, at least one USB interface 137, and the like, and is configured to communicate with an external device.
[0022] The storage medium 150, as a carrier of resource storage, may be a random storage medium, a magnetic disk, an optical disc, or the like. Resources stored on the storage medium 150 include an operating system 151, an application program 153, data 155, and the like, and storage modes may be transitory storage or permanent storage. The operating system 151 is used for managing and controlling various hardware devices and the application program 153 on the server 100, and calculating and processing the massive data 155 by the central processing unit 170. The operating system 151 may be Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, and the like. The application program 153 is a computer program that performs at least one specific task based on the operating system 151, and may include at least one module (not shown in FIG 2). Each module can contain a series of operation instructions for the server 100. The data 155 can be photos, pictures, and the like stored on the magnetic disk.
[0023] The central processing unit 170 may include one or more processors and is configured to communicate with the storage medium 150 through a bus, for calculating and processing the massive data 155 in the storage medium 150.
[0024] As described in detail above, the server 100 applicable to this application reads, by using the central processing unit 170, a series of operation instructions stored in the storage medium 150, to track a target.
[0025] In addition, the technical solutions of this application can also be realized by a hardware circuit or a hardware circuit in combination with a software instruction. Therefore, realization of this application is not limited to any specific hardware circuit, software, and a combination thereof.
[0026] Referring to FIG. 3, in an exemplary embodiment, a target tracking method is applicable to the server 100 in the implementation environment shown in FIG. 1. The server 100 may have the hardware structure shown in FIG. 2 in the exemplary embodiment.
[0027] As shown in FIG. 3, the target tracking method may be performed by the server 100, and may include the following steps:
[0028] Step 310. Obtain image data, and perform target detection on the image data to obtain at least one target instance.
[0029] The image data includes dynamic image data and static image data. The dynamic image data refers to a plurality of image frames, such as a video, while the static image data may be a static image including one frame, such as a picture. Based on this, the target tracking in this embodiment may be carried out based on a video including multi-frame images or a picture of a single-frame image.
[0030] The image data may come from image data collected by an image capturing device in real time, or may be image data prestored in the server. That is, the server may process the image data in real time after the image data is collected by the image capturing device, or the server may prestore the image data and then process the image data. For example, the server processes image data when the server processes fewer tasks or at a time specified by an operator. Therefore, the image data obtained in this application may be image data collected by an image capturing device at present, or image data prestored in the server, that is, image data obtained by invoking an image capturing devices at a historical time, which is not limited herein.
[0031] The image capturing devices may be located everywhere, such as client devices, for example, in a corner of a ceiling inside a building, on a lamp post outside a building, and of an intelligent robot. Correspondingly, the image data may be any image in any scene where an image capturing device is disposed, any image inside a building, or any image outside a building, which is not limited herein.
[0032] The tracked target refers to any object on an image in any scene, such as a person, a car, or a mobile phone in the image, which is not limited herein.
[0033] The target detection of the image data is realized by using a target detection model. For example, the target detection model may be a deformable-part model, a single Gaussian model, a mixture Gaussian model, or the like, which is not limited herein. The target detection model is pre-created by the server before performing target detection on the image data.
[0034] It may be understood that, because the image data is any image in any scene where an image capturing device is deployed, sometimes, some images include one tracked target, some images include a plurality of tracked targets, and some images include no tracked target. Therefore, the target instance refers to image data including one tracked target.
[0035] It is noted that based on same image data, possibly no target instance can be obtained, that is, there is no tracked target in an image indicated by the image data, or a plurality of target instances may be obtained, that is, an image indicated by the image data contains a plurality of tracked targets, and each tracked target corresponds to one target instance.
[0036] It is further noted that, regardless how many target instances can be obtained from image data, the target instances overlap with each other in time. That is, a plurality of target instances included in image data are obtained from image data collected by the image capturing devices at a same collection time.
[0037] Therefore, at least one target instance may be obtained through the target detection performed on the image data by using the pre-created target detection model, where the at least one target instance contains one tracked target.
[0038] Further, because the image data may be a video of a plurality of frames or a picture of a single frame, target detection is performed on the image data in frame units, that is, one frame of image is used as input of the target detection model to implement a target detection process of the image data.
[0039] Step 330. Search the at least one target instance for target instances including a same tracked target, and connect the target instances including the same tracked target to form a trajectory segment.
[0040] First, it is noted that, a preset tracking algorithm, such as a KLT algorithm, may be used for performing global search on all tracked targets included in many target instances, to track a tracked target, that is, to find same tracked targets in many target instances.
[0041] Therefore, target instances including a same tracked target may be obtained by using the preset tracking algorithm, and then all target instances including a same tracked target may be connected to form a trajectory segment.
[0042] For example, a target instance A1 includes a tracked target A, a target instance A2 includes a tracked target B, and a target instance A3 includes a tracked target A. Correspondingly, a trajectory segment 1 is formed by connecting the target instances A1 and A3 including the tracked target A, and a trajectory segment 2 is formed by connecting the target instance A2 including the tracked target B.
[0043] It is understood that the image data is collected in chronological order, that is, the target instances are in chronological order.
[0044] Therefore, when a trajectory segment is obtained, all target instances including a same tracked target are connected in chronological order.
[0045] Still in the foregoing example, the first target instance of the trajectory segment 1 is A1, and the second target instance is A3. It is noted that, as time goes on, a quantity of target instances included in the trajectory segments increases, for example, the third target instance, the fourth target instance, ..., the last target instance is added to the trajectory segment 1 in time order.
[0046] Step 350. Perform feature construction on the trajectory segment by using the target instances in the trajectory segment, to obtain trajectory segment feature information.
[0047] The trajectory segment feature information is used for accurately describing an entire and/or a part of the trajectory segment, to uniquely identify the trajectory segment in a form of information. It is understood that, if the target instances in the trajectory segment include different tracked targets, the trajectory segment are also different, which makes the trajectory segment feature information different as well.
[0048] Therefore, after a trajectory segment is formed by connecting target instances including a same tracked target, corresponding trajectory segment feature information may be obtained by using the target instances in the trajectory segment.
[0049] Specifically, the trajectory segment feature information is obtained by performing feature construction on the trajectory segment by using the target instances in the trajectory segment.
[0050] Further, the feature construction includes but is not limited to, target feature construction on the target instances, and local feature construction and global feature construction on the trajectory segment. Correspondingly, the trajectory segment feature information includes but is not limited to target feature information, local feature information, and global feature information.
[0051] Still further, the target feature information is related to a tracked target included in a corresponding target instance. The local feature information is related to at least one target instance in a corresponding trajectory segment. For example, the local feature information is related to the first target instance and the last target instance in the trajectory segment. The global feature information is related to all target instances in a corresponding trajectory segment.
[0052] Step 370. Perform clustering on specified trajectory segments according to the trajectory segment feature information, to obtain a trajectory segment type distribution.
[0053] In this embodiment, the clustering refers to a process of classifying the specified trajectory segments into a plurality of different types according to different tracked targets, that is, each type of trajectory segments corresponds to a tracked target through clustering.
[0054] It is noted that, the specified trajectory segment are trajectory segments that need to be clustered, which may be all of the trajectory segments obtained through step 330, or just any few of the trajectory segments that need to be clustered.
[0055] As described above, if target instances in trajectory segments include different tracked targets, the trajectory segments are also different, which leads to different trajectory segment feature information. In other words, the trajectory segment feature information may be used to represent a tracked target. In this way, the corresponding trajectory segment feature information has high similarity or even consistency, regardless that the tracked targets appear in which scenes, that is, which collected image data, or target instances that the tracked targets are included.
[0056] Therefore, the trajectory segments can be clustered through the trajectory segment feature information. To be specific, for extremely similar or even consistent trajectory segment feature information, target instances included in corresponding trajectory segments may be regarded as including a same tracked target, that is, the corresponding trajectory segments belong to a same type. Otherwise, for non-similar or inconsistent trajectory segment feature information, target instances in the corresponding trajectory segments may be regarded as including different tracked targets, that is, the corresponding trajectory segments belong to different types.
[0057] Based on the above, the trajectory segment type distribution refers to trajectory segments included in different types and a quantity of the trajectory segments. Different types are for different tracked targets. For example, for all trajectory segments belonging to a type A, tracked targets included in target instances is B, and for all trajectory segments belonging to a type C, tracked targets included in target instances is D.
[0058] For example, a Dirichlet mixture model is used for performing clustering on the specified trajectory segments. Specifically, the trajectory segment feature information corresponding to the specified trajectory segments is used as an input of the Dirichlet mixture model. The Dirichlet mixture model is used for performing clustering on the specified trajectory segments, to output a type of the specified trajectory segments. Further, the trajectory segment type distribution may be output by inputting the trajectory segment feature information for a plurality of specified trajectory segments.
[0059] In an embodiment, a clustering algorithm used for performing clustering on the trajectory segments may be flexibly adjusted to improve target tracking accuracy.
[0060] In the foregoing process, exclusive constraints of the target tracking are realized by performing clustering, that is, same tracked targets definitely belong to a same type, which provides a reliable guarantee for target tracking accuracy simply and effectively.
[0061] Step 390. Connect trajectory segments of a same type in the trajectory segment type distribution to form a target tracking result.
[0062] A trajectory segment is formed by connecting at least one target instance including a same tracked target. Further, based on the foregoing steps, for trajectory segments classified into a same type, target instances include a same tracked target. Therefore, the target tracking result formed by connecting the trajectory segments of the same type is also definitely based on a same tracked target. That is, the target tracking result corresponds to a unique tracked target, thereby accurately tracking a target.
[0063] Further, a plurality of targets can be tracked for a plurality of target tracking results formed by connecting different types of trajectory segments.
[0064] A connection for the target tracking result is carried out in chronological order of the trajectory segments. For example, the last target instance of the trajectory segment 1 is prior to the first target instance of the trajectory segment 2 in time. In this case, that is, the trajectory segment 1 is earlier than the trajectory segment 2 in time. Correspondingly, in the target tracking result, the trajectory segment 1 is connected before the trajectory segment 2 is connected.
[0065] Through the foregoing process, under the action of the trajectory segment feature information, a plurality of trajectory segments may be classified based on a same tracked target, to obtain a target tracking result of the tracked target. Therefore, in the process of target tracking, regardless whether the tracked target disappears or reappears, multi-target tracking may be performed on any plurality of tracked targets according to a requirement.
[0066] Referring to FIG. 4, in an exemplary embodiment, step 350 may include the following steps:
Step 351. Perform target feature construction on the target instances in the trajectory segment, to obtain target feature information.
[0067] The target feature information is used for accurately describing the target instance through feature construction of the target instance. Further, the target feature information is related to a tracked target including in a corresponding target instance, thereby uniquely identifying the tracked target in a form of information.
[0068] It is understood that, different tracked targets leads to different target feature information. For example, if the tracked target is a person in the image, the target feature information may include a visual feature vector and a structure feature vector of the person; if the tracked target is a vehicle in the image, the target feature information may include a plate number; if the tracked target is a mobile phone in the image, the target feature information may include a device identification code or a device signal.
[0069] Herein, the target feature information is not enumerated herein. Different tracked targets each have corresponding target feature information, so as to accurately describe and identify a tracked target.
[0070] Step 353. Perform local feature construction and global feature construction on the trajectory segment according to the target feature information, to obtain local feature information and global feature information.
[0071] It is noted first that, the local feature information is used for accurately describing some trajectory segments through local feature construction of the trajectory segments.
[0072] Further, the local feature information is related to at least one target instance in a corresponding trajectory segment. For example, the local feature information is related to the first target instance and the last target instance in the corresponding trajectory segment.
[0073] Therefore, the local feature information may be defined by using the target feature information corresponding to the at least one target instance. For example, the target feature information includes a visual feature vector of a tracked target. In this case, a visual feature vector is extracted from the target feature information corresponding to the at least one target instance, and the visual feature vector is used as the local feature information.
[0074] In addition, the global feature information is used for accurately describing an entire trajectory segment through global feature construction of the trajectory segment.
[0075] Further, the global feature information is related to all target instances in a corresponding trajectory segment.
[0076] Therefore, the global feature information may be defined by using the target feature information corresponding to each target instance. Still using the foregoing example for description, the target feature information includes a structure feature vector of the tracked target. In this case, a structure feature vector is extracted from the target feature information respectively corresponding to the target instances, and the global feature information is obtained by using the structure feature vectors.
[0077] Step 355. Generate the trajectory segment feature information according to the local feature information and the global feature information.
[0078] As described above, the trajectory segment feature information is used for accurately describing an entire and/or a part of a trajectory segment. Therefore, after the local feature information and the global feature information are obtained, trajectory segment feature information including the local feature information and the global feature information can be correspondingly obtained.
[0079] The foregoing process provides sufficient basis for a clustering process in which the specified trajectory segments are correspondingly classified into different types according to the different tracked targets. That is, the trajectory segment feature information having extremely high similarity or even consistency is used for representing same tracked targets.
[0080] Further, in an exemplary embodiment, step 310 of performing target detection on the image data to obtain at least one target instance may include the following step:
performing annotation information identification on a plurality of deformable parts of a tracked target in the image data by using a pre-created deformable-part model, to obtain the at least one target instance.
[0081] A deformable part of the tracked target corresponding to the target instance is identified by using annotation information.
[0082] For the image data, target detection is performed by using the deformable-part model, where an obtained tracked target is a non-rigid target. The non-rigid target is a tracked target that deforms in a target tracking process, for example, a person, an animal, or another deformable object.
[0083] Specifically, the tracked target is represented by using the deformable-part model as a global rectangular frame and a plurality of part rectangular frames. The global means an entire tracked target, and the part means a deformable part of the tracked target.
[0084] Images annotated by the global rectangular frame and the part rectangular frames are defined as annotation information in the target instance, so that a plurality of deformable parts of the tracked target are identified by using a plurality of pieces of annotation information in the target instance.
[0085] In an example, the tracked target is a person. As shown in FIG. 5, the person is represented as one global rectangular frame and six part rectangular frames. The global means an entire person, and the part means a deformable part such as the head, the left hand, the right hand, the left leg, the right leg, the left foot and the right foot of the person. Correspondingly, the deformable part of the person is identified by using seven pieces of standard information in the target instance.
[0086] In an embodiment, a quantity of part rectangular frames may be flexibly adjusted to satisfy different requirements for target tracking accuracy in different application scenarios.
[0087] Correspondingly, referring to FIG. 6, step 351 may include the following steps:
Step 3511. Obtain a visual feature vector and a structure feature vector of the tracked target.
[0088] Specifically, histogram feature vector extraction is performed on a plurality of pieces of annotation information, and an extracted histogram feature vector is used as the visual feature vector of the tracked target.
[0089] The histogram feature vector includes a histogram of oriented gradients feature vector and a color histogram feature vector. The histogram of oriented gradients feature vector is used for describing a texture feature of the tracked target, and the color histogram feature vector is used for describing a color feature of the tracked target.
[0090] In a histogram feature vector extraction process, histogram feature vector extraction is essentially performed on a deformable part identified by using a plurality of pieces of annotation information. For example, a person's head is identified by using the annotation information. In this case, histogram feature vector extraction is performed on an image annotated by using a part rectangular frame in which the person's head is located.
[0091] Based on this, the visual feature vector of the tracked target is defined by using an extracted histogram feature vector, thereby reflecting external appearance information of the tracked target.
[0092] For example, for one piece of the annotation information, the histogram of oriented gradients feature vector is a1, and the color histogram feature vector is b1. In this case, the visual feature vector is obtained by using the annotation information as {al, b1}.
[0093] By analogy, as an amount of annotation information increases, a length of the visual feature vector generated by using the annotation information also correspondingly increases, thereby improving tracked target describing accuracy.
[0094] For example, a visual feature vector obtained by using eight pieces of annotation information is {a1, b1, a2, b2, a3, b3, a4, b4, a5, b5, a6, b6, a7, b7, a8, b8}.
[0095] A deformable part identified by using one of the plurality of pieces of annotation information is used as an anchor point to calculate location deviations between the anchor point and deformable parts identified by using remaining annotation information, and calculated deviation values are used as the structure feature vector of the tracked target.
[0096] Still using the foregoing example for description, the annotation information is used for identifying a deformable part of a person in the target instance. Preferably, the deformable part of the person is identified by using eight pieces of annotation information in the target instance. That is, an image annotated by using one global rectangular frame represents the head of the person, and images annotated by using seven part rectangular frames respectively represent parts of the left hand, the right hand, the body, the left leg, the right leg, the left food, and the right food of the person.
[0097] Herein, a person's head is used as an anchor point. There are seven deviation values obtained through calculation based on the person's head and remaining part of the person, so as to obtain a structure feature vector when the tracked target is a person, thereby reflecting internal structural information when the tracked target is the person.
[0098] For example, the deviation values are c1, c2, c3, c4, c5, c6, and c7 respectively. In this case, the structure feature vector is {c1, c2, c3, c4, c5, c6, c7}.
[0099] Step 3513. Generate target feature information corresponding to the target instances according to the visual feature vector and the structure feature vector of the tracked target.
[0100] After obtaining the visual feature vector and the structure feature vector of the tracked target, the target instance can be accurately described. That is, the target feature information corresponding to the target instance includes the visual feature vector and the structure feature vector of the tracked target, thereby uniquely identify, in a form of information. that the person is the tracked target.
[0101] In the foregoing process, the target feature information is used to reflect the external appearance information and the internal structural information of the tracked target, thereby accurately describing the tracked target, facilitating subsequent accurate target tracking.
[0102] Referring to FIG. 7, in an exemplary embodiment, step 353 may include the following steps:
Step 3531. Extract at least one target instance from the trajectory segment, and use a visual feature vector in target feature information corresponding to the at least one target instance as the local feature information corresponding to the trajectory segment.
[0103] As described above, the local feature information is related to at least one target instance in a corresponding trajectory segment. The local feature information may be defined by using the target feature information corresponding to the at least one target instance.
[0104] Preferably, the local feature information is related to the first target instance and the last target instance in the corresponding trajectory segment.
[0105] Specifically, the first target instance and the last target instance are extracted from the trajectory segment in chronological order, and the visual feature vector in the target feature information corresponding to the first target instance and the visual feature vector in the target feature information corresponding to the last target instance are obtained, so that the local feature information corresponding to the trajectory segment includes the foregoing visual feature vectors.
[0106] In this process, the local feature information may be regarded as accurate descriptions of the tracked target included in the at least one target instance in the trajectory segment. That is, accurate description of a part of the trajectory segment is implemented, facilitating subsequent accurate target tracking.
[0107] Step 3533. For the target instances in the trajectory segment, calculate an average value of a structure feature vector in the corresponding target feature information, and perform a covariance operation according to the structure feature vector to obtain a covariance matrix.
[0108] Step 3535. Use the average value and the covariance matrix as the global feature information corresponding to the trajectory segment.
[0109] As described above, the global feature information is related to all target instances in a corresponding trajectory segment. The global feature information may be defined by using the target feature information corresponding to each target instance.
[0110] Specifically, various structure feature vectors in the target feature information corresponding to all target instances are calculated for an average value and a covariance matrix, and the average value and the covariance matrix are defined as the global feature information corresponding to the trajectory segment.
[0111] The structure feature vectors in the target feature information corresponding to the target instances are used as elements, and elements in the covariance matrix are the covariance between the foregoing various elements.
[0112] In this process, all target instances in the trajectory segment are averaged and de-correlated by using the global feature information, thereby accurately describing an entire trajectory segment, facilitating subsequent accurate target tracking.
[0113] Referring to FIG. 8, in an exemplary embodiment, step 370 may include the following steps:
Step 371. Calculate, for predefined at least one type, likelihood between the at least one type and the specified trajectory segments according to the trajectory segment feature information.
[0114] The type is a set of including at least one trajectory segment. In the set, target instances in all trajectory segments include same tracked target.
[0115] As described above, for extremely similar or even consistent trajectory segment feature information, target instances included in corresponding trajectory segments may be regarded as including a same tracked target, that is, the corresponding trajectory segments belong to a same type.
[0116] Otherwise, if a type of trajectory segments and the specified trajectory segments have extremely similar or even consistent trajectory segment feature information, the specified trajectory segments may possibly belongs to the type.
[0117] Therefore, before the trajectory segment clustering, the likelihood between each type of trajectory segments and the specified trajectory segments needs to be obtained according to the trajectory segment feature information, to learn whether extremely similar or even consistent trajectory segment feature information exists between the specified trajectory segments and each trajectory segment in the type.
[0118] Step 373. Calculate, according to the likelihood, a probability that the specified trajectory segments follows a uniform distribution in the at least one type.
[0119] It is noted that herein, after the likelihood between each type of trajectory segments and the specified trajectory segments is calculated, normalization processing is first performed on the calculated likelihood, to ensure all likelihood for probability calculating are of a same quantity level, thereby improving probability calculating accuracy, and further improving target tracking accuracy.
[0120] Step 375. Classify the specified trajectory segments into a type of corresponding to a maximum probability.
[0121] Step 377. Complete clustering of the specified trajectory segments to form the trajectory segment type distribution.
[0122] When the specified trajectory segments that need to be clustered are classified into a same type or different types, clustering is completed once. In this case, a clustering result is a trajectory segment type distribution.
[0123] Further, to improve target tracking accuracy, a plurality of times of clustering may be performed, and the last clustering result is used as a trajectory segment type distribution. A quantity of iterations of the clustering can be flexibly adjusted according to a requirement in an actual application scenario.
[0124] In an exemplary embodiment, before step 371, the method may further include the following steps:
for target instances included in a type of trajectory segments, determining whether a trajectory segment in the type of trajectory segments and the specified trajectory segments include target instances temporally overlapping with each other; and
if the trajectory segment exists, setting likelihood between the type of trajectory segments and the specified trajectory segments to zero.
[0125] As described above, the target instances overlapping with each other in time are obtained from image data collected by image capturing devices at a same collection time. That is, these target instances are derived from the same image data.
[0126] It may be understood that same image data may include more than one tracked target. Correspondingly, after the target detection, a plurality of target instances containing different tracked targets may be obtained. The plurality of target instances are connected to form a plurality of trajectory segments, making it impossible that the trajectory segments belong to a same type.
[0127] Therefore, if the specified trajectory segments and a trajectory segment in one type have target instances that overlap with each other in time, it indicates that the specified trajectory segments cannot belong to the type. In other words, the likelihood between the type of trajectory segments and the specified trajectory segments is necessarily zero.
[0128] As can be seen from the above, if a trajectory segment in one type includes a target instance that overlap with a target instance in the specified trajectory segments in time, there is no need to calculate the likelihood between the type of trajectory segments and the specified trajectory segments, and the between the type of trajectory segments and the specified trajectory segments can be directly to set to zero.
[0129] In combination with the foregoing embodiment, exclusive constraints for the tracked target, that is, trajectory segments corresponding to different tracked targets in same image data belong to different types, providing a guarantee for target tracking accuracy simply and effectively.
[0130] Referring to FIG. 9, in an exemplary embodiment, step 371 may include the following steps:
Step 3711. For at least one target instance in the specified trajectory segment, obtain, from a type of trajectory segments, a trajectory segment to which a target instance closest to the at least one target instance in time belongs.
[0131] Preferably, the likelihood calculation is calculated by using the first and the last target instances in a specified trajectory segment.
[0132] It is understood that a connection of the target tracking result is carried out in chronological order of the trajectory segments. Therefore, each trajectory segment in a same type is bound to have a time sequence, and target instances in each trajectory segment also have a time sequence. In other words, the last target instance connected in the preceding trajectory segment is prior to the first target instance connected in the following trajectory segment on the time axis.
[0133] Correspondingly, in one type, a target instance closest to the first target instance in the specified trajectory segments in time is the last target instance in a trajectory segment of the type.
[0134] A target instance closest to the last target instance in the specified trajectory segments in time is the first target instance in a trajectory segment of the type.
[0135] Step 3713. Calculate local similarity between the specified trajectory segments and the obtained trajectory segment according to local feature information respectively corresponding to the trajectory segments.
[0136] It is noted that the trajectory segments are the specified trajectory segments and the obtained trajectory segment. Correspondingly, the local feature information corresponding to the trajectory segments respectively is local feature information of the specified trajectory segments and local feature information of the obtained trajectory segment.
[0137] Step 3715. Calculate global similarity between the type of trajectory segments and the specified trajectory segments according to global feature information corresponding to the specified trajectory segments and a type parameter of the type of trajectory segments.
[0138] Step 3717. Calculate likelihood between the type of trajectory segments and the specified trajectory segments by using the local similarity and the global similarity.
[0139] Specifically, for the specified trajectory segments and the type, the likelihood between them can be calculated as follows:
where f represents the likelihood between the type of trajectory segments and the specified trajectory segments; and
xi represents an ith specified trajectory segment, (φk,xk,[]) represents the type, the type is a kth type in the trajectory segment type distribution, and the kth type includes several trajectory segments, which is represented by [].
[0140] In the first term s, Aitail represents a visual feature vector in target feature information corresponding to the first target instance in the ith specified trajectory segment, and Ak,nhead represents a visual feature vector in target feature information corresponding to the last target instance in an mth trajectory segment of the kth type.
[0141] In the second term s, Aitail represents a visual feature vector in target feature information corresponding to the last target instance in the ith specified trajectory segment, and Ak,n head represents a visual feature vector in target feature information corresponding to the first target instance in an nth trajectory segment of the kth type.
[0142] In the third item p, Di and Vi respectively represent an average value and a covariance matrix in the global feature information corresponding to the ith specified trajectory segment. φk is a type parameter of the kth type, where the type parameter is obtained from Gaussian model modeling by using an average value and a covariance matrix in the global feature information corresponding to all trajectory segments [] of the kth type.
[0143] Further, the function s represents calculation, for the foregoing two target instances, of similarity between histogram feature vectors in visual feature vectors corresponding to the two target instances. Then, all calculated histogram feature vectors are accumulated to obtain local similarity between the ith specified trajectory segment and the mth trajectory segment of the kth type and between the ith specified trajectory segment and the nth trajectory segment of the kth type. The histogram feature vector includes a histogram of oriented gradients feature vector and a color histogram feature vector.
[0144] The function p represents that Gaussian model modeling was carried out on an average value and a covariance matrix in the global feature information corresponding to all trajectory segments [] of the kth type and the ith specified trajectory segment, next a distance between the two Gaussian models obtained through the modeling was compared, and then comparison results are converted into global similarity between the specified trajectory segments and the type.
[0145] Further, after calculating the local similarity s1 between the ith specified trajectory segment and the mth trajectory segment of the kth type, the local similarity s2 between the ith specified trajectory segment and the nth trajectory segment of the kth type, and the global similarity p between the ith specified trajectory segment and the kth type, the likelihood f between the ith specified trajectory segment and the kth type can be calculated by using the foregoing formula.
[0146] In an exemplary embodiment, before step 377, the method may further include the following step:
determining whether an iteration quantity of the clustering satisfies a preset iteration threshold.
[0147] For example, the preset iteration threshold is set to 500 times. Certainly, the preset iteration threshold can be set flexibly according to an actual requirement. For example, to improve target tracking accuracy, the preset iteration threshold is increased, and to reduce processing pressure on the server, the preset iteration threshold is reduced.
[0148] If the quantity of iterations of the clustering satisfies the preset iteration threshold, the iterative process of the clustering is stopped and a result obtained through the last clustering is used as a trajectory segment type distribution, that is, step 377 is performed.
[0149] Otherwise, if the iteration quantity of the clustering does not satisfy the preset iteration threshold, type parameter update is triggered, and for at least one type for which a type parameter has been updated, likelihood between the at least one type and the specified trajectory segments is calculated according to the trajectory segment feature information. That is, step 371 is returned to, until the iteration quantity of the clustering satisfies the preset iteration threshold.
[0150] In addition, according to different application scenarios, the stop condition of the cyclic iteration can be flexibly set, or the cyclic iteration can be stopped when a calculation time reaches a preset calculation time, or the cyclic iteration is stopped when a clustering result remains unchanged.
[0151] Under the action of the foregoing embodiment, clustering accuracy is improved through cyclic iteration, making tracked targets based on a target tracking result more consistent, thereby improving target tracking accuracy.
[0152] The following is an apparatus embodiment in this application. The apparatus may be configured to implement the target tracking method in this application. For details not disclosed in the apparatus embodiment in this application, refer to the embodiment of the target tracking method in this application.
[0153] Referring to FIG. 10, in an exemplary embodiment, a target tracking apparatus 700 includes, but is not limited to, a target instance obtaining module 710, a trajectory segment obtaining module 730, a feature information obtaining module 750, a type distribution obtaining module 770, and a tracking result obtaining module 790.
[0154] The target instance obtaining module 710 is configured to: obtain image data, and perform target detection on the image data to obtain at least one target instance. Each target instance corresponds to one tracked target.
[0155] The trajectory segment obtaining module 730 is configured to: search the at least one target instance for target instances including a same tracked target, and connect the target instances including the same tracked target to form a trajectory segment.
[0156] The feature information obtaining module 750 is configured to perform feature construction on the trajectory segment by using the target instances in the trajectory segment, to obtain trajectory segment feature information.
[0157] The type distribution obtaining module 770 is configured to perform clustering on specified trajectory segments according to the trajectory segment feature information, to obtain a trajectory segment type distribution.
[0158] The tracking result obtaining module 790 is configured to connect trajectory segments of a same type in the trajectory segment type distribution to form a target tracking result.
[0159] Referring to FIG. 11, in an exemplary embodiment, the feature information obtaining module 750 includes, but is not limited to, a target feature construction unit 751, a trajectory segment feature construction unit 753, and a feature information defining unit 755.
[0160] The target feature construction unit 751 is configured to perform target feature construction on the target instances in the trajectory segment, to obtain target feature information.
[0161] The trajectory segment feature construction unit 753 is configured to perform local feature construction and global feature construction on the trajectory segment according to the target feature information, to obtain local feature information and global feature information.
[0162] The feature information defining unit 755 is configured to generate the trajectory segment feature information according to the local feature information and the global feature information.
[0163] In an exemplary embodiment, the target instance obtaining module includes an annotation information identification unit.
[0164] The annotation information identification unit is configured to perform annotation information identification on a plurality of deformable parts of a tracked target in the image data by using a pre-created deformable-part model, to obtain the at least one target instance. A deformable part of the tracked target corresponding to the target instance is identified by using annotation information.
[0165] Correspondingly, referring to FIG. 12, the target feature construction unit 751 includes, but is not limited to, a feature vector obtaining subunit 7511 and a feature information forming subunit 7513.
[0166] The feature vector obtaining subunit 7511 is configured to obtain a visual feature vector and a structure feature vector of the tracked target.
[0167] Specifically, histogram feature vector extraction is performed on a plurality of pieces of annotation information, and an extracted histogram feature vector is used as the visual feature vector of the tracked target. A deformable part identified by using one of the plurality of pieces of annotation information is used as an anchor point to respectively calculate location deviations between the anchor point and deformable parts identified by using remaining annotation information, and calculated deviation values are used as the structure feature vector of the tracked target.
[0168] The feature information forming subunit 7513 is configured to generate target feature information corresponding to the target instances according to the visual feature vector and the structure feature vector of the tracked target.
[0169] Referring to FIG. 13, in an exemplary embodiment, the trajectory segment feature construction unit 753 includes, but is not limited to, a local feature information defining subunit 7531, a structure feature vector calculation subunit 7533, and a global feature information defining subunit 7535.
[0170] The local feature information defining subunit 7531 is configured to: extract at least one target instance from the trajectory segment, and use a visual feature vector in target feature information corresponding to the at least one target instance as the local feature information corresponding to the trajectory segment.
[0171] The structure feature vector calculation subunit 7533 is configured to: for the target instances in the trajectory segment, calculate an average value of a structure feature vector in the corresponding target feature information, and perform a covariance operation according to the structure feature vector to obtain a covariance matrix.
[0172] The global feature information defining subunit 7535 is configured to use the average value and the covariance matrix as the global feature information corresponding to the trajectory segment.
[0173] Referring to FIG. 14, in an exemplary embodiment, the type distribution obtaining module 770 includes, but is not limited to, a first likelihood calculation unit 771, a probability calculation unit 773, a clustering unit 775, and a type distribution forming unit 777.
[0174] The first likelihood calculation unit 771 is configured to calculate, for predefined at least one type, likelihood between the at least one type and the specified trajectory segments according to the trajectory segment feature information.
[0175] The probability calculation unit 773 is configured to calculate, according to the likelihood, a probability that the specified trajectory segments follows a uniform distribution in the at least one type.
[0176] The clustering unit 775 is configured to classify the specified trajectory segments into a type of corresponding to a maximum probability.
[0177] The type distribution forming unit 777 is configured to complete clustering of the specified trajectory segments to form the trajectory segment type distribution.
[0178] Referring to FIG. 15, in an exemplary embodiment, the type distribution obtaining module 770 further includes, but is not limited to, an overlapping determining unit 810 and a likelihood setting unit 830.
[0179] The overlapping determining unit 810 is configured to: for target instances included in a type of trajectory segments, determine whether a trajectory segment in the type of trajectory segments and the specified trajectory segments include target instances temporally overlapping with each other. If the trajectory segment exists, the likelihood setting unit is notified.
[0180] The likelihood setting unit 830 is configured to set likelihood between the type of trajectory segments and the specified trajectory segments to zero.
[0181] Referring to FIG. 16, in an exemplary embodiment, the first likelihood calculation unit 771 includes, but is not limited to, a trajectory segment feature vector obtaining subunit 7711, a local similarity calculation subunit 7713, a global similarity calculation subunit 7715, and a likelihood calculation subunit 7717.
[0182] The trajectory segment feature vector obtaining subunit 7711 is configured to: for at least one target instance in the specified trajectory segment, obtain, from a type of trajectory segments, a trajectory segment to which a target instance closest to the at least one target instance in time belongs.
[0183] The local similarity calculation subunit 7713 is configured to calculate local similarity between the specified trajectory segments and the obtained trajectory segment according to local feature information respectively corresponding to the trajectory segments.
[0184] The global similarity calculation subunit 7715 is configured to calculate global similarity between the type of trajectory segments and the specified trajectory segments according to global feature information corresponding to the specified trajectory segments and a type parameter of the type of trajectory segments.
[0185] The likelihood calculation subunit 7717 is configured to calculate likelihood between the type of trajectory segments and the specified trajectory segments by using the local similarity and the global similarity.
[0186] Referring to FIG. 17, in an exemplary embodiment, the type distribution obtaining module 770 further includes, but is not limited to, an iteration determining unit 910 and a second likelihood calculation unit 930.
[0187] The iteration determining unit 910 is configured to determine whether an iteration quantity of the clustering satisfies a preset iteration threshold. If the iteration quantity of the clustering does not satisfy the preset iteration threshold, the second likelihood calculation unit is notified.
[0188] The second likelihood calculation unit 930 is configured to: trigger to perform type parameter update, and for at least one type for which a type parameter has been updated, calculate likelihood between the at least one type and the specified trajectory segments according to the trajectory segment feature information.
[0189] It is noted that, when the target tracking apparatus provided in the foregoing embodiment performs target tracking, the divisions of the foregoing functional modules are described by using an example. During actual application, the foregoing functions may be allocated to and completed by different functional modules according to requirements, that is, the internal structure of the target tracking apparatus is divided into different functional modules, to complete all or some of the foregoing described functions.
[0190] In addition, the target tracking apparatus provided in the foregoing embodiment and the embodiment of the target tracking method belong to a same ideal. Specific operations manners of the modules have been described in detail in the method embodiment, and the details are not described herein again.
[0191] In an exemplary embodiment, an electronic device includes one or more processors and one or more memories.
[0192] The memory stores computer-readable instructions, the computer-readable instructions, when executed by the processor, implementing the target tracking method in the foregoing embodiment.
[0193] In an exemplary embodiment, a computer-readable storage medium stores a computer program, the computer program, when executed by a processor, implementing the target tracking method in the foregoing embodiment.
[0194] The foregoing descriptions are merely exemplary embodiments of this application, and are not intended to limit the embodiments of this application. A person of ordinary skill in the art can make corresponding modifications and variations with ease without departing from the spirit and scope of the embodiments of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.
obtaining, according to a target detection on image data, a plurality of target instances, each of the plurality of target instances corresponding to one of a plurality of tracked targets (310);
determining, a plurality of trajectory segments, each of the plurality of trajectory segments is formed by connecting a subset of the target instances corresponding to a same tracked target of the plurality of tracked targets (330);
determining, feature information of the plurality of trajectory segments by performing feature construction on the trajectory segment according to the target instances in the trajectory segment (350);
performing clustering on specified trajectory segments according to the feature information, to obtain a type distribution of the specified trajectory segments (370); and
determining, according to the type distribution of the specified trajectory segments, a target tracking result including a same type of the specified trajectory segments (390).
determining, target feature information of the target instances associated with one of the plurality of trajectory segments by performing target feature construction on the target instances associated with one of the plurality of trajectory segments (351);
determining, according to the target feature information, local feature information and global feature information by performing local feature construction and global feature construction on one of the plurality of trajectory segments (353); and
determining, feature information of the one of the plurality of trajectory segments according to the local feature information and the global feature information (355).
obtaining, one of the plurality of target instances by performing annotation information identification on a plurality of deformable parts of a tracked target of the plurality of tracked targets in the image data by using a pre-created deformable-part model, one of the plurality of deformable parts of the tracked target corresponding to the target instance being identified by using annotation information; and
wherein the determining, feature information of the plurality of trajectory segments by performing feature construction on the trajectory segment according to the target instances in the trajectory segment comprises:
obtaining a visual feature vector and a structure feature vector of one of the tracked targets (3511),
wherein the visual feature vector being a histogram feature vector that is extracted from the annotation information of the plurality of deformable parts of the tracked target;
the structure feature vector being based on location deviation values between one of the plurality of deformable parts and another one of the plurality of deformable parts of the tracked target; and
determining, the target feature information of the one of the target instances according to the visual feature vector and the structure feature vector of the tracked target (3513).
determining, the local feature information corresponding to the trajectory segment according to a visual feature vector in target feature information corresponding to the at least one target instances, the at least one target instances being extracted from the plurality of trajectory segments (3531);
determining, an average value of structure feature vectors and a covariance matrix of the structure feature vectors, the structure feature vectors corresponding to the target instances associated with the one of the plurality of trajectory segments (3533) ; and
determining, the global feature information corresponding to the trajectory segment according to the average value and the covariance matrix of the structure feature vectors (3535).
calculating, for each of at least one predefined type of trajectory segment, a likelihood between the respective predefined type of trajectory segment and one of the specified trajectory segments according to the feature information of the one of the specified trajectory segments (371);
calculating, according to the likelihoods between the at least one predefined type of trajectory segment and the one of the specified trajectory segments, a plurality of probabilities that the one of the specified trajectory segments follows a uniform distribution in the at least one predefined type of trajectory segment (373);
classifying, the one of the specified trajectory segments into one of the at least one predefined type of corresponding to a maximum probability in the plurality of probabilities (375); and
completing clustering of the specified trajectory segments to form the type distribution of the specified trajectory segments (377).
determining, for target instances associated with one of the at least one predefined type of trajectory segment, whether a first target instance associated with the respective predefined type of trajectory segment and a second target instance associated with the one of the specified trajectory segments overlap in time, ; and
determining, the likelihood between the respective predefined type of trajectory segment and the one of specified trajectory segments to be zero when the first target instance and the second target instance are determined to overlap in time.
obtaining, by the processing circuitry, a predefined trajectory segment associated with the respective predefined type of trajectory segment, a first target instance associated with the predefined trajectory segment being closest in time to a second target instance associated with the one of the specified trajectory segments (3711);
calculating, by the processing circuitry, a local similarity between the one of the specified trajectory segments and the predefined trajectory segment associated with the respective predefined type of trajectory segment according to local feature information of the one of the specified trajectory segments and local feature information of the predefined trajectory segment associated with the respective predefined type of trajectory segments (3713);
calculating, by the processing circuitry, a global similarity between the respective predefined type of trajectory segment and the one of the specified trajectory segments according to the global feature information of the one of the specified trajectory segments and a type parameter of the respective predefined type of trajectory segment (3715); and
calculating, by the processing circuitry, the likelihood between the respective predefined type of trajectory segment and the one of the specified trajectory segments according to the local similarity and the global similarity (3717).
determining whether an iteration quantity of the clustering satisfies a preset iteration threshold;
updating, by the processing circuitry, the type parameter of the respective predefined type of trajectory segment when the iteration quantity of the clustering is determined not to satisfy the preset iteration threshold; and
calculating, by the processing circuitry, a likelihood between the updated respective predefined type of trajectory segment and the one of the specified trajectory segments according to the feature information of the one of the specified trajectory segments.
a target instance obtaining module (710), configured to: obtain, according to a target detection on image data, a plurality of target instances, each of the plurality of target instances corresponding to one of a plurality of tracked targets;
a trajectory segment obtaining module (730), configured to: determine, a plurality of trajectory segments, each of the plurality of trajectory segments is formed by connecting a subset of the target instances corresponding to a same tracked target of the plurality of tracked targets;
a feature information obtaining module (750), configured to determine, feature information of the plurality of trajectory segments by performing feature construction on the trajectory segment according to the target instances in the trajectory segment;
a type distribution obtaining module (770), configured to perform clustering on specified trajectory segments according to the feature information, to obtain a type distribution of the specified trajectory segments; and
a tracking result obtaining module (790), configured to determine, according to the type distribution of the specified trajectory segments, a target tracking result including a same type of the specified trajectory segments.
a target feature construction unit (751), configured to determine, target feature information of the target instances associated with one of the plurality of trajectory segments by performing target feature construction on the target instances associated with one of the plurality of trajectory segments;
a trajectory segment feature construction unit (753), configured to determine, according to the target feature information, local feature information and global feature information by performing local feature construction and global feature construction on one of the plurality of trajectory segments; and
a feature information defining unit (755), configured to determine, feature information of the one of the plurality of trajectory segments according to the local feature information and the global feature information.
an annotation information identification unit, configured to obtain, one of the plurality of target instances by performing annotation information identification on a plurality of deformable parts of a tracked target of the plurality of tracked targets in the image data by using a pre-created deformable-part model, one of the plurality of deformable parts of the tracked target corresponding to the target instance being identified by using annotation information; and
wherein the target feature construction unit (751) comprises:
a feature vector obtaining subunit (7511), configured to obtain a visual feature vector and a structure feature vector of one of the tracked targets, wherein the visual feature vector being a histogram feature vector that is extracted from the annotation information of the plurality of deformable parts of the tracked target;
the structure feature vector being based on location deviation values between one of the plurality of deformable parts and another one of the plurality of deformable parts of the tracked target; and
a feature information forming subunit (7513), configured to determine, the target feature information of the one of the target instances according to the visual feature vector and the structure feature vector of the tracked target.
a local feature information defining subunit (7531), configured to: determine, the local feature information corresponding to the trajectory segment according to a visual feature vector in target feature information corresponding to the at least one target instances, the at least one target instances being extracted from the plurality of trajectory segments;
a structure feature vector calculation subunit (7533), configured to: determine, an average value of structure feature vectors and a covariance matrix of the structure feature vectors, the structure feature vectors corresponding to the target instances associated with the one of the plurality of trajectory segments; and
a global feature information defining subunit (7535), configured to determine, the global feature information corresponding to the trajectory segment according to the average value and the covariance matrix of the structure feature vectors.
a first likelihood calculation unit (771), configured to calculate, for each of at least one predefined type of trajectory segment, a likelihood between the respective predefined type of trajectory segment and one of the specified trajectory segments according to the feature information of the one of the specified trajectory segments
a probability calculation unit (773), configured to calculate, according to the likelihoods between the at least one predefined type of trajectory segment and the one of the specified trajectory segments, a plurality of probabilities that the one of the specified trajectory segments follows a uniform distribution in the at least one predefined type of trajectory segment;
a clustering unit (775), configured to classify the one of the specified trajectory segments into one of the at least one predefined type of corresponding to a maximum probability in the plurality of probabilities; and
a type distribution forming unit (777), configured to complete clustering of the specified trajectory segments to form the type distribution of the specified trajectory segments.
one or more processors; and
one or more memories, the memory storing computer-readable instructions, and the computer-readable instructions, when executed by the processor, implementing the target tracking method according to any one of claims 1 to 8.
REFERENCES CITED IN THE DESCRIPTION
This list of references cited by the applicant is for the reader's convenience only. It does not form part of the European patent document. Even though great care has been taken in compiling the references, errors or omissions cannot be excluded and the EPO disclaims all liability in this regard.
Patent documents cited in the description