(19)
(11)EP 3 460 385 B1

(12)EUROPEAN PATENT SPECIFICATION

(45)Mention of the grant of the patent:
29.04.2020 Bulletin 2020/18

(21)Application number: 17798552.0

(22)Date of filing:  30.03.2017
(51)International Patent Classification (IPC): 
G01B 11/00(2006.01)
G06T 7/62(2017.01)
G06T 7/11(2017.01)
(86)International application number:
PCT/CN2017/078768
(87)International publication number:
WO 2017/197988 (23.11.2017 Gazette  2017/47)

(54)

METHOD AND APPARATUS FOR DETERMINING VOLUME OF OBJECT

VERFAHREN UND VORRICHTUNG ZUR BESTIMMUNG DES VOLUMENS EINES OBJEKTES

PROCÉDÉ ET APPAREIL DE DÉTERMINATION DE VOLUME D'OBJET


(84)Designated Contracting States:
AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

(30)Priority: 16.05.2016 CN 201610323084

(43)Date of publication of application:
27.03.2019 Bulletin 2019/13

(73)Proprietor: Hangzhou Hikrobot Technology Co., Ltd
Hangzhou, Zhejiang 310051 (CN)

(72)Inventors:
  • ZHANG, Wencong
    Hangzhou Zhejiang 310051 (CN)
  • WU, Kuang
    Hangzhou Zhejiang 310051 (CN)
  • JIA, Yonghua
    Hangzhou Zhejiang 310051 (CN)

(74)Representative: Guérin, Jean-Philippe 
Opilex 32 rue Victor Lagrange
69007 Lyon
69007 Lyon (FR)


(56)References cited: : 
WO-A1-2014/147863
CN-A- 102 564 338
CN-A- 104 330 038
CN-A- 106 839 975
US-A1- 2015 302 594
CN-A- 101 846 503
CN-A- 103 983 334
CN-A- 104 517 095
CN-U- 204 881 572
US-B2- 8 381 976
  
  • GREFF K ET AL: "A COMPARISON BETWEEN BACKGROUND SUBTRACTION ALGORITHMS USING A CONSUMER DEPTH CAMERA", INTERNATIONAL CONFERENCE ON COMPUTER VISION THEORY AND APPLICATIONS (VISAPP 2012); ROME, ITALY; 24 - 26 FEBRUARY, 2012, SCITEPRESS , 24 February 2012 (2012-02-24), pages 431-436, XP002742816, DOI: 10.5220/0003849104310436 Retrieved from the Internet: URL:http://www2.ic.uff.br/~medialab/Andre/ visapp2012.pdf [retrieved on 2015-07-29]
  
Note: Within nine months from the publication of the mention of the grant of the European patent, any person may give notice to the European Patent Office of opposition to the European patent granted. Notice of opposition shall be filed in a written reasoned statement. It shall not be deemed to have been filed until the opposition fee has been paid. (Art. 99(1) European Patent Convention).


Description

TECHNICAL FIELD



[0001] The present application relates to the technical field of machine vision, and in particular to a method and apparatus for determining volume of an object.

BACKGROUND



[0002] As the information about one of the most basic attributes of an object, volume data is widely used in production, logistics and the like, especially in volume-based logistics billing, automatic loading of an object, and so on. The object here means an object that is basically cuboid.

[0003] In the prior art, the common method for determining volume of an object includes a laser-based determination method and a manual scale-based determination method. The laser-based determination method achieves a high accuracy, but is not cost effective due to the need for expensive laser measurement devices, and thus is difficult to be widely used by users. The manual scale-based determination method requires manual cooperation and is susceptible to user's operation and emotion, and as a result, neither accuracy nor efficiency can be guaranteed.

[0004] CN 104330038 A describes a size measurement method. The method comprises steps of acquiring sensing points on an object to be measured through a depth sensor to generate a depth matrix; calculating the maximum height of the object to be measured depending on the depth matrix; and then calculating a minimum bounding rectangle depending on a coordinate set of all sensing points, thus obtaining the maximum length and the maximum width of the object to be measured and realizing rapid measurement of three-dimensional size of the object to be measured. The method is suitable for regular and irregular cargos, and in particular irregular cargos; the method can calculate the maximum length, width and height of cargo, and is applicable to logistics transportation charging and loading operations.

[0005] CN 101846503 A describes a luggage information on-line obtaining system based on stereoscopic vision and a method thereof. An image collecting part is arranged in an image collecting work table in the system, the image collecting part obtains stereoscopic image pairs in an on-line noncontact way when the luggage passes through the image collecting work table; an image processing module is connected with the image collecting part, and is used for feature extracting and matching processing on the stereoscopic image pairs to reconstruct the three-dimensional deep information on the surface of the luggage and calculate the length, width and height dimension of the luggage; the luggage image carries out color and space conversation, the color features of the luggage are extracted and classified to obtain the color information of the luggage; and a server is connected with the image processing module and is used for receiving the dimension and color information of the luggage and the stereoscopic image pairs of the luggage and providing the inquiring service of the luggage information. The method comprises the following steps: carrying out feature extraction and feature matching on the stereoscopic image pairs of the luggage; reconstructing the three-dimensional deep information on the surface of the luggage; calculating the length, width and height dimension of the luggage; carrying out color and space conversation on the luggage images; extracting the color features; carrying out classification; and obtaining the color information of the luggage.

SUMMARY



[0006] Embodiments of the present application aim to provide a method and apparatus for determining volume of an object, so as to achieve a high accuracy, a high efficiency and a lower economic cost in determining the volume of the object. Such a method and apparatus are specified in claim 1 and claim 6 respectively. Further embodiments are specified in the dependent claims.

[0007] In the embodiments of the present application, after obtaining a target depth image containing a target object which is captured by a depth image capturing device, the depth data of the target depth image is segmented to obtain a target image region corresponding to the target object. A target circumscribed rectangle that corresponds to the target image region is determined. The volume of the target object is determined based on the target circumscribed rectangle and the depth data of the target depth image. Compared to the laser-based determination method in the prior art, the present scheme employs a depth image capturing device without a laser measurement device, and thus the economic cost is lower. In addition, compared to the manual scale-based determination method in the prior art, the present scheme employs a software program to automatically determine volume without manual cooperation, leading to higher accuracy and efficiency. It can be seen that the present scheme determines the volume of the object in a high accuracy, high efficiency and cost effective manner.

BRIEF DESCRIPTION OF THE DRAWINGS



[0008] In order to more clearly describe the technical solutions of embodiments of the present application and the prior art, drawings used in the embodiments and the prior art will be briefly described below. Obviously, the drawings described below are for only some embodiments of the present application; those skilled in the art can obtain other drawings based on these drawings without any creative efforts.

Fig. 1 is a flow chart of a method for determining volume of an object provided by an embodiment of the present application;

Fig. 2 is another flow chart of a method for determining volume of an object provided by an embodiment of the present application;

Fig. 3 is another flow chart of a method for determining volume of an object provided by an embodiment of the present application;

Fig. 4 is a schematic diagram of the structure of an apparatus for determining volume of an object provided by an embodiment of the present application;

Fig. 5 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application.


DETAILED DESCRIPTION



[0009] The technical solutions of the embodiments of the application will be described clearly and completely below with reference to the accompanying drawings. Obviously, the embodiments described are merely some of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments herein without any creative efforts shall fall within the protection scope of the application.

[0010] In order to solve the technical problem in the prior art, the embodiments of the present application provide a method and apparatus for determining volume of an object, so as to achieve a high accuracy, a high efficiency and a lower economic cost in determining volume of an object.

[0011] A method for determining volume of an object provided by an embodiment of the present application will be introduced below.

[0012] It should be noted that the method for determining volume of an object provided by the embodiment of the present application may be implemented by an apparatus for determining volume of an object. And in practical applications, the apparatus for determining volume of an object may be functional software arranged in a depth image capturing device, or may be functional software arranged in a backend server in communication with the depth image capturing device. In addition, the object whose volume is to be determined involved in the embodiment of the present application may be: an object, such as a workpiece or a package, having a shape of a basically cuboid in production and logistics.

[0013] As shown in Fig. 1, the method for determining volume of an object provided by an embodiment of the present application, may include:
S101, obtaining a target depth image containing a target object which is captured by a depth image capturing device.

[0014] In the process of determining volume of a target object, the apparatus for determining volume of an object may first obtain a target depth image containing the target object which is captured by a depth image capturing device, and then perform subsequent processes using the obtained target depth image containing the target object. There may be at least one target object in one target depth image. It can be understood that, in a case where the apparatus for determining volume of an object is functional software in the depth image capturing device, this apparatus for determining volume of an object may directly obtain the target depth image captured by the depth image capturing device. In a case where the apparatus for determining volume of an object is functional software in a backend server, this apparatus for determining volume of an object may obtain the target depth image obtained by this backend server from the depth image capturing device, and the manner in which the backend server obtains the target depth image from the depth image capturing device may be active capturing or passive reception.

[0015] It should be noted that, in order to ensure that the depth image capturing device can capture the depth image of the target object, this depth image capturing device may be placed at a position where the depth data of the target object can be captured. In addition, the manner of capturing the target depth image may be a triggered capture manner. The triggered capture manner may be that capturing the depth image is triggered only when a target object to be measured appears in the scene. For example, the manner of triggering a capture may include a manner in which the capture is externally and physically triggered by a photoelectric signal or a manner in which the capture is automatically triggered through intelligent analysis. The manner in which the capture is externally and physically triggered by a photoelectric signal specifically means that a photoelectric signal is interrupted when a target object whose volume needs to be determined passes, and thereby a trigger signal is transmitted to the depth image capturing device. The manner in which the capture is automatically triggered through intelligent analysis means that an automatic detection is performed using a motion detection algorithm to determine whether the target object appears, and then the depth image capturing device can capture the target depth image of the target object when the determination result indicates that it appears.

[0016] Specifically, in an implementation, the depth image capturing device may be a TOF (Time-of-Flight) camera. In this case, obtaining a target depth image containing a target object which is captured by a depth image capturing device, may include: obtaining the target depth image containing the target object which is captured by the TOF (Time-of-Flight) camera. The principle of the TOF camera capturing a depth image is to continuously transmit a light pulse to the target object, receive light returned from the target object using a sensor, and detect the flight time of the light pulse and obtain the distance to the target object. Of course, the depth image capturing device is not limited to the TOF camera. Other devices that may capture the depth image may also be used as the depth image capturing device used in the embodiment of the present application.

[0017] For better understanding of the embodiment of the present application, the depth of the image will be introduced now. A picture is composed of pixels. All the pixels of different colors constitute an entire image. The image is stored in a computer in binary. If 1 bit is used for storage, that is, a pixel is stored with one bit, then this pixel has a value of 0 or 1, and the picture is thus either black or white. If 4 bits are used for storage, then this pixel has a value range from 0 to 24. If 8 bits are used for storage, this pixel has a value range from 0 to 28, and so on. In the prior art, the number of bits used by the computer to store a single pixel is called the depth of an image, and the depth image is a picture that can reflect the depth of the picture.

[0018] It should be noted that the specific implementation of obtaining the target depth image containing the target object which is captured by the depth image capturing device above is only an example and should not be construed as a limitation to the embodiment of the present application.

[0019] S102, performing segmentation based on depth data in the target depth image to obtain a target image region corresponding to the target object.

[0020] To determine the volume of the target object, after obtaining the target depth image containing the target object, the depth data of the target depth image may be segmented to obtain the target image region corresponding to the target object.

[0021] Specifically, in an implementation, performing segmentation based on depth data in the target depth image to obtain the target image region corresponding to the target object, may include:
performing segmentation based on the depth data in the target depth image using a depth image frame difference method to obtain the target image region corresponding to the target object.

[0022] It should be noted that the specific implementation of performing segmentation based on depth data in the target depth image to obtain the target image region corresponding to the target object is only an example and should not be construed as a limitation to the embodiment of the present application. In addition, for a clear layout, the specific implementation of performing segmentation based on the depth data in the target depth image using the depth image frame difference method to obtain the target image region corresponding to the target object will be explained subsequently.

[0023] S103, determining a target circumscribed rectangle that corresponds to the target image region and meets a predetermined condition.

[0024] After the segmentation for obtaining the target image region corresponding to the target object, in order to determine volume of the target object, a target circumscribed rectangle that corresponds to the target image region and meets a predetermined condition may be determined, and subsequent processes are then performed based on the target circumscribed rectangle.

[0025] It can be understood that, in practical applications, a target circumscribed rectangle that corresponds to the target image region and meets a predetermined condition may be determined by a connected region analysis algorithm or an edge detection fitting algorithm without being limited thereto. Specifically, the basic principle of the connected region analysis algorithm is that: the connected regions in the binarized image are marked, the convex hull of each of the connected regions is calculated, and the minimum circumscribed rectangle corresponding to the target object is calculated using the feature of the circumscribed rectangle, having a minimum area, of the convex hull. The feature is that one edge of the convex hull coincides with one edge of the circumscribed rectangle and each edge of the rectangle must contain a vertex of the convex hull. The convex hull is an existing basic concept in computational geometry and is the smallest convex polygon that contains the set of all points in this connected region. The basic principle of the so-called edge detection fitting algorithm is that: the edges of each target image region are fitted using a straight line fitting method, and the circumscribed rectangle of the target image region is calculated according to an edge straight line equation. The straight line fitting method is a common method in the prior art, and mainly includes Hough transform and least-squares fitting. Specifically, the Hough transform is a parameter estimation technique using a voting strategy. The principle is to convert a detection problem in an image space into a parameter space using the point-line duality between the image space and the Hough parameter space.

[0026] The least-squares fitting is defined as that: (xi)2 is the minimum, and the fitting function defined by the standard ni=1 is called the least-squares fitting, which is the best squares approximation in the discrete case. For the given data points {(Xi, Yi)}(i=0, 1, ..., m), in a given function class Φ, the least-squares fitting finds p(x) ∈ Φ that minimizes the sum E^2 of squares of errors, where E^2=Σ [p(Xi)-Yi]^2. Further, from a geometrical point of view, it is to find the curve y=p(x), to which the sum of squares of the distances from the given points {(Xi, Yi)}(i=0,1,...,m) is the least. The function p(x) is called a fitting function or a least-squares solution. The method for finding the fitting function p(x) is called the least-squares method of curve fitting.

[0027] In addition, it should be noted that there may be multiple target circumscribed rectangles corresponding to the target image region. One target circumscribed rectangle meeting a predetermined condition may be obtained from the multiple target circumscribed rectangles, and then the subsequent process of determining volume is performed based on the obtained target circumscribed rectangle meeting the predetermined condition. Based on the above requirements, determining a target circumscribed rectangle that corresponds to the target image region and meets the predetermined condition may include: determining a target circumscribed rectangle that corresponds to the target image region and has a minimum area;
or,
determining a target circumscribed rectangle that corresponds to the target image region and has a minimum difference between its area and a predetermined area threshold.

[0028] The target circumscribed rectangle having a minimum area is the circumscribed rectangle that best fits the edges of the target image region, and therefore may be used for the subsequent process of determining the volume. The target circumscribed rectangle having a minimum difference between its area and the predetermined area threshold is a circumscribed rectangle with the minimum error from the reference circumscribed rectangle, and therefore may also be used for subsequent process of determining the volume. The area of the reference circumscribed rectangle equals to the predetermined area threshold.

[0029] It should be noted that the above specific implementation of determining a target circumscribed rectangle that corresponds to the target image region and meets the predetermined condition is only an example and should not be construed as a limitation to the embodiment of the present application.

[0030] S104, determining volume of the target object based on the target circumscribed rectangle and the depth data of the target depth image.

[0031] After the target circumscribed rectangle is determined, the volume of the target object may be determined by a specific process based on the target circumscribed rectangle and the depth data in the target depth image.

[0032] It should be noted that there are various implementations of determining the volume of the target object based on the target circumscribed rectangle and the depth data in the target depth image. For the clarity of the solution and the layout, the specific implementation of determining volume of the target object based on the target circumscribed rectangle and the depth data in the target depth image will be introduced subsequently by way of example.

[0033] In the embodiment of the present application, after obtaining a target depth image containing a target object which is captured by a depth image capturing device, the depth data in the target depth image is segmented to obtain a target image region corresponding to the target object; a target circumscribed rectangle that corresponds to the target image region and meets a predetermined condition is determined; and the volume of the target object is determined based on the target circumscribed rectangle and the depth data in the target depth image. Compared to the laser-based determination method in the prior art, the present scheme employs a depth image capturing device without a laser measurement device, and thus the economic cost is lower. In addition, compared to the manual scale-based determination method in the prior art, the present scheme employs a software program to automatically determine volume without user's cooperation, and thus has higher accuracy and efficiency. It can be seen that the present scheme achieves a high accuracy, a high efficiency and a lower economic cost in determining volume of an object.

[0034] The specific implementation of performing segmentation based on the depth data in the target depth image using a depth image frame difference method to obtain the target image region corresponding to the target object will be described in detail below.

[0035] As shown in Fig. 2, performing segmentation based on the depth data in the target depth image using a depth image frame difference method to obtain a target image region corresponding to the target object (S102) may include:
S1021, obtaining a difference between the depth data of each of pixels in the target depth image and the depth data of a corresponding pixel in a predetermined background depth image.

[0036] The predetermined background depth image is an image of the background environment where the target object is located, which is captured in advance by a depth image capturing device and does not contain the target object.

[0037] S 1022, generating a frame difference image corresponding to the target depth image based on the difference for each of the pixels.

[0038] S1023, binarizing the frame difference image.

[0039] S1024, obtaining the target image region corresponding to the target object from the binarized frame difference image.

[0040] Specifically, obtaining a difference between the depth data of each of pixels in the target depth image and the depth data of a corresponding pixel in the predetermined background depth image means: for each pixel in the target depth image, obtaining the difference between the depth data of the pixel and the depth data of a corresponding pixel in the predetermined background depth image. For example, obtaining the difference between the pixel 1 in the target depth image and the corresponding pixel 2 in the predetermined background depth image may be: subtracting the value of the corresponding pixel 2 from the value of the pixel 1.

[0041] Specifically, assuming that the two values of the binarization are 0 and 1, binarizing the frame difference image is: comparing the absolute value of each of the pixels in the frame difference image with a predetermined threshold, and if the absolute value is greater than the threshold, changing the pixel value of the pixel into 1, otherwise, changing the pixel value of the pixel into 0. Alternatively, if the absolute value is greater than the threshold, the pixel value of the pixel may be changed into 0, otherwise, the pixel value of the pixel may be changed into 1. Through such processes, the pixel value of each pixel in the target image region, corresponding to the target object, in the frame difference image is different from the pixel value of each pixel outside the target image region. The target image region corresponding to this target object can then be obtained from the binarized frame difference image. Of course, the two values of the binarization may also be 0 and 255. In this case, the specific process of binarizing the frame difference image is similar to that described with respect to the values of 0 and 1, and will not be described again.

[0042] The specific implementation of determining volume of the target object based on the target circumscribed rectangle and the depth data of the target depth image will be explained below by way of example. Of course, this specific implementation is only an example and should not be construed as a limitation to the embodiment of the present application.

[0043] As shown in Fig. 3, determining volume of the target object based on the target circumscribed rectangle and the depth data of the target depth image may include:

S1041, extracting image coordinates of each vertex of the target circumscribed rectangle in the binarized frame difference image;

S1042, projecting the extracted image coordinates of each vertex into the target depth image to generate a reference point located in the target depth image;

S1043, calculating three-dimensional coordinates for each reference point in a camera world coordinate system according to a principle of perspective projection in camera imaging; and

S1044, obtaining the volume of the target object by using the three-dimensional coordinates of the reference points and the depth data of the target depth image.



[0044] It can be understood that the frame difference image corresponds to a two-dimensional coordinate system, and therefore, the image coordinates of each vertex of the target circumscribed rectangle in the binarized frame difference image may be extracted. In addition, since the frame difference image is determined based on the target depth image, the dimension of the frame difference image is the same as that of the target depth image, and the two-dimensional coordinate system for the frame difference image is thus the same as that for the target depth image. Therefore, the image coordinates of a reference point located in the target depth image are the same as the image coordinates of a corresponding vertex located in the binarized frame difference image.

[0045] Technologies in prior art may be used to calculate three-dimensional coordinates for each reference point in a camera world coordinate system according to the principle of perspective projection in camera imaging, and will not be described herein.

[0046] The specific implementation of obtaining the volume of the target object by using the three-dimensional coordinates of the reference points and the depth data of the target depth image may include: calculating Euclidean distances between every two of 4 reference points; determining the length and width of the target object based on the calculated Euclidean distances; subtracting the Z value of the target object from the Z value corresponding to the predetermined background depth image, to obtain the height of the target object; and determining the product of the determined length, width and height of the target object as the volume of the target object. The Z value of the target object is the Z value of a region corresponding to the four reference points, i.e., the depth value. The Z value corresponding to the predetermined background depth image is a depth value. It should be noted that since the pattern formed by the four reference points as the vertexes is a rectangle, lines between every two of the four reference points includes the diagonal lines of the rectangle. That is, the Euclidean distance between reference points at two ends of each diagonal line is included in the Euclidean distances between every two of the four reference points. For this reason, when determining the length and width of the target object based on the calculated Euclidean distances, the Euclidean distance between reference points at two ends of each diagonal line should be removed firstly, that is, the maximum Euclidean distances are removed. The remaining Euclidean distances are determined as the length and width of the target object.

[0047] Of course, in calculating the length and width of the target object, one reference point may alternatively be selected as the target reference point. The Euclidean distances between the target reference point and the other three reference points are calculated. The two Euclidean distances whose values are smaller are determined as the length and width of the target object.

[0048] Corresponding to the above method embodiment, an embodiment of the present application further provides an apparatus for determining volume of an object. As shown in Fig. 4, the apparatus may include:

a depth image obtaining module 410, configured for obtaining a target depth image containing a target object which is captured by a depth image capturing device;

an image region segmentation module 420, configured for performing segmentation based on depth data in the target depth image to obtain a target image region corresponding to the target object;

a circumscribed rectangle determination module 430, configured for determining a target circumscribed rectangle that corresponds to the target image region and meets a predetermined condition; and

a volume determination module 440, configured for determining volume of the target object based on the target circumscribed rectangle and the depth data in the target depth image.



[0049] In the embodiment of the present application, after obtaining the target depth image containing the target object which is captured by a depth image capturing device, the depth data in the target depth image is segmented to obtain a target image region corresponding to the target object. A target circumscribed rectangle that corresponds to the target image region and meets a predetermined condition is determined. The volume of the target object is then determined based on the target circumscribed rectangle and the depth data of this target depth image. Compared to the laser-based determination method in the prior art, the present scheme employs a depth image capturing device without a laser measurement device, and thus the economic cost is lower. In addition, compared to the manual scale-based determination method in the prior art, the present scheme employs a software program to automatically determine volume without manual cooperation, and has a higher accuracy and efficiency. It can be seen that the present scheme achieves a high accuracy, a high efficiency and a lower economic cost in determining volume of an object.

[0050] The depth image obtaining module 410 may include:
a depth image obtaining unit, configured for obtaining the target depth image containing the target object which is captured by a time-of-flight camera.

[0051] The image region segmentation module 420 may include:
an image region segmentation unit, configured for performing segmentation based on the depth data in the target depth image using a depth image frame difference method to obtain the target image region corresponding to the target object.

[0052] Further, specifically, the image region segmentation unit may include:

a subtraction sub-unit, configured for obtaining a difference between the depth data of each of pixels in the target depth image and the depth data of a corresponding pixel in a predetermined background depth image; wherein the predetermined background depth image is an image of the background environment where the target object is located, which is captured in advance by the depth image capturing device and does not contain the target object;

a frame difference image generating sub-unit, configured for generating a frame difference image corresponding to the target depth image based on the difference for each of the pixels;

a binarization sub-unit, configured for binarizing the frame difference image; and

an image region segmentation sub-unit, configured for obtaining the target image region corresponding to the target object from the binarized frame difference image.



[0053] The circumscribed rectangle determination module 430 may include:
a first circumscribed rectangle determination unit, configured for determining the target circumscribed rectangle that corresponds to the target image region and meets a predetermined condition by using a connected region analysis algorithm or an edge detection fitting algorithm.

[0054] The circumscribed rectangle determination module 430 may include:

a second circumscribed rectangle determination unit, configured for determining a target circumscribed rectangle that corresponds to the target image region and has a minimum area;

or,

a third circumscribed rectangle determination unit, configured for determining a target circumscribed rectangle that corresponds to the target image region and has a minimum difference between its area and a predetermined area threshold.



[0055] The volume determination module 440 may include:

an image coordinate extracting unit, configured for extracting image coordinates of each vertex of the target circumscribed rectangle in the binarized frame difference image;

a reference point generating unit, configured for projecting the extracted image coordinates of each vertex into the target depth image to generate a reference point located in the target depth image;

a three-dimensional coordinate calculation unit, configured for calculating three-dimensional coordinates for each reference point in a camera world coordinate system according to a principle of perspective projection in camera imaging;

a volume determination unit, configured for obtaining the volume of the target object by using the three-dimensional coordinates of the reference points and the depth data of the target depth image.



[0056] Corresponding to the above method embodiment, an embodiment of the present application further provides a storage medium storing executable program codes, which are executed to carry out the method for determining volume of an object provided by the embodiments of the present application. Specifically, the method for determining volume of an object may include:

obtaining a target depth image containing a target object which is captured by a depth image capturing device;

performing segmentation based on depth data in the target depth image to obtain a target image region corresponding to the target object;

determining a target circumscribed rectangle that corresponds to the target image region and meets a predetermined condition; and

determining volume of the target object based on the target circumscribed rectangle and the depth data of the target depth image.



[0057] Optionally, obtaining a target depth image containing a target object which is captured by a depth image capturing device, includes:
obtaining the target depth image containing the target object which is captured by a time-of-flight camera.

[0058] Optionally, performing segmentation based on depth data in the target depth image to obtain a target image region corresponding to the target object, includes:
performing segmentation based on the depth data in the target depth image using a depth image frame difference method to obtain the target image region corresponding to the target object;

[0059] Optionally, performing segmentation based on the depth data in the target depth image using a depth image frame difference method to obtain the target image region corresponding to the target object, includes:

obtaining a difference between the depth data of each of pixels in the target depth image and the depth data of a corresponding pixel in a predetermined background depth image; wherein the predetermined background depth image is an image of the background environment where the target object is located, which is captured in advance by the depth image capturing device and does not contain the target object;

generating a frame difference image corresponding to the target depth image based on the difference for each of the pixels;

binarizing the frame difference image; and

obtaining the target image region corresponding to the target object from the binarized frame difference image.



[0060] Optionally, determining a target circumscribed rectangle that corresponds to the target image region and meets a predetermined condition, includes:
determining the target circumscribed rectangle that corresponds to the target image region and meets a predetermined condition by using a connected region analysis algorithm or an edge detection fitting algorithm.

[0061] Optionally, determining a target circumscribed rectangle that corresponds to the target image region and meets a predetermined condition, includes:

determining a target circumscribed rectangle that corresponds to the target image region and has a minimum area;

or,

determining a target circumscribed rectangle that corresponds to the target image region and has a minimum difference between its area and a predetermined area threshold.



[0062] Optionally, determining volume of the target object based on the target circumscribed rectangle and the depth data of the target depth image, includes:

extracting image coordinates of each vertex of the target circumscribed rectangle in the binarized frame difference image;

projecting the extracted image coordinates of each vertex into the target depth image to generate a reference point located in the target depth image;

calculating three-dimensional coordinates for each reference point in a camera world coordinate system according to a principle of perspective projection in camera imaging; and

obtaining the volume of the target object by using the three-dimensional coordinates of the reference points and the depth data of the target depth image.



[0063] In this embodiment, the storage medium stores the executable codes, which is executed to carry out the method for determining volume of an object provided by the embodiments of the present application, thereby being capable of achieving a high accuracy, a high efficiency and a lower economic cost in determining volume of an object.

[0064] Corresponding to the above method embodiment, an embodiment of the present application further provides an application program, which is executed to carry out the method for determining volume of an object provided by the embodiments of the present application. Specifically, the method for determining volume of an object may include:

obtaining a target depth image containing a target object which is captured by a depth image capturing device;

performing segmentation based on depth data in the target depth image to obtain a target image region corresponding to the target object;

determining a target circumscribed rectangle that corresponds to the target image region and meets a predetermined condition; and

determining volume of the target object based on the target circumscribed rectangle and the depth data of the target depth image.



[0065] Optionally, obtaining a target depth image containing a target object which is captured by a depth image capturing device, includes:
obtaining the target depth image containing the target object which is captured by a time-of-flight camera.

[0066] Optionally, performing segmentation based on depth data in the target depth image to obtain a target image region corresponding to the target object, includes:
performing segmentation based on the depth data in the target depth image using a depth image frame difference method to obtain the target image region corresponding to the target object.

[0067] Optionally, performing segmentation based on the depth data in the target depth image using a depth image frame difference method to obtain the target image region corresponding to the target object, includes:

obtaining a difference between the depth data of each of pixels in the target depth image and the depth data of a corresponding pixel in a predetermined background depth image; wherein the predetermined background depth image is an image of the background environment where the target object is located, which is captured in advance by the depth image capturing device and does not contain the target object;

generating a frame difference image corresponding to the target depth image based on the difference for each of the pixels;

binarizing the frame difference image; and

obtaining the target image region corresponding to the target object from the binarized frame difference image.



[0068] Optionally, determining a target circumscribed rectangle that corresponds to the target image region and meets a predetermined condition, includes:
determining the target circumscribed rectangle that corresponds to the target image region and meets a predetermined condition by using a connected region analysis algorithm or an edge detection fitting algorithm.

[0069] Optionally, determining a target circumscribed rectangle that corresponds to the target image region and meets a predetermined condition, includes:

determining a target circumscribed rectangle that corresponds to the target image region and has a minimum area;

or,

determining a target circumscribed rectangle that corresponds to the target image region and has a minimum difference between its area and a predetermined area threshold.



[0070] Optionally, determining volume of the target object based on the target circumscribed rectangle and the depth data of the target depth image, includes:

extracting image coordinates of each vertex of the target circumscribed rectangle in the binarized frame difference image;

projecting the extracted image coordinates of each vertex into the target depth image to generate a reference point located in the target depth image;

calculating three-dimensional coordinates for each reference point in a camera world coordinate system according to a principle of perspective projection in camera imaging; and

obtaining the volume of the target object by using the three-dimensional coordinates of the reference points and the depth data of the target depth image.



[0071] In this embodiment, the application program, when executed, carries out the method for determining volume of an object provided by the embodiments of the present application, thereby being capable of achieving a high accuracy, a high efficiency and a lower economic cost in determining volume of an object.

[0072] Corresponding to the above method embodiment, an embodiment of the present application further provides an electronic device. The electronic device includes: a housing 510, a processor 520, a memory 530, a circuit board 540, and a power circuit 550. The circuit board 550 is arranged inside a space surrounded by the housing 510. The processor 520 and the memory 530 are arranged on the circuit board 540. The power circuit 540 is used for supplying power to various circuits or components. The memory 530 is used for storing executable program codes. The processor 520 carries out the method for determining volume of an object provided by the embodiments of the present application by executing the executable program codes stored in the memory. The method for determining volume of an object may include:

obtaining a target depth image containing a target object which is captured by a depth image capturing device;

performing segmentation based on depth data in the target depth image to obtain a target image region corresponding to the target object;

determining a target circumscribed rectangle that corresponds to the target image region and meets a predetermined condition; and

determining volume of the target object based on the target circumscribed rectangle and the depth data of the target depth image.



[0073] The electronic device may be a depth image capturing device or a backend server in communication with the depth image capturing device.

[0074] Optionally, obtaining a target depth image containing a target object which is captured by a depth image capturing device, includes:
obtaining the target depth image containing the target object which is captured by a time-of-flight camera.

[0075] Optionally, performing segmentation based on depth data in the target depth image to obtain a target image region corresponding to the target object, includes:
performing segmentation based on the depth data in the target depth image using a depth image frame difference method to obtain the target image region corresponding to the target object;

[0076] Optionally, performing segmentation based on the depth data in the target depth image using a depth image frame difference method to obtain the target image region corresponding to the target object, includes:

obtaining a difference between the depth data of each of pixels in the target depth image and the depth data of a corresponding pixel in a predetermined background depth image; wherein the predetermined background depth image is an image of the background environment where the target object is located, which is captured in advance by the depth image capturing device and does not contain the target object;

generating a frame difference image corresponding to the target depth image based on the difference for each of the pixels;

binarizing the frame difference image; and

obtaining the target image region corresponding to the target object from the binarized frame difference image.



[0077] Optionally, determining a target circumscribed rectangle that corresponds to the target image region and meets a predetermined condition, includes:
determining the target circumscribed rectangle that corresponds to the target image region and meets a predetermined condition by using a connected region analysis algorithm or an edge detection fitting algorithm.

[0078] Optionally, determining a target circumscribed rectangle that corresponds to the target image region and meets a predetermined condition, includes:

determining a target circumscribed rectangle that corresponds to the target image region and has a minimum area;

or,

determining a target circumscribed rectangle that corresponds to the target image region and has a minimum difference between its area and a predetermined area threshold.



[0079] Optionally, determining volume of the target object based on the target circumscribed rectangle and the depth data of the target depth image, includes:

extracting image coordinates of each vertex of the target circumscribed rectangle in the binarized frame difference image;

projecting the extracted image coordinates of each vertex into the target depth image to generate a reference point located in the target depth image;

calculating three-dimensional coordinates for each reference point in a camera world coordinate system according to a principle of perspective projection in camera imaging; and

obtaining the volume of the target object by using the three-dimensional coordinates of the reference points and the depth data of the target depth image.



[0080] In this embodiment, the processor of the electronic device reads the executable program codes stored in the memory and then carries out a program corresponding to the executable program codes. The program, when being executed, carries out the method for determining volume of an object provided by the embodiments of the present application, thereby achieving a high accuracy, a high efficiency and a lower economic cost in determining volume of the object.

[0081] It should be emphasized that the embodiments of the electronic device, the application program and the storage medium are described briefly, since the operations involved therein are substantially similar to the foregoing embodiments of the method. The related contents can refer to the description of the embodiments of the method.

[0082] It should be noted that the relationship terms used herein, such as "first", "second" and the like are only used to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply that there is an actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise", or any variants thereof are intended to cover a non-exclusive inclusion, such that processes, methods, articles, or devices, including a series of elements, include not only those elements that have been listed, but also other elements that is not specifically listed or the elements intrinsic to these processes, methods, articles, or devices. Without further limitations, elements defined by the wording "comprise(s) a/an..." do not exclude additional identical elements in the processes, methods, articles, or devices that includes the listed elements.

[0083] All of the embodiments herein are described in a correlated manner, and identical or similar parts in the embodiments can refer to one another. In addition, the description for each embodiment focuses on the differences from other embodiments. In particular, the embodiment of the device is described briefly, since it is substantially similar to the embodiment of the method, and the related contents can refer to the description of the embodiment of the method.

[0084] The embodiments described above are simply preferred embodiments of the invention, which is defined by the appended claims.


Claims

1. A method for determining volume of an object, comprising:

obtaining a target depth image containing a target object which is captured by a depth image capturing device (S101);

performing segmentation based on depth data in the target depth image to obtain a target image region corresponding to the target object (S102);

determining a target circumscribed rectangle that corresponds to the target image region and meets a predetermined condition (S103), which comprises:

determining a target circumscribed rectangle that corresponds to the target image region and has a minimum area, or,

determining a target circumscribed rectangle that corresponds to the target image region and has a minimum difference between its area and a predetermined area threshold;

extracting image coordinates of each vertex of the target circumscribed rectangle in a binarized frame difference image (S1041), wherein the frame difference image is an image that is obtained based on the difference between the depth data of the target depth image and depth data of a predetermined background depth image and characterised by

projecting the extracted image coordinates of each vertex into the target depth image to generate a reference point located in the target depth image (S1042);

calculating three-dimensional coordinates for each reference point in a camera world coordinate system according to a principle of perspective projection in camera imaging (S1043); and

with the three-dimensional coordinates of the reference points, calculating Euclidean distance between every two of the reference points, determining a length and a width of the target object as two distances other than the longest distance in the calculated Euclidean distances, subtracting a depth value of a region corresponding to the reference points from a depth value of the predetermined background depth image to obtain a height of the target object, and determining a volume of the target object as the product of the length, width and height of the target object.


 
2. The method of claim 1, wherein obtaining a target depth image containing a target object which is captured by a depth image capturing device (S101), comprises:
obtaining the target depth image containing the target object which is captured by a time-of-flight camera.
 
3. The method of claim 1, wherein performing segmentation based on depth data in the target depth image to obtain a target image region corresponding to the target object (S102), comprises:
performing segmentation based on the depth data in the target depth image using a depth image frame difference method to obtain the target image region corresponding to the target object.
 
4. The method of claim 3, wherein performing segmentation based on the depth data in the target depth image using a depth image frame difference method to obtain the target image region corresponding to the target object, comprises:

obtaining a difference between the depth data of each of pixels in the target depth image and the depth data of a corresponding pixel in the predetermined background depth image (S1021); wherein the predetermined background depth image is an image of the background environment where the target object is located, which is captured in advance by the depth image capturing device and does not contain the target object;

generating a frame difference image corresponding to the target depth image based on the difference for each of the pixels (S1022);

binarizing the frame difference image (S1023); and

obtaining the target image region corresponding to the target object from the binarized frame difference image (S1024).


 
5. The method of claim 1, wherein determining a target circumscribed rectangle that corresponds to the target image region and meets a predetermined condition (S103), comprises:
determining the target circumscribed rectangle that corresponds to the target image region and meets the predetermined condition by using a connected region analysis algorithm or an edge detection fitting algorithm.
 
6. An apparatus for determining volume of an object, comprising:

a depth image obtaining module (410), configured for obtaining a target depth image containing a target object which is captured by a depth image capturing device (S101);

an image region segmentation module (420), configured for performing segmentation based on depth data in the target depth image to obtain a target image region corresponding to the target object (S102);

a circumscribed rectangle determination module (430), configured for determining a target circumscribed rectangle that corresponds to the target image region and meets a predetermined condition (S103), which comprises:

determining a target circumscribed rectangle that corresponds to the target image region and has a minimum area, or,

determining a target circumscribed rectangle that corresponds to the target image region and has a minimum difference between its area and a predetermined area threshold; and

a volume determination module (440), configured for:
extracting image coordinates of each vertex of the target circumscribed rectangle in a binarized frame difference image (S1041), wherein the frame difference image is an image that is obtained based on the difference between the depth data of the target depth image and

depth data of a predetermined background depth image and characterised by

projecting the extracted image coordinates of each vertex into the target depth image to generate a reference point located in the target depth image (S1042);

calculating three-dimensional coordinates for each reference point in a camera world coordinate system according to a principle of perspective projection in camera imaging (S1043); and

with the three-dimensional coordinates of the reference points, calculating Euclidean distance between every two of the reference points, determining a length and a width of the target object as two distances other than the longest distance in the calculated Euclidean distances, subtracting a depth value of a region corresponding to the reference points from a depth value of the predetermined background depth image to obtain a height of the target object, and determining a volume of the target object as the product of the length, width and height of the target object.


 
7. The apparatus of claim 6, wherein the depth image obtaining module (410) comprises:
a depth image obtaining unit, configured for obtaining the target depth image containing the target object which is captured by a time-of-flight camera.
 
8. The apparatus of claim 6, wherein the image region segmentation module (420) comprises:
an image region segmentation unit, configured for performing segmentation based on the depth data in the target depth image using a depth image frame difference method to obtain the target image region corresponding to the target object.
 
9. The apparatus of claim 8, wherein the image region segmentation unit comprises:

a subtraction sub-unit, configured for obtaining a difference between the depth data of each of pixels in the target depth image and the depth data of a corresponding pixel in the predetermined background depth image (S1021); wherein the predetermined background depth image is an image of the background environment where the target object is located, which is captured in advance by the depth image capturing device and does not contain the target object;

a frame difference image generating sub-unit, configured for generating a frame difference image corresponding to the target depth image based on the difference for each of the pixels (S1022);

a binarization sub-unit, configured for binarizing the frame difference image (S1023); and

an image region segmentation sub-unit, configured for obtaining the target image region corresponding to the target object from the binarized frame difference image (S1024).


 
10. The apparatus of claim 6, wherein the circumscribed rectangle determination module (430) comprises:
a first circumscribed rectangle determination unit, configured for determining the target circumscribed rectangle that corresponds to the target image region and meets the predetermined condition by using a connected region analysis algorithm or an edge detection fitting algorithm.
 
11. A storage medium for storing executable program codes, which are executed to carry out the method for determining volume of an object of any of claims 1-5.
 
12. An application program executed to carry out the method for determining volume of an object of any of claims 1-5.
 
13. An electronic device comprising: a housing, a processor, a memory, a circuit board, and a power circuit; wherein the circuit board is arranged inside a space surrounded by the housing; the processor and the memory are arranged on the circuit board; the power circuit is used for supplying power to various circuits or components; the memory is used for storing executable program codes; and the processor carries out the method for determining volume of an object of any of claims 1-5 by executing the executable program codes stored in the memory.
 


Ansprüche

1. Verfahren zur Bestimmung des Volumens eines Objekts, umfassend:

Erhalten eines Ziel-Tiefenbilds, das ein Zielobjekt enthält, das durch eine Tiefenbildaufnahmevorrichtung (S101) erfasst wird;

Durchführen einer Segmentierung auf der Basis der Tiefendaten in dem Ziel-Tiefenbild, um einen Ziel-Bildbereich zu erhalten, der dem Zielobjekt (S102) entspricht;

Bestimmen eines umschriebenen Rechtecks des Ziels, das dem Ziel-Bildbereich entspricht und eine vorbestimmte Bedingung (S103) erfüllt, folgendes umfassend:

Bestimmen eines umschriebenen Rechtecks des Ziels, das dem Ziel-Bildbereich entspricht und eine Mindestfläche aufweist, oder

Bestimmen eines umschriebenen Rechtecks des Ziels, das dem Ziel-Bildbereich entspricht und eine Mindestdifferenz zwischen dessen Fläche und einem vorbestimmten Flächenschwellenwert aufweist;

Extrahieren der Bildkoordinaten jedes Eckpunkts des umschriebenen Rechtecks des Ziels in einem binarisierten Frame-Differenzbild (S1041), wobei das Frame-Differenzbild ein Bild ist, das erhalten wird auf der Basis der Differenz zwischen den Tiefendaten des Ziel-Tiefenbilds und den Tiefendaten eines vorbestimmten Hintergrund-Tiefenbilds, und gekennzeichnet durch

Projizieren der extrahierten Bildkoordinaten jedes Eckpunktes in das Ziel-Tiefenbild, um einen Referenzpunkt zu erzeugen, der sich in dem Ziel-Tiefenbild befindet (S1042);

Berechnen dreidimensionaler Koordinaten für jeden Referenzpunkt in einem Kamera-Weltkoordinatensystem gemäß einem Prinzip der entsprechenden Projektion in der Kamera-Bildgebung (S1043); und

mithilfe der dreidimensionalen Koordinaten der Referenzpunkte, Berechnen des euklidischen Abstands zwischen jeden zwei Referenzpunkten, Bestimmen einer Länge und einer Breite des Zielobjekts als zwei Abstände mit Ausnahme des längsten Abstands der euklidischen bestimmten Abstände, Subtrahieren eines Tiefenwerts eines Bereichs entsprechend den Referenzpunkten von einem Tiefenwert des vorbestimmten Hintergrund-Tiefenbilds, um eine Höhe des Zielobjekts zu erhalten, und Bestimmen eines Volumens des Zielobjekts als das Produkt der Länge, Breite und Höhe des Zielobjekts.


 
2. Verfahren nach Anspruch 1, wobei das Erhalten eines Ziel-Tiefenbilds, das ein Zielobjekt enthält, das durch eine Tiefenbildaufnahmevorrichtung (S101) erfasst wird, folgendes umfasst:
Erhalten des Ziel-Tiefenbilds, welches das Zielobjekt enthält, durch eine Laufzeitkamera.
 
3. Verfahren nach Anspruch 1, wobei das Durchführen einer Segmentierung auf der Basis der Tiefendaten in dem Ziel-Tiefenbild, um einen Ziel-Bildbereich zu erhalten, der dem Zielobjekt (S102) entspricht, folgendes umfasst:
Durchführen einer Segmentierung auf der Basis der Tiefendaten in dem Ziel-Tiefenbild unter Verwendung eines Tiefenbild-Frame-Differenzverfahrens, um den Ziel-Bildbereich zu erhalten, der dem Zielobjekt entspricht.
 
4. Verfahren nach Anspruch 3, wobei das Durchführen einer Segmentierung auf der Basis der Tiefendaten in dem Ziel-Tiefenbild unter Verwendung eines Tiefenbild-Frame-Differenzverfahrens, um den Ziel-Bildbereich zu erhalten, der dem Zielobjekt entspricht, folgendes umfasst:

Erhalten einer Differenz zwischen den Tiefendaten jedes Pixels in dem Ziel-Tiefenbild und den Tiefendaten eines entsprechenden Pixels in dem vorbestimmten Hintergrund-Tiefenbild (S1021); wobei das vorbestimmte Hintergrund-Tiefenbild ein Bild der Hintergrundumgebung ist, wo sich das Zielobjekt befindet, das vorab erfasst wird durch die Tiefenbildaufnahmevorrichtung und das das Zielobjekt nicht enthält;

Erzeugen eines dem Ziel-Tiefenbild entsprechenden Frame-Differenzbilds auf der Basis der Differenz für jedes der Pixel (S1022);

Binarisieren des Frame-Differenzbilds (S1023); und

Erhalten des Ziel-Bildbereichs, der dem Zielobjekt entspricht, aus dem binarisierten Frame-Differenzbild (S1024).


 
5. Verfahren nach Anspruch 1, wobei das Bestimmen eines umschriebenen Rechtecks des Ziels, das dem Ziel-Bildbereich entspricht und eine vorbestimmte Bedingung (S103) erfüllt, folgendes umfasst:
Bestimmen des umschriebenen Rechtecks des Ziels, das dem Ziel-Bildbereich entspricht und die vorbestimmte Bedingung erfüllt, unter Verwendung eines Analysealgorithmus verbundener Bereich oder eines Kantenerkennungsanpassungsalgorithmus.
 
6. Vorrichtung zur Bestimmung des Volumens eines Objekts, umfassend:

ein Tiefenbild-Erhaltungsmodul (410), das gestaltet ist zum Erhalten eines Ziel-Tiefenbilds, das ein Zielobjekt enthält, das durch eine Tiefenbildaufnahmevorrichtung (S101) erfasst wird;

ein Bildbereichs-Segmentierungsmodul (420), das gestaltet ist zum Durchführen einer Segmentierung auf der Basis der Tiefendaten in dem Ziel-Tiefenbild, um einen Ziel-Bildbereich zu erhalten, der dem Zielobjekt (S102) entspricht;

ein Bestimmungsmodul für ein umschriebenes Rechteck (430), das gestaltet ist zum Bestimmen eines umschriebenen Rechtecks des Ziels, das dem Ziel-Bildbereich entspricht und eine vorbestimmte Bedingung (S103) erfüllt, folgendes umfassend:

Bestimmen eines umschriebenen Rechtecks des Ziels, das dem Ziel-Bildbereich entspricht und eine Mindestfläche aufweist, oder

Bestimmen eines umschriebenen Rechtecks des Ziels, das dem Ziel-Bildbereich entspricht und eine Mindestdifferenz zwischen dessen Fläche und einem vorbestimmten Flächenschwellenwert aufweist; und

ein Volumenbestimmungsmodul (440), das gestaltet ist zum:

Extrahieren der Bildkoordinaten jedes Eckpunkts des umschriebenen Rechtecks des Ziels in einem binarisierten Frame- Differenzbild (S1041), wobei das Frame-Differenzbild ein Bild ist, das erhalten wird auf der Basis der Differenz zwischen den Tiefendaten des Ziel-Tiefenbilds und den Tiefendaten eines vorbestimmten Hintergrund-Tiefenbilds, und gekennzeichnet durch

Projizieren der extrahierten Bildkoordinaten jedes Eckpunktes in das Ziel-Tiefenbild, um einen Referenzpunkt zu erzeugen, der sich in dem Ziel-Tiefenbild befindet (S1042);

Berechnen dreidimensionaler Koordinaten für jeden Referenzpunkt in einem Kamera-Weltkoordinatensystem gemäß einem Prinzip der entsprechenden Projektion in der Kamera-Bildgebung (S1043); und

mithilfe der dreidimensionalen Koordinaten der Referenzpunkte, Berechnen des euklidischen Abstands zwischen jeden zwei Referenzpunkten, Bestimmen einer Länge und einer Breite des Zielobjekts als zwei Abstände mit Ausnahme des längsten Abstands der euklidischen bestimmten Abstände, Subtrahieren eines Tiefenwerts eines Bereichs entsprechend den Referenzpunkten von einem Tiefenwert des vorbestimmten Hintergrund-Tiefenbilds, um eine Höhe des Zielobjekts zu erhalten, und Bestimmen eines Volumens des Zielobjekts als das Produkt der Länge, Breite und Höhe des Zielobjekts.


 
7. Vorrichtung nach Anspruch 6, wobei das Tiefenbild-Erhaltungsmodul (410) folgendes umfasst:
eine Tiefenbild-Erhaltungseinheit, die gestaltet ist zum Erhalten eines Ziel-Tiefenbilds, welches das Zielobjekt enthält, das durch eine Laufzeitkamera erfasst wird;
 
8. Vorrichtung nach Anspruch 6, wobei das Bildbereichs-Segmentierungsmodul (420) folgendes umfasst:
eine Bildbereichs-Segmentierungseinheit, die gestaltet ist zum Durchführen einer Segmentierung auf der Basis der Tiefendaten in dem Ziel-Tiefenbild unter Verwendung eines Tiefenbild-Frame-Differenzverfahrens, um den Ziel-Bildbereich zu erhalten, der dem Zielobjekt entspricht.
 
9. Vorrichtung nach Anspruch 8, wobei die Bildbereichs-Segmentierungseinheit folgendes umfasst:

eine Subtraktions-Teileinheit, die gestaltet ist zum Erhalten einer Differenz zwischen den Tiefendaten jedes Pixels in dem Ziel-Tiefenbild und den Tiefendaten eines entsprechenden Pixels in dem vorbestimmten Hintergrund-Tiefenbild (S1021); wobei das vorbestimmte Hintergrund-Tiefenbild ein Bild der Hintergrundumgebung ist, wo sich das Zielobjekt befindet, das vorab erfasst wird durch die Tiefenbildaufnahmevorrichtung und das das Zielobjekt nicht enthält;

eine Frame-Differenzbilderzeugungs-Teileinheit, die gestaltet ist zum Erzeugen eines dem Ziel-Tiefenbild entsprechenden Frame-Differenzbilds auf der Basis der Differenz für jedes der Pixel (S1022);

eine Binarisierungs-Teileinheit, die gestaltet ist zum Binarisieren des Frame-Differenzbilds (S1023); und

eine Bildbereichssegmentierungs-Teileinheit, die gestaltet ist zum Erhalten des Ziel-Bildbereichs, der dem Zielobjekt entspricht, aus dem binarisierten Frame-Differenzbild (S1024).


 
10. Vorrichtung nach Anspruch 6, wobei das Bestimmungsmodul für ein umschriebenes Rechteck (430) folgendes umfasst:
Eine erste Bestimmungseinheit für ein umschriebenes Rechteck, die gestaltet ist zum Bestimmen des umschriebenen Rechtecks des Ziels, das dem Ziel-Bildbereich entspricht und die vorbestimmte Bedingung erfüllt, unter Verwendung eines Analysealgorithmus verbundener Bereich oder eines Kantenerkennungsanpassungsalgorithmus.
 
11. Speichermedium zum Speichern ausführbarer Programmcodes, die ausgeführt werden, um das Verfahren zur Bestimmung des Volumens eines Objekts nach einem der Ansprüche 1 bis 5 auszuführen.
 
12. Anwendungsprogramm, das ausgeführt wird, um das Verfahren zur Bestimmung des Volumens eines Objekts nach einem der Ansprüche 1 bis 5 auszuführen.
 
13. Elektronische Vorrichtung, die folgendes umfasst: ein Gehäuse, einen Prozessor, einen Speicher, einer Leiterplatte und einen Leistungskreis, wobei sich die Leiterplatte in einem Raum befindet, der von dem Gehäuse umgeben ist; wobei sich der Prozessor und der Speicher auf der Leiterplatte befinden; wobei der Leistungskreis zur Versorgung verschiedener Schaltkreise oder Komponenten mit Strom verwendet wird; wobei der Speicher zum Speichern ausführbarer Programmcodes verwendet wird; und wobei der Prozessor das Verfahren zur Bestimmung des Volumens eines Objekts nach einem der Ansprüche 1 bis 5 ausführt durch Ausführen der in dem Speicher gespeicherten ausführbaren Programmcodes.
 


Revendications

1. Procédé de détermination du volume d'un objet, comprenant les étapes consistant à :

obtenir une image de profondeur cible contenant un objet cible qui est capturée par un dispositif de capture d'image de profondeur (S101) ;

effectuer une segmentation sur la base de données de profondeur dans l'image de profondeur cible pour obtenir une région d'image cible correspondant à l'objet cible (S102) ;

déterminer un rectangle circonscrit cible qui correspond à la région d'image cible et qui remplit une condition prédéfinie (S103), qui comprend les étapes consistant à :

déterminer un rectangle circonscrit cible qui correspond à la région d'image cible et a une surface minimale, ou,

déterminer un rectangle circonscrit cible qui correspond à la région d'image cible et a une différence minimale entre sa surface et un seuil de surface prédéfini ;

extraire des coordonnées d'image de chaque sommet du rectangle circonscrit cible dans une image de différence de trame binarisée (S1041), l'image de différence de trame étant une image qui est obtenue sur la base de la différence entre les données de profondeur de l'image de profondeur cible et les données de profondeur d'une image de profondeur d'arrière-plan prédéfinie et caractérisé par les étapes consistant à

projeter les coordonnées d'image extraites de chaque sommet dans l'image de profondeur cible pour générer un point de référence situé dans l'image de profondeur cible (S1042) ;

calculer des coordonnées tridimensionnelles pour chaque point de référence dans un système de coordonnées mondial de caméra selon un principe de projection en perspective dans l'imagerie de caméra (S1043) ; et

avec les coordonnées tridimensionnelles des points de référence, calculer la distance euclidienne entre chaque deux points de référence, déterminer une longueur et une largeur de l'objet cible comme deux distances autres que la plus grande distance dans les distances euclidiennes calculées, soustraire une valeur de profondeur d'une région correspondant aux points de référence à une valeur de profondeur de l'image de profondeur d'arrière-plan prédéfinie pour obtenir une hauteur de l'objet cible, et déterminer un volume de l'objet cible comme étant le produit de la longueur, de la largeur et de la hauteur de l'objet cible.


 
2. Procédé selon la revendication 1, l'obtention d'une image de profondeur cible contenant un objet cible qui est capturée par un dispositif de capture d'image de profondeur (S101), comprenant l'étape consistant à :
obtenir l'image de profondeur cible contenant l'objet cible qui est capturée par une caméra à temps de vol.
 
3. Procédé selon la revendication 1, l'exécution d'une segmentation sur la base des données de profondeur dans l'image de profondeur cible pour obtenir une région d'image cible correspondant à l'objet cible (S102), comprenant l'étape consistant à :
effectuer une segmentation sur la base des données de profondeur dans l'image de profondeur cible à l'aide d'un procédé de différence de trame d'image de profondeur pour obtenir la région d'image cible correspondant à l'objet cible.
 
4. Procédé selon la revendication 3, l'exécution d'une segmentation sur la base des données de profondeur dans l'image de profondeur cible à l'aide d'un procédé de différence de trame d'image de profondeur pour obtenir la région d'image cible correspondant à l'objet cible, comprenant les étapes consistant à :

obtenir une différence entre les données de profondeur de chacun des pixels dans l'image de profondeur cible et les données de profondeur d'un pixel correspondant dans l'image de profondeur d'arrière-plan prédéfinie (S1021) ; l'image de profondeur d'arrière-plan prédéfinie étant une image de l'environnement d'arrière-plan où l'objet cible est situé, qui est capturée à l'avance par le dispositif de capture d'image de profondeur et ne contient pas l'objet cible ;

générer une image de différence d'image de trame correspondant à l'image de profondeur cible sur la base de la différence pour chacun des pixels (S1022) ;

binariser l'image de différence de trame (S1023) ; et

obtenir la région d'image cible correspondant à l'objet cible à partir de l'image de différence de trame binarisée (S1024).


 
5. Procédé selon la revendication 1, la détermination d'un rectangle circonscrit cible qui correspond à la région d'image cible et satisfait à une condition prédéfinie (S103), comprenant l'étape consistant à :
déterminer le rectangle circonscrit cible qui correspond à la région d'image cible et qui remplit la condition prédéfinie à l'aide d'un algorithme d'analyse de région connectée ou d'un algorithme d'ajustement par détection de bord.
 
6. Appareil de détermination du volume d'un objet, comprenant :

un module d'obtention d'image de profondeur (410), configuré pour obtenir une image de profondeur cible contenant un objet cible qui est capturée par un dispositif de capture d'image de profondeur (S101) ;

un module de segmentation de région d'image (420), configuré pour effectuer une segmentation sur la base des données de profondeur dans l'image de profondeur cible pour obtenir une région d'image cible correspondant à l'objet cible (S102) ;

un module de détermination de rectangle circonscrit (430), configuré pour déterminer un rectangle circonscrit cible qui correspond à la région d'image cible et satisfait à une condition prédéfinie (S103), qui comprend les étapes suivantes :

déterminer un rectangle circonscrit cible qui correspond à la région d'image cible et a une surface minimale, ou,

déterminer un rectangle circonscrit cible qui correspond à la région d'image cible et a une différence minimale entre sa surface et un seuil de surface prédéfini ; et

un module de détermination de volume (440), configuré pour :
extraire des coordonnées d'image de chaque sommet du rectangle circonscrit cible dans une image de différence de trame binarisée (S1041),

l'image de différence de trame étant une image qui est obtenue sur la base de la différence entre les données de profondeur de l'image de profondeur cible et les données de profondeur d'une image de profondeur d'arrière-plan prédéfinie et caractérisé par les étapes suivantes

projeter les coordonnées d'image extraites de chaque sommet dans l'image de profondeur cible pour générer un point de référence situé dans l'image de profondeur cible (S1042) ;

calculer des coordonnées tridimensionnelles pour chaque point de référence dans un système de coordonnées mondial de caméra selon un principe de projection en perspective dans l'imagerie de caméra (S1043) ; et

avec les coordonnées tridimensionnelles des points de référence, calculer la distance euclidienne entre chaque deux points de référence, déterminer une longueur et une largeur de l'objet cible comme deux distances autres que la plus grande distance dans les distances euclidiennes calculées, soustraire une valeur de profondeur d'une région correspondant aux points de référence à une valeur de profondeur de l'image de profondeur d'arrière-plan prédéfinie pour obtenir une hauteur de l'objet cible, et déterminer un volume de l'objet cible comme étant le produit de la longueur, de la largeur et de la hauteur de l'objet cible.


 
7. Appareil selon la revendication 6, le module d'obtention d'image de profondeur (410) comprenant :
une unité d'obtention d'image de profondeur, configurée pour obtenir l'image de profondeur cible contenant l'objet cible qui est capturée par une caméra à temps de vol.
 
8. Appareil selon la revendication 6, le module de segmentation de région d'image (420) comprenant :
une unité de segmentation de région d'image, configurée pour effectuer une segmentation sur les données de profondeur dans l'image de profondeur cible à l'aide d'un procédé de différence de trame d'image de profondeur pour obtenir la région d'image cible correspondant à l'objet cible.
 
9. Appareil selon la revendication 8, l'unité de segmentation de région d'image comprenant :

une sous-unité de soustraction, configurée pour obtenir une différence entre les données de profondeur de chacun des pixels dans l'image de profondeur cible et les données de profondeur d'un pixel correspondant dans l'image de profondeur d'arrière-plan prédéfinie (S1021) ; l'image de profondeur d'arrière-plan prédéfinie étant une image de l'environnement d'arrière-plan où se trouve l'objet cible, qui est capturée à l'avance par le dispositif de capture d'image de profondeur et ne contient pas l'objet cible ;

une sous-unité de génération d'image de différence de trame, configurée pour générer une image de différence de trame correspondant à l'image de profondeur cible sur la base de la différence pour chacun des pixels (S1022) ;

une sous-unité de binarisation, configurée pour binariser l'image de différence de trame (S1023) ; et

une sous-unité de segmentation de région d'image, configurée pour obtenir la région d'image cible correspondant à l'objet cible à partir de l'image de différence de trame binarisée (S1024).


 
10. Appareil selon la revendication 6, le module de détermination de rectangle circonscrit (430) comprenant :
une première unité de détermination de rectangle circonscrit, configurée pour déterminer le rectangle circonscrit cible qui correspond à la région d'image cible et remplit la condition prédéfinie en utilisant un algorithme d'analyse de région connectée ou d'un algorithme d'ajustement par détection de bord.
 
11. Support de stockage pour stocker des codes de programme exécutables, qui sont exécutés pour mettre en œuvre le procédé de détermination du volume d'un objet selon l'une quelconque des revendications 1 à 5.
 
12. Programme d'application exécuté pour mettre en œuvre le procédé de détermination du volume d'un objet selon l'une quelconque des revendications 1 à 5.
 
13. Dispositif électronique comprenant : un boîtier, un processeur, une mémoire, un circuit imprimé, et un circuit d'alimentation ; un circuit imprimé étant disposé à l'intérieur d'un espace entouré par le boîtier ; le processeur et la mémoire étant disposés sur un circuit imprimé ; le circuit d'alimentation étant utilisé pour alimenter en énergie divers circuits ou composants ; la mémoire étant utilisée pour stocker les codes de programme exécutables ; et le processeur mettant en œuvre le procédé pour déterminer un volume d'un objet selon l'une quelconque des revendications 1 à 5 en exécutant les codes de programme exécutables stockés dans la mémoire.
 




Drawing














Cited references

REFERENCES CITED IN THE DESCRIPTION



This list of references cited by the applicant is for the reader's convenience only. It does not form part of the European patent document. Even though great care has been taken in compiling the references, errors or omissions cannot be excluded and the EPO disclaims all liability in this regard.

Patent documents cited in the description