| (11) | EP 2 973 421 B1 |
| (12) | EUROPEAN PATENT SPECIFICATION |
|
|
| (54) | APPARATUS AND METHOD FOR APPLYING A TWO-DIMENSIONAL IMAGE ON A THREE-DIMENSIONAL MODEL VORRICHTUNG UND VERFAHREN ZUM AUFBRINGEN EINES ZWEIDIMENSIONALEN BILDES AUF EINEM DREIDIMENSIONALEN MODELL APPAREIL ET PROCÉDÉ D'APPLICATION D'IMAGE BIDIMENSIONNELLE SUR UN MODÈLE TRIDIMENSIONNEL |
|
| |||||||||||||||||||||||||||||||
| Note: Within nine months from the publication of the mention of the grant of the European patent, any person may give notice to the European Patent Office of opposition to the European patent granted. Notice of opposition shall be filed in a written reasoned statement. It shall not be deemed to have been filed until the opposition fee has been paid. (Art. 99(1) European Patent Convention). |
[0001] This invention relates to an apparatus and method for applying a two-dimensional image on a three-dimensional model. More specifically, but not exclusively, this invention relates to applying a label on a three-dimensional model.
[0002] In product development, the visual appearance of the product is an important part of the marketing strategy. Accordingly, developers typically use computer editing suites to design their packaging. In some industries, containers for products are manufactured separately to the brand material, and the brand materials are subsequently fixed to the container. This is common practice in, for example, the drinks industry, in which bottles are manufactured to contain the product, and a label is produced separately and applied to the bottle.
[0003] Separate computer editing suites exist for creating the label and creating a model of the bottle. To visualize the label on the bottle, a UV map of the model is created and the label is applied as a texture. This is computationally expensive and also requires a lot of user intervention to ensure the label is applied correctly. This leads to a significant delay for the computer to render the label onto the model. Accordingly, the label cannot be placed on the bottle and moved in real-time, as the processing requirements outweigh the processing power available.
[0004] EP0889437A2 describes rendering a two-dimensional resolution dependent raster image to a three-dimensional surface model object. A clipping path is established for a raster version of the 3D image which is mapped to a 2D parametric map and to a 2D display space. The raster image is edited and masked while displayed on a display of the 2D parametric map. The vector output and raster output are integrated to generate a final output including both vector and raster data and the final output is to be sent to a printing device and rasterised by the printing device.
[0005] Igarashi T et al "Adaptive Unwrapping for Interactive Texture Painting", proceedings of the 2001 Symposium on Interactive 3D Graphics, March 19 - 21, 2001, pages 209-216 describes a method for dynamically generating an efficient texture bit map and its associated UV - mapping in an interactive texture painting system for 3D models. The paper proposes an adaptive unwrapping mechanism which allows users to create a tailored UV mapping for newly painted polygons during an interactive painting process. The tailored UV mapping eliminates the distortion of brush strokes and the resulting textured bit map is made more compact because the system allocates texture space only for the painted polygons. The dynamic texture allocation also allows the user to paint smoothly at any zoom level.
[0007] According to a first aspect of the invention, there is provided a computer implemented method of applying a two-dimensional image to a three-dimensional model, as set out in claim 1 appended hereto.
[0008] The present invention provides a method which may extract UV-coordinates for a particular area of the three-dimensional model, corresponding to the area on which the two-dimensional image is to be applied. Accordingly, the computational power required to apply the two-dimensional image is significantly reduced, as the conventional step of UV-mapping the entire model is not carried out.
[0009] On a computing apparatus adapted to carry out the method of the present invention, the user may therefore select a first point in an area on the three-dimensional model to apply the two-dimensional image, and the processor extracts spatial data from that area until it identifies a point on the model being a greater distance from the first point than the distance from the centre to the boundary of the two-dimensional image. As the processing power required has been reduced significantly, the two-dimensional image may therefore be applied to the three-dimensional model and displayed on the computing apparatus' display device in real time. This also allows the user to 'drag' the two-dimensional image across the three-dimensional model, i.e. selecting successive first points on the model, and the processor may apply the two-dimensional image to the model on the fly.
[0010] The method may further comprise the step of generating a single three-dimensional model from a plurality of three-dimensional models to form a membrane, wherein the first point is on the membrane. The present invention may therefore apply the two-dimensional image to the membrane enveloping the models. The membrane may contain most or all the vertices of the three-dimensional model, or in the case of a plurality of three-dimensional models, each representing various parts of a complex product, the membrane may contain most or all the vertices of each three-dimensional model.
[0011] The method may further comprise the step of applying a smoothing technique to the membrane. Thus, any sudden changes in curvature on the membrane (e.g. due to a gap in the membrane) may be smoothed over. Accordingly, the two-dimensional image may then be applied in a real-life manner. For example, in the case of a label being applied to a three-dimensional model, the label may be applied to the model such that it passes over the gap, in a similar manner to real-life.
[0012] A computer program product comprising computer executable code which when executed on a computer may cause the computer to perform the method of the first aspect of the invention.
[0013] According to a second aspect of the invention, there is provided a computing apparatus as set out in claim 5 appended hereto.
[0014] The computing apparatus may further comprise a display device, wherein the processor is configured to cause the display device to display the three-dimensional model including the applied two-dimensional image.
[0015] Embodiments of the invention will now be described, by way of example, and with reference to the drawings in which:
Figure 1 is flow diagram representing an embodiment of a method of the present invention; and
Figure 2 is a schematic diagram illustrating a computing apparatus configured to carry out the steps of the method of Figure 1.
[0016] An embodiment of a method of applying a two-dimensional image to a three-dimensional model in a virtual environment will now be described with reference to Figure 1. The embodiment will detail an example of placing a label (the two-dimensional image) onto a bottle (the three-dimensional model), but, on reviewing the following description, the skilled person will understand that the method may be applied to any type of two-dimensional image being placed on a three-dimensional model.
[0017] As an initial step, a first image file representing a label is created. The first image file may be created in a suitable graphics editing computer program, such as Adobe Illustrator, and consists of a two-dimensional image representing a label (and shall hereinafter be referred to as the "label"). In this embodiment, the label is rectangular and includes a graphical portion enveloped by cutting lines.
[0018] As a further initial step, a first model file representing a bottle is created. The first model file may be created in any suitable three-dimensional modelling software, such as 3DS Max or AutoCAD. The first model file consists of a three-dimensional representation of a bottle (and shall hereinafter be referred to as the "bottle"), defined by a triangular mesh. The skilled person will understand that the triangular mesh represents an outer surface of the bottle, and is a non-UV mapped model.
[0019] The triangular mesh is used to create a set of data which records the adjacency information of all the triangles within the mesh, in one embodiment a structure such as a half-edge data structure could be used. This allows traversal of the mesh starting from a single point on the mesh. This is labelled as the membrane which consists of a set of triangles with adjacency information.
[0020] As explained in more detail below, a plurality of triangular meshes may be merged together (such as one representing a bottle and another representing a bottle cap).
[0021] In this embodiment, a single membrane which includes the form of both the cap and bottle is created, this is achieved by starting with a simple shape which encompasses both cap and bottle which is then shrunk to the form of the bottle and cap combined. This single mesh is then used to generate a membrane which is used to place the 2D label on both bottle and cap.
[0022] In this embodiment, a 'smoothing' operation is applied to the membrane. Various smoothing techniques may be used, such as Laplacian polygon smoothing (however, the skilled person will understand that other smoothing techniques may be used). This technique ensures that the membrane closely follows a real-life surface that the label will be applied to. For example, if the bottle contained a small gap, the membrane may be smoothed such that it passes over the gap without any change in curvature. This closely represents how the real-life label would be applied to the real-life bottle, as the real-life label would simply be pasted over the gap without any change in curvature.
[0023] The label is then applied to the membrane using the following algorithm. As a first step, an initial point of reference is taken as the centre of the label. A collision routine is used to correlate the centre of the label with an initial hit point on the triangle mesh. The initial hit point may be any arbitrary point on the triangle mesh.
[0024] In a next step of the algorithm (as shown in Figure 1), the triangle containing the starting point on the membrane is chosen. The starting point is determined as a point on the membrane which is closest to the initial hit point on the triangular mesh. The distance between this point (horizontal and vertical) and each of the three vertices in the starting triangle is calculated and stored, then each of the three neighbouring triangles is added to a set of triangles to be processed. An iterative process is then employed in the following manner. For each of the triangles in the triangle list, calculate the distance between the vertex in which the distance to the starting point is known and the vertex whose distance is still to be calculated, and store this distance. This distance is calculated using geodesic techniques. Check the remaining two edges of the triangle to see if the 2D coordinates of the triangle intersect the boundary rectangle of the two dimensional label. If an edge does intersect then add the triangle neighbour which uses that edge to the list of triangles to process. The process is run in an iterative manner until there are no more triangles within the list to process.
[0025] If the 2D label is likely to overlap itself, the process can be split so that the label is processed as two separate labels; the processing order defines the overlap direction.
[0026] In one embodiment, an additional 2D rotation can be applied to all the calculated distances, which would have the effect of rotating the label around the centre point.
[0027] The skilled person will understand that this process results in a data set containing a set of points on the membrane and a set of distances (horizontal and vertical) between each point in the set of points on the membrane and the starting point. The data set may then be converted into normalized UV coordinates using a transform. For example, the distances are stored as vectors from the starting point to each point, and are converted to UV coordinates by a translation by 0.5 and scaled to the particular height and width of the label.
[0028] The normalized UV coordinates may then be used to apply the label to the bottle. That is, the label may be treated as a texture which may then be correlated with the UV coordinates.
[0029] Accordingly, a plurality of points within the label may be correlated with particular UV coordinates, such that the label may then be placed on the bottle in the virtual environment.
[0030] The skilled person will understand that the process is suitable for applying a two-dimensional image onto any three-dimensional model. The three dimensional model extracts UV coordinates for only a part of the three-dimensional model that the label is to be applied to, and thus reduces the computational power required for prior art techniques which involve a fully mapped UV model. The two-dimensional image may be applied to any portion of the three-dimensional model, and the algorithm detailed above applies the centre of the label to a starting point on the membrane in that portion and extracts the UV coordinates from the membrane of the surrounding area.
[0031] The skilled person will also realise that by reducing the computational power required to apply the two-dimensional image to the three-dimensional model, the image may be applied in real-time. That is, the user may slide the image across the membrane of the three-dimensional model and the computational power required is reduced to the point that the image may be applied to the three-dimensional model "on-the-fly".
[0032] As noted above, a plurality of triangular meshes may be merged together. In this manner, a label may be applied to a complex product (i.e. an object having multiple constituent parts, such as a bottle and bottle cap). Next, a smoothing operation may be applied to the membrane to smooth any gaps or grooves between the parts of the complex product to ensure the label is applied appropriately.
[0033] In the above embodiments, the three-dimensional model consists of one or more triangular meshes to which a single membrane is generated. The skilled person will understand that any form of polygonal meshed model would be appropriate for the present invention. Furthermore, the skilled person will understand that the application of a membrane is preferable as it allows the user to apply a smoothing technique such that the two-dimensional image is applied in a real-life manner. However, the smoothing step is non-essential. For example, the algorithm to extract the UV coordinates may be based on the vertices of the triangular mesh rather than points of a smoothed membrane.
[0034] In the above embodiment, a label is applied to a model of a bottle. However, the skilled person will understand that the method of the present invention is suitable for applying any two-dimensional image to any three-dimensional model. For example, the method would be suitable for applying graphics to the bodywork of a vehicle in a virtual environment.
[0035] The method of the present invention described above may be implemented on a computing apparatus 1, such as a personal computer or mobile computing device (e.g. a tablet). As shown in Figure 2, the computing device 1 includes a processor 3 configured for implementing the method of the present invention, and a display device 5 configured for displaying the three-dimensional model including the applied two-dimensional image.
creating a membrane by generating an initial adjacency structure for all triangles within the polygon mesh,
identifying a triangle within the membrane which contains a desired centre point of the two-dimensional image
calculating horizontal and vertical spatial distances between the three vertices of the triangle and desired centre point of the two-dimensional image;
adding each of the three neighbouring triangles of the triangle to a list of triangles to process;
iteratively processing all the triangles in the list until the list is empty to generate a data set containing a set of points on the membrane and a set of horizontal and vertical distances between each point in the set of points on the membrane and the desired centre point, the iterative process comprising:
calculating the horizontal and vertical distances between a vertex in which the distance to the desired centre point is known and the single unknown vertex within the triangle and
checking the two remaining edges of the triangle to see if the 2D coordinates of the triangle intersect the boundary rectangle of the two-dimensional image, and if an intersection occurs adding this new triangle neighbour to the triangle list;
transforming the points and horizontal and vertical distances into normalised UV-coordinates; and
applying the two-dimensional image to the three-dimensional model using the normalised UV-coordinates.
creating a membrane by generating an initial adjacency structure for all triangles within the polygon mesh,
identifying a triangle within the membrane which contains a desired centre point of the two-dimensional image
calculating horizontal and vertical spatial distances between the three vertices of the triangle and desired centre point of the two-dimensional image;
adding each of the three neighbouring triangles of the triangle to a list of triangles to process; iteratively processing all the triangles in the list until the list is empty to generate a data set containing a set of points on the membrane and a set of horizontal and vertical distances between each point in the set of points on the membrane and the desired centre point, the iterative process comprising;
calculating the horizontal and vertical distances between a vertex in which the distance to the desired centre point is known and the single unknown vertex within the triangle; and
checking the two remaining edges of the triangle to see if the 2D coordinates of the triangle intersect the boundary rectangle of the two-dimensional image, and if an intersection occurs adding this new triangle neighbour to the triangle list;
transforming the points and horizontal and vertical distances into normalised UV-coordinates; and
applying the two-dimensional image to the three-dimensional model using the normalised UV-coordinates.
Erzeugen einer Membran durch Generieren einer anfänglichen Nachbarschaftsstruktur aller Dreiecke innerhalb des Polygonnetzes,
Identifizieren eines Dreiecks innerhalb der Membran, das einen gewünschten Mittelpunkt des zweidimensionalen Bildes enthält;
Berechnen horizontaler und vertikaler räumlicher Distanzen zwischen den drei Eckpunkten des Dreiecks und dem gewünschten Mittelpunkt des zweidimensionalen Bildes;
Hinzufügen jedes der drei benachbarten Dreiecke des Dreiecks zu einer Liste von zu verarbeitenden Dreiecken;
iteratives Verarbeiten aller Dreiecke in der Liste, bis die Liste leer ist, um einen Datensatz zu generieren, der einen Satz Punkte auf der Membran und einen Satz horizontale und vertikale Distanzen zwischen jedem Punkt in dem Satz Punkte auf der Membran und dem gewünschten Mittelpunkt enthält, wobei der iterative Prozess Folgendes umfasst:
Berechnen der horizontalen und vertikalen Distanzen zwischen einem Eckpunkt, bei dem die Distanz zu dem gewünschten Mittelpunkt bekannt ist, und dem einzelnen unbekannten Eckpunkt innerhalb des Dreiecks; und
Prüfen der zwei restlichen Kanten des Dreiecks, um zu sehen, ob die 2D-Koordinaten des Dreiecks das Grenzrechteck des zweidimensionalen Bildes überschneiden, und falls eine Überschneidung auftritt, Hinzufügen des neuen Dreiecknachbars zu der Dreieckliste;
Transformieren der Punkte und horizontalen und vertikalen Distanzen in normalisierte UV-Koordinaten; und
Anwenden des zweidimensionalen Bildes auf das dreidimensionale Modell unter Verwendung der normalisierten UV-Koordinaten.
Erzeugen einer Membran durch Generieren einer anfänglichen Nachbarschaftsstruktur aller Dreiecke innerhalb des Polygonnetzes,
Identifizieren eines Dreiecks innerhalb der Membran, das einen gewünschten Mittelpunkt des zweidimensionalen Bildes enthält;
Berechnen horizontaler und vertikaler räumlicher Distanzen zwischen den drei Eckpunkten des Dreiecks und dem gewünschten Mittelpunkt des zweidimensionalen Bildes;
Hinzufügen jedes der drei benachbarten Dreiecke des Dreiecks zu einer Liste von zu verarbeitenden Dreiecken;
iteratives Verarbeiten aller Dreiecke in der Liste, bis die Liste leer ist, um einen Datensatz zu generieren, der einen Satz Punkte auf der Membran und einen Satz horizontale und vertikale Distanzen zwischen jedem Punkt in dem Satz Punkte auf der Membran und dem gewünschten Mittelpunkt enthält, wobei der iterative Prozess Folgendes umfasst:
Berechnen der horizontalen und vertikalen Distanzen zwischen einem Eckpunkt, bei dem die Distanz zu dem gewünschten Mittelpunkt bekannt ist, und dem einzelnen unbekannten Eckpunkt innerhalb des Dreiecks; und
Prüfen der zwei restlichen Kanten des Dreiecks, um zu sehen, ob die 2D-Koordinaten des Dreiecks das Grenzrechteck des zweidimensionalen Bildes überschneiden, und falls eine Überschneidung auftritt, Hinzufügen des neuen Dreiecknachbars zu der Dreieckliste;
Transformieren der Punkte und horizontalen und vertikalen Distanzen in normalisierte UV-Koordinaten; und
Anwenden des zweidimensionalen Bildes auf das dreidimensionale Modell unter Verwendung der normalisierten UV-Koordinaten.
création d'une membrane en générant une structure de contiguïté initiale pour tous les triangles à l'intérieur du maillage polygonal,
identification d'un triangle à l'intérieur de la membrane qui contient un point central souhaité de l'image bidimensionnelle,
calcul des distances spatiales horizontales et verticales entre les trois sommets du triangle et le point central souhaité de l'image bidimensionnelle ;
ajout de chacun des trois triangles voisins du triangle à une liste de triangles à traiter ;
traitement de façon itérative de tous les triangles de la liste jusqu'à ce que la liste soit vide pour générer un ensemble de données contenant un ensemble de points sur la membrane et un ensemble de distances horizontales et verticales entre chaque point de l'ensemble de points sur la membrane et le point central souhaité, le processus itératif comprenant :
le calcul des distances horizontales et verticales entre un sommet pour lequel la distance au point central souhaité est connue et le sommet inconnu unique dans le triangle et la vérification des deux arêtes restantes du triangle pour voir si les coordonnées 2D du triangle coupent le rectangle frontière de l'image bidimensionnelle, et si une intersection se produit, l'ajout ce nouveau triangle voisin à la liste des triangles ;
la transformation des points et des distances horizontales et verticales en coordonnées UV normalisées ; et
l'application de l'image bidimensionnelle sur le modèle tridimensionnel à l'aide des coordonnées UV normalisées.
créer une membrane en générant une structure de contiguïté initiale pour tous les triangles à l'intérieur du maillage polygonal,
identifier un triangle à l'intérieur de la membrane qui contient un point central souhaité de l'image bidimensionnelle
calculer les distances spatiales horizontales et verticales entre les trois sommets du triangle et le point central souhaité de l'image bidimensionnelle ;
ajouter chacun des trois triangles voisins du triangle à une liste de triangles à traiter ;
traiter de façon itérative tous les triangles de la liste jusqu'à ce que la liste soit vide pour générer un ensemble de données contenant un ensemble de points sur la membrane et un ensemble de distances horizontales et verticales entre chaque point dans l'ensemble de points sur la membrane et le point central souhaité, le processus itératif comprenant ;
le calcul des distances horizontales et verticales entre un sommet pour lequel la distance au point central souhaité est connue et le sommet inconnu unique à l'intérieur du triangle ; et
la vérification des deux arêtes restantes du triangle pour voir si les coordonnées 2D du triangle coupent le rectangle frontière de l'image bidimensionnelle, et si une intersection se produit, l'ajout de ce nouveau triangle voisin à la liste de triangles ;
la transformation des points et des distances horizontales et verticales en coordonnées UV normalisées ; et
l'application de l'image bidimensionnelle sur le modèle tridimensionnel à l'aide des coordonnées UV normalisées.
REFERENCES CITED IN THE DESCRIPTION
This list of references cited by the applicant is for the reader's convenience only. It does not form part of the European patent document. Even though great care has been taken in compiling the references, errors or omissions cannot be excluded and the EPO disclaims all liability in this regard.
Patent documents cited in the description
Non-patent literature cited in the description
- IGARASHI T et al.Adaptive Unwrapping for Interactive Texture Paintingproceedings of the 2001 Symposium on Interactive 3D Graphics, 2001, 209-216 [0005]