|
(11) | EP 4 339 836 B1 |
| (12) | EUROPEAN PATENT SPECIFICATION |
|
|
| (54) |
NETWORK MODEL COMPRESSION METHOD, APPARATUS AND DEVICE, IMAGE GENERATION METHOD, AND MEDIUM NETZWERKMODELLKOMPRIMIERUNGSVERFAHREN, -VORRICHTUNG UND -VORRICHTUNG, BILDERZEUGUNGSVERFAHREN UND MEDIUM PROCÉDÉ, APPAREIL ET DISPOSITIF DE COMPRESSION DE MODÈLE DE RÉSEAU, PROCÉDÉ DE GÉNÉRATION D'IMAGE ET SUPPORT |
|
|
|||||||||||||||||||||||||||||||
| Note: Within nine months from the publication of the mention of the grant of the European patent, any person may give notice to the European Patent Office of opposition to the European patent granted. Notice of opposition shall be filed in a written reasoned statement. It shall not be deemed to have been filed until the opposition fee has been paid. (Art. 99(1) European Patent Convention). |
TECHNICAL FIELD
[0001] The present disclosure relates to the field of computer technology and, in particular, to a network model compression method, apparatus and device, an image generation method, and a medium.
BACKGROUND
[0002] Generative Adversarial Network (GAN) is a deep learning model and is one of the most promising methods for unsupervised learning on complex distributions in recent years, which is widely used in various image synthesis tasks, such as image generation, image resolution, and super-resolution.
[0003] The non-patent document "Slimmable Generative Adversarial Networks" introduces slimmable GANs (SlimGANs), which can flexibly switch the width of the generator to accommodate various quality-efficiency trade-offs at runtime. Specifically, multiple discriminators that share partial parameters are leveraged to train the slimmable generator. To facilitate the consistency between generators of different widths, a stepwise inplace distillation technique that encourages narrow generators to learn from wide ones is presented. As for class-conditional generation, a sliceable conditional batch normalization that incorporates the label information into different widths is proposed. The methods are validated, both quantitatively and qualitatively, by extensive experiments and a detailed ablation study.
[0004] The non-patent document "Teachers Do More Than Teach Compressing Image-To-Image Models" aims to address issues that Generative Adversarial Networks (GANs) suffer from low efficiency due to tremendous computational cost and bulky memory usage in generating high-fidelity images. This work is realized by introducing a teacher network that provides a search space in which efficient network architectures can be found, in addition to performing knowledge distillation. First, the search space of generative models is visited, introducing an inception-based residual block into generators. Second, to achieve target computation cost, a one-step pruning algorithm that searches a student architecture from the teacher model and substantially reduces searching cost is proposed. It requires no l1 sparsity regularization and its associated hyper-parameters, simplifying the training procedure. Finally, it proposes to distill knowledge through maximizing feature similarity between teacher and student via an index named Global Kernel Alignment (GKA). The compressed networks achieve similar or even better image fidelity (FID, mIoU) than the original models with much-reduced computational cost, e.g., MACs.
SUMMARY
[0005] At least one embodiment of the present disclosure provides a network model compression method, apparatus and device, an image generation method, and a medium, which may solve one or more problems in the art. The object is achieved by the features of the respective independent claims. Further embodiments are defined in the respective dependent claims.
BRIEF DESCRIPTION OF DRAWINGS
[0006] The drawings herein are incorporated into and form a part of the specification, illustrate the embodiments consistent with the present disclosure, and are used in conjunction with the specification to explain the principles of the present disclosure.
[0007] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or in prior art, the drawings to be used in the description of the embodiments or prior art will be briefly described below, and it will be obvious to those ordinarily skilled in the art that other drawings can be obtained on the basis of these drawings without inventive work.
FIG. 1 is a flowchart of a network model compression method according to an embodiment of the present disclosure;
FIG. 2 is a flowchart of a network model compression method according to an embodiment of the present disclosure;
FIG. 3 is a flowchart of a network model compression method according to an embodiment of the present disclosure;
FIG. 4 is a flowchart of an image generation method according to an embodiment of the present disclosure;
FIG. 5 is a schematic diagram of a structure of a network model compression apparatus according to an embodiment of the present disclosure; and
FIG. 6 is a schematic diagram of a structure of a network model compression device according to an embodiment of the present disclosure.
DETAILED DESCRIPTION
[0008] In order to understand the above objects, features and advantages of the present disclosure more clearly, the solutions of the present disclosure will be further described below. It should be noted that, in case of no conflict, the features in one embodiment or in different embodiments can be combined.
[0009] Many specific details are set forth in the following description to fully understand the present disclosure, but the present disclosure can also be implemented in other ways different from those described here; obviously, the embodiments in the specification are a part but not all of the embodiments of the present disclosure.
[0010] As GAN with a larger model usually consumes more computing resources, when it is applied to devices with a poor computing capacity such as mobile phones, the delay is long and real-time application requirements cannot be met. Therefore, in the related art, the overall model size of the GAN is reduced by compressing a generator. However, the applicant found that mode collapse phenomenon occurs when only the generator is compressed while the structure of a discriminator remains unchanged.
[0011] The applicant found through research that for a well-trained GAN, its generator and discriminator are comparable to each other in the state. After the generator is compressed, the performance of the generator will decrease, while the structure of the discriminator remains unchanged, that is, the performance of the discriminator remains unchanged, so that the Nash equilibrium between the generator and the discriminator is broken, resulting in mode collapse phenomenon.
[0012] In view of this, the embodiments of the present disclosure provide a network model compression method. By performing pruning processing on a first generator, a second generator is obtained, and states of convolution kernels in a first discriminator is configured to enable a part of the convolution kernels to be in an activated state and the other part of the convolution kernels to be in a suppressed state, so as to obtain a second discriminator, such that a first loss difference between the first generator and the first discriminator is close to a second loss difference between the second generator and the second discriminator, and thus the Nash equilibrium between the second generator and the second discriminator can be maintained, thereby avoiding the mode collapse phenomenon. Hereinafter, the method will be introduced with reference to specific embodiments.
[0013] FIG. 1 is a flowchart of a network model compression method according to an embodiment of the present disclosure, and the method may be performed by a network model compression device. The network model compression device may be illustratively understood as a device with a computing function such as a portable Android device, a laptop computer, or a desktop computer. The method can compress a network model to be compressed including a first generator and a first discriminator. As shown in FIG. 1, the method of the present embodiment includes the following S110-S120.
[0015] Specifically, the specific implementation for performing pruning processing on the first generator may be set by those skilled in the art according to the actual situation, and is not limited herein. In one possible embodiment, performing pruning processing on the first generator includes: selectively deleting convolution kernels in the first generator such that convolution kernels with the importance less than a preset importance threshold are deleted and convolution kernels with the importance greater than or equal to the preset importance threshold are retained.
[0016] Illustratively, each convolutional layer (CL) in the first generator is provided with a batch normalization (BN) layer, and each CL includes at least one convolution kernel. Each convolution kernel is correspondingly provided with a scaling factor in the BN layer corresponding to the CL to which the convolution kernel belongs, where the scaling factor is used for characterizing the importance of its corresponding convolution kernel. The scaling factor corresponding to each convolution kernel is added to an objective function of the first generator by direct summation to obtain
, where LG T is the objective function of the first generator, A is the total number of convolution kernels in the first generator, and scale(a) is a scaling factor for an a-th convolution kernel. Then, the first generator and the first discriminator are trained. Specifically, each training includes determining the scaling factor according to
. When the total number of training times is reached, the scaling factors for respective convolution kernels are ranked from small to large, and the convolution kernels with smaller scaling factors are deleted using a binary search algorithm until the computational amount of the first generator meets the preset given computational amount.
[0017] Illustratively, each CL in the first generator includes at least one convolution kernel, and each convolution kernel is correspondingly provided with a weight parameter, where the weight parameter of a convolution kernel is used for characterizing the importance of the convolution kernel corresponding to the weight parameter. The weight parameter corresponding to each convolution kernel is added to the objective function of the first generator by direct summation to obtain
, where LG T is the objective function of the first generator, A is the total number of convolution kernels in the first generator, and L(a) is a weight parameter for the a-th convolution kernel. Then, the first generator and the first discriminator are trained. Specifically, each training includes determining the weight parameter according to
. When the total number of training times is reached, the weight parameters for the respective convolution kernels are ranked from small to large, and the convolution kernels with smaller weight parameters are deleted using binary search algorithm until the computational amount of the first generator meets the preset given computational amount.
[0018] In the above-mentioned two manners, the specific number of training times and the specific value of the preset given computational amount may be set by those skilled in the art according to the actual situation, and are not limited herein.
[0019] S120: configuring states of convolution kernels in the first discriminator to enable a part of the convolution kernels to be in an activated state and the other part of the convolution kernels to be in a suppressed state, so as to obtain a second discriminator.
[0020] Specifically, when the compressed network model is put into use, the convolution kernels in the suppressed state do not work, and the convolution kernels in the activated state work normally.
[0021] Herein, a loss difference between the first generator and the first discriminator is a first loss difference, a loss difference between the second generator and the second discriminator is a second loss difference, and an absolute value of a difference value between the first loss difference and the second loss difference is less than a first preset threshold.
[0022] Specifically, the first loss difference is used for characterizing the difference between the performance of the first generator and the performance of the first discriminator, which may be obtained through calculation according to the objective function of the first generator and a loss function of the first discriminator with respect to false pictures. In the same way, the second loss difference is used for characterizing the difference between the performance of the second generator and the performance of the second discriminator, which may be obtained through calculation according to an objective function of the second generator and a loss function of the second discriminator with respect to false pictures.
[0023] It should be understood that because the network model to be compressed is a well-trained GAN, the first generator and the first discriminator are in a Nash equilibrium state. After the pruning processing is performed on the first generator and the states of the convolution kernels in the first discriminator are configured, the first loss difference can be close to the second loss difference, that is, the Nash equilibrium between the second generator and the second discriminator can be maintained. In this way, the mode collapse phenomenon can be avoided.
[0024] Specifically, the specific implementation for configuring states of the convolution kernels in the first discriminator may be set by those skilled in the art according to the actual situation, and is not limited herein.
[0025] In one possible embodiment, S120 includes: freezing a retention factor corresponding to each convolution kernel in the second discriminator, and determining a first weight parameter of the second discriminator; freezing the first weight parameter of the second discriminator and a second weight parameter of the second generator, and determining respective retention factors; and repeatedly performing operations of determining the first weight parameter of the second discriminator and determining the respective retention factors until the absolute value of the difference value between the first loss difference and the second loss difference is less than the first preset threshold.
[0026] In the step, the retention factor is used for characterizing the importance of a convolution kernel corresponding the retention factor.
[0027] Specifically, a retention factor is configured for each convolution kernel in the first discriminator to obtain a second discriminator, and an initial value of the retention factor corresponding to each convolution kernel may be 1. In the second discriminator obtained finally, the values of the retention factors corresponding to the convolution kernels in the activated state may be 1, and the values of the retention factors corresponding to the convolution kernels in the suppressed state may be 0.
[0028] Specifically, the first weight parameter described herein includes weight parameters corresponding to other elements in the second discriminator other than the respective retention factors for the convolution kernels. The second weight parameter described herein includes weight parameters corresponding to elements (e.g., convolution kernels) in the second generator.
[0029] Specifically, the specific implementation for determining the first weight parameter of the second discriminator may be set by those skilled in the art according to the actual situation, and is not limited herein. In one possible embodiment, determining the first weight parameter of the second discriminator includes: determining the first weight parameter of the second discriminator according to an objective function of the second discriminator. In one possible embodiment, before determining the first weight parameter of the second discriminator according to the objective function of the second discriminator, the method may further include determining the second weight parameter of the second generator according to an objective function of the second generator.
[0030] Specifically, the specific implementation for determining the respective retention factors may be set by those skilled in the art according to the actual situation, and is not limited herein. In one possible embodiment, the respective retention factors are determined according to an objective function of the respective retention factors.
[0031] Illustratively, firstly, the retention factor remains unchanged, and the second weight parameter of the second generator is determined by optimizing the objective function of the second generator; and the first weight parameter of the second discriminator is determined by optimizing the objective function of the second discriminator. Then, the first weight parameter of the second discriminator and the second weight parameter of the second generator remain unchanged, and the retention factors for the respective convolution kernels are determined by optimizing the objective function of the retention factors. When it is detected that the absolute value of the difference value between the first loss difference and the second loss difference is less than the first preset threshold, the training may be ended; when it is detected that the absolute value of the difference between the first loss difference and the second loss difference is greater than or equal to the first preset threshold, the method returns to perform the operations of "determining the second weight parameter of the second generator and determining the first weight parameter of the second discriminator" and "determining the retention factors for the respective convolution kernels" until the absolute value of the difference value between the first loss difference and the second loss difference is less than the first preset threshold.
[0032] In the embodiments of the present disclosure, the second generator is obtained by performing pruning processing on the first generator; and the states of the convolution kernels in the first discriminator are configured to enable a part of the convolution kernels to be in the activated state and the other part of the convolution kernels to be in the suppressed state, so as to obtain the second discriminator; the loss difference between the first generator and the first discriminator is the first loss difference, the loss difference between the second generator and the second discriminator is the second loss difference, and the absolute value of the difference value between the first loss difference and the second loss difference is less than the first preset threshold. Because the embodiments of the present disclosure can cooperatively compress the generator and the discriminator, the compressed generator and the compressed discriminator can maintain Nash equilibrium, thereby avoiding the mode collapse phenomenon. Moreover, the respective retention factors and the second weight parameter of the second generator and the first weight parameter of the second discriminator are alternately determined until the absolute value of the difference value between the first loss difference and the second loss difference is less than the first preset threshold, and the performance of the second generator and the second discriminator may be optimized in the process of approximating the second loss difference to the first loss difference.
[0033] FIG. 2 is a flowchart of a network model compression method according to an embodiment of the present disclosure. As shown in FIG. 2, the method of the present embodiment includes the following S210-S280.
[0035] S220: determining an objective function of the second generator according to a loss function of the second generator.
[0036] Specifically, the specific implementation for S220 may be set by those skilled in the art according to the actual situation, and is not limited herein.
[0037] Illustratively, the objective function of the second generator is as follows:
where LG S represents the objective function of the second generator, GS(Z) represents a false picture generated by the second generator according to a noise signal, DS(GS(Z)) represents a response value of the second discriminator to the false picture generated by the second generator, fS G(-DS(GS(Z))) represents the loss function of the second generator, E(*) represents an expected value of a distribution function, and p(z) represents the noise distribution.
[0038] S230: determining an objective function of a second discriminator according to a loss function of the second discriminator with respect to real pictures and a loss function of the second discriminator with respect to false pictures.
[0039] Specifically, the specific implementation for S230 may be set by those skilled in the art according to the actual situation, and is not limited herein.
[0040] Illustratively, the objective function of the second discriminator is as follows:
where LD S represents the objective function of the second discriminator, D(x) represents a response value of the second discriminator to a real picture, fS D(-D(x)) represents the loss function of the second discriminator with respect to real pictures, E(*) represents an expected value of the distribution function, pdata represents the distribution of real pictures, GS(Z) represents a false picture generated by the second generator according to a noise signal, DS(GS(Z)) represents a response value of the second discriminator to the false picture generated by the second generator, fS D(DS(GS(Z))) represents the loss function of the second discriminator with respect to false picture generated by the second generator, and p(z) represents the noise distribution.
[0041] S240: determining the objective function of the respective retention factors according to the objective function of the second generator, the objective function of the second discriminator, the loss function of the second discriminator with respect to false pictures, an objective function of the first generator, and a loss function of the first discriminator with respect to false pictures.
[0042] Specifically, the specific implementation for S240 may be set by those skilled in the art according to the actual situation, and is not limited herein.
[0043] Illustratively, the objective function of the retention factor is as follows:
where Larch represents the objective function of the retention factor, LG S represents the objective function of the second generator, LD S represents the objective function of the second discriminator, and LSDfake represents the loss function of the second discriminator with respect to false pictures, whose detailed explanations are shown above and will not be repeated herein; LG T represents the objective function of the first generator, LTDfake represents the loss function of the first discriminator with respect to false pictures, GT(Z) represents a false picture generated by the first generator according to a noise signal, DT(GT(Z)) represents a response value of the first discriminator to the false picture generated by the first generator, fT G(-DT(GT(Z))) represents the loss function of the first generator, fT D(DT(GT(Z))) represents the loss function of the first discriminator with respect to false pictures generated by the first generator, E(*) represents an expected value of the distribution function, and p(z) represents the noise distribution.
[0044] S250: freezing a retention factor corresponding to each convolution kernel in the second discriminator, and determining a second weight parameter of the second generator according to the objective function of the second generator; and determining a first weight parameter of the second discriminator according to the objective function of the second discriminator.
[0045] Specifically, the specific implementation for S250 may be set by those skilled in the art according to the actual situation, and is not limited herein.
[0046] Illustratively, the retention factor for each convolution kernel remains unchanged, and the second weight parameter of the second generator is updated such that the objective function LG S of the second generator becomes smaller, and thus the second weight parameter of the second generator is determined; the first weight parameter of the second discriminator is updated such that the objective function LD S of the second discriminator becomes smaller, and thus the first weight parameter of the second discriminator is determined.
[0047] S260: freezing the first weight parameter of the second discriminator and the second weight parameter of the second generator, and determining the retention factors according to the objective function of the retention factors.
[0048] Specifically, the specific implementation for S260 may be set by those skilled in the art according to the actual situation, and is not limited herein.
[0049] Illustratively, the second weight parameter of the second generator and the first weight parameter of the second discriminator remain unchanged, and the retention factor for each convolution kernel is updated, such that the objective function Larch of the retention factor becomes smaller, and thus the respective retention factors are determined.
[0050] S270: determining a second loss difference between the second generator and the second discriminator according to the objective function of the second generator and the loss function of the second discriminator with respect to false pictures.
[0051] Specifically, the specific implementation for S270 may be set by those skilled in the art according to the actual situation, and is not limited herein. In one possible embodiment, the absolute value of the difference value between the objective function of the second generator and the loss function of the second discriminator with respect to false pictures is taken as a second loss difference.
[0052] Illustratively, the second loss difference is as follows:
where ΔLS represents the second loss difference, and the specific explanations of LG S and LSDfake are shown above and will not be repeated herein.
[0053] S280: determining whether an absolute value of the difference value between a first loss difference and the second loss difference is less than a first preset threshold or not; if yes, ending the training; and if no, returning to perform S250.
[0054] Specifically, the specific implementation for S280 may be set by those skilled in the art according to the actual situation, and is not limited herein.
[0055] In one possible embodiment, the absolute value of the difference value between the objective function of the first generator and the loss function of the first discriminator with respect to false pictures is taken as the first loss difference; an absolute value of the difference value between the first loss difference and the second loss difference is calculated; and whether the absolute value of the difference between the first loss difference and the second loss difference is less than the first preset threshold or not is determined.
[0056] Illustratively, the first loss difference is as follows:
where ΔLT represents the first loss difference, and the specific explanations of LG T and LTDfake are shown above and will not be repeated herein.
[0057] Then the absolute value of the difference value between the first loss difference and the second loss difference is as follows:
then whether the absolute value ΔL of the difference between the first loss difference and the second loss difference is less than the first preset threshold or not is determined; if yes, the training is ended; if no, return to perform S250.
[0058] In the embodiments of the present disclosure, the objective function of the second generator is determined according to the loss function of the second generator, and the second weight parameter of the second generator is determined according to the objective function of the second generator; the objective function of the second discriminator is determined according to the loss function of the second discriminator with respect to real pictures and the loss function of the second discriminator with respect to false pictures, and the first weight parameter of the second discriminator is determined according to the objective function of the second discriminator; the objective function of the retention factors is determined according to the objective function of the first generator, the loss function of the first discriminator with respect to false pictures, the objective function of the second generator, the objective function of the second discriminator, and the loss function of the second discriminator with respect to false pictures, and the retention factors are determined according to the objective function of the retention factors. The loss function of the second generator and the loss function of the second discriminator with respect to false pictures can be reduced in the process of approximating the second loss difference to the first loss difference, and moreover, the loss function of the second discriminator with respect to real pictures can be improved, and thus the performance of the second generator and the second discriminator is optimized.
[0059] FIG. 3 is a flowchart of a network model compression method according to an embodiment of the present disclosure. As shown in FIG. 3, the method of the present embodiment includes the following S310-S390.
[0061] S320: taking the first generator and a first discriminator as a teacher generative adversarial network, and taking the second generator and a second discriminator as a student generative adversarial network.
[0062] S330: determining an objective function of the second generator according to a distillation objective function between the teacher generative adversarial network and the student generative adversarial network, and a loss function of the second generator.
[0063] Specifically, the first generator and the first discriminator are a well-trained network model with good accuracy and stability. The network model is taken as the teacher generative adversarial network to guide the student generative adversarial network for learning, which is beneficial to improving the performance of the second generator. There is usually a distillation loss when the teacher generative adversarial network guides the student generative adversarial network for learning, and the distillation loss can be reduced by optimizing the distillation objective function.
[0064] Specifically, the specific implementation for determining the distillation objective function may be set by those skilled in the art according to the actual situation, and is not limited herein.
[0065] In one possible embodiment, the specific implementation for determining the distillation objective function is as follows: determining a first similarity metric function according to a similarity between intermediate feature maps of at least one layer in the first generator and the second generator; inputting false pictures generated by the first generator into the first discriminator to obtain a first intermediate feature map of at least one layer in the first discriminator; inputting false pictures generated by the second generator into the first discriminator to obtain a second intermediate feature map of at least one layer in the first discriminator; determining a second similarity metric function according to a similarity between the first intermediate feature map of the at least one layer and the second intermediate feature map of the at least one layer; and determining the distillation objective function according to the first similarity metric function and the second similarity metric function.
[0066] Specifically, an intermediate feature map of the first generator refers to output information from a certain layer of the first generator; and an intermediate feature map of the second generator refers to output information from a certain layer of the second generator.
[0067] Specifically, the intermediate feature maps used for determining the first similarity metric function have the following characteristics: the intermediate feature maps obtained from the first generator correspond to the intermediate feature maps obtained from the second generator in a one-to-one manner; that is, when an intermediate feature map of a certain layer (e.g., the first layer) is obtained from the first generator, an intermediate feature map with the same number of network layer (e.g., the first layer) needs to be obtained from the second generator. The intermediate feature maps obtained from which layers of the first generator and the second generator may be set by those skilled in the art according to the actual situation, and are not limited herein.
[0068] Specifically, the first similarity metric function is used for characterizing the degree of approximation of the intermediate layer information of the first generator and the second generator. The specific method for determining the first similarity metric function may be set by those skilled in the art according to the actual situation.
[0069] In one possible embodiment, determining the first similarity metric function according to the similarity between intermediate feature maps of at least one layer in the first generator and the second generator includes: inputting an intermediate feature map of an i-th layer in the first generator and an intermediate feature map of an i-th layer in the second generator into a similarity metric function to obtain a first sub-similarity metric function corresponding to the i-th layer, where i is a positive integer, i takes a value from 1 to M, and M is the total number of layers of the first generator and the second generator; and determining the first similarity metric function according to first sub-similarity metric functions corresponding to respective layers.
[0070] Specifically, the specific structure of the similarity metric function may be set by those skilled in the art according to the actual situation, and it is not limited herein. In one possible embodiment, the similarity metric function includes an MSE loss function and a Texture loss function.
[0071] Illustratively, the similarity metric function is as follows:
where Ô, O represent two different intermediate feature maps inputted into the similarity metric function, respectively, Gpq(Ô) represents an inner product between the feature of a p-th channel and the feature of a q-th channel in an intermediate feature map Ô, Gij(O) represents an inner product between the feature of a p-th channel and the feature of a q-th channel in an intermediate feature map O, cl represents the total number of channels in the intermediate feature maps Ô, O inputted into the similarity metric function, and
represents the MSE loss function.
[0072] Illustratively, the first similarity metric function is as follows:
where D1 represents the first similarity metric function, LG represents the total number of intermediate feature maps obtained from the first generator, p(z) represents the noise distribution,
represents a first sub-similarity metric function corresponding to the i-th layer,
represents the intermediate feature map obtained from the i-th layer of the second generator,
represents the intermediate feature map obtained from the i-th layer of the first generator, and
represents a learnable 1×1 convolutional layer used for converting the total number of channels of
into the same number of the total number of channels of
.
[0073] Specifically, the first intermediate feature map refers to output information from a certain layer of the first discriminator when a false picture generated by the first generator is input into the first discriminator; and the second intermediate feature map refers to output information from a certain layer of the first discriminator when a false picture generated by the second generator is input into the first discriminator.
[0074] It should be understood that, compared with an additional network model used for extracting the intermediate feature maps of the first generator and the second generator, the first discriminator used for extracting the intermediate feature maps of the first generator and the second generator in the embodiments of the present disclosure has the advantage of having a high correlation to the generation tasks of the first generator and the second generator and having a good capability of distinguishing the real images from the false images.
[0075] Specifically, the first intermediate feature maps correspond to the second intermediate feature maps in a one-to-one manner; that is, when the first intermediate feature map is an intermediate feature map of a certain layer (e.g., the first layer) in the first discriminator, its corresponding second intermediate feature map is an intermediate feature map of this layer (e.g., the first layer) in the second discriminator. The intermediate feature maps obtained from which layers of the first discriminator and the second discriminator may be set by those skilled in the art according to the actual situation, and are not limited herein.
[0076] Specifically, the second similarity metric function is used for characterizing the degree of approximation of the intermediate layer information of the first discriminator and the second discriminator. The specific method for determining the second similarity metric function may be set by those skilled in the art according to the actual situation.
[0077] In one possible embodiment, determining the second similarity metric function according to a similarity between the first intermediate feature map and the second intermediate feature map corresponding to each layer includes: inputting a first intermediate feature map and a second intermediate feature map corresponding to a j-th layer into a similarity metric function to obtain a second sub-similarity metric function corresponding to the j-th layer, where j is a positive integer, 1 ≤ j ≤ N, j take a value from 1 to N, and N is the total number of layers of the first discriminator; and determining the second similarity metric function according to second sub-similarity metric functions corresponding to respective layers.
[0078] Illustratively, the second similarity metric function is as follows:
where D2 represents the second similarity metric function, LD represents the total number of intermediate feature maps obtained from the first discriminator, p(z) represents the noise distribution,
represents a second sub-similarity metric function corresponding to the j-th layer,
represents the first intermediate feature map corresponding to the j-th layer, and
represents a second intermediate feature map corresponding to the j-th layer.
[0079] Specifically, the specific implementation for determining the distillation objective function according to the first similarity metric function and the second similarity metric function may be set by those skilled in the art according to the actual situation, and is not limited herein.
[0080] In one possible embodiment, the first similarity metric function and the second similarity metric function are summed according to weights to determine the distillation objective function.
[0081] In one possible embodiment, the first similarity metric function and the second similarity metric function are directly summed to determine the distillation objective function.
[0082] Illustratively, the distillation objective function is as follows:
where
represents the distillation objective function, D1 represents the first similarity metric function, and D2 represents the second similarity metric function.
[0083] Specifically, the specific implementation for S330 may be set by those skilled in the art according to the actual situation, and is not limited herein.
[0084] In one possible embodiment, the distillation objective function and the objective function component determined according to the loss function of the second generator are directly summed to determine the objective function of the second generator.
[0085] In one possible embodiment, the distillation objective function and the objective function component determined according to the loss function of the second generator are summed according to weights to determine the objective function of the second generator.
[0086] Illustratively, the objective function of the second generator is as follows:
where (LG S)' represents the objective function component determined according to the loss function of the second generator. It should be understood by those skilled in the art that when the first generator and the first discriminator are not taken as the teacher generative adversarial network, (LG S)' may be the objective function of the second generator, as shown in the example of the method in FIG. 2. Therefore, the specific explanation of (LG S)' is shown above and will not be repeated herein. Ldistill represents the distillation objective function, and γ represents the weight parameter of the distillation objective function.
[0087] S340: determining an objective function of the second discriminator according to a loss function of the second discriminator with respect to real pictures and a loss function of the second discriminator with respect to false pictures.
[0088] S350: determining an objective function of a retention factor according to the objective function of the second generator, the objective function of the second discriminator, the loss function of the second discriminator with respect to false pictures, an objective function of the first generator, and a loss function of the first discriminator with respect to false pictures.
[0089] S360: freezing the retention factors corresponding the respective convolution kernels in the second discriminator, and determining a second weight parameter of the second generator according to the objective function of the second generator; and determining a first weight parameter of the second discriminator according to the objective function of the second discriminator.
[0090] S370: freezing the first weight parameter of the second discriminator and the second weight parameter of the second generator, and determining the respective retention factors according to the objective function of the respective retention factors; when a retention factor is less than a second preset threshold, determining the retention factor to be 0; and when a retention factor is greater than or equal to the second preset threshold, determining the retention factor to be 1.
[0091] Specifically, the specific value of the second preset threshold may be set by those skilled in the art according to the actual situation, and it is not limited herein.
[0092] It should be understood that, by setting the retention factor to be 0 when the value of the retention factor is less than the second preset threshold, and setting the retention factor to be 1 when the value of the retention factor is greater than or equal to the second preset threshold, the process of setting the convolution kernels in a part of the first discriminator to be in the activated state and setting the convolution kernels in the other part of the first discriminator to be in the suppressed state is sped up, thereby improving the compression efficiency of the network model.
[0093] S380: determining a second loss difference between the second generator and the second discriminator according to the objective function of the second generator and the loss function of the second discriminator with respect to false pictures.
[0094] S390: determining whether an absolute value of the difference value between a first loss difference and the second loss difference is less than a first preset threshold or not; if yes, ending the training; and if no, returning to perform S360.
[0095] In the embodiments of the present disclosure, the intermediate layer feature maps in the first generator and the first discriminator are simultaneously utilized through the distillation method as additional supervision information to help the second generator to generate high-quality images. In this way, a lightweight second generator can be obtained, and it can also be ensured that the lightweight second generator can generate high-quality images.
[0096] FIG. 4 is a flowchart of an image generation method according to an embodiment of the present disclosure, and the method can be executed by an electronic device. The electronic device may be illustratively understood as a device with a computing function, such as a portable Android device, a laptop computer, or a desktop computer. As shown in FIG. 4, the method of the present embodiment includes the following S410-S420.
[0097] S410: inputting a random noise signal into a second generator to enable the second generator to generate a false image according to the random noise signal.
[0098] S420: inputting the false image into a second discriminator to enable that the second discriminator discriminates that the false image is true and then outputs the false image.
[0099] In the steps, the second generator and the second discriminator are obtained by using the method according to any of the embodiments in FIG. 1 to FIG.3 described above.
[0100] In the embodiments of the present disclosure, the second generator for generating a false image and the second discriminator for outputting the false image are obtained by using the network model compression method provided in the embodiments of the present disclosure. Because the second generator and the second discriminator are relatively lightweight, even if they are deployed to edge devices with limited resources, there will be no longer delay.
[0101] FIG. 5 is a schematic diagram of a structure of a network model compression apparatus according to an embodiment of the present disclosure, and the network model compression apparatus may be understood as the network model compression device described above or part of functional modules in the network model compression device described above. The network model compression apparatus can compress a network model to be compressed, and the network model to be compressed includes a first generator and a first discriminator. As shown in FIG. 5, the network model compression apparatus 500 includes a pruning module 510 and a configuration module 520.
[0102] The pruning module 510 is configured to perform pruning processing on the first generator to obtain a second generator.
[0103] The configuration module 520 is configured to configure states of convolution kernels in the first discriminator to enable a part of the convolution kernels to be in an activated state and the other part of the convolution kernels to be in a suppressed state, so as to obtain a second discriminator.
[0104] A loss difference between the first generator and the first discriminator is a first loss difference, a loss difference between the second generator and the second discriminator is a second loss difference, and an absolute value of a difference between the first loss difference and the second loss difference is less than a first preset threshold.
[0105] In one embodiment, the configuration module 520 includes:
a first determination submodule that may be configured to freeze a retention factor corresponding to each convolution kernel in the second discriminator, and determine a first weight parameter of the second discriminator, in which the retention factor is used for characterizing the importance of a convolution kernel corresponding the retention factor;
a second determination submodule that may be configured to freeze the first weight parameter of the second discriminator and a second weight parameter of the second generator, and determine respective retention factors;
and a repetition submodule that may be configured to repeatedly perform operations of determining the first weight parameter of the second discriminator and determining the respective retention factors until the absolute value of the difference value between the first loss difference and the second loss difference is less than the first preset threshold.
[0106] In one embodiment, the first weight parameter includes weight parameters corresponding to other elements in the second discriminator other than the respective retention factors.
[0107] In one embodiment, the second weight parameter includes weight parameters corresponding to elements in the second generator.
[0108] In one embodiment, the first determination submodule includes:
a first determination unit that may be configured to determine the second weight parameter of the second generator according to an objective function of the second generator;
and a second determination unit that may be configured to determine the first weight parameter of the second discriminator according to an objective function of the second discriminator.
[0109] In one embodiment, the apparatus further includes:
an objective function determination module of the second generator, which may be configured to determine, before the second weight parameter of the second generator are determined according to the objective function of the second generator, the objective function of the second generator according to a loss function of the second generator;
and an objective function determination module of the second discriminator, which may be configured to determine the objective function of the second discriminator according to a loss function of the second discriminator with respect to real pictures and a loss function of the second discriminator with respect to false pictures.
[0110] In one embodiment, the apparatus further includes a teacher and student determination module that may be configured to take, before the objective function of the second generator is determined according to the loss function of the second generator, the first generator and the first discriminator as a teacher generative adversarial network, and take the second generator and the second discriminator as a student generative adversarial network;
and the objective function determination module of the second generator may be specifically configured to determine the objective function of the second generator according to a distillation objective function between the teacher generative adversarial network and the student generative adversarial network, and the loss function of the second generator.
[0111] In one embodiment, the objective function determination module of the second generator may be specifically configured to sum, according to weights, the distillation objective function and an objective function component determined according to the loss function of the second generator, to determine the objective function of the second generator.
[0112] In one embodiment, the apparatus further includes:
a first similarity metric function determination module that may be configured to determine, before the objective function of the second generator is determined according to the distillation objective function between the teacher generative adversarial network and the student generative adversarial network, and the loss function of the second generator, a first similarity metric function according to a similarity between intermediate feature maps of at least one layer in the first generator and the second generator;
a first intermediate feature map acquisition module that may be configured to input false pictures generated by the first generator into the first discriminator to obtain a first intermediate feature map of at least one layer in the first discriminator;
a second intermediate feature map acquisition module that may be configured to input false pictures generated by the second generator into the first discriminator to obtain a second intermediate feature map of at least one layer in the first discriminator;
a second similarity metric function determination module that may be configured to determine a second similarity metric function according to a similarity between the first intermediate feature map of the at least one layer and the second intermediate feature map of the at least one layer;
and a distillation objective function determination module that may be configured to determine the distillation objective function according to the first similarity metric function and the second similarity metric function.
[0113] In one embodiment, the first similarity metric function determination module includes:
a first sub-similarity metric function determination submodule that may be configured to input an intermediate feature map of an i-th layer in the first generator and an intermediate feature map of an i-th layer in the second generator into a similarity metric function to obtain a first sub-similarity metric function corresponding to the i-th layer, where i is a positive integer, i takes a value from 1 to M, and M is the total number of layers of the first generator and the second generator;
and a first similarity metric function determination submodule that may be configured to determine the first similarity metric function according to first sub-similarity metric functions corresponding to respective layers.
[0114] In one embodiment, the second similarity metric function determination module includes:
a second sub-similarity metric function determination submodule that may be configured to input a first intermediate feature map and a second intermediate feature map corresponding to a j-th layer into a similarity metric function to obtain a second sub-similarity metric function corresponding to the j-th layer, where j is a positive integer, 1 ≤ j ≤N, j takes a value from 1 to N, and N is the total number of layers of the first discriminator;
and a second similarity metric function determination submodule that may be configured to determine the second similarity metric function according to second sub-similarity metric functions corresponding to respective layers.
[0115] In one embodiment, the second determination submodule includes:
a third determination unit that may be configured to determine the respective retention factors according to an objective function of the respective retention factors;
a fourth determination unit that may be configured to determine a retention factor to be 0 when the retention factor is less than a second preset threshold;
and a fifth determination unit that may be configured to determine a retention factor to be 1 when the retention factor is greater than or equal to the second preset threshold.
[0116] In one embodiment, the apparatus further includes an objective function determination module for the retention factor, which may be configured to determine, before the respective retention factors are determined according to the objective function of the respective retention factors, the objective function of the respective retention factors according to an objective function of the second generator, an objective function of the second discriminator, a loss function of the second discriminator with respect to false pictures, an objective function of the first generator, and a loss function of the first discriminator with respect to false pictures.
[0117] The apparatus of the present embodiment can execute the method of any of the embodiments in FIG. 1 to FIG.3 described above, whose execution mode and beneficial effects are similar and will not be repeated herein.
[0118] Illustratively, FIG. 6 is a schematic diagram of a structure of a network model compression device according to an embodiment of the present disclosure. Referring specifically to FIG. 6 below, a schematic diagram of a structure of a network model compression device 600 suitable for implementation in the embodiments of the present disclosure is shown. The network model compression device 600 in the embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcasting receiver, a personal digital assistant (PDA), a portable Android device (PAD), a portable media player (PMP), a vehicle-mounted terminal (e.g., a vehicle-mounted navigation terminal) or the like, and a fixed terminal such as a digital TV, a desktop computer, or the like. The network model compression device illustrated in FIG. 6 is merely an example, and should not pose any limitation to the functions and the range of use of the embodiments of the present disclosure.
[0119] As shown in FIG. 6, the network model compression device 600 may include a processing apparatus 601 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various suitable actions and processing according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage apparatus 608 into a random-access memory (RAM) 603. The RAM 603 further stores various programs and data required for operations of the network model compression device 600. The processing apparatus 601, the ROM 602, and the RAM 603 are interconnected by means of a bus 604. An input/output (I/O) interface 605 is also connected to the bus 604.
[0120] Usually, the following apparatuses may be connected to the I/O interface 605: an input apparatus 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, or the like; an output apparatus 607 including, for example, a liquid crystal display (LCD), a loudspeaker, a vibrator, or the like; a storage apparatus 608 including, for example, a magnetic tape, a hard disk, or the like; and a communication apparatus 609. The communication apparatus 609 may allow the network model compression device 600 to be in wireless or wired communication with other devices to exchange data. While FIG. 6 illustrates the network model compression device 600 having various apparatuses, it should be understood that not all of the illustrated apparatuses are necessarily implemented or included. More or fewer apparatuses may be implemented or included alternatively.
[0121] Particularly, according to some embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried by a non-transitory computer-readable medium. The computer program includes program code for performing the methods shown in the flowcharts. In such embodiments, the computer program may be downloaded online through the communication apparatus 609 and installed, or may be installed from the storage apparatus 608, or may be installed from the ROM 602. When the computer program is executed by the processing apparatus 601, the above-mentioned functions defined in the methods of some embodiments of the present disclosure are performed.
[0122] It should be noted that the above-mentioned computer-readable medium in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. For example, the computer-readable storage medium may be, but not limited to, an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof. More specific examples of the computer-readable storage medium may include but not be limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random-access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination of them. In the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus or device. In the present disclosure, the computer-readable signal medium may include a data signal that propagates in a baseband or as a part of a carrier and carries computer-readable program code. The data signal propagating in such a manner may take a plurality of forms, including but not limited to an electromagnetic signal, an optical signal, or any appropriate combination thereof. The computer-readable signal medium may also be any other computer-readable medium than the computer-readable storage medium. The computer-readable signal medium may send, propagate or transmit a program used by or in combination with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted by using any suitable medium, including but not limited to an electric wire, a fiber-optic cable, radio frequency (RF) and the like, or any appropriate combination of them.
[0123] In some implementation modes, the client and the server may communicate with any network protocol currently known or to be researched and developed in the future such as hypertext transfer protocol (HTTP), and may communicate (via a communication network) and interconnect with digital data in any form or medium. Examples of communication networks include a local area network (LAN), a wide area network (WAN), the Internet, and an end-to-end network (e.g., an ad hoc end-to-end network), as well as any network currently known or to be researched and developed in the future.
[0124] The above-mentioned computer-readable medium may be included in the above-mentioned network model compression device, or may also exist alone without being assembled into the network model compression device.
[0125] The computer-readable medium carries one or more programs. The one or more programs, when executed by the network model compression device, cause the network model compression device to: perform pruning processing on the first generator to obtain the second generator; and configure the states of the convolution kernels in the first discriminator to enable a part of the convolution kernels to be in the activated state and the other part of the convolution kernels to be in the suppressed state, so as to obtain the second discriminator, where the loss difference between the first generator and the first discriminator is the first loss difference, the loss difference between the second generator and the second discriminator is the second loss difference, and the absolute value of the difference between the first loss difference and the second loss difference is less than the first preset threshold.
[0126] The computer program code for performing the operations of the present disclosure may be written in one or more programming languages or a combination thereof. The above-mentioned programming languages include but are not limited to object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" programming language or similar programming languages. The program code may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the scenario related to the remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
[0127] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, a program segment, or a portion of code, including one or more executable instructions for implementing specified logical functions. It should also be noted that, in some alternative implementations, the functions noted in the blocks may also occur out of the order noted in the accompanying drawings. For example, two blocks shown in succession may, in fact, can be executed substantially concurrently, or the two blocks may sometimes be executed in a reverse order, depending upon the functionality involved. It should also be noted that, each block of the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may also be implemented by a combination of dedicated hardware and computer instructions.
[0128] The modules or units involved in the embodiments of the present disclosure may be implemented in software or hardware. Among them, the name of the module or unit does not constitute a limitation of the unit itself under certain circumstances.
[0129] The functions described herein above may be performed, at least partially, by one or more hardware logic components. For example, without limitation, available exemplary types of hardware logic components include: a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), an application specific standard product (ASSP), a system on chip (SOC), a complex programmable logical device (CPLD), etc.
[0130] In the context of the present disclosure, the machine-readable medium may be a tangible medium that may include or store a program for use by or in combination with an instruction execution system, apparatus or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium includes, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semi-conductive system, apparatus or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage medium include electrical connection with one or more wires, portable computer disk, hard disk, random-access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing.
[0131] The embodiments of the present disclosure further provide a computer-readable storage medium storing a computer program. The computer program, when executed by a processor, implements the method of any of the embodiments in FIG. 1 to FIG.3, whose execution mode and beneficial effects are similar and will not be repeated herein.
[0132] It should be noted that in the present disclosure, relational terms such as "first," "second," etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any actual relationship or order between these entities or operations. Furthermore, the terms "comprise," "comprising," "include," "including," etc., or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a set of elements includes not only those elements, but also other elements not expressly listed, or other elements not expressly listed for the purpose of such a process, method, article or device, or elements that are inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "includes a ..." does not preclude the existence of additional identical elements in the process, method, article or device that includes the element.
inputting a random noise signal into a second generator to enable the second generator to generate a false image according to the random noise signal (S410); and
inputting the false image into a second discriminator to enable that the second discriminator discriminates that the false image is true and then outputs the false image (S420),
wherein the second generator and the second discriminator are obtained by using a network model compression method, a network model to be compressed comprises a first generator and a first discriminator, characterized in that the network model compression method comprises:
performing pruning processing on the first generator to obtain the second generator (S110) (S210) (S310); and
configuring states of convolution kernels in the first discriminator to enable a part of the convolution kernels to be in an activated state and the other part of the convolution kernels to be in a suppressed state, so as to obtain the second discriminator (S120),
wherein a loss difference between the first generator and the first discriminator is a first loss difference, a loss difference between the second generator and the second discriminator is a second loss difference, and an absolute value of a difference value between the first loss difference and the second loss difference is less than a first preset threshold,
wherein configuring states of convolution kernels in the first discriminator to enable a part of the convolution kernels to be in an activated state and the other part of the convolution kernels to be in a suppressed state, so as to obtain a second discriminator (S120), comprising:
freezing a retention factor corresponding to each convolution kernel in the second discriminator, and determining a first weight parameter of the second discriminator, wherein the retention factor is used for characterizing importance of a convolution kernel corresponding the retention factor (S250);
freezing the first weight parameter of the second discriminator and a second weight parameter of the second generator, and determining respective retention factors (S260); and
repeatedly performing operations of determining the first weight parameter of the second discriminator and determining the respective retention factors until the absolute value of the difference value between the first loss difference and the second loss difference is less than the first preset threshold,
wherein determining the respective retention factors comprises:
determining the respective retention factors according to an objective function of the respective retention factors;
in response to a retention factor being less than a second preset threshold, determining the retention factor to be 0; and
in response to a retention factor being greater than or equal to the second preset threshold, determining the retention factor to be 1 (S370);
wherein before determining the respective retention factors according to an objective
function of the respective retention factors, the network model compression method
further comprises:
determining the objective function of the respective retention factors according to
an objective function of the second generator, an objective function of the second
discriminator, a loss function of the second discriminator with respect to false pictures,
an objective function of the first generator, and a loss function of the first discriminator
with respect to false pictures (S240) (S350).
determining the first weight parameter of the second discriminator according to an objective function of the second discriminator; and
the network model compression method further comprises:
determining the second weight parameter of the second generator according to an objective
function of the second generator.
determining the objective function of the second generator according to a loss function of the second generator (S220); and
determining the objective function of the second discriminator according to a loss function of the second discriminator with respect to real pictures and a loss function of the second discriminator with respect to false pictures (S230) (S340).
taking the first generator and the first discriminator as a teacher generative adversarial network, and taking the second generator and the second discriminator as a student generative adversarial network (S320); and
determining the objective function of the second generator according to a loss function of the second generator (S220) comprises:
determining the objective function of the second generator according to a distillation objective function between the teacher generative adversarial network and the student generative adversarial network, and the loss function of the second generator (S330).
summing, according to weights, the distillation objective function and an objective function component determined according to the loss function of the second generator, to determine the objective function of the second generator; and
before determining the objective function of the second generator according to a distillation objective function between the teacher generative adversarial network and the student generative adversarial network, and the loss function of the second generator (S330), the network model compression method further comprises:
determining a first similarity metric function according to a similarity between intermediate feature maps of at least one layer in the first generator and the second generator;
inputting false pictures generated by the first generator into the first discriminator to obtain a first intermediate feature map of at least one layer in the first discriminator;
inputting false pictures generated by the second generator into the first discriminator to obtain a second intermediate feature map of at least one layer in the first discriminator;
determining a second similarity metric function according to a similarity between the first intermediate feature map of the at least one layer and the second intermediate feature map of the at least one layer; and
determining the distillation objective function according to the first similarity metric function and the second similarity metric function.
inputting an intermediate feature map of an i-th layer in the first generator and an intermediate feature map of an i-th layer in the second generator into a similarity metric function to obtain a first sub-similarity metric function corresponding to the i-th layer, wherein i is a positive integer, i takes a value from 1 to M, and M is a total number of layers of the first generator and the second generator; and
determining the first similarity metric function according to first sub-similarity metric functions corresponding to respective layers;
determining a second similarity metric function according to a similarity between the first intermediate feature map of the at least one layer and the second intermediate feature map of the at least one layer, comprises:
inputting a first intermediate feature map and a second intermediate feature map corresponding to a j-th layer into a similarity metric function to obtain a second sub-similarity metric function corresponding to the j-th layer, wherein j is a positive integer, 1 ≤ j ≤N, j takes a value from 1 to N, and N is a total number of layers of the first discriminator; and
determining the second similarity metric function according to second sub-similarity metric functions corresponding to respective layers.
a second generator, configured to receive a random noise signal to generate a false image according to the random noise signal; and
a second discriminator, configured to receive the false image to discriminate that the false image is true and then output the false image,
wherein the second generator and the second discriminator are obtained by using a network model compression apparatus (500),
a network model to be compressed comprises a first generator and a first discriminator, characterized in that the network model compression apparatus comprises a pruning module (510) and a configuration module (520);
wherein the pruning module (510) is configured to perform pruning processing on the first generator to obtain the second generator;
the configuration module (520) is configured to configure states of convolution kernels in the first discriminator to enable a part of the convolution kernels to be in an activated state and the other part of the convolution kernels to be in a suppressed state, so as to obtain the second discriminator; and
a loss difference between the first generator and the first discriminator is a first loss difference, a loss difference between the second generator and the second discriminator is a second loss difference, and an absolute value of a difference value between the first loss difference and the second loss difference is less than a first preset threshold,
wherein the configuration module (520) is configured to configure states of convolution kernels in the first discriminator to enable a part of the convolution kernels to be in an activated state and the other part of the convolution kernels to be in a suppressed state, so as to obtain a second discriminator, comprising:
freezing a retention factor corresponding to each convolution kernel in the second discriminator, and determining a first weight parameter of the second discriminator, wherein the retention factor is used for characterizing importance of a convolution kernel corresponding the retention factor (S250);
freezing the first weight parameter of the second discriminator and a second weight parameter of the second generator, and determining respective retention factors (S260); and
repeatedly performing operations of determining the first weight parameter of the second discriminator and determining the respective retention factors until the absolute value of the difference value between the first loss difference and the second loss difference is less than the first preset threshold,
wherein determining the respective retention factors comprises:
determining the respective retention factors according to an objective function of the respective retention factors;
in response to a retention factor being less than a second preset threshold, determining the retention factor to be 0; and
in response to a retention factor being greater than or equal to the second preset threshold, determining the retention factor to be 1 (S370);
wherein before determining the respective retention factors according to an objective
function of the respective retention factors, the network model compression method
further comprises:
determining the objective function of the respective retention factors according to
an objective function of the second generator, an objective function of the second
discriminator, a loss function of the second discriminator with respect to false pictures,
an objective function of the first generator, and a loss function of the first discriminator
with respect to false pictures (S240) (S350).
a memory, storing a computer program; and
a processor, configured to execute the computer program, wherein the computer program, when executed by the processor, causes the processor to perform the image generation method according to any one of claims 1 to 8.
Eingeben eines zufälligen Rauschsignals in einen zweiten Generator, um dem zweiten Generator zu ermöglichen, ein falsches Bild gemäß dem zufälligen Rauschsignal zu erzeugen (S410); und
Eingeben des falschen Bilds in einen zweiten Diskriminator, um zu ermöglichen, dass der zweite Diskriminator ausschließt, dass das falsche Bild echt ist, und dann das falsche Bild ausgibt (S420),
wobei der zweite Generator und der zweite Diskriminator unter Verwendung eines Netzwerkmodellkomprimierungsverfahrens erhalten werden, ein zu komprimierendes Netzwerkmodell einen ersten Generator und einen ersten Diskriminator umfasst, dadurch gekennzeichnet, dass das Netzwerkmodellkomprimierungsverfahren Folgendes umfasst:
Durchführen von Pruning-Verarbeitung an dem ersten Generator, um den zweiten Generator zu erhalten (S110) (S210) (S310); und
Konfigurieren von Zuständen von Faltungskernen im ersten Diskriminator, um einem Teil der Faltungskerne zu ermöglichen, sich in einem aktivierten Zustand zu befinden, und dem anderen Teil der Faltungskerne zu ermöglichen, sich in einem unterdrückten Zustand zu befinden, um den zweiten Diskriminator (S120) zu erhalten,
wobei eine Verlustdifferenz zwischen dem ersten Generator und dem ersten Diskriminator eine erste Verlustdifferenz ist, eine Verlustdifferenz zwischen dem zweiten Generator und dem zweiten Diskriminator eine zweite Verlustdifferenz ist und ein Absolutwert eines Differenzwerts zwischen der ersten Verlustdifferenz und der zweiten Verlustdifferenz kleiner als ein erster voreingestellter Schwellenwert ist,
wobei Konfigurieren von Zuständen von Faltungskerne im ersten Diskriminator, um einem Teil der Faltungskerne zu ermöglichen, sich in einem aktivierten Zustand zu befinden, und dem anderen Teil der Faltungskerne zu ermöglichen, sich in einem unterdrückten Zustand zu befinden, um einen zweiten Diskriminator (S120) zu erhalten, Folgendes umfasst:
Einfrieren eines Retentionsfaktors, der jedem Faltungskern im zweiten Diskriminator entspricht, und Bestimmen eines ersten Gewichtungsparameters des zweiten Diskriminators, wobei der Retentionsfaktor verwendet wird, um eine Wichtigkeit eines Faltungskerns entsprechend dem Retentionsfaktor zu charakterisieren (S250);
Einfrieren des ersten Gewichtungsparameters des zweiten Diskriminators und eines zweiten Gewichtungsparameters des zweiten Generators, und Bestimmen jeweiliger Retentionsfaktoren (S260); und
wiederholtes Durchführen von Operationen zum Bestimmen des ersten Gewichtungsparameters des zweiten Diskriminators und Bestimmen der jeweiligen Retentionsfaktoren, bis der Absolutwert des Differenzwerts zwischen der ersten Verlustdifferenz und der zweiten Verlustdifferenz kleiner als der erste voreingestellte Schwellenwert ist,
wobei Bestimmen der jeweiligen Retentionsfaktoren Folgendes umfasst:
Bestimmen der jeweiligen Retentionsfaktoren gemäß einer Zielfunktion der jeweiligen Retentionsfaktoren;
infolgedessen, dass ein Retentionsfaktor kleiner als ein zweiter voreingestellter Schwellenwert ist, Bestimmen des Retentionsfaktor als 0; und
infolgedessen, dass ein Retentionsfaktor größer oder gleich einem zweiten voreingestellten Schwellenwert ist, Bestimmen des Retentionsfaktor als 1 (S370);
wobei vor dem Bestimmen der jeweiligen Retentionsfaktoren gemäß einer Zielfunktion
der jeweiligen Retentionsfaktoren, das Netzwerkmodellkomprimierungsverfahren ferner
Folgendes umfasst:
Bestimmen der Zielfunktion der jeweiligen Retentionsfaktoren gemäß einer Zielfunktion
des zweiten Generators, einer Zielfunktion des zweiten Diskriminators, einer Verlustfunktion
des zweiten Diskriminators in Bezug auf falsche Bilder, einer Zielfunktion des ersten
Generators und einer Verlustfunktion des ersten Diskriminators in Bezug auf falsche
Bilder (S240) (S350).
Bestimmen des ersten Gewichtungsparameters des zweiten Diskriminators gemäß einer Zielfunktion des zweiten Diskriminators; und
wobei das Netzwerkmodellkomprimierungsverfahren ferner Folgendes umfasst:
Bestimmen des zweiten Gewichtungsparameters des zweiten Generators gemäß einer Zielfunktion
des zweiten Generators.
Bestimmen der Zielfunktion des zweiten Generators gemäß einer Verlustfunktion des zweiten Generators (S220); und
Bestimmen der Zielfunktion des zweiten Diskriminators gemäß einer Verlustfunktion des zweiten Diskriminators in Bezug auf reale Bilder und einer Verlustfunktion des zweiten Diskriminators in Bezug auf falsche Bilder (S230) (S340).
Heranziehen des ersten Generators und des ersten Diskriminators als ein generatives adversariales Lehrernetzwerk, und Heranziehen des zweiten Generators und des zweiten Diskriminators als ein generatives adversariales Schülernetzwerk (S320); und
wobei Bestimmen der Zielfunktion des zweiten Generators gemäß einer Verlustfunktion
des zweiten Generators (S220) Folgendes umfasst:
Bestimmen der Zielfunktion des zweiten Generators gemäß einer Destillationszielfunktion
zwischen dem generativen adversarialen Lehrernetzwerk und dem generativen adversarialen
Schülernetzwerk und der Verlustfunktion des zweiten Generators (S330).
Summieren, gemäß Gewichten, der Destillationszielfunktion und einer gemäß der Verlustfunktion des zweiten Generators bestimmten Zielfunktionskomponente, um die Zielfunktion des zweiten Generators zu bestimmen; und
wobei vor Bestimmen der Zielfunktion des zweiten Generators gemäß einer Destillationszielfunktion zwischen dem generativen adversarialen Lehrernetzwerk und dem generativen adversarialen Schülernetzwerk und der Verlustfunktion des zweiten Generators (S330), das Netzwerkmodellkomprimierungsverfahren ferner Folgendes umfasst:
Bestimmen einer ersten Ähnlichkeitsmetrikfunktion gemäß einer Ähnlichkeit zwischen Zwischenmerkmalskarten von mindestens einer Schicht in dem ersten Generator und dem zweiten Generator;
Eingeben von falschen Bildern, die vom ersten Generator erzeugt wurden, in den ersten Diskriminator, um eine erste Zwischenmerkmalskarte von mindestens einer Schicht im ersten Diskriminator zu erhalten;
Eingeben von falschen Bildern, die vom zweiten Generator erzeugt wurden, in den ersten Diskriminator, um eine zweite Zwischenmerkmalskarte von mindestens einer Schicht im ersten Diskriminator zu erhalten;
Bestimmen einer zweiten Ähnlichkeitsmetrikfunktion gemäß einer Ähnlichkeit zwischen der ersten Zwischenmerkmalskarte der mindestens einen Schicht und der zweiten Zwischenmerkmalskarte der mindestens einen Schicht; und
Bestimmen der Destillationszielfunktion gemäß der ersten Ähnlichkeitsmetrikfunktion und der zweiten Ähnlichkeitsmetrikfunktion.
Eingeben einer Zwischenmerkmalskarte einer i-ten Schicht im ersten Generator und einer Zwischenmerkmalskarte einer i-ten Schicht im zweiten Generator in eine Ähnlichkeitsmetrikfunktion, um eine erste Teilähnlichkeitsmetrikfunktion entsprechend der i-ten Schicht zu erhalten, wobei i eine positive ganze Zahl ist, i einen Wert von 1 bis M annimmt und M eine Gesamtzahl von Schichten des ersten Generators und des zweiten Generators ist; und
Bestimmen der ersten Ähnlichkeitsmetrikfunktion gemäß der ersten Teilähnlichkeitsmetrikfunktionen, die den jeweiligen Schichten entsprechen;
wobei Bestimmen einer zweiten Ähnlichkeitsmetrikfunktion gemäß einer Ähnlichkeit zwischen der ersten Zwischenmerkmalskarte der mindestens einen Schicht und der zweiten Zwischenmerkmalskarte der mindestens einen Schicht, Folgendes umfasst:
Eingeben einer ersten Zwischenmerkmalskarte und einer zweiten Zwischenmerkmalskarte entsprechend einer j-ten Schicht in eine Ähnlichkeitsmetrikfunktion, um eine zweite Teilähnlichkeitsmetrikfunktion entsprechend der j-ten Schicht zu erhalten, wobei j eine positive ganze Zahl ist, 1 ≤ j ≤N, i einen Wert von 1 bis N annimmt und N eine Gesamtzahl von Schichten des ersten Diskriminators ist; und
Bestimmen der zweiten Ähnlichkeitsmetrikfunktion gemäß der zweiten Teilähnlichkeitsmetrikfunktionen, die den jeweiligen Schichten entsprechen.
einen zweiten Generator, um dem zweiten Generator, der dazu konfiguriert ist, ein zufälliges Rauschsignal zu empfangen, um ein falsches Bild gemäß dem zufälligen Rauschsignal zu erzeugen; und
einen zweiten Diskriminator, der dazu konfiguriert ist, das falsche Bild zu empfangen, um auszuschließen, dass das falsche Bild echt ist, und dann das falsche Bild auszugeben,
wobei der zweite Generator und der zweite Diskriminator durch Verwendung einer Netzwerkmodell-Kompressionseinrichtung (500) erhalten werden,
wobei ein zu komprimierendes Netzwerkmodell einen ersten Generator und einen ersten Diskriminator umfasst, dadurch gekennzeichnet, dass die Netzwerkmodellkomprimierungseinrichtung ein Pruning-Modul (510) und ein Konfigurationsmodul (520) umfasst;
wobei das Pruning-Modul (510) dazu konfiguriert ist, einen Pruning-Prozess am ersten Generator durchzuführen, um den zweiten Generator zu erhalten;
das Konfigurationsmodul (520) dazu konfiguriert ist, Zustände von Faltungskernen im ersten Diskriminator zu konfigurieren, um einem Teil der Faltungskerne zu ermöglichen, sich in einem aktivierten Zustand zu befinden, und dem anderen Teil der Faltungskerne zu ermöglichen, sich in einem unterdrückten Zustand zu befinden, um den zweiten Diskriminator zu erhalten; und
eine Verlustdifferenz zwischen dem ersten Generator und dem ersten Diskriminator eine erste Verlustdifferenz ist, eine Verlustdifferenz zwischen dem zweiten Generator und dem zweiten Diskriminator eine zweite Verlustdifferenz ist und ein Absolutwert eines Differenzwerts zwischen der ersten Verlustdifferenz und der zweiten Verlustdifferenz kleiner als ein erster voreingestellter Schwellenwert ist,
wobei das Konfigurationsmodul (520) dazu konfiguriert ist, Zustände von Faltungskernen im ersten Diskriminator zu konfigurieren, um einem Teil der Faltungskerne zu ermöglichen, sich in einem aktivierten Zustand zu befinden, und dem anderen Teil der Faltungskerne zu ermöglichen, sich in einem unterdrückten Zustand zu befinden, um einen zweiten Diskriminator zu erhalten, Folgendes umfasst:
Einfrieren eines Retentionsfaktors, der jedem Faltungskern im zweiten Diskriminator entspricht, und Bestimmen eines ersten Gewichtungsparameters des zweiten Diskriminators, wobei der Retentionsfaktor verwendet wird, um eine Wichtigkeit eines Faltungskerns entsprechend dem Retentionsfaktor zu charakterisieren (S250);
Einfrieren des ersten Gewichtungsparameters des zweiten Diskriminators und eines zweiten Gewichtungsparameters des zweiten Generators, und Bestimmen jeweiliger Retentionsfaktoren (S260); und
wiederholtes Durchführen von Operationen zum Bestimmen des ersten Gewichtungsparameters des zweiten Diskriminators und Bestimmen der jeweiligen Retentionsfaktoren, bis der Absolutwert des Differenzwerts zwischen der ersten Verlustdifferenz und der zweiten Verlustdifferenz kleiner als der erste voreingestellte Schwellenwert ist,
wobei Bestimmen der jeweiligen Retentionsfaktoren Folgendes umfasst:
Bestimmen der jeweiligen Retentionsfaktoren gemäß einer Zielfunktion der jeweiligen Retentionsfaktoren;
infolgedessen, dass ein Retentionsfaktor kleiner als ein zweiter voreingestellter Schwellenwert ist, Bestimmen des Retentionsfaktor als 0; und
infolgedessen, dass ein Retentionsfaktor größer oder gleich einem zweiten voreingestellten Schwellenwert ist, Bestimmen des Retentionsfaktor als 1 (S370);
wobei vor dem Bestimmen der jeweiligen Retentionsfaktoren gemäß einer Zielfunktion
der jeweiligen Retentionsfaktoren, das Netzwerkmodellkomprimierungsverfahren ferner
Folgendes umfasst:
Bestimmen der Zielfunktion der jeweiligen Retentionsfaktoren gemäß einer Zielfunktion
des zweiten Generators, einer Zielfunktion des zweiten Diskriminators, einer Verlustfunktion
des zweiten Diskriminators in Bezug auf falsche Bilder, einer Zielfunktion des ersten
Generators und einer Verlustfunktion des ersten Diskriminators in Bezug auf falsche
Bilder (S240) (S350).
einen Speicher, der ein Computerprogramm speichert; und
einen Prozessor, der dazu konfiguriert ist, das Computerprogramm auszuführen, wobei das Computerprogramm, wenn es vom Prozessor ausgeführt wird, den Prozessor veranlässt, das Bilderzeugungsverfahren nach einem der Ansprüche 1 bis 8 durchzuführen.
l'entrée d'un signal de bruit aléatoire dans un second générateur pour permettre au second générateur de générer une fausse image selon le signal de bruit aléatoire (S410) ; et
l'entrer de la fausse image dans un second discriminateur pour permettre au second discriminateur de discriminer que la fausse image est vraie, puis émet la fausse image (S420),
dans lequel le second générateur et le second discriminateur sont obtenus par un procédé de compression de modèle de réseau, le modèle de réseau à compresser comprend un premier générateur et un premier discriminateur, caractérisé en ce que le procédé de compression de modèle de réseau comprend :
la réalisation d'un traitement d'élagage sur le premier générateur pour obtenir le second générateur (S110) (S210) (S310) ; et
la configuration d'états de noyaux de convolution dans le premier discriminateur pour permettre à une partie des noyaux de convolution d'être dans un état activé et à l'autre partie des noyaux de convolution d'être dans un état supprimé, de manière à obtenir le second discriminateur (S120),
dans lequel une différence de perte entre le premier générateur et le premier discriminateur est une première différence de perte, une différence de perte entre le second générateur et le second discriminateur est une seconde différence de perte, et une valeur absolue d'une valeur de différence entre la première différence de perte et la seconde différence de perte est inférieure à un premier seuil prédéfini,
dans lequel la configuration d'états de noyaux de convolution dans le premier discriminateur pour permettre à une partie des noyaux de convolution d'être dans un état activé et à l'autre partie des noyaux de convolution d'être dans un état supprimé, de manière à obtenir un second discriminateur (S120), comprend :
le gel d'un facteur de rétention correspondant à chaque noyau de convolution dans le second discriminateur, et la détermination d'un premier paramètre de poids du second discriminateur, dans lequel le facteur de rétention est utilisé pour la caractérisation de l'importance d'un noyau de convolution correspondant au facteur de rétention (S250) ;
le gel du premier paramètre de poids du second discriminateur et d'un second paramètre de poids du second générateur, et la détermination de facteurs de rétention respectifs (S260) ; et
la réalisation de manière répétée d'opérations de détermination du premier paramètre de poids du second discriminateur et la détermination des facteurs de rétention respectifs jusqu'à ce que la valeur absolue de la valeur de différence entre la première différence de perte et la seconde différence de perte soit inférieure au premier seuil prédéfini,
dans lequel la détermination des facteurs de rétention respectifs comprend :
la détermination des facteurs de rétention respectifs selon une fonction objective des facteurs de rétention respectifs ;
en réponse à un facteur de rétention étant inférieur à un second seuil prédéfini, la détermination que le facteur de rétention est de 0 ; et
en réponse à un facteur de rétention étant supérieur ou égal au second seuil prédéfini, la détermination que le facteur de rétention est de 1 (S370) ;
dans lequel, avant la détermination des facteurs de rétention respectifs selon une
fonction objective des facteurs de rétention respectifs, le procédé de compression
de modèle de réseau comprend en outre :
la détermination de la fonction objective des facteurs de rétention respectifs selon
une fonction objective du second générateur, une fonction objective du second discriminateur,
une fonction de perte du second discriminateur par rapport à de fausses images, une
fonction objective du premier générateur et une fonction de perte du premier discriminateur
par rapport à de fausses images (S240) (S350).
la détermination du premier paramètre de poids du second discriminateur selon une fonction objective du second discriminateur ; et
le procédé de compression de modèle de réseau comprend en outre :
la détermination du second paramètre de poids du second générateur selon une fonction
objective du second générateur.
la détermination de la fonction objective du second générateur selon une fonction de perte du second générateur (S220) ; et
la détermination de la fonction objective du second discriminateur selon une fonction de perte du second discriminateur par rapport à des images réelles et une fonction de perte du second discriminateur par rapport à de fausses images (S230) (S340).
la prise du premier générateur et du premier discriminateur comme un réseau antagoniste génératif d'enseignant, et la prise du second générateur et du second discriminateur comme un réseau antagoniste génératif d'étudiant (S320) ; et
la détermination de la fonction objective du second générateur selon une fonction
de perte du second générateur (S220) comprend :
la détermination de la fonction objective du second générateur selon une fonction
objective de distillation entre le réseau antagoniste génératif d'enseignant et le
réseau antagoniste génératif d'étudiant, et la fonction de perte du second générateur
(S330).
l'addition, selon des poids, de la fonction objective de distillation et d'une composante de fonction objective déterminée selon la fonction de perte du second générateur, pour déterminer la fonction objective du second générateur; et
avant la détermination de la fonction objective du second générateur selon une fonction objective de distillation entre le réseau antagoniste génératif d'enseignant et le réseau antagoniste génératif d'étudiant, et la fonction de perte du second générateur (S330), le procédé de compression de modèle de réseau comprend en outre :
la détermination d'une première fonction métrique de similarité selon une similarité entre des cartes de caractéristique intermédiaires d'au moins une couche du premier générateur et du second générateur ;
l'entrée de fausses images générées par le premier générateur dans le premier discriminateur pour obtenir une première carte de caractéristique intermédiaire d'au moins une couche dans le premier discriminateur ;
l'entrée de fausses images générées par le second générateur dans le premier discriminateur pour obtenir une seconde carte de caractéristique intermédiaire d'au moins une couche dans le premier discriminateur ;
la détermination d'une seconde fonction métrique de similarité selon une similarité entre la première carte de caractéristique intermédiaire de l'au moins une couche et la seconde carte de caractéristique intermédiaire de l'au moins une couche ; et
déterminer la fonction objective de distillation selon la première fonction métrique de similarité et la seconde fonction métrique de similarité.
l'entrée d'une carte de caractéristique intermédiaire d'une ie couche du premier générateur et une carte de caractéristique intermédiaire d'une ie couche du second générateur dans une fonction métrique de similarité pour obtenir une première fonction métrique de sous-similarité correspondant à la ie couche, dans lequel i est un entier positif, i prend une valeur de 1 à M, et M est un nombre total de couches du premier générateur et du second générateur ; et
la détermination de la première fonction métrique de similarité selon de premières fonctions métriques de sous-similarité correspondant à des couches respectives ;
la détermination d'une seconde fonction métrique de similarité selon une similarité entre la première carte de caractéristique intermédiaire de l'au moins une couche et la seconde carte de caractéristique intermédiaire de l'au moins une couche comprend :
l'entrée d'une première carte de caractéristique intermédiaire et d'une seconde carte de caractéristique intermédiaire correspondant à une je couche dans une fonction métrique de similarité pour obtenir une seconde fonction métrique de sous-similarité correspondant à la je couche, dans lequel j est un entier positif, 1 ≤ j ≤ N, j prend une valeur de 1 à N, et N est un nombre total de couches du premier discriminateur ; et
la détermination de la seconde fonction métrique de similarité selon de secondes fonctions métriques de sous-similarité correspondant à des couches respectives.
un second générateur, configuré pour recevoir un signal de bruit aléatoire pour générer une fausse image selon le signal de bruit aléatoire ; et
un second discriminateur, configuré pour recevoir la fausse image pour discriminer que la fausse image est vraie, puis émettre la fausse image,
dans lequel le second générateur et le second discriminateur sont obtenus en utilisant un appareil de compression de modèle de réseau (500),
un modèle de réseau à compresser comprend un premier générateur et un premier discriminateur, caractérisé en ce que l'appareil de compression de modèle de réseau comprend un module d'élagage (510) et un module de configuration (520) ;
dans lequel le module d'élagage (510) est configuré pour réaliser un traitement d'élagage sur le premier générateur pour obtenir le second générateur ;
le module de configuration (520) est configuré pour configurer des états de noyaux de convolution dans le premier discriminateur pour permettre à une partie des noyaux de convolution d'être dans un état activé et à l'autre partie des noyaux de convolution d'être dans un état supprimé, de manière à obtenir le second discriminateur ; et
une différence de perte entre le premier générateur et le premier discriminateur est une première différence de perte, une différence de perte entre le second générateur et le second discriminateur est une seconde différence de perte, et une valeur absolue d'une valeur de différence entre la première différence de perte et la seconde différence de perte est inférieure à un premier seuil prédéfini,
dans lequel le module de configuration (520) est configuré pour configurer des états de noyaux de convolution dans le premier discriminateur pour permettre à une partie des noyaux de convolution d'être dans un état activé et à l'autre partie des noyaux de convolution d'être dans un état supprimé, de manière à obtenir un second discriminateur, comprenant :
le gel d'un facteur de rétention correspondant à chaque noyau de convolution dans le second discriminateur, et la détermination d'un premier paramètre de poids du second discriminateur, dans lequel le facteur de rétention est utilisé pour la caractérisation de l'importance d'un noyau de convolution correspondant au facteur de rétention (S250) ;
le gel du premier paramètre de poids du second discriminateur et d'un second paramètre de poids du second générateur, et la détermination de facteurs de rétention respectifs (S260) ; et
la réalisation de manière répétée d'opérations de détermination du premier paramètre de poids du second discriminateur et la détermination des facteurs de rétention respectifs jusqu'à ce que la valeur absolue de la valeur de différence entre la première différence de perte et la seconde différence de perte soit inférieure au premier seuil prédéfini,
dans lequel la détermination des facteurs de rétention respectifs comprend :
la détermination des facteurs de rétention respectifs selon une fonction objective des facteurs de rétention respectifs ;
en réponse à un facteur de rétention étant inférieur à un second seuil prédéfini, la détermination que le facteur de rétention est de 0 ; et
en réponse à un facteur de rétention étant supérieur ou égal au second seuil prédéfini, la détermination que le facteur de rétention est de 1 (S370) ;
dans lequel, avant la détermination des facteurs de rétention respectifs selon une
fonction objective des facteurs de rétention respectifs, le procédé de compression
de modèle de réseau comprend en outre :
la détermination de la fonction objective des facteurs de rétention respectifs selon
une fonction objective du second générateur, une fonction objective du second discriminateur,
une fonction de perte du second discriminateur par rapport à de fausses images, une
fonction objective du premier générateur et une fonction de perte du premier discriminateur
par rapport à de fausses images (S240) (S350).
une mémoire, stockant un programme informatique ; et
un processeur, configuré pour exécuter le programme informatique, dans lequel le programme informatique, lorsqu'il est exécuté par le processeur, amène le processeur à réaliser le procédé de génération d'image selon l'une quelconque des revendications 1 à 8.