(19)
(11)EP 3 065 090 B1

(12)EUROPEAN PATENT SPECIFICATION

(45)Mention of the grant of the patent:
04.11.2020 Bulletin 2020/45

(21)Application number: 16157040.3

(22)Date of filing:  24.02.2016
(51)International Patent Classification (IPC): 
G06N 3/04(2006.01)
G06K 9/62(2006.01)

(54)

LEARNING METHOD AND RECORDING MEDIUM BACKGROUND

LERNVERFAHREN UND AUFZEICHNUNGSMEDIENHINTERGRUND

PROCÉDÉ D'APPRENTISSAGE ET SUPPORT D'ENREGISTREMENT D'ARRIÈRE-PLAN


(84)Designated Contracting States:
AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

(30)Priority: 06.03.2015 JP 2015044838
30.10.2015 JP 2015214657

(43)Date of publication of application:
07.09.2016 Bulletin 2016/36

(73)Proprietor: Panasonic Intellectual Property Management Co., Ltd.
Osaka-shi, Osaka 540-6207 (JP)

(72)Inventors:
  • HAGAWA, Reiko
    Osaka-shi, Osaka 540-6207 (JP)
  • TSUKIZAWA, Sotaro
    Osaka-shi, Osaka 540-6207 (JP)
  • ISHII, Yasunori
    Osaka-shi, Osaka 540-6207 (JP)

(74)Representative: Grünecker Patent- und Rechtsanwälte PartG mbB 
Leopoldstraße 4
80802 München
80802 München (DE)


(56)References cited: : 
  
  • Jeff Donahue ET AL: "DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition", , 5 October 2013 (2013-10-05), XP055326544, Retrieved from the Internet: URL:http://www.jmlr.org/proceedings/papers /v32/donahue14.pdf
  • Zhicheng Yan ET AL: "HD-CNN: HIERARCHICAL DEEP CONVOLUTIONAL NEURAL NETWORK FOR IMAGE CLASSIFICATION", arXiv, 19 December 2014 (2014-12-19), XP055327803, Retrieved from the Internet: URL:https://arxiv.org/pdf/1410.0736v2.pdf [retrieved on 2016-12-09]
  • DAN C CIRESAN ET AL: "Transfer learning for Latin and Chinese characters with Deep Neural Networks", NEURAL NETWORKS (IJCNN), THE 2012 INTERNATIONAL JOINT CONFERENCE ON, IEEE, 10 June 2012 (2012-06-10), pages 1-6, XP032210239, DOI: 10.1109/IJCNN.2012.6252544 ISBN: 978-1-4673-1488-6
  
Note: Within nine months from the publication of the mention of the grant of the European patent, any person may give notice to the European Patent Office of opposition to the European patent granted. Notice of opposition shall be filed in a written reasoned statement. It shall not be deemed to have been filed until the opposition fee has been paid. (Art. 99(1) European Patent Convention).


Description

BACKGROUND


1. Technical Field



[0001] The present disclosure relates to a learning method and a recording medium, and more particularly, to a method of learning a classifier including a neural network and configured to classify images and a recording medium therefor.

2. Description of the Related Art



[0002] In recent years, there has been a need for a technique for quickly and accurately recognizing various objects by using an image recognition technique that is a technique for recognizing what kind of object is included at which location on an image. For example, Z. Yan, V. Jagadeesh, D. DeCoste, W. Di and R. Piramuthu, "HD-CNN: Hierarchical Deep Convolutional Neural Network for Image Classification" (Under review as a conference paper at ICLR 2015, URL: http://arxiv.org/pdf/1410.0736v2.pdf, Dec. 2014) (hereinafter referred to simply as NPL 1) discloses an image recognition technique capable of performing a high-accuracy image recognition using a plurality of convolutional neural network classifiers.
The paper titled "DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition discloses in section 4.3 to test the performance of DeCAF on the task of subcategory recognition.
The paper for the International Joint Conference on Neural Networks by Dan C Cirsean et.al titled "Transfer learning for Latin and Chinese characters with Deep Neural Networks" analyzes transfer learning with Deep Neural Networks (DNN) on various character recognition tasks. DNN trained on digits are perfectly capable of recognizing uppercase letters with minimal retraining.

SUMMARY



[0003] The invention provides a method of learning a classifier according to claim 1 and a non-transitory computer-readable recording medium according to claim 6. Embodiments that are not covered by the claims are to be understood as useful for understanding the invention.
In one general aspect, the techniques disclosed here feature that a method of learning a classifier for classifying an image includes performing a first process in which a coarse class classifier configured with a first neural network is made to classify a plurality of images given as a set of images each attached with a label indicating a detailed class into a plurality of coarse classes including a plurality of detailed classes and is then made to learn a first feature that is a feature common in each of the coarse classes, and performing a second process in which a detailed class classifier is made to classify the set of images into a plurality of detailed classes and is then made to learn a second feature that is a feature common in each of the detailed classes, the detailed class classifier configured with a second neural network that is the same in terms of layers other than a final layer as, but different in terms of the final layer from, the first neural network made to perform the learning in the first process.

[0004] The present disclosure makes it possible to realize a method of learning a classifier capable of accurately recognizing an image with an improved image recognition processing speed, a recording medium therefor, and the like.

[0005] It should be noted that general or specific embodiments may be implemented as a system, a method, an integrated circuit, a computer program, a computer-readable storage medium such as a CD-ROM disk or like, or any selective combination thereof.

[0006] Additional benefits and advantages of the disclosed embodiments will become apparent from the specification and drawings. The benefits and/or advantages may be individually obtained by the various embodiments and features of the specification and drawings, which need not all be provided in order to obtain one or more of such benefits and/or advantages.

BRIEF DESCRIPTION OF THE DRAWINGS



[0007] 

Fig. 1 is a block diagram illustrating an example of a configuration of a learning system according to a first embodiment;

Fig. 2A is a diagram illustrating a configuration of a coarse class classifier used in a learning process performed by the learning system according to the first embodiment;

Fig. 2B is a diagram illustrating a configuration of a detailed class classifier used in a learning process performed by the learning system according to the first embodiment;

Fig. 3A is a diagram illustrating an example of a first neural network used as a coarse class classifier in Fig. 2A;

Fig. 3B is a diagram illustrating an example of a second neural network used as a detailed class classifier in Fig. 2B;

Fig. 4 is a diagram illustrating a configuration in performing a recognition process by a classifier according to the first embodiment;

Fig. 5 is a flow chart illustrating a learning process according to the first embodiment;

Fig. 6A illustrates an example of a result of coarse class learning according to the first embodiment;

Fig. 6B illustrates an example of a result of detailed class learning according to the first embodiment;

Fig. 7 is a flow chart illustrating an example of a detailed process of step S1 in Fig. 5;

Fig. 8 is a flow chart illustrating an example of a detailed process of step S2 in Fig. 5;

Fig. 9A is a block diagram illustrating an example of a configuration of a learning system according to a first modification;

Fig. 9B is a block diagram illustrating an example of a detailed configuration of a coarse class generation unit shown in Fig. 9A;

Fig. 10A is a flow chart illustrating an example of a detailed process of coarse class learning according to the first modification;

Fig. 10B is a flow chart illustrating an example of a detailed process of step S10 in

Fig. 10A;

Fig. 11 is a diagram illustrating an example of a third neural network according to a second modification;

Fig. 12A is a diagram illustrating an example of a first neural network used as a coarse class classifier according to the second modification;

Fig. 12B is a diagram illustrating an example of a second neural network used as a detailed class classifier according to the second modification;

Fig. 13 is a block diagram illustrating an example of a configuration of a learning system according to a second embodiment;

Fig. 14 is a diagram illustrating a configuration of a detailed class classifier used in a learning process performed by the learning system according to the second embodiment;

Fig. 15 is a flow chart illustrating an example of a detailed process of detailed class learning according to the second embodiment; and

Fig. 16 is a diagram illustrating a configuration of a conventional classifier and a conventional learning method.


DETAILED DESCRIPTION


Underlying Knowledge Forming Basis of the Present Disclosure



[0008] The image recognition technique generally includes following two steps:
  1. 1) Extraction process to extract a feature value from an image; and
  2. 2) Discrimination process to discriminate an object based on the feature value. For example, in the extraction process, a feature value such as a luminance distribution, a luminance difference (edge), or the like used in the recognition is extracted from an object to be recognized in an image. On the other hand, in the discrimination process, for example, a determination is made based on the extracted feature value as to whether the object is that to be recognized by using a statistical matching learning method such as boosting, SVM (Support Vector Machine), or the like.


[0009] Conventionally, a feature value extraction filter used in the extraction process is designed manually. In recent years, image capturing environments have been established, and it has become possible to practically collect a large amount of learning data via the Internet. Furthermore, infrastructure has been established in terms of large-scale computer environments such as GPU or the like. Thus investigation has been performed on methods of automatically designing a feature value extraction filter. One of such methods is called deep learning. In particular, in the field of the image recognition, investigation has been performed on a convolutional neural network in which a feature value extraction filter is used in a convolutional filtering process on a two-dimensional image. Of various deep learning techniques, the convolutional neural network, is suitable for image recognition.

[0010] NPL 1 discloses techniques associated with a configuration of a classifier and a learning method using deep learning. Referring to Fig. 16, the classifier disclosed in NPL 1 is described below. Fig. 16 is a diagram illustrating a configuration of a conventional classifier and a conventional learning method.

[0011] The classifier shown in Fig. 16 includes a coarse class classifier 900 and detailed class classifiers 901, 902, and 903. Note that the detailed class classifier is also called a fine class classifier. The coarse class classifier 900 is a classifier that coarsely classifies objects into coarse classes, for example, a dog, a cat, and the like. The detailed class classifiers 901, 902, and 903 are classifiers that classify objects into detailed classes, for example, Shiba dog, corgi dog, tortoiseshell cat, and the like. By making the detailed class classifiers 901, 902, and 903 perform learning particular classes, it is possible to obtain capabilities of such detailed class classification.

[0012] In the classifier shown in Fig. 16, as described above, the detailed class classifiers 901, 902, and 903 that perform the detailed class classification in terms of, for example, Shiba dog, corgi dog, tortoiseshell cat, and the like are connected to an output of the coarse class classifier 900 that performs the coarse class classification in terms of, for example, dog, cat, and the like. In the classifier shown in Fig. 16, for example, when an image of a corgi dog is given as an input image, the coarse class classifier 900 recognizes that the image is of a dog. After that, a detailed class classifier having high ability in terms of classification of a dog class is selected. More specifically, for example, the detailed class classifier 901 is selected, and the detailed class classifier 901 determines that the image is of a corgi dog.

[0013] As described above, in the technique disclosed in NPL 1, detailed class classifiers are connected to a coarse class classifier. This makes it possible to simplify problems to be solved by the coarse class classifier and the detailed class classifiers, respectively. By combining these classifiers, it becomes possible to accurately recognize images and accurately classify images.

[0014] However, in the classifier disclosed in the NPL 1, a plurality of detailed class classifiers are connected to an output of a coarse class classifier, which results in an increase in a size of a neural network (that is, an increase in the number of neurons and the number of couplings). As a result, to perform image recognition, a greater amount of calculation is needed, which results in a reduction in the processing speed of the image recognition. Furthermore, in the classifier disclosed in NPL 1, when an error occurs in the classification by the coarse class classifier, the following detailed classification process is performed by a detailed class classifier optimized for a wrong coarse class, which results in a reduction in correct answer rate in terms of the classification.

[0015] In an aspect of the present disclosure, a method of learning a classifier for classifying an image, includes performing a first process in which a coarse class classifier configured with a first neural network is made to classify a plurality of images given as a set of images each attached with a label indicating a detailed class into a plurality of coarse classes including a plurality of detailed classes and is then made to learn a first feature that is a feature common in each of the coarse classes, and performing a second process in which a detailed class classifier is made to classify the set of images into a plurality of detailed classes and is then made to learn a second feature that is a feature common in each of the detailed classes, the detailed class classifier configured with a second neural network that is the same in terms of layers other than a final layer as, but different in terms of the final layer from, the first neural network made to perform the learning in the first process.

[0016] Thus the present disclosure makes it possible to maintain the size of the neural network structure. Therefore, the present disclosure makes it possible to perform a high-accuracy image recognition without reducing a processing speed of the image recognition (classification) process compared with a processing speed of the conventional technique.

[0017] For example, the first process may include a first design process and a first learning process, the first design process including designing the first neural network so as to include as many neurons in the final layer as there are coarse classes generated based on contents of labels indicating the respective detailed classes, the first learning process including making the coarse class classifier configured with the designed first neural network acquire the set of images and learn the first feature, and the second process may include a second design process and a second learning process, the second design process including designing the second neural network such that the neurons in the final layer of the first neural network made to perform the learning in the first process are changed by neurons whose number is the same as the number of detailed classes and the resultant neural network is employed as the second neural network, the second learning process including making the detailed class classifier configured with the designed second neural network acquire the set of images and learn the second feature thereby learning the classifier.

[0018] For example, in the second design process, the second neural network may be designed such that the neurons in the final layer of the first neural network made to perform the learning in the first learning process are replaced with neurons whose number is the same as the number of detailed classes and the resultant neural network is employed as the second neural network.

[0019] For example, the detailed class classifier and the coarse class classifier each may be configured using a third neural network, and the method may further include a third design process before the first process, the third design process including designing the third neural network such that a final layer of the third neural network includes neurons whose number is equal to or greater than the sum of the number of coarse classes and the number of detailed classes, wherein in the first design process, the first neural network may be given by the third neural network but in the final layer of the third neural network, particular neurons whose number is equal to the number of coarse classes are selected and only the selected neurons are used thereby designing the first neural network so as to include as many neurons in the final layer as there are coarse classes generated based on contents of labels indicating the respective detailed classes, and in the second design process, the second neural network may be given by the third neural network but in the final layer of the third neural network, particular neurons which are different from the neurons employed as the neurons in the final layer of the first neural network and whose number is equal to the number of detailed classes are used thereby designing the second neural network obtained by changing the neurons in the final layer of the first neural network made to perform the learning in the first process by neurons whose number is equal to the number of detailed classes.

[0020] For example, the second design process may further include setting a weight of the final layer of the second neural network to a random number, the weight indicating a coupling efficiency between the final layer of the second neural network and a layer immediately in front of the final layer.

[0021] For example, the second design process may further include setting a second weight of the final layer of the second neural network based on a first weight of the final layer of the first neural network, the first weight indicating a coupling efficiency, obtained via the learning in the first learning process, between the final layer of the first neural network and the layer immediately in front of the final layer, the second weight indicating a coupling efficiency between the final layer of the second neural network and the layer immediately in front of the final layer, the second weight determined by proportionally dividing the first weight according to a relationship between the plurality of coarse classes and the plurality of detailed classes included in the plurality of coarse classes.

[0022] For example, the method may further include storing, as the classifier, the detailed class classifier made to learn the second feature in the second process.

[0023] For example, the first process may further include generating the plurality of coarse classes based on contents of labels attached to respective images in the plurality of images.

[0024] For example, the first process may further include generating the plurality of coarse classes based on similarity of each image in the plurality of images.

[0025] For example, the method may further include performing a third process in which a third coarse class classifier configured with a third neural network is made to classify the set of images into a plurality of third coarse classes including a plurality of detailed classes and is then made to learn a third feature that is a feature common in each third coarse class in the plurality of third coarse class, wherein the first process includes a first learning process in which the detailed class classifier, configured with the first neural network that is the same in terms of layers other than the final layer as but different in terms of the final layer from the third neural network made to perform the learning in the third process, is made to classify the set of images into the plurality of coarse classes and learn the first feature of each detailed class, and wherein each coarse class in the plurality of coarse classes includes a plurality of third coarse classes and thus includes a plurality of detailed classes included in the plurality of third coarse classes.

[0026] For example, at least one of the first process and the second process may be performed by a processor.

[0027] Note that embodiments described below are given by way of example only and not limitation on the scope of the present disclosure. Values, shapes, constituent elements, steps, an order of steps, etc., described in the following embodiments are given by way of example only, and not limitation on the scope of the present disclosure. Among constituent elements in the embodiments described below, constituent elements that are not included in independent claims representing highest concepts are arbitrary constituent elements. Also note that a combination of arbitrary embodiments is possible.

First Embodiment



[0028] A classifier 10 according to a first embodiment is described below as to a learning method and the like with reference to drawings.

Configuration of Learning System 1



[0029] Fig. 1 is a block diagram illustrating an example of a configuration of a learning system 1 according to a first embodiment. Fig. 2A is a diagram illustrating a configuration of a coarse class classifier 10a used in a learning process performed by the learning system 1 according to the first embodiment. Fig. 2B is a diagram illustrating a configuration of a detailed class classifier 10b used in a learning process performed by the learning system 1 according to the first embodiment. Note that the detailed class classifier is also called a fine class classifier. Fig. 3A is a diagram illustrating an example of a first neural network used as a coarse class classifier 10a shown in Fig. 2A. Fig. 3B is a diagram illustrating an example of a second neural network used as a detailed class classifier 10b shown in Fig. 2B.

[0030] A learning system 1 shown in Fig. 1 includes an information acquisition unit 2, a classifier design unit 3, a classifier learning unit 4, and a storage unit 5, and is configured to learn the classifier 10 that classifies images.

[0031] The information acquisition unit 2 acquires a data set of a plurality of images (a set of images) prepared in advance, that is, a set of images each attached with a label indicating a detailed class. Furthermore, the information acquisition unit 2 acquires a coarse class classifier 10a stored in the storage unit 5, that is, a first neural network (parameters indicating a structure, a threshold value, a weight, a coupling state, and the like) used to realize the coarse class classifier 10a stored in the storage unit 5.

[0032] The classifier learning unit 4 learns the classifier 10 by performing a learning process (coarse class learning) on the coarse class classifier 10a shown in Fig. 2A and a learning process (detailed class learning) on the detailed class classifier 10b shown in Fig. 2B using a coarse-to-fine approach in the image recognition technique.

[0033] More specifically, the classifier learning unit 4 controls the coarse class classifier 10a including a first neural network designed by the classifier design unit 3 to acquire a set of images respective attached with detailed classes. The classifier learning unit 4 then controls the coarse class classifier 10a to perform coarse class learning so as to classify the set of images into a plurality of coarse classes including a plurality of detailed classes and learn a first feature that is a feature common in each of the coarse classes. The classifier learning unit 4 may store, in the storage unit 5, the coarse class classifier 10a made to perform the coarse class learning, that is, the first neural network (parameters indicating the structure, the threshold value, the weight, the coupling state, and the like) used to realize the coarse class classifier 10a.

[0034] The classifier learning unit 4 also controls the detailed class classifier 10b to acquire the set of images described above, wherein the detailed class classifier 10b is configured with a second neural network designed by the classifier design unit 3 so as to be the same in terms of layer other than the final layer as but different in terms of the final layer from the first neural network made to perform the coarse class learning. The classifier learning unit 4 then controls the detailed class classifier 10b to perform detailed class learning so as to classify the set of images into detailed classes and learn a second feature that is a feature common in each detailed class. Thereafter, the classifier learning unit 4 stores the detailed class classifier 10b made to perform the detailed class learning as the classifier 10 in the storage unit 5, that is, the classifier learning unit 4 stores the second neural network (parameters indicating the structure, the threshold value, the weight, the coupling state, and the like) used to realize the detailed class classifier 10b as the classifier 10 in the storage unit 5.

[0035] As described above, the classifier learning unit 4 employs a multi-stage (two stage, in the present example) approach in learning the classifier 10.

[0036] The classifier design unit 3 determines the number of outputs (the number of neurons) such that the number of outputs is equal to the number of detailed classes or the number of generated coarse classes, and designs the configuration and the parameters for the coarse class classifier 10a or the detailed class classifier 10b.

[0037] More specifically, the classifier design unit 3 designs the coarse class classifier 10a such that the final layer of the first neural network of the coarse class classifier 10a has as many neurons as there are coarse classes generated based on contents of labels indicating the detailed classes. In the present embodiment, the classifier design unit 3 designs the first neural network including an input layer, an intermediate layer, and a final layer, for example, as illustrated in Fig. 3A, and employs the resultant first neural network as the coarse class classifier 10a shown in Fig. 2A. Note that the number of neurons of the final layer of the first neural network is set to be equal to the number of coarse classes.

[0038] Furthermore, the classifier design unit 3 designs the second neural network such that the neurons of the final layer of the first neural network of the coarse class classifier 10a made to perform the coarse class learning are replaced by neurons whose number is equal to the number of detailed classes, and the resultant neural network is employed as the second neural network. For example, the classifier design unit 3 designs the second neural network such that the neurons of the final layer of the first neural network made to perform the coarse class learning are replaced by neurons whose number is equal to the number of detailed classes, and the resultant neural network is employed as the second neural network. In this design process, the classifier design unit 3 may employ a random value as a weight indicating a coupling efficiency between the final layer of the second neural network and a layer immediately before the final layer.

[0039] In the present embodiment, the classifier design unit 3 designs the second neural network so as to include an input layer, an intermediate layer, and a final layer, for example, as illustrated in Fig. 3B, and employs the resultant second neural network as the detailed class classifier 10b shown in Fig. 2B. Note that the second neural network is designed by replacing the final layer thereof such that the second neural network and the first neural network are the same in terms of layers other than their final layers but different only in terms of their final layers and such that the final layer of the second neural network has as many neurons as there are detailed classes. The weight indicating the coupling efficiency between the final layer of the second neural network and the layer immediately before the final layer is learned (additionally learned) when the detailed class learning is performed, and thus the weight may be set to an arbitrary value, for example, a random number.

[0040] The storage unit 5 stores the classifier 10 (the coarse class classifier 10a) made to perform the coarse class learning. That is, the storage unit 5 stores the first neural network (the parameters indicating the structure, the threshold value, the weight, the coupling state, and the like) of the coarse class classifier 10a. Furthermore, the storage unit 5 stores the classifier 10 (the detailed class classifier 10b) made to perform the detailed class learning. That is, the storage unit 5 stores the second neural network (the parameters indicating the structure, the threshold value, the weight, the coupling state, and the like) of the detailed class classifier 10b.

[0041] Fig. 4 is a diagram illustrating a configuration in performing a recognition process by the classifier 10 according to the first embodiment. The classifier 10 is a classifier for classifying images, and more particularly, when an object to be classified (an input image) is input, the classifier 10 performs a recognition process and a classification process, and outputs a result thereof (a classification result). The classifier 10 is configured as illustrated in Fig. 4, that is, the classifier 10 is the same in configuration as the detailed class classifier 10b made to perform the detailed class learning.

[0042] As described above, the classifier 10 is configured using a neural network with a small size (in terms of the number of neurons and the number of couplings), and the learning system 1 performs multi-stage learning based on a coarse-to-fine approach in the image recognition, and thus it is possible to achieve high accuracy and high speed in the image recognition (classification).

Learning Process by Learning System 1



[0043] A learning process performed by the learning system 1 configured in the above-described manner is described below with reference to drawings.

[0044] Fig. 5 is a flow chart illustrating a learning process according to the first embodiment. In the example shown in Fig. 5, it is assumed by way of example that a 2-stage learning process is performed as the multi-stage learning process. Fig. 6A illustrates an example of a result of a coarse class learning according to the first embodiment. Fig. 6B illustrates an example of a result of a detailed class learning according to the first embodiment.

[0045] First, in step S1, the learning system 1 performs a learning process (coarse class learning) in terms of the coarse class classifier 10a.

[0046] More specifically, the learning system 1 controls the coarse class classifier 10a configured with the first neural network to classify a set of images attached with labels respectively indicating detailed classes into a plurality of coarse classes including a plurality of detailed classes, and learn a first feature that is a feature common in each of the coarse classes. More specifically, for example, as illustrated in Fig. 6A, the learning system 1 controls the coarse class classifier 10a to classify a set of images given as an image data set into coarse classes such as dog, bird, machine, etc., (that is, to determine boundaries between them), and further extract a feature (a first feature) common among images attached with labels indicating detailed classes belonging to the same coarse class. The learning system 1 then controls the coarse class classifier 10a to perform learning (coarse class learning) so as to recognize the extracted first feature.

[0047] Next, in step S2, the learning system 1 performs a learning process (a detailed class learning process) in terms of the detailed class classifier 10b.

[0048] More specifically, learning of the classifier 10 is performed by controlling the detailed class classifier 10b configured with the second neural network that is the same in terms of layers other than the final layer but different in terms of the final layer from the first neural network made to perform the learning in step S1 such that the detailed class classifier 10b classifies the set of images into detailed classes and learn a second feature that is a feature common in each of the detailed classes. For example, as illustrated in Fig. 6B, the learning system 1 controls the detailed class classifier 10b to set initial values to the set of images given as the image data set in terms of coarse classes such as dog, bird, machine and the like classified in step S1 (among which borderlines were drawn). The learning system 1 further controls the detailed class classifier 10b to divide the coarse classes into detailed classes (among which borderlines are drawn) such as corgi, Rottweiler, golden retriever, radio, printer, computer, white stork, albatross, ruddy turnstone, and the like and extract a feature (a second feature) common in each of the resultant detailed classes. The learning system 1 then controls the detailed class classifier 10b to perform learning (detailed class learning) so as to recognize the extracted second feature.

[0049] Thus, compared with the conventional classifier, it is possible to suppress an occurrence of a local solution or overlearning while maintaining a small size of the neural network structure, and thus it is possible to achieve high accuracy in the image recognition (classification) without reducing a processing speed in the image recognition.

[0050] The processes of step S1 (coarse class learning) and step S2 (detailed class learning) are described in further detail below.

[0051] Fig. 7 is a flow chart illustrating an example of the detailed process in step S1 in Fig. 5.

[0052] In step S1, first, the learning system 1 designs the first neural network in the coarse class classifier 10a such that the final layer thereof includes as many neurons as there are coarse classes generated based on contents of labels indicating the detailed classes (S11). That is, the learning system 1 designs the first neural network of the coarse class classifier 10a.

[0053] The learning system 1 then acquires a set of images each attached with a label indicating a detailed class (S12).

[0054] Next, the learning system 1 controls the coarse class classifier 10a configured with the designed first neural network to acquire the set of images and learn the first feature (S13). More specifically, the coarse class classifier 10a performs learning (coarse class learning) so as to recognize the first feature thereby learning parameters indicating a threshold value, a weight, a coupling state, and the like for the first neural network to recognize the first feature.

[0055] Next, the learning system 1 stores, in the storage unit 5, the coarse class classifier 10a made to learn the first feature in step S13, that is, the learning system 1 stores the first neural network used to realize the coarse class classifier 10a (S14).

[0056] Fig. 8 is a flow chart illustrating an example of the detailed process in step S2 in Fig. 5.

[0057] In step S2, first, the learning system 1 designs the second neural network used to realize the detailed class classifier 10b such that the neurons of the final layer of the first neural network of the coarse class classifier 10a used to realize the coarse class classifier 10a made to perform the learning in step S1 are replaced by neurons whose number is equal to the number of detailed classes and the resultant neural network is employed as the second neural network (S21).

[0058] That is, the learning system 1 designs the second neural network by directly employing the first neural network (the parameters indicating the structure, the threshold value, the weight, the coupling state, and the like) of the coarse class classifier 10a made to perform the learning in step S1 except for the final layer, and employing the resultant second neural network as the detailed class classifier 10b. The learning system 1 then changes the neurons in the final layer of the second neural network such that there are as many neurons as there are detailed classes. In the present embodiment, the learning system 1 designs the second neural network such that the neurons of the final layer of the first neural network are replaced by neurons whose number is equal to the number of detailed classes and the resultant neural network is employed as the second neural network.

[0059] Next, the learning system 1 acquires, from the outside, a set of images each attached with a label indicating a detailed class (S22). More specifically, the learning system 1 acquires the same set of images as the set of images acquired in step S12.

[0060] Next, the learning system 1 controls the detailed class classifier 10b configured with the designed second neural network to acquire the above-described set of images and learn the second feature (S23). More specifically, the detailed class classifier 10b performs learning (detailed class learning) so as to recognize the second feature thereby learning parameters indicating a threshold value, a weight, a coupling state, and the like for second neural network to recognize the second feature.

[0061] Next, the learning system 1 stores the detailed class classifier 10b made to learn the second feature in step S23 as the classifier 10 in the storage unit 5. That is, the learning system 1 stores the second neural network (the parameters indicating the structure, the threshold value, the weight, the coupling state, and the like) used to realize the detailed class classifier 10b as the classifier 10 in the storage unit 5 (S14).

Effects



[0062] As described above, the first embodiment makes it possible to realize the method of learning the classifier capable of performing high-accuracy image recognition without reducing a processing speed in the image recognition compared with a processing speed of the conventional technique. More specifically, in the learning method according to the first embodiment, the detailed class classifier 10b is made to learn detailed classes such that an initial value of the learning is given by a result of the coarse class learning performed by the coarse class classifier 10a configured with the neural network that is the same as the detailed class classifier 10b except for the final layer (that is, the multi-stage learning is performed). That is, in the classifier 10 (including the coarse class classifier 10a and the detailed class classifier 10b) according to the present embodiment, the same neural network is used in both the coarse class learning and the detailed class learning while only changing the number of outputs (the number of neurons in the final layer) between the coarse class learning and the detailed class learning. The detailed class learning is performed using the detailed class classifier 10b configured with the neural network for which the same parameters as those obtained in the coarse class learning are set to layers other than the final layer. As described above, the multi-stage learning is performed without changing the structure of the layers, other than the final layer, of the neural network of the classifier 10 (that is, while maintaining the size of the layers other than the final layer). This makes it possible to suppress erroneous employment of a wrong detailed class in the classification process, which results in an improvement in image recognition accuracy. Besides, it becomes possible to reduce the size of the neural network, which allows it to increase the processing speed of the image recognition by a factor of about 4 compared with the processing speed of the conventional technique.

[0063] In the first embodiment, by way of example, the classifier 10 is configured to perform the 2-stage learning. However, the number of states in the multi-stage learning may be three or greater.

[0064] For example, in a case where 3-stage learning is employed, the learning system 1 controls a third coarse class classifier configured with a third neural network to classify the set of images into a plurality of third coarse classes including a plurality of detailed classes and learn a third feature that is a feature common in each of the third coarse classes. Furthermore, the learning system 1 controls the detailed class classifier 10b configured with the first neural network which is the same in terms of layers other than the final layer as but different in terms of the final layer from the third neural network made to learn the third feature such that the set of images are classified into the coarse classes and a first feature of each detailed class is learned. Note that when each coarse class in the plurality of coarse classes includes a plurality of third coarse classes and thus includes a plurality of detailed classes included in the plurality of third coarse classes.

[0065] By performing the coarse class classification via two or more stages in the above-described manner, the classifier 10 is allowed to perform multi-stage learning via three or more stages. This method is useful in particular when the image set includes so many images attached with labels indicating detailed classes that use of one-stage coarse class classification would results in many outputs (neurons in the final layer).

First Modification



[0066] In the first embodiment described above, it is assumed by way of example but not limitation that the learning system 1 designs the coarse class classifier 10a using coarse classes generated based on contents of labels indicating detailed classes. Alternatively, the learning system 1 may generate coarse classes based on contents of labels indicating detailed classes. In this case, for example, the learning system 1 may perform a learning process according to a first modification as described below.

Configuration of Learning System 1A



[0067] Fig. 9A is a block diagram illustrating an example of a configuration of a learning system 1A according to the first modification. Fig. 9B is a block diagram illustrating an example of a detailed configuration of a coarse class generation unit 6 shown in Fig. 9A. Elements similar to those in Fig. 1 are denoted by similar reference numerals, and a further description thereof is omitted.

[0068] The learning system 1A shown in Fig. 9A is different in configuration from the learning system 1 shown in Fig. 1 in that the learning system 1A additionally includes a coarse class generation unit 6.

[0069] The coarse class generation unit 6 generates a plurality of coarse classes based on contents of labels indicating detailed classes assigned to respective images in the image set described above. Alternatively, the coarse class generation unit 6 may generate a plurality of coarse classes based on similarity of each image in the image set described above. Configuration of Coarse Class Generation Unit 6

[0070] In the present modification, the coarse class generation unit 6 includes, for example, a coarse class generation processing unit 61 and a coarse class label application unit 62, as shown in Fig. 9B.

[0071] The coarse class generation processing unit 61 may perform, for example, unsupervised clustering to classify a plurality of images given as the set of images into a plurality of coarse classes thereby generating the plurality of the coarse classes. For example, the coarse class generation processing unit 61 classifies a plurality of images given as the image set into a plurality of coarse classes based on contents of labels indicating detailed classes attached to the respective images in the set images or based on similarity of each image in the plurality of images given as the set of images. Furthermore, the coarse class generation processing unit 61 generates names or the like uniquely identifying the respective obtained coarse classes. That is, the coarse class generation processing unit 61 is capable of generating the plurality of coarse classes in the above-described manner.

[0072] The coarse class label application unit 62 attaches labels indicating coarse classes classified by the coarse class generation processing unit 61 to the respective images given as the set of images attached with labels indicating detailed classes.

Learning Process by the Learning System 1A



[0073] Next, referring to drawings, a description is given below as to a learning process performed by the learning system 1A configured in the above-described manner. The two-stage learning process performed by the learning system 1A is similar to that described above with reference to Fig. 5, and the detailed process of the detailed class learning (step S2) is similar to that described above with reference to Fig. 8, and thus further descriptions thereof are omitted. Thus the following description is focused on differences from the first embodiment.

[0074] Fig. 10A is a flow chart illustrating an example of a detailed process of coarse class learning (step S1) according to the first modification. Fig. 10B is a flow chart illustrating an example of a detailed process of step S10 in Fig. 10A. Elements similar to those shown in Fig. 7 are denoted by similar symbols, and a further description thereof is omitted.

[0075] The detailed process of the coarse class learning shown in Fig. 10A is different from the detailed process of the coarse class learning shown in Fig. 7 in that an additional process in step S10 is included. In step S10, the learning system 1A makes the coarse class generation unit 6 generate a plurality of coarse classes based on contents of labels indicating detailed classes attached to the respective images in the plurality of images given as the set of images set described above. Alternatively, the learning system 1A may make the coarse class generation unit 6 generate a plurality of coarse class based on similarity of each image in the plurality of images given as the image set described above.

[0076] More specifically, as illustrated in Fig. 10B, in step 10, first, learning system 1A performs a coarse class generation process (S101). In the present modification, the learning system 1A makes the coarse class generation processing unit 61 acquire, from the outside, a set of images attached with labels indicating detailed classes. The learning system 1A then makes the coarse class generation processing unit 61 classify a plurality of images in the acquired set of images into a plurality of coarse classes thereby generating the plurality of coarse classes.

[0077] Next, the learning system 1A performs a coarse class label application process (S102). In the learning system 1A according to the present modification, based on the result of the classification performed in step S101, the coarse class label application unit 62 attaches labels indicating the coarse classes classified in step S101 to the respective images in the plurality of images given as the set of images each attached with a label indicating a detailed class.

Second Modification



[0078] In the first embodiment described above, the final layer of the second neural network is designed, by way of example but not limitation, by exchanging (replacing) the final layer of the first neural network. Alternatively, the third neural network may be used as the first neural network and the second neural network such that neurons are selected from the neurons of the final layer of the third neural network and only the selected neurons are used in the final layer. In this case, neurons selected and used in the final layer are changed between the first neural network and the second neural network. This modification (that is, the second modification) is described in further detail below.

Configuration



[0079] Fig. 11 is a diagram illustrating an example of a third neural network according to the second modification. Fig. 12A is a diagram illustrating an example of a first neural network 10c used to realize a coarse class classifier according to the second modification. Fig. 12B is a diagram illustrating an example of a second neural network 10d used to realize a detailed class classifier according to the second modification. Elements similar to those in Fig. 2A or Fig. 2B are denoted by similar reference symbols, and a further detailed description thereof is omitted.

[0080] In the multi-stage learning on the classifier 10 performed by the learning system 1, coarse class learning is performed using the coarse class classifier 10c, and detailed class learning is performed using the detailed class classifier 10d.

[0081] In the present modification, the coarse class classifier 10c and the detailed class classifier 10d are realized using the third neural network such as that illustrated in Fig. 11. The final layer of the third neural network includes neurons whose number is equal to or greater than the sum of the number of coarse classes and the number of detailed classes.

[0082] As illustrated in Fig. 12A, the coarse class classifier 10c is realized using the first neural network that is realized fully using the layers other than the final layer of the third neural network shown in Fig. 11 and using part of the final layer. More specifically, the first neural network used to realize the coarse class classifier 10c is given by the third neural network but in the final layer thereof, particular neurons whose number is equal to the number of coarse classes are selected from the neurons of the final layer and only the selected neurons are used.

[0083] The second neural network used to realize the detailed class classifier 10d is given, as illustrated in Fig. 12B, by the third neural network shown in Fig. 11 but the layers other than the final layer and part of the final layer remaining without being used in the final layer of first neural network are used. More specifically, the second neural network used to realize the detailed class classifier 10d is given by the third neural network but in the final layer thereof, particular neurons which are different from the neurons employed as the neurons in the final layer of the first neural network and whose number is equal to the number of detailed classes are used.

Learning Process by Learning System 1



[0084] In the present modification, the learning system 1 performs a learning process such as that described above with reference to Fig. 5 using the coarse class classifier 10c and the detailed class classifier 10d configured in the above-described manner.

[0085] More specifically, the learning process according to the present modification includes, before the learning process on the coarse class classifier 10c in step S1, an additional step of designing the third neural network so as to include, in its final layer, as many neurons as the sum of the number of coarse classes and the number of detailed class, or more, as illustrated, for example, in Fig. 11.

[0086] For the purpose described above, in the present modification, the learning system 1 designs the first neural network in step S1 in which the first neural network used to realize the coarse class classifier 10c is given by the third neural network but in the final layer thereof, particular neurons whose number is equal to the number of coarse classes are selected and only the selected neurons are used thereby achieving the first neural network whose final layer includes as many neurons as there are coarse classes generated based on contents of labels indicating detailed classes.

[0087] Thereafter, the learning system 1 of the present modification designs the second neural network in step S2 in which the second neural network used to realize the detailed class classifier 10d is given by the third neural network but in the final layer thereof, particular neurons which are different from the neurons employed as the neurons in the final layer of the first neural network and whose number is equal to the number of detailed classes are used thereby achieving the second neural network that is the same as the first neural network made to perform the learning in step S1 except that the neurons in the final layer is changed by the neuron whose number is equal to the number of detailed classes.

Second Embodiment



[0088] In the first embodiment described above, the weight for the final layer of the second neural network is set, by way of example but not limitation, to a random number. Alternatively, the weight for the final layer of the second neural network may be set based on the weight for the final layer of the first neural network made to learn the coarse classes and the relationship between the detailed classes and the coarse classes. This alternative setting is described below focusing on differences from the first embodiment.

Configuration of Learning System 1B



[0089] Fig. 13 is a block diagram illustrating an example of a configuration of a learning system 1B according to a second embodiment. Fig. 14 is a diagram illustrating a configuration of a detailed class classifier 10e used in a learning process performed by the learning system 1B according to the second embodiment. In Fig. 14, a, b, k, a1 to a1, b1 to bm, and k1 to kn are positive integers. In Fig. 14, elements similar to those in Fig. 1 are denoted by similar reference symbols, and a further description thereof is omitted.

[0090] The learning system 1B shown in Fig. 13 includes an information acquisition unit 2, a classifier design unit 3, a classifier learning unit 4, a storage unit 5, and a weight setting unit 7. The learning system 1B shown in Fig. 13 is different from the learning system 1 shown in Fig. 1 in that the weight setting unit 7 is additionally provided.

[0091] Based on the weight for the final layer of the first neural network acquired from the coarse class classifier 10a made to learn the coarse classes and based on the relationship such as an inclusion relationship between the detailed classes and the coarse classes, the weight setting unit 7 calculates the weight for the new final layer, that is, the weight for the final layer of the second neural network. More specifically, the weight setting unit 7 determines the weight as follows. The first weight for the final layer of the first neural network indicating the coupling efficiency between the final layer of the first neural network of the coarse class classifier 10a obtained via the process of learning the coarse classes and the layer immediately in front of the final layer is divided proportionally in accordance with the relationship between the plurality of coarse classes and the plurality of detailed classes included in the plurality of coarse classes. Thereafter, using the value calculated in the above-described manner, the weight setting unit 7 sets the second weight for the final layer of the second neural network indicating the coupling efficiency between the final layer of the second neural network and the layer immediately in front of the final layer.

[0092] For example, Fig. 14 shows Wa, Wb,..., Wk as first weights, and Wa1 to Wal, Wb1 to Wbm,..., Wk1 to Wkn as second weights. Furthermore, Fig. 14 also shows a plurality of coarse classes (coarse class a, coarse class b,..., coarse class k) output from neurons of the final layer of the first neural network, and a plurality of detailed classes (detailed classes a1 to al, detailed classes b1 to bm,..., detailed classes k1 to kn) output from neurons of the final layer of the second neural network.

[0093] That is, the weight setting unit 7 obtains the first weights for the final layer of the first neural network via the learning on the first neural network in terms of the coarse classes, that is, the weight setting unit 7 obtains first weights Wa, Wb,..., Wk for the neurons in the final layer that respectively output coarse class a, coarse class b,..., coarse class k. Thereafter, in the detailed class classifier 10e, the plurality of coarse classes (coarse class a, coarse class b,..., coarse class k) are replaced with the plurality of detailed classes (detailed classes a1 to al, detailed classes b1 to bm,..., detailed classes k1 to kn) included in the plurality of coarse classes (coarse class a, coarse class b,..., coarse class k). In this process, the weight setting unit 7 calculates values by proportionally dividing the acquired first weights Wa, Wb,..., Wk by the number (I, m,..., n) of detailed classes included in the corresponding coarse classes based on the relationship among the plurality of detailed classes (detailed classes a1 to al, detailed classes b1 to bm,..., detailed classes k1 to kn) included in the plurality of coarse classes (coarse class a, coarse class b,... coarse class k).

Learning Process by Learning System 1B



[0094] A learning process performed by the learning system 1B configured in the above-described manner is described below. The two-stage learning process performed by the learning system 1B is similar to that described above with reference to Fig. 5, and the detailed process of the coarse class learning (step S1) is similar to that described above with reference to Fig. 7, and thus further descriptions thereof are omitted. The following description is focused on differences from the first embodiment. Fig. 15 is a flow chart illustrating an example of a detailed class learning (step S2) according to the second embodiment. In Fig. 15, step S32, steps S34 to S36 are similar to steps S21 to S24 described above with reference to Fig. 8, and thus a further description thereof is omitted.

[0095] First, in step S31, the learning system 1B controls the weight setting unit 7 to acquire the first weights for the final layer of the first neural network used to realize the coarse class classifier 10a, wherein the first weights are obtained when the coarse class classifier 10a is made to learn the coarse classes in step S10.

[0096] In step S33, the learning system 1B sets the weight setting unit 7 in terms of the second weights for the final layer of the second neural network used to realize the detailed class classifier 10b by using the values obtained by proportionally dividing the acquired first weights according to the relationship between the detailed classes and the coarse classes. Here the relationship between the detailed classes and the coarse classes is the relationship between the plurality of coarse classes and the plurality of detailed classes included in the plurality of coarse classes. In the specific example described above, the relationship is the number of detailed classes included in each coarse class. Effects

[0097] As described above, the present embodiment makes it possible to realize the method of learning the classifier capable of performing high-accuracy image recognition without reducing a processing speed in the image recognition compared with a processing speed of the conventional technique.

[0098] The learning method according to the second embodiment, as described above, includes an additional process, in addition to the process of the learning method according to the first embodiment, to set the weights for the final layer of the detailed class classifier 10b used in the detailed class classification process, based on the weights for the final layer of the coarse class classifier 10a obtained via the coarse class classification process in the learning process. This makes it possible to improve the accuracy of the image recognition by the classifier 10 and the processing speed of the image recognition.

[0099] The learning method according to the present disclosure has been described above with reference to the first embodiment, the first modification, the second modification, and the second embodiment. Note that there is no particular restriction on an entity or an apparatus that performs each process. Processes may be performed by a processor or the like disposed in a particular local apparatus or by a cloud server or the like disposed at a location different from the location of the local apparatus, as described below.

[0100] In addition to the above, the present disclosure may be realized in many other ways, some examples of which are described below.
  1. (1) A specific example of the apparatus described above is a computer system including a microprocessor, a ROM, a RAM, a hard disk unit, a display unit, a keyboard, a mouse, and the like. A computer program is stored in the RAM or the hard disk unit. The microprocessor operates according to the computer program such that each apparatus achieves its function. The computer program includes a plurality of instruction codes each indicating an instruction to be executed by the computer to achieve a particular function.
  2. (2) Part or all of constituent elements of the apparatus described above may be realized with one system LSI (Large Scale Integration). The system LSI is a super-multifunction LSI produced by integrating a plurality of parts on a single chip. More specifically, the system LSI may be a computer system configured using a microprocessor, a ROM, a RAM, and the like. In the RAM, a computer program is stored. The microprocessor operates according to the computer program such that the system LSI achieves its functionalities.
  3. (3) Part or all of the constituent elements of the apparatus described above may be implemented in the form of an IC card attachable to the apparatus or in the form of a single module. The IC card or the module is a computer system including a microprocessor, a ROM, a RAM, and so on. The IC card or the module may include the super-multifunction LSI described above. In the IC card or the module, the microprocessor operates according to the computer program thereby achieving the function of the IC card or the module. The IC card or the module may be configured so as to be resistant against tampering.
  4. (4) The present disclosure may be implemented as a method. The method may be realized by a computer program that is to be executed by a computer or the method may be realized by a digital signal associated with the computer program.
  5. (5) The present disclosure may be implemented by a computer readable storage medium, such as a flexible disk, a hard disk, a CD-ROM disk, an MO disk, a DVD disk, a DVD-ROM disk, a DVD-RAM disk, a BD (Blu-ray (registered trademark) Disc), a semiconductor memory, or the like in which the computer program or the digital signal is stored. Alternatively, the present disclosure may be implemented by the digital signal stored in the storage medium described above.
    The present disclosure may be implemented by transmitting the computer program or the digital signal via a telecommunication line, a wired or wireless communication line, a network typified by the Internet, data broadcasting, or the like.
    The present disclosure may be implemented by a computer system including a microprocessor and a memory, wherein the computer program is stored in the memory and the microprocessor operates according to the computer program.
    The program or the digital signal may be stored in the storage medium and the storage medium may be transported, or the program or the digital signal may be transferred via the network or the like thereby allowing the present invention to be implemented in another computer system.
  6. (6) The details of the above-described embodiments and the modifications may be combined.


[0101] The present disclosure may be applied to a method of learning a classifier that classifies an image and a recording medium therefor, and more particularly, to an image capturing apparatus such as a digital camera, a movie camera, a monitor camera, an in-vehicle camera, a wearable camera, and the like in which an object recognition method or system is implemented to analyze an image to accurately recognize what kind of object is included in the image.


Claims

1. A method of learning a classifier (10), the classifier (10) for classifying an image into a detailed class, comprising the steps:

learning the classifier (10), by:

providing a set of images, which is given as a set of images each attached with a label indicating a detailed class;

performing a first process (S1) which comprises the steps of:

designing, in a first design process, a coarse class classifier (10a), wherein the coarse class classifier (10a) is configured with a first neural network, wherein the first neural network is composed of a first final layer and a first set of layers other than the first final layer; wherein the first final layer of the first neural network of the coarse class classifier has as many neurons as there are coarse classes;

clustering, by a coarse class generation unit (61), the set of images into a plurality of coarse classes, based on similarity of each image in the set of images;

learning, by the coarse class classifier with the first neural network, to classify the set of images into the plurality of coarse classes;

performing a second process (S2) which comprises the steps of:

designing, in a second design process, a detailed class classifier (10b), wherein the detailed class classifier is configured with a second neural network, wherein the second neural network is composed of a second final layer and the first set of layers;

wherein the second design process includes designing the second neural network such that the neurons in the first final layer of the first neural network are changed by neurons whose number is the same as the number of detailed classes, and the resultant neural network is employed as the second neural network,

learning, by the detailed class classifier with the second neural network, to classify the set of images into the plurality of detailed classes;

storing the detailed class classifier as the classifier for classifying images into detailed classes.


 
2. The method according to claim 1, wherein
the detailed class classifier and the coarse class classifier are each formed using a third neural network,
wherein the method further comprises a third design process before the first process, the third design process including designing the third neural network such that a third final layer of the third neural network includes neurons whose number is equal to the sum of the number of coarse classes and the number of detailed classes,
wherein in the first design process, the first neural network is given by the third neural network but in the third final layer of the third neural network, particular neurons whose number is equal to the number of coarse classes are selected and only the selected neurons are used thereby designing the first neural network so as to include as many neurons in the first final layer as there are coarse classes, and
in the second design process, the second neural network is given by the third neural network but in the third final layer of the third neural network, particular neurons which are different from the neurons employed as the neurons in the first final layer of the first neural network and whose number is equal to the number of detailed classes are used thereby designing the second neural network obtained by changing the neurons in the first final layer of the first neural network made to perform the learning in the first process by neurons whose number is equal to the number of detailed classes.
 
3. The method according to any of claims 1 to 2, wherein the second design process further includes setting a weight of the second final layer of the second neural network to a random number, the weight indicating a coupling efficiency between the second final layer of the second neural network and a layer immediately in front of the second final layer.
 
4. The method according to any of claims 1 to 2, wherein the second design process further includes setting a second weight for neurons of a detailed class of the second final layer of the second neural network based on a first weight of the first final layer of the first neural network,
a coarse class of the first final layer of the first neural network is replaced by the neurons of the detailed class of the second final layer of the second neural network,
the first weight indicating a coupling efficiency, obtained via the learning in the first learning process, between the first final layer of the first neural network and the layer immediately in front of the first final layer,
the second weight indicating a coupling efficiency between the second final layer of the second neural network and the layer immediately in front of the second final layer,
the second weight determined by proportionally dividing the first weight according to a relationship between the coarse class and the neurons of the detailed class included in the coarse class.
 
5. The method according to claim 1, wherein the first process and the second process are performed by a processor.
 
6. A non-transitory computer-readable recording medium storing a program for learning a classifier that classifies an image, the program causing a computer to execute the method of claim 1.
 


Ansprüche

1. Verfahren zum Lernen eines Klassifikators (10), wobei der Klassifikators (10) zum Klassifizieren eines Bildes in eine detaillierte Klasse bestimmt ist, umfassend folgende Schritte:

Lernen des Klassifikators (10) durch:

Bereitstellen eines Satzes von Bildern, der als ein Satz von Bildern angegeben wird, die jeweils mit einem Etikett versehen sind, das eine detaillierte Klasse angibt;

Ausführen eines ersten Vorgangs (S 1), der die folgenden Schritte umfasst:

Entwerfen eines Grobklassenklassifikators (10a) in einem ersten Entwurfsvorgang, wobei der Grobklassenklassifikator (10a) mit einem ersten neuronalen Netzwerk konfiguriert ist, wobei das erste neuronale Netzwerk aus einer ersten letzten Schicht und einem ersten Satz von Schichten besteht, die sich von der ersten letzten Schicht unterscheiden; wobei die erste letzte Schicht des ersten neuronalen Netzwerks des Grobklassenklassifikators so viele Neuronen aufweist, wie es Grobklassen gibt;

Gruppieren des Satzes von Bildern durch eine Grobklassenerzeugungseinheit (61) in mehrere grobe Klassen, basierend auf der Ähnlichkeit jedes Bildes in dem Satz von Bildern;

Lernen durch den Grobklassenklassifikator mit dem ersten neuronalen Netzwerk, die Menge von Bildern in die Vielzahl von Grobklassen zu klassifizieren;

Ausführen eines zweiten Vorgangs (S2), der die folgenden Schritte umfasst:

Entwerfen eines detaillierten Klassenklassifikators (10b) in einem zweiten Entwurfsvorgang, wobei der detaillierte Klassenklassifikator mit einem zweiten neuronalen Netzwerk konfiguriert ist, wobei das zweite neuronale Netzwerk aus einer zweiten letzten Schicht und dem ersten Satz von Schichten besteht;

wobei der zweite Entwurfsvorgang das Entwerfen des zweiten neuronalen Netzwerks derart umfasst, dass die Neuronen in der ersten letzten Schicht des ersten neuronalen Netzwerks durch Neuronen geändert werden, deren Anzahl der Anzahl detaillierter Klassen entspricht, und das resultierende neuronale Netzwerk als das zweite neuronale Netzwerk verwendet wird,

Lernen durch den detaillierten Klassenklassifikator mit dem zweiten neuronalen Netzwerk, den Satz von Bildern in die Vielzahl detaillierter Klassen zu klassifizieren;

Speichern des detaillierten Klassenklassifikators als Klassifikator zum Klassifizieren von Bildern in detaillierte Klassen.


 
2. Verfahren nach Anspruch 1, bei dem
der detaillierte Klassenklassifikator und der Grobklassenklassifikator jeweils unter Verwendung eines dritten neuronalen Netzwerks ausgebildet werden,
wobei das Verfahren weiterhin einen dritten Entwurfsvorgang vor dem ersten Vorgang umfasst, wobei der dritte Entwurfsvorgang das Entwerfen des dritten neuronalen Netzwerks derart umfasst, dass eine dritte letzte Schicht des dritten neuronalen Netzwerks Neuronen enthält, deren Anzahl gleich der Summe der Anzahl von Grobklassen und der Anzahl der detaillierten Klassen ist,
wobei in dem ersten Entwurfsvorgang das erste neuronale Netzwerk durch das dritte neuronale Netzwerk gegeben ist, aber in der dritten letzten Schicht des dritten neuronalen Netzwerks bestimmte Neuronen ausgewählt werden, deren Anzahl gleich der Anzahl von Grobklassen ist und nur die ausgewählten Neuronen verwendet werden, um das erste neuronale Netzwerk so zu gestalten, dass so viele Neuronen in der ersten Endschicht enthalten sind, wie es grobe Klassen gibt, und
in dem zweiten Entwurfsvorgang das zweite neuronale Netzwerk durch das dritte neuronale Netzwerk gegeben ist, aber in der dritten letzten Schicht des dritten neuronalen Netzwerks bestimmte Neuronen, die sich von den Neuronen unterscheiden, die als Neuronen in der ersten letzten Schicht des ersten neuronalen Netzwerks verwendet werden und deren Anzahl gleich der Anzahl detaillierter Klassen ist, verwendet werden, wodurch das zweite neuronale Netzwerk entworfen wird, das man durch Ändern der Neuronen in der ersten letzten Schicht des ersten neuronalen Netzwerks erhält, das dazu bestimmt ist, das Lernen in dem ersten Vorgang durch Neuronen auszuführen, deren Anzahl gleich der Anzahl der detaillierten Klassen ist.
 
3. Verfahren nach einem der Ansprüche 1 bis 2, bei dem der zweite Entwurfsvorgang weiterhin das Setzen eines Gewichts der zweiten letzten Schicht des zweiten neuronalen Netzwerks auf eine Zufallszahl umfasst, wobei das Gewicht eine Kopplungseffizienz zwischen der zweiten letzten Schicht des zweiten neuronalen Netzwerks und eine Schicht unmittelbar vor der zweiten letzten Schicht anzeigt.
 
4. Verfahren nach einem der Ansprüche 1 bis 2, bei dem der zweite Entwurfsvorgang weiterhin das Einstellen eines zweiten Gewichts für Neuronen einer detaillierten Klasse der zweiten letzten Schicht des zweiten neuronalen Netzwerks basierend auf einem ersten Gewicht der ersten letzten Schicht des ersten neuronalen Netzwerks umfasst,
eine Grobklasse der ersten letzten Schicht des ersten neuronalen Netzwerks durch die Neuronen der detaillierten Klasse der zweiten letzten Schicht des zweiten neuronalen Netzwerks ersetzt wird,
das erste Gewicht eine Kopplungseffizienz anzeigt, die man durch das Lernen in dem ersten Lernvorgang zwischen der ersten letzten Schicht des ersten neuronalen Netzwerks und der Schicht unmittelbar vor der ersten letzten Schicht erhält,
das zweite Gewicht eine Kopplungseffizienz zwischen der zweiten letzten Schicht des zweiten neuronalen Netzwerks und der Schicht unmittelbar vor der zweiten letzten Schicht anzeigt, und
das zweite Gewicht durch proportionales Teilen des ersten Gewichts gemäß einer Beziehung zwischen der Grobklasse und den Neuronen der in der Grobklasse enthaltenen detaillierten Klasse bestimmt wird.
 
5. Verfahren nach Anspruch 1, bei dem der erste Vorgang und der zweite Vorgang von einem Prozessor ausgeführt werden.
 
6. Nicht flüchtiges computerlesbares Aufzeichnungsmedium, das ein Programm zum Lernen eines Klassifikators speichert, der ein Bild klassifiziert, wobei das Programm einen Computer veranlasst, das Verfahren nach Anspruch 1 auszuführen.
 


Revendications

1. Procédé d'apprentissage d'un classificateur (10), le classificateur (10) servant à classifier une image dans une classe détaillée, comprenant les étapes suivantes :

apprentissage du classificateur (10), par :

la fourniture d'un ensemble d'images, qui est donné comme un ensemble d'images attachées chacune à une étiquette indiquant une classe détaillée ;

la mise en œuvre d'un premier processus (S1) qui comprend les étapes suivantes :

la conception, dans un premier processus de conception, d'un classificateur de classe brute (10a), dans lequel le classificateur de classe brute (10a) est configuré avec un premier réseau neuronal, dans lequel le premier réseau neuronal est composé d'une première couche finale et d'un premier ensemble de couches autres que la première couche finale ; dans lequel la première couche finale du premier réseau neuronal du classificateur de classe brute comporte autant de neurones qu'il n'y a de classes brutes ;

le groupement, par une unité de génération de classe brute (61), de l'ensemble d'images en une pluralité de classes brutes, sur base de la similarité de chaque image dans l'ensemble d'images ;

l'apprentissage, par le classificateur de classe brute avec le premier réseau neuronal, de la classification de l'ensemble d'images dans ladite pluralité de classes brutes ;

la mise en œuvre d'un deuxième processus (S2) qui comprend les étapes suivantes :

conception, dans un deuxième processus de conception, d'un classificateur de classe détaillée (10b), dans lequel le classificateur de classe détaillée est configuré avec un deuxième réseau neuronal, dans lequel le deuxième réseau neuronal est composé d'une deuxième couche finale et du premier ensemble de couches ;

dans lequel le deuxième processus de conception comprend la conception du deuxième réseau neuronal de telle sorte que les neurones dans la première couche finale du premier réseau neuronal sont échangés avec des neurones dont le nombre est le même que le nombre de classes détaillées, et le réseau neuronal résultant est employé comme deuxième réseau neuronal,

l'apprentissage, par le classificateur de classe détaillée avec le deuxième réseau neuronal, de la classification de l'ensemble d'images dans ladite pluralité de classes détaillées ;

le stockage du classificateur de classe détaillée comme classificateur pour classifier des images en classes détaillées.


 
2. Procédé selon la revendication 1, dans lequel
le classificateur de classe détaillée et le classificateur de classe brute sont formés chacun en utilisant un troisième réseau neuronal,
dans lequel le procédé comprend en outre un troisième processus de conception avant le premier processus, le troisième processus de conception comprenant la conception du troisième réseau neuronal de telle sorte qu'une troisième couche finale du troisième réseau neuronal comprend des neurones dont le nombre est égal à la somme du nombre de classes brutes et du nombre de classes détaillées,
dans lequel, dans le premier processus de conception, le premier réseau neuronal est donné par le troisième réseau neuronal, mais dans la troisième couche finale du troisième réseau neuronal, des neurones particuliers dont le nombre est égal au nombre de classes brutes sont sélectionnés et uniquement les neurones sélectionnés sont ainsi utilisés pour la conception du premier réseau neuronal de manière à inclure autant de neurones dans la première couche finale qu'il y a de classes brutes, et
dans le deuxième processus de conception, le deuxième réseau neuronal est donné par le troisième réseau neuronal, mais dans la troisième couche finale du troisième réseau neuronal, des neurones particuliers qui sont différents des neurones employés comme neurones dans la première couche finale du premier réseau neuronal et dont le nombre est égal au nombre de classes détaillées sont utilisés pour ainsi concevoir le deuxième réseau neuronal obtenu en échangeant les neurones dans la première couche finale du premier réseau neuronal construit pour mettre en œuvre l'apprentissage dans le premier processus avec des neurones dont le nombre est égal au nombre de classes détaillées.
 
3. Procédé selon l'une quelconque des revendications 1 et 2, dans lequel le deuxième processus de conception comprend en outre le réglage d'un poids de la deuxième couche finale du deuxième réseau neuronal à un nombre aléatoire, le poids indiquant une efficacité de couplage entre la deuxième couche finale du deuxième réseau neuronal et une couche immédiatement devant la deuxième couche finale.
 
4. Procédé selon l'une quelconque des revendications 1 et 2, dans lequel
le deuxième processus de conception comprend en outre le réglage d'un deuxième poids pour les neurones d'une classe détaillée de la deuxième couche finale du deuxième réseau neuronal sur base d'un premier poids de la première couche finale du premier réseau neuronal,
une classe brute de la première couche finale du premier réseau neuronal est remplacée par les neurones de la classe détaillée de la deuxième couche finale du deuxième réseau neuronal,
le premier poids indiquant une efficacité de couplage, obtenue via l'apprentissage dans le premier processus d'apprentissage, entre la première couche finale du premier réseau neuronal et la couche immédiatement devant la première couche finale,
le deuxième poids indiquant une efficacité de couplage entre la deuxième couche finale du deuxième réseau neuronal et la couche immédiatement devant la deuxième couche finale,
le deuxième poids étant déterminé en divisant proportionnellement le premier poids conformément à une relation entre la classe brute et les neurones de la classe détaillée incluse dans la classe brute.
 
5. Procédé selon la revendication 1, dans lequel le premier processus et le deuxième processus sont mis en œuvre par un processeur.
 
6. Support d'enregistrement non volatile lisible par un ordinateur sur lequel est stocké un programme pour l'apprentissage d'un classificateur qui classe une image, le programme commandant à un ordinateur d'exécuter le procédé selon la revendication 1.
 




Drawing
























































Cited references

REFERENCES CITED IN THE DESCRIPTION



This list of references cited by the applicant is for the reader's convenience only. It does not form part of the European patent document. Even though great care has been taken in compiling the references, errors or omissions cannot be excluded and the EPO disclaims all liability in this regard.

Non-patent literature cited in the description