COMMUNICATION OPTIMIZATIONS FOR DISTRIBUTED MACHINE LEARNING

(19)

(11)

EP 3 506 095 A3

(12)	EUROPEAN PATENT APPLICATION

(88)	Date of publication A3:
	25.09.2019 Bulletin 2019/39

(43)	Date of publication A2:
	03.07.2019 Bulletin 2019/27

(21)	Application number: 18209320.3

(22)	Date of filing: 29.11.2018

(51)

Int. Cl.:

G06F 9/50 ^(2006.01)

(84)	Designated Contracting States:
	AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR
	Designated Extension States:
	BA ME
	Designated Validation States:
	KH MA MD TN

(30)

Priority:

29.12.2017 US 201715859180

(71)	Applicant: INTEL Corporation
	Santa Clara, CA 95054 (US)

(72)	Inventors:
	SRIDHARAN, Srinivas 560008 Bangalore (IN) VAIDYANATHAN, Karthikeyan 560034 Bangalore (IN) DAS, Dipankar 411003 Pune (IN) SAKTHIVEL, Chandrasekaran Sunnyvale, CA California 94087 (US) SMORKALOV, Mikhail E. 60312 Nizhniy Novgorod (RU)

(74)	Representative: Goddar, Heinz J.
	Boehmert & Boehmert Anwaltspartnerschaft mbB Pettenkoferstrasse 22 80336 München 80336 München (DE)

(54)	COMMUNICATION OPTIMIZATIONS FOR DISTRIBUTED MACHINE LEARNING

(57) Embodiments described herein provide a system to configure distributed training of a neural network, the system comprising memory to store a library to facilitate data transmission during distributed training of the neural network; a network interface to enable transmission and receipt of configuration data associated with a set of worker nodes, the worker nodes configured to perform distributed training of the neural network; and a processor to execute instructions provided by the library, the instructions to cause the processor to create one or more groups of the worker nodes, the one or more groups of worker nodes to be created based on a communication pattern for messages to be transmitted between the worker nodes during distributed training of the neural network.