(19)
(11)EP 3 602 290 B1

(12)EUROPEAN PATENT SPECIFICATION

(45)Mention of the grant of the patent:
06.07.2022 Bulletin 2022/27

(21)Application number: 18718017.9

(22)Date of filing:  22.03.2018
(51)International Patent Classification (IPC): 
G06F 9/455(2018.01)
G06F 9/48(2006.01)
(52)Cooperative Patent Classification (CPC):
G06F 9/4843; G06F 9/485; G06F 2009/45575; G06F 9/45558; G06F 9/4881
(86)International application number:
PCT/US2018/023674
(87)International publication number:
WO 2018/183064 (04.10.2018 Gazette  2018/40)

(54)

COOPERATIVE VIRTUAL PROCESSOR SCHEDULING

KOOPERATIVES ABLAUFPLANUNG VON VIRTUELLEN PROZESSOREN

ORDONNANCEMENT COOPERATIF DE PROCESSEURS VIRTUELS


(84)Designated Contracting States:
AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

(30)Priority: 31.03.2017 US 201715476925

(43)Date of publication of application:
05.02.2020 Bulletin 2020/06

(73)Proprietor: Microsoft Technology Licensing, LLC
Redmond, WA 98052-6399 (US)

(72)Inventors:
  • OKS, Artem
    Redmond, Washington 98052-6399 (US)
  • HEPKIN, David
    Redmond, Washington 98052-6399 (US)

(74)Representative: CMS Cameron McKenna Nabarro Olswang LLP 
Cannon Place 78 Cannon Street
London EC4N 6AF
London EC4N 6AF (GB)


(56)References cited: : 
US-A1- 2015 007 170
US-B1- 8 539 499
  
  • Tianxiang Miao ET AL: "FlexCore: Dynamic Virtual Machine Scheduling Using VCPU Ballooning", , February 2015 (2015-02), pages 7-16, XP055483662, DOI: 10.1109/TST.2015.7040515 Retrieved from the Internet: URL:https://ieeexplore.ieee.org/ielx7/5971 803/7040506/07040515.pdf?tp=&arnumber=7040 515&isnumber=7040506 [retrieved on 2018-06-13]
  
Note: Within nine months from the publication of the mention of the grant of the European patent, any person may give notice to the European Patent Office of opposition to the European patent granted. Notice of opposition shall be filed in a written reasoned statement. It shall not be deemed to have been filed until the opposition fee has been paid. (Art. 99(1) European Patent Convention).


Description

FIELD



[0001] The invention relates to moving scheduling of processor time for virtual processors (VPs) out of a virtualization hypervisor.

BACKGROUND



[0002] Among the many forms of computer virtualization, machine or system virtualization has become common due to many known advantages. System virtualization involves abstracting the hardware resources of a computer and presenting the computer as virtual machines. A layer of software referred to as a hypervisor or virtual machine monitor (VMM) runs directly on the hardware of a computer. The hypervisor manages access to the hardware of the computer by virtual machines (VMs), which are also known as partitions, domains, or guests. Each VM is a software environment or construct capable of hosting its own guest operating system. The hypervisor manages sharing of the computer's hardware, in particular processing hardware, by the VMs. The presence of a layer of software - the hypervisor- between the guest operating system and the computer hardware is mostly transparent to the guest operating system.

[0003] One type of hypervisor is the type I hypervisor, also referred to as a stand-alone, native, bare metal, or embedded hypervisor. Examples of type I hypervisors are Microsoft Hyper-V (TM) from Microsoft Corp., KVM (TM) (Kernel-based Virtual Machine) from Red Hat Corp., VMware ESXi (TM) from VMware Inc., and Xen (TM), which is available from various sources. Traditionally, in virtualization architectures that use a type I hypervisor ("hypervisor", hereafter), the hypervisor is responsible for managing the virtualization of processors. Specifically, the hypervisor applies resource policies to virtual processors and schedules execution of virtual processors on physical processors. A hypervisor is usually accompanied by a privileged VM that hosts a special guest operating system that handles other virtualization-related tasks that need not reside in the hypervisor. This special operating system will be referred to as the host operating system or the host kernel. The hypervisor and host operating system together will be referred to as a virtualization layer. The virtualization tasks handled by the host operating system varies among hypervisor implementations. Some may handle aspects of device emulation/virtualization, I/O virtualization, inter-VM communication, etc. Some may provide a user interface to enable a user to configure the virtualization layer, VMs, etc. The host operating system may also have administrative tools or other software that is not necessary for virtualization.

[0004] Although which virtualization responsibilities are handled by the hypervisor and which are handled by the host operating system can vary, all type I hypervisors (including kernel-based hypervisors such as the KVM hypervisor) handle processor virtualization, including scheduling of processor time for virtual processors, managing the kernel objects (e.g. threads or processes) that back the virtual processors, etc. This approach has shortcomings.

[0005] One shortcoming of having the hypervisor handle processor scheduling is scheduler redundancy. Operating systems that are employed as VM guests or as host operating systems are usually designed to be capable of running directly on hardware. They have their own kernel scheduler designed for scheduling their own threads on processors (or on virtual processors, as the case may be). These operating systems' schedulers are usually heavily engineered for resource control and for supporting advanced power management policies. To date, hypervisors have also had their own scheduler implementation that is independent of the scheduler in the host/operating systems. Because a hypervisor has its own scheduler, some of the sophisticated scheduler logic often found in host/guest operating systems must either be omitted or else duplicated in the hypervisor scheduler. For example, supposing a host/guest kernel's scheduler is enhanced with a new feature to make its scheduling more power efficient, if the benefit is to be had on a computer where the host kernel executes in a VM on top of a hypervisor (which actually controls scheduling), this new feature must be duplicated in the hypervisor's scheduler for the host computer to obtain the same benefit.

[0006] The stacked layers of scheduling also make it difficult to realize certain performance and management improvements. Regarding performance, it has been observed that the nature of VM workloads often vary. For instance, some VMs might be I/O bound, others might be processor bound. A hypervisor scheduler cannot self-adjust to suit VM workload conditions because those conditions may not be visible to the hypervisor. For example, if some I/O virtualization occurs at the host operating system, actionable information about I/O conditions may not be available in the hypervisor. Regarding management, when the hypervisor handles processor scheduling, CPU resource reporting for virtual machines is not naturally accounted for in the host operating system. Because the host operating system does not have direct visibility to how a VM's virtual processors are being scheduled, the host operating system cannot determine how physical CPU resources are actually being consumed. CPU resource reporting tools in the host operating system may report an incorrect view of CPU resource consumption, since such tools are in fact reporting CPU resource consumption of the host operating system (and not the virtual machines). Accordingly, system tools must be retooled if system-wide CPU usage is to be monitored accurately in a virtualized environment. Moreover, hypervisor-based scheduling can make execution tracing and analysis difficult because of the need to correlate between events from two schedulers.

[0007] Implementing scheduling in the hypervisor also carries security risks. Because each VM's execution is controlled in the same security domain (the hypervisor), there is a risk that a hypervisor breach of one VM could lead to compromise of the hypervisor and thus compromise of the other VMs.

[0008] Some of the problems discussed above may be solved by scheduling virtual processors outside of the hypervisor, as described below.

[0009] US 8,539,499 B1 discloses a system, method and computer program product, including a primary operating system (OS) having access to at least some hardware resources of the computer system. A Hypervisor controls a plurality of virtualization spaces and at least some of the remaining hardware resources. Each virtualization space maintains data of a corresponding instance of a Virtual Machine with a guest operating system running inside the Virtual Machine. Each Virtual Machine is associated with at least one virtual processor. A hyperswitch controlled by the Hypervisor starts and stops execution of the Virtual Machines. A scheduler dedicates a quantum of running time to each virtual processor. When the quantum is given to the Virtual Machine, the Hypervisor forces the hyperswitch to activate the Virtual Machine on its virtual processor.

SUMMARY



[0010] According to aspects of the present invention there is provided a method, a device and a computing storage as defined in the accompanying claims.

[0011] The following summary is included only to introduce some concepts discussed in the Detailed Description below. This summary is not comprehensive and is not intended to delineate the scope of the claimed subject matter, which is set forth by the claims presented at the end.

[0012] The invention relates to moving scheduling of processor time for virtual processors (VPs) out of a virtualization hypervisor. A host operating system schedules VP processor time. The host operating system creates VP backing threads, one for each VP of each VM. There is a one-to-one mapping between each VP thread in the host operating system and each VP in the hypervisor. When a VP thread is dispatched for a slice of processor time, the host operating system calls into the hypervisor to have the hypervisor start executing the VP, and the hypervisor may perform a processor context switch for the VP. Of note is the security separation between VP scheduling and VP context switching. The hypervisor manages VP contexts and controls VP context switching while VP scheduling is performed in kernel mode in the context of the host operating system's VM. There is a security/interface boundary between the unit that schedules VP processor time and the hypervisor.

[0013] Many of the attendant features will be explained below with reference to the following detailed description considered in connection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS



[0014] The present description will be better understood from the following detailed description read in light of the accompanying drawings, wherein like reference numerals are used to designate like parts in the accompanying description.

Figure 1 shows an example virtualization environment that includes a known type of a system virtualization hypervisor.

Figure 2 shows details of hypervisor-based processor scheduling.

Figure 3 shows a virtualization architecture that implements processor scheduling for VMs in the host operating system.

Figure 4 shows a control flow for handling intercepts.

Figure 5 shows a flow for handling interrupts.

Figure 6 shows a control flow for handling context switches.

Figure 7 shows a control flow for unblocking a VP.

Figure 8 shows details of a computing device.


DETAILED DESCRIPTION



[0015] Embodiments described below relate to shifting scheduling of processor time for virtual processors (VPs) from a hypervisor to a host operating system. In one embodiment, the host operating system schedules a VP by making a blocking call into the hypervisor to run a VP. The host operating system creates VP threads, one for each VP of each VM. There is a one-to-one mapping between each VP thread in the host operating system and each VP in the hypervisor. When a VP thread is dispatched for a slice of processor time, the host operating system calls into the hypervisor to have the hypervisor start executing the VP. When an interrupt occurs, execution of the VP stops, and the hypervisor returns control back to the host operating system. This allows the host operating system to then schedule a different thread, which can be a VP thread or just a regular software thread that has no relation to a virtual processor.

[0016] Figure 1 shows an example virtualization environment that includes a known type of hypervisor 100. A computer 102 has hardware 104, including a central processing unit (CPU) 106, memory 108, a network interface 110, non-volatile storage 112, and other components not shown, such as a bus, a display and/or display adapter, etc. The hypervisor 100 manages and facilitates execution of virtual machines (VMs) 114, 116. Each virtual machine 114, 116 typically has virtualized devices including a virtual disk within which a guest/host operating system 116, 118 is stored. Machine or system virtualization is provided by the hypervisor 100 cooperating with a host operating system 118 that executes in a privileged VM 116. The tasks of virtualization may be distributed among the hypervisor 100 and the host operating system 118 in a number of ways, as discussed in the Background. In some cases, the host operating system 118 might consist only of minimal virtualization elements such as tools and user interfaces for managing the hypervisor 100. In other cases, the host operating system 118 might include one or more of: device virtualization management, inter-VM communication facilities, running device drivers, starting or stopping other VMs. In short, the guest operating system 118 can potentially handle any virtualization tasks outside core functions of a hypervisor such as CPU and memory sharing. Despite a wide range of virtualization layer designs, as discussed next, all previous virtualization systems have implemented all aspects of processor scheduling in the hypervisor 100.

[0017] Figure 2 shows details of hypervisor-based processor scheduling. All previous virtualization hypervisors have been kernels 130 that implement processor sharing completely within the hypervisor. With this kernel-based processor sharing approach, the hypervisor kernel 130 typically backs virtual processors (VPs) 132 with kernel-level execution units such as processes and/or threads. The hypervisor kernel 130 has a scheduler 134 that implements perhaps complex scheduling algorithms to determine which VPs are to execute on which physical processors (PPs) 134, and when. Often, the scheduler 134 is the same type of scheduler ordinary operating system kernels use for time-sharing a processor among processes or threads. The hypervisor kernel 130 has also been responsible for the mechanics of time-slicing, for instance performing PP context switches. This kernel-based approach has shortcomings. As shown in Figure 2, there is a redundancy issue; the kernels 136 of guest and host operating systems have their own schedulers 138. In addition, implementing scheduling in the hypervisor creates security risks because if the scheduling code is compromised any VM could potentially access the data of any other VM. Other problems with hypervisor-based scheduling are discussed in the Background. Regarding terminology, as used herein "physical processor" refers to both physical cores of a CPU as well as logical cores of CPUs that implement hyperthreading; from software's perspective, logical and physical processors/cores are mostly indistinguishable.

[0018] Figure 3 shows a virtualization architecture that implements processor scheduling for VMs in the host operating system 118. This new architecture shifts processor scheduling for VMs out of the hypervisor 100 and into the host operating system 118. Conceptually, the host operating system's scheduler 138 makes decisions about which VPs will execute and when. The host operating system's scheduler 138 tells the hypervisor 100 which VP to execute. More specifically, the host operating system maintains a map 146 indicating which threads (or other execution backing units) correspond to which VPs (for all VMs). The hypervisor has a state manager 148 that manages processor states of the VPs in a state table 149. The state manager 148 uses the state table 149 to perform context switches to implement the processor-sharing instructions (hypercalls) of the host operating system 118. The host kernel 136 instantiates and manages the kernel-level execution units/objects that back the VPs. For discussion, the backing objects will be described as threads, although processes or other units of execution can be used.

[0019] When the host operating system scheduler 138 determines that a VM is to be started, the host operating system creates VP backing threads for the respective VPs of the starting VM. Each VP of each VM has a VP backing thread. Each VP backing thread runs a dispatch loop until, for whatever reason, the corresponding VP is stopped. The scheduler 138 in the host VM schedules VP backing threads as regular threads subject to additional VM/VP-specific scheduling policies and possibly enlightenments (information provided by guests). The hypervisor 100 is configured with an interface 150 that exposes one or more hypercalls allowing the host operating system to make calls into the hypervisor.

[0020] To elaborate, the VP backing/dispatch threads are thread objects in the host operating system that function like any other thread object. The host operating system scheduler 138 schedules all of its threads including the VP backing threads. In one embodiment, each VP backing thread knows the id of the corresponding VP, and when a VP backing thread is scheduled to run, it calls into the hypervisor to dispatch the VP on the current processor. Each VP backing thread runs a dispatch loop. When a VP backing thread receives time to run, it makes a hypercall intercepted by the hypervisor that instructs the hypervisor to "run this VP". The host operating system might need to tell the hypervisor, through hypercall parameters, which VP and/or VM to start running. The hypervisor only needs to know which VP (or VM) to run. The processor state for the VP need not be visible to the host VM and preferably the host VM is not permitted or able to access same directly.

[0021] From the perspective of the host VM, a VP is just another thread and execution of the VP by the hypervisor is transparent. When a VP backing thread is scheduled and starts executing, it is running on a PP (the one currently executing a VP of the host VM when the VP backing thread is started). The VP backing thread executes and invokes the hypercall to switch into the hypervisor (a privileged hypervisor operation). When entering hypervisor mode, the VP backing thread is initially still executing on the same PP with the host VM's context. As noted, the hypervisor maintains processor states of all VPs in the state table 149 and knows which VP the VP backing thread corresponds to. After saving the current PP state 152 (corresponding to the host VM's VP), the hypervisor swaps into the current PP the previously-stored processor state 154 of the VP that issued the hypercall, effectively beginning execution of the relevant VP. At this point, the VP is executing guest code on the PP. The hypervisor lets the VP run until either the VP blocks voluntarily, the VP generates an intercept for the root, or an interrupt has arrived for the root VP. On any of these three events the hypervisor context-switches back to the host's VP and the latter completes the hypercall and returns to the host VM. In sum, the hypervisor merely executes the VPs it is told to execute, and the host operating system controls the scheduling of VP execution.

[0022] Because the hypervisor is informed, through a hypercall parameter, which VP is executing, the hypervisor knows which processor state to swap in from the state table 149. Note that in one embodiment the host operating system sees each root/host VP as a logical/physical processor and does not need to schedule its own processing time in the same way that it schedules other VM processing time. When a VP is executing and there is an interrupt or other event that needs to be taken care of by host operating system, the hypervisor just switches context to the corresponding root/host VP (there is a one-to-one mapping between each root/host VP and PP). When the root VP enters (returns to) the host VM, it is running in the context of whatever thread made the hypercall, and the host kernel decides what to do next.

[0023] To summarize, there is no scheduler in the hypervisor. There may be one stack per PP in the hypervisor. The host VM owns all PP scheduling and idling decisions. Each VP has VP backing thread or other unit of execution. The host VM schedules the VP backing threads. Each VP backing thread: runs a dispatch loop, makes a hypercall to switch from the host VP to the corresponding target VP (e.g., VP context switch hypercall), and optionally provides host targeted work requests on return from the VP context switch hypercall.

[0024] Figure 4 shows a control flow for handling intercepts. An intercept is a return from a VP context switch hypercall. When the host kernel 136 decides to schedule a VP, the host kernel 136 issues a hypercall with an identifier of the VP. The hypervisor 100 begins executing the thread with the context of the VP for the relevant VM 114. When the VM 114 triggers an intercept, the hypervisor kernel returns control to the host kernel 136 with a return from the initial hypercall. The host kernel 136 then processes the intercept, issues another hypercall, and the VM/VP necessary for the intercept.

[0025] Figure 5 shows a flow for handling an interrupt. After the hypercall and context switch to the relevant VP, an interrupt occurs. In response, the VP exits, control returns to the hypervisor, and the host kernel processes the interrupt, eventually returning control to the VP through another hypercall.

[0026] Figure 6 shows a control flow for handling a context switch. After an initial hypercall and while a first VM's VP is executing, a timed scheduling interrupt is generated and the VP of the first VM exits. The hypercall returns to the host kernel, which switches threads to the next scheduled thread and issues a hypercall for the VP of the new thread.

[0027] Figure 7 shows how a control flow for unblocking a VP. As can be seen, unblocking of a VP may require an extra trip to the host if there are not any optimizations. The control flow operates as follows. A VM's first VP is running while the VM's second VP is idle. At some point, the first VP requests to send an IPI to the second VP. As the result, the first VP exits into the hypervisor and sets a pending virtual interrupt for the second VP. The hypervisor switches to the host VP to deliver the wakeup (unblock) signal to the VP backing thread of the second VP. The host VP returns, signals the requested VP backing thread, and makes another hypercall to dispatch and continue running the VM's first VP. At some time shortly thereafter, the host operating system schedules to run a VP backing thread on another host VP. The latter calls into the hypervisor and the VM's second VP starts executing the guest code.

[0028] Although a host operating system is convenient, other techniques for separating VP scheduling from the hypervisor may be used. In one embodiment, the privileged extra-hypervisor functionality of the host operating system need not be executed in a VM or container. Any module that interfaces with the hypervisor performs the VP scheduling. For example, a module with an execution context permanently running on a PP can avoid context switches between host VPs and hypervisor idle threads. This may also reduce code path lengths and latencies for entering leaving idle states. In addition, as mentioned above, enlightenments may be implemented to allow guest VMs to provide the host VM with information relevant to scheduling, such as workload profiles, scheduling priorities, I/O statistics, etc.

[0029] Removing VP scheduling from the hypervisor can provide other advantages. Newer CPUs may have different types of cores on the same chip (so-called "big.LITTLE" designs). This type of CPU can be leveraged by exposing the CPU topology to guest VMs (e.g., big VPs and LITTLE VPs). The kernel scheduler in a big.LITTLE guest VM makes its own scheduling decisions. The host operating system is aware of which VP backing threads are big and which ones are LITTLE. The host operating system's scheduler decides how to schedule big and LITTLE VP backing threads according to the host operating system's power management policies and constraints.

[0030] Removing VP scheduling from the hypervisor can have other advantages. The host operating system may have tools for analyzing system performance. If host threads are used to back VPs, tools for analyzing thread performance can provide direct insights into VP performance. (e.g., CPU usage, context switches, dispatch latencies, etc.). That is, because each VP is backed by a host thread, the host kernel can maintain the counters inside the host VM such as time spent in the hypervisor, time spent inside of its partition, time spent in the host kernel, etc.

[0031] In some embodiments, the VM guest operating systems and the host operating system might have the same kernel; i.e., each might have an instance of the same basic operating system kernel, albeit within different VMs and for different purposes. In such a case, enlightenments can allow for better integration of lightweight VM containers that can host applications. This technique provides improved isolation and sandboxing of applications (i.e., better security but with acceptable performance). As noted above, other guest enlightenments might be supported. For example, a guest might invoke a hypercall to share information about its workload, processing priorities or characteristics, etc. The host operating system may take this into account when making scheduling decisions.

[0032] Figure 8 shows details of the computing device 102 on which embodiments described above may be implemented. The technical disclosures herein will suffice for programmers to write software, and/or configure reconfigurable processing hardware (e.g., field-programmable gate arrays (FPGAs)), and/or design application-specific integrated circuits (ASICs), etc., to run on the computing device 220 to implement any of the features or embodiments described herein.

[0033] The computing device 102 has one or more displays 222, a network interface 224 (or several), as well as storage hardware 226 and processing hardware 228, which may be a combination of any one or more: central processing units, graphics processing units, analog-to-digital converters, bus chips, FPGAs, ASICs, Application-specific Standard Products (ASSPs), or Complex Programmable Logic Devices (CPLDs), etc. The storage hardware 226 may be any combination of magnetic storage, static memory, volatile memory, non-volatile memory, optically or magnetically readable matter, etc. The meaning of the term "storage", as used herein does not refer to signals or energy per se, but rather refers to physical apparatuses and states of matter. The hardware elements of the computing device 102 may cooperate in ways well understood in the art of machine computing. In addition, input devices may be integrated with or in communication with the computing device 102. The computing device 102 may have any form-factor or may be used in any type of encompassing device. The computing device 102 may be in the form of a handheld device such as a smartphone, a tablet computer, a gaming device, a server, a rack-mounted or backplaned computer-on-a-board, a system-on-a-chip, or others.

[0034] Embodiments and features discussed above can be realized in the form of information stored in volatile or non-volatile computer or device readable storage hardware. This is deemed to include at least hardware such as optical storage (e.g., compact-disk read-only memory (CD-ROM)), magnetic media, flash read-only memory (ROM), or any means of storing digital information in to be readily available for the processing hardware 228. The stored information can be in the form of machine executable instructions (e.g., compiled executable binary code), source code, bytecode, or any other information that can be used to enable or configure computing devices to perform the various embodiments discussed above. This is also considered to include at least volatile memory such as random-access memory (RAM) and/or virtual memory storing information such as central processing unit (CPU) instructions during execution of a program carrying out an embodiment, as well as non-volatile media storing information that allows a program or executable to be loaded and executed. The embodiments and features can be performed on any type of computing device, including portable devices, workstations, servers, mobile wireless devices, and so on.


Claims

1. A method performed by a computing device (102) comprising a central processing unit, CPU, (106) comprising physical processors, PPs, (134), the method comprising:

executing a hypervisor (100) comprising a first kernel (130), and executing an operating system (118) comprising a second kernel (136), wherein the operating system and the second kernel execute in a scheduling virtual machine (116), VM, comprising a corresponding scheduling virtual processor, VP;

managing by the first kernel, scheduled VMs (114) comprising respective scheduled virtual processors (132) the managing comprising maintaining physical processor states of the respective scheduled VPs and swapping the physical processor states (149) in and out of the PPs, the physical processor states being stored in the hypervisor, wherein the scheduled VPs (132) do not include the scheduling VP; and

scheduling (138) the scheduled VPs to execute on the PPs, wherein the scheduling is performed by the scheduling VP executing the second kernel (136) in the scheduling VM (116), wherein when the scheduling by the second kernel determines to execute a given scheduled VP, the second kernel identifies the given scheduled VP to the hypervisor and based thereon the first kernel (130) performs a processor context switch on a PP for the given scheduled VP, the processor context switch performed using the physical processor state corresponding to the identified given scheduled VP.


 
2. A method according to claim 1, wherein the first kernel (130) implements a hypercall, and
wherein when the second kernel (136) determines to execute the given scheduled VP, the second kernel invokes the hypercall to identify the given VP to the first kernel.
 
3. A method according to claim 2, wherein the first kernel maintains physical processor states (149) for the respective scheduled VPs, wherein the hypercall is invoked with an identifier of the given scheduled VP, and based thereon the first kernel performs the processor context switch.
 
4. A method according to any preceding claim 1, wherein neither the hypervisor nor the first kernel schedule execution of the VPs.
 
5. A method according to any preceding claim , further comprising executing an operating system as a guest in one of the VMs, the operating system comprising the second kernel.
 
6. A computing device (100) comprising:

a central processing unit, CPU, (106) comprising physical processors, PPs, (134);

storage hardware (226) storing a hypervisor comprising a hypervisor kernel (130) and a operating system (118), OS, comprising a OS kernel (136), the OS kernel configured to run in a scheduling virtual machine (116), VM, executed by a corresponding scheduling virtual processor, VP, the hypervisor kernel (130) configured to:

maintain physical processor states (149) of respective scheduled virtual processors, VPs, (132) for virtual machines, VMs, (114) that are not the scheduling VM, wherein the scheduling VP is not one of the scheduled VPs;

the OS kernel (136) configured to make scheduling decisions for scheduling execution of the scheduled VPs (132), where operations to implement the scheduling decisions are performed by the hypervisor kernel; and

wherein, each time the OS kernel (136) makes a scheduling decision for a given scheduled VP to execute on a PP, the OS kernel (136) makes a corresponding call to identify the given scheduled VP to the hypervisor to cause the hypervisor kernel (130) to perform a context switch to begin executing the scheduled VP on the PP using the physical processor state corresponding to the identified given scheduled VP.


 
7. A computing device according to claim 6, wherein the operating system kernel is configured to run within the scheduling VM (116), wherein the representations comprise respective threads of the operating system kernel, and wherein the hypervisor kernel executes a scheduled VP by executing a corresponding thread on a PP.
 
8. A computing device according to claim 6 or claim 7, wherein the operating system kernel is configured to control which scheduled VPs will execute on which PPs, and wherein the hypervisor kernel is configured to, each time the operating system kernel issues a call to execute a scheduled VP, execute the corresponding scheduled VP on a PP.
 
9. A computing device according to any one of claims 6 to 8, wherein the operating system kernel is configured to control which scheduled VPs will execute on which PPs, and wherein the hypervisor kernel is configured to, each time the operating system kernel issues a call to execute a scheduled VP, execute the corresponding scheduled VP on a PP.
 
10. A computing device according to any one of claims 6 to 9, wherein the hypervisor kernel is configured to handle an interrupt of execution of a scheduled VP on a PP by returning corresponding control to the operating system kernel.
 
11. A computing device according to any one of claims 6 to 10 wherein the hypervisor kernel is not configured to schedule execution of VPs on PPs.
 
12. Computer storage hardware (226) storing information to enable a computing device to perform a method according to any one of claims 1 to 5.
 


Ansprüche

1. Verfahren, das von einer Rechenvorrichtung (102) durchgeführt, das eine zentrale Verarbeitungseinheit, CPU, (106) umfasst, die physische Prozessoren, PP, (134) umfasst, wobei das Verfahren Folgendes umfasst:

Ausführen eines Hypervisors (100), der einen ersten Kern (130) umfasst, und Ausführen eines Betriebssystems (118), das einen zweiten Kern (136) umfasst, wobei das Betriebssystem und der zweite Kern auf einer planenden virtuellen Maschine (116), VM, ausgeführt werden, die einen entsprechenden planenden virtuellen Prozessor, VP, umfasst;

Verwalten durch den ersten Kern von geplanten VM (114), die jeweilige geplante virtuelle Prozessoren (132) umfassen, wobei Verwalten Aufrechterhalten von physischen Prozessorzuständen der jeweilige geplanten VP und Umladen der physischen Prozessorzustände (149) in und aus den PP umfasst, wobei die physischen Prozessorzustände in dem Hypervisor gespeichert sind, wobei die geplanten VP (132) nicht den planenden VP einschließen; und

Planen (138) der geplanten VP, damit sie auf den PP ausgeführt werden, wobei Planen von dem planenden VP durchgeführt wird, der auf dem zweiten Kern (136) auf der planenden VM (116) ausgeführt wird, wobei, wenn Planen durch den zweiten Kern bestimmt, einen gegebenen geplanten VP auszuführen, der zweite Kern den gegebenen geplanten VP gegenüber dem Hypervisor identifiziert, und darauf basierend der erste Kern (130) eine Prozessorkontextumschaltung an einem PP für den gegebenen geplanten VP durchführt, wobei die Prozessorkontextumschaltung unter Verwendung des physischen Prozessorzustands durchgeführt wird, der dem identifizierten gegebenen geplanten VP entspricht.


 
2. Verfahren nach Anspruch 1, wobei der erste Kern (130) einen Hypercall implementiert, und wobei, wenn der zweite Kern (136) bestimmt, den gegebenen geplanten VP auszuführen, der zweite Kern den Hypercall aufruft, um den gegebenen VP gegenüber dem ersten Kern zu identifizieren.
 
3. Verfahren nach Anspruch 2, wobei der erste Kern physische Prozessorzustände (149) für die jeweilige geplanten VP aufrecht erhält, wobei der Hypercall mit einer Kennung des gegebenen geplanten VP aufgerufen wird, und darauf basierend der erste Kern Prozessorkontextumschaltung durchführt.
 
4. Verfahren nach einem der vorstehenden Ansprüche 1, wobei weder der Hypervisor noch der erste Kern Ausführung der VP planen.
 
5. Verfahren nach einem der vorstehenden Ansprüche, weiter umfassend Ausführen eines Betriebssystems als Gast auf einer der VM, wobei das Betriebssystem den zweiten Kern umfasst.
 
6. Rechenvorrichtung (100), umfassend:

eine zentrale Verarbeitungseinheit, CPU, (106) die physische Prozessoren, PP, (134) umfasst;

Speicherhardware (226), die einen Hypervisor speichert, der einen Hypervisor-Kern (130) umfasst, und ein Betriebssystem (118), OS, das einen OS-Kern (136) umfasst, wobei der OS-Kern konfiguriert ist, um auf einer planenden virtuellen Maschine (116), VM, zu laufen, die von einem entsprechenden
virtuellen Prozessor, VP, ausgeführt wird,

wobei der Hypervisor-Kern (130) konfiguriert ist, um:

physische Prozessorzustände (149) von jeweiligen geplanten virtuellen Prozessoren, VP,(132) für virtuelle Maschinen, VM, (114), die nicht die planende VM sind, aufrecht zu erhalten, wobei der planende VP nicht einer der geplanten VP ist;

wobei der OS-Kern (136) konfiguriert ist, um Planungsentscheidungen zur Planung von Ausführung der geplanten VP (132) zu treffen, wobei durch den Hypervisor-Kern Operationen durchgeführt werden, um die Planungsentscheidungen zu implementieren; und

wobei, jedes Mal, wenn der OS-Kern (136) eine Planungsentscheidung trifft, einen gegebenen geplanten VP auf einem PP auszuführen, der OS-Kern (136) einen entsprechenden Anruf vornimmt, um den gegebenen geplanten VP gegenüber dem Hypervisor zu identifizieren, um zu verursachen, dass der Hypervisor-Kern (130) eine Kontextumschaltung durchführt, um Ausführen des geplanten VP auf dem PP unter Verwendung des physischen Prozessorzustands, der dem identifizierten gegebenen geplanten VP entspricht, zu beginnen.


 
7. Rechenvorrichtung nach Anspruch 6, wobei der Betriebssystemkern konfiguriert ist, um innerhalb der planenden VM (116) zu laufen, wobei die Darstellungen jeweilige Threads des Betriebssystemkerns umfassen und wobei der Hypervisor-Kern einen geplanten VP ausführt durch Ausführen eines entsprechenden Threads auf einem PP.
 
8. Rechenvorrichtung nach Anspruch 6 oder Anspruch 7, wobei der Betriebssystemkern konfiguriert ist, um zu steuern, welche geplanten VP auf welchen PP ausgeführt werden, und wobei der Hypervisor-Kern konfiguriert ist, um, jedes Mal, wenn der Betriebssystemkern einen Anruf ausgibt, um einen geplanten VP auszuführen, den entsprechenden geplanten VP auf einem PP ausführt.
 
9. Rechenvorrichtung nach einem der Ansprüche 6 bis 8, wobei der Betriebssystemkern konfiguriert ist, um zu steuern, welche geplanten VP auf welchen PP ausgeführt werden, und wobei der Hypervisor-Kern konfiguriert ist, um, jedes Mal, wenn der Betriebssystemkern einen Anruf ausgibt, um einen geplanten VP auszuführen, den entsprechenden geplanten VP auf einem PP ausführt.
 
10. Rechenvorrichtung nach einem der Ansprüche 6 bis 9, wobei der Hypervisor-Kern konfiguriert ist, um eine Unterbrechung von Ausführung eines geplanten VP auf einem PP durch Rücksenden von entsprechender Steuerung an den Betriebssystemkern zu handeln.
 
11. Rechenvorrichtung nach einem der Ansprüche 6 bis 10, wobei der Hypervisor-Kern nicht konfiguriert ist, um Ausführung von VP auf PP zu planen.
 
12. Computerspeicherhardware (226), auf der Informationen gespeichert sind, um ein Rechenvorrichtung zu befähigen, ein Verfahren nach einem der Ansprüche 1 bis 5 durchzuführen.
 


Revendications

1. Procédé réalisé par un dispositif informatique (102) comprenant un processeur, CPU, (106) comprenant des processeurs physiques, PP, (134), le procédé comprenant :

l'exécution d'un hyperviseur (100) comprenant un premier noyau (130), et l'exécution d'un système d'exploitation (118) comprenant un second noyau (136), dans lequel le système d'exploitation et le second noyau sont exécutés dans une machine virtuelle (116), VM, de planification comprenant un processeur virtuel, VP, de planification correspondant ;

la gestion, par le premier noyau, de VM (114) planifiées comprenant des processeurs virtuels (132) planifiés respectifs, la gestion comprenant le maintien d'états de processeurs physiques des VP planifiés respectifs et la permutation des états de processeurs physiques (149) dans et hors des PP, les états de processeurs physiques étant stockés dans l'hyperviseur, dans lequel les VP (132) planifiés n'incluent pas le VP de planification ; et

la planification (138) des VP planifiés devant être exécutés sur les PP, dans lequel la planification est réalisée par le VP de planification exécutant le second noyau (136) dans la VM de planification (116), dans lequel lorsque la planification par le second noyau détermine d'exécuter un VP planifié donné, le second noyau identifie le VP planifié donné à l'hyperviseur et sur la base de ceci le premier noyau (130) réalise une commutation de contexte de processeur sur un PP pour le VP planifié donné, la commutation de contexte de processeur réalisée en utilisant l'état de processeur physique correspondant au VP planifié donné identifié.


 
2. Procédé selon la revendication 1, dans lequel le premier noyau (130) met en oeuvre un hyperappel, et dans lequel lorsque le second noyau (136) détermine d'exécuter le VP planifié donné, le second noyau invoque l'hyperappel pour identifier le VP donné au premier noyau.
 
3. Procédé selon la revendication 2, dans lequel le premier noyau maintient des états de processeurs physiques (149) pour les VP planifiés respectifs, dans lequel l'hyperappel est invoqué avec un identifiant du VP planifié donné, et sur la base de ceci le premier noyau réalise la commutation de contexte de processeur.
 
4. Procédé selon une quelconque revendication précédente 1, dans lequel ni l'hyperviseur ni le premier noyau ne planifie l'exécution des VP.
 
5. Procédé selon une quelconque revendication précédente, comprenant en outre l'exécution d'un système d'exploitation en tant qu'invité dans l'une des VM, le système d'exploitation comprenant le second noyau.
 
6. Dispositif informatique (100) comprenant :

un processeur, CPU, (106) comprenant des processeurs physiques, PP, (134) ;

un dispositif de stockage matériel (226) stockant un hyperviseur comprenant un noyau (130) d'hyperviseur et un système d'exploitation (118), OS, comprenant un noyau (136) d'OS, le noyau d'OS configuré pour tourner dans une machine virtuelle (116), VM, de planification exécutée par un processeur virtuel, VP, de planification correspondant,

le noyau (130) d'hyperviseur configuré pour :

maintenir des états de processeurs physiques (149) de processeurs virtuels, VP, (132) planifiés respectifs pour des machines virtuelles, VM, (114) qui ne sont pas les VM de planification, dans lequel le VP de planification n'est pas l'un des VP planifiés ;

le noyau (136) d'OS configuré pour prendre des décisions de planification pour planifier l'exécution des VP (132) planifiés où des opérations pour mettre en oeuvre les décisions de planification sont réalisées par le noyau de l'hyperviseur ; et

dans lequel, chaque fois que le noyau (136) d'OS prend une décision de planification pour qu'un VP planifié donné soit exécuté sur un PP, le noyau (136) d'OS effectue un appel correspondant pour identifier le VP planifié donné à l'hyperviseur pour amener le noyau (130) d'hyperviseur à réaliser une commutation de contexte pour commencer à exécuter le VP planifié sur le PP en utilisant l'état de processeur physique correspondant au VP planifié donné identifié.


 
7. Dispositif informatique selon la revendication 6, dans lequel le noyau du système d'exploitation est configuré pour tourner à l'intérieur de la VM (116) de planification, dans lequel les représentations comprennent des fils respectifs du noyau du système d'exploitation, et dans lequel le noyau de l'hyperviseur exécute un VP planifié en exécutant un fil correspondant sur un PP.
 
8. Dispositif informatique selon la revendication 6 ou la revendication 7, dans lequel le noyau du système d'exploitation est configuré pour commander quels VP planifiés seront exécutés sur quels PP, et dans lequel le noyau de l'hyperviseur est configuré pour, chaque fois que le noyau du système d'exploitation émet un appel pour exécuter un VP planifié, exécuter le VP planifié correspondant sur un PP.
 
9. Dispositif informatique selon l'une quelconque des revendications 6 à 8, dans lequel le noyau du système d'exploitation est configuré pour commander quels VP planifiés seront exécutés sur quels PP, et dans lequel le noyau de l'hyperviseur est configuré pour, chaque fois que le noyau du système d'exploitation émet un appel pour exécuter un VP planifié, exécuter le VP planifié correspondant sur un PP.
 
10. Dispositif informatique selon l'une quelconque des revendications 6 à 9, dans lequel le noyau de l'hyperviseur est configuré pour traiter une interruption d'exécution d'un VP planifié sur un PP en renvoyant une commande correspondante au noyau du système d'exploitation.
 
11. Dispositif informatique selon l'une quelconque des revendications 6 à 10, dans lequel le noyau de l'hyperviseur n'est pas configuré pour planifier une d'exécution de VP sur des PP.
 
12. Dispositif informatique de stockage matériel (226) stockant des informations pour permettre à un dispositif informatique de réaliser un procédé selon l'une quelconque des revendications 1 à 5.
 




Drawing





























Cited references

REFERENCES CITED IN THE DESCRIPTION



This list of references cited by the applicant is for the reader's convenience only. It does not form part of the European patent document. Even though great care has been taken in compiling the references, errors or omissions cannot be excluded and the EPO disclaims all liability in this regard.

Patent documents cited in the description