After running fine since installation about a month ago, vmmservice.exe stared to fail very quickly after starting. Enabled debug and got this in the log:
[0]0C74.0FE0::2013-05-22 22:02:46.410 [Microsoft-VirtualMachineManager-Debug]26,1,VmRefresher.cs,1899,System.NullReferenceException: Object reference not set to an instance of an object. at Microsoft.VirtualManager.Engine.WorkloadCreation.COMPort..ctor(ICOMPortContainerInternal
parent, Byte portNumber) at Microsoft.VirtualManager.Engine.ImageLibrary.VMBase.ReadChildren(Boolean ignoreErrorLoadingChildren) at Microsoft.VirtualManager.Engine.ImageLibrary.VM.ReadChildren(Boolean ignoreErrorLoadingChildren)
at Microsoft.VirtualManager.Engine.ImageLibrary.VM..ctor(VMData objData, CarmineObjectLock objLock, Boolean loadShallow, Boolean cloneData, Boolean ignoreErrorLoadingChildren) at Microsoft.VirtualManager.Engine.ImageLibrary.LibraryUtils.CreateLibObject(LibObjectData
objData, CarmineObjectLock objLock, Boolean loadShallow, Boolean ignoreErrorLoadingChildren) at Microsoft.VirtualManager.Engine.ImageLibrary.LibraryUtils.GetLibObject(Guid id, CarmineObjectType objType, Guid taskID, CarmineObjectLockType
lockType, Boolean loadShallow, Boolean ignoreErrorLoadingChildren) at Microsoft.VirtualManager.Engine.ImageLibrary.VM.Get(Guid id, Guid taskID, CarmineObjectLockType lockType, Boolean shallowLoad, Boolean ignoreErrorLoadingChildren)
at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.AcquireVMLock(CarmineObjectLockType lockType, VMLightData vmLightData),{00000000-0000-0000-0000-000000000000}
[0]0C74.0FE0::2013-05-22 22:02:46.410 [Microsoft-VirtualMachineManager-Debug]4,2,LockableObjectCacheBase.cs,406, Object 146da0a4-c87b-424b-af27-2339c5957c1b:VMHost (NoDelete) removed from the cache @ Refresher:Refresh virtual machine properties,{00000000-0000-0000-0000-000000000000}
[0]0C74.0FE0::2013-05-22 22:02:46.410 [Microsoft-VirtualMachineManager-Debug]4,4,CarmineObjectLock.cs,179,CarmineObjectLock, Task 0a7c77ce-a38c-43af-88cc-6ddd6416c726(Refresher:Refresh virtual machine properties) Releasing NoDelete lock on VMHost:146da0a4-c87b-424b-af27-2339c5957c1b,{0a7c77ce-a38c-43af-88cc-6ddd6416c726}
[1]0C74.0FE0::2013-05-22 22:02:46.411 [Microsoft-VirtualMachineManager-Debug]26,1,VmRefresher.cs,234,VM Light Refresher caught an unexpected exception and will crash the engine. Host clusterhost1.lab.no,{00000000-0000-0000-0000-000000000000}
[1]0C74.0FE0::2013-05-22 22:02:46.412 [Microsoft-VirtualMachineManager-Debug]26,1,VmRefresher.cs,234,System.NullReferenceException: Object reference not set to an instance of an object. at Microsoft.VirtualManager.Engine.WorkloadCreation.COMPort..ctor(ICOMPortContainerInternal
parent, Byte portNumber) at Microsoft.VirtualManager.Engine.ImageLibrary.VMBase.ReadChildren(Boolean ignoreErrorLoadingChildren) at Microsoft.VirtualManager.Engine.ImageLibrary.VM.ReadChildren(Boolean ignoreErrorLoadingChildren)
at Microsoft.VirtualManager.Engine.ImageLibrary.VM..ctor(VMData objData, CarmineObjectLock objLock, Boolean loadShallow, Boolean cloneData, Boolean ignoreErrorLoadingChildren) at Microsoft.VirtualManager.Engine.ImageLibrary.LibraryUtils.CreateLibObject(LibObjectData
objData, CarmineObjectLock objLock, Boolean loadShallow, Boolean ignoreErrorLoadingChildren) at Microsoft.VirtualManager.Engine.ImageLibrary.LibraryUtils.GetLibObject(Guid id, CarmineObjectType objType, Guid taskID, CarmineObjectLockType
lockType, Boolean loadShallow, Boolean ignoreErrorLoadingChildren) at Microsoft.VirtualManager.Engine.ImageLibrary.VM.Get(Guid id, Guid taskID, CarmineObjectLockType lockType, Boolean shallowLoad, Boolean ignoreErrorLoadingChildren)
at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.AcquireVMLock(CarmineObjectLockType lockType, VMLightData vmLightData) at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.CheckIfNeedLock(VMLightData vmLightData)
at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.UpdateVMSummary(IVMComputerSystemSummary vmComputer, VMLightData vmData, Boolean updateStateOnly, UpdateRequired updateRequired, Boolean vmObjectHasChanged) at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.UpdateVMs(List`1
toUpdate) at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.RunLightRefresher() at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.UpdateHostAndVMs(VMRefresherType refresherType, Guid vmObjectId,
VM tempVm) at Microsoft.VirtualManager.Engine.BitBos.VmLightRefresher.RefreshData(HostReference hostRef),{00000000-0000-0000-0000-000000000000}
[0]0C74.0FE0::2013-05-22 22:02:46.414 [Microsoft-VirtualMachineManager-Debug]19,2,Auditor.cs,273,Auditor: Discarding audit record changes for task.,{0a7c77ce-a38c-43af-88cc-6ddd6416c726}
[0]0C74.0FE0::2013-05-22 22:02:46.415 [Microsoft-VirtualMachineManager-Debug]4,4,WsmanAPIWrapper.cs,1199,WsmanAPIWrapper:UseWSManCaching; set to [False] for thread: [10],{00000000-0000-0000-0000-000000000000}
[0]0C74.0FE0::2013-05-22 22:02:46.415 [Microsoft-VirtualMachineManager-Debug]19,2,RefresherSubtask.cs,150,RefresherSubtask: MarkCompleted called for task LightVmRefresher with error Success (0) and IsTaskPersisted False,{0a7c77ce-a38c-43af-88cc-6ddd6416c726}
[0]0C74.0FE0::2013-05-22 22:02:46.415 [Microsoft-VirtualMachineManager-Debug]4,4,CarmineObjectLock.cs,420,Releasing all locks for task 0a7c77ce-a38c-43af-88cc-6ddd6416c726,{0a7c77ce-a38c-43af-88cc-6ddd6416c726}
[0]0C74.0FE0::2013-05-22 22:02:46.416 [Microsoft-VirtualMachineManager-Debug]4,4,CarmineObjectLock.cs,749,CarmineObjectLock - Release All, Task 0a7c77ce-a38c-43af-88cc-6ddd6416c726(Refresher:Refresh virtual machine properties) Releasing Delete lock on
VirtualNetworkAdapter:db255f6d-8df8-41f4-8deb-09835437a7ca,{00000000-0000-0000-0000-000000000000}
[0]0C74.0FE0::2013-05-22 22:02:46.416 [Microsoft-VirtualMachineManager-Debug]4,4,CarmineObjectLock.cs,749,CarmineObjectLock - Release All, Task 0a7c77ce-a38c-43af-88cc-6ddd6416c726(Refresher:Refresh virtual machine properties) Releasing NoDelete lock on
UserRole:75700cd5-893e-4f68-ada7-50ef4668acc6,{00000000-0000-0000-0000-000000000000}
[0]0C74.0FE0::2013-05-22 22:02:46.416 [Microsoft-VirtualMachineManager-Debug]4,4,CarmineObjectLock.cs,749,CarmineObjectLock - Release All, Task 0a7c77ce-a38c-43af-88cc-6ddd6416c726(Refresher:Refresh virtual machine properties) Releasing NoDelete lock on
UserRole:75700cd5-893e-4f68-ada7-50ef4668acc6,{00000000-0000-0000-0000-000000000000}
[0]0C74.0FE0::2013-05-22 22:02:46.416 [Microsoft-VirtualMachineManager-Debug]4,4,CarmineObjectLock.cs,749,CarmineObjectLock - Release All, Task 0a7c77ce-a38c-43af-88cc-6ddd6416c726(Refresher:Refresh virtual machine properties) Releasing Write lock on VM:e3954892-b71f-4e5d-aa67-7bfd775a5974,{00000000-0000-0000-0000-000000000000}
[0]0C74.0FE0::2013-05-22 22:02:46.416 [Microsoft-VirtualMachineManager-Debug]4,4,CarmineObjectLock.cs,749,CarmineObjectLock - Release All, Task 0a7c77ce-a38c-43af-88cc-6ddd6416c726(Refresher:Refresh virtual machine properties) Releasing Delete lock on
VirtualNetworkAdapter:7a1ad55e-16c3-4f0a-a476-b9ae1041e868,{00000000-0000-0000-0000-000000000000}
[1]0C74.0FE0::2013-05-22 22:02:46.439 [Microsoft-VirtualMachineManager-Debug]4,1,WatsonExceptionReport.cs,756,Unhandled exception caught.,{00000000-0000-0000-0000-000000000000}
[1]0C74.0FE0::2013-05-22 22:02:46.439 [Microsoft-VirtualMachineManager-Debug]4,1,WatsonExceptionReport.cs,757,Unhandled exception.,{00000000-0000-0000-0000-000000000000}
[1]0C74.0FE0::2013-05-22 22:02:46.440 [Microsoft-VirtualMachineManager-Debug]4,1,WatsonExceptionReport.cs,757,System.NullReferenceException: Object reference not set to an instance of an object. at Microsoft.VirtualManager.Engine.WorkloadCreation.COMPort..ctor(ICOMPortContainerInternal
parent, Byte portNumber) at Microsoft.VirtualManager.Engine.ImageLibrary.VMBase.ReadChildren(Boolean ignoreErrorLoadingChildren) at Microsoft.VirtualManager.Engine.ImageLibrary.VM.ReadChildren(Boolean ignoreErrorLoadingChildren)
at Microsoft.VirtualManager.Engine.ImageLibrary.VM..ctor(VMData objData, CarmineObjectLock objLock, Boolean loadShallow, Boolean cloneData, Boolean ignoreErrorLoadingChildren) at Microsoft.VirtualManager.Engine.ImageLibrary.LibraryUtils.CreateLibObject(LibObjectData
objData, CarmineObjectLock objLock, Boolean loadShallow, Boolean ignoreErrorLoadingChildren) at Microsoft.VirtualManager.Engine.ImageLibrary.LibraryUtils.GetLibObject(Guid id, CarmineObjectType objType, Guid taskID, CarmineObjectLockType
lockType, Boolean loadShallow, Boolean ignoreErrorLoadingChildren) at Microsoft.VirtualManager.Engine.ImageLibrary.VM.Get(Guid id, Guid taskID, CarmineObjectLockType lockType, Boolean shallowLoad, Boolean ignoreErrorLoadingChildren)
at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.AcquireVMLock(CarmineObjectLockType lockType, VMLightData vmLightData) at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.CheckIfNeedLock(VMLightData vmLightData)
at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.UpdateVMSummary(IVMComputerSystemSummary vmComputer, VMLightData vmData, Boolean updateStateOnly, UpdateRequired updateRequired, Boolean vmObjectHasChanged) at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.UpdateVMs(List`1
toUpdate) at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.RunLightRefresher() at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.UpdateHostAndVMs(VMRefresherType refresherType, Guid vmObjectId,
VM tempVm) at Microsoft.VirtualManager.Engine.BitBos.VmLightRefresher.RefreshData(HostReference hostRef) at Microsoft.VirtualManager.Engine.RefreshDriver`1.RefreshThreadFunction(Object obj),{00000000-0000-0000-0000-000000000000}
[0]0C74.0FE0::2013-05-22 22:02:46.454 [Microsoft-VirtualMachineManager-Debug]4,4,FileUtil.cs,435,Creating directory C:\Users\vmm-svc\AppData\Local\Temp\SCVMM.c9b4561e-0ce5-46f0-8a99-e5fdc9d79fdb,{00000000-0000-0000-0000-000000000000}
[0]0C74.0FE0::2013-05-22 22:02:46.556 [Microsoft-VirtualMachineManager-Debug]4,4,FileUtil.cs,497,Deleting directory C:\Users\vmm-svc\AppData\Local\Temp\SCVMM.c9b4561e-0ce5-46f0-8a99-e5fdc9d79fdb,{00000000-0000-0000-0000-000000000000}
[0]0C74.0FE0::2013-05-22 22:02:46.566 [Microsoft-VirtualMachineManager-Debug]4,1,VirtualManagerService.cs,759,Unhandled exception received by service. Process will terminate.,{00000000-0000-0000-0000-000000000000}
[0]0C74.0FE0::2013-05-22 22:02:46.567 [Microsoft-VirtualMachineManager-Debug]4,1,VirtualManagerService.cs,759,System.NullReferenceException: Object reference not set to an instance of an object. at Microsoft.VirtualManager.Engine.WorkloadCreation.COMPort..ctor(ICOMPortContainerInternal
parent, Byte portNumber) at Microsoft.VirtualManager.Engine.ImageLibrary.VMBase.ReadChildren(Boolean ignoreErrorLoadingChildren) at Microsoft.VirtualManager.Engine.ImageLibrary.VM.ReadChildren(Boolean ignoreErrorLoadingChildren)
at Microsoft.VirtualManager.Engine.ImageLibrary.VM..ctor(VMData objData, CarmineObjectLock objLock, Boolean loadShallow, Boolean cloneData, Boolean ignoreErrorLoadingChildren) at Microsoft.VirtualManager.Engine.ImageLibrary.LibraryUtils.CreateLibObject(LibObjectData
objData, CarmineObjectLock objLock, Boolean loadShallow, Boolean ignoreErrorLoadingChildren) at Microsoft.VirtualManager.Engine.ImageLibrary.LibraryUtils.GetLibObject(Guid id, CarmineObjectType objType, Guid taskID, CarmineObjectLockType
lockType, Boolean loadShallow, Boolean ignoreErrorLoadingChildren) at Microsoft.VirtualManager.Engine.ImageLibrary.VM.Get(Guid id, Guid taskID, CarmineObjectLockType lockType, Boolean shallowLoad, Boolean ignoreErrorLoadingChildren)
at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.AcquireVMLock(CarmineObjectLockType lockType, VMLightData vmLightData) at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.CheckIfNeedLock(VMLightData vmLightData)
at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.UpdateVMSummary(IVMComputerSystemSummary vmComputer, VMLightData vmData, Boolean updateStateOnly, UpdateRequired updateRequired, Boolean vmObjectHasChanged) at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.UpdateVMs(List`1
toUpdate) at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.RunLightRefresher() at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.UpdateHostAndVMs(VMRefresherType refresherType, Guid vmObjectId,
VM tempVm) at Microsoft.VirtualManager.Engine.BitBos.VmLightRefresher.RefreshData(HostReference hostRef) at Microsoft.VirtualManager.Engine.RefreshDriver`1.RefreshThreadFunction(Object obj)
at Microsoft.VirtualManager.Utils.Multithreading.ThrottledThreadPool.<>c__DisplayClass3.<ScheduleNextThread>b__2(Object dummy) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback
callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch(),{00000000-0000-0000-0000-000000000000}
[0]0C74.0FE0::2013-05-22 22:02:46.567 [Microsoft-VirtualMachineManager-Debug]4,1,VirtualManagerService.cs,765,Unhandled exception.,{00000000-0000-0000-0000-000000000000}
[0]0C74.0FE0::2013-05-22 22:02:46.567 [Microsoft-VirtualMachineManager-Debug]4,1,VirtualManagerService.cs,765,System.NullReferenceException: Object reference not set to an instance of an object. at Microsoft.VirtualManager.Engine.WorkloadCreation.COMPort..ctor(ICOMPortContainerInternal
parent, Byte portNumber) at Microsoft.VirtualManager.Engine.ImageLibrary.VMBase.ReadChildren(Boolean ignoreErrorLoadingChildren) at Microsoft.VirtualManager.Engine.ImageLibrary.VM.ReadChildren(Boolean ignoreErrorLoadingChildren)
at Microsoft.VirtualManager.Engine.ImageLibrary.VM..ctor(VMData objData, CarmineObjectLock objLock, Boolean loadShallow, Boolean cloneData, Boolean ignoreErrorLoadingChildren) at Microsoft.VirtualManager.Engine.ImageLibrary.LibraryUtils.CreateLibObject(LibObjectData
objData, CarmineObjectLock objLock, Boolean loadShallow, Boolean ignoreErrorLoadingChildren) at Microsoft.VirtualManager.Engine.ImageLibrary.LibraryUtils.GetLibObject(Guid id, CarmineObjectType objType, Guid taskID, CarmineObjectLockType
lockType, Boolean loadShallow, Boolean ignoreErrorLoadingChildren) at Microsoft.VirtualManager.Engine.ImageLibrary.VM.Get(Guid id, Guid taskID, CarmineObjectLockType lockType, Boolean shallowLoad, Boolean ignoreErrorLoadingChildren)
at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.AcquireVMLock(CarmineObjectLockType lockType, VMLightData vmLightData) at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.CheckIfNeedLock(VMLightData vmLightData)
at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.UpdateVMSummary(IVMComputerSystemSummary vmComputer, VMLightData vmData, Boolean updateStateOnly, UpdateRequired updateRequired, Boolean vmObjectHasChanged) at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.UpdateVMs(List`1
toUpdate) at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.RunLightRefresher() at Microsoft.VirtualManager.Engine.BitBos.VMRefresherBase.UpdateHostAndVMs(VMRefresherType refresherType, Guid vmObjectId,
VM tempVm) at Microsoft.VirtualManager.Engine.BitBos.VmLightRefresher.RefreshData(HostReference hostRef) at Microsoft.VirtualManager.Engine.RefreshDriver`1.RefreshThreadFunction(Object obj)
at Microsoft.VirtualManager.Utils.Multithreading.ThrottledThreadPool.<>c__DisplayClass3.<ScheduleNextThread>b__2(Object dummy) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback
callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch(),{00000000-0000-0000-0000-000000000000}
[0]0C74.0FE0::2013-05-22 22:02:46.567 [Microsoft-VirtualMachineManager-Debug]Unhandled exception in the VMM service.
[0]0C74.0FE0::2013-05-22 22:02:46.569 [Microsoft-VirtualMachineManager-Debug]4,1,VirtualManagerService.cs,827,Attempting to abort Indigo service host and release open communication ports.,{00000000-0000-0000-0000-000000000000}
[0]0C74.0FE0::2013-05-22 22:02:46.569 [Microsoft-VirtualMachineManager-Debug]4,1,VirtualManagerService.cs,831,Ready to abort the indigo service host and release open communication ports.,{00000000-0000-0000-0000-000000000000}
[0]0C74.0FE0::2013-05-22 22:02:46.569 [Microsoft-VirtualMachineManager-Debug]25,2,SQMSession.cs,92,Getting Instance for session Engine,{00000000-0000-0000-0000-000000000000}
[0]0C74.0FE0::2013-05-22 22:02:46.569 [Microsoft-VirtualMachineManager-Debug]25,2,SQMSession.cs,96,Found session Engine,{00000000-0000-0000-0000-000000000000}
[0]0C74.0FE0::2013-05-22 22:02:46.570 [Microsoft-VirtualMachineManager-Debug]25,2,SQMSession.cs,290,Ending session with release handleEngine,{00000000-0000-0000-0000-000000000000}
[0]0C74.0FE0::2013-05-22 22:02:46.572 [Microsoft-VirtualMachineManager-Debug]25,2,SQMSession.cs,130,Removing session Engine,{00000000-0000-0000-0000-000000000000}
[0]0C74.0FE0::2013-05-22 22:02:46.573 [Microsoft-VirtualMachineManager-Debug]4,2,ServiceHost.cs,103,Entering ServiceHost.Abort(),{00000000-0000-0000-0000-000000000000}
[0]0C74.0FE0::2013-05-22 22:02:46.573 [Microsoft-VirtualMachineManager-Debug]4,2,ServiceHost.cs,107,Aborting service...,{00000000-0000-0000-0000-000000000000}
[1]097C.0FF8::2013-05-22 22:02:46.580 [Microsoft-VirtualMachineManager-Debug]18,2,ServerConnection.cs,1121,Connection ID uuid:3d77a412-aa2b-4673-bc7c-4ad55b9f742f;id=1 faulted to carmine server scvmm.lab.no,{00000000-0000-0000-0000-000000000000}
[1]097C.0FF8::2013-05-22 22:02:46.581 [Microsoft-VirtualMachineManager-Debug]18,2,ServerConnection.cs,1136,Connection ID uuid:3d77a412-aa2b-4673-bc7c-4ad55b9f742f;id=1 closed to carmine server scvmm.lab.no,{00000000-0000-0000-0000-000000000000}
[0]0C74.0FE0::2013-05-22 22:02:46.581 [Microsoft-VirtualMachineManager-Debug]18,2,ClientConnection.cs,110,Closing client connection: uuid:855874c4-3e06-4204-b1a3-33cd22efc49e;id=1,{00000000-0000-0000-0000-000000000000}
Hostnames are changed
This is SCVMM 2012 SP1 RU2 running on a Windows 2012 Server