Recently, I had a problem unique to my test machine. Attempting to call CoCreateInstance on a COM class provided by one of our DLLs resulted in HRESULT 0x80040154, which corresponds to “Class not registered”.
This was a mystery since regsvr32 appeared to properly register our DLL, and all the obvious registry keys were intact!
After a bit of web searching, I experimented with using CLSCTX_ALL instead of CLSCTX_INPROC_SERVER. Now, the call succeeded! But why, since this is a DLL and, therefore, should be considered an in-process COM server?
MSDN documentation states that CLSCTX_ALL is defined as the combination of CLSCTX_INPROC_SERVER, CLSCTX_INPROC_HANDLER, and CLSCTX_LOCAL_SERVER. I began substituting each one in turn and found that the key was CLSCTX_LOCAL_SERVER.
Stepping through the code in the debugger revealed that the instant the CoCreateInstance call was made, a new instance of dllhost.exe appeared in the process list. This explained why CLSCTX_INPROC_SERVER would fail, since strictly speaking, the DLL was being hosted in dllhost’s process space.
Ultimately, the problem turned out to be an artifact from an old COM+ experiment. There was a COM+ entry corresponding to our COM server in the Component Services panel (Control Panel | Administrative Tools | Component Services, under Component Services | Computers | My Computer | COM+ Applications). The COM+ entry dictated that the COM server be hosted by proxy by dllhost.exe. Deleting this COM+ entry and re-running regsvr32 fixed the problem.
