API Server cannot establish connection to the database

Hi experts,

since few days I am experiencing issues with the API Server. It is apparently not able to establish connection to the database. It had previously worked seamlessly since the installation (about 1 year ago) and there were no intenational changes on the API itself. There were modifications in the dynamic module qer-app-portal upon the day, as the issue appeared. I wonder if there would be any correlation to the error...

When i call .../ApiServer/imx/ping in the browse, i get this response:

{"NoDbConnection":true,"HasPluginErrors":false,"IsWaitingToStartUpdate":false,"PluginsWithErrors":[]}

Logs on the server are saying this:

--- System.Exception: API could not be loaded. The API server will be running in connectionless mode. ---> System.Exception: Could not load API from provider: API from ScriptAssembly
at QBM.CompositionApi.ApiManager.DbMethodProvider.<GetBranchedMethodSetsAsync>d__12.MoveNext()
--- End of inner exception stack trace ---
 ERROR ( WebLog ) : An error occurred while processing the request: GET http://<host>/ApiServer/imx/entityschema System.Exception: An error occurred while processing the request: GET http://<host>/ApiServer/imx/entityschema ---> System.Exception: Error while building schema for imx/multilanguage/uicultures ---> System.NullReferenceException: Object reference not set to an instance of an object.
at VI.DB.Entities.SessionExtensions.MultiLanguage(ISession session)

>>>> I am using Version 8.2.1 <<<<<

I have already spent multiple hours on debugging. No results. 

I would really appreciate any suggestions or help.

Cheers

Toni

  • Hi Toni,

    Is there any error message in the logs before that? I've seen this one as a follow-up problem when the service connection to the database cannot be set up.

  • Hi Hanno,

    thanks for the answer!

    The service connection looks indeed suspicious. This is what appears in the log file after restarting the IIS Server:

    2024-01-24 13:44:00.7849 INFO ( WebLog ) : Ending API server process, goodbye.
    2024-01-24 13:44:30.7875 INFO ( WebLog ) : Ending API server process, goodbye.
    2024-01-24 13:44:30.8031 INFO ( WebLog ) : The application is shutting down, reason: HostingEnvironment
    2024-01-24 13:45:47.0691 INFO ( WebLog ) : Creating performance counters in category One Identity Manager API Server, instance _LM_W3SVC_1_ROOT_ApiServer
    2024-01-24 13:45:47.4910 INFO ( WebLog ) : Could not create performance counters: Access to the registry key 'Global' is denied.
    2024-01-24 13:45:47.4910 INFO ( WebLog ) : Updating performance counter values
    2024-01-24 13:45:47.4910 INFO ( WebLog ) : Performance counters initialized in category One Identity Manager API Server, instance _LM_W3SVC_1_ROOT_ApiServer
    2024-01-24 13:45:47.6004 INFO ( WebLog ) : Starting plugin: QBM.CompositionApi.PlugIns.ServiceUnavailablePlugIn
    2024-01-24 13:45:47.7412 INFO ( WebLog ) : Starting plugin: QBM.CompositionApi.PlugIns.UpdateCheckPlugIn
    2024-01-24 13:45:47.7412 INFO ( WebLog ) : Application set-up start
    2024-01-24 13:45:47.7412 INFO ( WebLog ) : Account: IIS APPPOOL\ApiServer_POOL
    2024-01-24 13:45:47.7412 INFO ( WebLog ) : Culture: en-US
    2024-01-24 13:45:47.8347 INFO ( WebLog ) : Loaded assemblies:
    ...
    2024-01-24 13:45:47.8347 INFO ( WebLog ) : Application set-up end
    2024-01-24 13:45:47.8505 INFO ( WebLog ) : Loading assembly: System.Security, Version=4.0.0.0, Culture=neutral
    2024-01-24 13:45:47.9597 INFO ( WebLog ) : Loading assembly: System.Transactions, Version=4.0.0.0, Culture=neutral
    2024-01-24 13:45:49.0847 INFO ( ObjectLog ) : User WebAppService_<GUID> (Dialog user: WebAppService_<GUID>, X fields: WebAppService_<GUID>) authenticated.
    2024-01-24 13:45:49.1159 INFO ( WebLog ) : New service connection requested
    2024-01-24 13:45:49.1472 INFO ( WebLog ) : Updater status on application start: NoUpdatesAvailable
    2024-01-24 13:45:49.1472 INFO ( WebLog ) : Starting plugin: QBM.CompositionApi.PlugIns.ReportConfigurationPlugIn
    ...
    024-01-24 13:45:50.0846 INFO ( WebLog ) : FileLoader: Loading stream from "C:\inetpub\wwwroot\ApiServer\bin\imxweb\Html_uci.zip"
    2024-01-24 13:45:50.1315 INFO ( WebLog ) : Decompressing plugin uci to C:\inetpub\wwwroot\ApiServer\App_Data\WebCache\html_zldpdjo1.qe4
    2024-01-24 13:45:50.1472 INFO ( WebLog ) : Starting plugin: QBM.CompositionApi.ApiManager.RouteActivatorPlugIn
    2024-01-24 13:45:50.1472 INFO ( WebLog ) : Loading assembly: System.Net.Http, Version=4.0.0.0, Culture=neutral
    2024-01-24 13:45:50.1629 INFO ( WebLog ) : Loading assembly: SMDiagnostics, Version=4.0.0.0, Culture=neutral
    2024-01-24 13:45:50.4130 INFO ( ObjectLog ) : User WebAppService_<GUID> (Dialog user: WebAppService_<GUID>, X fields: WebAppService_<GUID>) authenticated.
    2024-01-24 13:45:50.4130 INFO ( WebLog ) : New service connection requested
    2024-01-24 13:45:50.4597 INFO (VI.DB.Meta ) : Loaded configuration from database for 'http://<host>/ApiServer/'
    2024-01-24 13:45:50.4753 INFO (VI.DB.Meta ) : Loaded configuration from database for 'http://<host>/ApiServer/'
    2024-01-24 13:45:50.5534 WARN ( WebLog ) : System.Exception: API could not be loaded. The API server will be running in connectionless mode. ---> System.Exception: Could not load API from provider: API from ScriptAssembly
    at QBM.CompositionApi.ApiManager.DbMethodProvider.<GetBranchedMethodSetsAsync>d__12.MoveNext()
    --- End of inner exception stack trace --- System.Exception: API could not be loaded. The API server will be running in connectionless mode. ---> System.Exception: Could not load API from provider: API from ScriptAssembly
    at QBM.CompositionApi.ApiManager.DbMethodProvider.<GetBranchedMethodSetsAsync>d__12.MoveNext()
    --- End of inner exception stack trace ---
    2024-01-24 13:45:50.7096 INFO ( WebLog ) : API Server state is now: Running

    The Service user WebAppService_<GUID> is active and without changes since the installation, as I can see in the DialogUser table. I masked the guid in this post.

    I can also ping the DB Server from the Web Server.

    Do you have any further suggestions? 

    There is always an emergency exit: reinstall the API Server. But I would really like to understand what is this error about and avoid installing it again... 

    Thank you for your time. 

    Regards

    Toni

  • After I was unable to find a solution to the database connection problem, I uninstalled the API Server and installed it agiain with same configurations.

    Unfortunately; all I get is an antother exception:

    2024-01-29 12:48:19.9235 INFO ( WebLog ) : Application set-up end
    2024-01-29 12:48:19.9397 INFO ( WebLog ) : Loading assembly: System.Security, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a from C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Security\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Security.dll, file version: 4.7.3701.0 built by: NET472REL1LAST_B
    2024-01-29 12:48:19.9397 WARN ( WebLog ) : Updater status could not be determined on application start. System.Configuration.ConfigurationErrorsException: Failed to decrypt using provider 'RsaProtectedConfigurationProvider'. Error message from the provider: The parameter is incorrect.
    (C:\inetpub\wwwroot\ApiServerNew\web.config line 10) ---> System.Security.Cryptography.CryptographicException: The parameter is incorrect.

    at System.Security.Cryptography.CryptographicException.ThrowCryptographicException(Int32 hr)
    at System.Security.Cryptography.RSACryptoServiceProvider.DecryptKey(SafeKeyHandle pKeyContext, Byte[] pbEncryptedKey, Int32 cbEncryptedKey, Boolean fOAEP, ObjectHandleOnStack ohRetDecryptedKey)
    at System.Security.Cryptography.RSACryptoServiceProvider.Decrypt(Byte[] rgb, Boolean fOAEP)
    at System.Security.Cryptography.RSACryptoServiceProvider.Decrypt(Byte[] data, RSAEncryptionPadding padding)
    at System.Security.Cryptography.RSAPKCS1KeyExchangeDeformatter.DecryptKeyExchange(Byte[] rgbIn)
    at System.Security.Cryptography.Xml.EncryptedXml.GetDecryptionKey(EncryptedData encryptedData, String symmetricAlgorithmUri)
    at System.Security.Cryptography.Xml.EncryptedXml.DecryptDocument()
    at System.Configuration.RsaProtectedConfigurationProvider.Decrypt(XmlNode encryptedNode)
    at System.Configuration.ProtectedConfigurationSection.DecryptSection(String encryptedXml, ProtectedConfigurationProvider provider)
    at System.Configuration.BaseConfigurationRecord.DecryptConfigSection(ConfigXmlReader reader, ProtectedConfigurationProvider protectionProvider)
    --- End of inner exception stack trace ---

    ...

    2024-01-29 12:48:20.8354 INFO ( WebLog ) : Starting plugin: QBM.CompositionApi.ApiManager.RouteActivatorPlugIn
    2024-01-29 12:48:20.8447 INFO ( WebLog ) : Loading assembly: System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a from C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Net.Http\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Net.Http.dll, file version: 4.7.3190.0 built by: NET472REL1LAST_C
    2024-01-29 12:48:20.8447 INFO ( WebLog ) : Loading assembly: SMDiagnostics, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 from C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\SMDiagnostics\v4.0_4.0.0.0__b77a5c561934e089\SMDiagnostics.dll, file version: 4.7.3620.0 built by: NET472REL1LAST_B
    2024-01-29 12:48:20.8823 WARN ( WebLog ) : System.Exception: API could not be loaded. The API server will be running in connectionless mode. ---> System.Configuration.ConfigurationErrorsException: Failed to decrypt using provider 'RsaProtectedConfigurationProvider'. Error message from the provider: The parameter is incorrect.
    (C:\inetpub\wwwroot\ApiServerNew\web.config line 10) ---> System.Security.Cryptography.CryptographicException: The parameter is incorrect.

    This is what the log is pointing to (web.config line 10):

     <EncryptedData Type="">www.w3.org/.../xmlenc

    I find it strange to get the decryption error. The API Server is installed on the same machine, with the same certificate and with the same dialog user for SQL authentification. I am really kind of desperate right now...

    Maybe any other suggestions or troubleshooting tips?

    Thanks in advance,

    Toni

  • Hello Toni,

    Unfortunately certificate errors thrown by the ASP.NET crypto API are often not very descriptive -- all that it tells us is that it does not accept the certificate.

    Here are two things to try to get to the root cause:

    - Check the ACLs on any certificate you are using. The web application is running under a dedicated user account (usually the application pool identity) and that user needs to have read permission on the certificate.

    - Decrypt and re-encrypt the web.config. Using an elevated command prompt you can do this as described here. stackoverflow.com/.../30400269

  • Hi Hanno,

    thank for getting back to me!

    Your second hint helped me to solve the issue. Decrypting with aspnet_regiis.exe -pd "connectionStrings" -app "/ApiServer" failed with the same error message as in 1IM Logs. But I cheated a little bit. I built the connection stiring in web.config by myself. I let it there unencrypted (psst, don't tell anyone). For a testing environment it should be ok. If someone would be reading this post - DON'T DO THIS IN A PRODUCTIVE ENVIRONMENT!

    So far so good. Unfortunately, after all the troubles I am getting the original error:

    System.Exception: API could not be loaded. The API server will be running in connectionless mode. ---> System.Exception: Could not load API from provider: API from ScriptAssembly

    How could I possibly recover (altough it wasn't modified) or fix the API ScriptAssembly?

    I have installed the AppServer and ApiServer on a another machine and the same error occurs.

  • Hi Toni,


    To clear that error, run the Database Compiler to compile the API assembly.

  • I had done that, of course. It didn't help.

    But I am not planning on giving up. I shall post the solution here when I find one. 

    Hanno, thank you for your time!

  • The error dissapered after upgrading to Version 9.0.  It's not a real solution, though it helped me to get over that hurdle.