error when requesting DB Queue from Job Server

Hi everybody!

We’ve encountered a critical error while requesting the database queue from the Job Server.
The process failed and generated multiple errors related to transaction handling, lock timeouts, and retry failures.

This is now blocking all scheduled operations across the platform.

Currently, we’re restarting the OneIMService on each Job Server as a workaround.
This temporarily restores system functionality, but only for a few days.
After that, the same errors recur and scheduled tasks fail again.

To help with troubleshooting, this is the full error log:

2025-07-28 06:00:24 +02:00 - Serious: Last process step request failed with error: '"[810023] Error running statement: exec QBM_PJobQueueLoad N'\[xxx]', 31, -1, 242775872, N'', N'[xxx]'-->[810143] Database error 266: Transaction count after EXECUTE indicates a mismatching number of BEGIN and COMMIT statements. Previous count = 2, current count = 3.-->[810143] Database error 266: detected in (SRV=[xxx], DB=OneIM) procedure QBM_PExecuteSQLWithRetry_LLP Line 33-->[810143] Database error 50000: No success after all retries. Run the schema changes without transaction.-->[810143] Database error 50000: detected in (SRV=[xxx], DB=OneIM) procedure QBM_PExecuteSQLWithRetry_LLP Line 229 code starting with : set lock_timeout 50;set XACT_ABORT OFF; update QBMServer set LastJobFetchTime = '2025-07-28 04:00:21.243' where UID_QBMServer = '[xxx]' -->[810143] Database error 1222: Lock request time out period exceeded.-->[810143] Database error 1222: detected in (SRV=[xxx], DB=OneIM) procedure QBM_PExecuteSQLWithRetry_LLP Line 2 code starting with : set lock_timeout 50;set XACT_ABORT OFF; update QBMServer set LastJobFetchTime = '2025-07-28 04:00:21.243' where UID_QBMServer = '[xxx]' "'!
2025-07-28 06:00:24 +02:00 - Serious - Error occurred in DbRequestQueue.Process (thread: Database Job Requests):
[821002] Error requesting queue '\[xxx]' for database '[xxx]\OneIM'.
[810023] Error running statement: exec QBM_PJobQueueLoad N'\[xxx]', 31, -1, 242775872, N'', N'[xxx]'
[810143] Database error 266: Transaction count after EXECUTE indicates a mismatching number of BEGIN and COMMIT statements. Previous count = 2, current count = 3.
[810143] Database error 266: detected in (SRV=[xxx], DB=OneIM) procedure QBM_PExecuteSQLWithRetry_LLP Line 33
[810143] Database error 50000: No success after all retries. Run the schema changes without transaction.
[810143] Database error 50000: detected in (SRV=[xxx], DB=OneIM) procedure QBM_PExecuteSQLWithRetry_LLP Line 229 code starting with : set lock_timeout 50;set XACT_ABORT OFF; update QBMServer set LastJobFetchTime = '2025-07-28 04:00:21.243' where UID_QBMServer = '[xxx]'
[810143] Database error 1222: Lock request time out period exceeded.
[810143] Database error 1222: detected in (SRV=[xxx], DB=OneIM) procedure QBM_PExecuteSQLWithRetry_LLP Line 2 code starting with : set lock_timeout 50;set XACT_ABORT OFF; update QBMServer set LastJobFetchTime = '2025-07-28 04:00:21.243' where UID_QBMServer = '[xxx]'
at VI.JobService.DbProvider.DbRequestQueue.Process(ProviderRequest request)
at VI.JobService.MSSqlJobProvider._MsSqlRequestQueue._HandleGetJobs(IDbSession dbSession, GetJobsProviderRequest request)
---- Start of Inner Exception ----
at VI.JobService.MSSqlJobProvider._MsSqlRequestQueue._HandleGetJobs(IDbSession dbSession, GetJobsProviderRequest request)
at VI.Base.SyncActions.Do[T,T1,T2](T1 p1, T2 p2, Func`3 function)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
--- End of stack trace from previous location where exception was thrown ---
at VI.DB.DataAccess.ReadOnlyDbSession.<SqlExecuteAsync>d__39.MoveNext()
---- Start of Inner Exception ----
at VI.DB.DataAccess.ReadOnlyDbSession.<SqlExecuteAsync>d__39.MoveNext()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
--- End of stack trace from previous location where exception was thrown ---
at VI.DB.DataAccess.ReadWriteDbSession.<IgnoreBrokenConnectionAsync>d__56`1.MoveNext()
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
--- End of stack trace from previous location where exception was thrown ---
at VI.DB.DataAccess.ReadOnlyDbSession.<>c__DisplayClass39_0.<<SqlExecuteAsync>b__0>d.MoveNext()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
--- End of stack trace from previous location where exception was thrown ---
at VI.DB.DataAccess.SafeDbCommand.<_CheckedAsync>d__43`1.MoveNext()
---- Start of Inner Exception ----

---- Start of Inner Exception ----

---- Start of Inner Exception ----

---- Start of Inner Exception ----

---- Start of Inner Exception ----

Thanks
A. Dell'Atti