Hello,
I am seeing the high CPU utilization by the StdioProcessor.exe for the last few days. Due to this lots of jobs are not processing and getting stuck in JobQ. I m not able to find anything for this issue.
Any suggestion?
Thanks
What version of Identity Manager are you using? Also, can you confirm that Debug or Trace logging is not enabled in the global log config?
What version are you using?
Newer versions show the process-id of the StdioProcessor.exe executing some workload in the Job Server Monitor page for the currently executed jobs. So you are able to identify the long-running job.
We are using the Q1IM 6.1.4 version. I checked and found that there are no. jobs stuck. It seems that’s those ones are stuck due to some unsupported characters being in use. That is my assumption for one of the reasons. Any other reasons where should I look into?
Thanks,
Rajneesh Ram
What makes you think this "It seems that’s those ones are stuck due to some unsupported characters being in use."?
It is just my assumption, it happened in the past due to that jobs were stuck, but at that time CPU utilization was not high. Now I m seeing 5-6 processes are running simultaneously with high CPU utilization.
What type of task is running?
Basically, jobs create a SNOW ticket. Due to high CPU utilization jobs are running very slow/stuck in the Q's. I m not seeing anything in the Event Viewer which relates to the issue. In the log file generally, it says that the:
ErrorMessages = [810222] Error executing script 'XXX_SNOWCreateRequest'.
[System.Net.WebException] The remote server returned an error: (500) Internal Server Error.
at StdioProcessor.StdioProcessor._Execute(Job job)
at VI.JobService.JobComponents.ScriptComponent.Activate(String task)
at VI.JobService.JobComponents.ScriptComponent._TaskScriptExec()
at VI.DB.Scripting.ScriptInstance.Eval(String key, Object[] parameters)
---- Start of Inner Exception ----
at DynScripts.Scripts_DA122C804FEC5AF1E0431782020A8A85.XXX_SNOWCreateRequest(String sLoginPassword, String UID_Person, String ObjectKeyOrdered, String sResId, String xObjectKey, String orderState, String lActionType, String sRequestRef)
at System.Net.WebClient.UploadString(Uri address, String method, String data)
at System.Net.WebClient.UploadDataInternal(Uri address, String method, Byte[] data, WebRequest& request)
Reactivation of frozen Job afbd17bd-5c94-41e7-ac2c-121adcc1a22f with parameter SELF
[810222] Error executing script 'XXX_SNOWCreateRequest'.
[System.Net.WebException] The remote server returned an error: (500) Internal Server Error.
at StdioProcessor.StdioProcessor._Execute(Job job)
at VI.JobService.JobComponents.ScriptComponent.Activate(String task)
at VI.JobService.JobComponents.ScriptComponent._TaskScriptExec()
at VI.DB.Scripting.ScriptInstance.Eval(String key, Object[] parameters)
---- Start of Inner Exception ----
at DynScripts.Scripts_DA122C804FEC5AF1E0431782020A8A85.XXX_SNOWCreateRequest(String sLoginPassword, String UID_Person, String ObjectKeyOrdered, String sResId, String xObjectKey, String orderState, String lActionType, String sRequestRef)
at System.Net.WebClient.UploadString(Uri address, String method, String data)
at System.Net.WebClient.UploadDataInternal(Uri address, String method, Byte[] data, WebRequest& request)
Reactivation of frozen Job afbd17bd-5c94-41e7-ac2c-121adcc1a22f with parameter SELF
[810222] Error executing script 'XXX_SNOWCreateRequest'.
[810019] Could not create single object for table PersonWantsOrg.
[810029] Object of type Request procedures does not exist in database or you do not have the relevant viewing permissions.
at StdioProcessor.StdioProcessor._Execute(Job job)
at VI.JobService.JobComponents.ScriptComponent.Activate(String task)
at VI.JobService.JobComponents.ScriptComponent._TaskScriptExec()
at VI.DB.Scripting.ScriptInstance.Eval(String key, Object[] parameters)
---- Start of Inner Exception ----
at DynScripts.Scripts_DA122C804FEC5AF1E0431782020A8A85.XXX_SNOWCreateRequest(String sLoginPassword, String UID_Person, String ObjectKeyOrdered, String sResId, String xObjectKey, String orderState, String lActionType, String sRequestRef)
at VI.DB.Implementation.Connection.CreateSingle(String tablename, String primaryKeyValue)
at VI.DB.Implementation.Connection._CreateSingle(String tablename, DbObjectKey key, IColElem elem, VarContext creationVariables, SingleDbObjectFlags flags)
---- Start of Inner Exception ----
at VI.DB.Implementation.Connection._CreateSingle(String tablename, DbObjectKey key, IColElem elem, VarContext creationVariables, SingleDbObjectFlags flags)
at VI.DB.Implementation.SingleDbObject.Load(IColElem elem)
Reactivation of frozen Job afbd17bd-5c94-41e7-ac2c-121adcc1a22f with parameter SELF
[810222] Error executing script 'XXX_SNOWCreateRequest'.
[810019] Could not create single object for table PersonWantsOrg.
[810029] Object of type Request procedures does not exist in database or you do not have the relevant viewing permissions.
at StdioProcessor.StdioProcessor._Execute(Job job)
at VI.JobService.JobComponents.ScriptComponent.Activate(String task)
at VI.JobService.JobComponents.ScriptComponent._TaskScriptExec()
at VI.DB.Scripting.ScriptInstance.Eval(String key, Object[] parameters)
---- Start of Inner Exception ----
at DynScripts.Scripts_DA122C804FEC5AF1E0431782020A8A85.XXX_SNOWCreateRequest(String sLoginPassword, String UID_Person, String ObjectKeyOrdered, String sResId, String xObjectKey, String orderState, String lActionType, String sRequestRef)
at VI.DB.Implementation.Connection.CreateSingle(String tablename, String primaryKeyValue)
at VI.DB.Implementation.Connection._CreateSingle(String tablename, DbObjectKey key, IColElem elem, VarContext creationVariables, SingleDbObjectFlags flags)
---- Start of Inner Exception ----
at VI.DB.Implementation.Connection._CreateSingle(String tablename, DbObjectKey key, IColElem elem, VarContext creationVariables, SingleDbObjectFlags flags)
at VI.DB.Implementation.SingleDbObject.Load(IColElem elem)
I m not sure how it is related or not.
Basically, jobs create a SNOW ticket. Due to high CPU utilization jobs are running very slow/stuck in the Q's. I m not seeing anything in the Event Viewer which relates to the issue. In the log file generally, it says that the:
ErrorMessages = [810222] Error executing script 'XXX_SNOWCreateRequest'.
[System.Net.WebException] The remote server returned an error: (500) Internal Server Error.
at StdioProcessor.StdioProcessor._Execute(Job job)
at VI.JobService.JobComponents.ScriptComponent.Activate(String task)
at VI.JobService.JobComponents.ScriptComponent._TaskScriptExec()
at VI.DB.Scripting.ScriptInstance.Eval(String key, Object[] parameters)
---- Start of Inner Exception ----
at DynScripts.Scripts_DA122C804FEC5AF1E0431782020A8A85.XXX_SNOWCreateRequest(String sLoginPassword, String UID_Person, String ObjectKeyOrdered, String sResId, String xObjectKey, String orderState, String lActionType, String sRequestRef)
at System.Net.WebClient.UploadString(Uri address, String method, String data)
at System.Net.WebClient.UploadDataInternal(Uri address, String method, Byte[] data, WebRequest& request)
Reactivation of frozen Job afbd17bd-5c94-41e7-ac2c-121adcc1a22f with parameter SELF
[810222] Error executing script 'XXX_SNOWCreateRequest'.
[System.Net.WebException] The remote server returned an error: (500) Internal Server Error.
at StdioProcessor.StdioProcessor._Execute(Job job)
at VI.JobService.JobComponents.ScriptComponent.Activate(String task)
at VI.JobService.JobComponents.ScriptComponent._TaskScriptExec()
at VI.DB.Scripting.ScriptInstance.Eval(String key, Object[] parameters)
---- Start of Inner Exception ----
at DynScripts.Scripts_DA122C804FEC5AF1E0431782020A8A85.XXX_SNOWCreateRequest(String sLoginPassword, String UID_Person, String ObjectKeyOrdered, String sResId, String xObjectKey, String orderState, String lActionType, String sRequestRef)
at System.Net.WebClient.UploadString(Uri address, String method, String data)
at System.Net.WebClient.UploadDataInternal(Uri address, String method, Byte[] data, WebRequest& request)
Reactivation of frozen Job afbd17bd-5c94-41e7-ac2c-121adcc1a22f with parameter SELF
[810222] Error executing script 'XXX_SNOWCreateRequest'.
[810019] Could not create single object for table PersonWantsOrg.
[810029] Object of type Request procedures does not exist in database or you do not have the relevant viewing permissions.
at StdioProcessor.StdioProcessor._Execute(Job job)
at VI.JobService.JobComponents.ScriptComponent.Activate(String task)
at VI.JobService.JobComponents.ScriptComponent._TaskScriptExec()
at VI.DB.Scripting.ScriptInstance.Eval(String key, Object[] parameters)
I m not sure how it is related or not.
The error is: "The remote server returned an error: (500) Internal Server Error."
Sound like your Service Now WebRequest has some issues.
Did you change something in our code recently? If the job server is running in a VM, does your resource setting for the VM have changed?
I am suggesting contacting support to help you figure this out.
