• State Verified Answer
  • Replies 4 replies
  • Answers 1 answer
  • Subscribers 95 subscribers
  • Views 238 views
  • Users 0 members are here
Options
Related

Implementing Random Delay in Orchestrated Processes to Prevent API Overload

fernando pacheco ext
3 days ago

Hi everyone,

I'm working with an Orchestrated Process that calls an external API. Currently, all requests hit the endpoint simultaneously, causing it to return a 500 Internal Server Error due to overload. I'm seeing errors like this in the logs:

(2025-06-03 01:27:21.697) [810222] Error running script 'CCC_Custom_Universidade_ASAD'.
[System.Net.WebException] The remote server returned an error: (500) Internal Server Error.
   at StdioProcessor.StdioProcessor._Execute(Job job)
   at VI.JobService.JobComponents.ScriptComponent.Activate(String task)
   at VI.JobService.JobComponents.ScriptComponent._TaskScriptExec()
   at VI.DB.Scripting.ScriptRunner.Eval(String key, Object[] parameters)
   ---- Start of Inner Exception ----
   at DynScripts.Scripts_YJbLef26FdtP02l5bRVmtvwiRlf5MWGx.CCC_Custom_Universidade_DASA(String identificacao)
   at System.Net.WebClient.UploadData(Uri address, String method, Byte[] data)
   at System.Net.WebClient.UploadDataInternal(Uri address, String method, Byte[] data, WebRequest& request)

The script making the API call is CCC_Custom_Universidade_ASAD. I've included the full script below for reference.

My goal is to introduce a random delay within this process so that the API requests are staggered, preventing the endpoint from being overwhelmed.

Does anyone know if there's a built-in component or a recommended method in One Identity Manager (within an Orchestrated Process or a script like this) to achieve a random delay? I'm looking for a way to pause the execution for a random period (e.g., between 1 to 5 minutes) before each API call.

Any ideas or examples would be greatly appreciated!

Thanks in advance.

  • +1 3 days ago

    Hi Fernando,

    Well you could use the DelayComponent - Delay ..... and generate the value for the delay - mins/secs I guess - in the pre-script of your process.  But there's always going to be a chance that you'll hit the 500 error.  I had a similar issue at my customer so in my process I call the API ..... if I go down the red path (error occurred) the next step checks the error ...... if it was a 400 simply end ..... if it was a 500 go to the next step ..... next step uses SQL to empty the ErrorMessages value from the API call step ..... then next step is DelayComponent - ReActivateJob .... so it loops back to the start.

    This way the process will loop back until it stops getting a 500 error ...... to avoid an infinite loop the DelayComponent - ReActivateJob has a 'Retries' parameter.

    Maybe this helps.

    B.

  • 0 2 days ago in reply to Barry Jackson

    Hi Barry,

    Thank you so much for the insights — they’re incredibly helpful and definitely pointed me in the right direction!

    I managed to randomize the delay time using a script:

    <BaseObjectTypeAttribute("PersonWantsOrg")> _
Public Function CCC_DelayRandomizer(ByVal dbPersonWantsOrg As ISingleDbObject) As Integer
	
    Dim rnd As New Random(CInt(DateTime.Now.Ticks Mod Int32.MaxValue))
	
    Dim delayInSeconds As Integer = rnd.Next(30, 301)

    Return delayInSeconds

End Function


    Then in the Delay Component on the Orchestrated Process I used the following in the DelayTime field:
    Value = $SCRIPT(CCC_DelayRandomizer)delayInSeconds$

    And It worked perfectly for randomizing the time between 30 and 300 seconds (it is adjustable)

    Now I’m going to implement the error validations as you suggested — your approach with handling 400 vs. 500 responses, resetting the ErrorMessages, and looping with a retry counter is really clever. I believe this will help resolve the issue completely. (Really great insight!)

    Thanks again!

    Best,
    Fernando



  • 0 2 days ago in reply to Barry Jackson

    Hi again Barry,

    I had a quick question regarding that SQL you shared — I’m not entirely clear on how the error validation part works. Would you mind explaining it a bit more, or if possible, could you attach a screenshot showing how that SQL works in your environment?

    I'm especially unsure about how the “500” error is being captured and validated in the query.

    Thanks a lot!

  • 0 2 days ago in reply to fernando pacheco ext

    Hi Fernando,

    Sure .... so the process looks like this:

    Ignore the 'Get the XDateInserted...' step .... that's something extra I needed for this target.

    The step: 'CCC Check Error Returned'

    Is:  SQLComponent - Exists

    WhereClause =

    Dim f As ISqlFormatter = Session.SqlFormatter

    Value = f.AndRelation ( _

    f.Comparison("GenProcID", CStr(Variables("GenProcID")), ValType.String, CompareOperator.Equal, FormatterOptions.NonUnicodeLiterals), _

    f.Comparison("JobChainName", myChain.Name, ValType.String, CompareOperator.Equal, FormatterOptions.UnicodeLiterals), _

    f.Comparison("ErrorMessages", "%The remote server returned an error: (400) Bad Request%", ValType.String, CompareOperator.Like, FormatterOptions.UnicodeLiterals), _

    f.Comparison("TaskName", "ExecuteCommand", ValType.String, CompareOperator.Equal, FormatterOptions.UnicodeLiterals) , _

    f.Comparison("BasisObjectKey", $XObjectKey$, ValType.String, CompareOperator.Equal, FormatterOptions.NonUnicodeLiterals))

    Then: 'CCC Clean up error messages of Provision XXXX Account'

    Is: SQLComponent - Execute SQL

    SQLStmt =

    Dim Query As String = <![CDATA[

    UPDATE JobQueue SET ErrorMessages=NULL

    WHERE GenProcID = '{0}'

    AND JobChainName = '{1}'

    AND ErrorMessages <> ''

    AND TaskName = 'ExecuteCommand'

    AND BasisObjectKey = '{2}'

    ]]>.Value()

    Value = String.Format(Query,CStr(Variables("GenProcID")),myChain.Name,$XObjectKey$)

    HTH, Barry.

    (And I hope the image of the process works as it's not showing up in this edit window!)