Process doesn't work on schedule and doesn't go on error path when it has to

Hi,

I am facing an issue with a scheduled process that reads a CSV file using the DataImport script. This process is scheduled to run three times a day, but it fails when executed via the schedule, producing the following error:

2025-02-24 09:16:16 +00:00 - Error - \comau-test-oneidentity - VI.JobService.JobComponents.HandleObjectComponent - AFAA9544-9629-4980-93DB-A72A8353EEA5: Errors occurred
    Error saving Person: [810103] Error generating processes for event CCC_NotifyErrorRegardingCSVFile_PersonManagerByPersonnelNumber.
    [810222] Error running script 'Event_CCC_NotifyErrorRegardingCSVFile_PersonManagerByPersonnelNumber'.
    [810108] Error generation process step event CCC_NotifyErrorRegardingCSVFile_PersonManagerByPersonnelNumber.
    [810109] Error generating process CCC_UpdateIdentitiesManagerByPersonnelNumber.
    [810155] Error running prescript.
    [System.IO.DirectoryNotFoundException] Could not find a part of the path 'C:\Users\Public\Documents\csv\log.txt'.
    The event CCC_NotifyErrorRegardingCSVFile_PersonManagerByPersonnelNumber was triggered for 1 object(s) of type DialogTable.
  

Behavior When Running Manually from Object Browser:

  • If the file exists → The process runs successfully, processes the file, and archives it as expected (implemented in VB.NET).
  • If the file does not exist → The process fails with the following error:


[810103] Error generating processes for event CCC_NotifyErrorRegardingCSVFile_PersonManagerByPersonnelNumber.
[810222] Error running script 'Event_CCC_NotifyErrorRegardingCSVFile_PersonManagerByPersonnelNumber'.
[810108] Error generation process step event CCC_NotifyErrorRegardingCSVFile_PersonManagerByPersonnelNumber.
[810109] Error generating process CCC_UpdateIdentitiesManagerByPersonnelNumber.
[810110] Error generating process step Assert file exists.
[810156] Error generating parameter FName.
Object reference not set to an instance of an object.

Attempted Solution:

To prevent this failure, I tried modifying the parameter’s value template as follows:



Dim fictitious As String = "C:\Users\Public\Documents\csv\fictitious.csv"
Dim file As String
If String.IsNullOrEmpty(values("FilePathAndName").ToString) Then
	file = fictitious 
Else
	file = values("FilePathAndName").ToString
End If

Value = file



Questions:

  1. How can I prevent the process from failing when executed via the schedule?
  2. How can I ensure that the process handles the case where the CSV file is missing, instead of failing?

Info:

1. One Identity Manager version: 9.1

2. Process looks like: https://photos.app.goo.gl/X17qJtAprSu5AJzm8 

3. Including the step that asserts if the file exists, doesn't make any difference, gives the same error.

Any insights or recommendations would be greatly appreciated!


Regards,

Abdalla

  • The initial problem is in your "Pre-Script" of the importing process CCC_UpdateIdentitiesManagerByPersonnelNumber.


        [System.IO.DirectoryNotFoundException] Could not find a part of the path 'C:\Users\Public\Documents\csv\log.txt'.


    Remember, that the import process is created by a HandleObject process task running on the JobService. So, the pre-script, the generation codes and parameter code snippets need to work on the JobService and not only on your client machine where you test it manually (ObjectBrowser).

    And remember, that the code snippets in the process will be executed during generation, so you need to ensure that your code handles NULL cases correctly.

    By the way, a disconnected "Assert" process steps does not help at all. It will never be part of the final generated process, but the process step's code is run during generation and may lead to additional errors.

  • If the prescript is designed to retrieve the latest CSV file and assign it as a parameter, it should also handle errors efficiently by logging any issues. Additionally, it should create a log entry that can be included in an email notification to inform the sender team. A timestamped log should also be maintained to track the history of processed files. The prescript is as follows:

    Dim directoryPath As String = "\\ip\csv\test" ' the path is reachable from the cloud machine and testing machine
    Try
        ' Ensure directory exists
        If Not Directory.Exists(directoryPath) Then
            Throw New Exception("Directory does not exist: " & directoryPath)
        End If
    
        ' Get all CSV files in the directory
        Dim csvFiles As String() = Directory.GetFiles(directoryPath, "*.csv")
    
        ' Ensure there are CSV files
        If csvFiles Is Nothing OrElse csvFiles.Length = 0 Then
            Throw New Exception("No CSV files found in directory: " & directoryPath)
        End If
    
        ' Get the latest modified CSV file
        Dim latestFile As String = csvFiles.OrderByDescending(Function(f) File.GetLastWriteTime(f)).First()
    
        values("FilePathAndName") = latestFile
    	values("DirectoryPath") = directoryPath
    	values("FileName") = Path.GetFileName(latestFile)
    	
    Catch ex As Exception
        
    	' Define the directory where the log will be stored
        Dim logDirectory As String = "\\ip\csv\test"
    	
    	' Define the log file name
        Dim logFileName As String = "log.txt"
        Dim logFilePath As String = Path.Combine(logDirectory, logFileName)
    	Dim logMessage As String = "Error: " & ex.Message 
    	
        ' Prepare the log message with filename and timestamp
        Dim timestamp As String = DateTime.Now.ToString("yyyy-MM-dd HH:mm:ss")
    	Dim logEntry As String = String.Format("[{0}] - Executed in {1}{2}Timestamp: {0}{2}Log Message: {3}{2}{4}", 
    	timestamp, logFilePath, Environment.NewLine, logMessage, New String("-"c, 50))
    	
    	values("logEntry") = logEntry
    		
        ' Use StreamWriter to append log entries to the existing file
        Using writer As New StreamWriter(logFilePath, append:=True)
            ' Write the log entry to the log file
            writer.WriteLine(logEntry)
        End Using
    End Try


    How can this be specifically implemented to work within the JobService? Do you have any insights or recommendations on the best approach?

    Note: The machine is accessible from both the cloud and the testing environment.