I am attempting to add new RHEL8 servers into our OID 8.1.1 environment, and I'm running into a puzzling issue. I've added three users. The first went through without issue. The user was provisioned and a random password was setup and confirmed as functional. The next two failed. One was an insert and the other an update, but the message is the same:
<e>2020-10-06 05:46:14 +02:00 - \<JobServiceQueue> - VI.Projector.JobComponent.ProjectorComponent - 2932fe70-dad6-4b81-af0a-922985f7dd46: Errors occurred
Executed on: <JobServiceQueue>
[1777018] Error executing synchronization project (Unix user/group synchronization of host <hostname>)'s workflow (Provisioning).
[1777124] Error executing projection step (User) of projection configuration (Provisioning (Provisioning)).
[1777219] Error executing synchronization step (User)!
[1777004] Method ( (Update)) could not be executed successfully.
[Sugi.Common.Exceptions.SugiSessionException] Could not execute sugi action for <IP address>: Received unexpected EOF while parsing action results
[Sugi.Common.Exceptions.SugiParserException] Received unexpected EOF while parsing action results
at VI.Projector.JobComponent.ProjectorComponent._AdHocProjection()
at VI.Projector.Database.ProjectorExecutor.Project(ISession session, IProjectionConfiguration configuration, ProjectionOption options, ISystemObject adHocObject, CancellationToken cancellationToken)
at VI.Projector.Projection.Extensions.Execute(IProjectorEngine engine, IProjectionConfiguration configuration, ProjectionOption options, ISystemObject adHocObject, CancellationToken cancellationToken)
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
---- Start of Inner Exception ----
at VI.Projector.Projection.ProjectorEngine.<ExecuteAsync>d__3.MoveNext()
---- Start of Inner Exception ----
at VI.Projector.Projection.ProjectorEngine.<ExecuteAsync>d__3.MoveNext()
---- Start of Inner Exception ----
at VI.Projector.Projection.ProjectorEngine.<ExecuteAsync>d__3.MoveNext()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
--- End of stack trace from previous location where exception was thrown ---
at VI.Projector.Projection.ProjectionStrategy.<ExecuteStepAsync>d__3.MoveNext()
---- Start of Inner Exception ----
at VI.Projector.Projection.ProjectionStrategy.<ExecuteStepAsync>d__3.MoveNext()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
--- End of stack trace from previous location where exception was thrown ---
at VI.Projector.Projection.ProjectionStrategyBase.<OnExecuteStepAsync>d__6.MoveNext()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
--- End of stack trace from previous location where exception was thrown ---
at VI.Projector.Projection.ProjectionStrategyBase.<_ProcessIntersectionSetAsync>d__21.MoveNext()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
--- End of stack trace from previous location where exception was thrown ---
at VI.Projector.Projection.ProjectionStrategyBase.<_ProcessMappingOfSystemObjectPairsAsync>d__25.MoveNext()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
--- End of stack trace from previous location where exception was thrown ---
at System.Threading.Tasks.Task.Execute()
at System.Threading.Tasks.Task`1.InnerInvoke()
at VI.Projector.Projection.ProjectionStrategyBase.<>c__DisplayClass25_0.<_ProcessMappingOfSystemObjectPairsAsync>b__9()
at VI.Projector.Projection.ProjectionStrategyBase._ExecuteMethods(IProjectionStep step, SystemObjectMatchingSet matchingSet, ISystemConnection connection, ISchemaMethodAssignment[] methodAssignments, IList`1 systemObjectPairs, SystemMappingSide side, ProjectionDirection projectionDirection, Boolean forceReloadObject, Boolean ignoreQuotas)
at VI.Projector.Projection.ProjectionStrategyBase._ExecuteMethodNative(ISystemConnection connection, SystemObjectMatchingSet matchingSet, ISchemaMethod method, IEnumerable`1 objectsWithAncestryToExecute, Boolean reloadObjects, Boolean isImport)
at VI.Projector.Connection.Connections.BottleNeckConnection._Redirect[T](Func`1 redirection)
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
---- Start of Inner Exception ----
at VI.Projector.Connection.Connections.BottleNeckConnectionTask.Execute()
at VI.Projector.Connection.SystemConnection.ExecuteMethod(ISchemaMethod method, IEnumerable`1 systemObjects, ExecuteMethodOptions option)
---- Start of Inner Exception ----
at VI.Projector.Connection.SystemConnection.ExecuteMethod(ISchemaMethod method, IEnumerable`1 systemObjects, ExecuteMethodOptions option)
at VI.Projector.Connection.Connectors.BottleNeckConnector._Redirect[T](Func`1 redirection)
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
---- Start of Inner Exception ----
at VI.Projector.Connection.Connectors.BottleNeckConnectorTask.Execute()
at VI.Projector.Connection.SystemConnector.ExecuteMethod(ExecuteMethodRequest request)
at VI.Projector.Connection.SystemConnector.CommitObject(CommitObjectRequest request)
at VI.Projector.Unix.Connector.BaseTypeHandler.CommitObject(CommitObjectRequest request, UnixConnectionContext context)
at VI.Projector.Unix.Connector.UserTypeHandler.Update(UnixSystemObjectData target, UnixConnectionContext context)
at VI.Projector.Unix.Connector.UserTypeHandler.SetEtcAccountPassword(UnixConnectionContext context, IEnumerable`1 modifications, String username, String uid)
at VI.Projector.Unix.Connector.UnixConnectionContext.RunSugiAction[T](ISugiAction`1 action, Func`2 skipFunction)
at Platform.Sugi.Actions.BaseSugiAction`1.Execute(ISugiSession session, Func`2 skipFunction)
at Sugi.Common.SugiSession.ExecuteAction[T](Stream package, String arguments, Func`2 skipFunction)
---- Start of Inner Exception ----
at Sugi.Common.SugiSession.ExecuteAction[T](Stream package, String arguments, Func`2 skipFunction)
at Sugi.Common.SugiResultsParser`1.Parse(Func`2 skipFunction)
Synchronization ended with errors.
You can find more detailed information in the log from Tuesday, October 6, 2020.
I'm not understanding why some users are provisioned without issue and others are not. Obviously, OID is getting an EOF indication before it's expected, but how do I determine what is coming back?
When the job freezes in the queue and I check the target RHEL8 system, I see that the user account is provisioned. If I continue with success, the emails are sent with the username and the temporary random password. If I try to log in, it fails. If I reset the user password for the new account using the randomly generated password and the unix account utilized by the sync project, the password is changed successfully and I can successfully log in with the newly provisioned account. It appears the password setup is failing, but I know the target system will accept the random password given I can manually set it utilizing the same admin account. I'm next going to check the logs on the Unix host, but I'm curious if there is any way to see what is coming back to the OID job server and what it doesn't like about the response.
