Integrating ARS with source control (git)

Morning all,

Currently within ARS, you write sript modules of various types and trigger them via policies, schedules, events, etc. Those scripts however must live within the ARS environment, specifically written in the inline script window ARS provides, with little to no code intelligence or syntax assistance. Further, there's no version control and for those of us that use git, no integration that I know of.

So, I'm here hoping I've completely overlooked something. Is it possible to have ARS load scripts at run time from a git repo? Is it possible to determine the physical script location on the ARS server, and perhaps create an external job using something like Jenkins that would keep those files updated when commits happened to a specific repo? Anything that would help us integrate source control with ARS? Right now I'm manually committing important changes and scripts to a git repo but it's subject to the failures of... well being manual of course.

I've had a few thoughts on potential solutions but none seem great:

1) If there were an API to import files into ARS, that would be one potential solution. 

2) If we knew the physical script location on the ARS server, we could write something back end to replace those files based on git commits. 

3) We could source the script content in real time from git, within the script module itself. This one I feel could actually work, but it's hacky. I'll explain.

We would still require the script module to exist in ARS. For simple scripts which just run, the only line in the script would be to source the script content from our desired git repo. This could be a few lines or could call a single internal function we write to help out. We would essentially then leave this script as is, forever. Any changes would be done downstream using our IDE of choice. Changes would be committed to git, and when ARS ran again, the new content would be sourced at run time.

For scripts which have multiple functions that we may want to call/trigger from policies, we would need to define the function (at minimum) within the script module within ARS still. If the function weren't defined the workflow or policy wouldn't see it as an available starting place when we're configuring it. I've knocked my head against this but I don't see a way around it. You would leave the function empty however and then source the content for it from the git source. 

Put it all together, and the best I can come up with is this approach. A bunch of script modules, some just sourcing git, others with empty defined functions which source their content from git. On the git end you'd have a bunch of seemingly disconnected pieces of code in separate files which would link up to these various sourcing/imports actions.

It's dirty, and any time you made a chance to functions you'd have to change your sourcing/git stuff. But it's all I can come up with.

-----------

Alright, so can anyone think of any other way to integrate ARS with git? I'm really surprised they haven't come up with a solution... or I'm really hoping I've just completely overlooked something.

Thanks folks,

  • An interesting topic and one I have thought some about strictly in the context of editing and saving scripts using "better" editors.

    You are of course aware than when you create a script, you have to in effect "register" it as a script module in AR.

    When you update it, what happens underneath is that a virtual attribute of the "registered" script gets updated with your new code.

    So technically, you could update these virtual attributes containing the code yourself using Set-QADObject -proxy <script module DN> thereby allowing to do all of your version control etc. externally.  In fact, you could probably create your own version control VA on your "registered" script modules that you could update every time you update the script.

    Some food for thought anyway.

  • We have two environments (lab/prod) that i have to constantly move scripts between.  Each one has different information such as Powershell code location, sql server, domain info, credential storage locations, etc..  I store that data in a .psd1 file in each environment and import it when ARS runs scripts.  That allows my code to be much more easier to move with copy/paste.

    I also store a ton of functions outside of ARS and then dot source them when ARS launches policy scripts.  For example,

    function onInit($context)
    {
        $context.UseLibraryScript('LibraryScript')
        $ProvConfig = Import-LocalizedData -BaseDirectory 'C:\functions' -FileName ARSConfigurations.psd1
        Get-ChildItem -Path "$($ProvConfig.FunctionLocation)\*.ps1" | % { . $_.FullName }
    }
    

    You could probably implement something like that and it points to a published repo location?

    ARS can have small functions that just dot source your real code and process everything in the ARS way.

  • This is interesting.

    So this here:

        $ProvConfig = Import-LocalizedData -BaseDirectory 'C:\functions' -FileName ARSConfigurations.psd1

    ,,, "pre-loads" your custom cmdlets into the AR PoSh runtime?

  • Get-ChildItem -Path "$($ProvConfig.FunctionLocation)\*.ps1" | % { . $_.FullName }

    that line is loading my custom functions.  It goes out to $ProvConfig.FunctionLocation and finds any .ps1 files.  It then loops through and dot sources them which makes them available to your PS console.

    $ProvConfig stores the data pertinent to whatever environment i'm in.. Lab/Prod.

  • That works for policies which fire on events and know precisely where to look for the script they are trying to run. You'd still need to have the script exist in ARS and at minimum, it would (as indicated in the image) need the 'onInit' method present to get things going.

    What about when you want to use a workflow though? I have workflows which, right now, can be targeted at specific functions within scripts. If those scripts didn't exist in ARS in their full version the workflow would have no knowledge of them and I wouldn't be able to specify what function should be running.

    Your solution is the closest I've seen in actual practice though, and it does partially some of the problems, definitely. 

  • That works for policies which fire on events and know precisely where to look for the script they are trying to run.

    The only difference with a workflow is that the start condition of the workflow is your event handler (e.g. onPostCreate) - the code you want to run sits in a script activity and you can still use OnInit to instantiate / make the AR PoSh runtime aware of your external cmdlets.  What I am not sure about is whether running an onInit in the "initialization script" that you can setup in the start conditions of the workflow will persist the called library scripts for all scripts executed by the workflow.  This is the location I am referring to:

  • Although I hate to bump up an ancient thread, based on your idea, I created a script for pushing our scripts in ARS, thought I'd share it here just in case it's useful for anyone else.

    It requires a little work for moving scripts between folders, but it's proved useful for us for managing our scripts in source code and vscode.

    The script checks for any files/folders under Script Modules and compares them to the relevant script in ARS.  I've also got another script for downloading the existing scripts.

    [CmdletBinding()]
    param (
        [Parameter()]
        [switch]
        $ShowChanges
    )
    
    $root = Resolve-Path "./"
    
    $scripts = Get-ChildItem -Path "./Script Modules" -Recurse -File
    $scriptDistinguishedNames = [System.Collections.Generic.List[string]]@()
    
    function Get-DistinguisedName {
        param (
            [string]$path
        )
        $path = $path -replace ".ps1"
        $containers = $path -split "\\"
        [array]::Reverse($containers)
        $distinguishedName = $null
        foreach ($container in $containers) {
            if ($container -eq "") { continue }
            $distinguishedName = $distinguishedName + "CN=$container," 
        }
        $distinguishedName = $distinguishedName + "CN=Configuration"
        return $distinguishedName
    }
    
    function Get-ParentContainer {
        [CmdletBinding()]
        param (
            [Parameter()]
            [string]
            $path,
            [switch]
            $ShowChanges
        )
        $dn = Get-DistinguisedName -path $path
        $container = [adsi]"EDMS://$dn"
        if ($null -eq $container.Path) {
            if ($ShowChanges) {
                Write-Information "$dn needs to be created" -InformationAction Continue
            }
            else {
                Write-Information "Creating $dn" -InformationAction Continue
                $parentContainer = Get-ParentContainer -path (Split-Path -Path $path -Parent) -ShowChanges:$ShowChanges
                $params = @{
                    Name            = Split-Path -Path $path -Leaf
                    ParentContainer = $parentContainer
                    Type            = "edsScriptModuleContainer"
                    Proxy           = $true
                }
                $container = New-QADObject @params
            }
        }
        return $dn
    }
    
    foreach ($script in $scripts) {
        $dn = Get-DistinguisedName -path (($script.FullName).Replace($root, ""))
        [void]$scriptDistinguishedNames.Add($dn)
        $arsScript = [adsi]"EDMS://$dn"
        $scriptContent = $(Get-Content -Path $script.FullName -Raw).TrimEnd()
        $createScript = $false
        try {
            $arsScriptContent = $($arsScript.edsaScriptText[0]).TrimEnd()
        }
        catch {
            $arsScriptContent = ""
            $createScript = $true
        }
        if ($createScript) {
            if ($ShowChanges) {
                $parentContainer = Get-ParentContainer -path (Split-Path -Path ($script.FullName).Replace($root, "") -Parent) -ShowChanges:$ShowChanges
                Write-Output "$dn needs to be created"
            }
            else {
                $edsaScriptType = 0
                if ($dn -like "*CN=Library,CN=Script Modules,CN=Configuration") {
                    $edsaScriptType = 2 
                }
                $parentContainer = Get-ParentContainer -path (Split-Path -Path ($script.FullName).Replace($root, "") -Parent) -ShowChanges:$ShowChanges
                $params = @{
                    Name             = $script.Name -replace "\.ps1", ""
                    ParentContainer  = $parentContainer
                    Type             = "edsScriptModule"
                    Proxy            = $true
                    ObjectAttributes = @{
                        edsaScriptText     = $scriptContent
                        edsaScriptType     = $edsaScriptType
                        edsaScriptLanguage = "PowerShell"
                    }
                }
                Write-Information "Creating $dn" -InformationAction Continue
                New-QADObject @params | Out-Null
            }
        }
        elseif ($arsScriptContent -ne "") {
            if ($arsScriptContent -ne $scriptContent) {
                if ($ShowChanges) {
                    Write-Output "$dn needs to be updated" 
                }
                else {
                    Write-Output "Updating $dn"
                    $arsScript.Put("edsaScriptText", $scriptContent)
                    $arsScript.SetInfo()
                }
            }
        }
    }
    
    $searcher = [adsisearcher]"(&(objectClass=edsScriptModule))"
    $searcher.SearchRoot = "EDMS://CN=Script Modules,CN=Configuration"
    foreach ($arsScript in $searcher.FindAll()) {
        if ($arsScript.Path -notlike "*CN=Builtin,CN=Script Modules,CN=Configuration") {
            if ($arsScript.Properties.name -ne "Change Auditor Integration Script") {
                if ($arsScript.Properties.distinguishedname -notin $scriptDistinguishedNames) {
                    if ($ShowChanges) {
                        Write-Output "$($arsScript.Properties.distinguishedname) should be deleted"
                    }
                    else {
                        Remove-QADObject -Identity $($arsScript.Properties.distinguishedname) -Proxy
                    }
                }
            }
        }
    }