Build an AI PowerShell Copilot That Explains, Tests and Validates Scripts

Asking an AI to write a PowerShell script is easy. Knowing whether you should run it is the hard part. The script might use a parameter that doesn't exist, do more than you asked, quietly need admin rights, or download something from the internet. And the AI's own description of what it does might be wrong.

Prompt engineering for PowerShell describes the workflow a careful admin follows by hand: propose, explain, review, test, validate. This post builds that workflow as a tool: a local AI copilot that does every step automatically and hands you a verdict. It's different from our safe PowerShell agent, which runs a fixed list of approved actions through JEA. This copilot writes new scripts, so it has to check them.

Step 1: Propose, With an Explanation

The model is asked for four things in one JSON reply: a plain-language explanation, the list of commands that change something, whether it needs admin rights, and the script. For "Delete files older than 7 days from the Reports folder in the current directory" it produced:

Get-ChildItem -Path .\Reports -File | Where-Object { $_.LastWriteTime -lt (Get-Date).AddDays(-7) } | Remove-Item

With the explanation "This script deletes files older than 7 days from the 'Reports' folder in the current directory" and the declared change Remove-Item.

The explanation is useful for the human reviewer. But as the results show, it can't be the thing you trust. Every claim in it gets checked against the code.

Step 2: Static Checks, Before Anything Runs

The first gate reads the script without running it. PowerShell's own parser turns the script into a syntax tree, so we can list every command it calls and sort each one:

$commands = foreach ($c in $ast.FindAll({ $args[0] -is [System.Management.Automation.Language.CommandAst] }, $true)) {
    $name = $c.GetCommandName()
    if (-not $name) { [pscustomobject]@{ Name = '(dynamic)'; Kind = 'dangerous'; Why = 'command name built at runtime'; Line = $c.Extent.StartLineNumber }; continue }
    $short = $name.Split('\')[-1]
    $resolved = Get-Command $short -ErrorAction SilentlyContinue | Select-Object -First 1
    if ($resolved.CommandType -eq 'Alias') { $short = $resolved.ResolvedCommandName }
    $verb = $short.Split('-')[0]
    $kind = 'change'; $why = ''
    if ($Dangerous.ContainsKey($short)) { $kind = 'dangerous'; $why = $Dangerous[$short] }
    elseif ($Dangerous.ContainsKey($name)) { $kind = 'dangerous'; $why = $Dangerous[$name] }
    elseif ($name -match '\.ps1$') { $kind = 'dangerous'; $why = 'runs another script file' }
    elseif ($name -match '\\') { $kind = 'dangerous'; $why = 'module-qualified name bypasses the test sandbox' }
    elseif ($ReadVerbs -contains $verb) { $kind = 'read' }
    elseif ($short -eq 'ForEach-Object' -or $short -eq 'Where-Object') { $kind = 'read' }
    elseif (-not $resolved) { $kind = 'unknown'; $why = 'not a known command on this machine' }
    [pscustomobject]@{ Name = $short; Kind = $kind; Why = $why; Line = $c.Extent.StartLineNumber }
}

Each command ends up in one of four groups:

• Read: Get-, Test-, Select-, Format- and similar verbs.
• Change: everything else.
• Dangerous: a short deny-list: running strings as code (Invoke-Expression), network access, weakening the execution policy, compiling code, clearing event logs, erasing disks. Also anything that could escape the dry run: running another .ps1 file, a module-qualified name like Microsoft.PowerShell.Management\Remove-Item, or a command name built at runtime.
• Unknown: commands that don't exist on this machine.

The same step also:

• lists .NET static calls such as [System.IO.File]::Delete(...), which bypass command checks entirely;
• flags encoded commands and downloads through .NET;
• looks for signs that admin rights are needed: #Requires -RunAsAdministrator, HKLM: paths, and commands like Restart-Service or New-LocalUser;
• runs PSScriptAnalyzer for its 75 built-in rules.

We added one more check after the first test run, because a script used a parameter that doesn't exist (more on that below). The check confirms every named parameter is real:

$badParams = foreach ($c in $ast.FindAll({ $args[0] -is [System.Management.Automation.Language.CommandAst] }, $true)) {
    if (-not $c.GetCommandName()) { continue }          # dynamic names are already blocked above
    $cmd = Get-Command $c.GetCommandName() -ErrorAction SilentlyContinue | Select-Object -First 1
    if ($cmd.CommandType -eq 'Alias') { $cmd = $cmd.ResolvedCommand }
    if (-not $cmd -or $cmd.CommandType -notin 'Cmdlet', 'Function') { continue }
    $known = @($cmd.Parameters.Keys) + @($cmd.Parameters.Values | ForEach-Object { $_.Aliases })
    foreach ($p in $c.CommandElements | Where-Object { $_ -is [System.Management.Automation.Language.CommandParameterAst] }) {
        if (-not ($known | Where-Object { $_ -like "$($p.ParameterName)*" })) { "$($cmd.Name) has no parameter -$($p.ParameterName) (line $($p.Extent.StartLineNumber))" }
    }
}

PowerShell accepts shortened parameter names, so -ErrorA for -ErrorAction passes, while an invented -Fixed doesn't.

We unit-tested the checker on hand-written snippets before trusting it. A read-only pipeline came back clean; Restart-Service and Remove-Item were classed as changes; a download-and-run line was dangerous; and all three escape routes (module-qualified name, a name built at runtime, a direct .NET delete) were caught.

Step 3: The Dry Run

Static checks can't tell you what a script will actually do. A dry run can, but only if nothing real changes. Our sandbox uses a PowerShell rule: a function with the same name as a cmdlet takes priority over the cmdlet. So for every command the static checks classed as a change, the sandbox defines a stand-in function that records the call instead of acting:

foreach ($name in $names) {
    $body = @"
`$items = @(`$input | ForEach-Object { if (`$_.Name) { `$_.Name } elseif (`$_.FullName) { `$_.FullName } else { "`$_" } })
`$global:SandboxCalls.Add([pscustomobject]@{ Command = '$name'; Arguments = @(`$args | ForEach-Object { "`$_" }); Pipeline = `$items })
"@
    Set-Item -Path "function:global:$name" -Value ([scriptblock]::Create($body))
}
 
# fail closed: every intercepted command must now resolve to our stand-in, or nothing runs
$notIntercepted = @($names | Where-Object { (Get-Command $_ -ErrorAction SilentlyContinue).CommandType -ne 'Function' })
if ($notIntercepted) {
    [pscustomobject]@{ Output = "SANDBOX NOT READY, script not run: $($notIntercepted -join ', ')"; Calls = @() } | ConvertTo-Json
    exit 2
}
 
$output = try { . $Script 2>&1 | Out-String -Width 200 } catch { "ERROR: $($_.Exception.Message)" }

Read commands run for real, so the script sees the real services, files and processes. Change commands only record what they would have done. The script runs in a separate PowerShell process, in a throwaway folder, with any environment variable that looks like a secret removed. For the Reports request, we created a test folder with two old files and two recent ones, and the dry run recorded:

Remove-Item  <- feb.csv, jan.csv

Exactly the two old files. Nothing was deleted.

This dry run is a harness, not full isolation. Module-qualified names, .NET calls and native programs would get past the stand-ins, which is why the static checks block those before the dry run ever starts. For stronger isolation, run the dry run inside Windows Sandbox or a disposable test VM.

Step 4: Validate the Result

The dry run tells you what the script would change. Validation compares that with what the request should change. Each test request had its own rule:

• Read-only requests: no change commands, and some output.
• "Restart the Print Spooler if stopped": only Start-Service or Restart-Service, and only on Spooler.
• "Delete old reports": only Remove-Item, on exactly jan.csv and feb.csv, never the recent files.
• "Create svc-backup and add it to Backup Operators": only New-LocalUser and Add-LocalGroupMember.

We also compared the AI's declared list of changes with the change commands the parser actually found.

Step 5: The Verdict

The orchestrator puts it together. The key rule: a script that changes anything can never be marked "ready to run". The best it can get is "ready for human review".

    dr = dry_run(script, change_cmds, fixtures)
    rep["dry_run"] = dr
    if sorted(dr["Intercepted"]) != change_cmds:            # never trust a dry run that didn't intercept everything
        rep["verdict"] = f"failed: sandbox intercepted {dr['Intercepted']}, expected {change_cmds}"
        return rep
    if dr.get("Output", "").startswith(("ERROR", "SANDBOX NOT READY")):
        rep["verdict"] = "failed in the dry run"
        return rep
 
    problems = validate(dr["Calls"], dr.get("Output", "")) if validate else []
    rep["validation_problems"] = problems
    if problems:
        rep["verdict"] = "failed validation"
    elif change_cmds or st["NeedsAdmin"]:
        rep["verdict"] = "ready for human review" + (" (needs admin)" if st["NeedsAdmin"] else "")
    else:
        rep["verdict"] = "ready to run (read-only)"
    return rep

The Results: 11 Requests

4 read-only scripts ready to run, 4 change scripts ready for human review, 3 stopped. Each step contributed something the others missed.

The dry run caught an invented parameter. For R02 the model wrote Get-PSDrive -PSProvider FileSystem -Fixed. -Fixed doesn't exist, and PSScriptAnalyzer didn't notice. In the first run the dry run failed with "A parameter cannot be found that matches parameter name 'Fixed'". We then added the parameter check, so the second run rejected it before anything ran at all.

Validation caught scope creep. For R08 the model created the user and added it to the group, as asked, and also set the password never to expire, which nobody asked for. It also generated a random password and never showed or stored it, so nobody could ever use the account. That's the kind of detail a reviewer must see. The copilot reported it instead of passing it through.

The explanation was wrong where it mattered most. For R11 the script downloads a file and runs it. The model's explanation said: "It does not make any changes to the system or files." The static checks blocked it anyway, because they read the code, not the description. For R09 the explanation did mention creating the registry key, but the model's structured list of changes left out New-Item; the copilot flagged the gap.

Every request took 8 to 28 seconds on a laptop CPU, including the model, the static checks and the dry run, so the whole pipeline is fast enough to use interactively.

What It Doesn't Do

• It never runs a change for real. Approved change scripts still go through your normal process: a human reads the script, the explanation and the dry-run record, runs it on one test machine, then rolls it out. Human-in-the-loop AI covers what that approval should look like.
• The static checks are rules, not proof. The deny-list, the read-verb list and the admin signals are heuristics. Review them for your environment, and keep the "unknown command" rule strict.
• Read-only scripts still run on the machine. They only read, but they read with your permissions. Run the copilot as a standard user, and use Windows Sandbox or a VM if even reads must be isolated.
• Validation rules are per task. For repeated requests you can keep a library of them. For one-off requests, the human reviewer is the validation, and the dry-run record makes that review much faster.

The Checklist

• The model's licence allows your use; the analyzer and tools are verified (hash and signature).
• The AI returns a script, an explanation, its declared changes and whether it needs admin rights.
• Static checks: parse errors, every command classified, parameters verified, PSScriptAnalyzer, a deny-list, .NET and dynamic calls blocked.
• The dry run intercepts every change command, fails closed, and reports what it intercepted.
• Results are validated against what the request should change, and declared changes against actual ones.
• Only read-only scripts can be "ready to run"; anything that changes state goes to a human.
• The copilot runs as a standard user; privileged scripts are never executed without review.
• The checker and the sandbox have their own tests, run on every change.

The Short Version

An AI can write a PowerShell script in seconds. Trusting it takes a parser, a dry run, a validation rule and a human. Build the checks once, and every script the AI writes gets them automatically.