One of my personal goals with my new position at Microsoft is to learn more about Windows and its ecosystem. As you know, I’m a systems person and I’ve been in the Unix and macOS realm for the last… 22 years or so. During this time, I’ve certainly touched Windows here and there but, for the most part, a lot of it remains a mystery to me.

Yesterday already, I uncovered a couple of interesting things worth discussing. In this post I’m going to focus on just one of those two, and that is: how PowerShell handles input parameters in its scripts. I’ll be looking at this through the lenses of a long-time Unix user and I’ll hopefully be posting more articles of this style in the future. Follow me, wink, wink.

The Unix way

Parsing flags gets old very, very quickly. I have written code to parse flags and arguments countless times in a multitude of languages. And no matter what, every time I have to do this I have to look at the getopts description in the sh(1) manual page for shell scripts, the getopt(3) and getopt_long(3) manual pages for C and C++ programs, and any other kind of documentation for the language of the day.

Now, of course there are plenty of libraries to simplify flags parsing. You just have to hit a rock and uncover at least 10. However, no matter what you do in your code, the shell is oblivious to what’s going on under your program.

Say you have a script named example.sh with the following contents:

#! /bin/sh

force=false
path=
while getopts fp: flag; do
    case "${flag}" in
        f) force=true ;;
        p) path="${OPTARG}" ;;
        \?) exit 1 ;;
    esac
done
shift $((${OPTIND} - 1))

[ -n "${path}" ] || { echo "Path (-p) is empty" 1>&2; exit 1; }

echo "Force: ${force}"
echo "Path: ${path}"

When we run the script above from the shell, we get an experience like this:

$ ./example.sh
Path (-p) is empty

$ ./example.sh -h
Illegal option -h

$ ./example.sh -p some/path
Force: false
Path: some/path

$ ./example.sh -p some/path -f
Force: true
Path: some/path

For every command invocation above, the shell:

1. tokenized the command line to extract words, and it did so using the IFS environment variable (have you ever heard of this?);
2. looked for the executable in the given location because the program name contained a slash, else it would have checked its in-memory cache of programs (did you know about the rehash or hash builtins?) and fallen back to scanning the PATH; and
3. executed the program with the arguments as they were, in a subprocess.

So far so good: there is no magic. The words we typed were exactly what the executed program got (after tokenization), and it did not matter that example.sh was a shell script: the exact same thing would have happened if we had run a native program like ls(1).

Once running, though, our example.sh script had to scan the arguments, find that -f and -p looked like options, interpret them, and do something with them. What you should notice is that there were two erroneous executions: one where we forgot to specify a required flag (-p) and one where we used an unknown flag (-h), and both of these required special handling in our code.

And this is were is where things get more interesting.

What if you wanted the shell to support autocompletion for the flags that the program supports? You can do that, of course—if the shell supports such a feature. I bet that, if you navigate to /etc/bash_completion.d/ on your machine, you’ll find some stuff in there. Those are Bash-specific scripts that Bash loads and uses to extend its autocompletion features. The content of these scripts is usually awful-looking though, and I remember having to recurrently fight problems in the implementation we had in Bazel…

I digress. The point I want you to remember is that programs run by the shell are black boxes, no matter if they are scripts or arbitrary native binaries.

The PowerShell way

In PowerShell, you can run arbitrary programs, yes, but you can also run cmdlets. A cmdlet is, essentially, an extension point for PowerShell. Builtin commands are cmdlets, but you can define your own in a variety of languages. Cmdlets can be written in C# and VB.NET… and, of course, they can be written in PowerShell’s own language too. In this latter case, we can call them scripts.

The thing that differentiates arbitrary programs from cmdlets is that when you run programs, PowerShell knows nothing about them. As a result, the shell essentially replicates “the Unix way” of running the called program as a black box. But when you run cmdlets, PowerShell knows stuff about them and can do pretty interesting things with that information.

Let’s take a look at an equivalent example named example.ps1:

#! /opt/microsoft/powershell/7/pwsh

param(
    [Parameter(HelpMessage="Forces execution")]
    [switch]$Force = $False,

    [Parameter(Mandatory, HelpMessage="The path to affect")]
    [string]$Path
)

Write-Output "Force: $Force"
Write-Output "Path: $Path"

(Yes, I installed PowerShell within my Debian WSL instance; hence the #! header.) Here is what happens when we run this script with the same sample cases as above:

> example.ps1
cmdlet example.ps1 at command pipeline position 1
Supply values for the following parameters:
(Type !? for Help.)
Path: !?
The path to affect
Path: some/path
Force: False
Path: some/path

> example.ps1 -?
example.ps1 [-Path] <string> [-Force] [<CommonParameters>]

> example.ps1 -Path some/path
Force: False
Path: some/path

> example.ps1 -Path some/path -Force
Force: True
Path: some/path

Note the differences? Here are the key ones:

• When we ran the script without arguments, the shell figured out which ones were missing and interactively asked us for values. We could even type !? at the prompt of each parameter to access the built-in help strings.
• When we ran the script with the standard -? flag, the shell auto-generated a help string based on the parameter definitions.
• When we specified all required parameters as arguments, the script ran unattended (without stopping to ask for user input).
• And when we tried to autocomplete the flag names (not pictured, sorry), the shell was able to do so.

Every cmdlet gets these integration features, and the syntax for defining how parameters behave is much richer than what I showed here. For example, you can even attach full docstrings with inline markup, and these are then exposed as part of the interactive help for the cmdlet. Peek through the documentation for some more details.

The question is, then: how does this all work? How can PowerShell be aware of the flags within the script before executing it? With metadata!

WARNING: From this point on I’m speculating based on the few things I’ve seen so far while researching this post, so I might be wrong. Please let me know if you find any lies.

You see, if you opened the documentation I linked to above, you’ll notice that param() is an annotation for functions: this statement must come first within a function definition, and the contents of the param() list get attached to the function. If you are the shell and you have such information attached to a function object, then you can do whatever you want with that metadata before running any of the function’s code. You can detect missing flags, validate their contents, or offer autocompletion.

“But, but, but: that example.ps1 does not contain any function definition!” Right. I can imagine that, in such a case, the shell just treats the whole file as if it were a function and expects param() to come first. In fact, it’s a syntax error if it doesn’t. With that assumption, the shell can read the parameters list first if it is present and do whatever it wants with it. It might even be reading the whole file first and putting its contents into an anonymous function object, I don’t know.

As for cmdlets implemented in C# or VB.NET, things have to be different though because they are native managed code. But that should be no problem. Managed code has great introspection features, so it’s perfectly feasible for PowerShell to extract such information from class/function annotations from the code.

And that’s all for now. As a long-time Unix user, I found this quite an interesting difference in paradigms, and I can start to see how cmdlet pipelines can be very powerful. More surprises in future posts!