A bit of fun with PowerShell

I admit this serves no real purpose, but it's something I've had on the back burner for a few years, and by back burner I mean in my head.

A recent challenge at work put all of the components on the table, so I finally took the time to put them together.

The Recent Challenge

My recent (massively oversimplified) challenge was to extract GUIDs from 1.6 TB (14 million) flat files on disk going back 17 years.

No problem I hear you say, use PowerShell get-content with regular expression pattern matching.

I initially tried to use the following regular expression, and it does work with properly formatted GUIDs.

$regex = "[A-Z0-9]{8}-([A-Z0-9]{4}-){3}[A-Z0-9]{12}"

However these files are ADOS files, which contain human readable metadata as well as other binary content, such as an original PDF or PNG file.

There does seem to be a human recognizable GUID towards the end of the file, usually down in the last three lines, but nothing I can easily match with regex for two reasons.

1. There is a space between every character in the GUID.
2. The GUID often spans a new line. CR\LF

I know this is a GUID, but PowerShell doesn't!

I thought about adding a space character to the regular expression

$regex = "[A-Z0-9 ]{16}-([A-Z0-9 ]{8}-){3}[A-Z0-9 ]{24}"

but this won't allow for the break over lines, and the regex special character for new line can't be used within the square brackets, so I'm kind of stuck at this point.

Interestingly I found that using PowerShell -replace " ","" didn't work for me. I could highlight and replace any random character on the screen, but I couldn't trim or replace the spaces, which got me thinking. PowerShell wasn't seeing what I was seeing!

[int][char]"a"

I needed to see what PowerShell was seeing.

I've played with casting to char and int before to get ASCII characters.

Work with me here. Open PowerShell, and try this.

[char]"a"

Nothing happens that you can see. So try this.

[int]"a"

Hmm, that didn't work. "Cannot convert value "a" to type "System.Int32"."

Let's put them together. Let's get the integer value of the char value of "a"

[int][char]"a"

Now we're getting somewhere, and true enough the ASCII value for lower case a is in fact 97. Let's see what a space is.

[int][char]" "

The ASCII value for a space is 32. No surprises.

Try this.

[char]"Hello"

Not so good. "String must be exactly one character long."

I need to work with a longer string, so I need an array to hold the char values.

[char[]]"Hello"

Great, so let's get the ASCII of the char values in another array

[int[]][char[]]"Hello"

So at this point, I can convert text to ASCII.

Now let's see what's contained in the GUID in the file.

I need the third last line of my file, which I'm sure I could get a more elegant way, but for now I'm going with:

$text = cat .\TestFile.Ados -last 3 | Select -first 1

I appreciate you don't have access to the test file, so to keep playing along, copy and paste this PowerShell here string:

$text = @"


                    ☺   ☻   ♥   ♦   ♣   ♠                  þÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿJ   3 5 7 b 9 7 0 b - c 0 d 1 - 4 9 0 5 - b a 3 5 - d 2 b
f c 3 e d 6 7 2 c     ♥   ☺           .ã@


          ☺☺  ☻☺  ♥☺  ♦☺  ♣☺  ♠☺  ☺☺    ☺
"@

Now we can convert the string to ASCII.

[int[]][char[]]$text | % {write-host "$_ " -nonewline}

This will give you an integer array of the values in the char array of the text, and for each element in the array it will write the ASCII character on the screen, separated with a space for readability.

In my case, I get the output below. Now I already know that lower case "a" is 97 and that lower case "b" will be 98 etc.. so I can easily match the GUID to the ASCII by looking for these values.

What's obvious from looking at this, is that the "spaces" I'm seeing on screen are actually ASCII 0, not ASCII 32. ASCII 0 is a null character, which explains why -replace " ","" wouldn't remove it.

If you used the text cut and paste from above, you should be seeing 32s.

OK, this gives me an idea. My Regular Expression for a GUID only contains the following: A-Z, a-z, 0-9 and "-"

What I need to do is get rid of everything else which will remove everything causing me an issue, and still leave me with enough information to match and extract the GUID.

I only need ASCII 45, 48-57, 65-90 and 97 - 122

Let's quickly build an array of the ASCII values we need.

$ascii = 48..57
$ascii += 65..90
$ascii += 97..122
$ascii += 45

Why didn't I start with 45? Try it an see...

So now I have an array of the ASCII values I need, let's build a hash table

$table = @{}

$ascii | % {$table[$_] = [char]$_}

Now I can check each character in my text to see if it potentially fits within the constraints I have for a GUID.

[int[]][char[]]$text | % {

  if ($table.contains($_) ) 
  
  {
    write-host "$($table[$_])" -nonewline 
  }
}

This is now starting to look truly GUID like, and I can do things from here.

OK, so this is where the current work related challenge took a back seat, and I got very distracted.

Open the Pod Bay Doors Please HAL

A few years ago I wrote function I called HAL

function HAL

{

    $text = $input | out-string

    [char[]]$text | % {

    sleep -Milliseconds 75

    write-host -nonewline $_}

}

HAL is very simple, it takes input and echo's it back to the screen, with a 1968's style delay between each character.

Cut and paste it, and then try this.

$text | HAL

Welcome to 1970's Science Fiction.

This also works with other input methods, just as Get-Content (cat)

Get-Content | Do-Something | Set-Content

A few years ago I was working in Manchester Airport Group on a Domino to Office 365 Migration using the Quest product.

The migration process was heavily dependent on Tab Separated Value .tsv files containing the primary SMTP addresses of the Office 365 mailboxes in the migration batch. These files were used both by the product set, and also for scripted tasks and automation outside of the product itself.

We quickly found that extra spaces or empty lines caused issues with scripting, so I used a workaround, where I trimmed leading and trailing spaces and blank lines from the input files, and then overwrote the original files in the process.

(gc $inputFile) |  % {$_.TrimEnd()} | ? {$_ -ne ""} | sc $inputFile

This get-content | do-things | set-content is the final piece of the puzzle, and now we have all the components we need to get creative.

We can convert to and from ASCII, read input and write output to the screen, read and write content with get-content and set-content, and we can manipulate the content on the fly.

This is where the fun starts

One of the first things I tried was to convert content to ASCII, increment the ASCII and then convert it back.

function Scramble
	{
	    $text = $input | Out-String 
		[int[]][char[]]$text | % { $a = $_ + 1
			$b = [char]$a
			write-host -nonewline "$b" -f Green
		}
	}

Copy this

$test = @"

I thought about adding a space character to the regular expression, but this won't allow for the break over lines, and the regex special character for new line can't be used within the square brackets, so I'm kind of stuck at this point.

Interestingly using PowerShell -replace " ","" didn't work for me. I could highlight and replace any random character on the screen, but I couldn't trim or replace the spaces, which got me thinking. PowerShell wasn't seeing what I was seeing!

"@

Now try this

$test | scramble

And for the record, converting it back is just as easy, we just decrement.

$revert = @"
J!uipvhiu!bcpvu!beejoh!b!tqbdf!dibsbdufs!up!uif!sfhvmbs!fyqsfttjpo-!cvu!uijt!xpo(u!bmmpx!gps!uif!c
sfbl!pwfs!mjoft-!boe!uif!sfhfy!tqfdjbm!dibsbdufs!gps!ofx!mjof!dbo(u!cf!vtfe!xjuijo!uif!trvbsf!csbd
lfut-!tp!J(n!ljoe!pg!tuvdl!bu!uijt!qpjou/♂Joufsftujohmz!vtjoh!QpxfsTifmm!.sfqmbdf!#!#-##!ejeo(u!xp
sl!gps!nf/!J!dpvme!ijhimjhiu!boe!sfqmbdf!boz!sboepn!dibsbdufs!po!uif!tdsffo-!cvu!J!dpvmeo(u!usjn!p
s!sfqmbdf!uif!tqbdft-!xijdi!hpu!nf!uijoljoh/!QpxfsTifmm!xbto(u!tffjoh!xibu!J!xbt!tffjoh"♫♂
"@

Don't get hung up on the formatting, CR\LFs get introduced with the cut and paste.

OK, what about converting to something else, like binary?

function scram
{
    $text = $input | out-string
    [int[]][char[]]$text | % {
    write-host -NoNewline "$([Convert]::ToString($_,2).PadLeft(8,"0"))" -f Green}
} 

This converts every character of the input text into binary, and pads the binary out to 8 bits.

Padding is important otherwise it would be impossible to ever decipher the text, as you would have no idea if 10001 is a 17, or 8 and a 1.

$test | scram

Now if I add the Set-Content, this get's as clever or as dangerous as you want it to me.

I could in theory:

1. Read the file contents.
2. Convert it to ASCII, or binary.
3. Increment, decrement or otherwise obfuscate the ASCII or binary content.
4. Overwrite the original file with the new content.

Unless somebody knows what manipulation we have done with the content on the fly, (if we've incremented or decremented and by how much) there is no easy way to decipher the new file content.

I've commented out the Set-Content line in the function below, so you can't accidentally cut paste and do any damage.

If you decide to play with this, PLEASE test it against copies of files, don't do it against the originals.

function scramble()
{
        param ( [String] $filename,
                [switch] $show)
        $text= gc $filename | out-string
        [int[]][char[]]$text | % {
                $a = $_
                $a ++
                $c = [Convert]::ToString($a ,2).PadLeft(8,"0")
                if ($show)
                {
                     write-host "$c" -F green -nonewline
                }
        } # | sc $filename -nonewline
}

Let's take a script I've been working on recently for a SharePoint Online project.

And let's look at it after we convert each character to ASCII, increment the ASCII, convert to binary, then overwrite the file.

It's very basic.

It's builds on some very simple components, but it's quite a nifty and very quick way of converting plain text into basically pretty indecipherable binary.

I haven't exactly worked out how to convert back from binary yet, I got bored at this point and moved on to other things :-)