Use PowerShell
The Shell Is Calling
The Shell Is Calling
Nov 13th
I’ve been exploring the Sync Framework for use in a couple of projects I have going and PowerShell is my preferred exploratory environment.
It was a bit of fun, since I got to work with eventing for the first time in V2.
First, I downloaded the Sync Framework Software Development Kit. That provided me with the Sync Framework runtime as well as some documentation.
The easiest way for me to get started was to take one of the samples and convert that to PowerShell.
I’m going to walk along the MSDN Sample and provide the equivalent PowerShell, as well as any changes I made to make it feel more PowerShell-y.
We are working with the File Sync Provider First up is setting the FileSyncOptions. FileSyncOptions are an enumeration (a limited list defined in code that maps to certain values) whose values are controlled by setting the appropriate bits to indicate the presence or absence of a flag. Mark Schill has a great post about how to set bitwise operations.
$options = [Microsoft.Synchronization.Files.FileSyncOptions]::ExplicitDetectChanges $options = $options -bor [Microsoft.Synchronization.Files.FileSyncOptions]::RecycleDeletedFiles $options = $options -bor [Microsoft.Synchronization.Files.FileSyncOptions]::RecyclePreviousFileOnUpdates $options = $options -bor [Microsoft.Synchronization.Files.FileSyncOptions]::RecycleConflictLoserFiles
With the File System provider, we can provide filters to include or exclude files and directories.
$FileNameFilter and $SubdirectoryNameFilter are parameters that take strings or string arrays.
$filter = New-Object Microsoft.Synchronization.Files.FileSyncScopeFilter if ($FileNameFilter.count -gt 0) { $FileNameFilter | ForEach-Object { $filter.FileNameExcludes.Add($_) } } if ($SubdirectoryNameFilter.count -gt 0) { $SubdirectoryNameFilter | ForEach-Object { $filter.SubdirectoryExcludes.Add($_) } }
After configuring the filter, we examine the folders and files located at the paths specified. If there has not been any previous synchronization, a metadata file will be created in each location to track any changes, updates, and deletes for later synchronization.
function Get-FileSystemChange() { param ($path, $filter, $options) try { $provider = new-object Microsoft.Synchronization.Files.FileSyncProvider -ArgumentList $path, $filter, $options $provider.DetectChanges() } finally { if ($provider -ne $null) { $provider.Dispose() } } }
Get-FileSystemChange $SourcePath $filter $options Get-FileSystemChange $DestinationPath $filter $options
Conflict resolution in the Sync Framework happens at the at the event level. An event is merely something that happens that can trigger other actions. Using Register-ObjectEvent, we can associate one or more scriptblocks with an event.
First, I defined scriptblocks to handle the conflicts. There is an enumeration, the ConflictResolutionAction enumeration, that provides some options for dealing with conflicts. For this example, we are going to pick the source object as the winner for any conflicts.
You will also notice another type of conflict defined, and that is a Constraint conflict. That can occur when an object of the same name is added on both sides in between synchronizations. The resolution options for these conflicts can be found in the ConstraintConflictResolutionAction enumeration.
$ItemConflictAction = { $event.SourceEventArgs.SetResolutionAction([Microsoft.Synchronization.ConflictResolutionAction]::SourceWins) [string[]]$global:FileSyncReport.Conflicted += $event.SourceEventArgs.DestinationChange.ItemId } $ItemConstraintAction = { $event.SourceEventArgs.SetResolutionAction([Microsoft.Synchronization.ConstraintConflictResolutionAction]::SourceWins) [string[]]$global:FileSyncReport.Constrained += $event.SourceEventArgs.DestinationChange.ItemId } # Configure the events for conflicts or constraints for the source and destination providers $destinationCallbacks = $destinationProvider.DestinationCallbacks Register-ObjectEvent -InputObject $destinationCallbacks -EventName ItemConflicting -Action $ItemConflictAction | Out-Null Register-ObjectEvent -InputObject $destinationCallbacks -EventName ItemConstraint -Action $ItemConstraintAction | Out-Null $sourceCallbacks = $SourceProvider.DestinationCallbacks Register-ObjectEvent -InputObject $sourceCallbacks -EventName ItemConflicting -Action $ItemConflictAction | Out-Null Register-ObjectEvent -InputObject $sourceCallbacks -EventName ItemConstraint -Action $ItemConstraintAction | Out-Null
We also see for the first time in the script blocks a variable called $event. This is an automatic variable exposed by the event and provides us information that we can use in our action.
Finally, I’m updating a variable in the global scope. There probably is a better way to handle this, but scriptblocks executed in response to events only have access to the global scope and any of the automatic variable exposed to it. Therefore, I use a variable in the global scope to gather my reporting information.
To start to synchronize the two sides, first we set up the synchronization via a SyncOrchestrator and assign it the local and remote providers, as well as defining the direction of the synchronization. In this example (sticking with the format from MSDN, we will do an Upload, which is in the SyncDirectionOrder enumeration (other options are Download, DownloadAndUpload, and UploadAndDownload).
# Create the agent that will perform the file sync $agent = New-Object Microsoft.Synchronization.SyncOrchestrator $agent.LocalProvider = $sourceProvider $agent.RemoteProvider = $destinationProvider # Upload changes from the source to the destination. $agent.Direction = [Microsoft.Synchronization.SyncDirectionOrder]::Upload Write-Host "Synchronizing changes from $($sourceProvider.RootDirectoryPath) to replica: $($destinationProvider.RootDirectoryPath)" $agent.Synchronize();
To achieve two way synchronization, we will do the upload twice, reversing the order of the providers.
Invoke-OneWayFileSync -SourcePath $SourcePath -DestinationPath $DestinationPath -Filter $null -Options $options Invoke-OneWayFileSync -SourcePath $DestinationPath -DestinationPath $SourcePath -Filter $null -Options $options
I modified the example to write out a custom object (and the logging is in the variable in the global scope as noted in the Handling Conflicts section) with the results of the synchronization (rather than logging it to the console).
In all, my translation is pretty similar to the example code, but there are some differences.
# Requires -Version 2 # Also depends on having the Microsoft Sync Framework 2.0 SDK or Runtime # --SDK-- # http://www.microsoft.com/downloads/details.aspx?FamilyID=89adbb1e-53ff-41b5-ba17-8e43a2e66254&displaylang=en # --Runtime-- # http://www.microsoft.com/downloads/details.aspx?FamilyId=109DB36E-CDD0-4514-9FB5-B77D9CEA37F6&displaylang=en # # [CmdletBinding(SupportsShouldProcess=$true)] param ( [Parameter(Position=1, Mandatory=$true, ValueFromPipelineByPropertyName=$true)] [Alias('FullName', 'Path')] [string]$SourcePath , [Parameter(Position=2, Mandatory=$true)] [string]$DestinationPath , [Parameter(Position=3)] [string[]]$FileNameFilter , [Parameter(Position=4)] [string[]]$SubdirectoryNameFilter ) <# .Synopsis Synchronizes to directory trees .Description Examines two directory structures (SourcePath and DestinationPath) and uses the Microsoft Sync Framework File System Provider to synchronize them. .Example An example of using the command #> begin { [reflection.assembly]::LoadWithPartialName('Microsoft.Synchronization') | Out-Null [reflection.assembly]::LoadWithPartialName('Microsoft.Synchronization.Files') | Out-Null function Get-FileSystemChange() { param ($path, $filter, $options) try { $provider = new-object Microsoft.Synchronization.Files.FileSyncProvider -ArgumentList $path, $filter, $options $provider.DetectChanges() } finally { if ($provider -ne $null) { $provider.Dispose() } } } function Invoke-OneWayFileSync() { param ($SourcePath, $DestinationPath, $Filter, $Options) $ApplyChangeJobs = @() $AppliedChangeJobs = @() try { # Scriptblocks to handle the events raised during synchronization $AppliedChangeAction = { $argument = $event.SourceEventArgs switch ($argument.ChangeType) { { $argument.ChangeType -eq [Microsoft.Synchronization.Files.ChangeType]::Create } {[string[]]$global:FileSyncReport.Created += $argument.NewFilePath} { $argument.ChangeType -eq [Microsoft.Synchronization.Files.ChangeType]::Delete } {[string[]]$global:FileSyncReport.Deleted += $argument.OldFilePath} { $argument.ChangeType -eq [Microsoft.Synchronization.Files.ChangeType]::Update } {[string[]]$global:FileSyncReport.Updated += $argument.OldFilePath} { $argument.ChangeType -eq [Microsoft.Synchronization.Files.ChangeType]::Rename } {[string[]]$global:FileSyncReport.Renamed += $argument.OldFilePath} } } $SkippedChangeAction = { [string[]]$global:FileSyncReport.Skipped += $event.SourceEventArgs.CurrentFilePath if ($event.SourceEventArgs.Exception -ne $null) { Write-Error '[' + "$($event.SourceEventArgs.Exception.Message)" +']' } } # Create source provider and register change events for it $sourceProvider = New-Object Microsoft.Synchronization.Files.FileSyncProvider -ArgumentList $SourcePath, $filter, $options $AppliedChangeJobs += Register-ObjectEvent -InputObject $SourceProvider -EventName AppliedChange -Action $AppliedChangeAction $AppliedChangeJobs += Register-ObjectEvent -InputObject $SourceProvider -EventName SkippedChange -Action $SkippedChangeAction $ApplyChangeJobs += $SourceApplyChangeJob # Create destination provider and register change events for it $destinationProvider = New-Object Microsoft.Synchronization.Files.FileSyncProvider -ArgumentList $DestinationPath, $filter, $options $AppliedChangeJobs += Register-ObjectEvent -InputObject $destinationProvider -EventName AppliedChange -Action $AppliedChangeAction $AppliedChangeJobs += Register-ObjectEvent -InputObject $destinationProvider -EventName SkippedChange -Action $SkippedChangeAction $ApplyChangeJobs += $DestApplyChangeJob # Use scriptblocks for the SyncCallbacks for conflicting items. $ItemConflictAction = { $event.SourceEventArgs.SetResolutionAction([Microsoft.Synchronization.ConflictResolutionAction]::SourceWins) [string[]]$global:FileSyncReport.Conflicted += $event.SourceEventArgs.DestinationChange.ItemId } $ItemConstraintAction = { $event.SourceEventArgs.SetResolutionAction([Microsoft.Synchronization.ConstraintConflictResolutionAction]::SourceWins) [string[]]$global:FileSyncReport.Constrained += $event.SourceEventArgs.DestinationChange.ItemId } #Configure the events for conflicts or constraints for the source and destination providers $destinationCallbacks = $destinationProvider.DestinationCallbacks $AppliedChangeJobs += Register-ObjectEvent -InputObject $destinationCallbacks -EventName ItemConflicting -Action $ItemConflictAction $AppliedChangeJobs += Register-ObjectEvent -InputObject $destinationCallbacks -EventName ItemConstraint -Action $ItemConstraintAction $sourceCallbacks = $SourceProvider.DestinationCallbacks $AppliedChangeJobs += Register-ObjectEvent -InputObject $sourceCallbacks -EventName ItemConflicting -Action $ItemConflictAction $AppliedChangeJobs += Register-ObjectEvent -InputObject $sourceCallbacks -EventName ItemConstraint -Action $ItemConstraintAction # Create the agent that will perform the file sync $agent = New-Object Microsoft.Synchronization.SyncOrchestrator $agent.LocalProvider = $sourceProvider $agent.RemoteProvider = $destinationProvider # Upload changes from the source to the destination. $agent.Direction = [Microsoft.Synchronization.SyncDirectionOrder]::Upload Write-Host "Synchronizing changes from $($sourceProvider.RootDirectoryPath) to replica: $($destinationProvider.RootDirectoryPath)" $agent.Synchronize(); } finally { # Release resources. if ($sourceProvider -ne $null) {$sourceProvider.Dispose()} if ($destinationProvider -ne $null) {$destinationProvider.Dispose()} } } # Set options for the synchronization session. In this case, options specify # that the application will explicitly call FileSyncProvider.DetectChanges, and # that items should be moved to the Recycle Bin instead of being permanently deleted. $options = [Microsoft.Synchronization.Files.FileSyncOptions]::ExplicitDetectChanges $options = $options -bor [Microsoft.Synchronization.Files.FileSyncOptions]::RecycleDeletedFiles $options = $options -bor [Microsoft.Synchronization.Files.FileSyncOptions]::RecyclePreviousFileOnUpdates $options = $options -bor [Microsoft.Synchronization.Files.FileSyncOptions]::RecycleConflictLoserFiles } process { $filter = New-Object Microsoft.Synchronization.Files.FileSyncScopeFilter if ($FileNameFilter.count -gt 0) { $FileNameFilter | ForEach-Object { $filter.FileNameExcludes.Add($_) } } if ($SubdirectoryNameFilter.count -gt 0) { $SubdirectoryNameFilter | ForEach-Object { $filter.SubdirectoryExcludes.Add($_) } } # Perform the detect changes operation on the two file locations Get-FileSystemChange $SourcePath $filter $options Get-FileSystemChange $DestinationPath $filter $options # Reporting Object - using the global scope so that it can be updated by the event scriptblocks. $global:FileSyncReport = New-Object PSObject | Select-Object SourceStats, DestinationStats, Created, Deleted, Overwritten, Renamed, Skipped, Conflicted, Constrained # We don't need to pass any filters here, since we are using the file detection that was previously completed. # this will only $global:FileSyncReport.SourceStats = Invoke-OneWayFileSync -SourcePath $SourcePath -DestinationPath $DestinationPath -Filter $null -Options $options $global:FileSyncReport.DestinationStats = Invoke-OneWayFileSync -SourcePath $DestinationPath -DestinationPath $SourcePath -Filter $null -Options $options # Write result to pipeline Write-Output $global:FileSyncReport }
Jul 21st
I started looking a little deeper at error handling in PowerShell after this StackOverflow question.
PowerShell has two kinds of errors – terminating errors and non-terminating errors.
Terminating errors are the errors that can stop command execution cold. Non-terminating errors provided an additional challenge, as you need to be notified of failed operations and continue with pipeline operations. To deal with this issue and to provide additional output options, PowerShell employs the concept of streams. There are three additional streams (other than the primary pipeline stream) available in PowerShell – Verbose, Warning, and Error (and . We’ll be concerning ourselves with the output from the Error stream.
There are some automatic variables in the shell that deal with errors as well. $ErrorActionPreference is a variable that describes how PowerShell will treat non-terminating errors. $ErrorActionPreference has four options: “Continue” (the default), “SilentlyContinue”, “Inquire”, and “Stop”. $error is an array of all the errors that have occurred in the current session up to the number specified in $MaximumErrorCount. $? is a boolean value that is $true if the previous operation succeeded and $false if it did not.
PowerShell does have some built in flexibility to turn non-terminating errors into terminating errors. Based on our setting of $ErrorActionPreference, PowerShell will treat a non-terminating error differently. If $ErrorActionPreference is set to ‘”Stop”, PowerShell will treat the errors from that scope and all sub scopes (unless explicitly overridden) as terminating errors. If “Inquire” is chosen as the $ErrorActionPreference, then PowerShell will prompt the user at the console upon each error to ask whether it should continue (treat as non-terminating) or halt (treat as terminating), as well as providing an option to drop into a nested prompt.
$ErrorActionPreference is a global preference that can be set, but the PowerShell runtime also allows more granular control by providing common parameters for -ErrorAction and -ErrorVariable (shorthand -EA and –EV) which allow you to determine per cmdlet (and in V2 per advanced function) how errors should be handled. The –ErrorAction parameter takes the same options as $ErrorActionPreference.
So what happens when you encounter an error?
If you hit a terminating error, your script, function, or command will stop. The error will be logged to the $error variable and written to the error stream. The $? will be set to false. You can use the trap construct (or the try/catch/finally construct in V2) to deal with terminating errors and we’ll cover that further in the fourth post in this series.
If you hit a non-terminating error, the result will depend on the $ErrorActionPreference in the scope or the –ErrorAction parameter. If the $ErrorActionPreference or –ErrorAction parameter is set to ‘Continue’, the error will be written to the error stream, added to the $error array, and the $? will be set to $false. ‘SilentlyContinue’ will not write an error to the Error stream, but $error and $? are updated. Finally, ‘Inquire’ provides you an option at each error as to whether to treat it as terminating or non-terminating. If treated as a non-terminating error, the error will be treated like the $ErrorActionPreference or –ErrorAction parameter is ‘Continue’.
Now, I’ve mentioned that some terminating and non-terminating errors both write to the Error stream, so where does that actually get reported back to the user? If an error has been written to the error stream, it will be written to the output stream (which may surface differently based on the PowerShell host) separately from the pipeline output unless it is redirected. This means that the exception objects, which are how the errors are represented, are not saved to a variable if you are assigning the pipeline output to a variable.
For example:
$results = Get-WMIObject Win32_Bios –ComputerName localhost, ComputerThatDoesNotExist
Get-WmiObject : The RPC server is unavailable. (Exception from HRESULT: 0x800706BA)
At line:1 char:22
+ $results = Get-WmiObject <<<< Win32_Bios -ComputerName localhost, ComputerThatDoesNotExist
creates this output to the screen, but $results will only hold the results of the successful WMI query. This is where the –ErrorVariable comes in. You can specify a variable to hold the exceptions generated by a particular cmdlet (or in V2 advanced function). Specifying a variable does not remove an item from being written to the error stream and displayed as output.
In a console session, you can redirect the error stream with the redirection operators ( 2> or 2>>) or you can merge the error stream with the output stream (2>&1) and write the output to a file, the pipeline, or assign it to a variable.
Up next:
Other great PowerShell Error Handling posts:
Jul 7th
Garbage collection is a process that the .NET Framework (upon which the PowerShell runtime works) uses to manage memory. The garbage collection (for applications – services are handled a bit differently) process basically covers X steps:
The first step is the critical point for PowerShell users. You may have noticed how the memory used in a PowerShell session can steadily climb (or suddenly climb if you’re dealing with a large number of objects). I talked about that in this tip. The .NET garbage collector looks for objects that are still in use or could possibly be in use in the near future (this is a gross oversimplification – if you are really interested in a detailed explanation of how garbage collection works in the .NET Framework, check out this article.)
What this means to scripters who are running into memory pressure issues is that the garbage collector will not reclaim any objects you have actively referenced (meaning that you’ve assigned them to a variable) in your current or higher scope.
Since this can sound a bit convoluted let’s look at an example.
I read a large file to parse it and leave the contents of the original stored in a variable
PS> $MyLargeFile = Get-Content MyLargeFile.txt
PS> Foreach ($line in $MyLargeFile) {
DoSomething $line | Out-File NewUpdatedLargeFile.txt
}
One of the problems with the above pattern is that the original file remains saved in memory and doesn’t get released unless you explicitly remove the reference, either by using Remove-Variable or Remove-Item or by changing what $MyLargeFile points to by assigning another value to it.
This isn’t VBScript where you have to set values to “Nothing”, but you should be aware of what larger object sets you are retaining in memory.
Apr 27th
One concept that is central to the understanding of the layout of classes in the Base Class Libraries is the Namespace. Namespaces are a method of providing context for classes (and other constructs like Enums and Structs), allowing developers to group related classes and not worry about name collisions with development in other areas.
In the Base Class Libraries, the root namespace is the System namespace. The System namespace defines a large amount of the base types, like Object (which all classes derive from), String, DateTime, Boolean, and numerous others. When you specify types, PowerShell can infer the “System” part.
Example:
PS C:\scripts\PowerShell> $random = New-Object –TypeName Random
is the same as
PS C:\scripts\PowerShell> $random = New-Object –TypeName System.Random
Namespaces are hierarchical, except in naming convention. The representation of a file in PowerShell is a great example. A file object is represented in PowerShell as a System.IO.FileInfo object. The FileInfo class lives in the System.IO namespace, which is a separate namespace than the System namespace. There is a hierarchical impression implied, but that is solely through naming convention.
The CLR (Common Language Runtime) does not care about namespaces, only the fully qualified name of the class, so why use namespaces at all?
Namespaces are a convenience for developers, as languages like C# and VB.NET have language constructs that allow the developer to not have to use the fully qualified type name every time that is needed to be referenced. C# provides the “using” syntax and VB.NET has the “imports” syntax.
How does this apply to PowerShell?
PowerShell does not currently have a “using” or “imports” syntax or language feature, but there are a couple of workarounds. You can use variables holding the namespace and string concatenation to save some typing
PS C:\scripts\PowerShell> $netinfo = ‘System.Net.NetworkInformation’
PS C:\scripts\PowerShell> $ping = New-Object “$netinfo.ping”
PS C:\scripts\PowerShell> $ping.gettype()
IsPublic IsSerial Name BaseType
——– ——– —- ——–
True False Ping System.ComponentModel.Component
This doesn’t work with type accelerators though (at least in V1).. V2 CTP3 has some other options and Oisin Grehan has a great discussion on how to find the type accelerators and add your own.
Another option would be to create a function to wrap the creation of objects for particular namespaces.
Use-Namespace is a function I came up with to create a function to wrap New-Object for specific namespaces.
The useage of Use-Namespace is
PS C:\scripts\PowerShell>Use-Namespace System.IO, System.Net
PS C:\scripts\PowerShell>$memorystream = New-System.IOObject MemoryStream
PS C:\scripts\PowerShell>$memorystream.GetType()
PS C:\scripts\PowerShell> $memorystream.gettype()
IsPublic IsSerial Name BaseType
——– ——– —- ——–
True True MemoryStream System.IO.Stream
And the function is here..
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | function Use-Namespace() { param ([string[]]$Namespace) BEGIN { if ($Namespace.length -gt 0) { $Namespace | Use-Namespace } } PROCESS { if ($_ -ne $null) { $NS = $_ $NSObject = "$NS" + "Object" $function = @" param (`$Class, [string[]]`$ArgumentList) if (`$ArgumentList.length -gt 0) { return New-Object -TypeName "$NS.`$Class" -ArgumentList `$ArgumentList } else { return New-Object -TypeName "$NS.`$Class" } "@ Set-Item -Path "function:global:New-$NSObject" -Value $function -force Write-Debug "Created function:global:New-$NSObject " } } } |
PSMDTAG:FAQ CLR
PSMDTAG:FAQ Namespace
PSMDTAG:FAQ .NET Framework