Use PowerShell
The Shell Is Calling
The Shell Is Calling
I’ve been exploring the Sync Framework for use in a couple of projects I have going and PowerShell is my preferred exploratory environment.
It was a bit of fun, since I got to work with eventing for the first time in V2.
First, I downloaded the Sync Framework Software Development Kit. That provided me with the Sync Framework runtime as well as some documentation.
The easiest way for me to get started was to take one of the samples and convert that to PowerShell.
I’m going to walk along the MSDN Sample and provide the equivalent PowerShell, as well as any changes I made to make it feel more PowerShell-y.
We are working with the File Sync Provider First up is setting the FileSyncOptions. FileSyncOptions are an enumeration (a limited list defined in code that maps to certain values) whose values are controlled by setting the appropriate bits to indicate the presence or absence of a flag. Mark Schill has a great post about how to set bitwise operations.
$options = [Microsoft.Synchronization.Files.FileSyncOptions]::ExplicitDetectChanges $options = $options -bor [Microsoft.Synchronization.Files.FileSyncOptions]::RecycleDeletedFiles $options = $options -bor [Microsoft.Synchronization.Files.FileSyncOptions]::RecyclePreviousFileOnUpdates $options = $options -bor [Microsoft.Synchronization.Files.FileSyncOptions]::RecycleConflictLoserFiles
With the File System provider, we can provide filters to include or exclude files and directories.
$FileNameFilter and $SubdirectoryNameFilter are parameters that take strings or string arrays.
$filter = New-Object Microsoft.Synchronization.Files.FileSyncScopeFilter if ($FileNameFilter.count -gt 0) { $FileNameFilter | ForEach-Object { $filter.FileNameExcludes.Add($_) } } if ($SubdirectoryNameFilter.count -gt 0) { $SubdirectoryNameFilter | ForEach-Object { $filter.SubdirectoryExcludes.Add($_) } }
After configuring the filter, we examine the folders and files located at the paths specified. If there has not been any previous synchronization, a metadata file will be created in each location to track any changes, updates, and deletes for later synchronization.
function Get-FileSystemChange() { param ($path, $filter, $options) try { $provider = new-object Microsoft.Synchronization.Files.FileSyncProvider -ArgumentList $path, $filter, $options $provider.DetectChanges() } finally { if ($provider -ne $null) { $provider.Dispose() } } }
Get-FileSystemChange $SourcePath $filter $options Get-FileSystemChange $DestinationPath $filter $options
Conflict resolution in the Sync Framework happens at the at the event level. An event is merely something that happens that can trigger other actions. Using Register-ObjectEvent, we can associate one or more scriptblocks with an event.
First, I defined scriptblocks to handle the conflicts. There is an enumeration, the ConflictResolutionAction enumeration, that provides some options for dealing with conflicts. For this example, we are going to pick the source object as the winner for any conflicts.
You will also notice another type of conflict defined, and that is a Constraint conflict. That can occur when an object of the same name is added on both sides in between synchronizations. The resolution options for these conflicts can be found in the ConstraintConflictResolutionAction enumeration.
$ItemConflictAction = { $event.SourceEventArgs.SetResolutionAction([Microsoft.Synchronization.ConflictResolutionAction]::SourceWins) [string[]]$global:FileSyncReport.Conflicted += $event.SourceEventArgs.DestinationChange.ItemId } $ItemConstraintAction = { $event.SourceEventArgs.SetResolutionAction([Microsoft.Synchronization.ConstraintConflictResolutionAction]::SourceWins) [string[]]$global:FileSyncReport.Constrained += $event.SourceEventArgs.DestinationChange.ItemId } # Configure the events for conflicts or constraints for the source and destination providers $destinationCallbacks = $destinationProvider.DestinationCallbacks Register-ObjectEvent -InputObject $destinationCallbacks -EventName ItemConflicting -Action $ItemConflictAction | Out-Null Register-ObjectEvent -InputObject $destinationCallbacks -EventName ItemConstraint -Action $ItemConstraintAction | Out-Null $sourceCallbacks = $SourceProvider.DestinationCallbacks Register-ObjectEvent -InputObject $sourceCallbacks -EventName ItemConflicting -Action $ItemConflictAction | Out-Null Register-ObjectEvent -InputObject $sourceCallbacks -EventName ItemConstraint -Action $ItemConstraintAction | Out-Null
We also see for the first time in the script blocks a variable called $event. This is an automatic variable exposed by the event and provides us information that we can use in our action.
Finally, I’m updating a variable in the global scope. There probably is a better way to handle this, but scriptblocks executed in response to events only have access to the global scope and any of the automatic variable exposed to it. Therefore, I use a variable in the global scope to gather my reporting information.
To start to synchronize the two sides, first we set up the synchronization via a SyncOrchestrator and assign it the local and remote providers, as well as defining the direction of the synchronization. In this example (sticking with the format from MSDN, we will do an Upload, which is in the SyncDirectionOrder enumeration (other options are Download, DownloadAndUpload, and UploadAndDownload).
# Create the agent that will perform the file sync $agent = New-Object Microsoft.Synchronization.SyncOrchestrator $agent.LocalProvider = $sourceProvider $agent.RemoteProvider = $destinationProvider # Upload changes from the source to the destination. $agent.Direction = [Microsoft.Synchronization.SyncDirectionOrder]::Upload Write-Host "Synchronizing changes from $($sourceProvider.RootDirectoryPath) to replica: $($destinationProvider.RootDirectoryPath)" $agent.Synchronize();
To achieve two way synchronization, we will do the upload twice, reversing the order of the providers.
Invoke-OneWayFileSync -SourcePath $SourcePath -DestinationPath $DestinationPath -Filter $null -Options $options Invoke-OneWayFileSync -SourcePath $DestinationPath -DestinationPath $SourcePath -Filter $null -Options $options
I modified the example to write out a custom object (and the logging is in the variable in the global scope as noted in the Handling Conflicts section) with the results of the synchronization (rather than logging it to the console).
In all, my translation is pretty similar to the example code, but there are some differences.
# Requires -Version 2 # Also depends on having the Microsoft Sync Framework 2.0 SDK or Runtime # --SDK-- # http://www.microsoft.com/downloads/details.aspx?FamilyID=89adbb1e-53ff-41b5-ba17-8e43a2e66254&displaylang=en # --Runtime-- # http://www.microsoft.com/downloads/details.aspx?FamilyId=109DB36E-CDD0-4514-9FB5-B77D9CEA37F6&displaylang=en # # [CmdletBinding(SupportsShouldProcess=$true)] param ( [Parameter(Position=1, Mandatory=$true, ValueFromPipelineByPropertyName=$true)] [Alias('FullName', 'Path')] [string]$SourcePath , [Parameter(Position=2, Mandatory=$true)] [string]$DestinationPath , [Parameter(Position=3)] [string[]]$FileNameFilter , [Parameter(Position=4)] [string[]]$SubdirectoryNameFilter ) <# .Synopsis Synchronizes to directory trees .Description Examines two directory structures (SourcePath and DestinationPath) and uses the Microsoft Sync Framework File System Provider to synchronize them. .Example An example of using the command #> begin { [reflection.assembly]::LoadWithPartialName('Microsoft.Synchronization') | Out-Null [reflection.assembly]::LoadWithPartialName('Microsoft.Synchronization.Files') | Out-Null function Get-FileSystemChange() { param ($path, $filter, $options) try { $provider = new-object Microsoft.Synchronization.Files.FileSyncProvider -ArgumentList $path, $filter, $options $provider.DetectChanges() } finally { if ($provider -ne $null) { $provider.Dispose() } } } function Invoke-OneWayFileSync() { param ($SourcePath, $DestinationPath, $Filter, $Options) $ApplyChangeJobs = @() $AppliedChangeJobs = @() try { # Scriptblocks to handle the events raised during synchronization $AppliedChangeAction = { $argument = $event.SourceEventArgs switch ($argument.ChangeType) { { $argument.ChangeType -eq [Microsoft.Synchronization.Files.ChangeType]::Create } {[string[]]$global:FileSyncReport.Created += $argument.NewFilePath} { $argument.ChangeType -eq [Microsoft.Synchronization.Files.ChangeType]::Delete } {[string[]]$global:FileSyncReport.Deleted += $argument.OldFilePath} { $argument.ChangeType -eq [Microsoft.Synchronization.Files.ChangeType]::Update } {[string[]]$global:FileSyncReport.Updated += $argument.OldFilePath} { $argument.ChangeType -eq [Microsoft.Synchronization.Files.ChangeType]::Rename } {[string[]]$global:FileSyncReport.Renamed += $argument.OldFilePath} } } $SkippedChangeAction = { [string[]]$global:FileSyncReport.Skipped += $event.SourceEventArgs.CurrentFilePath if ($event.SourceEventArgs.Exception -ne $null) { Write-Error '[' + "$($event.SourceEventArgs.Exception.Message)" +']' } } # Create source provider and register change events for it $sourceProvider = New-Object Microsoft.Synchronization.Files.FileSyncProvider -ArgumentList $SourcePath, $filter, $options $AppliedChangeJobs += Register-ObjectEvent -InputObject $SourceProvider -EventName AppliedChange -Action $AppliedChangeAction $AppliedChangeJobs += Register-ObjectEvent -InputObject $SourceProvider -EventName SkippedChange -Action $SkippedChangeAction $ApplyChangeJobs += $SourceApplyChangeJob # Create destination provider and register change events for it $destinationProvider = New-Object Microsoft.Synchronization.Files.FileSyncProvider -ArgumentList $DestinationPath, $filter, $options $AppliedChangeJobs += Register-ObjectEvent -InputObject $destinationProvider -EventName AppliedChange -Action $AppliedChangeAction $AppliedChangeJobs += Register-ObjectEvent -InputObject $destinationProvider -EventName SkippedChange -Action $SkippedChangeAction $ApplyChangeJobs += $DestApplyChangeJob # Use scriptblocks for the SyncCallbacks for conflicting items. $ItemConflictAction = { $event.SourceEventArgs.SetResolutionAction([Microsoft.Synchronization.ConflictResolutionAction]::SourceWins) [string[]]$global:FileSyncReport.Conflicted += $event.SourceEventArgs.DestinationChange.ItemId } $ItemConstraintAction = { $event.SourceEventArgs.SetResolutionAction([Microsoft.Synchronization.ConstraintConflictResolutionAction]::SourceWins) [string[]]$global:FileSyncReport.Constrained += $event.SourceEventArgs.DestinationChange.ItemId } #Configure the events for conflicts or constraints for the source and destination providers $destinationCallbacks = $destinationProvider.DestinationCallbacks $AppliedChangeJobs += Register-ObjectEvent -InputObject $destinationCallbacks -EventName ItemConflicting -Action $ItemConflictAction $AppliedChangeJobs += Register-ObjectEvent -InputObject $destinationCallbacks -EventName ItemConstraint -Action $ItemConstraintAction $sourceCallbacks = $SourceProvider.DestinationCallbacks $AppliedChangeJobs += Register-ObjectEvent -InputObject $sourceCallbacks -EventName ItemConflicting -Action $ItemConflictAction $AppliedChangeJobs += Register-ObjectEvent -InputObject $sourceCallbacks -EventName ItemConstraint -Action $ItemConstraintAction # Create the agent that will perform the file sync $agent = New-Object Microsoft.Synchronization.SyncOrchestrator $agent.LocalProvider = $sourceProvider $agent.RemoteProvider = $destinationProvider # Upload changes from the source to the destination. $agent.Direction = [Microsoft.Synchronization.SyncDirectionOrder]::Upload Write-Host "Synchronizing changes from $($sourceProvider.RootDirectoryPath) to replica: $($destinationProvider.RootDirectoryPath)" $agent.Synchronize(); } finally { # Release resources. if ($sourceProvider -ne $null) {$sourceProvider.Dispose()} if ($destinationProvider -ne $null) {$destinationProvider.Dispose()} } } # Set options for the synchronization session. In this case, options specify # that the application will explicitly call FileSyncProvider.DetectChanges, and # that items should be moved to the Recycle Bin instead of being permanently deleted. $options = [Microsoft.Synchronization.Files.FileSyncOptions]::ExplicitDetectChanges $options = $options -bor [Microsoft.Synchronization.Files.FileSyncOptions]::RecycleDeletedFiles $options = $options -bor [Microsoft.Synchronization.Files.FileSyncOptions]::RecyclePreviousFileOnUpdates $options = $options -bor [Microsoft.Synchronization.Files.FileSyncOptions]::RecycleConflictLoserFiles } process { $filter = New-Object Microsoft.Synchronization.Files.FileSyncScopeFilter if ($FileNameFilter.count -gt 0) { $FileNameFilter | ForEach-Object { $filter.FileNameExcludes.Add($_) } } if ($SubdirectoryNameFilter.count -gt 0) { $SubdirectoryNameFilter | ForEach-Object { $filter.SubdirectoryExcludes.Add($_) } } # Perform the detect changes operation on the two file locations Get-FileSystemChange $SourcePath $filter $options Get-FileSystemChange $DestinationPath $filter $options # Reporting Object - using the global scope so that it can be updated by the event scriptblocks. $global:FileSyncReport = New-Object PSObject | Select-Object SourceStats, DestinationStats, Created, Deleted, Overwritten, Renamed, Skipped, Conflicted, Constrained # We don't need to pass any filters here, since we are using the file detection that was previously completed. # this will only $global:FileSyncReport.SourceStats = Invoke-OneWayFileSync -SourcePath $SourcePath -DestinationPath $DestinationPath -Filter $null -Options $options $global:FileSyncReport.DestinationStats = Invoke-OneWayFileSync -SourcePath $DestinationPath -DestinationPath $SourcePath -Filter $null -Options $options # Write result to pipeline Write-Output $global:FileSyncReport }
December 16, 2009 - 2:38 pm
I am running Powershell on a WS08 R2 system. I have the sync framework installed, but
$provider = new-object Microsoft.Synchronization.Files.FileSyncProvider
produces an error all the time. — Constructor not found.
On the other hand
$filter = New-Object Microsoft.Synchronization.Files.FileSyncScopeFilter
works fine. Do you have any suggestions on how to do this?
I simply copied and pasted the code from your examples.
Thank you.
December 16, 2009 - 5:46 pm
Depending on the line wrapping on the post, there are some arguments that need to be passed to the Microsoft.Synchrononization.Files.FileSyncProvider, the filter object, the path, and the options. There are two ways to do it, either with the -ArgumentList parameter, or by adding them in parens after the class name.
December 16, 2009 - 6:14 pm
Curiously, I run your code in two windows.
One is a “runas /user:administrator powershell” and the other is just an ordinary powershell. The Administrator powershell works but the user one doesn’t. I haven’t tracked down where the differences are yet. I’ll post that when I learn more.
December 19, 2009 - 1:44 pm
I haven’t had that experience. I’ll try that out as well.
January 21, 2010 - 2:16 pm
To avoid the line wrap issues can you post a ZIP’d PS1download link in this blog?
January 21, 2010 - 2:18 pm
Sure thing.. will do that shortly.
April 23, 2010 - 12:54 am
Supernice example and it works perfectly, but i’m curious of whats happening here?
$ApplyChangeJobs += $SourceApplyChangeJob
and here
$ApplyChangeJobs += $DestApplyChangeJob
I cannot see that $SourceApplyChangeJob and $DestApplyChangeJob is used anywhere else?
Regards,
Johnnie
April 27, 2010 - 9:13 am
I think those were leftovers from a previous refactoring. Nice catch.
Steve
September 23, 2010 - 3:49 pm
I can not get this to work. I recieve the following error. Any help would be appricated.
Cannot find an overload for “FileSyncProvider” and the argument count: “3″.
At :line:43 char:34
+ $provider = new-object <<<< Microsoft.Synchronization.Files.FileSyncProvider -ArgumentList $path, $filter, $options
September 23, 2010 - 4:15 pm
Gary,
What version of the Sync Framework do you have installed? What OS are you running on? What version of PowerShell do you have installed? Are you running on a 64 bit OS? If so, what version of the Sync Framework and PowerShell are you using (32 bit or 64 bit)?
Steve
September 24, 2010 - 12:35 pm
Steven, thanks for the reply.
Sync Framework 2.1 64bit
Win 7 64bit
Powershell 2
I have tried the 32 bit and 64 bit consoles on my machine. same results
here is the load assembly output
GAC Version Location
— ——- ——–
True v2.0.50727 C:\Windows\assembly\GAC_MSIL\Microsoft.Synchronization\2.1.0.0__89845dcd8080cc91\Microsoft.Synchronization.dll
True v2.0.50727 C:\Windows\assembly\GAC_MSIL\Microsoft.Synchronization.Files\2.1.0.0__89845dcd8080cc91\Microsoft.Synchronization.Files.dll
September 28, 2010 - 9:53 am
Steve,
So what was your setup when you wrote this script?
September 29, 2010 - 5:18 am
I wrote and tested this script on 64 bit Server 2008 R2 and 32 bit Windows 7 with the Sync Framework 2.0 SDK.
December 31, 2010 - 10:45 pm
I am curious, using this method, can two clients sync to a common source?
Client A Client B Server C
Client A syncs to Server C
Client B syncs to Server C
Using this method, with the metadata file, it appears that it fails.
Exception calling “Synchronize” with “0″ argument(s): “The metadata store replica is already in use.”
March 24, 2011 - 3:55 pm
I’ve found this really helpful, but here are a couple of notes for people trying to use it:
Write-Error seems to have a syntax error and isn’t running correctly. It should be Write-Error “[$($event.SourceEventArgs.Exception.Message)]”
Errors in an event seem to get suck in the event thus the output from Write-error isn’t displayed and nor is the syntax error
$global:FileSyncReport has a property called Overwritten but the $AppliedChangeAction is writing to a field called Updated
April 28, 2011 - 5:35 am
Just stumbled across this looking for a windows sync alternative to rsycn. I guess I’m in the wrong place?
August 1, 2011 - 4:42 am
Has anyone tried to use the “ApplyingChange” Event.
The event is triggered, but setting
$argument.SkipChange=$true
has no effect. I want to disable file deletions at target folder.
thanx for help
September 17, 2011 - 11:47 am
Nice code. it works perfectly.
Do you know how to sync ntfs permissions as well?