Synchronising directories on more than 2 PCs

[John Gray]

This is a real-life 'puzzle'!

Problem 1

I have three PCs, A, B and C, each of which contains a directory DIR to which changes (additions, updates and deletions) may be made by that PC during the day.

I am interested at the end of each day in ensuring that the DIR directory on each PC contains exactly the same files at exactly the same update levels.

I have a program that will synchronise the DIR directory between two PCs.

What sequence of synchronisation runs must I do between the three PCs to achieve this result with the minimum amount of effort (that is, least number of synchronisation program runs and/or least amount of file copying)?

Problem 2

Once Problem 1 is sorted out, let us introduce an external hard drive connected to one of the PCs, or a NAS box on the LAN, I don't think it matters which.

Assume now that this external hard drive/NAS box contains the 'master copy' of the directory DIR, and at the end of the day we want to reflect all the changes that have been made on each PC to its own copy of the DIR directory. Here we must do more synchronisations runs, now between four 'devices', the three PCs and the external hard drive/NAS box, to ensure that all four copies of the DIR directory are identical. Is this problem similar to the first one, but now between four devices?

I have worked out an answer (I think) to Problem 1, but for Problem 2, "my brain hurts"!

I'd be interested in what people come up with...

PS I also posted this Puzzle (by accident!) on WSL...

[HansV]

Assuming that the 4th device (the external hard disk or whatever) will not be modified independently of the three PCs, the situation is not symmetric with respect to all 4 devices. Once you have synchronised the three PCs, you're left with a one-way synchronisation from any of the 3 PCs to the 4th device.

Do you know Beyond Compare? It's not free, but not expensive either, and it's a very handy comparison/synchronisation tool.

[HansV]

I mentioned Beyond Compare because it's used successfully where I work.

While Googling, I found Allway Sync which claims to be able to perform n-way file and folder synchronisation (n = 2, 3, 4, ...). I have no experience with it, however, so I don't know how well it works.

[ChrisGreaves]

I have three PCs, A, B and C, each of which contains a directory ...

This seems to be the same genre of puzzle as that of {poorly stated}"What is the minimum number of slices needed to cut a cube 1/3 the dimension of a cube from the centre of the cube"{/poorly stated}.
The answer is "six", for that is the number of faces on the smaller cube buried within the larger cube.

In your case you have "n" drives.
If you RoboCopy drives 2, 3, 4 ... to drive 1, then drive 1 is up to date. Each of the drives 2, 3, 4 ... must be accessed in order to be sure that each of those drives is identical (one-way) with drive 1.
Then drive 1 must be RoboCopy'd to each of 2, 3, 4 ... in order to be sure that each of those drives is identical (second-way) with drive 1.

For "n" drives you need 2*(n-1) operations.

Unless someone can come up with better logic.


FWIW I have 3 networked computers backed up to 2 external drives.
Each of the 3 networked computers writes to each of the 2 external drives, in a one-way direction. I have no need to make any networked computer drive identical to any other.
I have a batch job to synchronize the two external drives after any drive maintenance:

Code: Select all

TITLE %0 Synchronize the two 300GB drives

if not exist X: 	"C:\Program Files\TrueCrypt\TrueCrypt.exe" /q /lX  /v\Device\Harddisk1\Partition1
if not exist Y: 	"C:\Program Files\TrueCrypt\TrueCrypt.exe" /q /lY  /v\Device\Harddisk2\Partition1


	@echo %ComputerName% %Date% %Time% Synchronise drive X to Y >> Y:\Backup.LOG
	Title  %0 Synchronise drive X to Y
	roboCopy X:\ y:\ /s /r:0 /w:0
	@echo %ComputerName% %Date% %Time% Ended drive X to Y >> Y:\Backup.LOG


	@echo %ComputerName% %Date% %Time% Synchronise drive Y to X >> X:\Backup.LOG
	Title  %0 Synchronise drive Y to X
	roboCopy Y:\ X:\ /s /r:0 /w:0
	@echo %ComputerName% %Date% %Time% Ended drive Y to X >> X:\Backup.LOG

In this case 2*(n-1) evaluates as 2*(2-1) which is 2.
And yes, the batch file makes 2 passes.

[John Gray]

I mentioned Beyond Compare because it's used successfully where I work.
While Googling, I found Allway Sync which claims to be able to perform n-way file and folder synchronisation (n = 2, 3, 4, ...). I have no experience with it, however, so I don't know how well it works.

Hans - the software you suggested to solve the puzzle was initially a secondary consideration for me! Beyond Compare sounds very good, but it's a bit pricey. Allway Sync is spoken of very highly by its manufacturers, and, yes, it seems to do exactly what I want in one pass when running on a single machine! It also seems to do much of what Beyond Compare does, perhaps implicitly.

Chris - I like your idea of always copying to/from a single PC, because then there's no need to serialise the execution of synchronisation jobs on other PCs.
For three PCs, your sequence would be:

• Sync A with B
• Sync A with C
  A is now fully up-to-date with all changes
• Sync A with B
• Sync A with C
  B and C are now fully up-to-date with all changes

My plan was a sort of 'round robin':

• Sync A with B
• Sync B with C
• Sync C with A
• Sync A with B
  A, B and C are now fully up-to-date with all changes (I think!)

I have had a different (and one-pass-fewer) solution, as follows:

• Sync A with B (both A and B contain the same changes)
• Sync B with C (both B and C contain the same changes, but now differ from A)
• Sync A with B (A is now updated with the changes from B and C)

The answer for problem 2 seems to be rather more straightforward, surprisingly!
"If you want to use a central disk (a server, a USB disk, a NAS device) as your master file set - let's call it X - you need to do the following

A <> X
B <> X
C <> X
B <> X
A <> X
"

[ChrisGreaves]

... because then there's no need to serialise the execution of synchronisation jobs on other PCs.

In my office-wide system this is true. The DOS batch file, on each computer system, can be fired up when the user is about to leave the office. The batch file WAITs until the backup drives are detected, then does the backup and executes the end-of-backup procedure for that machine - which may be Shutdown, Restart, idle etc.[/quote]

For three PCs, your sequence would be:

• Sync A with B
• Sync A with C
  A is now fully up-to-date with all changes
• Sync A with B
• Sync A with C
  B and C are now fully up-to-date with all changes

Correct, and my formula for slices-of-the-cube analogy is 2*(n-1) which in this case is 2*(3-1) which evaluates to 4, so we are in harmony.

My plan was a sort of 'round robin':

• Sync A with B
• Sync B with C
• Sync C with A
• Sync A with B
  A, B and C are now fully up-to-date with all changes (I think!)

Correct, and my formula for slices-of-the-cube analogy is 2*(n-1) which in this case is 2*(3-1) which evaluates to 4, so we are in harmony.

I have had a different (and one-pass-fewer) solution, as follows:

• Sync A with B (both A and B contain the same changes)
• Sync B with C (both B and C contain the same changes, but now differ from A)
• Sync A with B (A is now updated with the changes from B and C)

You're cheating (grin!).
"Sync A with B (both A and B contain the same changes)" must involve TWO (Robo)Copy operations, if you want the drives to be synchronised; one operation updates A from B, and the second one updates B from A", so this solution (if I've read it correctly) occupies 6 operations.

"If you want to use a central disk (a server, a USB disk, a NAS device) as your master file set - let's call it X - you need to do the following
A <> X
B <> X
C <> X
B <> X
A <> X
"

This is a 4-drive problem, so 2*(4-1) evaluates to 2*3=6.
That's assuming you want to synchronize all 4 drives A, B, C and X.

I'm curious: Does anyone reading this agree with my 2*(n-1) formula? I think that the logic is correct. I know it is correct for the slicing-out-an-interior-cube problem (or the 2D problem of slicing-out-an-interior-square of a chessboard, or the 1D problem of slicing-out-an-interior slice of bread).

[StuartR]

,,,Does anyone reading this agree with my 2*(n-1) formula? ...

It looks right to me.

[ChrisGreaves]

It looks right to me.

Thanks Stuart.

[John Gray]

I think it depends what you consider to be a 'synchronisation operation'. I'm using it to mean the simultaneous updating in both directions of a directory held on each of two PCs. I'm thinking more of a synchronisation utility like SyncBack rather than RoboCopy, however much I love it!

[ChrisGreaves]

... I'm using it to mean the simultaneous updating in both directions of a directory held on each of two PCs. ...

Fair enough.
In that case a macro RoboCopy ("MoboCopy.BAT ??) that RoboCopys in both directions would be your two-way synchronization.
I suspect that my formula would then become 1*(n-1).
I'm struggling with the concept of "n-1" when I have only one drive; I suppose that means I need zero operations (of any sort) to maintain synchronicity.

I suspect too, but will have to mull a bit more, that using a 2-way copy always might introduce some inefficiencies; there may be an example where data is tested/copied twice.

If I understand the round-robin concept, I visualize a 5-drive system as (pencil-and-paper time) 5 circles, each identified by its contents. Circle "A" contains just the letter "A", etc.

I pencil an arrow from circle "A" to circle "B", appending "A" to the contents of "B" which now reads "AB".
A second arrow (i.e. "Robocopy command") is drawn from "B" to "C", so that "C" now contains "ABC".
Continue in this way once around the set of 5, and keep going until all drives contain "ABCDE".
The process stops after the second transfer to circle "E".
I count the arrows and find that I have executed 8 "Robocopy commands", which conforms (can I say that/) to my generic 2*(n-1) formula.
Hooray!

[ChrisGreaves]

Once Problem 1 is sorted out, let us introduce an external hard drive connected to one of the PCs, ...

I still think that puzzle 1 is a 3-drive situation and puzzle 2 is a 4-drive situation. That is, same puzzle, so same solution.
Quantities May Vary^(TM).