Isn't there a way to set the display name for the local and remote host inside
the prf file?

Sandro


> -----Urspr�ngliche Nachricht-----
> Von: "Konrad Hinsen" <konrad.hinsen@...>
> Gesendet: 12.11.09 11:24:02
> An: unison-users@yahoogroups.com
> Betreff: Re: [unison-users] Synchronizing to one server with different
hostnames


> On 11 Nov 2009, at 22:13, Cameron wrote:
> > I'm currently using Unison to sync my computer with my university's
> > networked file space using SSH. My university has a pool of
> machines
> > you can log into at one hostname, but then directs you to whichever
> > machine is available. Each mounts my networked file space as the
> > home directory.
> >
> > The problem is that Unison thinks I'm syncing to different machines
> > each time I run it. Is it possible to have Unison ignore that its
> > connecting to different machines, or make it treat every computer
> in
> > the pool as one archive?
> >
> Set the environment variable UNISONLOCALHOSTNAME to the same value on
> all those machines. Its value will be used instead of the hostname.
>
> Konrad.
>
>

Hello,

I have a problem: I need to sync two machines which are far away from each
other. Unison is installed on both and short sync IS okay. However when I try to
sync huge folder then after scanning the local folder (for few hours) it reports
"Lost connection with server". I am trying to have initial synchronisation, do
not know if it is any faster when initial sync has been done already.

I tried to play with ssh options on the remote machine (server) but that was not
OK. I can NOT synchronise by small pieces, since I have LOTS of data.

Therefore, if somebody had and overcame this problem, please tell how you did
that! Is there a way to speed up scanning process (now it takes hours)?

Cheers, Askar

On Thu, Nov 12, 2009 at 1:32 PM, askar.ibragimov
<askar.ibragimov@...> wrote:
> Hello,
>
> I have a problem: I need to sync two machines which are far away from each
other. Unison is installed on both and short sync IS okay. However when I try to
sync huge folder then after scanning the local folder (for few hours) it reports
"Lost connection with server". I am trying to have initial synchronisation, do
not know if it is any faster when initial sync has been done already.
>
> I tried to play with ssh options on the remote machine (server) but that was
not OK. I can NOT synchronise by small pieces, since I have LOTS of data.
>
> Therefore, if somebody had and overcame this problem, please tell how you did
that! Is there a way to speed up scanning process (now it takes hours)?

Why can't you synchronize by small pieces? Set the roots as they will
be at the end, and add synchronization paths little by little.

Alan

Hello,

I can not. Unless you have a good way to sync dozens of thousands of
these small pieces. As I stated, I am not interested in the solutions
which relate to small pieces.

For the most curious ones: I have a particular folder with  20784
subfolders inside (typical hierarchy depth is 5 folders deep).  Whole
work folder contains about 5 times more subfolders.

Cheers, Askar


2009/11/12 Alan Schmitt <alan.schmitt@...>:
> On Thu, Nov 12, 2009 at 1:32 PM, askar.ibragimov
> <askar.ibragimov@...> wrote:
>> Hello,
>>
>> I have a problem: I need to sync two machines which are far away from each
other. Unison is installed on both and short sync IS okay. However when I try to
sync huge folder then after scanning the local folder (for few hours) it reports
"Lost connection with server". I am trying to have initial synchronisation, do
not know if it is any faster when initial sync has been done already.
>>
>> I tried to play with ssh options on the remote machine (server) but that was
not OK. I can NOT synchronise by small pieces, since I have LOTS of data.
>>
>> Therefore, if somebody had and overcame this problem, please tell how you did
that! Is there a way to speed up scanning process (now it takes hours)?
>
> Why can't you synchronize by small pieces? Set the roots as they will
> be at the end, and add synchronization paths little by little.
>
> Alan
>

You could use the "root alias" option in a .prf, but I would use a script to
set the UNISONLOCALHOSTNAME if possible, because then you don't need to know
all the machine hostnames before syncing.  However, I would be careful about
doing this at all.  How do you know that all of the university's computers
are in sync?  If you get one machine one time and do a sync and then another
machine the next time, that for some reason on the universities end hasn't
been synced with the first machine you contacted, unison will "think" that
"you" changed the file(s) and so will end up propagating them in the wrong
direction.  I would be very careful doing this.  Root alias is really
intended for the same file system being mounted in a different location and
not different file systems being mounted in the same location.

Mike E.


-----Original Message-----
From: unison-users@yahoogroups.com
[mailto:unison-users@yahoogroups.com]On Behalf Of Sandro Plischewski
Sent: Thursday, November 12, 2009 5:40 AM
To: unison-users@yahoogroups.com
Subject: Re: [unison-users] Synchronizing to one server with different
hostnames


Isn't there a way to set the display name for the local and remote host
inside the prf file?

Sandro


> -----Urspr�ngliche Nachricht-----
> Von: "Konrad Hinsen" <konrad.hinsen@...>
> Gesendet: 12.11.09 11:24:02
> An: unison-users@yahoogroups.com
> Betreff: Re: [unison-users] Synchronizing to one server with different
hostnames


> On 11 Nov 2009, at 22:13, Cameron wrote:
> > I'm currently using Unison to sync my computer with my university's
> > networked file space using SSH. My university has a pool of
> machines
> > you can log into at one hostname, but then directs you to whichever
> > machine is available. Each mounts my networked file space as the
> > home directory.
> >
> > The problem is that Unison thinks I'm syncing to different machines
> > each time I run it. Is it possible to have Unison ignore that its
> > connecting to different machines, or make it treat every computer
> in
> > the pool as one archive?
> >
> Set the environment variable UNISONLOCALHOSTNAME to the same value on
> all those machines. Its value will be used instead of the hostname.
>
> Konrad.
>
>




------------------------------------

Yahoo! Groups Links

Unison is not appropriate tool.

Looking through maillist archives, found this:
<http://tech.groups.yahoo.com/group/unison-users/message/8426>
> A better solution is to put the image upload directory on a NFS share on linux
> or a shared directoy on Windows. That avoids the sync delay completely and
setup
> shouldn't be too complicated.
>Hope that helps,
> Dirk Dittert

and my own message:
> A little googling gives results like
<http://stackoverflow.com/questions/398305/file-upload-on-load-balanced-server>
and forth. Just try:
> <http://www.google.com/search?num=100&q=load-balanced+file+uploading>


on message of Thomas Manson <dev.mansonthomas@...> sent Thu, 12 Nov 2009
17:02:55 +0300 :

>    1. Is unison the right tool to do what I want ? if not, do you have some
>    other tool to recommend?
>    2. What is the best way to do it ?  (cron on each machine? how to handle
>    the hundred of dir)


--
Regards

Hello,

I can not. Unless you have a good way to sync dozens of thousands of
these small pieces. As I stated, I am not interested in the solutions
which relate to small pieces.

For the most curious ones: I have a particular folder with 20784
subfolders inside (typical hierarchy depth is 5 folders deep). Whole
work folder contains about 5 times more subfolders.

Cheers, Askar

> > Hello,
> > I have a problem: I need to sync two machines which are far away
from each other. Unison is installed on both and short sync
> IS okay. However when I try to sync huge folder then after scanning
the local folder (for few hours) it reports "Lost
> connection with server". I am trying to have initial synchronisation,
do not know if it is any faster when initial sync has
> been done already.
> >
> > I tried to play with ssh options on the remote machine (server) but
that was not OK. I can NOT synchronise by small pieces,
> since I have LOTS of data.
> >
> > Therefore, if somebody had and overcame this problem, please tell
how you did that! Is there a way to speed up scanning
> process (now it takes hours)?
>
> Why can't you synchronize by small pieces? Set the roots as they will
> be at the end, and add synchronization paths little by little.
>
> Alan
[_SUrbaniak_]

Alan,  you are talking about a very tedious manual process.  I have been
struggling with the very same issue for years.  (Connection Loss on
synchs that take more then 1h    -     no my WAN link that is a fact of
life).

Is there nothing that can be added to the software that would do the
"little by little"  breakup automatically for you ?

I even wrote myself a tiny VB app that lets me ignore/un-ignore paths
interactively/graphically in a prf.... but as much as it helps that is
still a super tedious manual process ... when you have to rerun Unison
10, 20, 30, ...   times.
There has got to be a way to make that automated behind the scenes !
What about something as simple as making the TCP connection a property
of the session (ie. persist in reconnecting as necessary to complete the
synch - unless expressly terminated), as opposed to the way it is now
where 1connection = 1synch-session.

Seb

-----Original Message-----
From: unison-users@yahoogroups.com [mailto:unison-users@yahoogroups.com]On Behalf Of Thomas Manson
Sent: Thursday, November 12, 2009 9:03 AM
To: unison-users@yahoogroups.com
Subject: [unison-users] file synchronisation between two node of a http cluster

 

Hi,

  I'm building a cluster of lighttpd (web server).

  It's two seperate machine, each one running one lighttpd server (among other thing).

  the incomming traffic is sent to a virtual ip address that point to a loadbalancer (haproxy) which loadbalance the traffic accross the two servers (and maybe more servers if needed).

  The webservers hosts hundred of virtualhost, so, there will be hundred of directories to sync.

  The problem is, if someone upload something like an image on a website one server, I need it to be replicated on the other server.

  One of the MySQL consultant I hire speak to me of unison.

  I've quickly read the doc, and it doesn't seem to have an unison daemon.

  So my question is  : 

1. Is unison the right tool to do what I want ? if not, do you have some other tool to recommend?
2. What is the best way to do it ?  (cron on each machine? how to handle the hundred of dir)

Regards,

Thomas.

It should be possible to write a perl or bash script to do this for
you. Might take a bit of work, but nothing too crazy I think.

Jason

On Thu, Nov 12, 2009 at 5:51 AM, Urbaniak, Sebastian
<surbaniak@...> wrote:
>> > Hello,
>> > I have a problem: I need to sync two machines which are far away
> from each other. Unison is installed on both and short sync
>> IS okay. However when I try to sync huge folder then after scanning
> the local folder (for few hours) it reports "Lost
>> connection with server". I am trying to have initial synchronisation,
> do not know if it is any faster when initial sync has
>> been done already.
>> >
>> > I tried to play with ssh options on the remote machine (server) but
> that was not OK. I can NOT synchronise by small pieces,
>> since I have LOTS of data.
>> >
>> > Therefore, if somebody had and overcame this problem, please tell
> how you did that! Is there a way to speed up scanning
>> process (now it takes hours)?
>>
>> Why can't you synchronize by small pieces? Set the roots as they will
>> be at the end, and add synchronization paths little by little.
>>
>> Alan
> [_SUrbaniak_]
>
> Alan, �you are talking about a very tedious manual process. �I have been
> struggling with the very same issue for years. �(Connection Loss on
> synchs that take more then 1h � �- � � no my WAN link that is a fact of
> life).
>
> Is there nothing that can be added to the software that would do the
> "little by little" �breakup automatically for you ?
>
> I even wrote myself a tiny VB app that lets me ignore/un-ignore paths
> interactively/graphically in a prf.... but as much as it helps that is
> still a super tedious manual process ... when you have to rerun Unison
> 10, 20, 30, ... � times.
> There has got to be a way to make that automated behind the scenes !
> What about something as simple as making the TCP connection a property
> of the session (ie. persist in reconnecting as necessary to complete the
> synch - unless expressly terminated), as opposed to the way it is now
> where 1connection = 1synch-session.
>
> Seb
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

The basic idea of what I was thinking was to put the entire profile
description in the script and then write out the profile for the first
path, and run unison. Then do this for each folder in the root
(assuming those folders are big enough). You could probably generalize
it even more by checking if unison fails (although this takes a while)
then set the root as a deeper folder. But I personally have not tried
this or even tried setting path separately since I have so few files.

Jason

On Thu, Nov 12, 2009 at 9:51 AM, Urbaniak, Sebastian
<surbaniak@...> wrote:
> Script to do which part ?
>
>> -----Original Message-----
>> From: Jason Axelson [mailto:bostonvaulter@...]
>> Sent: November 12, 2009 14:24
>> To: Urbaniak, Sebastian
>> Cc: unison-users@yahoogroups.com
>> Subject: Re: [unison-users] Unison "Lost connection with server"
>>
>> It should be possible to write a perl or bash script to do this for
>> you. Might take a bit of work, but nothing too crazy I think.
>>
>> Jason
>>
>> On Thu, Nov 12, 2009 at 5:51 AM, Urbaniak, Sebastian
>> <surbaniak@...> wrote:
>> >> > Hello,
>> >> > I have a problem: I need to sync two machines which are far away
>> > from each other. Unison is installed on both and short sync
>> >> IS okay. However when I try to sync huge folder then after scanning
>> > the local folder (for few hours) it reports "Lost
>> >> connection with server". I am trying to have initial synchronisation,
>> > do not know if it is any faster when initial sync has
>> >> been done already.
>> >> >
>> >> > I tried to play with ssh options on the remote machine (server) but
>> > that was not OK. I can NOT synchronise by small pieces,
>> >> since I have LOTS of data.
>> >> >
>> >> > Therefore, if somebody had and overcame this problem, please tell
>> > how you did that! Is there a way to speed up scanning
>> >> process (now it takes hours)?
>> >>
>> >> Why can't you synchronize by small pieces? Set the roots as they will
>> >> be at the end, and add synchronization paths little by little.
>> >>
>> >> Alan
>> > [_SUrbaniak_]
>> >
>> > Alan, �you are talking about a very tedious manual process. �I have been
>> > struggling with the very same issue for years. �(Connection Loss on
>> > synchs that take more then 1h � �- � � no my WAN link that is a fact of
>> > life).
>> >
>> > Is there nothing that can be added to the software that would do the
>> > "little by little" �breakup automatically for you ?
>> >
>> > I even wrote myself a tiny VB app that lets me ignore/un-ignore paths
>> > interactively/graphically in a prf.... but as much as it helps that is
>> > still a super tedious manual process ... when you have to rerun Unison
>> > 10, 20, 30, ... � times.
>> > There has got to be a way to make that automated behind the scenes !
>> > What about something as simple as making the TCP connection a property
>> > of the session (ie. persist in reconnecting as necessary to complete the
>> > synch - unless expressly terminated), as opposed to the way it is now
>> > where 1connection = 1synch-session.
>> >
>> > Seb
>> >
>> >
>> > ------------------------------------
>> >
>> > Yahoo! Groups Links
>> >
>> >
>> >
>> >
>

That works in some cases,  but not in general.

For example one of the cases where mine runs into  Connection Loss is where only
10 files (9 of them in same folder) update,  but they are each 2GB in size.

The solution/algorithm for the client which I proposed handles the problem in
general.

	 Thread1:
		 While(<detecting_file_diffs>)
		 {
			 If(!CONNECTED)
				 Sleep(20seconds)
			 Else
				 <keep_detecting_file_diffs>
		 }
		 While(<synchronizing>)
		 {
			 If(!CONNECTED)
				 Sleep(20seconds)
			 Else
				 <keep_synchronizing>
		 }
	 Thread2:
		 While(TRUE)
		 {
			 If(!CONNECTED && SUPPOSED_TO_BE_CONNECTED)
				 AttemptConnectionToServer()
			 If(USER_ABORTED || PROCESS_TERMINATED)
				 Exit()
			 Else
				 Sleep(10seconds)
		 }
		 // add configurable MAX_OVERALL_CONNECT_RETRY_LIMIT & Counter if you like


Does it fit the current paradigm?  Anyone feel like implementing this in the
Unison core?


Seb

> -----Original Message-----
> From: Jason Axelson [mailto:bostonvaulter@...]
> Sent: November 12, 2009 15:53
> To: Urbaniak, Sebastian
> Cc: unison-users@yahoogroups.com
> Subject: Re: [unison-users] Unison "Lost connection with server"
>
> The basic idea of what I was thinking was to put the entire profile
> description in the script and then write out the profile for the first
> path, and run unison. Then do this for each folder in the root
> (assuming those folders are big enough). You could probably generalize
> it even more by checking if unison fails (although this takes a while)
> then set the root as a deeper folder. But I personally have not tried
> this or even tried setting path separately since I have so few files.
>
> Jason
>
> On Thu, Nov 12, 2009 at 9:51 AM, Urbaniak, Sebastian
> <surbaniak@...> wrote:
> > Script to do which part ?
> >
> >> -----Original Message-----
> >> From: Jason Axelson [mailto:bostonvaulter@...]
> >> Sent: November 12, 2009 14:24
> >> To: Urbaniak, Sebastian
> >> Cc: unison-users@yahoogroups.com
> >> Subject: Re: [unison-users] Unison "Lost connection with server"
> >>
> >> It should be possible to write a perl or bash script to do this for
> >> you. Might take a bit of work, but nothing too crazy I think.
> >>
> >> Jason
> >>
> >> On Thu, Nov 12, 2009 at 5:51 AM, Urbaniak, Sebastian
> >> <surbaniak@...> wrote:
> >> >> > Hello,
> >> >> > I have a problem: I need to sync two machines which are far away
> >> > from each other. Unison is installed on both and short sync
> >> >> IS okay. However when I try to sync huge folder then after scanning
> >> > the local folder (for few hours) it reports "Lost
> >> >> connection with server". I am trying to have initial synchronisation,
> >> > do not know if it is any faster when initial sync has
> >> >> been done already.
> >> >> >
> >> >> > I tried to play with ssh options on the remote machine (server) but
> >> > that was not OK. I can NOT synchronise by small pieces,
> >> >> since I have LOTS of data.
> >> >> >
> >> >> > Therefore, if somebody had and overcame this problem, please tell
> >> > how you did that! Is there a way to speed up scanning
> >> >> process (now it takes hours)?
> >> >>
> >> >> Why can't you synchronize by small pieces? Set the roots as they will
> >> >> be at the end, and add synchronization paths little by little.
> >> >>
> >> >> Alan
> >> > [_SUrbaniak_]
> >> >
> >> > Alan, �you are talking about a very tedious manual process. �I have been
> >> > struggling with the very same issue for years. �(Connection Loss on
> >> > synchs that take more then 1h � �- � � no my WAN link that is a fact of
> >> > life).
> >> >
> >> > Is there nothing that can be added to the software that would do the
> >> > "little by little" �breakup automatically for you ?
> >> >
> >> > I even wrote myself a tiny VB app that lets me ignore/un-ignore paths
> >> > interactively/graphically in a prf.... but as much as it helps that is
> >> > still a super tedious manual process ... when you have to rerun Unison
> >> > 10, 20, 30, ... � times.
> >> > There has got to be a way to make that automated behind the scenes !
> >> > What about something as simple as making the TCP connection a property
> >> > of the session (ie. persist in reconnecting as necessary to complete the
> >> > synch - unless expressly terminated), as opposed to the way it is now
> >> > where 1connection = 1synch-session.
> >> >
> >> > Seb

No, this doesn't fit Unison's view of the world very well.

I believe what's needed is to change the update detection code so
that, if it notices it is taking a long time, it quits early (without
scanning all of the replica), writes out the archive files, and then
restarts the scanning process.  I don't think this would be too hard
for someone to implement, but it does involve touching the Update
module, which is a little tangled.

      - B



On Nov 12, 2009, at 5:13 PM, Urbaniak, Sebastian wrote:

> That works in some cases,  but not in general.
>
> For example one of the cases where mine runs into  Connection Loss
> is where only 10 files (9 of them in same folder) update,  but they
> are each 2GB in size.
>
> The solution/algorithm for the client which I proposed handles the
> problem in general.
>
>  Thread1:
> 	 While(<detecting_file_diffs>)
> 	 {
> 		 If(!CONNECTED)
> 			 Sleep(20seconds)
> 		 Else
> 			 <keep_detecting_file_diffs>
> 	 }
> 	 While(<synchronizing>)
> 	 {
> 		 If(!CONNECTED)
> 			 Sleep(20seconds)
> 		 Else
> 			 <keep_synchronizing>
> 	 }
>  Thread2:
> 	 While(TRUE)
> 	 {
> 		 If(!CONNECTED && SUPPOSED_TO_BE_CONNECTED)
> 			 AttemptConnectionToServer()
> 		 If(USER_ABORTED || PROCESS_TERMINATED)
> 			 Exit()
> 		 Else
> 			 Sleep(10seconds)
> 	 }
> 	 // add configurable MAX_OVERALL_CONNECT_RETRY_LIMIT & Counter if
> you like
>
>
> Does it fit the current paradigm?  Anyone feel like implementing
> this in the Unison core?
>
>
> Seb
>
>> -----Original Message-----
>> From: Jason Axelson [mailto:bostonvaulter@...]
>> Sent: November 12, 2009 15:53
>> To: Urbaniak, Sebastian
>> Cc: unison-users@yahoogroups.com
>> Subject: Re: [unison-users] Unison "Lost connection with server"
>>
>> The basic idea of what I was thinking was to put the entire profile
>> description in the script and then write out the profile for the
>> first
>> path, and run unison. Then do this for each folder in the root
>> (assuming those folders are big enough). You could probably
>> generalize
>> it even more by checking if unison fails (although this takes a
>> while)
>> then set the root as a deeper folder. But I personally have not tried
>> this or even tried setting path separately since I have so few files.
>>
>> Jason
>>
>> On Thu, Nov 12, 2009 at 9:51 AM, Urbaniak, Sebastian
>> <surbaniak@...> wrote:
>>> Script to do which part ?
>>>
>>>> -----Original Message-----
>>>> From: Jason Axelson [mailto:bostonvaulter@...]
>>>> Sent: November 12, 2009 14:24
>>>> To: Urbaniak, Sebastian
>>>> Cc: unison-users@yahoogroups.com
>>>> Subject: Re: [unison-users] Unison "Lost connection with server"
>>>>
>>>> It should be possible to write a perl or bash script to do this for
>>>> you. Might take a bit of work, but nothing too crazy I think.
>>>>
>>>> Jason
>>>>
>>>> On Thu, Nov 12, 2009 at 5:51 AM, Urbaniak, Sebastian
>>>> <surbaniak@...> wrote:
>>>>>>> Hello,
>>>>>>> I have a problem: I need to sync two machines which are far away
>>>>> from each other. Unison is installed on both and short sync
>>>>>> IS okay. However when I try to sync huge folder then after
>>>>>> scanning
>>>>> the local folder (for few hours) it reports "Lost
>>>>>> connection with server". I am trying to have initial
>>>>>> synchronisation,
>>>>> do not know if it is any faster when initial sync has
>>>>>> been done already.
>>>>>>>
>>>>>>> I tried to play with ssh options on the remote machine
>>>>>>> (server) but
>>>>> that was not OK. I can NOT synchronise by small pieces,
>>>>>> since I have LOTS of data.
>>>>>>>
>>>>>>> Therefore, if somebody had and overcame this problem, please
>>>>>>> tell
>>>>> how you did that! Is there a way to speed up scanning
>>>>>> process (now it takes hours)?
>>>>>>
>>>>>> Why can't you synchronize by small pieces? Set the roots as
>>>>>> they will
>>>>>> be at the end, and add synchronization paths little by little.
>>>>>>
>>>>>> Alan
>>>>> [_SUrbaniak_]
>>>>>
>>>>> Alan,  you are talking about a very tedious manual process.  I
>>>>> have been
>>>>> struggling with the very same issue for years.  (Connection Loss
>>>>> on
>>>>> synchs that take more then 1h    -     no my WAN link that is a
>>>>> fact of
>>>>> life).
>>>>>
>>>>> Is there nothing that can be added to the software that would do
>>>>> the
>>>>> "little by little"  breakup automatically for you ?
>>>>>
>>>>> I even wrote myself a tiny VB app that lets me ignore/un-ignore
>>>>> paths
>>>>> interactively/graphically in a prf.... but as much as it helps
>>>>> that is
>>>>> still a super tedious manual process ... when you have to rerun
>>>>> Unison
>>>>> 10, 20, 30, ...   times.
>>>>> There has got to be a way to make that automated behind the
>>>>> scenes !
>>>>> What about something as simple as making the TCP connection a
>>>>> property
>>>>> of the session (ie. persist in reconnecting as necessary to
>>>>> complete the
>>>>> synch - unless expressly terminated), as opposed to the way it
>>>>> is now
>>>>> where 1connection = 1synch-session.
>>>>>
>>>>> Seb
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

That would work for me.
In fact, why could the archives not get updated "AS YOU GO" (on a timer,
or for every N megabytes read)?
That way  when a sudden Connection Loss occurs you don't loose all the
precious processing you have already done up to this point.   (I believe
this methodology was already kind of applied in the propagation stage -
by resuming previously aborted transfers (if I am not mistaken))

Yes?
This way there would be no need to write any additional timers for
detection of 'long' scans, or logic for auto restarting.
The next unison cycle would simply pickup where the previous left off  -
currently if I understand things correctly everything starts from the
very beginning (even if previous cycle already scanned 9 out of 10 huge
files).

Any coders out there ,  that want to make this mod ?

S

> -----Original Message-----
> From: Benjamin Pierce [mailto:bcpierce@...]
> Sent: November 12, 2009 20:23
> To: Urbaniak, Sebastian
> Cc: Jason Axelson; unison-users@yahoogroups.com
> Subject: Re: [unison-users] Unison "Lost connection with server"
>
> No, this doesn't fit Unison's view of the world very well.
>
> I believe what's needed is to change the update detection code so
> that, if it notices it is taking a long time, it quits early (without
> scanning all of the replica), writes out the archive files, and then
> restarts the scanning process.  I don't think this would be too hard
> for someone to implement, but it does involve touching the Update
> module, which is a little tangled.
>
>      - B

dear unisoners,

sorry for this long post.

before reading along: the following happened with 2.27 since
this is currently the best one can get from MacPorts:

I  ran into massive problems when trying to sync between two
macos X machines, the first (old) one being  a  PPC  running
10.4, the second (new) being a macbook running 10.6.

I  first  copied  most  of  the  big chunks manually to help
unison along and then tried a standard unsion sync  of  both
machines.

these  led  to  many  apparent(?)  conflicts concerning file
props, although the reported  permissions  (when  using  'x'
during the conflict resolution) where exactly the same.

since  the number of 'conflicts' was way to large to confirm
interactively I then tried a

'unison force'

forcing  the  old root for all conflicts (assuming that they
concerned only spurious props differences).

but in the _very_ long list of detected conflicts (thousands
of  them)  there  where,  beside  the  maybe  95%    'props'
conflicts,   also   quite  some  'files  differ'   conflicts
although, as mentioned, I had transferred the affected files
manually  just before the unison run from the old to the new
machine.

so  my FIRST QUESTION is: what are these 'file differences'?
do they indicate actual differences  (which  would  be  very
strange  since  I do'nt assume that the previous manual copy
had any problems)? can I trust the transferred data  on  the
target machine for these files?

I   nevertheless  confirmed  the  run  (as  long  as  it  is
unidirectional I think I cannot mess  up  something  on  the
first machine ...).

this  produced some 150 errors (out of some 10^4 transferred
files) a la

Failed: The file .unison.Dr.doc.6711619009f4bc8f8ae6196d85070fc1.unison.tmp was
incorrectly transferred (fingerprint mismatch)

in  the  logfile  I  see  that the affected files seem to be
mostly ms-word .doc files (should be expected  anyway  :-)),
quicktime movies and pdf-files but I'm not sure whether that
has any significance.

my  SECOND  QUESTION is: is this a known 2.27 problem? I did
not find anything helpful in the mail archive.

I've mailed the macports maintainer of unison in parallel. a
few weeks ago  they  had  still  problems  getting  the  GUI
running  under  10.6  and  therefore  did  not  upgrade  the
package.

my  THIRD QUESTION is: if the macports people cannot provide
this for now, is there a well-defined way of getting the CLI
version   of   unison  running  under  10.6?  I've  got  the
impression that it's not as simple as calling ocamopt :-)


any  advice  what  to  do to make unison (and thus me) happy
again would be greatly appreciated.

best

joerg

Can unison syncronize advanced ACL, along with the chmod bits?

I added group:staff:rwx permissions to a file in unison_root1 directory, also
there is default mask:: and these don't seem to be synced to the second
root(unison_test4).

-bash-3.1$ getfacl /unison_root1/AddGroup
# file: unison_root1/AddGroup
# owner: asong
# group: asong
user::rwx
group::rwx
group:staff:rwx
mask::rwx
other::r--

-bash-3.1$ getfacl /unison_test4/AddGroup
# file: unison_test4/AddGroup
# owner: asong
# group: asong
user::rwx
group::rwx
other::r--

On Fri, Nov 13, 2009 at 6:32 PM, a227song <a.song@...> wrote:
> Can unison syncronize advanced ACL, along with the chmod bits?

I don't think so.

Alan

I am getting this error more and more recently:



   Waiting for changes from server
Reconciling changes
          error            configs/apps/Memeo
Error in scanning directory:
Error performing inpage operation.
  [opendir()]
No updates to propagate



Any ideas what I can do to make it not fail on these 'inpage operations'
?

Running: unison.win32-2.27.57-Text.exe         on both ends.

Hi Joerg,

> I've mailed the macports maintainer of unison in parallel. a
> few weeks ago  they  had  still  problems  getting  the  GUI
> running  under  10.6  and  therefore  did  not  upgrade  the
> package.

Oh, really?

I submitted minimal patches to MacPorts BTS which makes unison-2.27.157/2.32.52
works with MacGUI on MacOSX 10.6(64-bit):
http://trac.macports.org/ticket/21862
It is working without serious error on my Mac.

Please tell me the detail of the problems about GUI support which prevent to
upgrade the package.
I will try to fix the problems and re-submit updated patches.

-- Kiyoshi

Hello

I am trying to to sync some folders between a local machine and folders that are
mapped(network folder mapping) over Vpn(distant) under Windows. Both sides are
windows(server and the client). For some reason Unison spends incredible amount
of time per file, maybe couple mins per file. I am not sure if this is expected
but does not sound right. When I use another application to sync I can get
easily 80 kb/sec between my computer and the server so the speed is not like
super slow.

Any ideas why that might be happening?


thanks

On Wed, Nov 18, 2009 at 5:03 AM, dedebaykus <dedebaykus@...> wrote:
> Hello
>
> I am trying to to sync some folders between a local machine and folders that
are mapped(network folder mapping) over Vpn(distant) under Windows. Both sides
are windows(server and the client). For some reason Unison spends incredible
amount of time per file, maybe couple mins per file. I am not sure if this is
expected but does not sound right. When I use another application to sync I can
get easily 80 kb/sec between my computer and the server so the speed is not like
super slow.

Is it slow during the change detection or during the file transfer?
When you monitor network traffic, do you see a lot of it?

Alan

Hi dedebaykus,

On Wed, Nov 18, 2009 at 05:03, dedebaykus wrote:
> I am trying to to sync some folders between a local machine and folders that
are mapped(network folder mapping) over Vpn(distant) under Windows. Both sides
are windows(server and the client). For some reason Unison spends incredible
amount of time per file, maybe couple mins per file. I am not sure if this is
expected but does not sound right. When I use another application to sync I can
get easily 80 kb/sec between my computer and the server so the speed is not like
super slow.
>
> Any ideas why that might be happening?

The way you are using Unison - by pointing it to a mapped drive - it
is not using the client & server mode. Instead you are using local
mode.
http://www.cis.upenn.edu/~bcpierce/unison/download/releases/stable/unison-manual\
.html#local

Here are a few tips:
  - do the first sync when you are IN the office, not over VPN
  - use "fastcheck = true" otherwise (on Windows) Unison will always
read in the whole files for creating the checksums and comparing it
against the previous state
  - please note that some file extensions are excluded from fastcheck
optimization
http://www.cis.upenn.edu/~bcpierce/unison/download/releases/stable/unison-manual\
.html#fastcheck

HTH, Kai

-----Original Message-----
From: unison-users@yahoogroups.com [mailto:unison-users@yahoogroups.com]On Behalf Of dedebaykus
Sent: Tuesday, November 17, 2009 11:04 PM
To: unison-users@yahoogroups.com
Subject: [unison-users] Incredibly slow VPN Sync

 

Hello

I am trying to to sync some folders between a local machine and folders that are mapped(network folder mapping) over Vpn(distant) under Windows. Both sides are windows(server and the client). For some reason Unison spends incredible amount of time per file, maybe couple mins per file. I am not sure if this is expected but does not sound right. When I use another application to sync I can get easily 80 kb/sec between my computer and the server so the speed is not like super slow.

Any ideas why that might be happening?

thanks

Hello,

I've run into an issue with unison (Debian package version 2.13.16-5) on
FAT filesystems:  If I rename a file from lower case to upper case (eg.
a --> A), unison fails to propagate this change, saying: "Failed:
Destination updated during synchronization"

Probably the reason for this is that in a FAT filesystem "a" and "A"
indeed are the same file, it's only the representation of the filename
which changes.  A simple mv doesn't work either:

$ mv a A
mv: `a' and `A' are the same file

The obvious work-around is:
$ mv a a_
$ mv a_ A

What do you think?  Can it be fixed?

Cheers!

Thiemo Nagel

Hi,

I'm new to your mailing list, and French.

Some times ago, I'd choosen to try unison for automatic backups in a small society.

Things works fine, at the start. Now i'm trapped in some trouble, and asking you some advice/help/anti-depressors.

Some regular informations :

- I use stable version of unison (2.32.52)

- I use the mirroring only mode ( -force switch)

- Backups are made from local network to local HD

(ie: \\pc\the\src\path =>> c:\dest\path )

- Some other switch i use :

* -fastcheck

* -batch

*-logfile

*-backups

*-maxbackups

*-retry

There is 2 things (perhaps not new to you, but i don't find anything in the net).

First one NTFS max length problem:

There's some really long path in the src tree to backup. Unfortunatly, during the operation unison create more longer ones in the destination dir by prefixing and a appending with .unison[.temp] stuffs. The result is that some path for the operation will grow over 255 chars and results in ERRORs.

Is there is some workaround for this ?

Second problem in which i have no clue.

Sometimes on some regular files (after checking nothing particular with these files), unison report an error in the log file saying :

/some/.unison.dir.1212.unison.temp/thefile.ext ERROR FILE EXIST.

Errors repeat over relaunch of unison.

These errors make unison stop the backup operation, leaving some temporary directory in the system.

- Is it possible that unison doesn't handle correctly the temporary stuffs it left in a previously crashed session ?

- Why unison doesn't continue the backuping and only report the error in the log file ?

- Did i miss something ?

Thank's for your patients and useful jobs.

Cheers Patrick

--

J'aime pas le marketing glouton