[packman] Repository switch delayed

[Pascal Bleser]

We didn't do the switch to the new repositories today, as was
announced, because Marc and I have been working on how to
properly transfer the repositories from our OBS instance to the
primary server all night, but given the extreme complexity of
the matter, we didn't manage to get done with it.

We will continue to work on it tonight, and as we managed to at
least find the right concept, it should be working very soon,
hopefully tonight (CET).

JFYI, in essence, the issue is properly synchronizing OBS
pushing with rsync and mirrors pulling from there with rsync.
And now that you think that you have an easy solution, stop
before replying and think about it again ;)

The plan, right now, is to
* have our OBS rsync diffs to a staging directory, instead of
  into the real package tree directly (with rsync's very, very
  neat --compare-dest feature)
* a cron job on the main server will, on a regular basis:
  * disable/block rsync for our OBS
  * disable/block rsync for the mirrors
  * wait for any running rsyncs to finish
  * rsync locally from the staging tree into the real tree
  * verify and re-sign the packages with the key that is used to
    sign the repository metadata (to avoid the issues people
    have had for so long with the signature verification on our
    packages)
  * re-enable rsync for OBS and mirrors

The issue, basically, is to have a mechanism that gives a window
of exclusive access to the file trees to that cron job on the
main server, to avoid having incomplete and corrupt repositories
(because e.g. our OBS is pushing files while mirrors are pulling
from the same directory).

If anyone else has an idea how to safely and atomically
lock/sync processes across the network (with only ssh), please
let me know ;) (and yes, we did try *many* things before giving
up and choosing to let the cron job cut off rsync for our OBS
instance, including many attempts at ssh'ing with flock etc...).

cheers
-- 
  -o) Pascal Bleser
  /\\ http://opensuse.org -- we haz green
 _\_v http://fosdem.org   -- we haz conf
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 190 bytes
Desc: not available
URL: <http://lists.links2linux.de/pipermail/packman/attachments/20110303/12aa1362/attachment.sig>