darcs

Patch 1383 made connection caching in Darcs.Repository.Ssh thread safe

Title made connection caching in Darcs.Repository.Ssh thread safe
Superseder Nosy List bfrk
Related Issues
Status accepted Assigned To
Milestone 2.10.1

Created on 2015-06-28.18:09:28 by bfrk, last changed 2015-08-03.05:31:43 by ganesh.

Files
File name Status Uploaded Type Edit Remove
made-connection-caching-in-darcs_repository_ssh-thread-safe.dpatch bfrk, 2015-06-28.18:09:27 application/x-darcs-patch
made-connection-caching-in-darcs_repository_ssh-thread-safe.dpatch bfrk, 2015-06-28.18:12:16 application/x-darcs-patch
patch-preview.txt bfrk, 2015-06-28.18:09:27 text/x-darcs-patch
patch-preview.txt bfrk, 2015-06-28.18:12:16 text/x-darcs-patch
patch-preview.txt bfrk, 2015-07-03.17:28:52 text/x-darcs-patch
simplified-code-in-d_r_ssh-by-using-withmvar.dpatch bfrk, 2015-07-03.17:28:52 application/x-darcs-patch
unnamed bfrk, 2015-06-28.18:09:27
unnamed bfrk, 2015-06-28.18:12:16
unnamed bfrk, 2015-07-03.17:28:52
See mailing list archives for discussion on individual patches.
Messages
msg18635 (view) Author: bfrk Date: 2015-06-28.18:09:27
This came up when I tested my latest changes to D.R.Packs: I wondered why
this is so extremely slow when I cloned a remote repo over ssh from my home
machine. Then I added --debug and saw that re-using the connection failed
each time.

1 patch for repository http://darcs.net/screened:

patch f4d8302036b82a26296f5201cc6959c836909595
Author: Ben Franksen <benjamin.franksen@helmholtz-berlin.de>
Date:   Sun Jun 28 03:53:30 CEST 2015
  * made connection caching in Darcs.Repository.Ssh thread safe
  
  The module in question had a global variable implemented as an IORef which
  does not play well with concurrent threads. The revised implementation uses
  MVars and also takes care that a connection is only used by one thread at a
  time. This fixes severe performance problems and intermittent failures when
  cloning repos over ssh due to concurrent access from D.R.Packs.
  
  Another problem was that the connections were identified by the user@host
  part only, but not by the directory of the repo. This causes problems when
  e.g. pulling from multiple ssh repos on the same user@host. We now use a
  (user@host, repodir) pair as the key for cached connections.
Attachments
msg18636 (view) Author: bfrk Date: 2015-06-28.18:12:16
Following up with a small re-factoring that depends on the fix.

2 patches for repository http://darcs.net/screened:

patch f4d8302036b82a26296f5201cc6959c836909595
Author: Ben Franksen <benjamin.franksen@helmholtz-berlin.de>
Date:   Sun Jun 28 03:53:30 CEST 2015
  * made connection caching in Darcs.Repository.Ssh thread safe
  
  The module in question had a global variable implemented as an IORef which
  does not play well with concurrent threads. The revised implementation uses
  MVars and also takes care that a connection is only used by one thread at a
  time. This fixes severe performance problems and intermittent failures when
  cloning repos over ssh due to concurrent access from D.R.Packs.
  
  Another problem was that the connections were identified by the user@host
  part only, but not by the directory of the repo. This causes problems when
  e.g. pulling from multiple ssh repos on the same user@host. We now use a
  (user@host, repodir) pair as the key for cached connections.

patch 09283ac15a43a4f182726a583a8901fc474f48b9
Author: Ben Franksen <benjamin.franksen@helmholtz-berlin.de>
Date:   Sun Jun 28 03:44:43 CEST 2015
  * use transferModeHeader from D.R.Ssh in D.U.C.TransferMode
Attachments
msg18642 (view) Author: bfrk Date: 2015-06-28.20:21:28
Bugfix, so should go into 2.10.1.
msg18649 (view) Author: gh Date: 2015-06-30.21:53:54
I agree it would be nice to have (and it applies cleanly to 2.10).  I
have quickly looked at it and it looks sane. It would be nice if
someone could review it.

Guillaume
msg18679 (view) Author: bfrk Date: 2015-07-03.17:28:52
A small re-factoring on top of the ssh connection caching fix.

1 patch for repository http://darcs.net/screened:

patch 050d575189bd701f7a214b7e1d01aebe17ea2ad9
Author: Ben Franksen <benjamin.franksen@helmholtz-berlin.de>
Date:   Thu Jul  2 15:06:28 CEST 2015
  * simplified code in D.R.Ssh by using withMVar
Attachments
msg18710 (view) Author: ganesh Date: 2015-07-30.06:05:03
Looks good to me, sorry for not reviewing it sooner.
History
Date User Action Args
2015-06-28 18:09:28bfrkcreate
2015-06-28 18:12:16bfrksetfiles: + patch-preview.txt, made-connection-caching-in-darcs_repository_ssh-thread-safe.dpatch, unnamed
messages: + msg18636
title: made connection caching in Darcs.Repository.Ssh thread... -> made connection caching in Darcs.Repository.Ssh threadsafe
2015-06-28 18:12:53bfrksettitle: made connection caching in Darcs.Repository.Ssh threadsafe -> made connection caching in Darcs.Repository.Ssh thread safe
2015-06-28 20:21:28bfrksetstatus: needs-screening -> needs-review
messages: + msg18642
milestone: 2.10.1
2015-06-30 21:53:54ghsetmessages: + msg18649
2015-07-03 17:28:53bfrksetfiles: + patch-preview.txt, simplified-code-in-d_r_ssh-by-using-withmvar.dpatch, unnamed
messages: + msg18679
2015-07-30 06:05:04ganeshsetstatus: needs-review -> accepted-pending-tests
messages: + msg18710
2015-08-03 05:31:43ganeshsetstatus: accepted-pending-tests -> accepted