darcs

Issue 504 wish: Problem with darcsdir distribution when using SSH_PORT

Title wish: Problem with darcsdir distribution when using SSH_PORT
Priority wishlist Status duplicate
Milestone Resolved in
Superseder accept URIs for ssh repos (eg sftp://droundy@darcs.net:10022/foo/bar), darcs assumes -p for SSH_PORT; but putty takes -P, use SSH URI to distinguish between ssh implementations
View: 1575, 1371, 1576
Nosy List darcs-devel, dlambe, dmitry.kurochkin, eivuokko, jaredj, kirby, kowey, markstos, rgm, thorkilnaur, tommy, wglozer
Assigned To
Topics SSH, Windows

Created on 2007-08-01.23:10:30 by dlambe, last changed 2009-10-24.00:07:50 by admin.

Messages
msg1971 (view) Author: dlambe Date: 2007-08-01.23:10:29
I am using the darcsdir-cygwin-1.0.9.tar.bz2 distribution of darcs 
provided by zooko.com on Windows XP.  I'm using the SSH_PORT environment 
variable to connect to an SSH server that is running on a non-standard 
port.  Darcs calls ssh with the -p option to set the alternate port, but 
since your ssh.exe is really PuTTY's plink.exe it's looking for -P 
(capital 'p') instead and certain operations (push, at least) fail with
	plink: unknown option "-p"

I've hacked around this problem on my installation by renaming ssh.exe 
to realssh.exe and building ssh.exe from this C source:

     #include <stdio.h>
     #include <unistd.h>

     int main(int argc, char **argv)
     {
         int i;

         for (i = 0; i < argc; i++) {
             if (argv[i][0] == '-' && argv[i][1] == 'p')
                 argv[i][1] = 'P';
         }

         execvp("realssh.exe", argv);
     	return 0;
     }

This works for me, but it's not terribly elegant.  Patching darcs  on 
win32 to supply PuTTY-style command line arguments is probably the 
cleanest solution without dragging in a cygwin dependency (for the real 
ssh.exe).
-- 
Dennis Lambe Jr.
Preferred Utilities
<dlambe@preferred-mfg.com>

"The key to performance is elegance, not battalions of special cases."
--Jon Bentley and Doug McIlroy
msg1972 (view) Author: droundy Date: 2007-08-01.23:24:03
On Wed, Aug 01, 2007 at 11:10:30PM -0000, Dennis Lambe wrote:
> This works for me, but it's not terribly elegant.  Patching darcs  on 
> win32 to supply PuTTY-style command line arguments is probably the 
> cleanest solution without dragging in a cygwin dependency (for the real 
> ssh.exe).

I am not a Windows developer (IANAWD), but wouldn't this break useage of
darcs with openssh on Windows?
-- 
David Roundy
Department of Physics
Oregon State University
msg1973 (view) Author: dlambe Date: 2007-08-01.23:44:08
David Roundy wrote:
> David Roundy <droundy@darcs.net> added the comment:
> 
> On Wed, Aug 01, 2007 at 11:10:30PM -0000, Dennis Lambe wrote:
>> This works for me, but it's not terribly elegant.  Patching darcs  on 
>> win32 to supply PuTTY-style command line arguments is probably the 
>> cleanest solution without dragging in a cygwin dependency (for the real 
>> ssh.exe).
> 
> I am not a Windows developer (IANAWD), but wouldn't this break useage of
> darcs with openssh on Windows?

Yeah, it would.

The basic problem is that the way zooko's distro does things is a tool 
renaming hack that *almost* works. (no offense to zooko)

If darcs is serious about supporting multiple SSH clients on Windows, 
then it would be better to stop pretending that plink, pscp and psftp 
are drop-in replacements for OpenSSH's ssh, scp and sftp.

Instead you could have code to support the different command line 
syntaxes of the two SSH suites and select which syntax to use by ether
1) A build flag (#if BUILD_FOR_PUTTY or something)
2) A command-line option (--use-putty --use-openssh)
3) A search through the PATH at run-time for either of those sets of 
binaries (by name).

Putty's command line syntax (which is simpler because it is homogeneous 
throughout the suite) is documented at 
http://the.earth.li/~sgtatham/putty/0.60/htmldoc/Chapter3.html#using-cmdline
-- 
Dennis Lambe Jr.
Preferred Utilities
<dlambe@preferred-mfg.com>

"The key to performance is elegance, not battalions of special cases."
--Jon Bentley and Doug McIlroy
msg1975 (view) Author: kowey Date: 2007-08-02.04:23:23
On Wed, Aug 01, 2007 at 23:44:09 -0000, Dennis Lambe wrote:
> If darcs is serious about supporting multiple SSH clients on Windows, 
> then it would be better to stop pretending that plink, pscp and psftp 
> are drop-in replacements for OpenSSH's ssh, scp and sftp.

Well put.  I wish it were reasonable for us to say 'sorry, we only know
how to use OpenSSH'

> Instead you could have code to support the different command line 
> syntaxes of the two SSH suites and select which syntax to use by ether
> 1) A build flag (#if BUILD_FOR_PUTTY or something)
> 2) A command-line option (--use-putty --use-openssh)
> 3) A search through the PATH at run-time for either of those sets of 
> binaries (by name).

Hmm.  This reminds me of issue362 (there is a commercial SSH whose sftp
command switches the meaning of -B and -b).  I suppose one thing we
could do is to have some really modular code that supports multiple ssh
implementations. Something like

data SSHImpl = SSHImpl { name :: String,
                       , isImplOf :: IO Bool
                       , copyFile :: FilePath -> IO ()    -- scp
                       , copyFiles :: [FilePath] -> IO () -- sftp
                       , executeRemotely :: String -> IO  -- ssh }

putty, opennssh :: SSHImpl
putty = SSHImpl "putty" isPutty puttyCopyFile puttyCopyFiles...
openssh = ...

This would code would either assume that the user uses the same
implementation everywhere (if plink, then pscp) or have a separate
check for each command.
msg1976 (view) Author: droundy Date: 2007-08-02.16:44:35
On Thu, Aug 02, 2007 at 04:23:24AM -0000, Eric Kow wrote:
> On Wed, Aug 01, 2007 at 23:44:09 -0000, Dennis Lambe wrote:
> > If darcs is serious about supporting multiple SSH clients on Windows, 
> > then it would be better to stop pretending that plink, pscp and psftp 
> > are drop-in replacements for OpenSSH's ssh, scp and sftp.
> 
> Well put.  I wish it were reasonable for us to say 'sorry, we only know
> how to use OpenSSH'

What if we were to introduce a putty-specific URL:

putty://droundy@darcs.net:foo/bar?

Then we wouldn't need to guess at runtime how to call ssh.

> > Instead you could have code to support the different command line 
> > syntaxes of the two SSH suites and select which syntax to use by ether
> > 1) A build flag (#if BUILD_FOR_PUTTY or something)
> > 2) A command-line option (--use-putty --use-openssh)
> > 3) A search through the PATH at run-time for either of those sets of 
> > binaries (by name).
> 
> Hmm.  This reminds me of issue362 (there is a commercial SSH whose sftp
> command switches the meaning of -B and -b).  I suppose one thing we
> could do is to have some really modular code that supports multiple ssh
> implementations. Something like
> 
> data SSHImpl = SSHImpl { name :: String,
>                        , isImplOf :: IO Bool
>                        , copyFile :: FilePath -> IO ()    -- scp
>                        , copyFiles :: [FilePath] -> IO () -- sftp
>                        , executeRemotely :: String -> IO  -- ssh }
> 
> putty, opennssh :: SSHImpl
> putty = SSHImpl "putty" isPutty puttyCopyFile puttyCopyFiles...
> openssh = ...
> 
> This would code would either assume that the user uses the same
> implementation everywhere (if plink, then pscp) or have a separate
> check for each command.

This would be a nice approach, if we can reliably figure out which ssh
we're talking with.  Putty might be pretty easy, but distinguishing between
"standard" ssh's might be tricky.  :(
-- 
David Roundy
Department of Physics
Oregon State University
msg1979 (view) Author: dlambe Date: 2007-08-02.17:16:25
> What if we were to introduce a putty-specific URL:
> 
> putty://droundy@darcs.net:foo/bar?
> 
> Then we wouldn't need to guess at runtime how to call ssh.

I'd suggest putty://droundy@darcs.net/foo/bar (with a slash instead of a 
colon separating host from path) in order to make it legal URL syntax.

At the risk of getting off-topic for this bug report, I'd like to 
suggest that the SSH syntax be amended to support URL style as well.  My 
reason is pragmatic: there's currently no way to describe an ssh 
repository on a non-standard port.  You have to remember to set SSH_PORT 
before running any darcs commands on that repo (and un-set it when 
you're done).  URL syntax would allow something like 
ssh://droundy@darcs.net:10022/foo/bar to be used on command lines and 
stored in prefs/repos.  The port is, afterall, a property of the remote 
repository, not of the local machine or local login session.
-- 
Dennis Lambe Jr.
Preferred Utilities
<dlambe@preferred-mfg.com>

"The key to performance is elegance, not battalions of special cases."
--Jon Bentley and Doug McIlroy
msg1980 (view) Author: droundy Date: 2007-08-02.17:29:48
On Thu, Aug 02, 2007 at 05:16:26PM -0000, Dennis Lambe wrote:
> I'd suggest putty://droundy@darcs.net/foo/bar (with a slash instead of a 
> colon separating host from path) in order to make it legal URL syntax.

Sounds good.

> At the risk of getting off-topic for this bug report, I'd like to 
> suggest that the SSH syntax be amended to support URL style as well.  My 
> reason is pragmatic: there's currently no way to describe an ssh 
> repository on a non-standard port.  You have to remember to set SSH_PORT 
> before running any darcs commands on that repo (and un-set it when 
> you're done).  URL syntax would allow something like 
> ssh://droundy@darcs.net:10022/foo/bar to be used on command lines and 
> stored in prefs/repos.  The port is, afterall, a property of the remote 
> repository, not of the local machine or local login session.

Sounds even better, and I'm not sure how off-topic we really are... because
I've forgotten what the bug report was.  :)

Actually, we might use the URL format that konqueror uses, which I think is
something like:

sftp://droundy@darcs.net:10022/foo/bar

I have no idea, maybe this is even somewhat standard.

And actually, if we used this format, putty users would be able to use the
existing mechanisms for redefining interactions for URL types
(DARCS_GET_FOO, DARCS_MGET_FOO and DARCS_APPLY_FOO, see
http://www.darcs.net/manual/node5.html#SECTION00520000000000000000).

Actually, I believe Zooko's windows distribution could switch to this
approach immediately, without modifying darcs itself, provided someone can
convince windows users to use this sort of URL...
-- 
David Roundy
Department of Physics
Oregon State University
msg1981 (view) Author: dlambe Date: 2007-08-02.18:11:08
> Actually, we might use the URL format that konqueror uses, which I think is
> something like:
> 
> sftp://droundy@darcs.net:10022/foo/bar
> 
> I have no idea, maybe this is even somewhat standard.

Somewhat.  Nautilus supports sftp:// as well as ssh://.  I think that 
between ssh:// and sftp://, you've got your bases pretty well covered, 
but you might want to consider having both as synonyms.

> And actually, if we used this format, putty users would be able to use the
> existing mechanisms for redefining interactions for URL types
> (DARCS_GET_FOO, DARCS_MGET_FOO and DARCS_APPLY_FOO, see
> http://www.darcs.net/manual/node5.html#SECTION00520000000000000000).
> 
> Actually, I believe Zooko's windows distribution could switch to this
> approach immediately, without modifying darcs itself, provided someone can
> convince windows users to use this sort of URL...

Well, PuTTY supports the telnet:// schema, but doesn't support any URL 
format for SSH connections.  No matter what, I think at some point the 
URLs will have to be translated into command line syntax for whatever 
SSH backend you're planning on using.
-- 
Dennis Lambe Jr.
Preferred Utilities
<dlambe@preferred-mfg.com>

"The key to performance is elegance, not battalions of special cases."
--Jon Bentley and Doug McIlroy
msg1982 (view) Author: droundy Date: 2007-08-03.00:02:50
On Thu, Aug 02, 2007 at 06:11:09PM -0000, Dennis Lambe wrote:
> Dennis Lambe <dlambe@preferred-mfg.com> added the comment:
> > And actually, if we used this format, putty users would be able to use the
> > existing mechanisms for redefining interactions for URL types
> > (DARCS_GET_FOO, DARCS_MGET_FOO and DARCS_APPLY_FOO, see
> > http://www.darcs.net/manual/node5.html#SECTION00520000000000000000).
> > 
> > Actually, I believe Zooko's windows distribution could switch to this
> > approach immediately, without modifying darcs itself, provided someone can
> > convince windows users to use this sort of URL...
> 
> Well, PuTTY supports the telnet:// schema, but doesn't support any URL 
> format for SSH connections.  No matter what, I think at some point the 
> URLs will have to be translated into command line syntax for whatever 
> SSH backend you're planning on using.

Right, but you can do that right now!  (...in whatever programming language
you wish!)  :) All you need do is write one little program that accepts a
sftp://user@host URL and calls the equivalent of scp user@host ., and one
little program that accepts the same URL and pipes its standard input to
ssh user@host, and then define two environment variables giving darcs the
names of those two programs.
-- 
David Roundy
Department of Physics
Oregon State University
msg1987 (view) Author: dlambe Date: 2007-08-03.14:58:16
> Right, but you can do that right now!  (...in whatever programming language
> you wish!)  :) All you need do is write one little program that accepts a
> sftp://user@host URL and calls the equivalent of scp user@host ., and one
> little program that accepts the same URL and pipes its standard input to
> ssh user@host, and then define two environment variables giving darcs the
> names of those two programs.

An excellent point!  It's actually not that simple in Windows - since 
there's no concept of shell scripts, it has to be a language that 
compiles to .exe files (native or .NET).  Fortunately, Haskell is a 
compiled language!  If I make something worth distributing, I'll attach 
it to this bug report.
-- 
Dennis Lambe Jr.
Preferred Utilities
<dlambe@preferred-mfg.com>

"The key to performance is elegance, not battalions of special cases."
--Jon Bentley and Doug McIlroy
msg2021 (view) Author: dlambe Date: 2007-08-07.23:40:38
> An excellent point!  It's actually not that simple in Windows - since 
> there's no concept of shell scripts, it has to be a language that 
> compiles to .exe files (native or .NET).  Fortunately, Haskell is a 
> compiled language!  If I make something worth distributing, I'll attach 
> it to this bug report.

I have the first version of a set of wrappers around plink and psftp 
implementing SSH URL support with PuTTY as a back-end.  URLs starting 
with ssh:// and sftp:// are both accepted, and both expect absolute 
paths relative to / on the remote server.

darcs get http://malsyned.net/repos/darcsputty

They're a little rough around the edges, but not much more than the 
current solution and they offer the benefit of supporting the URL format 
we've been discussing including both username and password in the URL 
(but consider the security implications wrt 
_darcs/prefs/{repos,defaultrepo}), and of being guaranteed to work with 
PuTTY out of the box including push support.

These programs all expect to find psftp and plink in your path.

The only bug that I know of is that the programs will hang darcs if you 
try to run them without a password in the URL on a repository that won't 
authenticate against your public key.  I belive this was a bug in 
Zooko's putty-based distro as well, though, and I don't think you can 
get around this bug with the current interface, since you can't count on 
being able to read the password from stdin.

Zooko: Do you have any interest in making something like this part of 
your darcs w32 distribution?

David: I've used runInteractiveProcess in these tools to communicate 
with the PuTTY processes.  If I try to integrate ssh:// and sftp:// 
support into a patch to darcs itself, would you prefer I try to rewrite 
in terms of darcs' "exec" function? The only reason I can see for that 
function is if runInteractiveProcess is not reliable on all platforms, 
but I don't have any information on whether it is or not.
-- 
Dennis Lambe Jr.
Preferred Utilities
<dlambe@preferred-mfg.com>

"The key to performance is elegance, not battalions of special cases."
--Jon Bentley and Doug McIlroy
msg3282 (view) Author: markstos Date: 2008-02-09.19:24:27
Could someone provide an update on this ticket? I know the "plink" error has
been addressed. Does that close this bug, or is there still more to be done here?
msg8540 (view) Author: kowey Date: 2009-08-27.15:43:58
Update as requested:

1. Original msg1971 observed that the Darcs assumption that ssh accepts -p
breaks for putty (which takes -P).  This is now taken over by issue1371

2. issue362 is slightly related dealing with a commercial SSH that has different
feelings about -B/-b than OpenSSH

3. David proposed that Darcs to include native support for URLs like:
putty://droundy@darcs.net:foo/bar which may eliminate the problem of how to
figure out which SSH we're talking to. (Now issue1576)

4. This later expanded to a proposal that Darcs support URL format like
sftp://droundy@darcs.net:10022/foo/bar (issue1575)

5. David seized on #4 to generalise away #3, suggesting that the
putty-recognition code be moved out to third party wrapper called via the
DARCS_GET_FOO mechanism.

4. Dennis implemented putty wrapper scripts (in Haskell) at:  darcs get
http://malsyned.net/repos/darcsputty

5. I'll note that in Darcs-2 we introduced darcs transfer-mode which makes it
seem like just relying on DARCS_GET_FOO would be problematic (performance-wise).

I think all the pieces of this issue are in separate tickets now, so I'm closing
this as a duplicate
History
Date User Action Args
2007-08-01 23:10:30dlambecreate
2007-08-01 23:24:04droundysetstatus: unread -> unknown
messages: + msg1972
2007-08-01 23:44:09dlambesetmessages: + msg1973
2007-08-02 04:23:24koweysetmessages: + msg1975
2007-08-02 04:30:12koweysettitle: Problem with darcsdir distribution when using SSH_PORT -> Framework for using different SSH implementations
2007-08-02 04:30:33koweylinkissue362 superseder
2007-08-02 04:31:10koweysettopic: + SSH
nosy: + rgm
2007-08-02 16:44:36droundysetmessages: + msg1976
title: Framework for using different SSH implementations -> Problem with darcsdir distribution when using SSH_PORT
2007-08-02 17:16:26dlambesetmessages: + msg1979
2007-08-02 17:29:50droundysetmessages: + msg1980
2007-08-02 18:11:10dlambesetmessages: + msg1981
2007-08-03 00:02:52droundysetmessages: + msg1982
2007-08-03 14:58:21dlambesetmessages: + msg1987
2007-08-07 23:40:39dlambesetmessages: + msg2021
2008-02-09 19:24:28markstossetstatus: unknown -> deferred
nosy: + markstos
messages: + msg3282
title: Problem with darcsdir distribution when using SSH_PORT -> wish: Problem with darcsdir distribution when using SSH_PORT
2009-08-06 17:40:18adminsetnosy: + jast, Serware, dmitry.kurochkin, darcs-devel, zooko, dagit, mornfall, simon, thorkilnaur, - droundy, rgm, dlambe
2009-08-06 20:37:28adminsetnosy: - beschmi
2009-08-10 22:06:40adminsetnosy: + rgm, dlambe, - darcs-devel, zooko, jast, dagit, Serware, mornfall
2009-08-25 17:53:53adminsetnosy: + darcs-devel, - simon
2009-08-27 13:54:46adminsetnosy: tommy, kowey, markstos, darcs-devel, rgm, thorkilnaur, dlambe, dmitry.kurochkin
2009-08-27 15:44:04koweysetstatus: deferred -> duplicate
nosy: + wglozer, eivuokko, kirby, jaredj
topic: + Windows
superseder: + darcs assumes -p for SSH_PORT; but putty takes -P, accept URIs for ssh repos (eg sftp://droundy@darcs.net:10022/foo/bar), use SSH URI to distinguish between ssh implementations
messages: + msg8540
2009-10-23 22:42:00adminsetnosy: + robmoss, - rgm
2009-10-24 00:07:50adminsetnosy: + rgm, - robmoss