darcs

Patch 2063 add a test for gpg signed send/apply

Title add a test for gpg signed send/apply
Superseder Nosy List bf
Related Issues
Status needs-screening Assigned To
Milestone

Created on 2020-07-20.12:11:04 by bf, last changed 2020-08-01.06:51:58 by ganesh.

Files
File name Status Uploaded Type Edit Remove
add-a-test-for-gng-signed-send_apply.dpatch dead bf, 2020-07-20.12:11:04 application/x-darcs-patch
add-a-test-for-gng-signed-send_apply.dpatch dead bf, 2020-07-27.10:23:44 application/x-darcs-patch
add-a-test-for-gpg-signed-send_apply.dpatch dead bf, 2020-07-27.11:31:38 application/x-darcs-patch
add-a-test-for-gpg-signed-send_apply.dpatch bf, 2020-07-28.08:50:23 application/x-darcs-patch
add-a-test-for-gpg-signed-send_apply.dpatch dead bf, 2020-07-28.11:23:04 application/x-darcs-patch
gpg.txt ganesh, 2020-07-27.10:48:53 text/plain
gpg.txt ganesh, 2020-08-01.06:51:56 text/plain
patch-preview.txt bf, 2020-07-20.12:11:04 text/x-darcs-patch
patch-preview.txt bf, 2020-07-27.10:23:44 text/x-darcs-patch
patch-preview.txt bf, 2020-07-27.11:31:38 text/x-darcs-patch
patch-preview.txt bf, 2020-07-28.08:50:23 text/x-darcs-patch
patch-preview.txt bf, 2020-07-28.11:23:04 text/x-darcs-patch
unnamed bf, 2020-07-20.12:11:04 text/plain
unnamed bf, 2020-07-27.10:23:44 text/plain
unnamed bf, 2020-07-27.11:31:38 text/plain
unnamed bf, 2020-07-28.08:50:23 text/plain
unnamed bf, 2020-07-28.11:23:04 text/plain
See mailing list archives for discussion on individual patches.
Messages
msg22217 (view) Author: bf Date: 2020-07-20.12:11:04
Following up on review of patch2044.

The reason for bundling the .gnupg directory is that key generation requires
lots of system entropy which is pretty soon exhausted if we generate keys in
the test scripts. This will then make gpg --key-gen (and thus the test
suite) hang without apparent reason.

1 patch for repository http://darcs.net/screened:

patch 2556f6181ff4661217b1f22546a5c3969692bdf8
Author: Ben Franksen <ben.franksen@online.de>
Date:   Mon Jul 20 14:14:16 CEST 2020
  * add a test for gng signed send/apply
Attachments
msg22222 (view) Author: ganesh Date: 2020-07-20.22:45:55
FYI the test harness just gets stuck on Windows when I run this test. I
haven't yet figured out why, darcs-test doesn't even seem to have
child processes running when it stops. I will try to debug it properly
more promptly than the previous tests I reported problems with...
msg22226 (view) Author: bf Date: 2020-07-21.09:43:43
> FYI the test harness just gets stuck on Windows when I run this test.
> I haven't yet figured out why, darcs-test doesn't even seem to have 
> child processes running when it stops. I will try to debug it
> properly more promptly than the previous tests I reported problems
> with...

This is most probably because the directory where gpg stores its
internal data is named differently i.e. not $HOME/.gnupg. If this is the
case then we will have to rename the directory after unpacking it on
Windows.
msg22261 (view) Author: ganesh Date: 2020-07-26.00:13:05
>> FYI the test harness just gets stuck on Windows when I run this test.
>> I haven't yet figured out why, darcs-test doesn't even seem to have 
>> child processes running when it stops. I will try to debug it
>> properly more promptly than the previous tests I reported problems
>> with...
> 
> This is most probably because the directory where gpg stores its
> internal data is named differently i.e. not $HOME/.gnupg. If this is the
> case then we will have to rename the directory after unpacking it on
> Windows.

I'm using gpg from mingw/msys and I do have a $HOME/.gnupg

But I experimented with the test a bit more and noticed that gpg-agent
was running trying to use $APPDATA/gnupg, which also has some plausible
looking content. When I kill gpg-agent then the test stops hanging and
fails, even though gpg itself didn't appear to be running at that point.

This is all even though the test is of course supposed to be isolated -
this is probably tricky to do perfectly when running under my user.

So I think I need to figure out exactly what is going on with my GPG
setup. But also I'm now a bit nervous that the test does an 'rm -rf
.gnupg' in case I ever run it in a non-isolated way :-)
msg22267 (view) Author: bf Date: 2020-07-27.10:23:44
I was having similar problems when I tried to run the test remotely via ssh
on another Linux machine. After several trials including re-creating the
gnupg home directory more than once, I have now got it working for me. I
hope this also fixes the problem on Windows.


1 patch for repository http://darcs.net/screened:

patch d18a97df3bde6a17b2ae33ed61b9d655b13e645d
Author: Ben Franksen <ben.franksen@online.de>
Date:   Mon Jul 20 14:14:16 CEST 2020
  * add a test for gng signed send/apply
Attachments
msg22268 (view) Author: ganesh Date: 2020-07-27.10:48:53
fails - see attached. I haven't had a chance to investigate at all.
Attachments
msg22269 (view) Author: ganesh Date: 2020-07-27.10:49:09
PS next time you amend please fix gng => gpg in the description :-)
msg22270 (view) Author: bf Date: 2020-07-27.11:31:38
> fails - see attached.

Hmm. It says

| export DARCS_TEST_GPG_OPTIONS="--homedir $(pwd)/gpghome"
| ++ pwd
| ++ ghc --make -o hspwd /home/ganesh/darcs/screened/tests/bin/hspwd.hs
| ++ ./hspwd
| + export 'DARCS_TEST_GPG_OPTIONS=--homedir
C:\Users\Ganesh\AppData\Local\Temp\tmpC31E/gpghome'
| + DARCS_TEST_GPG_OPTIONS='--homedir
C:\Users\Ganesh\AppData\Local\Temp\tmpC31E/gpghome'

[...]

| gpg: keyblock resource
'/c/Users/Ganesh/AppData/Local/Temp/tmpC31E/S/C:\Users\Ganesh\AppData\Local\Temp\tmpC31E/gpghome/pubring.kbx':
No such file or directory

I guess we should not use the hspwd here, because that means we get a
path with mixed '\\' and '/' separators, which cannot be properly translated
back to a native Windows path. I replaced it with '/bin/pwd'. Might work or
not.

1 patch for repository http://darcs.net/screened:

patch f5c6bd16d406ca5d8ddb0cda2e7417512a9e0c6b
Author: Ben Franksen <ben.franksen@online.de>
Date:   Mon Jul 20 14:14:16 CEST 2020
  * add a test for gpg signed send/apply
Attachments
msg22274 (view) Author: ganesh Date: 2020-07-27.22:23:07
Unfortunately this now gets stuck again until I kill gpg-agent. Before
debugging it, I started googling, and came across this long thread:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=840669

My two takeaways were that gpg is amazingly unfriendly to programmatic
usage, and that we should be setting GNUPGHOME. I'll have a go at that
the next time I look at this patch, if you don't figure something out
first.
msg22275 (view) Author: bf Date: 2020-07-28.08:50:24
Here is yet another version. I removed the DARCS_TEST_GPG_OPTIONS hack in
favour of using GNUPGHOME.

From the thread you linked I learned that to manipulate the gpg-agent
explicitly, you need to use gpgconf.

I added explicit steps to create a socket directory, kill the gpg-agent, and
finally remove the socket directory. Removing the socket directories is
needed because otherwise these directories accumulate and fill up /run/user
each time you run the test.


1 patch for repository http://darcs.net/screened:

patch 83c95346d569717ac2fcda9256d56567371be4bb
Author: Ben Franksen <ben.franksen@online.de>
Date:   Tue Jul 28 10:48:18 CEST 2020
  * add a test for gpg signed send/apply
Attachments
msg22277 (view) Author: bf Date: 2020-07-28.11:23:04
2 patches for repository http://darcs.net/screened:

patch 83c95346d569717ac2fcda9256d56567371be4bb
Author: Ben Franksen <ben.franksen@online.de>
Date:   Tue Jul 28 10:48:18 CEST 2020
  * add a test for gpg signed send/apply

patch 13a28cdd2ada9c1ce2deee15a2cc92facd1c0851
Author: Ben Franksen <ben.franksen@online.de>
Date:   Tue Jul 28 12:08:15 CEST 2020
  * tests/send_apply_signed.sh: also kill gpg-agent initially
Attachments
msg22278 (view) Author: bf Date: 2020-07-28.11:50:47
Please disregard the latest patch I sent. It destroys the ability to 
run the tests in parallel.
msg22290 (view) Author: ganesh Date: 2020-07-30.11:08:23
Do you want me to try the first one (83c95346d569717ac2fcda9256d56567371be4bb)
or to disregard both for now and wait for another update?
msg22298 (view) Author: bf Date: 2020-07-30.12:57:02
Yes, please test the first one 
(83c95346d569717ac2fcda9256d56567371be4bb), ignoring 
13a28cdd2ada9c1ce2deee15a2cc92facd1c0851 which is broken. It's in 
the second to last patch bundle:

http://bugs.darcs.net/file8852/add-a-test-for-gpg-signed-
send_apply.dpatch

If this also hangs I think we have to investigate your gpg 
installation more closely.
msg22340 (view) Author: ganesh Date: 2020-08-01.06:51:56
Failed again. 'gpgconf --create-socketdir' fails even when I run it from the
command-line with no special environment, so I agree something must be up
with my gpg installation in some sense, though I haven't done anything weird
to it deliberately.
Attachments
History
Date User Action Args
2020-07-20 12:11:04bfcreate
2020-07-20 22:45:56ganeshsetmessages: + msg22222
2020-07-21 09:43:44bfsetmessages: + msg22226
2020-07-26 00:13:06ganeshsetmessages: + msg22261
2020-07-27 10:23:45bfsetfiles: + patch-preview.txt, add-a-test-for-gng-signed-send_apply.dpatch, unnamed
messages: + msg22267
2020-07-27 10:48:54ganeshsetfiles: + gpg.txt
messages: + msg22268
2020-07-27 10:49:09ganeshsetmessages: + msg22269
2020-07-27 11:31:39bfsetfiles: + patch-preview.txt, add-a-test-for-gpg-signed-send_apply.dpatch, unnamed
messages: + msg22270
2020-07-27 11:32:27bfsettitle: add a test for gng signed send/apply -> add a test for gpg signed send/apply
2020-07-27 22:23:07ganeshsetmessages: + msg22274
2020-07-28 08:50:25bfsetfiles: + patch-preview.txt, add-a-test-for-gpg-signed-send_apply.dpatch, unnamed
messages: + msg22275
2020-07-28 11:23:04bfsetfiles: + patch-preview.txt, add-a-test-for-gpg-signed-send_apply.dpatch, unnamed
messages: + msg22277
2020-07-28 11:50:48bfsetmessages: + msg22278
2020-07-30 11:08:23ganeshsetmessages: + msg22290
2020-07-30 12:57:02bfsetmessages: + msg22298
2020-08-01 06:51:58ganeshsetfiles: + gpg.txt
messages: + msg22340