Yay!  We have a windows buildslave!

http://allmydata.org/buildbot-darcs/waterfall

But the Windows build fails, in a funny way that I could use some help improving.

I know what's going wrong, and I know how I could configure my Windows machine
to fix it, but then the next person who tries to build on Windows might
encounter this same error and not know what is going wrong. 

The problem is that the autoconf/configure script -- see the log here: 

http://allmydata.org/buildbot-darcs/builders/zooko%20windows/builds/10/steps/configure/logs/stdio


detects that gcc can successfully use -lz to link to the zlib.
but the build: 

http://allmydata.org/buildbot-darcs/builders/zooko%20windows/builds/10/steps/compile/logs/stdio


fails, because ghc cannot link to zlib.  Or more precisely, the gcc that comes
with ghc and is used internally by ghc cannot link to zlib. 

So the technique used in the configure script to determine whether ghc will be
able to use zlib by testing whether gcc can use zlib is to falsely passing.

Which leaves the hypothetical next-person who builds on Windows with a 
build failure: "c:\ghc\ghc-6.8.2\gcc-lib\ld.exe: cannot find -lz"
 
instead of with a configure script message saying "We require libz to build
darcs.", or whatever it says.

So one way to detect whether we have a working libz that would be less
error-prone than the current way
(http://allmydata.org/trac/darcs-2/browser/configure.ac?rev=5553#L522 ) would be
to use the TRY_COMPILE_GHC macro
(http://allmydata.org/trac/darcs-2/browser/aclocal.m4?rev=5385#L7 ) to compile a
minimal program that requires libz.

$ ghc --make -fffi tmp/Zlibtest.hs
[1 of 1] Compiling Main             ( tmp/Zlibtest.hs, tmp/Zlibtest.o )
Linking tmp/Zlibtest ...
tmp/Zlibtest.o: In function `rG0_info':
(.text+0x2a): undefined reference to `gzopen'
collect2: ld returned 1 exit status
[04:14 PM][dagit@olive~]
$ cat tmp/Zlibtest.hs
module Main where
import Foreign.C.String
import Foreign.Ptr


foreign import ccall unsafe "static zlib.h gzopen" c_gzopen
  :: CString -> CString -> IO (Ptr ())

main :: IO ()
main = do empty <- newCString ""
          c_gzopen empty empty
          return ()

I sent zooko a patch that correctly detects the presence of zlib on my system
using TRY_COMPILE_GHC. I'm curious if it can also detect the absence of it :-)

How about just sending this patch to the official repository? If David accepts,
we'll find out if it works through Zooko's waterfall page.

Resolved.  Thanks tux_rocker!

Yes, thank you tux_rocker and droundy!  This made my Windows buildslave go from
a falsely successful configure:

http://allmydata.org/buildbot-darcs/builders/zooko%20windows/builds/19/steps/configure/logs/stdio

followed by a failed compile:

http://allmydata.org/buildbot-darcs/builders/zooko%20windows/builds/19/steps/compile/logs/stdio

to a correctly failed configure:

http://allmydata.org/buildbot-darcs/builders/zooko%20windows/builds/19/steps/compile/logs/stdio

I will now go install zlib on that box.  :-)

So I ran ./configure on my windows system and it said:

checking for library z... no; and not with -lz either
configure: error: Cannot find system's zlib library; please set the LDFLAGS
environment variable!

So then I set the LDFLAGS environment variable to point to where the zlib is
installed, but then it gave the exact same error.  Looking at the config.log
file, it appears that the LDFLAGS variable does not have any influence on ghc:

configure:6702: ghc  -package regex-compat -package QuickCheck -package mtl
-package parsec -package html -package containers -O -funbox-strict
-fields -o conftest conftest.hs
conftest.o(.text+0x11c):fake: undefined reference to `deflate'
collect2: ld returned 1 exit status

So what should that error message say?  Once I know, then I will do it and we
will see if darcs builds against my zlib here.  :-)

Zooko, perhaps you can ask this on the GHC users list? I doubt that  
we have experts on GHC-on-Windows here...

At least you can see that it's not finding the library: so it does  
find the header, otherwise it wouldn't get that far.

Reinier
>

Hmmm... this is definitely another configure-script bug.  It could be that we've
broken the LDFLAGS variable.  You could try passing it as GHCFLAGS (or is it
GHC_FLAGS?), but the correct thing is to fix the configure script to use LDFLAGS
when linking (as the makefile does).

David

Thanks David for enlightening me on this one. Zooko, it seems that  
you could try to put " $LDFLAGS" after "$GHCFLAGS" in aclocal.m4 in  
line 16 and then rerun autoconf. If it works, submit a patch...

I'm just now pushing a change.  It's a bit trickier than that, since ghc expects
LDFLAGS to be preceeded by -optl.  But I think I've gotten it working. 
Incidentally, this *did* work, prior to our fix, since autoconf automatically
handles LDFLAGS and friends.

David

On May 2, 2008, at 10:21 AM, David Roundy wrote:

> I'm just now pushing a change.  It's a bit trickier than that,  
> since ghc expects
> LDFLAGS to be preceeded by -optl.  But I think I've gotten it working.
> Incidentally, this *did* work, prior to our fix, since autoconf  
> automatically
> handles LDFLAGS and friends.

It looks like this caused some buildbots to go from green to red:

http://allmydata.org/buildbot-darcs/waterfall

Regards,

Zooko

Yes, but can't that be fixed by running "autoconf ; ./configure" on those
buildbots?

On May 2, 2008, at 11:16 AM, Reinier Lamers wrote:

> Yes, but can't that be fixed by running "autoconf ; ./configure" on  
> those
> buildbots?

The buildbots programmed to be as near stateless as possible.  They  
start with an empty directory, do a complete darcs checkout into it,  
run autoconf, then run configure, then run make, then run the tests.

The owners and administrators of buildslaves have not been instructed  
to install any special tools or configure anything in the user  
account that the buildslave runs in.

The goal (which may or may not be actually achieved in each case) is  
that the buildslave is testing only a basic, un-tweaked operating  
system environment plus whatever comes from the buildmaster's config  
file [1] and the darcs source code [2].  (The buildmaster's config  
file is, as mentioned above, currently programmed to just do a  
completely clean checkout, autoconf, configure, etc.)

And by the way I just configured it to capture the config.log from  
the configure step so that you can see that file from the status  
page, which might help:

http://allmydata.org/buildbot-darcs/waterfall

Regards,

Zooko

[1] http://allmydata.org/buildbot-darcs-master.cfg
[2] http://darcs.net

On Fri, May 02, 2008 at 11:36:32AM -0600, zooko wrote:
> On May 2, 2008, at 11:16 AM, Reinier Lamers wrote:
> >Yes, but can't that be fixed by running "autoconf ; ./configure" on
> >those buildbots?
> 
> The buildbots programmed to be as near stateless as possible.  They  
> start with an empty directory, do a complete darcs checkout into it,  
> run autoconf, then run configure, then run make, then run the tests.
> 
> The owners and administrators of buildslaves have not been instructed  
> to install any special tools or configure anything in the user  
> account that the buildslave runs in.
> 
> The goal (which may or may not be actually achieved in each case) is  
> that the buildslave is testing only a basic, un-tweaked operating  
> system environment plus whatever comes from the buildmaster's config  
> file [1] and the darcs source code [2].  (The buildmaster's config  
> file is, as mentioned above, currently programmed to just do a  
> completely clean checkout, autoconf, configure, etc.)
> 
> And by the way I just configured it to capture the config.log from  
> the configure step so that you can see that file from the status  
> page, which might help:
> 
> http://allmydata.org/buildbot-darcs/waterfall

I am at a complete loss.  It seems that ghc 6.8.2 is failing for some
reason with the new configure tests, but I can't see any useful error
output.  Unless someone who's got ghc 6.8.2 steps forward to fix this, I'll
roll back to the version that worked fine, but incorrectly failed on
Zooko's cygwin system.
-- 
David Roundy
Department of Physics
Oregon State University

On May 3, 2008, at 10:04 AM, David Roundy wrote:

> I am at a complete loss.  It seems that ghc 6.8.2 is failing for some
> reason with the new configure tests, but I can't see any useful error
> output.  Unless someone who's got ghc 6.8.2 steps forward to fix  
> this, I'll
> roll back to the version that worked fine, but incorrectly failed on
> Zooko's cygwin system.

So the config.log doesn't help?

Here is a configure/build that passed:

http://allmydata.org/buildbot-darcs/builders/ertai/builds/23

(Ertai's buildslave.)

And then David committed this patch and the resulting build failed:

http://allmydata.org/buildbot-darcs/builders/ertai/builds/24

Then I configured the buildmaster to capture config.log and re-ran  
the build:

http://allmydata.org/buildbot-darcs/builders/ertai/builds/25

Hm...  There are some weird non-ascii characters in that config log.   
Search for "does not exist".

Regards,

Zooko

On Sat, May 03, 2008 at 10:24:59AM -0600, zooko wrote:
> On May 3, 2008, at 10:04 AM, David Roundy wrote:
> 
> >I am at a complete loss.  It seems that ghc 6.8.2 is failing for some
> >reason with the new configure tests, but I can't see any useful error
> >output.  Unless someone who's got ghc 6.8.2 steps forward to fix  
> >this, I'll
> >roll back to the version that worked fine, but incorrectly failed on
> >Zooko's cygwin system.
> 
> So the config.log doesn't help?

I've very rarely known a config.log to help...

> Here is a configure/build that passed:
> 
> http://allmydata.org/buildbot-darcs/builders/ertai/builds/23
> 
> (Ertai's buildslave.)
> 
> And then David committed this patch and the resulting build failed:
> 
> http://allmydata.org/buildbot-darcs/builders/ertai/builds/24
> 
> Then I configured the buildmaster to capture config.log and re-ran  
> the build:
> 
> http://allmydata.org/buildbot-darcs/builders/ertai/builds/25
> 
> Hm...  There are some weird non-ascii characters in that config log.   
> Search for "does not exist".

Yes, I saw those weird characters, but have no idea what they mean or where
they came from.  I know that sometimes the buildbot generates/modifies
weird characters, which is one reason why it seems wise for a real human to
try configuring darcs on a machine with ghc 6.8.2.

I'm in the process of doing the stupid thing that I've told myself before
that I should never again attempt:  debugging via the buildbot system by
inserting debug code into the darcs repository.  I imagine I'll regret
this...
-- 
David Roundy
Department of Physics
Oregon State University

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I have ghc 6.8.2 on Linux and I'm going to try to configure darcs now...

Reinier

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iEYEARECAAYFAkgcrEkACgkQR3bgBiWODBfhigCeMikxnM0TB1+CwU7FOWqkmMC8
yVEAn0erjPkEngoPY+1JDIvRF1SghBKe
=p29f
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi all,

It just configured fine and now it's compiling... no problem, it seems.

Looking at the config log in zooko's buildbot page, http:// 
allmydata.org/buildbot-darcs/builders/ertai/builds/29/steps/configure/ 
logs/config.log, we see:

ghc-6.8.2: does not exist: [?1034h

This is the error ghc utters when you give it a nonexistent file  
name. So apparently, ghc starts looking for "[?1034h". That looks  
like it might be some part of a terminal metacharacter (to set your  
font color, for example), but I'm not an expert on such things.

Reinier
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iEYEARECAAYFAkgcsTUACgkQR3bgBiWODBfMqwCfShQ6MwhrfRX5b4QrvAdHI6f2
rYsAnAg95jtod8PcOPIASddOu09FS8wX
=YbOJ
-----END PGP SIGNATURE-----

On May 3, 2008, at 12:31 PM, Reinier Lamers wrote:

> Looking at the config log in zooko's buildbot page, http://
> allmydata.org/buildbot-darcs/builders/ertai/builds/29/steps/configure/
> logs/config.log, we see:
>
> ghc-6.8.2: does not exist: [?1034h
>
> This is the error ghc utters when you give it a nonexistent file
> name. So apparently, ghc starts looking for "[?1034h". That looks
> like it might be some part of a terminal metacharacter (to set your
> font color, for example), but I'm not an expert on such things.

Oh, good clue, Reinier -- could it have to do with spaces in filenames?

I will experiment on my "zooko nexenta" system.

--Z

You can see that that string [?1034h seems to end up in the  
GHCLDFLAGS and GHCLIBS variables.

Grepping for GHCLIBS, I find among other stuff:

GHCLIBS=$(echo $LIBS | $GHC -e 'getContents >>= putStrLn . unwords .  
map ("-optl"++) . words')

Googling for 1034h, I see that it's appeared to other people who pipe  
stuff through ghc (http://ircarchive.info/haskell/2007/4/16/23.html).  
So that line is highly suspect to me.

So, uh, I don't take it has anything in particular to do with spaces  
in filenames.

Reinier

I submitted a patch replacing that GHC call by pure bash code. Let's hope it helps.