Git help please - checking out old commit doesn't find lib/gnulib.mk.in.

classic Classic list List threaded Threaded
26 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Git help please - checking out old commit doesn't find lib/gnulib.mk.in.

Alan Mackenzie
Hello, Emacs.

I'm attempting to check out a moderately old version of Emacs with:

    $ git checkout 10c0e1c

.  This appears to work.  I then attempt to build it with:

    $ time make -j17 bootstrap

.  This fails with:

    make: *** No rule to make target 'lib/gnulib.mk.in', needed by 'Makefile'.  Stop.

.  It would appear that the git checkout has failed to restore the state
of the given commit, in that it hasn't restored the indicated file.  Is
this in fact the case?

What do I have to do to restore this version in a state I can build.
(System: 64-bit GNU (Gentoo)).

--
Alan Mackenzie (Nuremberg, Germany).

Reply | Threaded
Open this post in threaded view
|

Re: Git help please - checking out old commit doesn't find lib/gnulib.mk.in.

Andreas Schwab-2
On Nov 12 2017, Alan Mackenzie <[hidden email]> wrote:

> Hello, Emacs.
>
> I'm attempting to check out a moderately old version of Emacs with:
>
>     $ git checkout 10c0e1c
>
> .  This appears to work.  I then attempt to build it with:
>
>     $ time make -j17 bootstrap
>
> .  This fails with:
>
>     make: *** No rule to make target 'lib/gnulib.mk.in', needed by 'Makefile'.  Stop.
>
> .  It would appear that the git checkout has failed to restore the state
> of the given commit, in that it hasn't restored the indicated file.  Is
> this in fact the case?

Makefile and a lot of other files aren't tracked, so they cannot be
restored in any way.  You need to rerun autogen.sh.

Andreas.

--
Andreas Schwab, [hidden email]
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Reply | Threaded
Open this post in threaded view
|

Re: Git help please - checking out old commit doesn't find lib/gnulib.mk.in.

Óscar Fuentes
In reply to this post by Alan Mackenzie
Alan Mackenzie <[hidden email]> writes:

> Hello, Emacs.
>
> I'm attempting to check out a moderately old version of Emacs with:
>
>     $ git checkout 10c0e1c
>
> .  This appears to work.  I then attempt to build it with:
>
>     $ time make -j17 bootstrap
>
> .  This fails with:
>
>     make: *** No rule to make target 'lib/gnulib.mk.in', needed by 'Makefile'.  Stop.
>
> .  It would appear that the git checkout has failed to restore the state
> of the given commit, in that it hasn't restored the indicated file.  Is
> this in fact the case?
>
> What do I have to do to restore this version in a state I can build.
> (System: 64-bit GNU (Gentoo)).

lib/gnulib.mk.in was introduced on March 2017 while the version you are
checking out corresponds to February 2016. Hence it is expected that the
file is missing.

It seems that you are re-using a build directory, so the problem you see
is expected as well.

As the build system (and Emacs in general) changed quite a bit since,
I'll start with a pristine state. In you emacs checkout:

$ git clean -f -d -x

This will delete everything that is not versioned (build artifacts,
etc). Then create a new build directory and proceed as usual.


Reply | Threaded
Open this post in threaded view
|

Re: Git help please - checking out old commit doesn't find lib/gnulib.mk.in.

Lele Gaifax
In reply to this post by Alan Mackenzie
Alan Mackenzie <[hidden email]> writes:

> Hello, Emacs.
>
> I'm attempting to check out a moderately old version of Emacs with:
>
>     $ git checkout 10c0e1c

$ git show 10c0e1c | head -3
commit 10c0e1ca40237224aa229c538fe49983ec905748
Author: Alan Mackenzie <[hidden email]>
Date:   Fri Feb 19 17:10:57 2016 +0000

>     make: *** No rule to make target 'lib/gnulib.mk.in', needed by 'Makefile'.  Stop.

$ git log lib/gnulib.mk.in | grep '^Date' | tail -2
Date:   Sun Mar 19 13:07:03 2017 -0700
Date:   Fri Mar 17 11:33:47 2017 -0700

It seems that at that time it wasn't there yet, so maybe the checkout of that
old version left some "debris" that confuses the makefiles?

What does "git status" gives you? Does it report "untracked files"? If so,
maybe a "git clean" could remove the problem...

ciao, lele.
--
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
[hidden email]  |                 -- Fortunato Depero, 1929.


Reply | Threaded
Open this post in threaded view
|

Re: Git help please - checking out old commit doesn't find lib/gnulib.mk.in.

John Wiegley-6
In reply to this post by Alan Mackenzie
>>>>> "AM" == Alan Mackenzie <[hidden email]> writes:

AM> I'm attempting to check out a moderately old version of Emacs with: $ git
AM> checkout 10c0e1c
AM> .  This appears to work.  I then attempt to build it with:
AM>     $ time make -j17 bootstrap
AM> .  This fails with:
AM>     make: *** No rule to make target 'lib/gnulib.mk.in', needed by
AM> 'Makefile'. Stop.

It's missing from that working tree at that commit. I see it added back in by
6bbbc38b3421723521f7cdd4fd617a4fc889aceb, for example.

--
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Reply | Threaded
Open this post in threaded view
|

Re: Git help please - checking out old commit doesn't find lib/gnulib.mk.in.

Eli Zaretskii
In reply to this post by Alan Mackenzie
> Date: Sun, 12 Nov 2017 18:09:59 +0000
> From: Alan Mackenzie <[hidden email]>
>
> I'm attempting to check out a moderately old version of Emacs with:
>
>     $ git checkout 10c0e1c
>
> .  This appears to work.  I then attempt to build it with:
>
>     $ time make -j17 bootstrap
>
> .  This fails with:
>
>     make: *** No rule to make target 'lib/gnulib.mk.in', needed by 'Makefile'.  Stop.
>
> .  It would appear that the git checkout has failed to restore the state
> of the given commit, in that it hasn't restored the indicated file.  Is
> this in fact the case?
>
> What do I have to do to restore this version in a state I can build.

I think you have some stale files that you need to clean up, basically
anything that is not a versioned file.  For best results, I'd checkout
into a separate directory, starting with "git worktree add".

Reply | Threaded
Open this post in threaded view
|

Re: Git help please - checking out old commit doesn't find lib/gnulib.mk.in.

Alan Mackenzie
Hello, Eli.

On Sun, Nov 12, 2017 at 20:29:26 +0200, Eli Zaretskii wrote:
> > Date: Sun, 12 Nov 2017 18:09:59 +0000
> > From: Alan Mackenzie <[hidden email]>

> > I'm attempting to check out a moderately old version of Emacs with:

> >     $ git checkout 10c0e1c

> > .  This appears to work.  I then attempt to build it with:

> >     $ time make -j17 bootstrap

> > .  This fails with:

> >     make: *** No rule to make target 'lib/gnulib.mk.in', needed by 'Makefile'.  Stop.

> > .  It would appear that the git checkout has failed to restore the state
> > of the given commit, in that it hasn't restored the indicated file.  Is
> > this in fact the case?

> > What do I have to do to restore this version in a state I can build.

> I think you have some stale files that you need to clean up, basically
> anything that is not a versioned file.

I though that was what make bootstrap was for.  In that working
directory, I tried ./autogen.sh all.  This gave the error messages:

Running 'autoreconf -fi -I m4' ...
/usr/bin/m4:aclocal.m4:9: cannot open `m4/count-leading-zeros.m4': No such file or directory
+ 11 similar lines.

> For best results, I'd checkout into a separate directory, starting with
> "git worktree add".

Thanks.  In the end, I just cloned another repository from my own
emacs-26 repo and checked out that version.  This build without problems.

But it would be nice to be able to do what make bootstrap purports to do.
Maybe one of the other less used make targets will do this.

--
Alan Mackenzie (Nuremberg, Germany).

Reply | Threaded
Open this post in threaded view
|

Re: Git help please - checking out old commit doesn't find lib/gnulib.mk.in.

Eli Zaretskii
> Date: Sun, 12 Nov 2017 20:06:51 +0000
> Cc: [hidden email]
> From: Alan Mackenzie <[hidden email]>
>
> > I think you have some stale files that you need to clean up, basically
> > anything that is not a versioned file.
>
> I though that was what make bootstrap was for.

No, you are mistaken.  Bootstrap removes products of compilation, but
not other generated files.  That's what the various "*-clean" targets
are for, but generally it's much more reliable to just checkout or
clone into a fresh directory.

Reply | Threaded
Open this post in threaded view
|

Re: Git help please - checking out old commit doesn't find lib/gnulib.mk.in.

Stefan Monnier
>> I though that was what make bootstrap was for.
> No, you are mistaken.

FWIW, I'm in Alan's camp: I can never remember which kind of "clean"
does what in the makefile.

The only "clean" I know is either to create a fresh checkout (Bzr
terminology, aka worktree in Git), or to use "git clean".


        Stefan


Reply | Threaded
Open this post in threaded view
|

Re: Git help please - checking out old commit doesn't find lib/gnulib.mk.in.

Paul Eggert
Stefan Monnier wrote:
> FWIW, I'm in Alan's camp: I can never remember which kind of "clean"
> does what in the makefile.

We could have a new rule ("make bootgit", say?) that does a "git clean -fdx"
followed by a "make bootstrap". This rule would be useful only for developers
who are using git.

Reply | Threaded
Open this post in threaded view
|

Re: Git help please - checking out old commit doesn't find lib/gnulib.mk.in.

Alan Mackenzie
Hello, Paul.

On Sun, Nov 12, 2017 at 13:23:37 -0800, Paul Eggert wrote:
> Stefan Monnier wrote:
> > FWIW, I'm in Alan's camp: I can never remember which kind of "clean"
> > does what in the makefile.

> We could have a new rule ("make bootgit", say?) that does a "git clean -fdx"
> followed by a "make bootstrap". This rule would be useful only for developers
> who are using git.

And that would be utterly useless for me.  I have quite a few
non-registered files around my repositories, for example those used by
backup scripts, or diff files, or stdout/stderr from Emacs builds.

I do not want to get rid of these random files.  What I need to get rid
of are specifically those files which are preventing a build working.

From the documentation, you would expect make bootstrap to do this.  It
doesn't.  make clean and make distclean fail because around 12 files in
m4/ are missing.  Why on earth are these make targets, which are solely
for deleting files, invoking m4 macros?

Why can Emacs not have a reliable way of cleaning up configuration and
build products (and _only_ these) such that a clean build can then be
done?

--
Alan Mackenzie (Nuremberg, Germany).

Reply | Threaded
Open this post in threaded view
|

Re: Git help please - checking out old commit doesn't find lib/gnulib.mk.in.

Óscar Fuentes
Alan Mackenzie <[hidden email]> writes:

> Why can Emacs not have a reliable way of cleaning up configuration and
> build products (and _only_ these) such that a clean build can then be
> done?

How the Emacs build system could know the products generated of the
future? Because this is the problem you faced.


Reply | Threaded
Open this post in threaded view
|

Re: Git help please - checking out old commit doesn't find lib/gnulib.mk.in.

Yasushi SHOJI-2
In reply to this post by Alan Mackenzie
Hi,

On Mon, Nov 13, 2017 at 6:55 AM, Alan Mackenzie <[hidden email]> wrote:

> On Sun, Nov 12, 2017 at 13:23:37 -0800, Paul Eggert wrote:
>> Stefan Monnier wrote:
>> > FWIW, I'm in Alan's camp: I can never remember which kind of "clean"
>> > does what in the makefile.
>
>> We could have a new rule ("make bootgit", say?) that does a "git clean -fdx"
>> followed by a "make bootstrap". This rule would be useful only for developers
>> who are using git.
>
> And that would be utterly useless for me.  I have quite a few
> non-registered files around my repositories, for example those used by
> backup scripts, or diff files, or stdout/stderr from Emacs builds.

I agree.  It's too risky to blindly call 'git clean -fdx'.

Why don't we just use a build dir separate from the source dir?
I'm sure autotools supports it.
--
           yashi

Reply | Threaded
Open this post in threaded view
|

Re: Git help please - checking out old commit doesn't find lib/gnulib.mk.in.

Stefan Monnier
In reply to this post by Óscar Fuentes
>> Why can Emacs not have a reliable way of cleaning up configuration and
>> build products (and _only_ these) such that a clean build can then be
>> done?
> How the Emacs build system could know the products generated of the
> future? Because this is the problem you faced.

We don't need to solve this impossible problem.
Instead we just need to make a rule in GNUmakefile (which is not
autoconf-generated contrarily to Makefile) to remove all the
auto-generated files of the present.


        Stefan


Reply | Threaded
Open this post in threaded view
|

Re: Git help please - checking out old commit doesn't find lib/gnulib.mk.in.

Óscar Fuentes
Stefan Monnier <[hidden email]> writes:

>>> Why can Emacs not have a reliable way of cleaning up configuration and
>>> build products (and _only_ these) such that a clean build can then be
>>> done?
>> How the Emacs build system could know the products generated of the
>> future? Because this is the problem you faced.
>
> We don't need to solve this impossible problem.
> Instead we just need to make a rule in GNUmakefile (which is not
> autoconf-generated contrarily to Makefile) to remove all the
> auto-generated files of the present.

This works as long as we remember to remove the build products *before*
we switch to another revision and then perform a full rebuild. A bit
draconian.


Reply | Threaded
Open this post in threaded view
|

Re: Git help please - checking out old commit doesn't find lib/gnulib.mk.in.

Stefan Monnier
> This works as long as we remember to remove the build products *before*
> we switch to another revision and then perform a full rebuild. A bit
> draconian.

No: it can work as long as the "other revision" has followed the
principle correctly (i.e. it removes the files that *it* needs to be
removed).  But indeed, my description was too optimistic (it probably
needs to remove not just the files that it would auto-generate, but
also additional files that it wouldn't auto-generate yet (if
present) would be used anyway).


        Stefan


Reply | Threaded
Open this post in threaded view
|

Re: Git help please - checking out old commit doesn't find lib/gnulib.mk.in.

Eli Zaretskii
In reply to this post by Alan Mackenzie
> Date: Sun, 12 Nov 2017 21:55:47 +0000
> From: Alan Mackenzie <[hidden email]>
> Cc: Stefan Monnier <[hidden email]>, [hidden email]
>
> make clean and make distclean fail because around 12 files in m4/
> are missing.

These targets should be invoked _before_ checking out another
revision, not after.  Otherwise, you get a bunch of Makefile's that
are inconsistent with the working tree contents.

Reply | Threaded
Open this post in threaded view
|

Re: Git help please - checking out old commit doesn't find lib/gnulib.mk.in.

Andreas Schwab
In reply to this post by Yasushi SHOJI-2
On Nov 13 2017, Yasushi SHOJI <[hidden email]> wrote:

> Why don't we just use a build dir separate from the source dir?

You can use that already now.

Andreas.

--
Andreas Schwab, SUSE Labs, [hidden email]
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Reply | Threaded
Open this post in threaded view
|

Re: Git help please - checking out old commit doesn't find lib/gnulib.mk.in.

Alan Mackenzie
In reply to this post by Eli Zaretskii
Hello, Eli.

On Mon, Nov 13, 2017 at 05:30:32 +0200, Eli Zaretskii wrote:
> > Date: Sun, 12 Nov 2017 21:55:47 +0000
> > From: Alan Mackenzie <[hidden email]>
> > Cc: Stefan Monnier <[hidden email]>, [hidden email]

> > make clean and make distclean fail because around 12 files in m4/
> > are missing.

> These targets should be invoked _before_ checking out another
> revision, not after.  Otherwise, you get a bunch of Makefile's that
> are inconsistent with the working tree contents.

Yes, that works better, thanks.  But not perfectly.

If I start off having just built the emacs-26 head, then do
(i) make distclean.
(ii) git checkout 10c0e1ca.
(iii) ./autogen.sh
, I get the following error:

Running 'autoreconf -fi -I m4' ...
/usr/bin/m4:aclocal.m4:9: cannot open `m4/count-leading-zeros.m4': No such file or directory
(+ ~11 similar lines)

.  The error here is that (i) make distclean has failed to delete aclocal.m4.

If I do
(i) delete aclocal.m4 by hand.
(ii) ./autogen.sh
, this works.  I then follow up with
(iii) ./configure --with-tiff=no --with-gif=no --with-gpm
(iv) make -j17
, this fails with:

Loading /home/acm/emacs/emacs.git/sub-26-0/lisp/emacs-lisp/cl-preloaded.el (source)...
Eager macro-expansion failure: (void-function define-symbol-prop)
Eager macro-expansion failure: (void-function define-symbol-prop)
Symbol's function definition is void: define-symbol-prop

.  Grepping for define-symbol-prop shows that define-symbol-prop is used
in cl-loaddefs.el.  It would appear that make distclean has failed to
delete the *loaddefs.el files, too.

I now delete *loaddefs.el from the source tree, and try again
(i) make -j17
, I get lots of errors like:

Eager macro-expansion failure: (void-function define-symbol-prop)

.  I've still got _lots_ of .elc files which refer to
define-symbol-prop (but no source files).  My theory: make distclean
failed to delete *.elc, but I can check that later.

Finally, I try
(i) make -j17 bootstrap
.  This _must_ work, surely?  Yes, it does.

make distclean is broken.  It purports to remove all files generated by
"configuring or building the program".  It doesn't do this.  I'll be
submitting a bug report about this, probably this evening.

--
Alan Mackenzie (Nuremberg, Germany).

Reply | Threaded
Open this post in threaded view
|

Re: Git help please - checking out old commit doesn't find lib/gnulib.mk.in.

Paul Eggert
On 11/13/2017 10:53 AM, Alan Mackenzie wrote:
> make distclean is broken.  It purports to remove all files generated by
> "configuring or building the program".

The intent is that 'make distclean' gets you back to a distribution
tarball, if all you've done is 'configure; make'. But that doesn't seem
to be what you're asking for. If I understand you correctly, you want
something that gets you back to a fresh git checkout if all you've done
is './autogen.sh; ./configure; make', and since 'make distclean' doesn't
do what you're asking for, you're saying it's broken. If I'm right, it
would be more accurate to say that 'make distclean' is simply a tool
that doesn't do what you want.

Perhaps someone (you?) could implement a 'make gitclean' that does what
you want.


12