emacs26, ucs-normalize.el, Symbol’s function definition is void: regexp-opt-charset

[hx]

Hi list,

can't compile emacs26 today after git pull,

CentOS7 X64:

make[2]: Entering directory `/emacs-26-src/lisp'

  ELC      ../lisp/international/ucs-normalize.elc

In toplevel form:

international/ucs-normalize.el:266:56:Error: Symbol’s function definition is void: regexp-opt-charset

make[2]: *** [../lisp/international/ucs-normalize.elc] Error 1

$ git status

# On branch emacs-26

nothing to commit, working directory clean

[Eli Zaretskii]

> From: hx <[hidden email]>
> Date: Mon, 11 Feb 2019 09:21:20 +0800
>
> can't compile emacs26 today after git pull,
> CentOS7 X64:
>
> make[2]: Entering directory `/emacs-26-src/lisp'
>   ELC      ../lisp/international/ucs-normalize.elc
>
> In toplevel form:
> international/ucs-normalize.el:266:56:Error: Symbol’s function definition is void: regexp-opt-charset
> make[2]: *** [../lisp/international/ucs-normalize.elc] Error 1

I think I understand why, but just so I'm sure: can you tell how you
configured this build?  What was the command line with which you
invoked the 'configure' script?

[Paul Eggert]

On 2/10/19 5:21 PM, hx wrote:
>
> can't compile emacs26 today after git pull,
> CentOS7 X64:

I am not seeing any problem on Fedora 29 x86-64 with a fresh checkout
after I run:

./autogen.sh && ./configure && make

I'm using the emacs-26 commit 3aaa2d2058125a755bd6226cb4009811089812f8.

[hx]

CFLAGS="-march=native -O2 -pipe -fomit-frame-pointer" \

./configure --prefix=/home/xyz/emacs26 --disable-largefile --without-pop --without-sound \

--with-x-toolkit=no --without-xpm --without-jpeg --without-tiff --without-gif --without-png --without-rsvg --without-xft \

--without-libotf --without-m17n-flt --without-toolkit-scroll-bars --without-xaw3d --without-xim --without-gpm \

--without-dbus --without-gconf -without-x \

--without-selinux --disable-acl --with-modules --without-makeinfo \

--without-gnutls

yesterday, I manually open emacs and make byte-compile on the ucs-normalize.el, then run make again, it worked.

today, I just do make clean && make distclean, delete all .elc files, sh autogen.sh, configure with above options,

make success without error.

On Tue, Feb 12, 2019 at 12:39 AM Eli Zaretskii <[hidden email]> wrote:

> From: hx <[hidden email]>
> Date: Mon, 11 Feb 2019 09:21:20 +0800
>
> can't compile emacs26 today after git pull,
> CentOS7 X64:
>
> make[2]: Entering directory `/emacs-26-src/lisp'
>   ELC      ../lisp/international/ucs-normalize.elc
>
> In toplevel form:
> international/ucs-normalize.el:266:56:Error: Symbol’s function definition is void: regexp-opt-charset
> make[2]: *** [../lisp/international/ucs-normalize.elc] Error 1

I think I understand why, but just so I'm sure: can you tell how you
configured this build?  What was the command line with which you
invoked the 'configure' script?

[Eli Zaretskii]

> From: Paul Eggert <[hidden email]>
> Date: Mon, 11 Feb 2019 15:39:14 -0800
> Cc: emacs-devel <[hidden email]>
>
> On 2/10/19 5:21 PM, hx wrote:
> >
> > can't compile emacs26 today after git pull,
> > CentOS7 X64:
>
> I am not seeing any problem on Fedora 29 x86-64 with a fresh checkout
> after I run:
>
> ./autogen.sh && ./configure && make

Even if you configure --without-x?  regexp-opt is preloaded only if
x-create-frame is fboundp.  So I thought we'd need an autoload form in
ucs-normalize.el for when regexp-opt is not preloaded.

[Paul Eggert]

On 2/11/19 7:35 PM, Eli Zaretskii wrote:
>> I am not seeing any problem on Fedora 29 x86-64 with a fresh checkout
>> after I run:
>>
>> ./autogen.sh && ./configure && make
> Even if you configure --without-x?  regexp-opt is preloaded only if
> x-create-frame is fboundp.  So I thought we'd need an autoload form in
> ucs-normalize.el for when regexp-opt is not preloaded.

On my platform (Fedora 29 x86-64) I don't see a problem in the emacs-26
branch even if I configure --without-x. The relevant output lines are:

make[2]: Entering directory '/home/eggert/src/gnu/emacs/emacs-26-tmp1/lisp'
   ELC      ../lisp/international/ucs-normalize.elc
make[2]: Leaving directory '/home/eggert/src/gnu/emacs/emacs-26-tmp1/lisp'

This is with commit 2019-02-13T15:39:31Z![hidden email]
(6d89120b156576ca24e03472563f5ea0f01313f8).

I also could not reproduce the problem with commands very close to hx's
in a fresh checkout of the same commit. I used:

./autogen.sh && ./configure --prefix=/home/xyz/emacs26
--disable-largefile --without-pop --without-sound --with-x-toolkit=no
--without-xpm --without-jpeg --without-tiff --without-gif --without-png
--without-rsvg --without-xft --without-libotf --without-m17n-flt
--without-toolkit-scroll-bars --without-xaw3d --without-xim
--without-gpm --without-dbus --without-gconf -without-x
--without-selinux --disable-acl --with-modules --without-makeinfo
--without-gnutls CFLAGS='-march=native -O2 -pipe -fomit-frame-pointer'
&& make

[Eli Zaretskii]

> Cc: [hidden email], [hidden email]
> From: Paul Eggert <[hidden email]>
> Date: Wed, 13 Feb 2019 10:49:43 -0800
>
> On 2/11/19 7:35 PM, Eli Zaretskii wrote:
> >> I am not seeing any problem on Fedora 29 x86-64 with a fresh checkout
> >> after I run:
> >>
> >> ./autogen.sh && ./configure && make
> > Even if you configure --without-x?  regexp-opt is preloaded only if
> > x-create-frame is fboundp.  So I thought we'd need an autoload form in
> > ucs-normalize.el for when regexp-opt is not preloaded.
>
> On my platform (Fedora 29 x86-64) I don't see a problem in the emacs-26
> branch even if I configure --without-x.

OK, thanks for testing.

[Glenn Morris-3]

Although a --without-x bootstrap does work (eg this is what the
hydra.nixos "tarball" job is), it's not clear to me why it works.
(Maybe something calls regexp-opt during bootstrap, which triggers an
autoload?) There is a problem, however, as shown by the fact that in a
--without-x build post-bootstrap, this fails:

./src/emacs -Q -batch -f batch-byte-compile lisp/international/ucs-normalize.el

Error: Symbol's function definition is void: regexp-opt-charset

[Stefan Monnier]

> Although a --without-x bootstrap does work (eg this is what the
> hydra.nixos "tarball" job is), it's not clear to me why it works.
> (Maybe something calls regexp-opt during bootstrap, which triggers an
> autoload?) There is a problem, however, as shown by the fact that in a
> --without-x build post-bootstrap, this fails:

Most likely regexp-opt gets loaded during the first run of "loadup", so
the compilation of ucs-normalize by the initial bootstrap-emacs
succeeds, even tough the compilation of that same file by the
`emacs` fails.

So a later "make" can succeed or fail depending on which files needs to
be recompiled.


        Stefan

[Paul Eggert]

On 2/13/19 12:04 PM, Glenn Morris wrote:
> There is a problem, however, as shown by the fact that in a
> --without-x build post-bootstrap, this fails:
>
> ./src/emacs -Q -batch -f batch-byte-compile lisp/international/ucs-normalize.el
>
> Error: Symbol's function definition is void: regexp-opt-charset

Yes, I observe that problem too. I fixed it by installing the attached
patch into the master branch. It's not clear to me whether it's worth
backporting this into emacs-26, as the problem occurs only with unusual
actions after unusual builds.

[Eli Zaretskii]

> From: Stefan Monnier <[hidden email]>
> Date: Wed, 13 Feb 2019 15:15:25 -0500
>
> Most likely regexp-opt gets loaded during the first run of "loadup", so
> the compilation of ucs-normalize by the initial bootstrap-emacs
> succeeds, even tough the compilation of that same file by the
> `emacs` fails.

Why is the first run of loadup different from the rest?  The condition
to load regexp-opt depends only on what functions were compiled into
the binary, not on anything else.

[Eli Zaretskii]

> Cc: Eli Zaretskii <[hidden email]>, [hidden email], [hidden email]
> From: Paul Eggert <[hidden email]>
> Date: Wed, 13 Feb 2019 15:04:18 -0800
>
> diff --git a/lisp/international/ucs-normalize.el b/lisp/international/ucs-normalize.el
> index 9d55470d94..6f1e770c09 100644
> --- a/lisp/international/ucs-normalize.el
> +++ b/lisp/international/ucs-normalize.el
> @@ -109,7 +109,9 @@
>  
>  (defconst ucs-normalize-version "1.2")
>  
> -(eval-when-compile (require 'cl-lib))
> +(eval-when-compile
> +  (require 'cl-lib)
> +  (require 'regexp-opt))

I think it would be better to add the 'require' in the same
eval-when-compile form which calls regexp-opt-charset.

[Stefan Monnier]

>> Most likely regexp-opt gets loaded during the first run of "loadup", so
>> the compilation of ucs-normalize by the initial bootstrap-emacs
>> succeeds, even tough the compilation of that same file by the
>> `emacs` fails.
> Why is the first run of loadup different from the rest?

Because the first run of loadup is done with not-yet-compiled Elisp
files, so it loads a crapload more packages (because of things like
(eval-when-compile (require <foo>)) and autoloaded macros) than the
second run (performed after the preloaded files have all been
byte-compiled).


        Stefan

[Paul Eggert]

On 2/13/19 7:37 PM, Eli Zaretskii wrote:
>> -(eval-when-compile (require 'cl-lib))
>> +(eval-when-compile
>> +  (require 'cl-lib)
>> +  (require 'regexp-opt))
> I think it would be better to add the 'require' in the same
> eval-when-compile form which calls regexp-opt-charset.

There's more than one such form. At some point it's easier to treat
regexp-opt functions like cl-lib functions, i.e., available to any code
within an eval-when-compile in that module.