bug#41853: 28.0.50; Peculiar "args out of range" error when using Edebug

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

bug#41853: 28.0.50; Peculiar "args out of range" error when using Edebug

Philipp Stephani

This error is somewhat common when using `edebug-all-defuns'.  It's not
easy to reproduce with a minimal example, but happens in real-world
code.  For example, the following recipe works for me consistently:

1. Clone the Flycheck repository at commit
   c02cd773dded0215f9417ec04dfe8dabda63ef43 (probably most other commits
   also work, this is just for reproducibility).

2. Clone the dash.el repository at commit
   ea4a4cc7cce7c3b93862a22df8bca8b83052ccbf (probably the exact commit
   doesn't matter here as well).

3. Visit flycheck.el like so:

   emacs -Q -L /src/dash.el/ -l edebug -f edebug-all-defuns \
     -f toggle-debug-on-error flycheck.el

4. M-x eval-buffer

5. Step through macro expansions using the `G' key.  Repeat until
   `eval-buffer' is complete or has signalled and error.

6. At some point, there will be an error

   edebug--display: Args out of range: [66 86 129 138 139], 5

   without invoking the debugger or backtrace.

This looks like a bug in Edebug.


In GNU Emacs 28.0.50 (build 25, x86_64-pc-linux-gnu, GTK+ Version 3.24.14, cairo version 1.16.0)
 of 2020-06-14
Repository revision: b3e7d046c3a94556fcaf6f9ce72aa2ecb20262a6
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12007000
System Description: Debian GNU/Linux rodete

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure --enable-gcc-warnings=warn-only
 --enable-gtk-deprecation-warnings --without-pop --with-mailutils
 --enable-checking --enable-check-lisp-object-type --with-modules
 'CFLAGS=-O0 -ggdb3''

Configured features:
XPM JPEG TIFF GIF PNG CAIRO SOUND DBUS GSETTINGS GLIB NOTIFY INOTIFY
LIBSELINUX GNUTLS FREETYPE HARFBUZZ ZLIB TOOLKIT_SCROLL_BARS GTK3 X11
XDBE XIM MODULES THREADS PDUMPER GMP

Important settings:
  value of $LANG: en_US.utf8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec epa epg epg-config gnus-util
rmail rmail-loaddefs text-property-search time-date mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils phst skeleton
derived edmacro kmacro pcase ffap thingatpt url url-proxy url-privacy
url-expand url-methods url-history url-cookie url-domsuf url-util
url-parse auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs
password-cache json map url-vars mailcap subr-x rx gnutls puny seq
byte-opt gv bytecomp byte-compile cconv dbus xml compile comint
ansi-color ring cl-loaddefs cl-lib tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify dynamic-setting system-font-setting
font-render-setting cairo move-toolbar gtk x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 68581 6800)
 (symbols 48 8624 1)
 (strings 32 23721 1916)
 (string-bytes 1 764482)
 (vectors 16 13090)
 (vector-slots 8 166268 6503)
 (floats 8 25 36)
 (intervals 56 213 0)
 (buffers 992 11))

--
Google Germany GmbH
Erika-Mann-Straße 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg

Diese E-Mail ist vertraulich.  Falls Sie diese fälschlicherweise erhalten haben
sollten, leiten Sie diese bitte nicht an jemand anderes weiter, löschen Sie
alle Kopien und Anhänge davon und lassen Sie mich bitte wissen, dass die E-Mail
an die falsche Person gesendet wurde.

This e-mail is confidential.  If you received this communication by mistake,
please don’t forward it to anyone else, please erase all copies and
attachments, and please let me know that it has gone to the wrong person.



Reply | Threaded
Open this post in threaded view
|

bug#41853: 28.0.50; Peculiar "args out of range" error when using Edebug

Philipp Stephani
Am So., 14. Juni 2020 um 16:23 Uhr schrieb Philipp Stephani
<[hidden email]>:

>
>
> This error is somewhat common when using `edebug-all-defuns'.  It's not
> easy to reproduce with a minimal example, but happens in real-world
> code.  For example, the following recipe works for me consistently:
>
> 1. Clone the Flycheck repository at commit
>    c02cd773dded0215f9417ec04dfe8dabda63ef43 (probably most other commits
>    also work, this is just for reproducibility).
>
> 2. Clone the dash.el repository at commit
>    ea4a4cc7cce7c3b93862a22df8bca8b83052ccbf (probably the exact commit
>    doesn't matter here as well).
>
> 3. Visit flycheck.el like so:
>
>    emacs -Q -L /src/dash.el/ -l edebug -f edebug-all-defuns \
>      -f toggle-debug-on-error flycheck.el
>
> 4. M-x eval-buffer
>
> 5. Step through macro expansions using the `G' key.  Repeat until
>    `eval-buffer' is complete or has signalled and error.
>
> 6. At some point, there will be an error
>
>    edebug--display: Args out of range: [66 86 129 138 139], 5
>
>    without invoking the debugger or backtrace.
>
> This looks like a bug in Edebug.
>


The immediate trigger appears to be that in `edebug-slow-after' for
`flycheck--checker-property-name' the AFTER-INDEX is out of range for
the EDEBUG-FREQ-COUNT vector.  I still don't understand why though;
there must be some part in edebug that misinstruments these forms.
The vector [66 86 129 138 139] is likely a vector of offsets, not
frequencies; the vector matches the setter for flycheck-checker-get
quite well. So maybe there's an issue with how edebug instruments
gv-define-setter?



Reply | Threaded
Open this post in threaded view
|

bug#41853: 28.0.50; Peculiar "args out of range" error when using Edebug

Philipp Stephani
Am So., 14. Juni 2020 um 17:20 Uhr schrieb Philipp Stephani
<[hidden email]>:

>
> Am So., 14. Juni 2020 um 16:23 Uhr schrieb Philipp Stephani
> <[hidden email]>:
> >
> >
> > This error is somewhat common when using `edebug-all-defuns'.  It's not
> > easy to reproduce with a minimal example, but happens in real-world
> > code.  For example, the following recipe works for me consistently:
> >
> > 1. Clone the Flycheck repository at commit
> >    c02cd773dded0215f9417ec04dfe8dabda63ef43 (probably most other commits
> >    also work, this is just for reproducibility).
> >
> > 2. Clone the dash.el repository at commit
> >    ea4a4cc7cce7c3b93862a22df8bca8b83052ccbf (probably the exact commit
> >    doesn't matter here as well).
> >
> > 3. Visit flycheck.el like so:
> >
> >    emacs -Q -L /src/dash.el/ -l edebug -f edebug-all-defuns \
> >      -f toggle-debug-on-error flycheck.el
> >
> > 4. M-x eval-buffer
> >
> > 5. Step through macro expansions using the `G' key.  Repeat until
> >    `eval-buffer' is complete or has signalled and error.
> >
> > 6. At some point, there will be an error
> >
> >    edebug--display: Args out of range: [66 86 129 138 139], 5
> >
> >    without invoking the debugger or backtrace.
> >
> > This looks like a bug in Edebug.
> >
>
>
> The immediate trigger appears to be that in `edebug-slow-after' for
> `flycheck--checker-property-name' the AFTER-INDEX is out of range for
> the EDEBUG-FREQ-COUNT vector.  I still don't understand why though;
> there must be some part in edebug that misinstruments these forms.
> The vector [66 86 129 138 139] is likely a vector of offsets, not
> frequencies; the vector matches the setter for flycheck-checker-get
> quite well. So maybe there's an issue with how edebug instruments
> gv-define-setter?

Yup, looks like this is the root of the issue. Minimal example:

$ cat /tmp/a.el
(defun foo (b) b)
(defun my-get (a b) (get a (foo b)))
(gv-define-setter my-get (x a b) `(setf (get ,a (foo ,b)) ,x))
(push 'foo (my-get 'foo 'bar))

$ emacs -Q -l edebug -f edebug-all-defuns /tmp/a.el

Then run M-x eval-buffer and hit G three times. Error:
edebug--display: Args out of range: [33 47 55 60 61], 5

Could the problem here be that to find the instrumentation metadata of
SYMBOL edebug uses (get SYMBOL 'edebug), and that it doesn't detect
that the setter for `my-get' is a new entity?



Reply | Threaded
Open this post in threaded view
|

bug#41853: 28.0.50; Peculiar "args out of range" error when using Edebug

Philipp Stephani
Am So., 14. Juni 2020 um 17:35 Uhr schrieb Philipp Stephani
<[hidden email]>:

>
> Am So., 14. Juni 2020 um 17:20 Uhr schrieb Philipp Stephani
> <[hidden email]>:
> >
> > Am So., 14. Juni 2020 um 16:23 Uhr schrieb Philipp Stephani
> > <[hidden email]>:
> > >
> > >
> > > This error is somewhat common when using `edebug-all-defuns'.  It's not
> > > easy to reproduce with a minimal example, but happens in real-world
> > > code.  For example, the following recipe works for me consistently:
> > >
> > > 1. Clone the Flycheck repository at commit
> > >    c02cd773dded0215f9417ec04dfe8dabda63ef43 (probably most other commits
> > >    also work, this is just for reproducibility).
> > >
> > > 2. Clone the dash.el repository at commit
> > >    ea4a4cc7cce7c3b93862a22df8bca8b83052ccbf (probably the exact commit
> > >    doesn't matter here as well).
> > >
> > > 3. Visit flycheck.el like so:
> > >
> > >    emacs -Q -L /src/dash.el/ -l edebug -f edebug-all-defuns \
> > >      -f toggle-debug-on-error flycheck.el
> > >
> > > 4. M-x eval-buffer
> > >
> > > 5. Step through macro expansions using the `G' key.  Repeat until
> > >    `eval-buffer' is complete or has signalled and error.
> > >
> > > 6. At some point, there will be an error
> > >
> > >    edebug--display: Args out of range: [66 86 129 138 139], 5
> > >
> > >    without invoking the debugger or backtrace.
> > >
> > > This looks like a bug in Edebug.
> > >
> >
> >
> > The immediate trigger appears to be that in `edebug-slow-after' for
> > `flycheck--checker-property-name' the AFTER-INDEX is out of range for
> > the EDEBUG-FREQ-COUNT vector.  I still don't understand why though;
> > there must be some part in edebug that misinstruments these forms.
> > The vector [66 86 129 138 139] is likely a vector of offsets, not
> > frequencies; the vector matches the setter for flycheck-checker-get
> > quite well. So maybe there's an issue with how edebug instruments
> > gv-define-setter?
>
> Yup, looks like this is the root of the issue. Minimal example:
>
> $ cat /tmp/a.el
> (defun foo (b) b)
> (defun my-get (a b) (get a (foo b)))
> (gv-define-setter my-get (x a b) `(setf (get ,a (foo ,b)) ,x))
> (push 'foo (my-get 'foo 'bar))
>
> $ emacs -Q -l edebug -f edebug-all-defuns /tmp/a.el
>
> Then run M-x eval-buffer and hit G three times. Error:
> edebug--display: Args out of range: [33 47 55 60 61], 5
>
> Could the problem here be that to find the instrumentation metadata of
> SYMBOL edebug uses (get SYMBOL 'edebug), and that it doesn't detect
> that the setter for `my-get' is a new entity?

So it looks more and more like the implementation of
`edebug-read-and-maybe-wrap-form1' generally doesn't support this use
case well at all. If the Edebug spec is `(&define name...)', then it
appears to blindly assume that the form being instrumented is the only
form defining NAME. That's clearly incorrect as NAME can mean
different kinds of entities depending on what the defining macro
happens to define.
I guess for `gv-define-setter' we can work around this by adding
`:name gv-setter' to the edebug specification, but the issue should be
fixed (or at least detected) in Edebug more generally. Also debugging
this is extremely hard because there's no backtrace etc.



Reply | Threaded
Open this post in threaded view
|

bug#41853: 28.0.50; Peculiar "args out of range" error when using Edebug

Philipp Stephani
Am So., 14. Juni 2020 um 17:48 Uhr schrieb Philipp Stephani
<[hidden email]>:

>
> Am So., 14. Juni 2020 um 17:35 Uhr schrieb Philipp Stephani
> <[hidden email]>:
> >
> > Am So., 14. Juni 2020 um 17:20 Uhr schrieb Philipp Stephani
> > <[hidden email]>:
> > >
> > > Am So., 14. Juni 2020 um 16:23 Uhr schrieb Philipp Stephani
> > > <[hidden email]>:
> > > >
> > > >
> > > > This error is somewhat common when using `edebug-all-defuns'.  It's not
> > > > easy to reproduce with a minimal example, but happens in real-world
> > > > code.  For example, the following recipe works for me consistently:
> > > >
> > > > 1. Clone the Flycheck repository at commit
> > > >    c02cd773dded0215f9417ec04dfe8dabda63ef43 (probably most other commits
> > > >    also work, this is just for reproducibility).
> > > >
> > > > 2. Clone the dash.el repository at commit
> > > >    ea4a4cc7cce7c3b93862a22df8bca8b83052ccbf (probably the exact commit
> > > >    doesn't matter here as well).
> > > >
> > > > 3. Visit flycheck.el like so:
> > > >
> > > >    emacs -Q -L /src/dash.el/ -l edebug -f edebug-all-defuns \
> > > >      -f toggle-debug-on-error flycheck.el
> > > >
> > > > 4. M-x eval-buffer
> > > >
> > > > 5. Step through macro expansions using the `G' key.  Repeat until
> > > >    `eval-buffer' is complete or has signalled and error.
> > > >
> > > > 6. At some point, there will be an error
> > > >
> > > >    edebug--display: Args out of range: [66 86 129 138 139], 5
> > > >
> > > >    without invoking the debugger or backtrace.
> > > >
> > > > This looks like a bug in Edebug.
> > > >
> > >
> > >
> > > The immediate trigger appears to be that in `edebug-slow-after' for
> > > `flycheck--checker-property-name' the AFTER-INDEX is out of range for
> > > the EDEBUG-FREQ-COUNT vector.  I still don't understand why though;
> > > there must be some part in edebug that misinstruments these forms.
> > > The vector [66 86 129 138 139] is likely a vector of offsets, not
> > > frequencies; the vector matches the setter for flycheck-checker-get
> > > quite well. So maybe there's an issue with how edebug instruments
> > > gv-define-setter?
> >
> > Yup, looks like this is the root of the issue. Minimal example:
> >
> > $ cat /tmp/a.el
> > (defun foo (b) b)
> > (defun my-get (a b) (get a (foo b)))
> > (gv-define-setter my-get (x a b) `(setf (get ,a (foo ,b)) ,x))
> > (push 'foo (my-get 'foo 'bar))
> >
> > $ emacs -Q -l edebug -f edebug-all-defuns /tmp/a.el
> >
> > Then run M-x eval-buffer and hit G three times. Error:
> > edebug--display: Args out of range: [33 47 55 60 61], 5
> >
> > Could the problem here be that to find the instrumentation metadata of
> > SYMBOL edebug uses (get SYMBOL 'edebug), and that it doesn't detect
> > that the setter for `my-get' is a new entity?
>
> So it looks more and more like the implementation of
> `edebug-read-and-maybe-wrap-form1' generally doesn't support this use
> case well at all. If the Edebug spec is `(&define name...)', then it
> appears to blindly assume that the form being instrumented is the only
> form defining NAME. That's clearly incorrect as NAME can mean
> different kinds of entities depending on what the defining macro
> happens to define.
> I guess for `gv-define-setter' we can work around this by adding
> `:name gv-setter' to the edebug specification, but the issue should be
> fixed (or at least detected) in Edebug more generally. Also debugging
> this is extremely hard because there's no backtrace etc.

I've now fixed the gv-setter issue with commit 62cf8f1649, but I'll
leave this bug open to track fixes for this class of issues in
edebug.el, as well as debuggability improvements.



Reply | Threaded
Open this post in threaded view
|

bug#41853: 28.0.50; Peculiar "args out of range" error when using Edebug

Philipp Stephani
Am So., 21. Juni 2020 um 15:40 Uhr schrieb Philipp Stephani
<[hidden email]>:

>
> Am So., 14. Juni 2020 um 18:23 Uhr schrieb Philipp Stephani
> <[hidden email]>:
> >
> > Am So., 14. Juni 2020 um 17:48 Uhr schrieb Philipp Stephani
> > <[hidden email]>:
> > >
> > > Am So., 14. Juni 2020 um 17:35 Uhr schrieb Philipp Stephani
> > > <[hidden email]>:
> > > >
> > > > Am So., 14. Juni 2020 um 17:20 Uhr schrieb Philipp Stephani
> > > > <[hidden email]>:
> > > > >
> > > > > Am So., 14. Juni 2020 um 16:23 Uhr schrieb Philipp Stephani
> > > > > <[hidden email]>:
> > > > > >
> > > > > >
> > > > > > This error is somewhat common when using `edebug-all-defuns'.  It's not
> > > > > > easy to reproduce with a minimal example, but happens in real-world
> > > > > > code.  For example, the following recipe works for me consistently:
> > > > > >
> > > > > > 1. Clone the Flycheck repository at commit
> > > > > >    c02cd773dded0215f9417ec04dfe8dabda63ef43 (probably most other commits
> > > > > >    also work, this is just for reproducibility).
> > > > > >
> > > > > > 2. Clone the dash.el repository at commit
> > > > > >    ea4a4cc7cce7c3b93862a22df8bca8b83052ccbf (probably the exact commit
> > > > > >    doesn't matter here as well).
> > > > > >
> > > > > > 3. Visit flycheck.el like so:
> > > > > >
> > > > > >    emacs -Q -L /src/dash.el/ -l edebug -f edebug-all-defuns \
> > > > > >      -f toggle-debug-on-error flycheck.el
> > > > > >
> > > > > > 4. M-x eval-buffer
> > > > > >
> > > > > > 5. Step through macro expansions using the `G' key.  Repeat until
> > > > > >    `eval-buffer' is complete or has signalled and error.
> > > > > >
> > > > > > 6. At some point, there will be an error
> > > > > >
> > > > > >    edebug--display: Args out of range: [66 86 129 138 139], 5
> > > > > >
> > > > > >    without invoking the debugger or backtrace.
> > > > > >
> > > > > > This looks like a bug in Edebug.
> > > > > >
> > > > >
> > > > >
> > > > > The immediate trigger appears to be that in `edebug-slow-after' for
> > > > > `flycheck--checker-property-name' the AFTER-INDEX is out of range for
> > > > > the EDEBUG-FREQ-COUNT vector.  I still don't understand why though;
> > > > > there must be some part in edebug that misinstruments these forms.
> > > > > The vector [66 86 129 138 139] is likely a vector of offsets, not
> > > > > frequencies; the vector matches the setter for flycheck-checker-get
> > > > > quite well. So maybe there's an issue with how edebug instruments
> > > > > gv-define-setter?
> > > >
> > > > Yup, looks like this is the root of the issue. Minimal example:
> > > >
> > > > $ cat /tmp/a.el
> > > > (defun foo (b) b)
> > > > (defun my-get (a b) (get a (foo b)))
> > > > (gv-define-setter my-get (x a b) `(setf (get ,a (foo ,b)) ,x))
> > > > (push 'foo (my-get 'foo 'bar))
> > > >
> > > > $ emacs -Q -l edebug -f edebug-all-defuns /tmp/a.el
> > > >
> > > > Then run M-x eval-buffer and hit G three times. Error:
> > > > edebug--display: Args out of range: [33 47 55 60 61], 5
> > > >
> > > > Could the problem here be that to find the instrumentation metadata of
> > > > SYMBOL edebug uses (get SYMBOL 'edebug), and that it doesn't detect
> > > > that the setter for `my-get' is a new entity?
> > >
> > > So it looks more and more like the implementation of
> > > `edebug-read-and-maybe-wrap-form1' generally doesn't support this use
> > > case well at all. If the Edebug spec is `(&define name...)', then it
> > > appears to blindly assume that the form being instrumented is the only
> > > form defining NAME. That's clearly incorrect as NAME can mean
> > > different kinds of entities depending on what the defining macro
> > > happens to define.
> > > I guess for `gv-define-setter' we can work around this by adding
> > > `:name gv-setter' to the edebug specification, but the issue should be
> > > fixed (or at least detected) in Edebug more generally. Also debugging
> > > this is extremely hard because there's no backtrace etc.
> >
> > I've now fixed the gv-setter issue with commit 62cf8f1649, but I'll
> > leave this bug open to track fixes for this class of issues in
> > edebug.el, as well as debuggability improvements.
>
> There are a couple similar bugs in Edebug. For example, there's code
> to ensure that nested named functions (cl-letf etc.) don't clash with
> each other, but that doesn't actually work. For example, edebugging
> the definition
>   (defun foo () (cl-flet ((bar ())) 1))
> results in the messages
>   Edebug: bar
>   Edebug: foo
> which is wrong because "bar" is local, so the symbol should be "foo@bar".

However, even that isn't enough because `foo' could contain multiple
non-overlapping definitions of `bar'. So probably this should use a
similar strategy as cl-macrolet, i.e. generate a unique name.