buffer.c/buffer.h: How to add new buffer-local variables?

classic Classic list List threaded Threaded
34 messages Options
12
Reply | Threaded
Open this post in threaded view
|

buffer.c/buffer.h: How to add new buffer-local variables?

Keith David Bershatsky
I am working on feature requests #22873 (multiple fake cursors) and #17684 (crosshairs that track the cursor position).

In an earlier version of the Emacs master branch from a few months ago, I was able to increase the number of buffer-local variables in buffer.c/buffer.h by changing the value of MAX_PER_BUFFER_VARS from 50 to 60.  That approach does not permit me to build a current version of the Emacs master branch, which fails with the following error:

    Dumping under the name bootstrap-emacs.pdmp
    dumping fingerprint: 50894efcc2cdb17747a4536c9f53c4d093895712c3604daa88d068c383ea4780

    pdumper.c:1786: Emacs fatal error: assertion failed: relpos < 1024
    Fatal error 6: Abort trapmake[1]: *** [bootstrap-emacs.pdmp] Abort trap
    make[1]: *** Deleting file `bootstrap-emacs.pdmp'
    make: *** [src] Error 2

    Compilation exited abnormally with code 2 at Sat Mar 30 15:53:18

I have narrowed the issue down to adding just one (1) more buffer local variable, which breaks the camel's back.  Attached are git diffs for a working build and a broken build.  I am using Emacs master branch from 03/28/2019:  2da9f8bf4222fda504f43b4757e154999cdbbf2c

My config options are as follows:

CFLAGS='-Wall -O0 -g3' ./configure \
--with-ns \
--enable-checking='yes,glyphs' \
--enable-check-lisp-object-type \
--without-compress-install \
--without-makeinfo \
--with-gnutls=no \
--with-mailutils \
--without-makeinfo

How can I increase the number of buffer local variables in buffer.c/buffer.h without breaking the ability to build Emacs?

Keith


patch_working.diff (14K) Download Attachment
patch_broken.diff (14K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: buffer.c/buffer.h: How to add new buffer-local variables?

Eli Zaretskii
> Date: Sat, 30 Mar 2019 16:19:50 -0700
> From: Keith David Bershatsky <[hidden email]>
>
> In an earlier version of the Emacs master branch from a few months ago, I was able to increase the number of buffer-local variables in buffer.c/buffer.h by changing the value of MAX_PER_BUFFER_VARS from 50 to 60.  That approach does not permit me to build a current version of the Emacs master branch, which fails with the following error:
>
>     Dumping under the name bootstrap-emacs.pdmp
>     dumping fingerprint: 50894efcc2cdb17747a4536c9f53c4d093895712c3604daa88d068c383ea4780
>
>     pdumper.c:1786: Emacs fatal error: assertion failed: relpos < 1024
>     Fatal error 6: Abort trapmake[1]: *** [bootstrap-emacs.pdmp] Abort trap
>     make[1]: *** Deleting file `bootstrap-emacs.pdmp'
>     make: *** [src] Error 2
>
>     Compilation exited abnormally with code 2 at Sat Mar 30 15:53:18

You need to update the dumping code to the new variables you added.
See the comments in pdumper.c.

Reply | Threaded
Open this post in threaded view
|

Re: buffer.c/buffer.h: How to add new buffer-local variables?

Eli Zaretskii
On March 31, 2019 5:37:14 AM GMT+03:00, Eli Zaretskii <[hidden email]> wrote:

> > Date: Sat, 30 Mar 2019 16:19:50 -0700
> > From: Keith David Bershatsky <[hidden email]>
> >
> > In an earlier version of the Emacs master branch from a few months
> ago, I was able to increase the number of buffer-local variables in
> buffer.c/buffer.h by changing the value of MAX_PER_BUFFER_VARS from 50
> to 60.  That approach does not permit me to build a current version of
> the Emacs master branch, which fails with the following error:
> >
> >     Dumping under the name bootstrap-emacs.pdmp
> >     dumping fingerprint:
> 50894efcc2cdb17747a4536c9f53c4d093895712c3604daa88d068c383ea4780
> >
> >     pdumper.c:1786: Emacs fatal error: assertion failed: relpos <
> 1024
> >     Fatal error 6: Abort trapmake[1]: *** [bootstrap-emacs.pdmp]
> Abort trap
> >     make[1]: *** Deleting file `bootstrap-emacs.pdmp'
> >     make: *** [src] Error 2
> >
> >     Compilation exited abnormally with code 2 at Sat Mar 30 15:53:18
>
> You need to update the dumping code to the new variables you added.
> See the comments in pdumper.c.

Could it be you are overflowing MAX_PER_BUFFER_VARS?

Reply | Threaded
Open this post in threaded view
|

Re: buffer.c/buffer.h: How to add new buffer-local variables?

Keith David Bershatsky
In reply to this post by Keith David Bershatsky
Thank you, Eli, for reading and responding to this particular thread.

I have tried increasing MAX_PER_BUFFER_VARS from 50 to 60 and even 75, but that no longer works.  That approach worked back in early/mid-December 2018, but does not work with a current version of Emacs master branch.

I searched pdumper.c for a few popular buffer-local variables such as mode_line_format and cache_long_scans, but did not find them.  As such, I have no idea where else to add the new buffer local variables other than the regular locations in buffer.c and buffer.h as demonstrated in the working/broken patches attached to the initial post of this particular thread.

The build messages gives little clues for us to investigate this issue.  I have learned how to extract the new hash values from dmpstruct.h and insert them at the appropriate location of pdumper.c, but that is not what causes the build to fail in this particular issue.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

> Date: [03-30-2019 20:49:57] <31 Mar 2019 06:49:57 +0300>
> From: Eli Zaretskii <[hidden email]>
> To: [hidden email],Keith David Bershatsky <[hidden email]>
> Subject: Re: buffer.c/buffer.h:  How to add new buffer-local variables?
>
> * * *
>
> Could it be you are overflowing MAX_PER_BUFFER_VARS?

Reply | Threaded
Open this post in threaded view
|

Re: buffer.c/buffer.h: How to add new buffer-local variables?

Andreas Schwab-2
In reply to this post by Keith David Bershatsky
On Mär 30 2019, Keith David Bershatsky <[hidden email]> wrote:

>     pdumper.c:1786: Emacs fatal error: assertion failed: relpos < 1024

The size of struct buffer is already close to that limit now.

Andreas.

--
Andreas Schwab, [hidden email]
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Reply | Threaded
Open this post in threaded view
|

Re: buffer.c/buffer.h: How to add new buffer-local variables?

Eli Zaretskii
On March 31, 2019 12:42:27 PM GMT+03:00, Andreas Schwab <[hidden email]> wrote:
> On Mär 30 2019, Keith David Bershatsky <[hidden email]> wrote:
>
> >     pdumper.c:1786: Emacs fatal error: assertion failed: relpos <
> 1024
>
> The size of struct buffer is already close to that limit now.
>
> Andreas.

So you are saying that the 1024 value is arbitrary and should be enlarged when more local vars are added?

Reply | Threaded
Open this post in threaded view
|

Re: buffer.c/buffer.h: How to add new buffer-local variables?

Andreas Schwab-2
On Mär 31 2019, Eli Zaretskii <[hidden email]> wrote:

> So you are saying that the 1024 value is arbitrary and should be enlarged when more local vars are added?

No.

Andreas.

--
Andreas Schwab, [hidden email]
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Reply | Threaded
Open this post in threaded view
|

Re: buffer.c/buffer.h: How to add new buffer-local variables?

Alan Mackenzie
In reply to this post by Keith David Bershatsky
Hello, Keith.

On Sat, Mar 30, 2019 at 16:19:50 -0700, Keith David Bershatsky wrote:
> I am working on feature requests #22873 (multiple fake cursors) and
> #17684 (crosshairs that track the cursor position).

> In an earlier version of the Emacs master branch from a few months ago,
> I was able to increase the number of buffer-local variables in
> buffer.c/buffer.h by changing the value of MAX_PER_BUFFER_VARS from 50
> to 60.  That approach does not permit me to build a current version of
> the Emacs master branch, which fails with the following error:

>     Dumping under the name bootstrap-emacs.pdmp
>     dumping fingerprint: 50894efcc2cdb17747a4536c9f53c4d093895712c3604daa88d068c383ea4780

>     pdumper.c:1786: Emacs fatal error: assertion failed: relpos < 1024
>     Fatal error 6: Abort trapmake[1]: *** [bootstrap-emacs.pdmp] Abort trap
>     make[1]: *** Deleting file `bootstrap-emacs.pdmp'
>     make: *** [src] Error 2

>     Compilation exited abnormally with code 2 at Sat Mar 30 15:53:18

> I have narrowed the issue down to adding just one (1) more buffer local
> variable, which breaks the camel's back.  Attached are git diffs for a
> working build and a broken build.  I am using Emacs master branch from
> 03/28/2019:  2da9f8bf4222fda504f43b4757e154999cdbbf2c

> My config options are as follows:

[ .... ]

> How can I increase the number of buffer local variables in
> buffer.c/buffer.h without breaking the ability to build Emacs?

You're surely aware that you can add a new local variable in C source by
declaring it with DEFVAR, then calling Fmake_variable_buffer_local in
syms_of_foo?  Or something like that (I've forgotten the exact details).

Is there some pressing reason why the new variable has to be part of
struct buffer rather than an "ordinary" buffer local?

It's worth noting that there's nothing in the elisp manual about how to
create buffer local variables in the C source.

It may be that there're BL variables in struct buffer that don't really
need to be there, and could be converted to "ordinary" variables.  I
think I might have added one or two of these myself, but don't remember.

> Keith

--
Alan Mackenzie (Nuremberg, Germany).

Reply | Threaded
Open this post in threaded view
|

Re: buffer.c/buffer.h: How to add new buffer-local variables?

Eli Zaretskii
In reply to this post by Andreas Schwab-2
> From: Andreas Schwab <[hidden email]>
> Date: Sun, 31 Mar 2019 13:41:51 +0200
> Cc: Keith David Bershatsky <[hidden email]>, [hidden email]
>
> On Mär 31 2019, Eli Zaretskii <[hidden email]> wrote:
>
> > So you are saying that the 1024 value is arbitrary and should be enlarged when more local vars are added?
>
> No.

"No" to which part?

Reply | Threaded
Open this post in threaded view
|

Re: buffer.c/buffer.h: How to add new buffer-local variables?

Keith David Bershatsky
In reply to this post by Keith David Bershatsky
Thank you, Alan.  I was unaware that there are other means of declaring buffer-local variables in C.

In a few areas of the code for the features that I am working on, I have statements such as:

if (!NILP (BVAR (XBUFFER (w->contents), crosshairs))) ...

I have been using the existing method in buffer.c/h to define buffer local variables for the past three years (while chiseling away at the multiple fake cursors feature).  The last proof concept patch applied to an Emacs version from 12/13/2018, and I am just about ready to submit a new proof concept patch for a current version of Emacs.

If there is a way to increase the maximum number of buffer local variables to continue using the same method as is done in buffer.c/h, then I would prefer to do that.  If not, then the method you suggested below could be fallback plan.  Other than personal preference and my existing code that has worked well up until recently, there is no pressing reason that it has to be done a certain way.

Keith

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

> Date: [03-31-2019 05:22:38] <31 Mar 2019 12:22:38 +0000>
> From: Alan Mackenzie <[hidden email]>
>
> You're surely aware that you can add a new local variable in C source by
> declaring it with DEFVAR, then calling Fmake_variable_buffer_local in
> syms_of_foo?  Or something like that (I've forgotten the exact details).
>
> Is there some pressing reason why the new variable has to be part of
> struct buffer rather than an "ordinary" buffer local?
>
> It's worth noting that there's nothing in the elisp manual about how to
> create buffer local variables in the C source.
>
> It may be that there're BL variables in struct buffer that don't really
> need to be there, and could be converted to "ordinary" variables.  I
> think I might have added one or two of these myself, but don't remember.

Reply | Threaded
Open this post in threaded view
|

Re: buffer.c/buffer.h: How to add new buffer-local variables?

Stefan Monnier
> if (!NILP (BVAR (XBUFFER (w->contents), crosshairs))) ...

Buffer-local variables are appropriate in the `struct buffer` when it's
likely that most buffers will have their own setting for it.

For variables that are likely to be used only in some particular
buffers, or only under some particular circumstances, it can make a lot
more sense to declare them as global variables and then rely on the
Elisp side using `make-local-variable` or `make-variable-buffer-local`,
so you don't pay for that slot in every buffer even tho it's almost
never used.


        Stefan


Reply | Threaded
Open this post in threaded view
|

Re: buffer.c/buffer.h: How to add new buffer-local variables?

Keith David Bershatsky
In reply to this post by Keith David Bershatsky
Thank you, Stefan, for the insight regarding how to handle this particular issue.  Greatly appreciated!

Keith

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

> From: Stefan Monnier
> Subject: Re: buffer.c/buffer.h: How to add new buffer-local variables?
> Date: Sun, 31 Mar 2019 16:02:49 -0400
>
>> if (!NILP (BVAR (XBUFFER (w->contents), crosshairs))) ...
>
> Buffer-local variables are appropriate in the `struct buffer` when it's
> likely that most buffers will have their own setting for it.
>
> For variables that are likely to be used only in some particular
> buffers, or only under some particular circumstances, it can make a lot
> more sense to declare them as global variables and then rely on the
> Elisp side using `make-local-variable` or `make-variable-buffer-local`,
> so you don't pay for that slot in every buffer even tho it's almost
> never used.
>
>         Stefan

Reply | Threaded
Open this post in threaded view
|

Re: buffer.c/buffer.h: How to add new buffer-local variables?

Eli Zaretskii
In reply to this post by Andreas Schwab-2
> From: Andreas Schwab <[hidden email]>
> Date: Sun, 31 Mar 2019 13:41:51 +0200
> Cc: Keith David Bershatsky <[hidden email]>, [hidden email]
>
> On Mär 31 2019, Eli Zaretskii <[hidden email]> wrote:
>
> > So you are saying that the 1024 value is arbitrary and should be enlarged when more local vars are added?
>
> No.

Daniel, could you please help me understand why the value 1024 was
used here:

  static dump_off
  field_relpos (const void *in_start, const void *in_field)
  {
    ptrdiff_t in_start_val = (ptrdiff_t) in_start;
    ptrdiff_t in_field_val = (ptrdiff_t) in_field;
    eassert (in_start_val <= in_field_val);
    ptrdiff_t relpos = in_field_val - in_start_val;
    eassert (relpos < 1024); /* Sanity check.  */  <<<<<<<<<<<<
    return (dump_off) relpos;
  }

The comment says "sanity check", but I would like to understand what
kind of sanity is being checked here, and what should be done when
some structure we dump becomes larger than this value.  E.g., is there
some other limit that requires that offsets of dumped fields never
exceed 1024 here?  I'd like to document in comments what to do when
the assertion is violated.

TIA

Reply | Threaded
Open this post in threaded view
|

Re: buffer.c/buffer.h: How to add new buffer-local variables?

Daniel Colascione-5
>> From: Andreas Schwab <[hidden email]>
>> Date: Sun, 31 Mar 2019 13:41:51 +0200
>> Cc: Keith David Bershatsky <[hidden email]>, [hidden email]
>>
>> On Mär 31 2019, Eli Zaretskii <[hidden email]> wrote:
>>
>> > So you are saying that the 1024 value is arbitrary and should be
>> enlarged when more local vars are added?
>>
>> No.
>
> Daniel, could you please help me understand why the value 1024 was
> used here:
>
>   static dump_off
>   field_relpos (const void *in_start, const void *in_field)
>   {
>     ptrdiff_t in_start_val = (ptrdiff_t) in_start;
>     ptrdiff_t in_field_val = (ptrdiff_t) in_field;
>     eassert (in_start_val <= in_field_val);
>     ptrdiff_t relpos = in_field_val - in_start_val;
>     eassert (relpos < 1024); /* Sanity check.  */  <<<<<<<<<<<<
>     return (dump_off) relpos;
>   }
>
> The comment says "sanity check", but I would like to understand what
> kind of sanity is being checked here, and what should be done when
> some structure we dump becomes larger than this value.  E.g., is there
> some other limit that requires that offsets of dumped fields never
> exceed 1024 here?  I'd like to document in comments what to do when
> the assertion is violated.

Indeed. That comment could have been a lot better. The general idea here
is this:

When we enter field_relpos, we're in the middle of some code that's
dumping some data structure field-by-field. The object that we're dumping
begins at in_start; and in_field is an interior pointer into that object.
We can't actually check that the two pointers refer to the same object: C
doesn't give us that level of introspection. But if the two pointers point
to addresses that differ by a lot, then the two pointers probably don't
refer to the same object, and in this case, we can fail an assertion. 1024
is probably too conservative here. We probably want to greatly increase
this number (say, to 32k) and also to give it a named constantly, maybe
something like PDUMPER_MAXIMUM_STRUCT_SIZE.

Note that this limit doesn't apply to big variable-length structures like
vectors: we dump these element-by-element instead of treating the whole
thing as one big "object" with a large and variable number of fields.


Reply | Threaded
Open this post in threaded view
|

Re: buffer.c/buffer.h: How to add new buffer-local variables?

Eli Zaretskii
> Date: Tue, 2 Apr 2019 11:46:22 -0700
> From: "Daniel Colascione" <[hidden email]>
> Cc: "Daniel Colascione" <[hidden email]>,
>  [hidden email]
>
> > The comment says "sanity check", but I would like to understand what
> > kind of sanity is being checked here, and what should be done when
> > some structure we dump becomes larger than this value.  E.g., is there
> > some other limit that requires that offsets of dumped fields never
> > exceed 1024 here?  I'd like to document in comments what to do when
> > the assertion is violated.
>
> Indeed. That comment could have been a lot better. The general idea here
> is this:
>
> When we enter field_relpos, we're in the middle of some code that's
> dumping some data structure field-by-field. The object that we're dumping
> begins at in_start; and in_field is an interior pointer into that object.
> We can't actually check that the two pointers refer to the same object: C
> doesn't give us that level of introspection. But if the two pointers point
> to addresses that differ by a lot, then the two pointers probably don't
> refer to the same object, and in this case, we can fail an assertion. 1024
> is probably too conservative here. We probably want to greatly increase
> this number (say, to 32k) and also to give it a named constantly, maybe
> something like PDUMPER_MAXIMUM_STRUCT_SIZE.

Thanks, I went with a smaller value, 2KB.  It should be good enough
for now, as the largest object is slightly below 1KB.

> Note that this limit doesn't apply to big variable-length structures like
> vectors: we dump these element-by-element instead of treating the whole
> thing as one big "object" with a large and variable number of fields.

Right.

Reply | Threaded
Open this post in threaded view
|

Re: buffer.c/buffer.h: How to add new buffer-local variables?

Keith David Bershatsky
In reply to this post by Keith David Bershatsky
Increasing the eassert value in field_relpos, and increasing the value of MAX_PER_BUFFER_VARS, are sufficient to build and use Emacs for the NS and X11 ports.

The w32 port of Emacs, however, crashes when adding one too many local-variables in buffer.c/h that breaks the camel's back:

Program received signal SIGSEGV, Segmentation fault.
0x01189584 in vector_marked_p (v=0xbaadf008) at alloc.c:3980
3980      return XVECTOR_MARKED_P (v);
(gdb) bt
#0  0x01189584 in vector_marked_p (v=0xbaadf008) at alloc.c:3980
#1  0x0118dc21 in mark_object (arg=...) at alloc.c:6684
#2  0x0118d2d4 in mark_vectorlike (header=0x5ab4b98) at alloc.c:6361
#3  0x0118d510 in mark_buffer (buffer=0x5ab4b98) at alloc.c:6428
#4  0x0118dd1f in mark_object (arg=...) at alloc.c:6717
#5  0x0118d69d in mark_localized_symbol (ptr=0x4a86c40) at alloc.c:6481

Attached is a gdb backtrace and a minimal diff working example that causes the w32 port to crash during garbage collection.  To reproduce the crash on the w32 port of Emacs, launch the GUI Emacs and at the *GNU Emacs* welcome screen, and hold down the right arrow key until a garbage collection occurs.

Keith


001.diff (16K) Download Attachment
gdb_001.txt (89K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: buffer.c/buffer.h: How to add new buffer-local variables?

Eli Zaretskii
> Date:  Thu, 04 Apr 2019 11:57:02 -0700
> From:  Keith David Bershatsky <[hidden email]>
> Cc:  [hidden email],Stefan Monnier <[hidden email]>,Andreas Schwab <[hidden email]>,Alan Mackenzie <[hidden email]>,Daniel Colascione <[hidden email]>
>
> Increasing the eassert value in field_relpos, and increasing the value of MAX_PER_BUFFER_VARS, are sufficient to build and use Emacs for the NS and X11 ports.
>
> The w32 port of Emacs, however, crashes when adding one too many local-variables in buffer.c/h that breaks the camel's back:
>
> Program received signal SIGSEGV, Segmentation fault.
> 0x01189584 in vector_marked_p (v=0xbaadf008) at alloc.c:3980
                                 ^^^^^^^^^^^^
This 0xbaadf008 is a clear sign of using uninitialized memory.

Reply | Threaded
Open this post in threaded view
|

Re: buffer.c/buffer.h: How to add new buffer-local variables?

Keith David Bershatsky
In reply to this post by Keith David Bershatsky
Using an unsophisticated method of going back in time and building various versions of Emacs on the w32 platform, I tracked down the inability to add a few new buffer-local variables in buffer.c/buffer.h to a commit that occurred on 01/31/2019:  05d2fc7170fb66a87601b1c76ddae2c1b7b4b934.

Steps to reproduce the issue:

1.  git reset --hard 05d2fc7170fb66a87601b1c76ddae2c1b7b4b934

2.  Apply the attached patch that adds a few new buffer local variables.

3.  Build a w32 version of Emacs [I use MinGW_32 and ezwinports downloaded a couple of years ago, on a Windows XP SP-3 box]:

CFLAGS='-O0 -g3' ./configure \
--prefix=/c/docume~1/admini~1/desktop/trunk \
--enable-checking='yes,glyphs' \
--enable-check-lisp-object-type \
--without-compress-install \
--without-makeinfo \
--with-gnutls=no \
--with-mailutils \
--without-makeinfo

4.  The build terminates abnormally at the following location:

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.
make[2]: *** [emacs-lisp/macroexp.elc] Error 3
make[2]: Leaving directory `/c/docume~1/admini~1/desktop/.0.2019_01_31__05d2fc7170fb66a87601b1c76ddae2c1b7b4b934/lisp'
make[1]: *** [bootstrap-emacs.pdmp] Error 2
make[1]: Leaving directory `/c/docume~1/admini~1/desktop/.0.2019_01_31__05d2fc7170fb66a87601b1c76ddae2c1b7b4b934/src'
make: *** [src] Error 2

Beyond pinpointing the exact commit where the Emacs build goes awry, I would be pleased to help further towards resolving this issue -- however, some pointers would be greatly appreciated.  Perhaps there is something that may stand out (to a trained programmer) in the 01/31/2019 commit ....

Keith


2019_04_06__19_33_22_280.diff (14K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: buffer.c/buffer.h: How to add new buffer-local variables?

Eli Zaretskii
> Date:  Sat, 06 Apr 2019 19:50:14 -0700
> From:  Keith David Bershatsky <[hidden email]>
> Cc:  [hidden email],Stefan Monnier <[hidden email]>,Andreas Schwab <[hidden email]>,Alan Mackenzie <[hidden email]>,Daniel Colascione <[hidden email]>,Paul Eggert <[hidden email]>
>
> Using an unsophisticated method of going back in time and building various versions of Emacs on the w32 platform, I tracked down the inability to add a few new buffer-local variables in buffer.c/buffer.h to a commit that occurred on 01/31/2019:  05d2fc7170fb66a87601b1c76ddae2c1b7b4b934.

Is your GDB version 8.1 or later?  If so, please do this:

  $ cd /path/to/emacs/src
  $ gdb ./emacs.exe
  ...
  (gdb) start -Q
  ...
  (gdb) ptype /o current_buffer

and post here the result.  Please do this both in the build before and
after the above commit, and let us see both results.

Reply | Threaded
Open this post in threaded view
|

Re: buffer.c/buffer.h: How to add new buffer-local variables?

Keith David Bershatsky
In reply to this post by Keith David Bershatsky
Thank you, Eli, for providing the steps to facilitate troubleshooting this issue.  The attached files may make it easier to do a diff between the gdb output for both commits.  I performed the gdb tests on the stock Emacs versions on the w32 platform without making any edits by me to the underlying source code; i.e., I did not add any buffer-local variables in these tests.

a68eee50eb515b28b448894299334afced26ef78

and

05d2fc7170fb66a87601b1c76ddae2c1b7b4b934

I am also posing the results in text form below just in case you prefer that instead of text files:

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

a68eee50eb515b28b448894299334afced26ef78

$ gdb ./emacs.exe

...

(gdb) start -Q

...

(gdb) ptype /o current_buffer

type = struct buffer {
/*    0      |     4 */    union vectorlike_header {
/*                 4 */        ptrdiff_t size;

                               /* total size (bytes):    4 */
                           } header;
/*    4      |     4 */    Lisp_Object name_;
/*    8      |     4 */    Lisp_Object filename_;
/*   12      |     4 */    Lisp_Object directory_;
/*   16      |     4 */    Lisp_Object backed_up_;
/*   20      |     4 */    Lisp_Object save_length_;
/*   24      |     4 */    Lisp_Object auto_save_file_name_;
/*   28      |     4 */    Lisp_Object read_only_;
/*   32      |     4 */    Lisp_Object mark_;
/*   36      |     4 */    Lisp_Object local_var_alist_;
/*   40      |     4 */    Lisp_Object major_mode_;
/*   44      |     4 */    Lisp_Object mode_name_;
/*   48      |     4 */    Lisp_Object mode_line_format_;
/*   52      |     4 */    Lisp_Object header_line_format_;
/*   56      |     4 */    Lisp_Object keymap_;
/*   60      |     4 */    Lisp_Object abbrev_table_;
/*   64      |     4 */    Lisp_Object syntax_table_;
/*   68      |     4 */    Lisp_Object category_table_;
/*   72      |     4 */    Lisp_Object case_fold_search_;
/*   76      |     4 */    Lisp_Object tab_width_;
/*   80      |     4 */    Lisp_Object fill_column_;
/*   84      |     4 */    Lisp_Object left_margin_;
/*   88      |     4 */    Lisp_Object auto_fill_function_;
/*   92      |     4 */    Lisp_Object downcase_table_;
/*   96      |     4 */    Lisp_Object upcase_table_;
/*  100      |     4 */    Lisp_Object case_canon_table_;
/*  104      |     4 */    Lisp_Object case_eqv_table_;
/*  108      |     4 */    Lisp_Object truncate_lines_;
/*  112      |     4 */    Lisp_Object word_wrap_;
/*  116      |     4 */    Lisp_Object ctl_arrow_;
/*  120      |     4 */    Lisp_Object bidi_display_reordering_;
/*  124      |     4 */    Lisp_Object bidi_paragraph_direction_;
/*  128      |     4 */    Lisp_Object bidi_paragraph_separate_re_;
/*  132      |     4 */    Lisp_Object bidi_paragraph_start_re_;
/*  136      |     4 */    Lisp_Object selective_display_;
/*  140      |     4 */    Lisp_Object selective_display_ellipses_;
/*  144      |     4 */    Lisp_Object minor_modes_;
/*  148      |     4 */    Lisp_Object overwrite_mode_;
/*  152      |     4 */    Lisp_Object abbrev_mode_;
/*  156      |     4 */    Lisp_Object display_table_;
/*  160      |     4 */    Lisp_Object mark_active_;
/*  164      |     4 */    Lisp_Object enable_multibyte_characters_;
/*  168      |     4 */    Lisp_Object buffer_file_coding_system_;
/*  172      |     4 */    Lisp_Object file_format_;
/*  176      |     4 */    Lisp_Object auto_save_file_format_;
/*  180      |     4 */    Lisp_Object cache_long_scans_;
/*  184      |     4 */    Lisp_Object width_table_;
/*  188      |     4 */    Lisp_Object pt_marker_;
/*  192      |     4 */    Lisp_Object begv_marker_;
/*  196      |     4 */    Lisp_Object zv_marker_;
/*  200      |     4 */    Lisp_Object point_before_scroll_;
/*  204      |     4 */    Lisp_Object file_truename_;
/*  208      |     4 */    Lisp_Object invisibility_spec_;
/*  212      |     4 */    Lisp_Object last_selected_window_;
/*  216      |     4 */    Lisp_Object display_count_;
/*  220      |     4 */    Lisp_Object left_margin_cols_;
/*  224      |     4 */    Lisp_Object right_margin_cols_;
/*  228      |     4 */    Lisp_Object left_fringe_width_;
/*  232      |     4 */    Lisp_Object right_fringe_width_;
/*  236      |     4 */    Lisp_Object fringes_outside_margins_;
/*  240      |     4 */    Lisp_Object scroll_bar_width_;
/*  244      |     4 */    Lisp_Object scroll_bar_height_;
/*  248      |     4 */    Lisp_Object vertical_scroll_bar_type_;
/*  252      |     4 */    Lisp_Object horizontal_scroll_bar_type_;
/*  256      |     4 */    Lisp_Object indicate_empty_lines_;
/*  260      |     4 */    Lisp_Object indicate_buffer_boundaries_;
/*  264      |     4 */    Lisp_Object fringe_indicator_alist_;
---Type <return> to continue, or q <return> to quit---
/*  268      |     4 */    Lisp_Object fringe_cursor_alist_;
/*  272      |     4 */    Lisp_Object display_time_;
/*  276      |     4 */    Lisp_Object scroll_up_aggressively_;
/*  280      |     4 */    Lisp_Object scroll_down_aggressively_;
/*  284      |     4 */    Lisp_Object cursor_type_;
/*  288      |     4 */    Lisp_Object extra_line_spacing_;
/*  292      |     4 */    Lisp_Object cursor_in_non_selected_windows_;
/*  296      |    72 */    struct buffer_text {
/*  296      |     4 */        unsigned char *beg;
/*  300      |     4 */        ptrdiff_t gpt;
/*  304      |     4 */        ptrdiff_t z;
/*  308      |     4 */        ptrdiff_t gpt_byte;
/*  312      |     4 */        ptrdiff_t z_byte;
/*  316      |     4 */        ptrdiff_t gap_size;
/*  320      |     4 */        EMACS_INT modiff;
/*  324      |     4 */        EMACS_INT chars_modiff;
/*  328      |     4 */        EMACS_INT save_modiff;
/*  332      |     4 */        EMACS_INT overlay_modiff;
/*  336      |     4 */        EMACS_INT compact;
/*  340      |     4 */        ptrdiff_t beg_unchanged;
/*  344      |     4 */        ptrdiff_t end_unchanged;
/*  348      |     4 */        EMACS_INT unchanged_modified;
/*  352      |     4 */        EMACS_INT overlay_unchanged_modified;
/*  356      |     4 */        INTERVAL intervals;
/*  360      |     4 */        struct Lisp_Marker *markers;
/*  364:31   |     4 */        bool_bf inhibit_shrinking : 1;
/*  364:30   |     4 */        bool_bf redisplay : 1;

                               /* total size (bytes):   72 */
                           } own_text;
/*  368      |     4 */    struct buffer_text *text;
/*  372      |     4 */    struct buffer *next;
/*  376      |     4 */    ptrdiff_t pt;
/*  380      |     4 */    ptrdiff_t pt_byte;
/*  384      |     4 */    ptrdiff_t begv;
/*  388      |     4 */    ptrdiff_t begv_byte;
/*  392      |     4 */    ptrdiff_t zv;
/*  396      |     4 */    ptrdiff_t zv_byte;
/*  400      |     4 */    struct buffer *base_buffer;
/*  404      |     4 */    int indirections;
/*  408      |     4 */    int window_count;
/*  412      |    50 */    char local_flags[50];
/* XXX  2-byte hole  */
/*  464      |    16 */    struct timespec {
/*  464      |     8 */        union {
/*                 4 */            time_t tv_sec;
/*                 4 */            __time32_t __tv32_sec;
/*                 8 */            __time64_t __tv64_sec;

                                   /* total size (bytes):    8 */
                               };
/*  472      |     4 */        long tv_nsec;

                               /* total size (bytes):   16 */
                           } modtime;
/*  480      |     4 */    off_t modtime_size;
/*  484      |     4 */    EMACS_INT auto_save_modified;
/*  488      |     4 */    EMACS_INT display_error_modiff;
/*  492      |     4 */    time_t auto_save_failure_time;
/*  496      |     4 */    ptrdiff_t last_window_start;
/*  500      |     4 */    struct region_cache *newline_cache;
/*  504      |     4 */    struct region_cache *width_run_cache;
/*  508      |     4 */    struct region_cache *bidi_paragraph_cache;
/*  512:31   |     4 */    bool_bf prevent_redisplay_optimizations_p : 1;
/*  512:30   |     4 */    bool_bf clip_changed : 1;
/* XXX  6-bit hole   */
/* XXX  3-byte hole  */
/*  516      |     4 */    struct Lisp_Overlay *overlays_before;
/*  520      |     4 */    struct Lisp_Overlay *overlays_after;
/*  524      |     4 */    ptrdiff_t overlay_center;
/*  528      |     4 */    Lisp_Object undo_list_;

---Type <return> to continue, or q <return> to quit---
                           /* total size (bytes):  536 */
                         } *
(gdb)

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

05d2fc7170fb66a87601b1c76ddae2c1b7b4b934

$ gdb ./emacs.exe

...

(gdb) start -Q

...

(gdb) ptype /o current_buffer

type = struct buffer {
/*    0      |     4 */    union vectorlike_header {
/*                 4 */        ptrdiff_t size;

                               /* total size (bytes):    4 */
                           } header;
/*    4      |     4 */    Lisp_Object name_;
/*    8      |     4 */    Lisp_Object filename_;
/*   12      |     4 */    Lisp_Object directory_;
/*   16      |     4 */    Lisp_Object backed_up_;
/*   20      |     4 */    Lisp_Object save_length_;
/*   24      |     4 */    Lisp_Object auto_save_file_name_;
/*   28      |     4 */    Lisp_Object read_only_;
/*   32      |     4 */    Lisp_Object mark_;
/*   36      |     4 */    Lisp_Object local_var_alist_;
/*   40      |     4 */    Lisp_Object major_mode_;
/*   44      |     4 */    Lisp_Object mode_name_;
/*   48      |     4 */    Lisp_Object mode_line_format_;
/*   52      |     4 */    Lisp_Object header_line_format_;
/*   56      |     4 */    Lisp_Object keymap_;
/*   60      |     4 */    Lisp_Object abbrev_table_;
/*   64      |     4 */    Lisp_Object syntax_table_;
/*   68      |     4 */    Lisp_Object category_table_;
/*   72      |     4 */    Lisp_Object case_fold_search_;
/*   76      |     4 */    Lisp_Object tab_width_;
/*   80      |     4 */    Lisp_Object fill_column_;
/*   84      |     4 */    Lisp_Object left_margin_;
/*   88      |     4 */    Lisp_Object auto_fill_function_;
/*   92      |     4 */    Lisp_Object downcase_table_;
/*   96      |     4 */    Lisp_Object upcase_table_;
/*  100      |     4 */    Lisp_Object case_canon_table_;
/*  104      |     4 */    Lisp_Object case_eqv_table_;
/*  108      |     4 */    Lisp_Object truncate_lines_;
/*  112      |     4 */    Lisp_Object word_wrap_;
/*  116      |     4 */    Lisp_Object ctl_arrow_;
/*  120      |     4 */    Lisp_Object bidi_display_reordering_;
/*  124      |     4 */    Lisp_Object bidi_paragraph_direction_;
/*  128      |     4 */    Lisp_Object bidi_paragraph_separate_re_;
/*  132      |     4 */    Lisp_Object bidi_paragraph_start_re_;
/*  136      |     4 */    Lisp_Object selective_display_;
/*  140      |     4 */    Lisp_Object selective_display_ellipses_;
/*  144      |     4 */    Lisp_Object minor_modes_;
/*  148      |     4 */    Lisp_Object overwrite_mode_;
/*  152      |     4 */    Lisp_Object abbrev_mode_;
/*  156      |     4 */    Lisp_Object display_table_;
/*  160      |     4 */    Lisp_Object mark_active_;
/*  164      |     4 */    Lisp_Object enable_multibyte_characters_;
/*  168      |     4 */    Lisp_Object buffer_file_coding_system_;
/*  172      |     4 */    Lisp_Object file_format_;
/*  176      |     4 */    Lisp_Object auto_save_file_format_;
/*  180      |     4 */    Lisp_Object cache_long_scans_;
/*  184      |     4 */    Lisp_Object width_table_;
/*  188      |     4 */    Lisp_Object pt_marker_;
/*  192      |     4 */    Lisp_Object begv_marker_;
/*  196      |     4 */    Lisp_Object zv_marker_;
/*  200      |     4 */    Lisp_Object point_before_scroll_;
/*  204      |     4 */    Lisp_Object file_truename_;
/*  208      |     4 */    Lisp_Object invisibility_spec_;
/*  212      |     4 */    Lisp_Object last_selected_window_;
/*  216      |     4 */    Lisp_Object display_count_;
/*  220      |     4 */    Lisp_Object left_margin_cols_;
/*  224      |     4 */    Lisp_Object right_margin_cols_;
/*  228      |     4 */    Lisp_Object left_fringe_width_;
/*  232      |     4 */    Lisp_Object right_fringe_width_;
/*  236      |     4 */    Lisp_Object fringes_outside_margins_;
/*  240      |     4 */    Lisp_Object scroll_bar_width_;
/*  244      |     4 */    Lisp_Object scroll_bar_height_;
/*  248      |     4 */    Lisp_Object vertical_scroll_bar_type_;
/*  252      |     4 */    Lisp_Object horizontal_scroll_bar_type_;
/*  256      |     4 */    Lisp_Object indicate_empty_lines_;
/*  260      |     4 */    Lisp_Object indicate_buffer_boundaries_;
/*  264      |     4 */    Lisp_Object fringe_indicator_alist_;
---Type <return> to continue, or q <return> to quit---
/*  268      |     4 */    Lisp_Object fringe_cursor_alist_;
/*  272      |     4 */    Lisp_Object display_time_;
/*  276      |     4 */    Lisp_Object scroll_up_aggressively_;
/*  280      |     4 */    Lisp_Object scroll_down_aggressively_;
/*  284      |     4 */    Lisp_Object cursor_type_;
/*  288      |     4 */    Lisp_Object extra_line_spacing_;
/*  292      |     4 */    Lisp_Object cursor_in_non_selected_windows_;
/*  296      |   104 */    struct buffer_text {
/*  296      |     4 */        unsigned char *beg;
/*  300      |     4 */        ptrdiff_t gpt;
/*  304      |     4 */        ptrdiff_t z;
/*  308      |     4 */        ptrdiff_t gpt_byte;
/*  312      |     4 */        ptrdiff_t z_byte;
/*  316      |     4 */        ptrdiff_t gap_size;
/*  320      |     8 */        modiff_count modiff;
/*  328      |     8 */        modiff_count chars_modiff;
/*  336      |     8 */        modiff_count save_modiff;
/*  344      |     8 */        modiff_count overlay_modiff;
/*  352      |     8 */        modiff_count compact;
/*  360      |     4 */        ptrdiff_t beg_unchanged;
/*  364      |     4 */        ptrdiff_t end_unchanged;
/*  368      |     8 */        modiff_count unchanged_modified;
/*  376      |     8 */        modiff_count overlay_unchanged_modified;
/*  384      |     4 */        INTERVAL intervals;
/*  388      |     4 */        struct Lisp_Marker *markers;
/*  392:31   |     4 */        bool_bf inhibit_shrinking : 1;
/*  392:30   |     4 */        bool_bf redisplay : 1;

                               /* total size (bytes):  104 */
                           } own_text;
/*  400      |     4 */    struct buffer_text *text;
/*  404      |     4 */    struct buffer *next;
/*  408      |     4 */    ptrdiff_t pt;
/*  412      |     4 */    ptrdiff_t pt_byte;
/*  416      |     4 */    ptrdiff_t begv;
/*  420      |     4 */    ptrdiff_t begv_byte;
/*  424      |     4 */    ptrdiff_t zv;
/*  428      |     4 */    ptrdiff_t zv_byte;
/*  432      |     4 */    struct buffer *base_buffer;
/*  436      |     4 */    int indirections;
/*  440      |     4 */    int window_count;
/*  444      |    50 */    char local_flags[50];
/* XXX  2-byte hole  */
/*  496      |    16 */    struct timespec {
/*  496      |     8 */        union {
/*                 4 */            time_t tv_sec;
/*                 4 */            __time32_t __tv32_sec;
/*                 8 */            __time64_t __tv64_sec;

                                   /* total size (bytes):    8 */
                               };
/*  504      |     4 */        long tv_nsec;

                               /* total size (bytes):   16 */
                           } modtime;
/*  512      |     4 */    off_t modtime_size;
/* XXX  4-byte hole  */
/*  520      |     8 */    modiff_count auto_save_modified;
/*  528      |     8 */    modiff_count display_error_modiff;
/*  536      |     4 */    time_t auto_save_failure_time;
/*  540      |     4 */    ptrdiff_t last_window_start;
/*  544      |     4 */    struct region_cache *newline_cache;
/*  548      |     4 */    struct region_cache *width_run_cache;
/*  552      |     4 */    struct region_cache *bidi_paragraph_cache;
/*  556:31   |     4 */    bool_bf prevent_redisplay_optimizations_p : 1;
/*  556:30   |     4 */    bool_bf clip_changed : 1;
/* XXX  6-bit hole   */
/* XXX  3-byte hole  */
/*  560      |     4 */    struct Lisp_Overlay *overlays_before;
/*  564      |     4 */    struct Lisp_Overlay *overlays_after;
/*  568      |     4 */    ptrdiff_t overlay_center;
/*  572      |     4 */    Lisp_Object undo_list_;
---Type <return> to continue, or q <return> to quit---

                           /* total size (bytes):  576 */
                         } *
(gdb)

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

> Date: [04-07-2019 08:48:21] <07 Apr 2019 18:48:21 +0300>
> From: Eli Zaretskii <[hidden email]>
> To: Keith David Bershatsky <[hidden email]>
> CC: [hidden email],[hidden email],[hidden email],[hidden email],[hidden email],[hidden email]
> Subject: Re: buffer.c/buffer.h:  How to add new buffer-local variables?
>
> > Date:  Sat, 06 Apr 2019 19:50:14 -0700
> > From:  Keith David Bershatsky <[hidden email]>
> > Cc:  [hidden email],Stefan Monnier <[hidden email]>,Andreas Schwab <[hidden email]>,Alan Mackenzie <[hidden email]>,Daniel Colascione <[hidden email]>,Paul Eggert <[hidden email]>
> >
> > Using an unsophisticated method of going back in time and building various versions of Emacs on the w32 platform, I tracked down the inability to add a few new buffer-local variables in buffer.c/buffer.h to a commit that occurred on 01/31/2019:  05d2fc7170fb66a87601b1c76ddae2c1b7b4b934.
>
> Is your GDB version 8.1 or later?  If so, please do this:
>
>   $ cd /path/to/emacs/src
>   $ gdb ./emacs.exe
>   ...
>   (gdb) start -Q
>   ...
>   (gdb) ptype /o current_buffer
>
> and post here the result.  Please do this both in the build before and
> after the above commit, and let us see both results.


05d2fc7170fb66a87601b1c76ddae2c1b7b4b934.txt (10K) Download Attachment
a68eee50eb515b28b448894299334afced26ef78.txt (10K) Download Attachment
12