bug#30539: 26.0; `char-displayable-p' is much slower in Emacs 25 and 26

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

bug#30539: 26.0; `char-displayable-p' is much slower in Emacs 25 and 26

Drew Adams
This is not from emacs -Q, and I do have many fonts installed.  I
repeated it from emacs -Q and I've attached those profiler reports
also.  But with emacs -Q the test code (`my-test') finished
immediately.  With my setup it took many seconds.  I use this font
by default - dunno whether that makes the difference:

(font . "-outline-Lucida Console-normal-normal-normal-mono-4-*-*-*-c-*-iso8859-1")
(font-parameter . "-*-Lucida Console-normal-r-*-*-14-*-*-*-c-*-iso8859-1")

Recipe I followed:

Evaluate the attached code (`throw-mule-bug.el'), then look at buffers
*CPU Profiler Report* and *Memory Profiler Report*.  I've attached these
reports as files:

throw-mule-bug-memory-report-E26-Q - Emacs 26 from emacs -Q
throw-mule-bug-cpu-report-E26-Q    - Emacs 26 from emacs -Q
throw-mule-bug-memory-report-E24 - Emacs 24.5 with my setup
throw-mule-bug-cpu-report-E24    - Emacs 24.5 with my setup
throw-mule-bug-memory-report-E26 - Emacs 26P2 with my setup
throw-mule-bug-cpu-report-E26    - Emacs 26P2 with my setup
 
In Emacs 25.3.1 and 26 `char-displayable-p' is MUCH slower
than it is in Emacs 24.5.  In 24.5 the evaluation of `my-test'
seems instantaneous.  In Emacs 26 it takes many seconds.

From the reports, using my setup:

E26 memory:

- my-delete-if-not                         225,165,075  52%
 - let                                     222,218,069  51%
  - while                                  222,218,069  51%
   - if                                    222,218,069  51%
    - not                                  222,218,069  51%
     - funcall                             222,218,069  51%
      - my-char-displayable-p              222,218,069  51%
       - char-displayable-p                222,218,069  51%
        - cond                             222,218,069  51%
           let                             189,022,646  43%
 - while                                     2,947,006   0%
  - and                                      2,947,006   0%
   - not                                     2,947,006   0%
    - funcall                                2,947,006   0%
     - my-char-displayable-p                 2,947,006   0%
      - char-displayable-p                   2,947,006   0%
       - cond                                2,947,006   0%
          let                                1,836,898   0%

E26 cpu:

- my-delete-if-not                                1609  70%
 - let                                            1602  70%
  - while                                         1602  70%
   - if                                           1602  70%
    - not                                         1602  70%
     - funcall                                    1602  70%
      - my-char-displayable-p                     1602  70%
       - char-displayable-p                       1602  70%
        - cond                                    1602  70%
           let                                    1602  70%
 - while                                             7   0%
  - and                                              7   0%
   - not                                             7   0%
    - funcall                                        7   0%
     - my-char-displayable-p                         7   0%
      - char-displayable-p                           7   0%
       - cond                                        7   0%
          let                                        7   0%

In GNU Emacs 26.0.91 (build 1, x86_64-w64-mingw32)
 of 2018-01-22
Repository revision: 752fba992b793a74d202c9cfc3e1a92fd458e748
Windowing system distributor `Microsoft Corp.', version 6.1.7601
Configured using:
 `configure --without-dbus --host=x86_64-w64-mingw32
 --without-compress-install 'CFLAGS=-O2 -static -g3''

throw-mule-bug-cpu-report-E26 (30K) Download Attachment
throw-mule-bug-memory-report-E26-Q (6K) Download Attachment
throw-mule-bug-cpu-report-E26-Q (1K) Download Attachment
throw-mule-bug-memory-report-E24 (28K) Download Attachment
throw-mule-bug-cpu-report-E24 (3K) Download Attachment
throw-mule-bug-memory-report-E26 (111K) Download Attachment
throw-mule-bug.el (49K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

bug#30539: 26.0; `char-displayable-p' is much slower in Emacs 25 and 26

Drew Adams
Sorry, the recipe I sent before was not good.  The initial
alist of chars had already been purged of any chars that
are not `char-displayable-p'.

Please try attached file `throw-mule-bug-2.el' instead.
This time the alist of chars in variable `char-names'
includes some chars that are not `char-displayable-p'.
Evaluating `char-displayable-p' for those chars is, I
think, where the bottleneck is.

Evaluate the code in `throw-mule-bug-2.el' and then check
buffers *CPU Profiler Report* and *Memory Profiler Report*.

I've attached those reports as these files:

throw-mule-bug-cpu-report2-E24-Q - Emacs 24.5 from `emacs -Q'
throw-mule-bug-mem-report2-E24-Q - Emacs 24.5 from `emacs -Q'
throw-mule-bug-cpu-report2-E26-Q - Emacs 26P2 from `emacs -Q'
throw-mule-bug-mem-report2-E26-Q - Emacs 26P2 from `emacs -Q'

You can see these reports by using `M-x profiler-find-profile'
and entering the report file name at the prompt.

In Emacs 24.5 evaluating `(my-test)' takes only a few _seconds_.
In Emacs 26 (Pretest 2) it takes about 13 _minutes_.

Emacs 25.3.1 has the same problem as Emacs 26.

throw-mule-bug-cpu-report2-E24-Q (1K) Download Attachment
throw-mule-bug-mem-report2-E24-Q (7K) Download Attachment
throw-mule-bug-cpu-report2-E26-Q (1K) Download Attachment
throw-mule-bug-mem-report2-E26-Q (7K) Download Attachment
throw-mule-bug-2.el (88K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

bug#30539: 26.0; `char-displayable-p' is much slower in Emacs 25 and 26

Drew Adams
Can someone please confirm that they can repro this problem?

I do have many fonts installed on my system.  Dunno whether
the recipe from `emacs -Q' has a different effect if you
do or don't have certain fonts installed.  But if not then
I think you should be able to easily and quickly repro the
problem.

Thx.



Reply | Threaded
Open this post in threaded view
|

bug#30539: 26.0; `char-displayable-p' is much slower in Emacs 25 and 26

Noam Postavsky
Drew Adams <[hidden email]> writes:

> Can someone please confirm that they can repro this problem?

I can reproduce on a Windows 10 box.  It looks like something was being
cached before, where now it's not.  E.g., try the following function
(char-names as defined in your throw-mule-bug-2.el).  In Emacs 24,
there's only one slow call.

(defun my-test-each-char ()
  (interactive)
  (view-echo-area-messages)
  (pcase-dolist (`(,name . ,ch) char-names)
    (read-char (format "continue? (next: %s)" name))
    (let ((t0 (current-time))
          dt displayable)
      (setq displayable (char-displayable-p ch))
      (setq dt (subtract-time (current-time) t0))
      (message "%s display:%s (%fs)" name displayable (float-time dt)))))

Doing (setq inhibit-compacting-font-caches t) brings back reasonable
performance.

I can't reproduce on my GNU/Linux box, although that may just be due to
different fonts installed.  In particular, char-displayable-p never gave
me nil.



Reply | Threaded
Open this post in threaded view
|

bug#30539: 26.0; `char-displayable-p' is much slower in Emacs 25 and 26

Drew Adams
> > Can someone please confirm that they can repro this problem?
>
> I can reproduce on a Windows 10 box.  It looks like something was being
> cached before, where now it's not.  E.g., try the following function
> (char-names as defined in your throw-mule-bug-2.el).  In Emacs 24,
> there's only one slow call.
>
> (defun my-test-each-char ()
>   (interactive)
>   (view-echo-area-messages)
>   (pcase-dolist (`(,name . ,ch) char-names)
>     (read-char (format "continue? (next: %s)" name))
>     (let ((t0 (current-time))
>           dt displayable)
>       (setq displayable (char-displayable-p ch))
>       (setq dt (subtract-time (current-time) t0))
>       (message "%s display:%s (%fs)" name displayable (float-time dt)))))
>
> Doing (setq inhibit-compacting-font-caches t) brings back reasonable
> performance.
>
> I can't reproduce on my GNU/Linux box, although that may just be due to
> different fonts installed.  In particular, char-displayable-p never gave
> me nil.

Thanks, Noam.  Yes, I see the same thing: over 4 sec for each
char that is not displayable.

I couldn't try your code with Emacs 24.5 because `pcase-dolist'
and `inhibit-compacting-font-caches' are both undefined.  (How
did you test it in 24.5, or did I misunderstand you?)

But everything else you say checks out, including the effect
of (setq inhibit-compacting-font-caches t).

Is this a bug that is likely to get fixed?

In any case, for Emacs 25-26, I wonder whether I should bind
`inhibit-compacting-font-caches to `t' in my code that uses
`char-displayable-p', or whether I should just skip the
`char-displayable-p' test for Emacs 25-26.

Another question is whether this bug should/will affect all
users or only some?  If the latter then I can let users
decide whether to test `char-displayable-p' (I have an
option for that anyway) or whether to bind
`inhibit-compacting-font-caches to `t'.  If only some users
are affected by the bug, do we know why?  Does it have to
do with the fonts they have installed, for example?



Reply | Threaded
Open this post in threaded view
|

bug#30539: 26.0; `char-displayable-p' is much slower in Emacs 25 and 26

Noam Postavsky
Drew Adams <[hidden email]> writes:

> I couldn't try your code with Emacs 24.5 because `pcase-dolist'

pcase-dolist is not autoloaded in 24.5, but is available after loading
pcase.el.

> and `inhibit-compacting-font-caches' are both undefined.

Of course setting inhibit-compacting-font-caches has no effect in 24.5,
I meant that remark just for 25+.

> But everything else you say checks out, including the effect
> of (setq inhibit-compacting-font-caches t).
>
> Is this a bug that is likely to get fixed?

Unfortunately, no, I don't think so (at least not soon).  My
understanding is that this inhibit-compacting-font-caches variable is
due to several mysterious font bugs with different users needing
different settings to work around them, and there isn't anyone who has a
good idea of how to sort it out.

> Another question is whether this bug should/will affect all
> users or only some?  If the latter then I can let users
> decide whether to test `char-displayable-p' (I have an
> option for that anyway) or whether to bind
> `inhibit-compacting-font-caches to `t'.  If only some users
> are affected by the bug, do we know why?  Does it have to
> do with the fonts they have installed, for example?

Well, as I mentioned, I don't see it on my GNU/Linux box, so it's not
universal.  I would guess the fonts installed is the main factor.




Reply | Threaded
Open this post in threaded view
|

bug#30539: 26.0; `char-displayable-p' is much slower in Emacs 25 and 26

Drew Adams
> > Is this a bug that is likely to get fixed?
>
> Unfortunately, no, I don't think so (at least not soon).  My
> understanding is that this inhibit-compacting-font-caches variable is
> due to several mysterious font bugs with different users needing
> different settings to work around them, and there isn't anyone who has a
> good idea of how to sort it out.
>
> > Another question is whether this bug should/will affect all
> > users or only some?  If the latter then I can let users
> > decide whether to test `char-displayable-p' (I have an
> > option for that anyway) or whether to bind
> > `inhibit-compacting-font-caches to `t'.  If only some users
> > are affected by the bug, do we know why?  Does it have to
> > do with the fonts they have installed, for example?
>
> Well, as I mentioned, I don't see it on my GNU/Linux box, so it's not
> universal.  I would guess the fonts installed is the main factor.

I googled a bit for that variable, and there are a bunch of
Emacs bugs and other posts about it.  Seems like (to be
confirmed) it is a problem only for MS Windows (?), and
maybe only for TrueType fonts (?).

And it seems like lots of folks run into it (though others
do not), so that lots of people (particularly with CJK
fonts?) are just systematically setting the variable to t.

I do wonder what the best approach is for my library.  If
I knew that the problem didn't exist for non-Windows that
would let me at least remove non-Windows from cases where
I try to do something.  I'll probably make the code, when
on Windows, by default use a non-nil value of the var by
default (e.g. as an option default value).  But it would be
good to know more about the cases where the problem can arise.



Reply | Threaded
Open this post in threaded view
|

bug#30539: 26.0; `char-displayable-p' is much slower in Emacs 25 and 26

Eli Zaretskii
> Date: Thu, 22 Feb 2018 20:07:09 -0800 (PST)
> From: Drew Adams <[hidden email]>
> Cc: [hidden email]
>
> > > Another question is whether this bug should/will affect all
> > > users or only some?  If the latter then I can let users
> > > decide whether to test `char-displayable-p' (I have an
> > > option for that anyway) or whether to bind
> > > `inhibit-compacting-font-caches to `t'.  If only some users
> > > are affected by the bug, do we know why?  Does it have to
> > > do with the fonts they have installed, for example?
> >
> > Well, as I mentioned, I don't see it on my GNU/Linux box, so it's not
> > universal.  I would guess the fonts installed is the main factor.
>
> I googled a bit for that variable, and there are a bunch of
> Emacs bugs and other posts about it.  Seems like (to be
> confirmed) it is a problem only for MS Windows (?), and
> maybe only for TrueType fonts (?).

AFAIR, this isn't seen only on Windows.  And yes, only some fonts need
this, if you have them installed.



Reply | Threaded
Open this post in threaded view
|

bug#30539: 26.0; `char-displayable-p' is much slower in Emacs 25 and 26

Charles A. Roelli
In reply to this post by Drew Adams
> Date: Tue, 20 Feb 2018 10:08:36 -0800 (PST)
> From: Drew Adams <[hidden email]>
>
> Sorry, the recipe I sent before was not good.  The initial
> alist of chars had already been purged of any chars that
> are not `char-displayable-p'.
>
> Please try attached file `throw-mule-bug-2.el' instead.
> This time the alist of chars in variable `char-names'
> includes some chars that are not `char-displayable-p'.
> Evaluating `char-displayable-p' for those chars is, I
> think, where the bottleneck is.
>
> Evaluate the code in `throw-mule-bug-2.el' and then check
> buffers *CPU Profiler Report* and *Memory Profiler Report*.
>
> I've attached those reports as these files:
>
> throw-mule-bug-cpu-report2-E24-Q - Emacs 24.5 from `emacs -Q'
> throw-mule-bug-mem-report2-E24-Q - Emacs 24.5 from `emacs -Q'
> throw-mule-bug-cpu-report2-E26-Q - Emacs 26P2 from `emacs -Q'
> throw-mule-bug-mem-report2-E26-Q - Emacs 26P2 from `emacs -Q'
>
> You can see these reports by using `M-x profiler-find-profile'
> and entering the report file name at the prompt.
>
> In Emacs 24.5 evaluating `(my-test)' takes only a few _seconds_.
> In Emacs 26 (Pretest 2) it takes about 13 _minutes_.
>
> Emacs 25.3.1 has the same problem as Emacs 26.

FWIW, on macOS 10.6, evaluating (my-test) the first time takes
~4.7 seconds, then further runs take about 0.01 seconds.  Setting
inhibit-compacting-font-caches to `t' seems to have no effect on
evaluation time in either case.

But I have noticed that displaying files containing certain Unicode
characters can lock Emacs for a little while.  I wonder if that is
also some manifestation of this bug.  Do you also see a slow down when
you visit a file containing the characters in the `char-names'
variable you defined?  Or is the slowness limited to running them
through `char-displayable-p'?



Reply | Threaded
Open this post in threaded view
|

bug#30539: 26.0; `char-displayable-p' is much slower in Emacs 25 and 26

Drew Adams
In reply to this post by Noam Postavsky
> From: Noam Postavsky Sent: Thursday, February 22, 2018 7:32 PM
>
> > Is this a bug that is likely to get fixed?
>
> Unfortunately, no, I don't think so (at least not soon).  My
> understanding is that this inhibit-compacting-font-caches variable is
> due to several mysterious font bugs with different users needing
> different settings to work around them, and there isn't anyone who has a
> good idea of how to sort it out.

Just checking, to see if this situation might have
changed in the last 2 1/2 years.

Thx.