Let's make the GC safe and iterative (Was: Re: bug#30626)

classic Classic list List threaded Threaded
196 messages Options
1234 ... 10
Reply | Threaded
Open this post in threaded view
|

Let's make the GC safe and iterative (Was: Re: bug#30626)

Daniel Colascione-5
Noam mentioned that I should make a new thread for this proposal, so I'm
posting an edited version of my original message.

tl;dr: we should be able to make the GC non-recursive with minimal
overhead, solving the "Emacs crashed because we ran out of stack space
in GC" problem once and for all.

On 02/27/2018 10:08 AM, Eli Zaretskii wrote:
> What can we do instead in such cases?  Stack-overflow protection
> cannot work in GC, so you are shooting yourself in the foot by
> creating such large recursive structures.  By the time we get to GC,
> where the problem will happen, it's too late, because the memory was
> already allocated.
>
> Does anyone has a reasonable idea for avoiding the crash in such
> programs?

We need to fix GC being deeply recursive once and for all. Tweaking
stack sizes on various platforms and trying to spot-fix GC for the
occasional deeply recursive structure is annoying. Here's my proposal:

I. NAIVE APPROACH

Turn garbage_collect_1 into a queue-draining loop, initializing the
object queue with the GC roots before draining it. We'll make
mark_object put an object on this queue, turning the existing
mark_object code into a mark_queued_object function.

garbage_collect_1 will just call mark_queued_object in a loop;
mark_queued_object can call mark_object, but since mark_object just
enqueues an object and doesn't recurse, we can't exhaust the stack with
deep object graphs. (We'll repurpose the mark bit to mean that the
object is on the to-mark queue; by the time we fully drain the queue,
just before we sweep, the mark bit will have the same meaning it does now.)

We can't allocate memory to hold the queue during GC, so we'll have to
pre-allocate it. We can implement the queue as a list of queue blocks,
where each queue block is an array of 16k or so Lisp_Objects. During
allocation, we'll just make sure we have one Lisp_Object queue-block
slot for every non-self-representing Lisp object we allocate.

Since we know that we'll have enough queue blocks for the worst GC case,
we can have mark_object pull queue blocks from a free list, aborting if
for some reason it ever runs out of queue blocks. (The previous
paragraph guarantees we won't.) garbage_collect_1 will churn through
these heap blocks and place each back on the free list after it's called
mark_queued_object on every Lisp_Object in the queue block.

In this way, in non-pathological cases of GC, we'll end up using the
same few queue blocks over and over. That's a nice optimization, because
we can MADV_DONTNEED unused queue blocks so the OS doesn't actually have
to remember their contents.

In this way, I think we can make the current GC model recursion-proof
without drastically changing how we allocate Lisp objects. The
additional memory requirements should be modest: it's basically one
Lisp_Object per Lisp object allocated.

II. ELABORATION

The naive version of this scheme needs about 4.6MB of overhead on my
current 20MB Emacs heap, but it should be possible to reduce the
overhead significantly by taking advantage of the block allocation we do
for conses and other types --- we can put whole blocks on the queue
instead of pointers to individual block parts, so we can get away with a
much smaller queue.

It's also interesting to note that we don't need separate queue blocks
to put a block on the queue, as we do if we want to enqueue individual
Lisp_Object pointers. Instead, we can add to each block type a pointer
to the next block *on the to-be-marked queue* and a bitmask yielding the
positions within that block that we want to mark.

For example, cons_block right now looks like this:

struct cons_block
{
   /* Place `conses' at the beginning, to ease up CONS_INDEX's job.  */
   struct Lisp_Cons conses[CONS_BLOCK_SIZE];
   bits_word gcmarkbits[1 + CONS_BLOCK_SIZE / BITS_PER_BITS_WORD];
   struct cons_block *next;
};

We'd turn it into something like this:

struct cons_block
{
   /* Place `conses' at the beginning, to ease up CONS_INDEX's job.  */
   struct Lisp_Cons conses[CONS_BLOCK_SIZE];
   bits_word gcmarkbits[1 + CONS_BLOCK_SIZE / BITS_PER_BITS_WORD];
   bits_word scan_pending[1 + CONS_BLOCK_SIZE / BITS_PER_BITS_WORD];
   struct cons_block *next;
   struct cons_block *next_scan_pending;
};

When we call mark_object on a cons, we'll look up its cons_block and
look up the cons in gcmarkbits. If we find the cons mark bit set, we're
done. Otherwise, we look at the scan_pending bit for the cons cell. If
_that's_ set, we're also done. If we find the scan_pending bit unset,
however, we set it, and then look at next_scan_pending. If that's
non-zero, we know the block as a whole is enqueued for scanning, and
we're done. If *that's* zero, then we add the whole block to the
to-be-scanned queue.

We'll modify garbage_collect_1 to drain both the Lisp_Object queue I
described in the last section (which we still need for big objects like
buffers) *and* the queue of blocks pending scanning. When we get a cons
block, we'll scan all the conses with scan_pending bits set to one, set
their gcmarkbits, and remove the cons block from the queue.

That same cons block might make it back onto the queue later if someone
calls mark_object for one if its conses we didn't already scan, but
that's okay. Scanning scan_pending should be very cheap, especially on
modern CPUs with bit-prefix-scan instructions.

Under this approach, the reserved-queue-block scheme would impose an
overhead of somewhere around 1MB on the same heap. (I think it'd
actually be a bit smaller actually.) Conses, strings, and vectors are
the overwhelming majority of heap-allocated objects, and thanks to block
packing, we'd get bookkeeping for them for practically free. This amount
of overhead seems reasonable. I think we may end up actually using less
memory that we would for recursive mark_object stack invocation.

This scheme interacts well with the portable dumper too. pdumper already
uses a big bit array to store mark bits; we'd just add another array for
its scan_pending. We'd basically treat the entire pdumper region as one
big cons_block for GC purposes.

What do you think? I think this approach solves a longstanding fiddly
problem with Emacs GC without too much disruption to the internals. It
also paves the way for concurrent or generational GC if we ever want to
implement these features.

Reply | Threaded
Open this post in threaded view
|

Re: Let's make the GC safe and iterative (Was: Re: bug#30626)

Paul Eggert
On 03/01/2018 03:22 PM, Daniel Colascione wrote:
> What do you think?

Thanks, I like the idea. I suggest implementing the naive version first.
Although the more-elaborate versions saves virtual memory, it's not
clear that it'd be an overall performance win in the typical case, so my
guess is that it's better to start simple.


Reply | Threaded
Open this post in threaded view
|

Re: Let's make the GC safe and iterative

Stefan Monnier
In reply to this post by Daniel Colascione-5
> We need to fix GC being deeply recursive once and for all. Tweaking stack
> sizes on various platforms and trying to spot-fix GC for the occasional
> deeply recursive structure is annoying. Here's my proposal:

I'm OK with making the GC loop without recursing on the C stack, but
I have two comments:
1- Don't use a queue: use a stack.  Mark&sweep naturally work with
   a stack whereas stop&copy naturally works with a queue, so in most cases
   the choice is implicit/accidental, but experience shows that if we
   can choose, the stack is the better option (e.g. there's been
   various works that try to tweak stop&copy to use a stack rather than
   a queue), for reasons of locality.
2- Why do you say "We can't allocate memory to hold the queue during GC"?
   We very well can, and doing it should make things simpler (and make
   sure we don't preallocate way too much memory).

If instead of a queue we use a stack and we just push Lisp_Object values
onto that stack, the resulting space use can't be worse than what we
have now (it's just going to use our malloc-managed stack instead of the
C stack).


        Stefan


Reply | Threaded
Open this post in threaded view
|

Re: Let's make the GC safe and iterative

Daniel Colascione-5
On 03/01/2018 03:38 PM, Stefan Monnier wrote:

>> We need to fix GC being deeply recursive once and for all. Tweaking stack
>> sizes on various platforms and trying to spot-fix GC for the occasional
>> deeply recursive structure is annoying. Here's my proposal:
>
> I'm OK with making the GC loop without recursing on the C stack, but
> I have two comments:
> 1- Don't use a queue: use a stack.  Mark&sweep naturally work with
>     a stack whereas stop&copy naturally works with a queue, so in most cases
>     the choice is implicit/accidental, but experience shows that if we
>     can choose, the stack is the better option (e.g. there's been
>     various works that try to tweak stop&copy to use a stack rather than
>     a queue), for reasons of locality.

Sure. Using a stack instead of a queue is a minor detail; the basic
algorithm is the same.

Within blocks, we still need to be queue-based though: we'll scan the
scan_pending bitmap once; if more bits behind the scan_pending read
position become set during the scan, we won't know until the next time
we examine the block. But we *can* make sure that we re-examine this
block immediately after we finish the scan_pending bit-scan.

> 2- Why do you say "We can't allocate memory to hold the queue during GC"?
>     We very well can, and doing it should make things simpler (and make
>     sure we don't preallocate way too much memory).

We can try, but we need a plan if allocation fails. I don't want Emacs
to be one of those programs that just aborts if malloc fails. Too many
other programs get it wrong these days. I don't think the worst-case
preallocation overhead is severe enough that we have to give up malloc
robustness.

> If instead of a queue we use a stack and we just push Lisp_Object values
> onto that stack, the resulting space use can't be worse than what we
> have now (it's just going to use our malloc-managed stack instead of the
> C stack).

Sure, except that stack use is ephemeral, and the space we allocate for
the stack gets used for lisp too. OTOH, we need to preallocate these
linkage structures and keep them allocated between GCs. But like I said,
I don't think the overhead is worth worrying about.

Reply | Threaded
Open this post in threaded view
|

Re: Let's make the GC safe and iterative

Stefan Monnier
> Sure. Using a stack instead of a queue is a minor detail; the basic
> algorithm is the same.

The point is that a stack will be better locality-wise and (more
importantly) that it should be as good as what we have now w.r.t memory
use (actually, it should be significantly better, because at each
"recursive" step we only push one Lisp_Object on our stack whereas the
current code pushes an activation frame which is at least twice as large
since it also needs a return address).

So I think it does affect the algorithm: I'm not convinced we need to do
anything special for objects allocated from blocks.

>> 2- Why do you say "We can't allocate memory to hold the queue during GC"?
>>     We very well can, and doing it should make things simpler (and make
>>     sure we don't preallocate way too much memory).
> We can try, but we need a plan if allocation fails.

We can just abort the GC (only requires resetting the markbits).
It's not like we GC when our memory is full: we just GC on a regular
basis for "prophylactic" reasons.

> I don't think the worst-case preallocation overhead is severe enough
> that we have to give up malloc robustness.

It sounds pretty significant to me.  I could live with it if it were
indispensable but I find it difficult to justify.

> Sure, except that stack use is ephemeral,

Our manually-managed stack can have the same property.

> and the space we allocate for the stack gets used for lisp too.

Yes, that's a downside of such a change w.r.t average use of the
combined stack space, but I don't think it affect the maximal combined
stack space, so I'm not worried.  This said, we could also move the Lisp
stack to that same manually-managed stack (at least for the bytecode
stacks).


        Stefan

Reply | Threaded
Open this post in threaded view
|

Re: Let's make the GC safe and iterative (Was: Re: bug#30626)

Ken Raeburn-2
In reply to this post by Paul Eggert
On Mar 1, 2018, at 18:29, Paul Eggert <[hidden email]> wrote:

> On 03/01/2018 03:22 PM, Daniel Colascione wrote:
>> What do you think?
>
> Thanks, I like the idea. I suggest implementing the naive version first. Although the more-elaborate versions saves virtual memory, it's not clear that it'd be an overall performance win in the typical case, so my guess is that it's better to start simple.

The more-elaborate version looks like it’s more likely to win on locality of reference as it scans blocks sequentially. That could be significant… especially if anyone is so unfortunate as to still run Emacs on a system small enough that has to page out part of the Lisp memory.

I like the second approach a bit better, but either way, getting away from using the C stack sounds like a big improvement.

Ken
Reply | Threaded
Open this post in threaded view
|

What improvements would be truly useful?

Richard Stallman
[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

An improvement in GC wouldn't be a bad thing, but it may not be worth
the effort.  It is likely to lead to many bugs that would be hard to
fix.  Once working, it would not make much difference to users.
It would only permit some operations on larger problems than now.

When I was working at the AI Lab, one of the older programmers told me
that hackers are often eager to make improvements of this sort: which
make the program better in an abstract sense, but not better for
users.  I took that advice to heart.  Now I pass it on.

Changing Emacs to handle indentation and alignment with
variable-width fonts would be an important and useful change.
Certain kinds of use would make sense, which currently don't.

It would be a big step towards making Emacs do the job of
a word processor, which is what I would like to see some day.
Imagine if you could edit nicely formatted documents directly
with Emacs, instead of using LibreOffice?  LibreOffice is
fine to use, it is free software, but it isn't Emacs.

--
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.


Reply | Threaded
Open this post in threaded view
|

Re: What improvements would be truly useful?

John Yates-4
Richard,

I too had an early mentor impart similar wisdom, possibly around the
same that time yours did.  Since then I have always tried to keep the
end user experience foremost in my work.

"Many bugs that would be hard to fix" is not a given.  I cannot count
the number times I have replaced recursion with iteration.  With
discipline (no other changes to the algorithm) it is a straight
forward conversion.

I agree that merely being able to handle large files is not a strong
justification.  OTOH, in spite of running on a state of the art
desktop with 64GB RAM, 4GHz+ multi-core processor and high-end gamer
graphics Emacs feels progressively less responsive.  Any work
countering that trend gets my vote.

/john

Reply | Threaded
Open this post in threaded view
|

Re: What improvements would be truly useful?

Stefan Monnier
In reply to this post by Richard Stallman
> An improvement in GC wouldn't be a bad thing, but it may not be worth
> the effort.

Making the recursion use an explicitly managed stack should be a fairly
simple change, which will fix some crashes.  I agree that it's probably
not worth trying to be too fancy, but it's really just a bug fix.


        Stefan


Reply | Threaded
Open this post in threaded view
|

Re: What improvements would be truly useful?

Paul Eggert
In reply to this post by John Yates-4
On 03/05/2018 06:02 AM, John Yates wrote:
> in spite of running on a state of the art
> desktop with 64GB RAM, 4GHz+ multi-core processor and high-end gamer
> graphics Emacs feels progressively less responsive

I'm feeling that on my older work desktop too (8 GiB RAM, 2.6 GHz
quad-core, circa-2010 AMD Deneb). However, I suspect it has mostly to do
with things other than GC.

The main argument to improve GC is robustness, not typical-case
performance. We're still installing last-minute hacks into Emacs 26
because of real-world problems here. It would be useful if we didn't
have to mess with these hacks because we fixed GC to not blow the C stack.


Reply | Threaded
Open this post in threaded view
|

Re: What improvements would be truly useful?

Rostislav Svoboda
In reply to this post by Richard Stallman
> It would be a big step towards making Emacs do the job of
> a word processor, which is what I would like to see some day.
> Imagine if you could edit nicely formatted documents directly
> with Emacs, instead of using LibreOffice?  LibreOffice is
> fine to use, it is free software, but it isn't Emacs.

IMO the future of GUI apps and in our case editors, lies in browser
based frameworks programmable by some lisp dialect. For the moment it
would be a stack based on electron, clojurescript, codemirror, etc.

I know it's a bitter pill to swallow, but let's face it - do we think,
our bellowed Emacs will ever be able to display anything like the
examples from https://threejs.org ?

Bost

Reply | Threaded
Open this post in threaded view
|

Re: Variable-width font indentation (was: What improvements would be truly useful?)

Eli Zaretskii
In reply to this post by Richard Stallman
> From: Richard Stallman <[hidden email]>
> Date: Mon, 05 Mar 2018 08:11:38 -0500
> Cc: [hidden email], [hidden email], [hidden email]
>
> Changing Emacs to handle indentation and alignment with
> variable-width fonts would be an important and useful change.
> Certain kinds of use would make sense, which currently don't.

This has come up several times, and I think I asked at least once what
does it need to entail.  It would be helpful to have the answer to
that, which will have to be the result of looking at the relevant Lisp
code and figuring out which primitives/subroutines will need to be
taught fractional column values.

Emacs already knows how to align text at pixel resolution (which is
easy to express in fractions of the width of the default face's font),
and in fact already does so when it displays TAB characters on GUI
frames.  We have the 'space' display property to do the same with
characters other than a TAB.

So what else is needed?  Can someone who is familiar with indent.el
please tell?

Reply | Threaded
Open this post in threaded view
|

Re: What improvements would be truly useful?

Eli Zaretskii
In reply to this post by Rostislav Svoboda
> From: Rostislav Svoboda <[hidden email]>
> Date: Mon, 5 Mar 2018 18:32:06 +0100
> Cc: "[hidden email] Development" <[hidden email]>,
> Ken Raeburn <[hidden email]>, [hidden email],
> Paul Eggert <[hidden email]>
>
> I know it's a bitter pill to swallow, but let's face it - do we think,
> our bellowed Emacs will ever be able to display anything like the
> examples from https://threejs.org ?

Can you display something similar in Office?

Reply | Threaded
Open this post in threaded view
|

Re: What improvements would be truly useful?

Daniele Nicolodi
In reply to this post by Richard Stallman
On 3/5/18 6:11 AM, Richard Stallman wrote:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
> An improvement in GC wouldn't be a bad thing, but it may not be worth
> the effort.  It is likely to lead to many bugs that would be hard to
> fix.  Once working, it would not make much difference to users.
> It would only permit some operations on larger problems than now.
>
> When I was working at the AI Lab, one of the older programmers told me
> that hackers are often eager to make improvements of this sort: which
> make the program better in an abstract sense, but not better for
> users.  I took that advice to heart.  Now I pass it on.
>
> Changing Emacs to handle indentation and alignment with
> variable-width fonts would be an important and useful change.
> Certain kinds of use would make sense, which currently don't.
>
> It would be a big step towards making Emacs do the job of
> a word processor, which is what I would like to see some day.
> Imagine if you could edit nicely formatted documents directly
> with Emacs, instead of using LibreOffice?  LibreOffice is
> fine to use, it is free software, but it isn't Emacs.

Richard,

with all the due respect for your role in starting the Emacs project and
the Free software movement, I don't think that dismissing contributions
to Emacs on the base of their relative usefulness to implementing new
features that you think are important, does any good to Emacs.

Hackers work on Emacs because they like it, and scratch their own itch.
I don't think that anyone is in the position to suggest how they should
be spending their time. Even more so if (as far as I know) there is no
agreed upon roadmap for the project.

I think that what will help to keep Emacs relevant in the long run is
modernizing its structure and code base, moving away from the baroque
architecture that developed as a result of its very long history (and
some less than optimally future proof design decisions).

Personally, I find that better support for display with variable width
fonts will not improve my use of Emacs. What would improve my user
experience would be improvements to the display engine that will allow a
better document viewing capabilities in something like pdf-tools (like
continuous scrolling).

The only effective (and not detrimental to the project) way to steer
hacker attentions to features you find important is to pay them for
their work. With your popularity it should not be impossible to rise
enough money to do so.

Cheers,
Dan

Reply | Threaded
Open this post in threaded view
|

Re: What improvements would be truly useful?

Aaron Ecay
In reply to this post by Rostislav Svoboda
2018ko martxoak 5an, Rostislav Svoboda-ek idatzi zuen:
>

[...]

> I know it's a bitter pill to swallow, but let's face it - do we think,
> our bellowed Emacs will ever be able to display anything like the
> examples from https://threejs.org ?

Thanks to the xwidgets support (info "(elisp) Xwidgets"), it is possible
to embed a graphical web browser with full JS support in an emacs buffer.
So itʼs possible to have the best of both worlds: emacsʼs support for
text editing combined with the visualization and interactivity of modern
Javascript libraries.

Iʼm sure that there are ways that graphical programming (for lack of
a better term) in emacs could be made better, including Richardʼs
suggestions that launched this thread.  But we should be proud of
what we already have, rather than pessimistic.

Aaron

PS I donʼt know of any projects using three.js support in xwidgets
specifically; it could turn out that it doesnʼt quite work
(e.g. hardware-accelerated 3D or lack thereof might present an
obstacle).  But anyone can try it and find out, and hopefully if
problems arise figure out how to fix them.

--
Aaron Ecay

Reply | Threaded
Open this post in threaded view
|

Re: What improvements would be truly useful?

Daniel Colascione-5
In reply to this post by Eli Zaretskii
On 03/05/2018 10:04 AM, Eli Zaretskii wrote:
>> From: Rostislav Svoboda <[hidden email]>
>> Date: Mon, 5 Mar 2018 18:32:06 +0100
>> Cc: "[hidden email] Development" <[hidden email]>,
>> Ken Raeburn <[hidden email]>, [hidden email],
>> Paul Eggert <[hidden email]>
>>
>> I know it's a bitter pill to swallow, but let's face it - do we think,
>> our bellowed Emacs will ever be able to display anything like the
>> examples from https://threejs.org ?

It can already, with XEmbed --- same way Office does it, with OLE. :-)
It's a nice trick, but I don't think that sort of pretty display helps
anyone accomplish a task.

Personally, I don't think word processing is a good focus for Emacs.
There are two groups of people who want to prepare documents: those who
want a WYSIWYG system and those who don't. The former group is
well-served by LibreOffice, which is a free and powerful office suite.
The latter group is well-served by Emacs with its extensive LaTeX
integration.

Instead of focusing on areas where we're weak and will realistically
never catch up with projects dedicated to the task, we should focus on
improving existing strengths.

1) We should be the best editor around for text and program code.
There's an opportunity to do much better than the mainstream.
Conventional IDE groups put a ton of brute force effort into tuning IDEs
for specific coding styles in specific environments. We can be more
generic and more flexible, ultimately offering more power and greater
efficiency for people willing to invest time into learning the system.

a) We should do a better job of integrating interesting ideas like
undo-tree, ace-jump-mode, yasnippet, helm, and others into the core and
enabling them by default. I don't think we need to be as conservative as
we've been historically, and I think there's still a lot of room to
improve the core editing mechanics.

b) There are long-standing defects that are minor in the scheme of
things, but that tend to create a poor impression. In particular,
long-line handling is a sore point, as is support for very large files.
For long lines: I haven't sufficiently studied what the necessary
redisplay hacks would look like.

For large files: by moving from a gap buffer to a rope representation
for buffers, we can partially use memory-mapped backing storage, and
even when we do need private, modifiable memory for editing, we can
allocate only when we immediately need and not have to move the gap
around through humongous amounts of main memory. Such a system would not
only improve our support for humongous files, but would also make a
32-bit Emacs capable of editing files larger than its address space.

c) We need a project system. There's been some good work in this area,
but there's too much fragmentation, which hinders productive
integration. For example, there's no default keybinding to jump, in C++,
between an "implementation" and a "header" file, and that's because
Emacs by default has no idea what either concept means and there are
something like, what, a dozen(?) different ways to teach it the concept.

d) We need better fontification and indentation. We don't have good
language coverage, and support for more obscure languages is sometimes
spotty, limited to fontifying comments, strings, and keywords. Keeping
up with language development is a constant struggle, and it's easy to
introduce odd bugs, infloops, and so on in ad-hoc parsing code,
especially when this code needs to be simultaneously fast, incremental,
and error tolerant.

I'm now wondering whether the manual approach is wrong. We've been using
it along with everyone else, but there might be better options these
days. It's a somewhat radical idea: let's use a machine learning model
to classify program tokens, then apply manual fontification and
indentation rules to the resulting token classifications. We'd train the
model by taking labeled program text (say, from Savannah or GitHub, run
through a parser), then perturb the program text, rewarding the model
for retaining token labels under various editing and truncation operations.

In this way, we'd learn an approximate model for understanding even
damaged program text without having to manually write a lot of code.
Tons and tons of stuff in cc-mode is heuristics for dealing with damaged
program text, and I think we could learn this understanding instead. The
system is equivalent in power to anything we could write by hand: LSTMs
and other systems are Turing-complete. This way, to add support for a
new language, you'd just feed Emacs examples. I imagine you might even
be able to gently correct the system when it misunderstands and improve
the overall accuracy.

But it's probably a crazy idea. :-)

2) Startup should be instant in all cases. Now that we have a portable
dumper, we should automatically dump the results of user initialization
and regenerate the dump when we detect that something's changed. This
way, users perceive Emacs as a fast, modern system. I know that the
daemon exists and that it's possible to optimize even a customized
initialization so that it's fast even without hacks (I do), but users
shouldn't have to go to the trouble of this kind of manual setup and
tweaking

3) Mobile support. One of Emacs' strengths is its portability, and this
portability comes with a much lower footprint than other approaches.
Desktop and laptop sales have been declining for six years. There are
lots of tools built on Emacs that would be useful (like gnus, org-mode,
etc.), except that I can't use them on mobile, which means I end up
choosing other tools entirely for these tasks.

There is no reason that Emacs couldn't be a good Android citizen. A good
Java<->elisp bridge would let us transparently use various system APIs.
While we would probably need mobile-specific GUI code (because the plain
buffer interface wouldn't be suitable for most tasks, at least without
mobile-desktop convergence), all the logic and back-end glue would work
on mobile as well as it works anywhere else, greatly simplifying the
task of building general-purpose tools like org-mode that really ought
to work anywhere.

Reply | Threaded
Open this post in threaded view
|

Re: What improvements would be truly useful?

Eli Zaretskii
In reply to this post by Daniele Nicolodi
> From: Daniele Nicolodi <[hidden email]>
> Date: Mon, 5 Mar 2018 11:51:09 -0700
>
> I think that what will help to keep Emacs relevant in the long run is
> modernizing its structure and code base, moving away from the baroque
> architecture that developed as a result of its very long history (and
> some less than optimally future proof design decisions).

What exactly are you talking about here?  AFAIK, the Emacs
architecture didn't change since its inception, so its long history
has no relevance here.  But maybe I'm missing something or
misunderstanding what you intended to convey.

> What would improve my user experience would be improvements to the
> display engine that will allow a better document viewing
> capabilities in something like pdf-tools (like continuous
> scrolling).

AFAIU, that's already possible from the display engine POV, what's
missing is application-level Lisp code that would take advantage of
the infrastructure.

Reply | Threaded
Open this post in threaded view
|

Re: Variable-width font indentation (was: What improvements would be truly useful?)

Daniel Colascione-5
In reply to this post by Eli Zaretskii
>> From: Richard Stallman <[hidden email]>
>> Date: Mon, 05 Mar 2018 08:11:38 -0500
>> Cc: [hidden email], [hidden email], [hidden email]
>>
>> Changing Emacs to handle indentation and alignment with
>> variable-width fonts would be an important and useful change.
>> Certain kinds of use would make sense, which currently don't.
>
> This has come up several times, and I think I asked at least once what
> does it need to entail.  It would be helpful to have the answer to
> that, which will have to be the result of looking at the relevant Lisp
> code and figuring out which primitives/subroutines will need to be
> taught fractional column values.
>
> Emacs already knows how to align text at pixel resolution (which is
> easy to express in fractions of the width of the default face's font),
> and in fact already does so when it displays TAB characters on GUI
> frames.  We have the 'space' display property to do the same with
> characters other than a TAB.
>
> So what else is needed?  Can someone who is familiar with indent.el
> please tell?

Well, say you want to align the following code:

1: void foo(int a,
2:          int b)
3: {
4:  ...

How is any generic indentation system supposed to know that the "i" on
line 2 is supposed to line up with the "i" on line 1? You'd need to
communicate all indentation intent to the generic system in the form of
positions relative to some anchor. (Presumably, it could cache pixel
offsets per-line.) While that's doable, it's a ton of work, and I don't
think it's worthwhile.


Reply | Threaded
Open this post in threaded view
|

Re: Variable-width font indentation

Paul Eggert
On 03/05/2018 11:32 AM, [hidden email] wrote:
> 1: void foo(int a,
> 2:          int b)
> 3: {
> 4:  ...
>
> How is any generic indentation system supposed to know that the "i" on
> line 2 is supposed to line up with the "i" on line 1?

It assumes the input is fixed-width, and that leading white space is
intended to indent with respect to previous lines. Although it'd be some
work to get Emacs to do this, I don't offhand see any algorithmic
problem here.


Reply | Threaded
Open this post in threaded view
|

Re: Variable-width font indentation (was: What improvements would be truly useful?)

Eli Zaretskii
In reply to this post by Daniel Colascione-5
> Date: Mon, 5 Mar 2018 11:32:30 -0800
> From: [hidden email]
> Cc: [hidden email],
>  [hidden email],
>  [hidden email],
>  [hidden email],
>  [hidden email]
>
> Well, say you want to align the following code:
>
> 1: void foo(int a,
> 2:          int b)
> 3: {
> 4:  ...
>
> How is any generic indentation system supposed to know that the "i" on
> line 2 is supposed to line up with the "i" on line 1?

How does it know that today, when we only support well the fixed-pitch
fonts?  Or are you talking about a different problem?

> You'd need to communicate all indentation intent to the generic
> system in the form of positions relative to some
> anchor. (Presumably, it could cache pixel offsets per-line.) While
> that's doable, it's a ton of work, and I don't think it's
> worthwhile.

Maybe I still misunderstand you, but the pixel position of a character
on display is trivial to obtain, and I see no reason why we'd need to
cache them.  Again, the current fixed-pitch column-wise indentation
machinery doesn't, AFAIK.

1234 ... 10