bug#42028: [Feature Request] 27.0.91; Provide the ability dynamic modules to post events in emacs event loop

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

bug#42028: [Feature Request] 27.0.91; Provide the ability dynamic modules to post events in emacs event loop

Ivan Yonchovski


This is needed for the cases when the module is listening for extenal
events and wants to call back emacs to process them. ATM this is kind of
possible by using signals on linux and using WM_INPUTLANGCHANGE as
described in https://nullprogram.com/blog/2017/02/14/ but this looks
more a hack for a missing feature.






Reply | Threaded
Open this post in threaded view
|

bug#42028: [Feature Request] 27.0.91; Provide the ability dynamic modules to post events in emacs event loop

Eli Zaretskii
> From: Ivan Yonchovski <[hidden email]>
> Date: Wed, 24 Jun 2020 10:14:44 +0300
>
> This is needed for the cases when the module is listening for extenal
> events and wants to call back emacs to process them. ATM this is kind of
> possible by using signals on linux and using WM_INPUTLANGCHANGE as
> described in https://nullprogram.com/blog/2017/02/14/ but this looks
> more a hack for a missing feature.

It is almost trivial to let modules insert events into the event
queue.  The problem is that I expect a module that inserts such events
to want to be called to process those events as well.  Is that
expectation correct?  If it is, then we need to think about extending
the mechanism that calls event handlers, not just about exposing the
likes of kbd_buffer_store_event to modules.  We should also consider
how a module could define its own events.

IOW, this calls for a slightly more detailed specification, before
someone could sit down and code the stuff.  I suggest to take a look
at how existing events are handled, and propose such a detailed
specification.

Thanks.



Reply | Threaded
Open this post in threaded view
|

bug#42028: [Feature Request] 27.0.91; Provide the ability dynamic modules to post events in emacs event loop

Ivan Yonchovski

The problem is that I expect a module that inserts such events
to want to be called to process those events as well.  Is that
expectation correct?  

Yes. As a side note, this feature will be useful even for the core C part for anything
that wants to perform async processing not including LispObject. 

Thanks,
Ivan
Reply | Threaded
Open this post in threaded view
|

bug#42028: [Feature Request] 27.0.91; Provide the ability dynamic modules to post events in emacs event loop

Eli Zaretskii
> From: yyoncho <[hidden email]>
> Date: Wed, 24 Jun 2020 19:17:30 +0300
> Cc: [hidden email]
>
> As a side note, this feature will be useful even for the core C part for anything
> that wants to perform async processing not including LispObject.

How will it help that?  The event queue is processed synchronously.



Reply | Threaded
Open this post in threaded view
|

bug#42028: [Feature Request] 27.0.91; Provide the ability dynamic modules to post events in emacs event loop

Ivan Yonchovski
Here it is an example: If we oversimplify json parsing consist of 2 parts.

1. JSON parsing to Jansson data structures
2. Converting Jasson data structures to lisp data structures.

If we assume that 1 is taking most of the time then we could write the following code:

start(string, callback) -> convert string to char* -> post it to worker thread -> worker thread does the parsing 
-> worker thread post to main loop -> main loop converts Jansson ds to lisp ds -> main loop
calls the callback passed on start

The same strategy probably could be applied to Font locking with TreeSitter. 

Thanks,
Ivan

On Wed, Jun 24, 2020 at 7:29 PM Eli Zaretskii <[hidden email]> wrote:
> From: yyoncho <[hidden email]>
> Date: Wed, 24 Jun 2020 19:17:30 +0300
> Cc: [hidden email]
>
> As a side note, this feature will be useful even for the core C part for anything
> that wants to perform async processing not including LispObject.

How will it help that?  The event queue is processed synchronously.
Reply | Threaded
Open this post in threaded view
|

bug#42028: [Feature Request] 27.0.91; Provide the ability dynamic modules to post events in emacs event loop

Eli Zaretskii
> From: yyoncho <[hidden email]>
> Date: Wed, 24 Jun 2020 19:46:56 +0300
> Cc: [hidden email]
>
> 1. JSON parsing to Jansson data structures
> 2. Converting Jasson data structures to lisp data structures.
>
> If we assume that 1 is taking most of the time

Does it?  Did someone time these separately?

> start(string, callback) -> convert string to char* -> post it to worker thread -> worker thread does the parsing
> -> worker thread post to main loop -> main loop converts Jansson ds to lisp ds -> main loop
> calls the callback passed on start

You cannot easily post to the queue from a worker thread, because the
queue cannot be posted to asynchronously.

And I lack the larger picture: how would this work within a framework
of some Emacs feature?  E.g., how will this processed be triggered,
and by what kind of trigger?

> The same strategy probably could be applied to Font locking with TreeSitter.

Again, I don't see the wider picture.  Are you familiar with how the
current JIT font-lock works?  If so, can you explain which parts of
that will be replaced/modified, and how?



Reply | Threaded
Open this post in threaded view
|

bug#42028: [Feature Request] 27.0.91; Provide the ability dynamic modules to post events in emacs event loop

Ivan Yonchovski

Eli Zaretskii writes:

>
> Your feature just asked for a way to queue events, it didn't say
> anything about doing that asynchronously, not from a function invoked
> by the main thread.  If you mean the latter, then it AFAIU would need
> a serious redesign of the Emacs event queue, to make it accessible
> from several threads running in parallel at once.

Yes, sorry about that. I didn't mention it explicitly in the bug report
it was described in more details in the linked article.

> What you describe is very different from how JIT font-lock works now.
> Which is why I asked the question: I know (more or less) how it works
> in other editors, I just don't yet understand well enough how
> something like that would fit into the existing fontification
> framework.  I hope there is a way of fitting it, because otherwise it
> would mean a serious surgery of the display engine as well, which will
> make the job significantly larger and harder.

IMO we are good here. At least on lsp-mode side we were able to
implement delayed semantic fontification as a result of async source.
But in lsp-mode case we are reseiving the messages from the server on
the UI thread.

Thanks,
Ivan