Language Servers and Emacs

classic Classic list List threaded Threaded
26 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Language Servers and Emacs

Perry E. Metzger
Microsoft has invented an interesting new protocol for IDEs
and code editors to talk to language tools (like compilers) to do
things like smart autocompletion, jumping to code definitions, and the
like.

The idea is that compilers and similar tools generally know a lot
about the structure of a language and can provide help to an editor as
an external service, so the editor doesn't need to handle parsing,
symbol tables, etc. for the language on its own. The editor can just
ask the compiler or language tool for the information when it needs
it.

The protocol is open and they have no intellectual property claims on
it. It is JSON based and should be straightforward to support.

A general description is at:

http://langserver.org/

And detailed protocol information is at:

https://github.com/Microsoft/language-server-protocol

It was originally designed for their "Roslyn" C# compiler (which is
free software, it's Apache licensed) to integrate with their own
editors and IDEs, but other compiler projects and editors seem to be
adopting the thing. There are now language servers people have built
for a lot of languages, and there's support appearing for the
protocol in a bunch of editors.

I'm mentioning this here because I think the Emacs community would be
interested in this capability, though I suspect that it would also be
really neat if GCC developed a language server for C and C++.

I recall that a while ago, RMS had a lot of problems with the idea
of opening up GCC in ways that might have made it possible for people
to use it too much for proprietary compilers. Using the Language
Server Protocol, it should be possible for GCC and other free
language tools to talk to a variety of IDEs and Editors (including, I
hope, Emacs) to allow advanced modern code editing _without_ the
sorts of risks he was worried about.

Perry
--
Perry E. Metzger [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Language Servers and Emacs

Philippe Vaucher
I'm mentioning this here because I think the Emacs community would be
interested in this capability, though I suspect that it would also be
really neat if GCC developed a language server for C and C++.

Apparently someone started working on this outside of emacs-dev: https://github.com/sourcegraph/emacs-lsp

Philippe
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Language Servers and Emacs

Perry E. Metzger
On Tue, 11 Apr 2017 18:36:07 +0200 Philippe Vaucher
<[hidden email]> wrote:
> >
> > I'm mentioning this here because I think the Emacs community
> > would be interested in this capability, though I suspect that it
> > would also be really neat if GCC developed a language server for
> > C and C++.
>
> Apparently someone started working on this outside of emacs-dev:
> https://github.com/sourcegraph/emacs-lsp

Indeed, though it seems fairly simple compared to what is possible.
It should be feasible, for example, to do things like having a
language mode highlight errors (not just syntax errors but type
errors and the like) in real time, provide easy renaming of
identifiers across whole projects, provide smart completion, etc.

I'm also imagining features where you can request documentation for
any function or variable and it will pop up a formatted version of
the documentation comment, just like you can do for elisp today.

Especially if GCC and other Gnu tools gained Language Server
capabilities, this could provide a substantial win for programmers
using Emacs.

Perry
--
Perry E. Metzger [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Language Servers and Emacs

Eli Zaretskii
In reply to this post by Perry E. Metzger
> Date: Tue, 11 Apr 2017 12:28:16 -0400
> From: "Perry E. Metzger" <[hidden email]>
>
> I'm mentioning this here because I think the Emacs community would be
> interested in this capability, though I suspect that it would also be
> really neat if GCC developed a language server for C and C++.

https://gcc.gnu.org/ml/gcc/2017-03/msg00105.html

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Language Servers and Emacs

Evgeniy Dushistov
In reply to this post by Philippe Vaucher
On Tue, Apr 11, 2017 at 06:36:07PM +0200, Philippe Vaucher wrote:
>     I'm mentioning this here because I think the Emacs community would be
>     interested in this capability, though I suspect that it would also be
>     really neat if GCC developed a language server for C and C++.
>
>
> Apparently someone started working on this outside of emacs-dev: https://
> github.com/sourcegraph/emacs-lsp

I think this fork or just project with the same name is more polular:

https://github.com/vibhavp/emacs-lsp

--
/Evgeniy

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Language Servers and Emacs

Helmut Eller-2
In reply to this post by Perry E. Metzger
On Tue, Apr 11 2017, Perry E. Metzger wrote:

> I'm mentioning this here because I think the Emacs community would be
> interested in this capability, though I suspect that it would also be
> really neat if GCC developed a language server for C and C++.

The client side of the protocol seems fairly easy to implement in Emacs,
so I think this is primarily a (big) task for GCC developers.  We could
implement a server for Elisp on top of bytecomp.el, but I doubt that
anybody has time/motivation for that.

I note that some Clang developers are working on a server:
http://lists.llvm.org/pipermail/cfe-dev/2017-January/052458.html

So it seems likely that Clang will support the protocol long before GCC
does.  It might also have political implications if Emacs implements the
client side if Clang is the only server for C/C++.

Helmut


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Language Servers and Emacs

Vibhav Pant-2
In reply to this post by Perry E. Metzger
I have been working on an emacs implementation for the past few
months: https://github.com/vibhavp/emacs-lsp
It integrates with eldoc, completion-at-point, xref and flycheck.
Coupling it with various CEDET components (ECB comes to mind) is
planned for the future.

At some point, I'd like to get this into ELPA.

Regards,
Vibhav

On Tue, Apr 11, 2017 at 9:58 PM, Perry E. Metzger <[hidden email]> wrote:

> Microsoft has invented an interesting new protocol for IDEs
> and code editors to talk to language tools (like compilers) to do
> things like smart autocompletion, jumping to code definitions, and the
> like.
>
> The idea is that compilers and similar tools generally know a lot
> about the structure of a language and can provide help to an editor as
> an external service, so the editor doesn't need to handle parsing,
> symbol tables, etc. for the language on its own. The editor can just
> ask the compiler or language tool for the information when it needs
> it.
>
> The protocol is open and they have no intellectual property claims on
> it. It is JSON based and should be straightforward to support.
>
> A general description is at:
>
> http://langserver.org/
>
> And detailed protocol information is at:
>
> https://github.com/Microsoft/language-server-protocol
>
> It was originally designed for their "Roslyn" C# compiler (which is
> free software, it's Apache licensed) to integrate with their own
> editors and IDEs, but other compiler projects and editors seem to be
> adopting the thing. There are now language servers people have built
> for a lot of languages, and there's support appearing for the
> protocol in a bunch of editors.
>
> I'm mentioning this here because I think the Emacs community would be
> interested in this capability, though I suspect that it would also be
> really neat if GCC developed a language server for C and C++.
>
> I recall that a while ago, RMS had a lot of problems with the idea
> of opening up GCC in ways that might have made it possible for people
> to use it too much for proprietary compilers. Using the Language
> Server Protocol, it should be possible for GCC and other free
> language tools to talk to a variety of IDEs and Editors (including, I
> hope, Emacs) to allow advanced modern code editing _without_ the
> sorts of risks he was worried about.
>
> Perry
> --
> Perry E. Metzger                [hidden email]
>



--
Vibhav Pant
[hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Language Servers and Emacs

Perry E. Metzger
In reply to this post by Helmut Eller-2
On Wed, 12 Apr 2017 09:39:48 +0200 Helmut Eller
<[hidden email]> wrote:

> On Tue, Apr 11 2017, Perry E. Metzger wrote:
>
> > I'm mentioning this here because I think the Emacs community
> > would be interested in this capability, though I suspect that it
> > would also be really neat if GCC developed a language server for
> > C and C++.  
>
> The client side of the protocol seems fairly easy to implement in
> Emacs, so I think this is primarily a (big) task for GCC
> developers.  We could implement a server for Elisp on top of
> bytecomp.el, but I doubt that anybody has time/motivation for that.
>
> I note that some Clang developers are working on a server:
> http://lists.llvm.org/pipermail/cfe-dev/2017-January/052458.html
>
> So it seems likely that Clang will support the protocol long before
> GCC does.  It might also have political implications if Emacs
> implements the client side if Clang is the only server for C/C++.

There are servers for many other languages too, this is not a C/C++
only issue. (And I think GCC needs to gain this capability in any
case, it appears that it is being rapidly adopted. There are now
langserv implementations for dozens of languages -- Python, Java, PHP,
Go, Rust, Swift, JavaScript, C#, Julia, Scala, and the list goes on,
plus there are a bunch of IDEs adopting the protocol too, and support
for the vim and Atom editors are coming.)

Architecturally, this is certainly the right thing. One would
generally far prefer to have the language implementation help the
editor with parsing than have to re-implement parsing inside the
editor.

--
Perry E. Metzger [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Language Servers and Emacs

Perry E. Metzger
In reply to this post by Vibhav Pant-2
On Wed, 12 Apr 2017 14:33:07 +0530 Vibhav Pant <[hidden email]>
wrote:
> I have been working on an emacs implementation for the past few
> months: https://github.com/vibhavp/emacs-lsp
> It integrates with eldoc, completion-at-point, xref and flycheck.
> Coupling it with various CEDET components (ECB comes to mind) is
> planned for the future.
>
> At some point, I'd like to get this into ELPA.

That looks very cool! However, your warning on the web site
makes it sound like it is not ready to be used. (I.e. you say "This
package is still under development, and is not recommended for daily
use.") Is that just an abundance of caution?

And how hard is it to add support for other clients/languages besides
Go and Rust? (Some instructions in the repository might make it
easier for people to help bringing this up to speed...)

Perry
--
Perry E. Metzger [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Language Servers and Emacs

Vibhav Pant-2
On Wed, Apr 12, 2017 at 6:37 PM, Perry E. Metzger <[hidden email]> wrote:
> That looks very cool! However, your warning on the web site
> makes it sound like it is not ready to be used. (I.e. you say "This
> package is still under development, and is not recommended for daily
> use.") Is that just an abundance of caution?

In some ways, yes. lsp-mode still has issues with *some* language server
(the Python one is the most mature one AFAIK, and works just fine).

> And how hard is it to add support for other clients/languages besides
> Go and Rust? (Some instructions in the repository might make it
> easier for people to help bringing this up to speed...)
>

I've updated the list of supported languages on the README. Adding
supporting for other languages is rather easy, you just need to tell
lsp-mode about how the language server connects and communicates
with clients. The remaining is handled by the package itself. For instance,
this is how Python is supported:
https://github.com/vibhavp/emacs-lsp/blob/c7c6a477b0c8aff1ce8aa032992cd10085ee0143/lsp-mode.el#L92

--
Vibhav Pant
[hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Language Servers and Emacs

Richard Stallman
In reply to this post by Perry E. Metzger
[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

This sounds interesting.  I will look at what it does.

Thanks.

--
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Language Servers and Emacs

Lele Gaifax
In reply to this post by Vibhav Pant-2
Vibhav Pant <[hidden email]> writes:

> On Wed, Apr 12, 2017 at 6:37 PM, Perry E. Metzger <[hidden email]> wrote:
>> That looks very cool! However, your warning on the web site
>> makes it sound like it is not ready to be used. (I.e. you say "This
>> package is still under development, and is not recommended for daily
>> use.") Is that just an abundance of caution?
>
> In some ways, yes. lsp-mode still has issues with *some* language server
> (the Python one is the most mature one AFAIK, and works just fine).

Indeed it is very nice: I gave it a shot on a Python project, and will surely
try it again.

One thing that surprised me is that it asks about launching the underlying
"pyls" tool once for each visited source: is that right? As my project has
hundreds of Python modules, wouldn't that put Emacs under pressure?

One minor observation on the lisp sources: I think it's not necessary to
explicitly quote lambdas as in the following snippet

  (lsp-define-client 'python-mode "python" 'stdio #'(lambda () default-directory)
    :command '("pyls")
    :name "Python Language Server")

Thank you,
ciao, lele.
--
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
[hidden email]  |                 -- Fortunato Depero, 1929.


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Language Servers and Emacs

Philipp Stephani
In reply to this post by Perry E. Metzger


Perry E. Metzger <[hidden email]> schrieb am Di., 11. Apr. 2017 um 18:28 Uhr:
Microsoft has invented an interesting new protocol for IDEs
and code editors to talk to language tools (like compilers) to do
things like smart autocompletion, jumping to code definitions, and the
like.

The idea is that compilers and similar tools generally know a lot
about the structure of a language and can provide help to an editor as
an external service, so the editor doesn't need to handle parsing,
symbol tables, etc. for the language on its own. The editor can just
ask the compiler or language tool for the information when it needs
it.

The protocol is open and they have no intellectual property claims on
it. It is JSON based and should be straightforward to support.

A general description is at:

http://langserver.org/

And detailed protocol information is at:

https://github.com/Microsoft/language-server-protocol

It was originally designed for their "Roslyn" C# compiler (which is
free software, it's Apache licensed) to integrate with their own
editors and IDEs, but other compiler projects and editors seem to be
adopting the thing. There are now language servers people have built
for a lot of languages, and there's support appearing for the
protocol in a bunch of editors.

I'm mentioning this here because I think the Emacs community would be
interested in this capability, though I suspect that it would also be
really neat if GCC developed a language server for C and C++.

I recall that a while ago, RMS had a lot of problems with the idea
of opening up GCC in ways that might have made it possible for people
to use it too much for proprietary compilers. Using the Language
Server Protocol, it should be possible for GCC and other free
language tools to talk to a variety of IDEs and Editors (including, I
hope, Emacs) to allow advanced modern code editing _without_ the
sorts of risks he was worried about.

I agree and think that language servers are very promising and we should have strong support for the protocol in Emacs (preferably even in core). They neatly solve the problem of having to implement support for each and every language in Elisp. 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Language Servers and Emacs

Richard Stallman
[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I am trying to interest the GCC maintainers in supporting this
protocol.

--
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Language Servers and Emacs

Tom Tromey-4
In reply to this post by Philipp Stephani
Philipp> I agree and think that language servers are very promising and
Philipp> we should have strong support for the protocol in Emacs
Philipp> (preferably even in core).

I too would like to see this support in core.
Up-thread, Vibhav expressed interest in putting it into ELPA.
That would be reasonable as well, though core would be better, IMO.

What are the blockers to doing this?

Tom

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Language Servers and Emacs

Perry E. Metzger
In reply to this post by Richard Stallman
On Thu, 20 Apr 2017 22:08:29 -0400 Richard Stallman <[hidden email]>
wrote:
> [[[ To any NSA and FBI agents reading my email: please consider
> ]]] [[[ whether defending the US Constitution against all
> enemies,     ]]] [[[ foreign or domestic, requires you to follow
> Snowden's example. ]]]
>
> I am trying to interest the GCC maintainers in supporting this
> protocol.

That is good to hear. Please let us know how the discussion goes.

Perry
--
Perry E. Metzger [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Language Servers and Emacs

Phillip Lord-3
In reply to this post by Perry E. Metzger
"Perry E. Metzger" <[hidden email]> writes:

>> So it seems likely that Clang will support the protocol long before
>> GCC does.  It might also have political implications if Emacs
>> implements the client side if Clang is the only server for C/C++.
>
> There are servers for many other languages too, this is not a C/C++
> only issue. (And I think GCC needs to gain this capability in any
> case, it appears that it is being rapidly adopted. There are now
> langserv implementations for dozens of languages -- Python, Java, PHP,
> Go, Rust, Swift, JavaScript, C#, Julia, Scala, and the list goes on,
> plus there are a bunch of IDEs adopting the protocol too, and support
> for the vim and Atom editors are coming.)
>
> Architecturally, this is certainly the right thing. One would
> generally far prefer to have the language implementation help the
> editor with parsing than have to re-implement parsing inside the
> editor.


Apologies for the zombie posting -- easter holidays and all.

It would be interesting to do a comparison with "nrepl" which is the
protocol that CIDER, the emacs package for clojure, uses. This is pretty
feature complete now and it would give an good side-by-side
comparison. I think that ENSIME (for Scala) has a protocol in there as
well.

I agree with you, having a standard to use is a very good thing.

Phil

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Language Servers and Emacs

Katherine Cox-Buday
> I think that ENSIME (for Scala) has a protocol in there as
> well.

They do, and there is an open discussion[1] on supporting the LSP
protocol.

Personally, I'd like to see broad support for this in emacs so I can
learn the "emacs way" of interacting with code that works across various
languages with different backends. It seems like every language's
mode/backend does things slightly differently.

--
Katherine

[1] - https://github.com/ensime/ensime-server/issues/1498

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Language Servers and Emacs

Perry E. Metzger
On Tue, 25 Apr 2017 18:06:40 -0500 Katherine Cox-Buday
<[hidden email]> wrote:

> > I think that ENSIME (for Scala) has a protocol in there as
> > well.  
>
> They do, and there is an open discussion[1] on supporting the LSP
> protocol.
>
> Personally, I'd like to see broad support for this in emacs so I can
> learn the "emacs way" of interacting with code that works across
> various languages with different backends. It seems like every
> language's mode/backend does things slightly differently.

Indeed.

I suspect that most of the ways of interacting with external language
tools other than LSP will fade over time, but whether or not that's
the case, it clearly seems to be smarter to have an external language
tool than to have to duplicate the work of parsing and managing
symbol tables inside the editor.

Perry
--
Perry E. Metzger [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Language Servers and Emacs

Phillip Lord-3
"Perry E. Metzger" <[hidden email]> writes:

> On Tue, 25 Apr 2017 18:06:40 -0500 Katherine Cox-Buday
> <[hidden email]> wrote:
>> > I think that ENSIME (for Scala) has a protocol in there as
>> > well.  
>>
>> They do, and there is an open discussion[1] on supporting the LSP
>> protocol.
>>
>> Personally, I'd like to see broad support for this in emacs so I can
>> learn the "emacs way" of interacting with code that works across
>> various languages with different backends. It seems like every
>> language's mode/backend does things slightly differently.
>
> Indeed.
>
> I suspect that most of the ways of interacting with external language
> tools other than LSP will fade over time, but whether or not that's
> the case, it clearly seems to be smarter to have an external language
> tool than to have to duplicate the work of parsing and managing
> symbol tables inside the editor.

Well, this depends whether LSP is good or not. Even then, history
suggests that introduction of the one true standard, just results in yet
another technique for the same thing.

A brief look at LSP seems that it's relatively limited in scope,
though; it has some discussion of "capabilities" and "execute
command". How extensible is it? For instance, NREPL allows evalling
code, or loading files? Would it be possible to send enough data through
LSP to support edebug step through debugging for instance?

Phil

12
Loading...