docstrings and elisp reference

classic Classic list List threaded Threaded
72 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

docstrings and elisp reference

Jean-Christophe Helary
I was thinking (naively?) that having the docstrings on one side and the reference on the other side was not a very efficient way to maintain documentation. Is everything in the documentation actually written by hand based on the docstrings? Wouldn't it be nicer to have good docstrings and use them directly in the documentation to avoid duplication of work? Is there any reason I'm not seeing why this is not happening?

Jean-Christophe
Reply | Threaded
Open this post in threaded view
|

Re: docstrings and elisp reference

Tino Calancha-2


On Tue, 6 Jun 2017, Jean-Christophe Helary wrote:

> I was thinking (naively?) that having the docstrings on one side and the reference on the other side was not a very efficient way to maintain documentation. Is everything in the documentation actually written by hand based on the docstrings? Wouldn't it be nicer to have good docstrings and use them directly in the documentation to avoid duplication of work? Is there any reason I'm not seeing why this is not happening?
Hi,

you might wish to read the following thread:
https://lists.gnu.org/archive/html/emacs-devel/2016-04/msg00452.html

Reply | Threaded
Open this post in threaded view
|

Re: docstrings and elisp reference

Jean-Christophe Helary

> 2017/06/06 9:09、Tino Calancha <[hidden email]>のメール:
>
>> I was thinking (naively?) that having the docstrings on one side and the reference on the other side was not a very efficient way to maintain documentation. Is everything in the documentation actually written by hand based on the docstrings? Wouldn't it be nicer to have good docstrings and use them directly in the documentation to avoid duplication of work? Is there any reason I'm not seeing why this is not happening?

> Hi,
>
> you might wish to read the following thread:
> https://lists.gnu.org/archive/html/emacs-devel/2016-04/msg00452.html

Wow, did I open a can of worms... :)

Fascinating thread.

Just a few comments.

Having a mechanism that generates automatic documentation (a la javadoc) would be extremely valuable. By just having that "reference" it would suggest developers to adopt a more standardized format/style.

The elisp reference should definitely extend the descriptions by adding exemples so that people learn from reading (which is admittedly difficult with the docstrings only). Maybe not go as far as a cookbook, but a bit more than what we have would be very welcome (there is a *huge* gap between the Introduction and the Reference).

My initial worry is that the day we start l10n (and we'll get there eventually :), we're going to have a huge amount of redundancy. But the flip side is that we're going to have also a huge amount of feedback on both the docstrings and the elisp reference...

Jean-Christophe
Reply | Threaded
Open this post in threaded view
|

Re: docstrings and elisp reference

Eli Zaretskii
> From: Jean-Christophe Helary <[hidden email]>
> Date: Tue, 6 Jun 2017 14:10:19 +0900
>
> Having a mechanism that generates automatic documentation (a la javadoc) would be extremely valuable.

It could be valuable, but IMO only if we exercise a lot of discipline
in producing those comments.  Because every project which uses this
methodology that I looked at ends up providing _abysmally_ inadequate
documentation.  As one very typical example, look at the GTK docs,
e.g.  here:

  https://developer.gnome.org/gtk4/stable/GtkWindow.html

Reply | Threaded
Open this post in threaded view
|

Re: docstrings and elisp reference

Stephen Leake-3
Eli Zaretskii <[hidden email]> writes:

>> From: Jean-Christophe Helary <[hidden email]>
>> Date: Tue, 6 Jun 2017 14:10:19 +0900
>>
>> Having a mechanism that generates automatic documentation (a la
>> javadoc) would be extremely valuable.
>
> It could be valuable, but IMO only if we exercise a lot of discipline
> in producing those comments.  Because every project which uses this
> methodology that I looked at ends up providing _abysmally_ inadequate
> documentation.

+1

Doc strings are for short reference information for single functions/variables.

texinfo docs are for more indepth explanations of several related functions/features.

They are complementary; they should not duplicate each other.

--
-- Stephe

Reply | Threaded
Open this post in threaded view
|

RE: docstrings and elisp reference

Drew Adams
> >> Having a mechanism that generates automatic documentation (a la
> >> javadoc) would be extremely valuable.
> >
> > It could be valuable, but IMO only if we exercise a lot of discipline
> > in producing those comments.  Because every project which uses this
> > methodology that I looked at ends up providing _abysmally_ inadequate
> > documentation.
>
> +1
>
> Doc strings are for short reference information for single
> functions/variables.
>
> texinfo docs are for more indepth explanations of several related
> functions/features.
>
> They are complementary; they should not duplicate each other.

Yes.

And this is not JavaDoc.  The Elisp manual is not purely
and simply an API reference manual.

And now this question is being rehashed again.  Everything
said this time around has already been said before.  Please
consult the previous thread, which was already cited.

(If you feel you really have something *new* to say, feel free.)

Reply | Threaded
Open this post in threaded view
|

Re: docstrings and elisp reference

Joost Kremers-2
In reply to this post by Stephen Leake-3

On Tue, Jun 06 2017, Stephen Leake wrote:
> Doc strings are for short reference information for single
> functions/variables.
>
> texinfo docs are for more indepth explanations of several
> related functions/features.
>
> They are complementary; they should not duplicate each other.

And yet, the Elisp manual documents a large number of functions
and variables / options in a format that is very reminiscent of
doc strings and which provides basically the same information. I
kinda see the point in making the doc string and that description
identical. Obviously, the manual gives a lot more background info
and groups the functions and variables together in a meaningful
way, but wouldn't it make sense / be possible to extract the
descriptions in the manual directly from the source?


--
Joost Kremers
Life has its moments

Reply | Threaded
Open this post in threaded view
|

Re: docstrings and elisp reference

Dmitry Gutov
In reply to this post by Stephen Leake-3
On 6/6/17 11:25 PM, Stephen Leake wrote:

> they should not duplicate each other.

Then you have something else in mind than the current situation with
documentation in Emacs.

Reply | Threaded
Open this post in threaded view
|

RE: docstrings and elisp reference

Drew Adams
> > they should not duplicate each other.
>
> Then you have something else in mind than the current
> situation with documentation in Emacs.

No, you do, if you think the intention now is that
they duplicate each other.  It never has been.

Some of the information is often the same; that's all.

The presentation and context of use are different.
The level of detail is often different.  (Sometimes
there is more detail in the manual; sometimes there
is more in a doc string.)

In particular, doc strings are written as user help.
The interactive use, if any, is typically described
first, and from the point of an interactive user.
Not so, the Elisp manual.

The doc string for a command tells a user what it
does and how a prefix arg changes the behavior.
In many cases the user need have no little or no
knowledge of Emacs Lisp.

The Elisp doc for a command tells a Lisp programmer
what the function and its parameters are, and
describes their behavior.  It might or might not
mention that one of the parameters corresponds to
the prefix arg when called interactively.

Different contexts; different purposes; not
necessarily the same readers or readers with the
same immediate goal.

Reply | Threaded
Open this post in threaded view
|

Re: docstrings and elisp reference

Dmitry Gutov
On 6/7/17 12:21 AM, Drew Adams wrote:
>>> they should not duplicate each other.
>>
>> Then you have something else in mind than the current
>> situation with documentation in Emacs.
>
> No, you do, if you think the intention now is that
> they duplicate each other.  It never has been.

Please look up the difference between "intention" and "current situation".

> Some of the information is often the same; that's all.

That's called "duplication".

> In particular, doc strings are written as user help.
> The interactive use, if any, is typically described
> first, and from the point of an interactive user.
> Not so, the Elisp manual.

We have different manuals, with different goals.

The fact that docstrings often describe the interactive case (when it
exists) doesn't make them necessarily targeted at the end user. Just
exhaustive.

Reply | Threaded
Open this post in threaded view
|

Re: docstrings and elisp reference

Richard Stallman
In reply to this post by Jean-Christophe Helary
[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I was thinking (naively?) that having the docstrings on one side
  > and the reference on the other side was not a very efficient way
  > to maintain documentation. Is everything in the documentation
  > actually written by hand based on the docstrings? Wouldn't it be
  > nicer to have good docstrings and use them directly in the
  > documentation to avoid duplication of work?

Absolutely not!  That's a recipe for a bad manual.  The text of a good
manual section is totally different from a series of doc strings.

Doc strings are read one by one.  Each one describes one function or
variable in isolation.  They are written specifically for this kind of
use.

By contrast, a section in a manual describes several related functions
and variables, and it should be written in an integrated way that
flows smoothly.  Meanwhile, common material should be stated only
once.

--
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.


Reply | Threaded
Open this post in threaded view
|

Re: docstrings and elisp reference

Dmitry Gutov
On 6/7/17 1:42 AM, Richard Stallman wrote:

> Meanwhile, common material should be stated only
> once.

Stated where?

Reply | Threaded
Open this post in threaded view
|

Re: docstrings and elisp reference

Eli Zaretskii
> From: Dmitry Gutov <[hidden email]>
> Date: Wed, 7 Jun 2017 01:49:18 +0300
> Cc: [hidden email]
>
> On 6/7/17 1:42 AM, Richard Stallman wrote:
>
> > Meanwhile, common material should be stated only
> > once.
>
> Stated where?

Somewhere in the vicinity, often in the same node, but sometimes in a
sibling or parent node which discuss the general attributes and
features of a family of related functions/features.  (If the common
material is far away from the place where it's relevant, there should
be a cross-reference to it, for those who read only about a specific
feature.)

Reply | Threaded
Open this post in threaded view
|

Re: docstrings and elisp reference

Eli Zaretskii
In reply to this post by Dmitry Gutov
> From: Dmitry Gutov <[hidden email]>
> Date: Wed, 7 Jun 2017 00:50:23 +0300
>
> On 6/7/17 12:21 AM, Drew Adams wrote:
> >>> they should not duplicate each other.
> >>
> >> Then you have something else in mind than the current
> >> situation with documentation in Emacs.
> >
> > No, you do, if you think the intention now is that
> > they duplicate each other.  It never has been.
>
> Please look up the difference between "intention" and "current situation".

There is a difference, that's true.  But the fact that reality is
different from the ideal doesn't mean we should give up the ideal, at
least not lightly.  And we certainly should consider whether the
alternative proposal will produce a far worse situation than what we
have today.  Once again, I suggest that you take a good look at
manuals of GnuTLS and Guile, they are produced using the methodology
that you propose.  That is our future if we go that way.  Do you
really like the result?

> > Some of the information is often the same; that's all.
>
> That's called "duplication".

You are overloading the meaning of "duplication".  The original
proposal was for a literal copying of the same text.  Drew was talking
about duplicating the information, but expressed in a different,
sometimes very different, form.

> > In particular, doc strings are written as user help.
> > The interactive use, if any, is typically described
> > first, and from the point of an interactive user.
> > Not so, the Elisp manual.
>
> We have different manuals, with different goals.
>
> The fact that docstrings often describe the interactive case (when it
> exists) doesn't make them necessarily targeted at the end user. Just
> exhaustive.

This doesn't really resolve the issue pointed out by Drew.  And that
is only one of the issues, there are others, also valid ones.

Reply | Threaded
Open this post in threaded view
|

Re: docstrings and elisp reference

Stefan Monnier
In reply to this post by Joost Kremers-2
> And yet, the Elisp manual documents a large number of functions and
> variables / options in a format that is very reminiscent of doc strings and
> which provides basically the same information.

Agreed.  And I also think this is a problem.
From where I stand, the way to fix it is:
- Better integration between the manual browser and the docstring
  browser, so you can click on a function/variable name in the manual to
  get to its docstring and you can easily jump from a docstring to the
  relevant section of the manual.
- Remove the redundant var/fun documentation from the manual (this is
  a tedious job: there is some redundance but it's not systematic, so
  we can't just do this removal systematically).
- Come up with a way to re-add those var/fun documentation into the
  printed manual (e.g. by adding to the Texinfo source external
  references to docstrings, which are then processed by now ad-hoc
  script).


        Stefan


Reply | Threaded
Open this post in threaded view
|

Re: docstrings and elisp reference

Yuri Khan-2
On Wed, Jun 7, 2017 at 8:03 PM, Stefan Monnier <[hidden email]> wrote:

> - Better integration between the manual browser and the docstring
>   browser, so you can click on a function/variable name in the manual to
>   get to its docstring and you can easily jump from a docstring to the
>   relevant section of the manual.

In principle, you already can. Being able to press just RET or click
instead of <f1> o RET would be nice, though.

Reply | Threaded
Open this post in threaded view
|

Re: docstrings and elisp reference

Stefan Monnier
>> - Better integration between the manual browser and the docstring
>> browser, so you can click on a function/variable name in the manual to
>> get to its docstring and you can easily jump from a docstring to the
>> relevant section of the manual.
> In principle, you already can. Being able to press just RET or click
> instead of <f1> o RET would be nice, though.

Of course `C-h o RET` works OK most of the time, but it's not the same
as having the var/fun name highlighted as a link that you can click.
We could even have a way to dynamically bring the docstring into the
Info manual (the same external-refs that allows an ad-hoc script to add
the docstrings to the printed version could be used to dynamically
insert the docstring into the *Info* buffer on-the-fly).


        Stefan


Reply | Threaded
Open this post in threaded view
|

RE: docstrings and elisp reference

Drew Adams
In reply to this post by Dmitry Gutov
> >>> they should not duplicate each other.
> >>
> >> Then you have something else in mind than the current
> >> situation with documentation in Emacs.
> >
> > No, you do, if you think the intention now is that
> > they duplicate each other.  It never has been.
>
> Please look up the difference between "intention" and "current situation".

There may be some examples in the "current situation"
where the doc string text and the text in the Elisp manual
are identical.  That is hardly how I would characterize
"the current situation".  Laziness and sloppiness happens.
And there might even be a few cases where having the exact
same text is not inappropriate.

But in general the "current situation" jibes well, I think,
with the "intention": doc string and manual are generally
_not_ exactly the same - their presentations, and to some
extent their content, are different.

> > Some of the information is often the same; that's all.
>
> That's called "duplication".

INFORMATION.  Yes, much of the _information_ is duplicated.
Of course.  Almost all of the information contained in doc
strings is in the Elisp manual.

The text/presentation is different.  The contexts are
different.  The readers are often different.  A reader's
intention in consulting them is often different.

> > In particular, doc strings are written as user help.
> > The interactive use, if any, is typically described
> > first, and from the point of an interactive user.
> > Not so, the Elisp manual.
> >
> > The Elisp doc for a command tells a Lisp programmer
> > what the function and its parameters are, and
> > describes their behavior.  It might or might not
> > mention that one of the parameters corresponds to
> > the prefix arg when called interactively.
>
> We have different manuals, with different goals.
>
> The fact that docstrings often describe the interactive case (when it
> exists) doesn't make them necessarily targeted at the end user. Just
> exhaustive.

The question about interactive use is a one of emphasis
and audience.  Interactive use is presented _first_ in
doc strings.  The aim in presenting it is not merely to
be exhaustive.  It's about whom the different docs are
presented to and what the aims typically are in reading
them.  And yes, that involves judgment calls.

We are not (and should not be) aiming only at reducing
emacs-dev maintenance time.  We are (and should be)
aiming at the best possible help for Emacs users,
including Lisp users.

That involves trade-offs, of course.  What it does not
involve is throwing up our hands and saying "Too hard,
bothersome, and error prone.  Let's just write it once -
one size fits all - and reuse the same thing for both
Elisp manual and doc string."  (As if that were a new
idea...)

Doc/help has been extremely important to the Emacs
project since Day One.  It's part of the mantra:
self-documenting editor.

And a lot of care and time has been spent, including
by some of the most qualified and knowledgable Emacs
developers, getting it right, where "right" means
paying attention to the differences among the roles
of doc string, Emacs manual, and Elisp manual.

I'm pretty sure that those who have long held a high
priority for the quality of Emacs doc, including in
particular RMS and Eli, do not share a reductive,
JavaDoc-like view of it.  I can't speak for them, of
course.  Perhaps they will care to speak up; perhaps not.

In any case, I've given my opinion about this: using
only a JavaDoc-like approach to Emacs doc would be a
step backward, not forward.  That does not mean that
there can never be any reuse (we do that with `apropos'
functions, for example).  It means that human judgment
is called for.

Reply | Threaded
Open this post in threaded view
|

RE: docstrings and elisp reference

Drew Adams
In reply to this post by Stefan Monnier
> - Better integration between the manual browser and the docstring
>   browser, so you can click on a function/variable name in the manual to
>   get to its docstring and you can easily jump from a docstring to the
>   relevant section of the manual.

Yes, 100%.  I've long argued that.  Cross-reference, not text reuse.

And Emacs has already come a fair distance in that direction.  It
could still be improved.

> - Remove the redundant var/fun documentation from the manual (this is
>   a tedious job: there is some redundance but it's not systematic, so
>   we can't just do this removal systematically).

We should not remove redundant _information_, if there is a
reason that it will help readers.  And that's the case for 99.9%
of the var/fun presentations in the manual.  There is often a
discussion about whether a given var/fun should be explicitly
doc'd in the manual - it's a judgment call.  It's not because
we _can_ add the doc for a var/fun to the manual that we should.

Think redundancy NOT for maintainers (I'm not saying you did that)
but for readers.

A reader should be _able_ to click to go to additional info
or to a different presentation/orientation of the same material.

But it is important, IMO, for the manual to have the information
that it has, even when that means that the same information is
provided also elsewhere (e.g. in a doc string).  We lose, do not
gain, if we decide to remove the doc for a given function etc.
from the manual just because it has a doc string - or vice versa.

The question of redundancy _for the user_ means that we also do
not, in general, want to have exactly the same text in doc string
and manual.  A user does not want to click a function name in one
or the other, only to end up seeing exactly the same text in the
other location.

"Reuse" for readers is generally a negative - it means reading
the same thing again, generally without being forewarned.
Little is more frustrating for a reader.  ("Why did you send me
here?  This is the same text!")

That is not to say that repetition in doc is always negative.
Far from it.  It's about judgment: repeat when there is a good
(e.g. pedagogical) reason to repeat.

> - Come up with a way to re-add those var/fun documentation into the
>   printed manual (e.g. by adding to the Texinfo source external
>   references to docstrings, which are then processed by now ad-hoc
>   script).

Not a good idea, IMO.  See above.

Reply | Threaded
Open this post in threaded view
|

RE: docstrings and elisp reference

Drew Adams
In reply to this post by Drew Adams
> And a lot of care and time has been spent, including
> by some of the most qualified and knowledgable Emacs
> developers, getting it right, where "right" means
> paying attention to the differences among the roles
> of doc string, Emacs manual, and Elisp manual.
>
> I'm pretty sure that those who have long held a high
> priority for the quality of Emacs doc, including in
> particular RMS and Eli, do not share a reductive,
> JavaDoc-like view of it.  I can't speak for them, of
> course.  Perhaps they will care to speak up; perhaps not.

I see they've both done that now.  Those who have not
yet spent a long time concerned about doc would do
well to listen.

Doc, like helping users generally, and like lots of
other things, is not so trivial.  There is no silver
bullet or royal road to helpful doc.

Emacs deserves the best doc possible.  You might say
it's been lucky that it's always benefitted from
strong developers who are also very concerned about
doc.  But it's not luck.  Quality doc is in part what
Emacs is about.  If you're interested in Emacs then
you pretty much need to be also interested in doc/help.

UI development is, in principle, similar, especially
free-software dev.

1234