TRAMP, auth-source and Secret Service API "label" prompt

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

TRAMP, auth-source and Secret Service API "label" prompt

Gustavo Barros
Hi All,

This is a follow-up report, from a discussion which started between
Michael Albinus and me at Reddit (https://redd.it/l2dw9j).

The behavior I met, which I thought was somehow "normal" and was trying
to disable, was that, whenever I tried to use TRAMP's sudo method (I
haven't tested others), I was first prompted for a password, as
expected, but after entering the password, I received a second prompt
"Enter label for root@localhost (default root@localhost):". I just
accepted the default, and the authentication succeeded. And this
happened even though I have set `auth-source-save-behavior' to nil.

Well, Michael told me at Reddit this is not expected behavior, so I
started playing with my settings in the area, and found the culprit to
be my `auth-sources' setting, which was:

#+begin_src emacs-lisp
(setq auth-sources '(default
                     "secrets:session"
                     "secrets:Login"
                     "~/.authinfo.gpg"))
#+end_src

Being `auth-source-save-behavior' and `auth-sources' the only options
from `auth-source.el' which I (knowingly, deliberately) configure in my
init file.  That setting for `auth-sources' is literally given as
example in (info "(auth) Secret Service API").

It turns out that, if a source which is not from the Secret Service API
(`secrets.el') has the highest priority there, the prompt is gone.  So I
went with:

#+begin_src emacs-lisp
(setq auth-sources '("~/.authinfo.gpg"
                     default
                     "secrets:session"
                     "secrets:Login"))
#+end_src

Michael found it strange, as I did, and suggested this should probably
be reported to `report-emacs-bug'.  So I tried to generate a
reproduction recipe starting from `emacs -Q' which, unfortunately, I
could not.  There is some interaction with something in my init which I
could not pin down.

Nevertheless, this bothered me now, so I tried to understand why this
was happening.  I started following the code trail.  The prompt itself
is from `auth-source-secrets-create' (in `auth-source.el'), and going up
from there I eventually came to `auth-source-search' which is called by
`tramp-read-passwd' in the following manner:

#+begin_src emacs-lisp
(auth-source-search
 :max 1
 (and user :user)
 (if domain
     (concat
      user tramp-prefix-domain-format domain)
   user)
 :host
 (if port
     (concat
      host tramp-prefix-port-format port)
   host)
 :port method
 :require (cons :secret (and user '(:user)))
 :create t)
#+end_src

And, indeed, `tramp-read-passwd' is asking auth-source to create a new
entry if one does not exist with `:create t'.

What particularly bothered me in this prompt was the fact of being
prompted for a "label" to save a password which was never meant to be
saved in the first place, since I set `auth-source-save-behavior' to
nil.  And it appears "create" and "save" are indeed two separate
operations for `auth-source', the docstring for `auth-source-search'
implies that in: "The token can hold a :save-function key.  If you call
that, the user will be prompted to save the data to the backend.  You
can't request that this should happen right after creation, because
`auth-source-search' has no way of knowing if the token is actually
useful.  So the caller must arrange to call this function."

My first thought was thus to condition directly to
`auth-source-save-behavior', and I've suggested Michael at Reddit it
might be interesting to use `:create (if auth-source-save-behavior t)',
so that `tramp-read-passwd' never tried to create the `auth-source'
entry if none was found, and none was meant to be saved.

I did test this, and it does work, in the sense that when
`auth-source-save-behavior' is set to nil, regardless of the order of
the sources in `auth-sources', that prompt does not occur.

But, since then I had what I regard a better idea: why not provide the
label?  This is pretty much standard behavior for applications which
store passwords in the Login keyring.  And it is not specially
complicated, as the label must exist, but does not have to be unique.
The docstring for `secrets-create-item' states this explicitly: "The
label ITEM does not have to be unique in COLLECTION."

So, `tramp-read-passwd' could go with something like:

#+begin_src emacs-lisp
:label (concat "Tramp password"
               (when (or user host) " for ")
               (when user (format "%s" user))
               (when host (format "@%s" host)))
#+end_src

keeping the current value of `:create t'.

That would automatically generate a label "Tramp password for
root@localhost" for the case at hand.  I've tested this, and the prompt
is gone for all cases, regardless of the order of entries in
`auth-sources'.  When `auth-source-save-behavior' is nil, it works as
expected (no prompt).  And there is a change in behavior when
`auth-source-save-behavior' is non-nil, the prompt is also gone, and the
password is saved according to the relevant settings with that label
(and answer in case of 'ask).

A couple of caveats.  First, I really don't know well `auth-sources.el'
and just tried to figure out what was happening in this case.  As far as
I understand the `:label' key will just be ignored by other backends of
`auth-source' where it is not meaningful.  But someone more acquainted
with `auth-source' should probably judge.  Second, I'm not sure that
particular label I suggested is general enough to cover all relevant
TRAMP methods, some further polishing there by someone more experienced
in the ways of TRAMP might be needed.

And, well, third, I understand, of course, that the fact I could not
generate a clean reproduction recipe with `emacs -Q' does beg the
question of whether any change is granted at all, because we really
don't know how relevant the "issue" is.  So, feel free to disregard the
suggestion.

Still, I hope this is somehow useful.

Best regards,
Gustavo.

PS: Please keep me CC's, as I'm not subscribed to the list.

Reply | Threaded
Open this post in threaded view
|

Re: TRAMP, auth-source and Secret Service API "label" prompt

Michael Albinus
Gustavo Barros <[hidden email]> writes:

> Hi All,

Hi Gustavo,

I'm adding Ted in Cc, because he is the author of auth-source.el. For
the same reason, I quote your analysis completely.

> This is a follow-up report, from a discussion which started between
> Michael Albinus and me at Reddit (https://redd.it/l2dw9j).
>
> The behavior I met, which I thought was somehow "normal" and was trying
> to disable, was that, whenever I tried to use TRAMP's sudo method (I
> haven't tested others), I was first prompted for a password, as
> expected, but after entering the password, I received a second prompt
> "Enter label for root@localhost (default root@localhost):". I just
> accepted the default, and the authentication succeeded. And this
> happened even though I have set `auth-source-save-behavior' to nil.
>
> Well, Michael told me at Reddit this is not expected behavior, so I
> started playing with my settings in the area, and found the culprit to
> be my `auth-sources' setting, which was:
>
> #+begin_src emacs-lisp
> (setq auth-sources '(default
>                      "secrets:session"
>                      "secrets:Login"
>                      "~/.authinfo.gpg"))
> #+end_src

Tramp searches via auth-source-search with the parameter ":create t". In
case it doesn't find a password in the sources, it creates the password
in the *first* source specified in auth-sources, which is
"secrets:session" in your case. As said in the documentation, this is a
temporary password collection, which does not survive an Emacs session.

However, it shouldn't even try to create the entry, because you have set
auth-source-save-behavior to nil.

> Being `auth-source-save-behavior' and `auth-sources' the only options
> from `auth-source.el' which I (knowingly, deliberately) configure in my
> init file.  That setting for `auth-sources' is literally given as
> example in (info "(auth) Secret Service API").
>
> It turns out that, if a source which is not from the Secret Service API
> (`secrets.el') has the highest priority there, the prompt is gone.  So I
> went with:
>
> #+begin_src emacs-lisp
> (setq auth-sources '("~/.authinfo.gpg"
>                      default
>                      "secrets:session"
>                      "secrets:Login"))
> #+end_src

Indeed. The creation function acknowledges auth-source-save-behavior
being nil, and it is silent.

> Michael found it strange, as I did, and suggested this should probably
> be reported to `report-emacs-bug'.  So I tried to generate a
> reproduction recipe starting from `emacs -Q' which, unfortunately, I
> could not.  There is some interaction with something in my init which I
> could not pin down.
>
> Nevertheless, this bothered me now, so I tried to understand why this
> was happening.  I started following the code trail.  The prompt itself
> is from `auth-source-secrets-create' (in `auth-source.el'), and going up
> from there I eventually came to `auth-source-search' which is called by
> `tramp-read-passwd' in the following manner:
>
> #+begin_src emacs-lisp
> (auth-source-search
>  :max 1
>  (and user :user)
>  (if domain
>      (concat
>       user tramp-prefix-domain-format domain)
>    user)
>  :host
>  (if port
>      (concat
>       host tramp-prefix-port-format port)
>    host)
>  :port method
>  :require (cons :secret (and user '(:user)))
>  :create t)
> #+end_src
>
> And, indeed, `tramp-read-passwd' is asking auth-source to create a new
> entry if one does not exist with `:create t'.

More precisely, Tramp is proposing (sigh!) to create a password if it
doesn't exist yet. auth-source-save-behavior let you overrule this proposal.

> What particularly bothered me in this prompt was the fact of being
> prompted for a "label" to save a password which was never meant to be
> saved in the first place, since I set `auth-source-save-behavior' to
> nil.  And it appears "create" and "save" are indeed two separate
> operations for `auth-source', the docstring for `auth-source-search'
> implies that in: "The token can hold a :save-function key.  If you call
> that, the user will be prompted to save the data to the backend.  You
> can't request that this should happen right after creation, because
> `auth-source-search' has no way of knowing if the token is actually
> useful.  So the caller must arrange to call this function."
>
> My first thought was thus to condition directly to
> `auth-source-save-behavior', and I've suggested Michael at Reddit it
> might be interesting to use `:create (if auth-source-save-behavior t)',
> so that `tramp-read-passwd' never tried to create the `auth-source'
> entry if none was found, and none was meant to be saved.
>
> I did test this, and it does work, in the sense that when
> `auth-source-save-behavior' is set to nil, regardless of the order of
> the sources in `auth-sources', that prompt does not occur.

Tramp shouldn't handle auth-source-save-behavior on its own, I
believe. It is just a contract between auth-source.el and the user;
Tramp is not involved.

> But, since then I had what I regard a better idea: why not provide the
> label?  This is pretty much standard behavior for applications which
> store passwords in the Login keyring.  And it is not specially
> complicated, as the label must exist, but does not have to be unique.
> The docstring for `secrets-create-item' states this explicitly: "The
> label ITEM does not have to be unique in COLLECTION."
>
> So, `tramp-read-passwd' could go with something like:
>
> #+begin_src emacs-lisp
> :label (concat "Tramp password"
>                (when (or user host) " for ")
>                (when user (format "%s" user))
>                (when host (format "@%s" host)))
> #+end_src
>
> keeping the current value of `:create t'.
>
> That would automatically generate a label "Tramp password for
> root@localhost" for the case at hand.  I've tested this, and the prompt
> is gone for all cases, regardless of the order of entries in
> `auth-sources'.  When `auth-source-save-behavior' is nil, it works as
> expected (no prompt).  And there is a change in behavior when
> `auth-source-save-behavior' is non-nil, the prompt is also gone, and the
> password is saved according to the relevant settings with that label
> (and answer in case of 'ask).

Tramp doesn't know the Secret Sercice API of auth-source.el, and so it
doesn't know that something like a label is needed.

(Well, this is a shameless lie: I'm the maintainer of Tramp, and the
author of secrets.el. But from the design point of view, this stands.)

> A couple of caveats.  First, I really don't know well `auth-sources.el'
> and just tried to figure out what was happening in this case.  As far as
> I understand the `:label' key will just be ignored by other backends of
> `auth-source' where it is not meaningful.  But someone more acquainted
> with `auth-source' should probably judge.  Second, I'm not sure that
> particular label I suggested is general enough to cover all relevant
> TRAMP methods, some further polishing there by someone more experienced
> in the ways of TRAMP might be needed.
>
> And, well, third, I understand, of course, that the fact I could not
> generate a clean reproduction recipe with `emacs -Q' does beg the
> question of whether any change is granted at all, because we really
> don't know how relevant the "issue" is.  So, feel free to disregard the
> suggestion.

As far as I understand, auth-source.el shouldn't ask for a label, when
auth-source-save-behavior is nil, because no entry in the Secret Service
API shall be created. This seems to be a bug to me in
auth-source-secrets-create. It shall ask only sor a label, if
auth-source-save-behavior has the value 'ask. If it has the value t,
there shouldn't also be a question about the label; just the default
label shall be used.

> Still, I hope this is somehow useful.
>
> Best regards,
> Gustavo.
>
> PS: Please keep me CC's, as I'm not subscribed to the list.

Best regards, Michael.

Reply | Threaded
Open this post in threaded view
|

Re: TRAMP, auth-source and Secret Service API "label" prompt

Gustavo Barros
Hi Michael,
Hi Ted,
Hi All,

On Sun, 24 Jan 2021 at 15:55, Michael Albinus <[hidden email]>
wrote:

>
> Hi Gustavo,
>
> I'm adding Ted in Cc, because he is the author of auth-source.el. For
> the same reason, I quote your analysis completely.
>

Michael, I do understand your position.  And won't dispute it, on the
contrary, I find the design choice reasoning to be more than sound, and
do recognize my suggestion to be quite ad hoc.  Thank you for
considering it.

Given then we have to look at this from the perspective of
`auth-source', the fact still remains that I cannot reproduce the
behavior with `emacs -Q', and could not isolate in my init file whatever
is the missing intervening factor which triggers it.

So, unless Ted can figure this out in `auth-source', the best I can
still offer is to remain at your disposal to try things out on my end,
and see if, with your guidance, we can find the culprit.  That is, if
you think it's worth it, because, of course, we cannot exclude that this
is all the fault of "that piece between my keyboard and my chair".

Best regards,
Gustavo.

Reply | Threaded
Open this post in threaded view
|

Re: TRAMP, auth-source and Secret Service API "label" prompt

Michael Albinus
Gustavo Barros <[hidden email]> writes:

> Hi Michael,

Hi Gustavo,

> Given then we have to look at this from the perspective of
> `auth-source', the fact still remains that I cannot reproduce the
> behavior with `emacs -Q', and could not isolate in my init file
> whatever is the missing intervening factor which triggers it.

Give me some days in order to reproduce it. I'll work on it, when
there's a free time slot.

> Best regards,
> Gustavo.

Best regards, Michael.

Reply | Threaded
Open this post in threaded view
|

Re: TRAMP, auth-source and Secret Service API "label" prompt

Gustavo Barros
Hi Michael,

On Mon, 25 Jan 2021 at 09:51, Michael Albinus <[hidden email]>
wrote:

> Give me some days in order to reproduce it. I'll work on it, when
> there's a free time slot.

No hurry from my part.  And, again, if you have some ideas you'd me like
to try here, where the issue does occur, I'll be glad to do it.

Best regards,
Gustavo.

Reply | Threaded
Open this post in threaded view
|

Re: TRAMP, auth-source and Secret Service API "label" prompt

Gustavo Barros
In reply to this post by Michael Albinus
Hi Michael,

On Mon, 25 Jan 2021 at 09:51, Michael Albinus <[hidden email]>
wrote:

>> Given then we have to look at this from the perspective of
>> `auth-source', the fact still remains that I cannot reproduce the
>> behavior with `emacs -Q', and could not isolate in my init file
>> whatever is the missing intervening factor which triggers it.
>
> Give me some days in order to reproduce it. I'll work on it, when
> there's a free time slot.

Adding some information.  I'm using GNU Emacs 27.1 (build 1,
x86_64-pc-linux-gnu, GTK+ Version 3.24.20, cairo version 1.16.0) of
2020-08-11, with built-in versions of TRAMP, auth-source and secrets.  I
had failed to mention it in my initial report.

I've also just did a more thorough bisecting exercise here.  I disabled
literally everything in both init.el and early-init.el, except for the
two sexps:

#+begin_src emacs-lisp
(setq auth-source-save-behavior nil)
(setq auth-sources
      '(default
         "secrets:session"
         "secrets:Login"
         "~/.authinfo.gpg"))
#+end_src

And the behavior still occurs.  My though then was that it must be in
some package I have installed, which is being activated between
early-init.el and init.el during startup.  I went with `emacs -Q' and
`M-: (package-activate-all)', and the behavior does not occur.

Best regards,
Gustavo.

Reply | Threaded
Open this post in threaded view
|

Re: TRAMP, auth-source and Secret Service API "label" prompt

Michael Albinus
Gustavo Barros <[hidden email]> writes:

> Hi Michael,

Hi Gustavo,

> And the behavior still occurs.  My though then was that it must be in
> some package I have installed, which is being activated between
> early-init.el and init.el during startup.  I went with `emacs -Q' and
> `M-: (package-activate-all)', and the behavior does not occur.

IIRC, in case of "emacs -Q", Tramp does not call auth-source-search. I
made this in order to get better error reports, w/o the hazzle to
instruct people how to configure auth-sources well. Try the following
after loading tramp.el:

(setq tramp-cache-read-persistent-data t)

This enables also using cached properties, but this shouldn't hurt in
your case.

> Best regards,
> Gustavo.

Best regards, Michael.

Reply | Threaded
Open this post in threaded view
|

Re: TRAMP, auth-source and Secret Service API "label" prompt

Gustavo Barros

On Mon, 25 Jan 2021 at 11:03, Michael Albinus <[hidden email]>
wrote:

> IIRC, in case of "emacs -Q", Tramp does not call auth-source-search. I
> made this in order to get better error reports, w/o the hazzle to
> instruct people how to configure auth-sources well. Try the following
> after loading tramp.el:
>
> (setq tramp-cache-read-persistent-data t)
>
> This enables also using cached properties, but this shouldn't hurt in
> your case.

Ah, bingo! I had no idea, and was starting to think I was mad.  :-)

Anyway, then we have an `emacs -Q' reproduction recipe:

#+begin_src emacs-lisp
(load-library "tramp")
(setq tramp-cache-read-persistent-data t)
(setq auth-source-save-behavior nil)
(setq auth-sources
     '(default
        "secrets:session"
        "secrets:Login"
        "~/.authinfo.gpg"))
#+end_src

Now find file "/sudo::/tmp/test.txt", enter password, and meet the
infamous "label prompt".

Best regards,
Gustavo.

Reply | Threaded
Open this post in threaded view
|

Re: TRAMP, auth-source and Secret Service API "label" prompt

Michael Albinus
Gustavo Barros <[hidden email]> writes:

Hi Gustavo,

> Anyway, then we have an `emacs -Q' reproduction recipe:
>
> #+begin_src emacs-lisp
> (load-library "tramp")
> (setq tramp-cache-read-persistent-data t)
> (setq auth-source-save-behavior nil)
> (setq auth-sources
>      '(default
>         "secrets:session"
>         "secrets:Login"
>         "~/.authinfo.gpg"))
> #+end_src
>
> Now find file "/sudo::/tmp/test.txt", enter password, and meet the
> infamous "label prompt".
The appended patch fixes this for me, could you pls check? I've applied
it on top of Emacs 28.0.50, but it shall also apply for Emacs 27.

> Best regards,
> Gustavo.

Best regards, Michael.


attachment0 (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: TRAMP, auth-source and Secret Service API "label" prompt

Gustavo Barros
Hi Michael,

On Tue, 26 Jan 2021 at 15:59, Michael Albinus <[hidden email]>
wrote:
>
> The appended patch fixes this for me, could you pls check? I've
> applied
> it on top of Emacs 28.0.50, but it shall also apply for Emacs 27.
>

I'm running the released version, not from Git.  But I did redefine
`auth-source-secrets-create' according to the patch, and can confirm the
prompt no longer occurs when `auth-source-save-behavior' is nil, even
when the first entry in `auth-sources' is from the Secret Service API.

Thank you very much for looking into this and for the fix.

Best regards,
Gustavo.

Reply | Threaded
Open this post in threaded view
|

Re: TRAMP, auth-source and Secret Service API "label" prompt

Michael Albinus
Gustavo Barros <[hidden email]> writes:

> Hi Michael,

Hi Gustavo,

>> The appended patch fixes this for me, could you pls check? I've
>> applied
>> it on top of Emacs 28.0.50, but it shall also apply for Emacs 27.
>
> I'm running the released version, not from Git.  But I did redefine
> `auth-source-secrets-create' according to the patch, and can confirm
> the prompt no longer occurs when `auth-source-save-behavior' is nil,
> even when the first entry in `auth-sources' is from the Secret Service
> API.
>
> Thank you very much for looking into this and for the fix.

Thanks for the feedback. However, I'd like to hear from Ted whether this
is the proper way to fix it, before I push the patch.

> Best regards,
> Gustavo.

Best regards, Michael.

Reply | Threaded
Open this post in threaded view
|

Re: TRAMP, auth-source and Secret Service API "label" prompt

Michael Albinus
Michael Albinus <[hidden email]> writes:

Hi Gustavo,

>>> The appended patch fixes this for me, could you pls check? I've
>>> applied
>>> it on top of Emacs 28.0.50, but it shall also apply for Emacs 27.
>>
>> I'm running the released version, not from Git.  But I did redefine
>> `auth-source-secrets-create' according to the patch, and can confirm
>> the prompt no longer occurs when `auth-source-save-behavior' is nil,
>> even when the first entry in `auth-sources' is from the Secret Service
>> API.
>>
>> Thank you very much for looking into this and for the fix.
>
> Thanks for the feedback. However, I'd like to hear from Ted whether this
> is the proper way to fix it, before I push the patch.

No answer, unfortunately. So I have pushed a modified version of the
patch to the Emacs master branch, fixing this for now and ever.

>> Best regards,
>> Gustavo.

Best regards, Michael.

Reply | Threaded
Open this post in threaded view
|

Re: TRAMP, auth-source and Secret Service API "label" prompt

Gustavo Barros
Hi Michael,

On Sat, 06 Feb 2021 at 07:54, Michael Albinus <[hidden email]>
wrote:

>>
>> Thanks for the feedback. However, I'd like to hear from Ted whether
>> this
>> is the proper way to fix it, before I push the patch.
>
> No answer, unfortunately. So I have pushed a modified version of the
> patch to the Emacs master branch, fixing this for now and ever.
>
> Best regards, Michael.

Thank you very much for seeing this through.

And, just so I know, is the fix coming to 27.2, or only to 28+? (No
problem either way, just for me to add a corresponding note in my init
file to revert the workaround.)

Best regards,
Gustavo.

Reply | Threaded
Open this post in threaded view
|

Re: TRAMP, auth-source and Secret Service API "label" prompt

Michael Albinus
Gustavo Barros <[hidden email]> writes:

> Hi Michael,

Hi Gustavo,

> And, just so I know, is the fix coming to 27.2, or only to 28+? (No
> problem either way, just for me to add a corresponding note in my init
> file to revert the workaround.)

No, it won't go to Emacs 27.2. For several reasons:

- The applied patch uses a function, which has been introduced in Emacs
  28 (`format-prompt'). For Emacs 27, this patch would need changes.

- Emacs 27.2 is in pretest. That means, only corrections for blocking or
  regression errors are accepted.

I don't know, whether there will be an Emacs 27.3.

> Best regards,
> Gustavo.

Best regards, Michael.

Reply | Threaded
Open this post in threaded view
|

Re: TRAMP, auth-source and Secret Service API "label" prompt

Gustavo Barros
Hi Michael,

On Sat, 06 Feb 2021 at 10:12, Michael Albinus <[hidden email]>
wrote:

> Hi Gustavo,
>
>> And, just so I know, is the fix coming to 27.2, or only to 28+? (No
>> problem either way, just for me to add a corresponding note in my
>> init
>> file to revert the workaround.)
>
> No, it won't go to Emacs 27.2. For several reasons:
>
> - The applied patch uses a function, which has been introduced in
> Emacs
>   28 (`format-prompt'). For Emacs 27, this patch would need changes.
>
> - Emacs 27.2 is in pretest. That means, only corrections for blocking
> or
>   regression errors are accepted.
>
> I don't know, whether there will be an Emacs 27.3.
>

Thanks for clearing that up.

It's all good from my side.  The only "risk" I incur with the workaround
is that I forget about it, and now it is duly noted in my init file.

Best,
Gustavo.