bug#40676: 28.0.50; gnus locks when reading email

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

bug#40676: 28.0.50; gnus locks when reading email

Lars Ingebrigtsen
"Philip K." <[hidden email]> writes:

> That was part of the rationale behind #40355, but the best way I see to
> fix this would be to implement asynchronous DNS, since a libravatar
> lookup has two phases (DNS lookup + image retrieval), compared to
> Gravatar's single request.

A cache would be a band-aid -- it still will have to do these lookups
occasionally, and the user experience in Gnus suffers.

As it stands, librgravatar shouldn't be the default gravatar provider.

But, yes, Emacs should have asynchronous DNS support, and adding that
probably isn't too difficult, I'd have thought?

--
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



Reply | Threaded
Open this post in threaded view
|

bug#40676: 28.0.50; gnus locks when reading email

Philip K.
Lars Ingebrigtsen <[hidden email]> writes:

> "Philip K." <[hidden email]> writes:
>
>> That was part of the rationale behind #40355, but the best way I see to
>> fix this would be to implement asynchronous DNS, since a libravatar
>> lookup has two phases (DNS lookup + image retrieval), compared to
>> Gravatar's single request.
>
> A cache would be a band-aid -- it still will have to do these lookups
> occasionally, and the user experience in Gnus suffers.

Another idea would be to only wait for a few milliseconds (or whatever
is reasonable) and only return an image if the backend manages to find
something in that time. But the request is still finished and cached for
the next query.

> As it stands, librgravatar shouldn't be the default gravatar provider.

I'm somewhat split on this. On the one hand, the reason I implemented
libravatar support is to increase it's audience and make more people
aware if it's existence, as a free and federated alternative to
Gravatar.

On the other hand, there is a privacy issue in the system, as explained
by [0]. The problem basically is that if I send you an email and set up
a libravatar server, by querying my server, I can know if you opened my
message or not. I can imagine spammers being very interested in
something like this.

> But, yes, Emacs should have asynchronous DNS support, and adding that
> probably isn't too difficult, I'd have thought?

I started writing something a few months ago, but didn't have the time
to finish it. But you're right, it shouldn't be too much work to come up
with a rough draft.

[0] https://lobste.rs/s/nwgljm/libravatar_federated_avatar_hosting#c_00fsba

--
        Philip K.



Reply | Threaded
Open this post in threaded view
|

bug#40676: 28.0.50; gnus locks when reading email

Lars Ingebrigtsen
"Philip K." <[hidden email]> writes:

>> As it stands, librgravatar shouldn't be the default gravatar provider.
>
> I'm somewhat split on this. On the one hand, the reason I implemented
> libravatar support is to increase it's audience and make more people
> aware if it's existence, as a free and federated alternative to
> Gravatar.
>
> On the other hand, there is a privacy issue in the system, as explained
> by [0]. The problem basically is that if I send you an email and set up
> a libravatar server, by querying my server, I can know if you opened my
> message or not. I can imagine spammers being very interested in
> something like this.

Indeed -- it makes libravatar a much worse interface than the central
Gravatar service (where you just have to trust the Gravatar people
somewhat, instead of the entire internet).

So that's another reason not to make libravatar the default provider.

--
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



Reply | Threaded
Open this post in threaded view
|

bug#40676: 28.0.50; gnus locks when reading email

Robert Pluim
In reply to this post by Lars Ingebrigtsen
>>>>> On Sun, 19 Jul 2020 15:02:35 +0200, Lars Ingebrigtsen <[hidden email]> said:

    Lars> "Philip K." <[hidden email]> writes:
    >> That was part of the rationale behind #40355, but the best way I see to
    >> fix this would be to implement asynchronous DNS, since a libravatar
    >> lookup has two phases (DNS lookup + image retrieval), compared to
    >> Gravatar's single request.

    Lars> A cache would be a band-aid -- it still will have to do these lookups
    Lars> occasionally, and the user experience in Gnus suffers.

    Lars> As it stands, librgravatar shouldn't be the default gravatar provider.

    Lars> But, yes, Emacs should have asynchronous DNS support, and adding that
    Lars> probably isn't too difficult, I'd have thought?

emacs calls getaddrinfo_a already if itʼs available, when creating
connections, but I donʼt think thereʼs a lisp-level interface to it
(and itʼs not available on macOS, weʼd have to implement it using
Apple's platform-specific API).

If you were thinking of using the emacs threading API, I tried hooking
that up to dns.el, but the results were not great. [1]

Robert

Footnotes:
[1]  Iʼm saying this so that Lars will now go away and implement it :-)




Reply | Threaded
Open this post in threaded view
|

bug#40676: 28.0.50; gnus locks when reading email

Lars Ingebrigtsen
Robert Pluim <[hidden email]> writes:

> emacs calls getaddrinfo_a already if itʼs available, when creating
> connections, but I donʼt think thereʼs a lisp-level interface to it
> (and itʼs not available on macOS, weʼd have to implement it using
> Apple's platform-specific API).

Yes, I was thinking that we could use Emacs' normal process support for
it, but just call getaddrinf_a and then return the results.  I haven't
actually looked at how to do that, but it seems like it should
conceptually makes sense.

For instance, to make it very much like a normal network process, Emacs
could even just output the DNS data as bytes in the process buffer.  (We
have code to parse DNS protocol data in dns.el already.)

--
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



Reply | Threaded
Open this post in threaded view
|

bug#40676: 28.0.50; gnus locks when reading email

Robert Pluim
>>>>> On Mon, 20 Jul 2020 11:23:02 +0200, Lars Ingebrigtsen <[hidden email]> said:

    Lars> Lars Ingebrigtsen <[hidden email]> writes:
    >> For instance, to make it very much like a normal network process, Emacs
    >> could even just output the DNS data as bytes in the process buffer.  (We
    >> have code to parse DNS protocol data in dns.el already.)

    Lars> Actually, that'd make no sense, because we don't have access to the raw
    Lars> DNS packages on the C side.

    Lars> So the process could just output the result as ASCII, perhaps.  That is,
    Lars> a call to

    Lars> (make-network-process :lookup-address "gnu.org")

    Lars> would result in

    Lars> 209.51.188.148^@2001:470:142:3::a^@

    Lars> or something in the buffer.  Or perhaps

    Lars> ipv4-address^@209.51.188.148^@ipv6-address^@2001:470:142:3::a^@

    Lars> or however we want to do this.

You donʼt like `network-lookup-address-info'? Just asynchifying that
should be enough.

Robert



Reply | Threaded
Open this post in threaded view
|

bug#40676: 28.0.50; gnus locks when reading email

Robert Pluim
>>>>> On Mon, 20 Jul 2020 11:33:31 +0200, Robert Pluim <[hidden email]> said:

>>>>> On Mon, 20 Jul 2020 11:23:02 +0200, Lars Ingebrigtsen <[hidden email]> said:
    Lars> Lars Ingebrigtsen <[hidden email]> writes:
    >>> For instance, to make it very much like a normal network process, Emacs
    >>> could even just output the DNS data as bytes in the process buffer.  (We
    >>> have code to parse DNS protocol data in dns.el already.)

    Lars> Actually, that'd make no sense, because we don't have access to the raw
    Lars> DNS packages on the C side.

    Lars> So the process could just output the result as ASCII, perhaps.  That is,
    Lars> a call to

    Lars> (make-network-process :lookup-address "gnu.org")

    Lars> would result in

    Lars> 209.51.188.148^@2001:470:142:3::a^@

    Lars> or something in the buffer.  Or perhaps

    Lars> ipv4-address^@209.51.188.148^@ipv6-address^@2001:470:142:3::a^@

    Lars> or however we want to do this.

    Robert> You donʼt like `network-lookup-address-info'? Just asynchifying that
    Robert> should be enough.

Actually, no, that wouldnʼt work since network-lookup-address-info can
only lookup 'A' and 'AAAA' records, and gravatar needs 'SRV'.

Robert



Reply | Threaded
Open this post in threaded view
|

bug#40676: 28.0.50; gnus locks when reading email

Lars Ingebrigtsen
In reply to this post by Robert Pluim
Robert Pluim <[hidden email]> writes:

> You donʼt like `network-lookup-address-info'? Just asynchifying that
> should be enough.

I like that function.  :-)  I just see how to fit that into a
process-like async context.

--
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



Reply | Threaded
Open this post in threaded view
|

bug#40676: 28.0.50; gnus locks when reading email

Lars Ingebrigtsen
Lars Ingebrigtsen <[hidden email]> writes:

> Robert Pluim <[hidden email]> writes:
>
>> You donʼt like `network-lookup-address-info'? Just asynchifying that
>> should be enough.
>
> I like that function.  :-)  I just see how to fit that into a
> process-like async context.

There should be a "don't" after the "just" there, I guess.  :-/

--
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



Reply | Threaded
Open this post in threaded view
|

bug#40676: 28.0.50; gnus locks when reading email

Lars Ingebrigtsen
In reply to this post by Robert Pluim
Robert Pluim <[hidden email]> writes:

> Actually, no, that wouldnʼt work since network-lookup-address-info can
> only lookup 'A' and 'AAAA' records, and gravatar needs 'SRV'.

I should have actually looked at the libravatar code -- it uses dns.el.
Which can be rewritten to be asynchronous, since it just does network
chatter via processes...

--
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



Reply | Threaded
Open this post in threaded view
|

bug#40676: 28.0.50; gnus locks when reading email

Lars Ingebrigtsen
In reply to this post by Lars Ingebrigtsen
Lars Ingebrigtsen <[hidden email]> writes:

> Indeed -- it makes libravatar a much worse interface than the central
> Gravatar service (where you just have to trust the Gravatar people
> somewhat, instead of the entire internet).
>
> So that's another reason not to make libravatar the default provider.

I've now changed the default to use gravatar, since that doesn't have
the same security implications.

Emacs should still grow an async DNS implementation (reachable from Lisp
land), though.

--
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



Reply | Threaded
Open this post in threaded view
|

bug#40676: 28.0.50; gnus locks when reading email

Lars Ingebrigtsen
Lars Ingebrigtsen <[hidden email]> writes:

> Emacs should still grow an async DNS implementation (reachable from Lisp
> land), though.

I've now added a function dns-query-asynchronous to dns.el to have an
asynchronous implementation that works across all Emacs builds.

However, I'm not pushing it yet until somebody makes a decision as to
whether the Emacs git repo is to be recovered from backup, or we just
carry on as before after the mis-merge into emacs-27 a few hours ago...

--
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



Reply | Threaded
Open this post in threaded view
|

bug#40676: 28.0.50; gnus locks when reading email

Richard Stallman
In reply to this post by Lars Ingebrigtsen
[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Indeed -- it makes libravatar a much worse interface than the central
  > > Gravatar service (where you just have to trust the Gravatar people
  > > somewhat, instead of the entire internet).
  > >
  > > So that's another reason not to make libravatar the default provider.

I don't know about libravatar, but ISTR that gravatar was found
to do some sort of surveillance of users.  Is that correct?
Was libravatar intended as a replacement to respect users' privacy
more than gravatar?

But it seems there is a problem with using libravatar, right?

If that is the case, we should not think of "use gravatar"
as a _solution_.  It might fix an urgent problem, but we
would not want it as a permanent change.  A real solution
is to make libravatar work well.

If what is needed is async DNS lookup, how about running
a shell command asynchronously to do it?


--
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)





Reply | Threaded
Open this post in threaded view
|

bug#40676: 28.0.50; gnus locks when reading email

Lars Ingebrigtsen
In reply to this post by Lars Ingebrigtsen
Lars Ingebrigtsen <[hidden email]> writes:

> I've now added a function dns-query-asynchronous to dns.el to have an
> asynchronous implementation that works across all Emacs builds.

And now I've rewritten gravatar.el to be asynchronous, too, so I'm
closing this bug report.

--
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



Reply | Threaded
Open this post in threaded view
|

bug#40676: 28.0.50; gnus locks when reading email

Philip K.-3
In reply to this post by Richard Stallman
Richard Stallman <[hidden email]> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > > Indeed -- it makes libravatar a much worse interface than the central
>   > > Gravatar service (where you just have to trust the Gravatar people
>   > > somewhat, instead of the entire internet).
>   > >
>   > > So that's another reason not to make libravatar the default provider.
>
> I don't know about libravatar, but ISTR that gravatar was found
> to do some sort of surveillance of users.

I'm not sure if it was proven, but just like Google Analytics or the
Facebook Like-button, they offer a service with a hidden fee -- that
they can or cannot exploit.

In the case of Emacs' Gravatar Integration, they get "notified" whenever
someone opens a message from person X (at least once), meaning they
could more easily reconstruct a database of who communicates with whom.

> Is that correct?  Was libravatar intended as a replacement to respect
> users' privacy more than gravatar?

From libravatar's web page:

        FREEDOM AND FEDERATION

        How is Libravatar different from Gravatar though? The main
        difference is that while Libravatar.org is an online avatar
        hosting service just like Gravatar, the software that powers the
        former is also available for download under a free software
        license.

        Why would you want to run your own instance? Our belief is that
        centralised approaches don't put users in control. If you own
        your own domain name, you control how mail is delivered to your
        domain. You should also be able to control how avatars are
        served to other websites.

So kind of, thought it seems to be more about controlling the software
you use than privacy per se.

> But it seems there is a problem with using libravatar, right?

Yes, the same attack as above with Gravatar (getting notified when you
open a message), just that a third party can set up their own libravatar
server and get notified when I open their email, instead of one
centralised service. So kind of like tracking bits in HTML messages.

> If that is the case, we should not think of "use gravatar"
> as a _solution_.  It might fix an urgent problem, but we
> would not want it as a permanent change.  A real solution
> is to make libravatar work well.

That would require an architectural change to libravatar itself. It's
current mechanism to retrieve an image for [hidden email] is:

1. Check if domain.tld has a libravatar service (via DNS)
2. If yes, use that server, otherwise default to libravatar's server
3. Send a request to the server using the MD5/SHA256 hash of
   [hidden email]

Steps 1./2. are the trick to federation, but the crux of the issue as
well.

> If what is needed is async DNS lookup, how about running
> a shell command asynchronously to do it?

I guess that would also be possible, but wouldn't that require a DNS
tool to be installed on the system?

--
        Philip K.



Reply | Threaded
Open this post in threaded view
|

bug#40676: 28.0.50; gnus locks when reading email

Richard Stallman
[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > But it seems there is a problem with using libravatar, right?

  > Yes, the same attack as above with Gravatar (getting notified when you
  > open a message), just that a third party can set up their own libravatar
  > server and get notified when I open their email, instead of one
  > centralised service. So kind of like tracking bits in HTML messages.

I guess this is not a big practical difference.
Simply using libravatar instead of gravatar doesn't seem to protect privacy.
You've explained that perhaps some redesign could make it protect privacy,
but it seems to me that the difference _at present_ is not crucial.

Thanks.


--
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)