RFC: Adding BBDB to Emacs core

classic Classic list List threaded Threaded
51 messages Options
123
Reply | Threaded
Open this post in threaded view
|

RFC: Adding BBDB to Emacs core

Thomas Fitzsimmons-2
Hi,

Now that BBDB is copyright clear and available in GNU ELPA, thanks to
Roland Winkler, I'd like to see what people think about also adding it
to Emacs core.

While maintaining EUDC in Emacs core, I've encountered many references
to BBDB that are unresolved.  The only reason for this, as far as I can
tell, is that historically BBDB's copyright status didn't allow it to be
included in core.  Otherwise it probably would have been included all
along.  Now it's possible to fix this properly.

I've started an integration attempt on the scratch/eudc-bbdb-3 branch.
I merged a recent version of BBDB from GNU ELPA into lisp/bbdb, then I
resolved references to BBDB in EUDC.  For example, we can remove things
like:

   (declare-function bbdb-record-phones "ext:bbdb" t) ; via bbdb-defstruct

and apply changes like:

--- a/lisp/net/eudc-export.el
+++ b/lisp/net/eudc-export.el
@@ -31,10 +31,8 @@
 ;;; Code:
 
 (require 'eudc)
-
-;; NOERROR is so we can compile it.
-(require 'bbdb nil t)
-(require 'bbdb-com nil t)
+(require 'bbdb)
+(require 'bbdb-com)
 
 (defun eudc-create-bbdb-record (record &optional silent)
   "Create a BBDB record using the RECORD alist.

This makes the code cleaner and easier to maintain.  We can also rely
only on the version of BBDB in GNU Emacs (or a later one in GNU ELPA)
and so all the BBDB < 3 compatibility code can be deleted without risk
of breaking people's package sets (BBDB >= 3 auto-converts BBDB < 3
databases to the updated format).

I think applying this same type of effort to the other BBDB-dependent
core packages would simplify them too.

I'd like BBDB to become the default out-of-the-box local contact
management library for GNU Emacs, in particular so that Gnus/EUDC/BBDB
work together to provide email completion and snarfing out-of-the-box,
without extra configuration or package installation.

Thoughts?

Thomas

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Adding BBDB to Emacs core

Joshua Branson-4

That sounds pretty awesome, but does the bbdb package have any info
documentation?  Not that it really matters, but it would be nice to have.

Also, may I ask about ebdb?  Does ebdb ever have
a chance at making emacs core?

P.S.  I'm not a developer (at least not yet).  Just curious.


Thomas Fitzsimmons <[hidden email]> writes:

> Hi,
>
> Now that BBDB is copyright clear and available in GNU ELPA, thanks to
> Roland Winkler, I'd like to see what people think about also adding it
> to Emacs core.
>
> While maintaining EUDC in Emacs core, I've encountered many references
> to BBDB that are unresolved.  The only reason for this, as far as I can
> tell, is that historically BBDB's copyright status didn't allow it to be
> included in core.  Otherwise it probably would have been included all
> along.  Now it's possible to fix this properly.
>
> I've started an integration attempt on the scratch/eudc-bbdb-3 branch.
> I merged a recent version of BBDB from GNU ELPA into lisp/bbdb, then I
> resolved references to BBDB in EUDC.  For example, we can remove things
> like:
>
>    (declare-function bbdb-record-phones "ext:bbdb" t) ; via bbdb-defstruct
>
> and apply changes like:
>
> --- a/lisp/net/eudc-export.el
> +++ b/lisp/net/eudc-export.el
> @@ -31,10 +31,8 @@
>  ;;; Code:
>
>  (require 'eudc)
> -
> -;; NOERROR is so we can compile it.
> -(require 'bbdb nil t)
> -(require 'bbdb-com nil t)
> +(require 'bbdb)
> +(require 'bbdb-com)
>
>  (defun eudc-create-bbdb-record (record &optional silent)
>    "Create a BBDB record using the RECORD alist.
>
> This makes the code cleaner and easier to maintain.  We can also rely
> only on the version of BBDB in GNU Emacs (or a later one in GNU ELPA)
> and so all the BBDB < 3 compatibility code can be deleted without risk
> of breaking people's package sets (BBDB >= 3 auto-converts BBDB < 3
> databases to the updated format).
>
> I think applying this same type of effort to the other BBDB-dependent
> core packages would simplify them too.
>
> I'd like BBDB to become the default out-of-the-box local contact
> management library for GNU Emacs, in particular so that Gnus/EUDC/BBDB
> work together to provide email completion and snarfing out-of-the-box,
> without extra configuration or package installation.
>
> Thoughts?
>
> Thomas

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Adding BBDB to Emacs core

Glenn Morris-3
In reply to this post by Thomas Fitzsimmons-2
Thomas Fitzsimmons wrote:

> I'd like to see what people think about also adding it to Emacs core.

Personally I think this is the opposite direction to the one in which
Emacs components should be moving.

> is that historically BBDB's copyright status didn't allow it to be
> included in core. Otherwise it probably would have been included all
> along.

Bear in mind that ELPAs did not exist when bbdb was first created.

> I'd like BBDB to become the default out-of-the-box local contact
> management library for GNU Emacs, in particular so that Gnus/EUDC/BBDB
> work together to provide email completion and snarfing out-of-the-box,
> without extra configuration or package installation.

If GNU ELPA is a first-class citizen, then all the above can happen
without adding yet more stuff to the main Emacs repo. (Wistfully
thinking here yet again of the project to bundle GNU ELPA packages with
Emacs releases...)

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Adding BBDB to Emacs core

Radon Rosborough
>> I'd like to see what people think about also adding it to Emacs core.
>
> Personally I think this is the opposite direction to the one in which
> Emacs components should be moving.

I heartily agree here: as someone who is looking to modernize Emacs
package management, one of the biggest problems I see in the current
system is the fact that some (obsolete) versions of packages are
bundled with Emacs core. It is annoying in the same way that when
using a package manager on macOS, one must deal with the (obsolete)
versions of packages that are provided by macOS.

> (Wistfully thinking here yet again of the project to bundle GNU ELPA
> packages with Emacs releases...)

My ideal situation would be one in which one can additionally install
a version of Emacs that does *not* come bundled with any packages.
AFAICT, this project would help make that happen, so I am also in
support of that.

Best,
Radon

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Adding BBDB to Emacs core

Thomas Fitzsimmons-2
In reply to this post by Glenn Morris-3
Glenn Morris <[hidden email]> writes:

> Thomas Fitzsimmons wrote:
>
>> I'd like to see what people think about also adding it to Emacs core.
>
> Personally I think this is the opposite direction to the one in which
> Emacs components should be moving.

I understand that position in general, but I think BBDB should be an
exception for two reasons: because there are already dependencies on it
in many core packages, and because it is a library that provides a
contact management API to other packages.

Thomas

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Adding BBDB to Emacs core

Eric Abrahamsen-2
In reply to this post by Joshua Branson-4
Joshua Branson <[hidden email]> writes:

> That sounds pretty awesome, but does the bbdb package have any info
> documentation?  Not that it really matters, but it would be nice to have.
>
> Also, may I ask about ebdb?  Does ebdb ever have
> a chance at making emacs core?

Probably not! There's already push-back on including more packages in
core, and the main argument for making an exception for BBDB (many
packages are already "aware" of it) doesn't apply to EBDB.

Personally, I belong to the "don't put more stuff in core" camp, and am
happy keeping EBDB in ELPA.

E

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Adding BBDB to Emacs core

Thomas Fitzsimmons-2
In reply to this post by Joshua Branson-4
Joshua Branson <[hidden email]> writes:

> That sounds pretty awesome, but does the bbdb package have any info
> documentation?

Not currently, only a stub .texi file exists in GNU ELPA.

> Not that it really matters, but it would be nice to have.

Agreed, it would be nice to have.

Thomas

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Adding BBDB to Emacs core

Joshua Branson-4
In reply to this post by Eric Abrahamsen-2
Eric Abrahamsen <[hidden email]> writes:

> Joshua Branson <[hidden email]> writes:
>
>> That sounds pretty awesome, but does the bbdb package have any info
>> documentation?  Not that it really matters, but it would be nice to have.
>>
>> Also, may I ask about ebdb?  Does ebdb ever have
>> a chance at making emacs core?
>
> Probably not! There's already push-back on including more packages in
> core, and the main argument for making an exception for BBDB (many
> packages are already "aware" of it) doesn't apply to EBDB.
>
> Personally, I belong to the "don't put more stuff in core" camp, and am
> happy keeping EBDB in ELPA.

That makes sense.  The more packages you have, the harder it must be to
maintain a stable emacs.

>
> E

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Adding BBDB to Emacs core

Bozhidar Batsov
You can add my voice to "I'd rather just have in ELPA (and we should be moving more and more packages there).

There was some cool idea a while ago to have even Emacs stable releases package some core deps directly from ELPA (similar to
what languages like Ruby do). That simplifies the maintenance of Emacs itself and gives more flexibility to users to update stuff while waiting
for new Emacs releases. 

On 15 April 2018 at 01:46, Joshua Branson <[hidden email]> wrote:
Eric Abrahamsen <[hidden email]> writes:

> Joshua Branson <[hidden email]> writes:
>
>> That sounds pretty awesome, but does the bbdb package have any info
>> documentation?  Not that it really matters, but it would be nice to have.
>>
>> Also, may I ask about ebdb?  Does ebdb ever have
>> a chance at making emacs core?
>
> Probably not! There's already push-back on including more packages in
> core, and the main argument for making an exception for BBDB (many
> packages are already "aware" of it) doesn't apply to EBDB.
>
> Personally, I belong to the "don't put more stuff in core" camp, and am
> happy keeping EBDB in ELPA.

That makes sense.  The more packages you have, the harder it must be to
maintain a stable emacs.

>
> E


Reply | Threaded
Open this post in threaded view
|

Re: RFC: Adding BBDB to Emacs core

Phillip Lord-3
In reply to this post by Glenn Morris-3
Glenn Morris <[hidden email]> writes:

>> I'd like BBDB to become the default out-of-the-box local contact
>> management library for GNU Emacs, in particular so that Gnus/EUDC/BBDB
>> work together to provide email completion and snarfing out-of-the-box,
>> without extra configuration or package installation.
>
> If GNU ELPA is a first-class citizen, then all the above can happen
> without adding yet more stuff to the main Emacs repo. (Wistfully
> thinking here yet again of the project to bundle GNU ELPA packages with
> Emacs releases...)

I've part-written two different versions of this, both in git.

They work in different ways; but ultimately, I think we need to decide
what "ELPA as a first-class citizen" actually means.

This version:

http://git.savannah.gnu.org/cgit/emacs.git/log/?h=elparized-core

for example, just pulls out parts of ELPA using git magic, and copies
the files into core. Simple, straight-forward and it works. But,
ultimately, will it make maintaining core more easy? In the end, I think
not, because it is essentially an ad-hoc way of tying together emacs.git
and elpa.git.


This version:

http://git.savannah.gnu.org/cgit/emacs.git/log/?h=feature/integrated-elpa

uses package.el during the build process of Emacs, so that ELPA packages
could be added as packages. It requires more work. In the end, my own
feeling is that this is the right way. We could dramatically slim down
core Emacs to be enough to run package.el. The release would then be
"core plus what ever packages we think are important at the time".

This would decrease the complexity of the emacs git. But it might
increase the complexity of the release process, since you'd be dependent
on multiple other packages. I think ELPA and package.el need
to be able to cope with multiple versions of the same package,
supporting different versions of Emacs for this to work.


Phil

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Adding BBDB to Emacs core

Michael Welsh Duggan-5
[hidden email] (Phillip Lord) writes:

> This version:
>
> http://git.savannah.gnu.org/cgit/emacs.git/log/?h=elparized-core
>
> for example, just pulls out parts of ELPA using git magic, and copies
> the files into core. Simple, straight-forward and it works. But,
> ultimately, will it make maintaining core more easy? In the end, I think
> not, because it is essentially an ad-hoc way of tying together emacs.git
> and elpa.git.
>
>
> This version:
>
> http://git.savannah.gnu.org/cgit/emacs.git/log/?h=feature/integrated-elpa
>
> uses package.el during the build process of Emacs, so that ELPA packages
> could be added as packages. It requires more work. In the end, my own
> feeling is that this is the right way. We could dramatically slim down
> core Emacs to be enough to run package.el. The release would then be
> "core plus what ever packages we think are important at the time".
>
> This would decrease the complexity of the emacs git. But it might
> increase the complexity of the release process, since you'd be dependent
> on multiple other packages. I think ELPA and package.el need
> to be able to cope with multiple versions of the same package,
> supporting different versions of Emacs for this to work.

I would support any method that does _not_ require a working internet
connection in order to install packages.  I have had to work on Emacs in
many, many places that have limited or non-existent network connections.
The ability to "move a package to external medium," then "install from
medium as a 'package' maintained by package-mode" would be very nice to
have.

This may not actually have anything to do with the text I quoted, but I
would like to put it out there anyway, just so it is in the back of
implementers' minds.

--
Michael Welsh Duggan
([hidden email])

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Adding BBDB to Emacs core

John Wiegley-6
In reply to this post by Bozhidar Batsov
>>>>> "BB" == Bozhidar Batsov <[hidden email]> writes:

BB> You can add my voice to "I'd rather just have in ELPA (and we should be
BB> moving more and more packages there).

Same here. I really don't want to see BBDB moved into core. I do want ELPA
packages to become more "first class" than they are now, so that the desire to
add such packages to core would no longer have the same appeal.

--
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Adding BBDB to Emacs core

Stefan Monnier
In reply to this post by Michael Welsh Duggan-5
> I would support any method that does _not_ require a working internet
> connection in order to install packages.

We're discussing something quite different, tho: the proposal would be
something like "also clone elpa.git when you clone emacs.git", or "fetch
some packages from elpa.git while building the Emacs tarball".

> I have had to work on Emacs in
> many, many places that have limited or non-existent network connections.
> The ability to "move a package to external medium," then "install from
> medium as a 'package' maintained by package-mode" would be very nice to
> have.

There's `package-install-file` for that.


        Stefan


Reply | Threaded
Open this post in threaded view
|

Re: RFC: Adding BBDB to Emacs core

Thomas Fitzsimmons-2
In reply to this post by John Wiegley-6
"John Wiegley" <[hidden email]> writes:

>>>>>> "BB" == Bozhidar Batsov <[hidden email]> writes:
>
> BB> You can add my voice to "I'd rather just have in ELPA (and we should be
> BB> moving more and more packages there).
>
> Same here. I really don't want to see BBDB moved into core. I do want ELPA
> packages to become more "first class" than they are now, so that the desire to
> add such packages to core would no longer have the same appeal.

I think the end result from the perspective of core Emacs maintainers'
maintenance burden would not be different in beneficial ways than BBDB
just being in core.

Quoting Stefan later in the thread, the (1) "fetch some packages from
elpa.git while building the Emacs tarball" method may allow for nice
out-of-the-box BBDB integration for an Emacs release, and solve the
problems I'm trying to solve for users of Emacs major release tarballs
(but not users/developers who build the Emacs they use out of git, and
I'd worry about last-minute integration of all this stuff -- who makes
sure it all works together, at what point before release?).

But method (1) wouldn't solve the EUDC package maintenance aspects for
me (EUDC requiring something not in the tree).  For that, we'd need
Stefan's solution (2) "also clone elpa.git when you clone emacs.git"
solution.  That may solve both cases if it's done completely, see below.

To get the same benefits for BBDB as it being in core, I'd want solution
(2) to ensure that core maintainers always clone BBDB into their tree,
so that they always build it.  Then they can check for compile errors,
and usage of new features, as they change the core parts of Emacs around
BBDB.  I get very useful patches to EUDC from the core maintainers from
time to time even though they don't know EUDC functionality or
internals; I would hope that with solution (2) I'd still get those types
of patches.

Assuming (2) achieves all that, from the core maintainer's perspective,
what's the difference?  The downside is they have two repos to deal
with, and the interactions between the two to always consider (e.g., do
we branch all of ELPA to match Emacs branches (probably not), or write
all ELPA packages to work on any Emacs branch (probably), etc.).

Or is there some other way of bumping ELPA packages up to first class
status that you're envisioning?  I'm willing to help experiment with
different approaches to solve EUDC/BBDB issues, FWIW, but as yet I can't
envision the end result.

(Philosophical aside: I really don't want to see Emacs major releases
become just an Elisp language runtime, class libraries and package
management.  That would be sad.  Emacs is special, not just another
language environment.  Package discovery via GNU ELPA (over the network)
just isn't the same as feature discovery within the running Emacs
instance -- there's always an extra level of annoyance, network access
and configuration associated with external packages.  I'm hoping that by
Emacs 27.1 I'll have a window manager in my text editor.

Maybe Emacs could do like GNU/Linux distributions and publish e.g.,
emacs-minimal-27.1.tar.gz alongside emacs-27.1.tar.gz though.)

Thomas

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Adding BBDB to Emacs core

Achim Gratz
In reply to this post by Phillip Lord-3
Phillip Lord writes:
> http://git.savannah.gnu.org/cgit/emacs.git/log/?h=feature/integrated-elpa
>
> uses package.el during the build process of Emacs, so that ELPA packages
> could be added as packages. It requires more work. In the end, my own
> feeling is that this is the right way. We could dramatically slim down
> core Emacs to be enough to run package.el. The release would then be
> "core plus what ever packages we think are important at the time".

I think the point of contention previously was not from people using Git
anyway for all their builds, but from folks attuned to using tarballs.
So I think there'd need to be a way to create a tarball that looks like
a proper Git checkout of the release, probably with some info put
somewhere to record the exact state of these various checkouts at the
time of the archive creation and a build process that picks those up as
appropriate.

My personal preference would be if everything that isn't needed to
bootstrap Emacs moved to ELPA so that most of Emacs could be kept
up-to-date via the package manager.


Regards,
Achim.
--
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds


Reply | Threaded
Open this post in threaded view
|

Re: RFC: Adding BBDB to Emacs core

Eli Zaretskii
> From: Achim Gratz <[hidden email]>
> Date: Mon, 16 Apr 2018 19:09:24 +0200
>
> My personal preference would be if everything that isn't needed to
> bootstrap Emacs moved to ELPA so that most of Emacs could be kept
> up-to-date via the package manager.

Beware: XEmacs tried going that way, and failed, in that most users
ended up installing the "sumo" package.  Let's not repeat past
mistakes, mmmkay?

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Adding BBDB to Emacs core

Achim Gratz
Eli Zaretskii writes:
> Beware: XEmacs tried going that way, and failed, in that most users
> ended up installing the "sumo" package.  Let's not repeat past
> mistakes, mmmkay?

That was a different time when "most" users probably didn't have
always-on internet access.  Again, you could have a "curated" set of
specially blessed ELPA packages if you wanted to and even provide it as a
meta package via ELPA.


Regards,
Achim.
--
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra


Reply | Threaded
Open this post in threaded view
|

Re: RFC: Adding BBDB to Emacs core

Stefan Monnier
In reply to this post by Thomas Fitzsimmons-2
Before deciding on this, I think we'd need a clear picture of the ways
in which BBDB is needed/used by EUDC.

I see you saying things like "EUDC requiring something not in the tree",
but I don't see anything in EUDC that really requires BBDB, instead
I just see glue code that lets you use BBDB when it's available, just
like there's code that lets you use LDAP when it's available (but
there's clearly no corresponding push to try and add LDAP directly into
Emacs).

Could you give us a clear description of how BBDB is used by EUDC?
I'm thinking of a list of BBDB functions/variables that are used by
EUDC, along with a description of what they are used for (not what they
do themselves, but what EUDC uses them for).

Also good would be to describe concrete ways in which having BBDB in
Emacs itself would improve the user's experience (presumably for users
which don't already use BBDB).


        Stefan


Reply | Threaded
Open this post in threaded view
|

Re: RFC: Adding BBDB to Emacs core

Eric Abrahamsen-2
Stefan Monnier <[hidden email]> writes:

> Before deciding on this, I think we'd need a clear picture of the ways
> in which BBDB is needed/used by EUDC.
>
> I see you saying things like "EUDC requiring something not in the tree",
> but I don't see anything in EUDC that really requires BBDB, instead
> I just see glue code that lets you use BBDB when it's available, just
> like there's code that lets you use LDAP when it's available (but
> there's clearly no corresponding push to try and add LDAP directly into
> Emacs).

I'm also curious about this, as I've started implementing EUDC support
for EBDB. On "my" side, I'm seeing `eudc-register-protocol', and a bunch
of calls to `eudc-protocol-set'. Shouldn't those thing be enough? If
EUDC provides a generalized framework, shouldn't the whole point be that
it doesn't need to know about its implementations?

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Adding BBDB to Emacs core

Roland Winkler-5
In reply to this post by John Wiegley-6
On Sun, Apr 15 2018, John Wiegley wrote:
>>>>>> "BB" == Bozhidar Batsov <[hidden email]> writes:
>
> BB> You can add my voice to "I'd rather just have in ELPA (and we
> BB> should be moving more and more packages there).
>
> Same here. I really don't want to see BBDB moved into core. I do want
> ELPA packages to become more "first class" than they are now, so that
> the desire to add such packages to core would no longer have the same
> appeal.

In general I tend to agree.  Yet it appears to me that so far there is
one problem with GNU ELPA: I believe it would help to have a better
separation between ELPA being the repository for the distribution of
stable versions of packages to normal users and ELPA being the
repository for the development of such packages.

With core Emacs, we have a master branch with the latest and sometimes
buggy code.  And occasionally there is a proper release with
significant efforts that these releases work reliably for normal users.
So far ELPA does not support such a model.

- Take BBDB as an example: It is not too difficult to break a user's
database by trying to improve some of BBDB's internal functions.  That's
why right now I prefer to continue to use BBDB's repository on savannah
for code development.  When a code change appears to be sufficiently
stable, it is also added to BBDB in ELPA.

Certainly, this is a non-ideal approach.  I am wondering: Could it make
sense to have a separate infrastructure like "ELPA-devel" for the
on-going development of ELPA packages, where adventurous users find the
very latest code similar to the master branch of core Emacs.  On the
other hand, ELPA becomes the place for stable versions of these
packages.  My terminology "separate infrastructure" is intentionally
vague, because I do not yet have a clear idea how this can be
implemented efficiently.

A simple scheme could be a policy for naming development and release
branches of packages in the ELPA git repository (beyond the current
"externals" branches for individual ELPA packages), combined with tools
that allow one to access one or the other version of a package.

Or yet something different.

Roland

123