org-merger questions

classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

org-merger questions

Rasmus-13
Hi,

As discussed, Org urgently needs to be merged into Emacs.  I have started
a scratch branch with v9.0.8, although I haven’t pushed it yet.

So far, I do the following changes from the Org release tarball:

- Necessary white-space cleanup, required by probably some git hook to let
  me commit.  We will have to do this on the Org repo as well.
- Hardcode version in org.texi to remove dependency of "org-version.inc".
- Include org-version.el (normally generated at compile-time)
- Include org-loaddefs.el (normally generated at compile-time)

I have a couple of questions.

* Schemas

ox-odt.el needs to be able to find schemas files.  By default it looks
into <Emacs>/etc/schemas and some folders computed from
‘org-odt-data-dir’, which is normally computed at compile-time.  That is
not so nice, and it anyway doesn’t work with some packaging systems such
as "makepkg".

In principal, I’d be happy to just throw the 3 Org schemas files into
<Emacs>/etc/schemas but it already contain an schemas.xml file.  Does
anyone know if the two schemas files can be merged?  Here’s the two files:

    http://orgmode.org/cgit.cgi/org-mode.git/tree/etc/schema/schemas.xml
    http://git.savannah.gnu.org/cgit/emacs.git/tree/etc/schema/schemas.xml


I could also add a quick "fix" to ox-odt.el to look into
<Emacs>/etc/org/schemas for now.

* Compiling

Org seems to mostly "just work" when compiling without the specific Org
Makefiles.

Achim:
Are there any reasons to think that Emacs-core would need to include the
Org Makefiles assuming org-version.el and org-loaddefs.el is "statically"
included, as currently are for Org 8.3?  The "mk" folder is not included
in Emacs-core now.  I assume there is still no reason to do so.

Thanks,
Rasmus

--
Send from my Emacs


Reply | Threaded
Open this post in threaded view
|

Re: org-merger questions

Kaushal Modi
This doesn't have to do with your email directly. 

But on the topic of tests, org 9.0.8 will fail 2 tests on emacs master. The fix is already in org maint branch, post 9.0.8: http://orgmode.org/cgit.cgi/org-mode.git/commit/?h=maint&id=3bcfed071deca0e9c5e8dd298b28bfd5903c6f9a

Just saying in case someone comes across those 2 test failures:

   FAILED  test-org/custom-properties
   FAILED  test-org/forward-paragraph

So do not worry about those 2 as they will be fixed after the next minor release of org.
--

Kaushal Modi

Reply | Threaded
Open this post in threaded view
|

Re: org-merger questions

Kyle Meyer
In reply to this post by Rasmus-13
Rasmus <[hidden email]> writes:

> As discussed, Org urgently needs to be merged into Emacs.  I have started
> a scratch branch with v9.0.8, although I haven’t pushed it yet.
>
> So far, I do the following changes from the Org release tarball:
>
> - Necessary white-space cleanup, required by probably some git hook to let
>   me commit.  We will have to do this on the Org repo as well.
> - Hardcode version in org.texi to remove dependency of "org-version.inc".

The emacs-sync branch in the Org repo contains the changes above, as
well as a few other Emacs-specific changes that should not be in Org's
maint branch but should be in the synced files.

Even if you don't sync from emacs-sync rather than release_9.0.8,
running

    git diff release_9.0.8..origin/emacs-sync

in the Org repo can help you see the other Emacs-specific changes that
you shouldn't overwrite.

--
Kyle

Reply | Threaded
Open this post in threaded view
|

Re: org-merger questions

Bastien Guerry-5
In reply to this post by Kaushal Modi
Kaushal Modi <[hidden email]> writes:

> So do not worry about those 2 as they will be fixed after the next
> minor release of org.

The idea would be to release 9.0.9 from maint just before the merge
to merge a clean version.

--
 Bastien

Reply | Threaded
Open this post in threaded view
|

Re: org-merger questions

Rasmus-13
> The idea would be to release 9.0.9 from maint just before the merge
> to merge a clean version.

I have pushed to scratch/org-mode-merge on Emacs.git.

I had to do a few whitespace cleanup to be able to commit.  We should
backport those, I guess.  Other than that, it seemed to compile and work
without issues on my system (GNU/Linux).

Re: schemas.xml files, the Org one contained <?xml version="1.0"?>, which
I have left out.  I added the Org location rules to bottom.

Note that http://thaiopensource.com/ns/locating-rules/1.0 isn’t
accessible.

It does not include tests for now.

Rasmus

Reply | Threaded
Open this post in threaded view
|

Re: org-merger questions

Yuri Khan-2
On Thu, Jun 22, 2017 at 5:31 PM, Rasmus <[hidden email]> wrote:

> Note that http://thaiopensource.com/ns/locating-rules/1.0 isn’t
> accessible.

XML namespace URIs are not supposed to be accessible. They are used as
opaque identifiers, guaranteed to be unique because presumably people
won’t allocate XML namespaces in domains they don’t own, and will
ensure uniqueness within domains they do own.

Reply | Threaded
Open this post in threaded view
|

Re: org-merger questions

Kyle Meyer
In reply to this post by Rasmus-13
Rasmus <[hidden email]> writes:

>> The idea would be to release 9.0.9 from maint just before the merge
>> to merge a clean version.
>
> I have pushed to scratch/org-mode-merge on Emacs.git.

Thanks.

> I had to do a few whitespace cleanup to be able to commit.  We should
> backport those, I guess.

Hmm, I was hoping I had already caught most of them, but maybe the
issues you encountered were in the few files changed in
scratch/org-mode-merge that I didn't look at.  I only checked
lisp/org/*.el, lisp/doc/misc/org.texi, and etc/refcards/orgcard.tex.

A few problems in scratch/org-mode-merge:

  * scratch/org-mode-merge reverts Emacs-specific changes in
    etc/refcards/orgcard.tex.  This same mistake was made in a previous
    sync and fixed with Emacs's e90dec2be1.  Since then, there have been
    additional Emacs-specific changes added to orgcard.tex, and these
    are all included in Org's emacs-sync branch.

  * I made the mistake of not looking at ORG-NEWS for backports, so
    scratch/org-mode-merge is reverting a few Emacs commits here.  I'll
    backport these, and I can also add your ORG-NEWS formatting changes
    to the Org repo.

  * The copyright years in Org's etc/styles/README have not been kept up
    to date, so scratch/org-mode-merge is overwriting the updated years
    in Emacs's etc/org/README.

And a question:

  * Should we include the "@set DATE ..." when we replace "@include
    org-version.inc" in Emacs's org.texi?  It doesn't seem like we have
    in the past (e.g., in Emacs's 6f66f53f65).

--
Kyle

Reply | Threaded
Open this post in threaded view
|

Re: org-merger questions

Kyle Meyer
Hi Rasmus,

Kyle Meyer <[hidden email]> writes:

> Rasmus <[hidden email]> writes:
>
>> I had to do a few whitespace cleanup to be able to commit.  We should
>> backport those, I guess.
>
> Hmm, I was hoping I had already caught most of them, but maybe the
> issues you encountered were in the few files changed in
> scratch/org-mode-merge that I didn't look at.  I only checked
> lisp/org/*.el, lisp/doc/misc/org.texi, and etc/refcards/orgcard.tex.

It seems that etc/ORG-NEWS, od-manifest-schema-v1.2-os.rnc, and
od-schema-v1.2-os.rnc all required whitespace cleanups.  These cleanups
are now included in Org's maint (and cherry-picked to the emacs-sync branch).

> A few problems in scratch/org-mode-merge:

[...]

>   * I made the mistake of not looking at ORG-NEWS for backports, so
>     scratch/org-mode-merge is reverting a few Emacs commits here.  I'll
>     backport these

I've backported all the changes to ORG-NEWS in maint (and cherry-picked
them to the emacs-sync branch).

Please consider using Org files from the emacs-sync branch instead of
the release_9.0.9 tag.  The emacs-sync branch is a superset of the last
release, but it contains a few more changes that are specific for Emacs.

These are changes that should not be in the maint branch but should be
in the Emacs repo.  In other words, these are the changes that we should
make sure not to overwrite in a sync.  For example, Org org.texi has
"@include org-version.inc", but in the Emacs repo this should be "@set
VERSION 9.0.9" instead.

You can see these changes in the Org repo with

    git diff release_9.0.9..origin/emacs-sync

(This diff also includes the whitespace cleanup and ORG-NEWS backports
that were cherry-picked to emacs-sync because they are in the maint
branch after release_9.0.9 but should be in the 9.0.9 sync.)

> [...] and I can also add your ORG-NEWS formatting changes to the Org repo.

I don't understand the formatting changes made in 5cfdf8dd17 (; Fix
ORG-NEWS formatting in previous commit, 2017-06-22).  Aside from fixing
the alignment of a table that already seems correct in Org's ORG-NEWS,
all the changes seem to be space to tab conversions.  Are these
necessary?

--
Kyle

Reply | Threaded
Open this post in threaded view
|

Re: org-merger questions

Rasmus-13
In reply to this post by Kyle Meyer
Hi Kyle,

Thanks for carefully looking into this.

>> I had to do a few whitespace cleanup to be able to commit.  We should
>> backport those, I guess.
>
> Hmm, I was hoping I had already caught most of them, but maybe the
> issues you encountered were in the few files changed in
> scratch/org-mode-merge that I didn't look at.  I only checked
> lisp/org/*.el, lisp/doc/misc/org.texi, and etc/refcards/orgcard.tex.

Perhaps I did something wrong.  I used release 9.0.9, not the emacs-sync
branch.  So if there were additional changes on the emacs-sync branch on
top of 9.0.9 I won’t have picked them up.  This is the version I used:

    http://orgmode.org/cgit.cgi/org-mode.git/tag/?h=emacs-sync&id=release_9.0.9

> A few problems in scratch/org-mode-merge:
>
>   * scratch/org-mode-merge reverts Emacs-specific changes in
>     etc/refcards/orgcard.tex.  This same mistake was made in a previous
>     sync and fixed with Emacs's e90dec2be1.  Since then, there have been
>     additional Emacs-specific changes added to orgcard.tex, and these
>     are all included in Org's emacs-sync branch.

See above.

So do you think it’s best to just cherry-pick these changes back?

>   * I made the mistake of not looking at ORG-NEWS for backports, so
>     scratch/org-mode-merge is reverting a few Emacs commits here.  I'll
>     backport these, and I can also add your ORG-NEWS formatting changes
>     to the Org repo.

Note that the formatting itself is a bit weird in ORG-NEWS.  At least on
my normal Emacs setup it’s always changed when opened.  The only issue was
some dangling whitespace at the end of a couple of lines, which the Emacs
commit hook didn’t like.

>   * The copyright years in Org's etc/styles/README have not been kept up
>     to date, so scratch/org-mode-merge is overwriting the updated years
>     in Emacs's etc/org/README.

OK

> And a question:
>
>   * Should we include the "@set DATE ..." when we replace "@include
>     org-version.inc" in Emacs's org.texi?  It doesn't seem like we have
>     in the past (e.g., in Emacs's 6f66f53f65).

I don’t know.  I think we should be bound by the Emacs practice, not what
was done in the last sync aeon ago.  Principally, I think documents should
be dated.  I don’t know if it’s right in an Emacs context, as it carries
its own date.  idlwave.texi and elisp.texi are the only other dated texi
files.  

Rasmus

--
Together we will make the possible totalllly impossible!




Reply | Threaded
Open this post in threaded view
|

Re: org-merger questions

Rasmus-13
In reply to this post by Kyle Meyer
Hi Kyle,

Thanks again.  I synced your suggested changes.

Kyle Meyer <[hidden email]> writes:

> It seems that etc/ORG-NEWS, od-manifest-schema-v1.2-os.rnc, and
> od-schema-v1.2-os.rnc all required whitespace cleanups.  These cleanups
> are now included in Org's maint (and cherry-picked to the emacs-sync branch).

Indeed.  Those were the ones it complained about.

>> A few problems in scratch/org-mode-merge:
>
> [...]
>
>>   * I made the mistake of not looking at ORG-NEWS for backports, so
>>     scratch/org-mode-merge is reverting a few Emacs commits here.  I'll
>>     backport these
>
> I've backported all the changes to ORG-NEWS in maint (and cherry-picked
> them to the emacs-sync branch).
>
> Please consider using Org files from the emacs-sync branch instead of
> the release_9.0.9 tag.  The emacs-sync branch is a superset of the last
> release, but it contains a few more changes that are specific for Emacs.

Cool.  Thanks a lot.  As said in the previous post, I didn’t use this
branch.

> You can see these changes in the Org repo with
>
>     git diff release_9.0.9..origin/emacs-sync
>
> (This diff also includes the whitespace cleanup and ORG-NEWS backports
> that were cherry-picked to emacs-sync because they are in the maint
> branch after release_9.0.9 but should be in the 9.0.9 sync.)

I updated the files in question to the "emacs-sync" version.

I had to change the version in orgcard.tex as it wasn’t 9.0.9.

>> [...] and I can also add your ORG-NEWS formatting changes to the Org repo.
>
> I don't understand the formatting changes made in 5cfdf8dd17 (; Fix
> ORG-NEWS formatting in previous commit, 2017-06-22).  Aside from fixing
> the alignment of a table that already seems correct in Org's ORG-NEWS,
> all the changes seem to be space to tab conversions.  Are these
> necessary?

I must have done a mistake.  ORG-NEWS seems to always change when I open
with my normal Emacs configuration.  So I redid the whitespace cleanup
with "emacs -q" but apparently that also induced unnecessary changes.

Thanks for fixing this!

Rasmus

--
Enough with the blah blah!



Reply | Threaded
Open this post in threaded view
|

Re: org-merger questions

Kyle Meyer
Rasmus <[hidden email]> writes:

> I updated the files in question to the "emacs-sync" version.

Thanks.  Checking the diffs again, things look good to me.

> I had to change the version in orgcard.tex as it wasn’t 9.0.9.

Yes, thanks.  I think this is the only place in the Org repo where the
Org version is hardcoded into a file, and it hasn't been getting bumped
at release time.  Since we automatically generate version numbers
everywhere else, it'd be nice if we did it here too (though of course
we'd still have to manually add the version number to the synced file,
like we do for org.texi).

--
Kyle

Reply | Threaded
Open this post in threaded view
|

Re: org-merger questions

Kyle Meyer
In reply to this post by Rasmus-13
Hi Rasmus,

Rasmus <[hidden email]> writes:

[I see that most of this message is addressed by the next post, but ...]

>> And a question:
>>
>>   * Should we include the "@set DATE ..." when we replace "@include
>>     org-version.inc" in Emacs's org.texi?  It doesn't seem like we have
>>     in the past (e.g., in Emacs's 6f66f53f65).
>
> I don’t know.  I think we should be bound by the Emacs practice, not what
> was done in the last sync aeon ago.  Principally, I think documents should
> be dated.  I don’t know if it’s right in an Emacs context, as it carries
> its own date.  idlwave.texi and elisp.texi are the only other dated texi
> files.  

OK, I don't have a strong preference either way.  In the future, I'll
add the date line when I replace the "@include org-version.inc" line in
the emacs-sync branch.

--
Kyle

Reply | Threaded
Open this post in threaded view
|

Re: org-merger questions

Rasmus-13
Hi,

>> I don’t know.  I think we should be bound by the Emacs practice, not what
>> was done in the last sync aeon ago.  Principally, I think documents should
>> be dated.  I don’t know if it’s right in an Emacs context, as it carries
>> its own date.  idlwave.texi and elisp.texi are the only other dated texi
>> files.  
>
> OK, I don't have a strong preference either way.  In the future, I'll
> add the date line when I replace the "@include org-version.inc" line in
> the emacs-sync branch.

Neither do I.  If there's an Emacs policy we should follow that.
Otherwise, I think the best thing is to use the date of the release.  But
other people may think differently.  In any case, it’s a minor issue.

--
Lasciate ogni speranza o voi che entrate: siete nella mani di'machellaio


Reply | Threaded
Open this post in threaded view
|

Re: org-merger questions

Kaushal Modi
In reply to this post by Rasmus-13
On Thu, Jun 22, 2017 at 6:32 AM Rasmus <[hidden email]> wrote:
> The idea would be to release 9.0.9 from maint just before the merge
> to merge a clean version.

I have pushed to scratch/org-mode-merge on Emacs.git.

Hi Rasmus,

I have been using the org-mode-merge branch and it looks good to me.

Thanks.
--

Kaushal Modi

Reply | Threaded
Open this post in threaded view
|

Re: org-merger questions

Rasmus-13
Hi,

Kaushal Modi <[hidden email]> writes:

> I have been using the org-mode-merge branch and it looks good to me.

Thank you for testing it, Kaushal.  The branch also seems to be working
fine for me.

Rasmus
 
--
Enough with the blah blah!

Reply | Threaded
Open this post in threaded view
|

Re: org-merger questions

Bastien Guerry-5
Hi all,

I will merge the org-mode-merger branch into Emacs
master this sunday, around 12am (Paris time).

Let's continue to review this branch for a few days.

John and Eli, if there is any concern, let us know.

Thanks,

--
 Bastien

Reply | Threaded
Open this post in threaded view
|

Re: org-merger questions

John Wiegley
>>>>> Bastien Guerry <[hidden email]> writes:

> John and Eli, if there is any concern, let us know.

No concerns here, thanks for the heads up, Bastien.

--
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Reply | Threaded
Open this post in threaded view
|

Re: org-merger questions

Bastien Guerry-5
Hi John,

John Wiegley <[hidden email]> writes:

>>>>>> Bastien Guerry <[hidden email]> writes:
>
>> John and Eli, if there is any concern, let us know.
>
> No concerns here, thanks for the heads up, Bastien.

Done, thanks!  Emacs master now features Org 9.0.9.

--
 Bastien

Reply | Threaded
Open this post in threaded view
|

Emacs master now updated to Org 9.0.9 (Was: org-merger questions)

Kaushal Modi
Yay!

emacs -Q on emacs master will now feature the new Org 9.0.9.

Thanks to Rasmus, Kyle, Bastien in making this merge happen. This was a mega-merger and it happened in a seamless fashion.

Thanks to Nicolas and everyone involved in the Org development that enabled us getting Org updates with new features and fixes on timely basis.

On Mon, Jul 3, 2017 at 3:25 AM Bastien <[hidden email]> wrote:
Hi John,

John Wiegley <[hidden email]> writes:

>>>>>> Bastien Guerry <[hidden email]> writes:
>
>> John and Eli, if there is any concern, let us know.
>
> No concerns here, thanks for the heads up, Bastien.

Done, thanks!  Emacs master now features Org 9.0.9.
--

Kaushal Modi

Reply | Threaded
Open this post in threaded view
|

Re: Emacs master now updated to Org 9.0.9

Bastien Guerry-5
Indeed, thanks to you and to everyone involved.

I'm very grateful everyone has been patiently baring
with me for this task.

--
 Bastien

12