Continuous integration

classic Classic list List threaded Threaded
86 messages Options
12345
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Continuous integration

Andreas Politz, INF|INF-I-2

(Sorry for throwing around buzz words here.)

Michael and I are trying to fix some errors with file-notification.
None of us has all the participating libraries and OS at hand, which
means we are unable to test any patches thoroughly.

So, I wondered whether it be possible to run the test-suite
automatically for every commit/branch/OS.  The results could be
published via mailing-list.

What do you think ?

-ap

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Continuous integration

Phillip Lord-3
On Tue, March 21, 2017 3:45 pm, Andreas Politz wrote:
>

> (Sorry for throwing around buzz words here.)
>
>
> Michael and I are trying to fix some errors with file-notification.
> None of us has all the participating libraries and OS at hand, which
> means we are unable to test any patches thoroughly.
>
> So, I wondered whether it be possible to run the test-suite
> automatically for every commit/branch/OS.  The results could be published
> via mailing-list.
>
> What do you think ?


To some extent it's already there.

https://hydra.nixos.org/jobset/gnu/emacs-trunk/all

It doesnt work for every branch though, which is unfortunate, and I think
this is something that Emacs devel would really benefit from.


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Continuous integration

Michael Albinus
"Phillip Lord" <[hidden email]> writes:

>> Michael and I are trying to fix some errors with file-notification.
>> None of us has all the participating libraries and OS at hand, which
>> means we are unable to test any patches thoroughly.
>>
>> So, I wondered whether it be possible to run the test-suite
>> automatically for every commit/branch/OS.  The results could be published
>> via mailing-list.
>>
>> What do you think ?
>
>
> To some extent it's already there.
>
> https://hydra.nixos.org/jobset/gnu/emacs-trunk/all
>
> It doesnt work for every branch though, which is unfortunate, and I think
> this is something that Emacs devel would really benefit from.

hydra is helpful, although I've stopped reading the mailing list due to
too many false reports. Maybe the situation has been improved last
weeks?

The other point is that it focuses on GNU/Linux. This is OK, but it
doesn't help us with our recent problem, running tests for kqueue on
*BSD and/or OS X.

Best regards, Michael.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Continuous integration

Phillip Lord-3
On Tue, March 21, 2017 7:46 pm, Michael Albinus wrote:

> "Phillip Lord" <[hidden email]> writes:
>
>> To some extent it's already there.
>>
>>
>> https://hydra.nixos.org/jobset/gnu/emacs-trunk/all
>>
>>
>> It doesnt work for every branch though, which is unfortunate, and I
>> think this is something that Emacs devel would really benefit from.
>
> hydra is helpful, although I've stopped reading the mailing list due to
> too many false reports. Maybe the situation has been improved last weeks?

I don't think so. It's a vicious circle, unfortunately. If not enough
people check it, then its more likely to be broken.

> The other point is that it focuses on GNU/Linux. This is OK, but it
> doesn't help us with our recent problem, running tests for kqueue on *BSD
> and/or OS X.

I agree with this. I had a look at buildbot and got it working cleanly (on
gnu/linux). It supports BSD, MacOS and windows workers also, as well as
build-any-branch semantics.

From my own perspective, being able to start multi-OS CI builds on any
branch would be a very good addition to Emacs.

Phil


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Continuous integration

Michael Albinus
"Phillip Lord" <[hidden email]> writes:

>> hydra is helpful, although I've stopped reading the mailing list due to
>> too many false reports. Maybe the situation has been improved last weeks?
>
> I don't think so. It's a vicious circle, unfortunately. If not enough
> people check it, then its more likely to be broken.

It's even worse now. For failed test runs, there is no log file anymore.
This makes it useless.

> Phil

Best regards, Michael.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Continuous integration

Toon Claes-2
In reply to this post by Andreas Politz, INF|INF-I-2
Andreas Politz <[hidden email]> writes:

> So, I wondered whether it be possible to run the test-suite
> automatically for every commit/branch/OS.  The results could be
> published via mailing-list.

Some time ago Ted Zlatanov proposed to use GitLab to improve the
development process:

http://lists.gnu.org/archive/html/emacs-devel/2016-07/msg00937.html

GitLab could take care of running CI, because it runs CI when commits
gets pushed to it.

The GitLab Runner can be installed on a large number of platforms:

https://docs.gitlab.com/runner/install/

And the builds logs will be available on the GitLab installation.

I know several people on this list are not familiar with
GitLab/GitHub/BitBucket, that's why Ted asked
[hidden email] if it was possible to run a GitLab
installation on FSF/GNU hardware, but I've never heard anything else
from it.

http://lists.gnu.org/archive/html/emacs-devel/2016-07/msg01133.html

I think it could be really interesting to give GitLab a try in the Emacs
development workflow. And I am also willing to help to set this up.


-- Toon

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Continuous integration

Thien-Thi Nguyen-3

() Toon Claes <[hidden email]>
() Wed, 22 Mar 2017 09:46:40 +0100 (CET)

   Some time ago Ted Zlatanov proposed to use GitLab to improve
   the development process

(tangent) I tried to create GitLab account several times but it
gave me a 422 error (w/o further explanation) each time.  What's
the probem, i wonder?  My creds ain't good enough, i suppose...

(on topic) I wanted the GitLab account to be able to push (and
thus work on, publicly, using the nice GitLab facilities) ZOW:

 http://www.gnuvola.org/software/zow/

which can be used to whip up CI/CD for Emacs (or anything,
really) in Emacs Lisp.  (Admit it, you saw it coming! :-D)

--
Thien-Thi Nguyen -----------------------------------------------
 (defun responsep (query)
   (pcase (context query)
     (`(technical ,ml) (correctp ml))
     ...))                              748E A0E8 1CB8 A748 9BFA
--------------------------------------- 6CE4 6703 2224 4C80 7502

signature.asc (203 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Continuous integration

Ted Zlatanov
In reply to this post by Toon Claes-2
On Wed, 22 Mar 2017 09:46:40 +0100 (CET) Toon Claes <[hidden email]> wrote:

TC> Some time ago Ted Zlatanov proposed to use GitLab to improve the
TC> development process:

TC> http://lists.gnu.org/archive/html/emacs-devel/2016-07/msg00937.html

TC> GitLab could take care of running CI, because it runs CI when commits
TC> gets pushed to it.

Absolutely. I think the benefits reach beyond that--especially if a pull
request workflow could be set up. Right now it's "push into branch; ask
for comments" which is delightfully retro. Together with per-branch CI
(so the changes on the branch can be tested before they are merged, as
opposed to post-merge) this could result in a greatly improved developer
experience.

(Hydra is a good service, but it doesn't offer that level of integration
currently, and I think it would be a bit harder to set that up.)

TC> I know several people on this list are not familiar with
TC> GitLab/GitHub/BitBucket, that's why Ted asked
TC> [hidden email] if it was possible to run a GitLab
TC> installation on FSF/GNU hardware, but I've never heard anything else
TC> from it.

TC> http://lists.gnu.org/archive/html/emacs-devel/2016-07/msg01133.html

Also note the recent discussion about why the Docker Hub web site's
Javascript usage made the Docker Hub service unacceptable. I hope we
don't waste time on discussing a GitLab installation if it doesn't fit
that specific requirement (since it runs a web server).

TC> I think it could be really interesting to give GitLab a try in the Emacs
TC> development workflow. And I am also willing to help to set this up.

Same here.

On Wed, 22 Mar 2017 13:16:39 +0100 Thien-Thi Nguyen <[hidden email]> wrote:

TN> (tangent) I tried to create GitLab account several times but it
TN> gave me a 422 error (w/o further explanation) each time.  What's
TN> the probem, i wonder?  My creds ain't good enough, i suppose...

Oh, you mean the GitLab hosted CI/CD accounts on gitlab.com. Toon and I
are proposing something different: a FSF/GNU hosted installation of the
GPL-ed GitLab software on local hardware.

Ted


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Continuous integration

Alex Dunn
Ted Zlatanov <[hidden email]> writes:

> Also note the recent discussion about why the Docker Hub web site's
> Javascript usage made the Docker Hub service unacceptable. I hope we
> don't waste time on discussing a GitLab installation if it doesn't fit
> that specific requirement (since it runs a web server).

Other self-hosted options are https://gitea.io/en-US/ and
https://gogs.io; both also use a lot of JS but as far as I can tell it’s
all free.

They don’t have built-in CI, though, so that would probably require a
separate dedicated CI server like Jenkins.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Continuous integration

Thien-Thi Nguyen-3
In reply to this post by Ted Zlatanov

() Ted Zlatanov <[hidden email]>
() Wed, 22 Mar 2017 09:14:58 -0400

   TN> (tangent) [...] create GitLab account

   Oh, you mean the GitLab hosted CI/CD accounts on gitlab.com.

Yes.

   Toon and I are proposing something different: a FSF/GNU
   hosted installation of the GPL-ed GitLab software on local
   hardware.

I see.  Well, i hope ZOW can fill the gap until that time comes.
Certainly, it runs on my (local) machine, which has nowhere near
the requisite amount of RAM for a GitLab instance...

--
Thien-Thi Nguyen -----------------------------------------------
 (defun responsep (query)
   (pcase (context query)
     (`(technical ,ml) (correctp ml))
     ...))                              748E A0E8 1CB8 A748 9BFA
--------------------------------------- 6CE4 6703 2224 4C80 7502

signature.asc (203 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Continuous integration

Toon Claes-2
In reply to this post by Ted Zlatanov
Ted Zlatanov <[hidden email]> writes:
> Absolutely. I think the benefits reach beyond that--especially if a pull
> request workflow could be set up.

Let's not try to change too many thing at once. But I've been working
with a Pull Request/Merge Request workflow for quite some time now, and
I like it!

> Also note the recent discussion about why the Docker Hub web site's
> Javascript usage made the Docker Hub service unacceptable. I hope we
> don't waste time on discussing a GitLab installation if it doesn't fit
> that specific requirement (since it runs a web server).

The javascript on GitLab is free, but at the moment it is not compatible
with LibreJS. There is an issue about this, but there isn't much
progress on it:

https://gitlab.com/gitlab-org/gitlab-ce/issues/15621

If that is a blocking issue, we should trigger the GitLab team so maybe
it will get a higher priority.

> Oh, you mean the GitLab hosted CI/CD accounts on gitlab.com. Toon and I
> are proposing something different: a FSF/GNU hosted installation of the
> GPL-ed GitLab software on local hardware.

Yes, as far as I understand, the FSF/GNU community does not like relying
on third party hosting, so that why we are suggesting to install a
self-hosted GitLab instance on FSF/GNU systems.

Maybe as temporary solution, we might put a git mirror on GitLab.com and
set up GitLab CI to see if we can get it to run the tests.


-- Toon

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Continuous integration

Toon Claes-2
In reply to this post by Alex Dunn
Alex <[hidden email]> writes:

> Other self-hosted options are https://gitea.io/en-US/ and
> https://gogs.io; both also use a lot of JS but as far as I can tell it’s
> all free.

Even if the javascript is free, we should verify if it works properly
with LibreJS.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Continuous integration

Eli Zaretskii
In reply to this post by Ted Zlatanov
> From: Ted Zlatanov <[hidden email]>
> Date: Wed, 22 Mar 2017 09:14:58 -0400
>
> Absolutely. I think the benefits reach beyond that--especially if a pull
> request workflow could be set up. Right now it's "push into branch; ask
> for comments" which is delightfully retro.

Actually, it's more like "obtain write access, then push your changes
directly".

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Continuous integration

Toon Claes-2
Eli Zaretskii <[hidden email]> writes:

>> From: Ted Zlatanov <[hidden email]>
>> Date: Wed, 22 Mar 2017 09:14:58 -0400
>>
>> Absolutely. I think the benefits reach beyond that--especially if a pull
>> request workflow could be set up. Right now it's "push into branch; ask
>> for comments" which is delightfully retro.
>
> Actually, it's more like "obtain write access, then push your changes
> directly".

A platform like GitLab relies heavily on pushing code to feature
branches, and merge to master when someone approves. I know this is a
big change compared to the current workflow, that's why I am proposing
to start of by setting up CI (which will work with either workflow).

-- Toon

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Continuous integration

Richard Stallman
In reply to this post by Thien-Thi Nguyen-3
[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > (tangent) I tried to create GitLab account several times but it
  > gave me a 422 error (w/o further explanation) each time.  What's
  > the probem, i wonder?  My creds ain't good enough, i suppose...

That could be a serious flaw in GitLab, if it is a policy
rather than a bug.

What sort of "creds" are they checking?  What do they _say_ a person
needs in order to make an account?

--
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Continuous integration

Phillip Lord-3
In reply to this post by Ted Zlatanov
Ted Zlatanov <[hidden email]> writes:

> On Wed, 22 Mar 2017 09:46:40 +0100 (CET) Toon Claes <[hidden email]> wrote:
>
> TC> Some time ago Ted Zlatanov proposed to use GitLab to improve the
> TC> development process:
>
> TC> http://lists.gnu.org/archive/html/emacs-devel/2016-07/msg00937.html
>
> TC> GitLab could take care of running CI, because it runs CI when commits
> TC> gets pushed to it.
>
> Absolutely. I think the benefits reach beyond that--especially if a pull
> request workflow could be set up. Right now it's "push into branch; ask
> for comments" which is delightfully retro. Together with per-branch CI
> (so the changes on the branch can be tested before they are merged, as
> opposed to post-merge) this could result in a greatly improved developer
> experience.

I'd certainly agree with this. Inline code review would be nice also,
especially given that Emacs git is set up for non-squashing.


> TC> I know several people on this list are not familiar with
> TC> GitLab/GitHub/BitBucket, that's why Ted asked
> TC> [hidden email] if it was possible to run a GitLab
> TC> installation on FSF/GNU hardware, but I've never heard anything else
> TC> from it.
>
> TC> http://lists.gnu.org/archive/html/emacs-devel/2016-07/msg01133.html
>
> Also note the recent discussion about why the Docker Hub web site's
> Javascript usage made the Docker Hub service unacceptable. I hope we
> don't waste time on discussing a GitLab installation if it doesn't fit
> that specific requirement (since it runs a web server).

If we are going to test builds across different platforms then, it is
worth noting, that we technically have to use non-free software. We
cannot test the windows/macos builds without it. I don't know whether
this is a problem or not; it is not dissimilar from building and
distributing Emacs windows binaries.

I mentioned earlier that I've also tried buildbot. It works, the
building is nice, and it's just CI. It would plugin to the existing
savannah infrastructure, so might be less distruptive. The web front end
does not, however, work with librejs.


Phil

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Continuous integration

Phillip Lord-3
In reply to this post by Toon Claes-2
Toon Claes <[hidden email]> writes:

> Ted Zlatanov <[hidden email]> writes:
>> Absolutely. I think the benefits reach beyond that--especially if a pull
>> request workflow could be set up.
>
> Yes, as far as I understand, the FSF/GNU community does not like relying
> on third party hosting, so that why we are suggesting to install a
> self-hosted GitLab instance on FSF/GNU systems.
>
> Maybe as temporary solution, we might put a git mirror on GitLab.com and
> set up GitLab CI to see if we can get it to run the tests.

That should work, but is a slight pain to mirror since the CI
configuration is committed.

Phil

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Continuous integration

raman
In reply to this post by Phillip Lord-3
Would be nice to target a tool that is usable from within Emacs  --
rather than dumbing things down to a What You See Is All You Have
browser environment.

Is it safe to assume that people developing Emacs are using Emacs as an
editor?

Browser-based tools get inflicted on one in a heterogeneous  environment
where different individuals use a variety of editors, platforms and
machine types --- in this instance --- emacs-Development --- it would be
nice to leverage the strenghts of Emacs if it is indeed a common
component of our varied environments.
--

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Continuous integration

Phillip Lord-3
raman <[hidden email]> writes:

> Would be nice to target a tool that is usable from within Emacs  --
> rather than dumbing things down to a What You See Is All You Have
> browser environment.

To an extent, although, probably not to the extent that it is usuable
everywhere else. Nor to the extent that we have to develop something
specific rather than using software that already exists.

> Is it safe to assume that people developing Emacs are using Emacs as an
> editor?

As an editor, but not necessarily for everything else.


> Browser-based tools get inflicted on one in a heterogeneous  environment
> where different individuals use a variety of editors, platforms and
> machine types --- in this instance --- emacs-Development --- it would be
> nice to leverage the strenghts of Emacs if it is indeed a common
> component of our varied environments.

Both of the tools that have been mentioned (gitlab and buildbot) have
IRC interfaces, so that could be contacted this way. Another solution
would be to extend them to serve up an org file, with hyperlinks through
to logs, and so forth.

AFAICT, the current solution (hydra) doesn't offer an emacs front end.

Phil

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Continuous integration

raman
As long as we dont end up doing things like diff/merge  of source code
inside a browser window, I'll stay mostly happy.

Phillip Lord writes:
 > raman <[hidden email]> writes:
 >
 > > Would be nice to target a tool that is usable from within Emacs  --
 > > rather than dumbing things down to a What You See Is All You Have
 > > browser environment.
 >
 > To an extent, although, probably not to the extent that it is usuable
 > everywhere else. Nor to the extent that we have to develop something
 > specific rather than using software that already exists.
 >
 > > Is it safe to assume that people developing Emacs are using Emacs as an
 > > editor?
 >
 > As an editor, but not necessarily for everything else.
 >
 >
 > > Browser-based tools get inflicted on one in a heterogeneous  environment
 > > where different individuals use a variety of editors, platforms and
 > > machine types --- in this instance --- emacs-Development --- it would be
 > > nice to leverage the strenghts of Emacs if it is indeed a common
 > > component of our varied environments.
 >
 > Both of the tools that have been mentioned (gitlab and buildbot) have
 > IRC interfaces, so that could be contacted this way. Another solution
 > would be to extend them to serve up an org file, with hyperlinks through
 > to logs, and so forth.
 >
 > AFAICT, the current solution (hydra) doesn't offer an emacs front end.
 >
 > Phil

--

--

12345
Loading...