bug#33016: 26.1; (make-process ...) doesn't signal an error, when executable given as absolute Windows path does not exist

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

bug#33016: 26.1; (make-process ...) doesn't signal an error, when executable given as absolute Windows path does not exist

Klaus-Dieter Bauer
The error-handling of `make-process' for a non-existents
executable is inconsistent between invocation as a command
(on PATH) and invocation as a full path.

To reproduce, start `emacs -Q'.

Entering

    M-x eval-expression RET
      (make-process :name "test" :command '("No Such Command"))

will bring up the debugger with 

    (file-missing "Searching for program" "No such file or directory" "nosuchcommand")

However, entering

    M-x eval-expression RET 
      (make-process :name "test" :command '("c:/No Such Command"))

will merely display in the echo-area message:

    eval: Spawning child process: Invalid argument

I stumbled upon this when debugging a quick-and-dirty
script, that called a program by absolute path. When a new
version of the program changed the name of the executable
(tex2lyx2.3 -> tex2lyx), this issue occurred, and hindered
debugging the problem.

The wording of the message might indicate a 
Windows-specific issue.

regards, Klaus



In GNU Emacs 26.1 (build 1, x86_64-w64-mingw32)
 of 2018-05-30 built on CIRROCUMULUS
Repository revision: 07f8f9bc5a51f5aa94eb099f3e15fbe0c20ea1ea
Windowing system distributor 'Microsoft Corp.', version 10.0.17134
Recent messages:
(#<process test<1>> #<process test>)
Quit
Entering debugger...
Back to top level
eval: Spawning child process: Invalid argument
Quit
Type C-x 1 to delete the help window, C-M-v to scroll help.
Quit
Entering debugger...
Back to top level
read--expression: Trailing garbage following expression
Configured using:
 'configure --without-dbus --host=x86_64-w64-mingw32
 --without-compress-install 'CFLAGS=-O2 -static -g3''

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND NOTIFY ACL GNUTLS LIBXML2 ZLIB
TOOLKIT_SCROLL_BARS THREADS LCMS2

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1252

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny seq dired
dired-loaddefs format-spec rfc822 mml mml-sec password-cache epa derived
epg epg-config gnus-util rmail rmail-loaddefs mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils cl-extra
help-fns radix-tree help-mode easymenu cl-print byte-opt gv bytecomp
byte-compile cl-loaddefs cl-lib cconv debug elec-pair time-date
mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel dos-w32 ls-lisp disp-table term/w32-win w32-win
w32-vars term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote w32notify w32 lcms2 multi-tty make-network-process emacs)

Memory information:
((conses 16 100958 5519)
 (symbols 56 20557 1)
 (miscs 48 45 202)
 (strings 32 31151 1771)
 (string-bytes 1 808081)
 (vectors 16 14466)
 (vector-slots 8 493429 6988)
 (floats 8 53 305)
 (intervals 56 327 7)
 (buffers 992 14))

Reply | Threaded
Open this post in threaded view
|

bug#33016: 26.1; (make-process ...) doesn't signal an error, when executable given as absolute Windows path does not exist

Eli Zaretskii
> From: Klaus-Dieter Bauer <[hidden email]>
> Date: Thu, 11 Oct 2018 14:55:27 +0200
>
> Entering
>
>     M-x eval-expression RET
>       (make-process :name "test" :command '("No Such Command"))
>
> will bring up the debugger with
>
>     (file-missing "Searching for program" "No such file or directory" "nosuchcommand")
>
> However, entering
>
>     M-x eval-expression RET
>       (make-process :name "test" :command '("c:/No Such Command"))
>
> will merely display in the echo-area message:
>
>     eval: Spawning child process: Invalid argument
>
> I stumbled upon this when debugging a quick-and-dirty
> script, that called a program by absolute path. When a new
> version of the program changed the name of the executable
> (tex2lyx2.3 -> tex2lyx), this issue occurred, and hindered
> debugging the problem.
>
> The wording of the message might indicate a
> Windows-specific issue.

The error in the second case is Windows specific, but the
inconsistency isn't: on Unix the second case "succeeds", in that it
returns a process object without any error messages.

The error message you see in the first case is because Emacs searches
for the program along exec-path (because it is not an absolute file
name).  In the second case this search is not done, because the file
name is already absolute.

So I don't think this is a bug.



Reply | Threaded
Open this post in threaded view
|

bug#33016: 26.1; (make-process ...) doesn't signal an error, when executable given as absolute Windows path does not exist

Klaus-Dieter Bauer
On Thu, Oct 11, 2018 at 4:22 PM Eli Zaretskii <[hidden email]> wrote:
> From: Klaus-Dieter Bauer <[hidden email]>
> Date: Thu, 11 Oct 2018 14:55:27 +0200
>
> Entering
>
>     M-x eval-expression RET
>       (make-process :name "test" :command '("No Such Command"))
>
> will bring up the debugger with
>
>     (file-missing "Searching for program" "No such file or directory" "nosuchcommand")
>
> However, entering
>
>     M-x eval-expression RET
>       (make-process :name "test" :command '("c:/No Such Command"))
>
> will merely display in the echo-area message:
>
>     eval: Spawning child process: Invalid argument
>
> I stumbled upon this when debugging a quick-and-dirty
> script, that called a program by absolute path. When a new
> version of the program changed the name of the executable
> (tex2lyx2.3 -> tex2lyx), this issue occurred, and hindered
> debugging the problem.
>
> The wording of the message might indicate a
> Windows-specific issue.

The error in the second case is Windows specific, but the
inconsistency isn't: on Unix the second case "succeeds", in that it
returns a process object without any error messages.

The error message you see in the first case is because Emacs searches
for the program along exec-path (because it is not an absolute file
name).  In the second case this search is not done, because the file
name is already absolute.

So I don't think this is a bug.

Now I understand the intent of the implementation better. However, the Unix/Windows difference still seems like a bug to me.

On Unix, the elisp will succeed, but the output and exit-status of the process clarify the issue.
On Windows, a non-local exit occurs due to the resulting exception.
As example:

    (let (p)
      (setq p
        (make-process :name "test" :command '("/tmp/nosuchcommand") :buffer (current-buffer)))
      ;; -- Subsequent code never reached on Windows
      (while (process-live-p p)
        (sleep-for 0.01))
      (message "(Process exit status %d)"
        (process-exit-status p)))

So on Windows two issues occur:
  - The exception doesn't indicate what went wrong.
  - The control-flow of the Elisp program is different from Unix.

This different seems, like it may give rise to Windows-specific bugs, that would be unnecessarily hard to debug.

Then again, calling programs by full path is probably rare, so it's probably a pretty low-priority issue.

- Klaus

 
Reply | Threaded
Open this post in threaded view
|

bug#33016: 26.1; (make-process ...) doesn't signal an error, when executable given as absolute Windows path does not exist

Eli Zaretskii
> From: Klaus-Dieter Bauer <[hidden email]>
> Date: Fri, 19 Oct 2018 10:03:00 +0200
> Cc: [hidden email]
>
> On Unix, the elisp will succeed, but the output and exit-status of the process clarify the issue.
> On Windows, a non-local exit occurs due to the resulting exception.
> As example:
>
>      (let (p)
>        (setq p
>          (make-process :name "test" :command '("/tmp/nosuchcommand") :buffer (current-buffer)))
>        ;; -- Subsequent code never reached on Windows
>        (while (process-live-p p)
>          (sleep-for 0.01))
>        (message "(Process exit status %d)"
>          (process-exit-status p)))
>
> So on Windows two issues occur:
>   - The exception doesn't indicate what went wrong.
>   - The control-flow of the Elisp program is different from Unix.
>
> This different seems, like it may give rise to Windows-specific bugs, that would be unnecessarily hard to
> debug.

That's true, but the way Emacs invokes async subprocesses on Windows
cannot be similar to Unix, because Windows lacks the 'fork' system
call.  Therefore, on Windows, the Emacs process itself invokes the
program, whereas on Unix this is done by a separate "forked" process,
which means Emacs on Unix simply doesn't know whether running the
program failed, until much later.

What this means is if some Lisp program wants to produce a consistent
behavior in these situations, it should have slightly different
application-level code for Posix and non-Posix hosts.

> Then again, calling programs by full path is probably rare, so it's probably a pretty low-priority issue.

Right.



Reply | Threaded
Open this post in threaded view
|

bug#33016: 26.1; (make-process ...) doesn't signal an error, when executable given as absolute Windows path does not exist

Noam Postavsky
In reply to this post by Klaus-Dieter Bauer
On Thu, 11 Oct 2018 at 08:57, Klaus-Dieter Bauer
<[hidden email]> wrote:

>     M-x eval-expression RET
>       (make-process :name "test" :command '("c:/No Such Command"))
>
> will merely display in the echo-area message:
>
>     eval: Spawning child process: Invalid argument

The confusing thing here is that the error is signaled between
block_input()...unblock_input(), which prevents the debugger from
triggering. E.g., the "-unless-debug" part in the expression below
appears not to work, even though the error flows normally in other
respects:

(condition-case-unless-debug err
    (make-process :name "test" :command '("c:/No Such Command"))
  (error (list :error err)))
;=> (:error (file-error "Spawning child process" "Invalid argument"))

The attached patch fixes this by moving the signal to after the unblock_input().

v1-0001-Let-debugger-handle-process-spawn-errors-on-w32-B.patch (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

bug#33016: 26.1; (make-process ...) doesn't signal an error, when executable given as absolute Windows path does not exist

Eli Zaretskii
> From: Noam Postavsky <[hidden email]>
> Date: Mon, 8 Apr 2019 14:34:41 -0400
> Cc: [hidden email]
>
> >     M-x eval-expression RET
> >       (make-process :name "test" :command '("c:/No Such Command"))
> >
> > will merely display in the echo-area message:
> >
> >     eval: Spawning child process: Invalid argument
>
> The confusing thing here is that the error is signaled between
> block_input()...unblock_input(), which prevents the debugger from
> triggering. E.g., the "-unless-debug" part in the expression below
> appears not to work, even though the error flows normally in other
> respects:
>
> (condition-case-unless-debug err
>     (make-process :name "test" :command '("c:/No Such Command"))
>   (error (list :error err)))
> ;=> (:error (file-error "Spawning child process" "Invalid argument"))
>
> The attached patch fixes this by moving the signal to after the unblock_input().

Thanks, but could we have a test for this, please?



Reply | Threaded
Open this post in threaded view
|

bug#33016: 26.1; (make-process ...) doesn't signal an error, when executable given as absolute Windows path does not exist

Noam Postavsky
On Mon, 8 Apr 2019 at 14:59, Eli Zaretskii <[hidden email]> wrote:

> > The confusing thing here is that the error is signaled between
> > block_input()...unblock_input(), which prevents the debugger from
> > triggering.

> > The attached patch fixes this by moving the signal to after the unblock_input().
>
> Thanks, but could we have a test for this, please?

Yes (I had initially thought it wouldn't work because of the way ert
uses the debugger internally, but it actually turns out fine).

By the way, I modified the error message in call_process in addition
to create_process for completeness, but I can't see a way to trigger
this for call_process: it searches for PROGRAM and signals an error
early, regardless of whether the filename is absolute or not.

v2-0001-Let-debugger-handle-process-spawn-errors-on-w32-B.patch (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

bug#33016: 26.1; (make-process ...) doesn't signal an error, when executable given as absolute Windows path does not exist

Eli Zaretskii
> From: Noam Postavsky <[hidden email]>
> Date: Tue, 9 Apr 2019 10:13:58 -0400
> Cc: Klaus-Dieter Bauer <[hidden email]>, [hidden email]
>
> By the way, I modified the error message in call_process in addition
> to create_process for completeness, but I can't see a way to trigger
> this for call_process: it searches for PROGRAM and signals an error
> early, regardless of whether the filename is absolute or not.

One way is to delete the program between the time Emacs searches for
it and the time it actually invokes it.  Another way is to make the
program be a file whose name includes non-ASCII characters outside of
the current system codepage (I'm assuming the search for the program
uses file-oriented primitives which support any Unicode characters).

Having said that, this isn't worth too much of your time, if those
ideas cannot be easily implemented, or turn out wrong, and no others
present themselves.

Thanks.



Reply | Threaded
Open this post in threaded view
|

bug#33016: 26.1; (make-process ...) doesn't signal an error, when executable given as absolute Windows path does not exist

Noam Postavsky
On Tue, 9 Apr 2019 at 10:33, Eli Zaretskii <[hidden email]> wrote:

> > but I can't see a way to trigger
> > this for call_process: it searches for PROGRAM and signals an error
> > early, regardless of whether the filename is absolute or not.
>
> One way is to delete the program between the time Emacs searches for
> it and the time it actually invokes it.  Another way is to make the
> program be a file whose name includes non-ASCII characters outside of
> the current system codepage (I'm assuming the search for the program
> uses file-oriented primitives which support any Unicode characters).
>
> Having said that, this isn't worth too much of your time, if those
> ideas cannot be easily implemented, or turn out wrong, and no others
> present themselves.
I was inspired by your suggestions to think of a simpler idea: use "C:/nul.exe".

There is unfortunately one additional wrinkle: each of the test passes
on its own, but when running both together the second one fails due to
this check in maybe_call_debugger:

      /* RMS: What's this for?  */
      && when_entered_debugger < num_nonmacro_input_events)

RMS' question is (now) answered in the commentary for when_entered_debugger:

/* The value of num_nonmacro_input_events as of the last time we
   started to enter the debugger.  If we decide to enter the debugger
   again when this is still equal to num_nonmacro_input_events, then we
   know that the debugger itself has an error, and we should just
   signal the error instead of entering an infinite loop of debugger
   invocations.  */

static intmax_t when_entered_debugger;

So I guess we'd need some way of resetting it from Lisp? (which does
open the tiny danger of a debugger inf-looping by resetting
when_entered_debugger and then hitting an error).
As far as I can tell, the normal debugger resets it by calling
recursive-edit, but there's no way to return from that without human
intervention (I think?).

v3-0001-Let-debugger-handle-process-spawn-errors-on-w32-B.patch (8K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

bug#33016: 26.1; (make-process ...) doesn't signal an error, when executable given as absolute Windows path does not exist

Eli Zaretskii
> From: Noam Postavsky <[hidden email]>
> Date: Wed, 10 Apr 2019 17:58:43 -0400
> Cc: Klaus-Dieter Bauer <[hidden email]>, [hidden email]
>
> I was inspired by your suggestions to think of a simpler idea: use "C:/nul.exe".
>
> There is unfortunately one additional wrinkle: each of the test passes
> on its own, but when running both together the second one fails due to
> this check in maybe_call_debugger:
>
>       /* RMS: What's this for?  */
>       && when_entered_debugger < num_nonmacro_input_events)
>
> RMS' question is (now) answered in the commentary for when_entered_debugger:
>
> /* The value of num_nonmacro_input_events as of the last time we
>    started to enter the debugger.  If we decide to enter the debugger
>    again when this is still equal to num_nonmacro_input_events, then we
>    know that the debugger itself has an error, and we should just
>    signal the error instead of entering an infinite loop of debugger
>    invocations.  */
>
> static intmax_t when_entered_debugger;
>
> So I guess we'd need some way of resetting it from Lisp?

Doesn't it work to simply set its value before the second test?

> As far as I can tell, the normal debugger resets it by calling
> recursive-edit, but there's no way to return from that without human
> intervention (I think?).

Doesn't abort-recursive-edit work noninteractively?

> +                    ;; On Windows, "nul.FOO" is the empty file for any
> +                    ;; FOO, in any directory.  So this passes Emacs'

Instead of "is the empty file", I'd say something like "resolves to
the null device, reading from which sets the EOF condition".

Thanks.



Reply | Threaded
Open this post in threaded view
|

bug#33016: 26.1; (make-process ...) doesn't signal an error, when executable given as absolute Windows path does not exist

Noam Postavsky
On Thu, 11 Apr 2019 at 10:05, Eli Zaretskii <[hidden email]> wrote:

> > static intmax_t when_entered_debugger;
> >
> > So I guess we'd need some way of resetting it from Lisp?
>
> Doesn't it work to simply set its value before the second test?

Yes, or in each test, since the tests don't necessarily have knowledge
of what order they're called in (I think it's currently alphabetical
order of test name). See attached diff (against the state of my v3
patch), but it seems a bit silly to make the variable Lisp accessible
just for this obscure test case. I don't see any other way though.

> > As far as I can tell, the normal debugger resets it by calling
> > recursive-edit, but there's no way to return from that without human
> > intervention (I think?).
>
> Doesn't abort-recursive-edit work noninteractively?

Yes, but how can I arrange for it to be called without stopping to
read commands from the user first? E.g., in the following
abort-recursive-edit is too late to do any good:

(progn
  (recursive-edit)
  (abort-recursive-edit))

Using pre-command-hook is also too late, the user has to type
something to trigger the beginning of a certain command first.

(let ((pre-command-hook #'abort-recursive-edit))
  (recursive-edit))

> > +                    ;; On Windows, "nul.FOO" is the empty file for any
> > +                    ;; FOO, in any directory.  So this passes Emacs'
>
> Instead of "is the empty file", I'd say something like "resolves to
> the null device, reading from which sets the EOF condition".

Hmm, while technically more accurate, it seems like a little too much
detail to be useful; I think saying "acts like an always-empty file"
should be enough.

reset-when-entered.debugger.diff (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

bug#33016: 26.1; (make-process ...) doesn't signal an error, when executable given as absolute Windows path does not exist

Eli Zaretskii
> From: Noam Postavsky <[hidden email]>
> Date: Thu, 11 Apr 2019 13:34:25 -0400
> Cc: Klaus-Dieter Bauer <[hidden email]>, [hidden email]
>
> > Doesn't it work to simply set its value before the second test?
>
> Yes, or in each test, since the tests don't necessarily have knowledge
> of what order they're called in (I think it's currently alphabetical
> order of test name). See attached diff (against the state of my v3
> patch), but it seems a bit silly to make the variable Lisp accessible
> just for this obscure test case. I don't see any other way though.

OK.  That diff includes some unrelated stuff, though -- you didn't
mean to install it as is, right?



Reply | Threaded
Open this post in threaded view
|

bug#33016: 26.1; (make-process ...) doesn't signal an error, when executable given as absolute Windows path does not exist

Noam Postavsky
Eli Zaretskii <[hidden email]> writes:

> OK.  That diff includes some unrelated stuff, though -- you didn't
> mean to install it as is, right?

Ah, I left in the .gdbinit change completely by accident.  The other
stuff is related, but wasn't cleaned up properly yet.  Here's a proper patch:


v4-0001-Let-debugger-handle-process-spawn-errors-on-w32-B.patch (8K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

bug#33016: 26.1; (make-process ...) doesn't signal an error, when executable given as absolute Windows path does not exist

Eli Zaretskii
> From: Noam Postavsky <[hidden email]>
> Cc: [hidden email],  [hidden email]
> Date: Thu, 11 Apr 2019 20:44:15 -0400
>
> > OK.  That diff includes some unrelated stuff, though -- you didn't
> > mean to install it as is, right?
>
> Ah, I left in the .gdbinit change completely by accident.  The other
> stuff is related, but wasn't cleaned up properly yet.  Here's a proper patch:

Thanks.

> +                    ;; On Windows, "nul.FOO" is the empty file for any
> +                    ;; FOO, in any directory.  So this passes Emacs'
> +                    ;; test for the file's existence, and ensures we
> +                    ;; hit an error in the w32 process spawn code.
> +                    (call-process "c:/nul.exe")

What happened to mentioning the null device in this comment?

> +                     (setq when-entered-debugger -1))))

This should be internal-when-entered-debugger, right?  And the same in
the other test.



Reply | Threaded
Open this post in threaded view
|

bug#33016: 26.1; (make-process ...) doesn't signal an error, when executable given as absolute Windows path does not exist

Noam Postavsky
On Fri, 12 Apr 2019 at 04:45, Eli Zaretskii <[hidden email]> wrote:

> > +                    ;; On Windows, "nul.FOO" is the empty file for any
> > +                    ;; FOO, in any directory.  So this passes Emacs'
> > +                    ;; test for the file's existence, and ensures we
> > +                    ;; hit an error in the w32 process spawn code.
> > +                    (call-process "c:/nul.exe")
>
> What happened to mentioning the null device in this comment?

Yeah, I forgot to change this comment (though I don't think
specifically naming the null device is needed).

> > +                     (setq when-entered-debugger -1))))
>
> This should be internal-when-entered-debugger, right?  And the same in
> the other test.

Oops, right, that's what I get for rushing things. I think got
everything this time (include replacing RMS' question in eval.c).

v5-0001-Let-debugger-handle-process-spawn-errors-on-w32-B.patch (12K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

bug#33016: 26.1; (make-process ...) doesn't signal an error, when executable given as absolute Windows path does not exist

Eli Zaretskii
> From: Noam Postavsky <[hidden email]>
> Date: Fri, 12 Apr 2019 14:20:46 -0400
> Cc: [hidden email], Klaus-Dieter Bauer <[hidden email]>
>
> Oops, right, that's what I get for rushing things. I think got
> everything this time (include replacing RMS' question in eval.c).

OK, thanks.



Reply | Threaded
Open this post in threaded view
|

bug#33016: 26.1; (make-process ...) doesn't signal an error, when executable given as absolute Windows path does not exist

Noam Postavsky
tags 33016 fixed
close 33016 27.1
quit

Eli Zaretskii <[hidden email]> writes:

>> From: Noam Postavsky <[hidden email]>
>> Date: Fri, 12 Apr 2019 14:20:46 -0400
>> Cc: [hidden email], Klaus-Dieter Bauer <[hidden email]>
>>
>> Oops, right, that's what I get for rushing things. I think got
>> everything this time (include replacing RMS' question in eval.c).
>
> OK, thanks.

Pushed to master.

9800df69cb 2019-04-14T22:43:38-04:00 "Let debugger handle process spawn errors on w32 (Bug#33016)"