bug#35055: 27.0.50; async-shell-command truncates output lines

classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|

bug#35055: 27.0.50; async-shell-command truncates output lines

Juri Linkov-2
Test case:

1. Open a wide frame and type:

2. `M-& ps aux RET'

3. observe that output lines are truncated at column 80

Another example: `M-& dpkg -l '*emacs*' RET', then try to copy
full package names from the *Async Shell Command* buffer.
Nope, it's impossible because they are truncated.

OTOH, there is no such problem with `M-! ps aux RET'
where lines are not truncated at all.

So the question is: why `M-&' (async-shell-command) limits COLUMNS to 80,
even on wide frames, whereas `M-!' (shell-command) has no limitation.

A third case is `M-x shell' that limits COLUMNS to window-width,
but at least it makes possible to change the window width (e.g.
to make it wider) to influence the number of output columns.



Reply | Threaded
Open this post in threaded view
|

bug#35055: 27.0.50; async-shell-command truncates output lines

Michael Albinus
Juri Linkov <[hidden email]> writes:

Hi Juri,

> Test case:
>
> 1. Open a wide frame and type:
>
> 2. `M-& ps aux RET'
>
> 3. observe that output lines are truncated at column 80

In my case, output lines are truncated at column 89. In fact the
truncation happens exactly at the size of the *Async Shell Command*
buffer.

> OTOH, there is no such problem with `M-! ps aux RET'
> where lines are not truncated at all.
>
> So the question is: why `M-&' (async-shell-command) limits COLUMNS to 80,
> even on wide frames, whereas `M-!' (shell-command) has no limitation.

Internally, M-! uses `call-process', and M-& uses
`start-file-process'. They have different implementations.

Synchronous processes do not care about the buffer width. So you see
untruncated output.

Asynchronous processes care. They send the information about buffer
dimensions with the function `set-process-window-size', see
(info "(elisp) Process Buffers")

There is the variable `window-adjust-process-window-size-function' which
controls how the dimension information is given to the underlying
process.

I'm not sure whether there exists already a configuration option to
allow asynchronous shell commands using infinite line width, but it
could be implemented this way.

Note that for remote shell commands the situation is even worse, because
it sets the process property `adjust-window-size-function' to nil,
overwriting any setting in `window-adjust-process-window-size-function'.
This affects even synchronous `shell-command' calls, because they are
implemented Tramp internally as asynchronous process.

Best regards, Michael.



Reply | Threaded
Open this post in threaded view
|

bug#35055: 27.0.50; async-shell-command truncates output lines

Juri Linkov-2
>> 1. Open a wide frame and type:
>>
>> 2. `M-& ps aux RET'
>>
>> 3. observe that output lines are truncated at column 80
>
> In my case, output lines are truncated at column 89. In fact the
> truncation happens exactly at the size of the *Async Shell Command*
> buffer.

This is strange.  I tried in `emacs -Q' in a wide frame and
output lines of `ps aux' are always truncated at column 80.

Then I tried to affect the number of columns with the
environment variable COLUMNS, and when using

  M-& COLUMNS=89 ps aux

only then output lines are truncated at column 89.
Trying different values produced different truncation:

  COLUMNS=0 ps aux      - column 80
  COLUMNS=1 ps aux      - column 68
  COLUMNS=5 ps aux      - column 70
  COLUMNS=20 ps aux     - column 80
  COLUMNS=30 ps aux     - column 90
  COLUMNS=40 ps aux     - column 80
  COLUMNS=50 ps aux     - column 100
  COLUMNS=60 ps aux     - column 120
  COLUMNS=70 ps aux     - column 70
  COLUMNS=131071 ps aux - practically unlimited
  COLUMNS=131072 ps aux - again column 80 (looks like maximum reached)

i.e. dependency is not linear.

Also `M-x shell' uses `comint-term-environment' to set COLUMNS
to window-width, but it does this only initially,
so after resizing the shell window, COLUMNS remains
at the old initial value.

>> OTOH, there is no such problem with `M-! ps aux RET'
>> where lines are not truncated at all.
>>
>> So the question is: why `M-&' (async-shell-command) limits COLUMNS to 80,
>> even on wide frames, whereas `M-!' (shell-command) has no limitation.
>
> Internally, M-! uses `call-process', and M-& uses
> `start-file-process'. They have different implementations.
>
> Synchronous processes do not care about the buffer width. So you see
> untruncated output.
>
> Asynchronous processes care. They send the information about buffer
> dimensions with the function `set-process-window-size', see
> (info "(elisp) Process Buffers")
>
> There is the variable `window-adjust-process-window-size-function' which
> controls how the dimension information is given to the underlying
> process.

Thanks for the pointer to process-window-size functions.
I tried to debug them in `emacs -Q', not to affect its default
behavior with customization of async-shell-command-display-buffer
that complicates the issue more.

But it seems `set-process-window-size' has no effect on
the truncated columns.  I see that `set-process-window-size'
in `window--adjust-process-windows' is called with WIDTH
more than 80, but the output of `ps aux' is still truncated
to 80 columns in *Async Shell Command*.

> I'm not sure whether there exists already a configuration option to
> allow asynchronous shell commands using infinite line width, but it
> could be implemented this way.
>
> Note that for remote shell commands the situation is even worse, because
> it sets the process property `adjust-window-size-function' to nil,
> overwriting any setting in `window-adjust-process-window-size-function'.
> This affects even synchronous `shell-command' calls, because they are
> implemented Tramp internally as asynchronous process.

I had truncated output of remote `shell-command' for a long time
and thought that it's impossible to do anything with this,
but now that you mentioned remote shell commands, I tried
to affect their output with a quite large value of COLUMNS
and get untruncated output even on remote.



Reply | Threaded
Open this post in threaded view
|

bug#35055: 27.0.50; async-shell-command truncates output lines

Michael Albinus
Juri Linkov <[hidden email]> writes:

>>> 1. Open a wide frame and type:
>>>
>>> 2. `M-& ps aux RET'
>>>
>>> 3. observe that output lines are truncated at column 80
>>
>> In my case, output lines are truncated at column 89. In fact the
>> truncation happens exactly at the size of the *Async Shell Command*
>> buffer.
>
> This is strange.  I tried in `emacs -Q' in a wide frame and
> output lines of `ps aux' are always truncated at column 80.

Strange indeed. Today I'm not in office but @home. Now I get the same
behaviour as you.

On both machines I've used a fresh compiled Emacs from master. The only
difference is, that @office I use a Fedora 29 machine with a large
monitor, and @home Ubuntu 18.10 with a smaller monitor. Don't know
whether this makes the difference. Anyway, ...

> Then I tried to affect the number of columns with the
> environment variable COLUMNS, and when using
>
>   M-& COLUMNS=89 ps aux
>
> only then output lines are truncated at column 89.

Same here.

> Trying different values produced different truncation:
>
>   COLUMNS=0 ps aux      - column 80
>   COLUMNS=1 ps aux      - column 68
>   COLUMNS=5 ps aux      - column 70
>   COLUMNS=20 ps aux     - column 80
>   COLUMNS=30 ps aux     - column 90
>   COLUMNS=40 ps aux     - column 80
>   COLUMNS=50 ps aux     - column 100
>   COLUMNS=60 ps aux     - column 120
>   COLUMNS=70 ps aux     - column 70
>   COLUMNS=131071 ps aux - practically unlimited
>   COLUMNS=131072 ps aux - again column 80 (looks like maximum reached)
>
> i.e. dependency is not linear.

Same here. I guess that setting COLUMNS is not reliable for COLUMNS < 80
and > 131072. But at least it is a way to influence the output width of
asynchronous processes.

>> Note that for remote shell commands the situation is even worse, because
>> it sets the process property `adjust-window-size-function' to nil,
>> overwriting any setting in `window-adjust-process-window-size-function'.
>> This affects even synchronous `shell-command' calls, because they are
>> implemented Tramp internally as asynchronous process.
>
> I had truncated output of remote `shell-command' for a long time
> and thought that it's impossible to do anything with this,
> but now that you mentioned remote shell commands, I tried
> to affect their output with a quite large value of COLUMNS
> and get untruncated output even on remote.

That's because Tramp sets process property `adjust-window-size-function'
to `ignore' (and not to nil as I wrote above). This was 2 years ago;
unfortunately, I have neither a bug nor a message reference for that change.

Best regards, Michael.



Reply | Threaded
Open this post in threaded view
|

bug#35055: 27.0.50; async-shell-command truncates output lines

Juri Linkov-2
>>> Note that for remote shell commands the situation is even worse, because
>>> it sets the process property `adjust-window-size-function' to nil,
>>> overwriting any setting in `window-adjust-process-window-size-function'.
>>> This affects even synchronous `shell-command' calls, because they are
>>> implemented Tramp internally as asynchronous process.
>>
>> I had truncated output of remote `shell-command' for a long time
>> and thought that it's impossible to do anything with this,
>> but now that you mentioned remote shell commands, I tried
>> to affect their output with a quite large value of COLUMNS
>> and get untruncated output even on remote.
>
> That's because Tramp sets process property `adjust-window-size-function'
> to `ignore' (and not to nil as I wrote above). This was 2 years ago;
> unfortunately, I have neither a bug nor a message reference for that change.

Then I don't understand why remote shell output is limited to 80 columns
when Tramp sets `adjust-window-size-function' to `ignore', i.e. not to 80.



Reply | Threaded
Open this post in threaded view
|

bug#35055: 27.0.50; async-shell-command truncates output lines

Juri Linkov-2
>>>> Note that for remote shell commands the situation is even worse, because
>>>> it sets the process property `adjust-window-size-function' to nil,
>>>> overwriting any setting in `window-adjust-process-window-size-function'.
>>>> This affects even synchronous `shell-command' calls, because they are
>>>> implemented Tramp internally as asynchronous process.
>>>
>>> I had truncated output of remote `shell-command' for a long time
>>> and thought that it's impossible to do anything with this,
>>> but now that you mentioned remote shell commands, I tried
>>> to affect their output with a quite large value of COLUMNS
>>> and get untruncated output even on remote.
>>
>> That's because Tramp sets process property `adjust-window-size-function'
>> to `ignore' (and not to nil as I wrote above). This was 2 years ago;
>> unfortunately, I have neither a bug nor a message reference for that change.
>
> Then I don't understand why remote shell output is limited to 80 columns
> when Tramp sets `adjust-window-size-function' to `ignore', i.e. not to 80.

We could add a new defcustom process-window-width with a choice of
a numeric value to set COLUMNS env variable in the process, or
a symbol `window-width' to set COLUMNS to the width of the output
buffer's window.  Then it will handle both asynchronous processes
and synchronous remote shell commands as well.



Reply | Threaded
Open this post in threaded view
|

bug#35055: 27.0.50; async-shell-command truncates output lines

Michael Albinus
In reply to this post by Juri Linkov-2
Juri Linkov <[hidden email]> writes:

Hi Juri,

>>> That's because Tramp sets process property `adjust-window-size-function'
>>> to `ignore' (and not to nil as I wrote above). This was 2 years ago;
>>> unfortunately, I have neither a bug nor a message reference for that change.
>>
>> Then I don't understand why remote shell output is limited to 80 columns
>> when Tramp sets `adjust-window-size-function' to `ignore', i.e. not to 80.
>
> We could add a new defcustom process-window-width with a choice of
> a numeric value to set COLUMNS env variable in the process, or
> a symbol `window-width' to set COLUMNS to the width of the output
> buffer's window.  Then it will handle both asynchronous processes
> and synchronous remote shell commands as well.

Would be possible. However, I wouldn't speak about COLUMNS of the
underlying process, because this is implementation detail. For remote
processes the process object you see is related to the *local* process
Tramp has opened. But we want to influence the *remote* process, which
has been started from the local process via ssh (for example).

The following code snippet does this already (with an example value of
1024 chars for the width)

--8<---------------cut here---------------start------------->8---
(defun set-fixed-process-window-width ()
  (let ((proc (get-buffer-process (current-buffer))))
    (set-process-window-size proc (window-body-height (selected-window)) 1024)
    ;; Don't change size when Emacs window changes.
    (process-put proc 'adjust-window-size-function #'ignore)))

(add-hook 'shell-mode-hook 'set-fixed-process-window-width)
--8<---------------cut here---------------end--------------->8---

You can test it like this, for example

--8<---------------cut here---------------start------------->8---
(shell-command-to-string "ps aux")
(shell-command-to-string "ps aux &")
--8<---------------cut here---------------end--------------->8---

or in a shell

--8<---------------cut here---------------start------------->8---
M-x shell
## in the *shell* buffer
$ echo $COLUMNS
1024
--8<---------------cut here---------------end--------------->8---

Best regards, Michael.



Reply | Threaded
Open this post in threaded view
|

bug#35055: 27.0.50; async-shell-command truncates output lines

Juri Linkov-2
>>>> That's because Tramp sets process property `adjust-window-size-function'
>>>> to `ignore' (and not to nil as I wrote above). This was 2 years ago;
>>>> unfortunately, I have neither a bug nor a message reference for that change.
>>>
>>> Then I don't understand why remote shell output is limited to 80 columns
>>> when Tramp sets `adjust-window-size-function' to `ignore', i.e. not to 80.
>>
>> We could add a new defcustom process-window-width with a choice of
>> a numeric value to set COLUMNS env variable in the process, or
>> a symbol `window-width' to set COLUMNS to the width of the output
>> buffer's window.  Then it will handle both asynchronous processes
>> and synchronous remote shell commands as well.
>
> Would be possible. However, I wouldn't speak about COLUMNS of the
> underlying process, because this is implementation detail. For remote
> processes the process object you see is related to the *local* process
> Tramp has opened. But we want to influence the *remote* process, which
> has been started from the local process via ssh (for example).
>
> The following code snippet does this already (with an example value of
> 1024 chars for the width)
>
> (defun set-fixed-process-window-width ()
>   (let ((proc (get-buffer-process (current-buffer))))
>     (set-process-window-size proc (window-body-height (selected-window)) 1024)
>     ;; Don't change size when Emacs window changes.
>     (process-put proc 'adjust-window-size-function #'ignore)))
>
> (add-hook 'shell-mode-hook 'set-fixed-process-window-width)
>
>
> You can test it like this, for example
>
> (shell-command-to-string "ps aux")
> (shell-command-to-string "ps aux &")
>
>
> or in a shell
>
> M-x shell
> ## in the *shell* buffer
> $ echo $COLUMNS
> 1024

I didn't know that set-process-window-size changes the
value of COLUMNS.  I thought that COLUMNS is set only by
comint-term-environment.  One thing I still don't understand is
how it would be possible to do the same for the remote process?



Reply | Threaded
Open this post in threaded view
|

bug#35055: 27.0.50; async-shell-command truncates output lines

Michael Albinus
In reply to this post by Juri Linkov-2
Juri Linkov <[hidden email]> writes:

Hi Juri,

> I didn't know that set-process-window-size changes the
> value of COLUMNS.  I thought that COLUMNS is set only by
> comint-term-environment.

comint-term-environment doesn't set anything. It just returns env values
you could add to process-environmont, as in comint-exec-1 or compilation-start.

> One thing I still don't understand is
> how it would be possible to do the same for the remote process?

Well, don't know (yet), this needs more testing. At least, we could set
$COLUMNS directly:

(let ((default-directory "/ssh::")
      (process-environment (cons "COLUMNS=1024" process-environment)))
  (shell-command-to-string "ps aux"))

(let ((default-directory "/ssh::")
      (process-environment (cons "COLUMNS=1024" process-environment)))
  (shell-command-to-string "ps aux &"))

(let ((default-directory "/ssh::")
      (process-environment (cons "COLUMNS=1024" process-environment)))
  (shell))

Tramp could add the $COLUMNS setting by its own. Maybe, there exist a
more general solution, let's see.

If you add "COLUMNS=1024" to tramp-remote-process-environment, you would
get this effect already w/o any change in Tramp. Maybe I shall document
it somewhere in the Tramp manual.

Best regards, Michael.



Reply | Threaded
Open this post in threaded view
|

bug#35055: 27.0.50; async-shell-command truncates output lines

Juri Linkov-2
>> One thing I still don't understand is how it would be possible
>> to do the same for the remote process?
>
> Well, don't know (yet), this needs more testing. At least, we could set
> $COLUMNS directly:
>
> (let ((default-directory "/ssh::")
>       (process-environment (cons "COLUMNS=1024" process-environment)))
>   (shell-command-to-string "ps aux"))
>
> (let ((default-directory "/ssh::")
>       (process-environment (cons "COLUMNS=1024" process-environment)))
>   (shell-command-to-string "ps aux &"))
>
> (let ((default-directory "/ssh::")
>       (process-environment (cons "COLUMNS=1024" process-environment)))
>   (shell))
>
> Tramp could add the $COLUMNS setting by its own. Maybe, there exist a
> more general solution, let's see.
>
> If you add "COLUMNS=1024" to tramp-remote-process-environment, you would
> get this effect already w/o any change in Tramp. Maybe I shall document
> it somewhere in the Tramp manual.

It seems it's impossible to set COLUMNS to the width of the output window
neither in Tramp nor in async-shell-command.  shell-mode can do this
using set-process-window-size.  But Tramp can't do the same for the
remote process, and for async-shell-command it's too late to do this
because when it displays a window with the output buffer,
the command is already finished.



Reply | Threaded
Open this post in threaded view
|

bug#35055: 27.0.50; async-shell-command truncates output lines

Michael Albinus
Juri Linkov <[hidden email]> writes:

Hi Juri,

>> If you add "COLUMNS=1024" to tramp-remote-process-environment, you would
>> get this effect already w/o any change in Tramp. Maybe I shall document
>> it somewhere in the Tramp manual.
>
> It seems it's impossible to set COLUMNS to the width of the output window
> neither in Tramp nor in async-shell-command.  shell-mode can do this
> using set-process-window-size.  But Tramp can't do the same for the
> remote process, and for async-shell-command it's too late to do this
> because when it displays a window with the output buffer,
> the command is already finished.

Again, using tramp-remote-process-environment seems to be the
appropriate place.

Best regards, Michael.



Reply | Threaded
Open this post in threaded view
|

bug#35055: 27.0.50; async-shell-command truncates output lines

Juri Linkov-2
>>> If you add "COLUMNS=1024" to tramp-remote-process-environment, you would

>>> get this effect already w/o any change in Tramp. Maybe I shall document
>>> it somewhere in the Tramp manual.
>>
>> It seems it's impossible to set COLUMNS to the width of the output window
>> neither in Tramp nor in async-shell-command.  shell-mode can do this
>> using set-process-window-size.  But Tramp can't do the same for the
>> remote process, and for async-shell-command it's too late to do this
>> because when it displays a window with the output buffer,
>> the command is already finished.
>
> Again, using tramp-remote-process-environment seems to be the
> appropriate place.
Thanks for the suggestion, I customized it to '("COLUMNS=222")
that is approximately equal to my frame's width, thus `ps`
shows enough output, and `dpkg -l '*emacs*'` is not too wide
('dpkg' tries to fill all available columns).

Should a similar option be added for async-shell-command as well?


diff --git a/lisp/simple.el b/lisp/simple.el
index 857e0fc001..7212686add 100644
--- a/lisp/simple.el
+++ b/lisp/simple.el
@@ -3336,6 +3336,15 @@ async-shell-command-display-buffer
   :group 'shell
   :version "26.1")
 
+(defcustom async-shell-command-width nil
+  "Number of columns available for shell command.
+If nil, use the shell default number (usually 80).
+If a positive integer, use a fixed width."
+  :type '(choice (const :tag "Use system limit" nil)
+                 (integer :tag "Fixed width" :value 80))
+  :group 'shell
+  :version "27.1")
+
 (defcustom shell-command-dont-erase-buffer nil
   "If non-nil, output buffer is not erased between shell commands.
 Also, a non-nil value sets the point in the output buffer
@@ -3599,8 +3608,13 @@ shell-command
  (with-current-buffer buffer
                   (shell-command--save-pos-or-erase)
   (setq default-directory directory)
-                  (setq proc
-                        (start-process-shell-command "Shell" buffer command))
+  (let ((process-environment
+ (if (natnump async-shell-command-width)
+     (cons (format "COLUMNS=%d" async-shell-command-width)
+   process-environment)
+   process-environment)))
+    (setq proc
+  (start-process-shell-command "Shell" buffer command)))
   (setq mode-line-process '(":%s"))
   (require 'shell) (shell-mode)
                   (set-process-sentinel proc #'shell-command-sentinel)
Reply | Threaded
Open this post in threaded view
|

bug#35055: 27.0.50; async-shell-command truncates output lines

Michael Albinus
Juri Linkov <[hidden email]> writes:

Hi Juri,

[sorry for the late reply; it was a busy week]

>> Again, using tramp-remote-process-environment seems to be the
>> appropriate place.
>
> Thanks for the suggestion, I customized it to '("COLUMNS=222")
> that is approximately equal to my frame's width, thus `ps`
> shows enough output, and `dpkg -l '*emacs*'` is not too wide
> ('dpkg' tries to fill all available columns).
>
> Should a similar option be added for async-shell-command as well?
>
> diff --git a/lisp/simple.el b/lisp/simple.el
> index 857e0fc001..7212686add 100644
> --- a/lisp/simple.el
> +++ b/lisp/simple.el
> @@ -3336,6 +3336,15 @@ async-shell-command-display-buffer
>    :group 'shell
>    :version "26.1")
>
> +(defcustom async-shell-command-width nil
> +  "Number of columns available for shell command.
> +If nil, use the shell default number (usually 80).
> +If a positive integer, use a fixed width."
> +  :type '(choice (const :tag "Use system limit" nil)
> +                 (integer :tag "Fixed width" :value 80))
> +  :group 'shell
> +  :version "27.1")
> +
>  (defcustom shell-command-dont-erase-buffer nil
>    "If non-nil, output buffer is not erased between shell commands.
>  Also, a non-nil value sets the point in the output buffer
> @@ -3599,8 +3608,13 @@ shell-command
>   (with-current-buffer buffer
>                    (shell-command--save-pos-or-erase)
>    (setq default-directory directory)
> -                  (setq proc
> -                        (start-process-shell-command "Shell" buffer command))
> +  (let ((process-environment
> + (if (natnump async-shell-command-width)
> +     (cons (format "COLUMNS=%d" async-shell-command-width)
> +   process-environment)
> +   process-environment)))
> +    (setq proc
> +  (start-process-shell-command "Shell" buffer command)))
>    (setq mode-line-process '(":%s"))
>    (require 'shell) (shell-mode)
>                    (set-process-sentinel proc #'shell-command-sentinel)

What about calling this just `shell-command-width'? It wouldn't hurt for
synchronous shell commands, and I could adapt
`tramp-remote-process-environment' accordingly if set. Might be more
convenient for users.

Best regards, Michael.



Reply | Threaded
Open this post in threaded view
|

bug#35055: 27.0.50; async-shell-command truncates output lines

Juri Linkov-2
>> +(defcustom async-shell-command-width nil
>> +  "Number of columns available for shell command.
>> +If nil, use the shell default number (usually 80).
>> +If a positive integer, use a fixed width."
>> +  :type '(choice (const :tag "Use system limit" nil)
>> +                 (integer :tag "Fixed width" :value 80))
>> +  :group 'shell
>> +  :version "27.1")
>> +
>
> What about calling this just `shell-command-width'? It wouldn't hurt for
> synchronous shell commands, and I could adapt
> `tramp-remote-process-environment' accordingly if set. Might be more
> convenient for users.

Yes, this is a more general name.  I pushed to master.

BTW, I had a hard time finding the info about one of the most demanding
Tramp features - an ability to edit files with sudo on a remote server.

I searched for "ssh+sudo" by analogy with the git+ssh:// scheme,
but failed to find anything related among all documented methods in
the Tramp manual, even when searched separately for "ssh" and "sudo".
Eventually I found it in an unexpected chapter "Ad-hoc multi-hops"
that allows to do: C-x C-f /ssh:user@host|sudo::/path <RET>

Would it be possible to make this feature more prominent?



Reply | Threaded
Open this post in threaded view
|

bug#35055: 27.0.50; async-shell-command truncates output lines

Michael Albinus
Juri Linkov <[hidden email]> writes:

Hi Juri,

>> What about calling this just `shell-command-width'? It wouldn't hurt for
>> synchronous shell commands, and I could adapt
>> `tramp-remote-process-environment' accordingly if set. Might be more
>> convenient for users.
>
> Yes, this is a more general name.  I pushed to master.

Thanks. However, I've meant this also for synchronous
`shell-command'. You have implemented this only for asynchronous.

The synchronous `shell-command' does not suffer from the buffer width
restriction. But there might be use cases to set a fixed width, for
example if you want to get a well-defined output layout.

Would you also mind to document the new user option in the Emacs manual?

> BTW, I had a hard time finding the info about one of the most demanding
> Tramp features - an ability to edit files with sudo on a remote server.
>
> I searched for "ssh+sudo" by analogy with the git+ssh:// scheme,
> but failed to find anything related among all documented methods in
> the Tramp manual, even when searched separately for "ssh" and "sudo".
> Eventually I found it in an unexpected chapter "Ad-hoc multi-hops"
> that allows to do: C-x C-f /ssh:user@host|sudo::/path <RET>

Well, this is not a feature on this own. It is just an example how to
apply ad-hoc multi-hops. That's why it is described in that section.

> Would it be possible to make this feature more prominent?

There is an introduction chapter "Quick Start Guide". I've added some
words about ssh+sudo, and friends.



Reply | Threaded
Open this post in threaded view
|

bug#35055: 27.0.50; async-shell-command truncates output lines

Juri Linkov-2
>>> What about calling this just `shell-command-width'? It wouldn't hurt for
>>> synchronous shell commands, and I could adapt
>>> `tramp-remote-process-environment' accordingly if set. Might be more
>>> convenient for users.
>>
>> Yes, this is a more general name.  I pushed to master.
>
> Thanks. However, I've meant this also for synchronous
> `shell-command'. You have implemented this only for asynchronous.
>
> The synchronous `shell-command' does not suffer from the buffer width
> restriction. But there might be use cases to set a fixed width, for
> example if you want to get a well-defined output layout.

Oh, I don't know what to do in case when only the current limit
on the output of asynchronous commands should be increased, but
the output of synchronous commands needs to be left unlimited.
I have no idea how to customize it in this case.

> Would you also mind to document the new user option in the Emacs manual?

Will do as soon as the details of the new option will be clarified.

>> I searched for "ssh+sudo" by analogy with the git+ssh:// scheme,
>> but failed to find anything related among all documented methods in
>> the Tramp manual, even when searched separately for "ssh" and "sudo".
>> Eventually I found it in an unexpected chapter "Ad-hoc multi-hops"
>> that allows to do: C-x C-f /ssh:user@host|sudo::/path <RET>
>
> Well, this is not a feature on this own. It is just an example how to
> apply ad-hoc multi-hops. That's why it is described in that section.
>
>> Would it be possible to make this feature more prominent?
>
> There is an introduction chapter "Quick Start Guide". I've added some
> words about ssh+sudo, and friends.

Thanks, this is the same place where my search found the first occurrence
of "sudo" and there was no info about using it on a remove host
before your addition.

BTW, I see that in info/dir all manual names are capitalized, except Tramp.
This lower-case name looks a little disparaging.



Reply | Threaded
Open this post in threaded view
|

bug#35055: 27.0.50; async-shell-command truncates output lines

Michael Albinus
Juri Linkov <[hidden email]> writes:

Hi Juri,

>> Thanks. However, I've meant this also for synchronous
>> `shell-command'. You have implemented this only for asynchronous.
>>
>> The synchronous `shell-command' does not suffer from the buffer width
>> restriction. But there might be use cases to set a fixed width, for
>> example if you want to get a well-defined output layout.
>
> Oh, I don't know what to do in case when only the current limit
> on the output of asynchronous commands should be increased, but
> the output of synchronous commands needs to be left unlimited.
> I have no idea how to customize it in this case.

I don't believe there is a conflict. The main use case will be to
increase the output width of a shell command, in order not to loose
information. People will do this by setting a large value, say
1024. This will be used for both the synchronous and asynchronous case.

And then there's the use case to have a fixed output width for special
commands, in order to get a deterministic layout. This won't be applied
globally, neither for synchronous nor for asynchronous
`shell-command'. Rather, `shell-command-width' will be let-bound.

And we have the advantage, that synchronous `shell-command' behaves
consistently, for both the local and remote case.

So I don't see a problem.

> BTW, I see that in info/dir all manual names are capitalized, except Tramp.
> This lower-case name looks a little disparaging.

Ahh, good catch. I've used @value{tramp} for this, as in the whole
tramp.texi. But this cannot work in @direntry, which is supposed to be
applied outside the document. I've changed it accordingly.

Best regards, Michael.



Reply | Threaded
Open this post in threaded view
|

bug#35055: 27.0.50; async-shell-command truncates output lines

Juri Linkov-2
>>> Thanks. However, I've meant this also for synchronous
>>> `shell-command'. You have implemented this only for asynchronous.
>>>
>>> The synchronous `shell-command' does not suffer from the buffer width
>>> restriction. But there might be use cases to set a fixed width, for
>>> example if you want to get a well-defined output layout.
>>
>> Oh, I don't know what to do in case when only the current limit
>> on the output of asynchronous commands should be increased, but
>> the output of synchronous commands needs to be left unlimited.
>> I have no idea how to customize it in this case.
>
> I don't believe there is a conflict. The main use case will be to
> increase the output width of a shell command, in order not to loose
> information. People will do this by setting a large value, say
> 1024. This will be used for both the synchronous and asynchronous case.

The same value will increase the output width of async shell commands,
but decrease the output width of synchronous shell commands from unlimited.

> And then there's the use case to have a fixed output width for special
> commands, in order to get a deterministic layout. This won't be applied
> globally, neither for synchronous nor for asynchronous
> `shell-command'. Rather, `shell-command-width' will be let-bound.
>
> And we have the advantage, that synchronous `shell-command' behaves
> consistently, for both the local and remote case.
>
> So I don't see a problem.

If output truncation will apply to shell-command-on-region,
especially with its REPLACE arg, this would be a big problem.
After filtering the buffer contents with a shell command,
parts of the buffer will be lost.



Reply | Threaded
Open this post in threaded view
|

bug#35055: 27.0.50; async-shell-command truncates output lines

Michael Albinus
Juri Linkov <[hidden email]> writes:

Hi Juri,

>> I don't believe there is a conflict. The main use case will be to
>> increase the output width of a shell command, in order not to loose
>> information. People will do this by setting a large value, say
>> 1024. This will be used for both the synchronous and asynchronous case.
>
> The same value will increase the output width of async shell commands,
> but decrease the output width of synchronous shell commands from unlimited.

Yes. You must set a proper value, large enough.

>> And then there's the use case to have a fixed output width for special
>> commands, in order to get a deterministic layout. This won't be applied
>> globally, neither for synchronous nor for asynchronous
>> `shell-command'. Rather, `shell-command-width' will be let-bound.
>>
>> And we have the advantage, that synchronous `shell-command' behaves
>> consistently, for both the local and remote case.
>>
>> So I don't see a problem.
>
> If output truncation will apply to shell-command-on-region,
> especially with its REPLACE arg, this would be a big problem.
> After filtering the buffer contents with a shell command,
> parts of the buffer will be lost.

No. You can always undo the effect on buffer.

Best regards, Michael.



Reply | Threaded
Open this post in threaded view
|

bug#35055: 27.0.50; async-shell-command truncates output lines

Juri Linkov-2
>>> I don't believe there is a conflict. The main use case will be to
>>> increase the output width of a shell command, in order not to loose
>>> information. People will do this by setting a large value, say
>>> 1024. This will be used for both the synchronous and asynchronous case.
>>
>> The same value will increase the output width of async shell commands,
>> but decrease the output width of synchronous shell commands from unlimited.
>
> Yes. You must set a proper value, large enough.
>
>>> And then there's the use case to have a fixed output width for special
>>> commands, in order to get a deterministic layout. This won't be applied
>>> globally, neither for synchronous nor for asynchronous
>>> `shell-command'. Rather, `shell-command-width' will be let-bound.
>>>
>>> And we have the advantage, that synchronous `shell-command' behaves
>>> consistently, for both the local and remote case.
>>>
>>> So I don't see a problem.
>>
>> If output truncation will apply to shell-command-on-region,
>> especially with its REPLACE arg, this would be a big problem.
>> After filtering the buffer contents with a shell command,
>> parts of the buffer will be lost.
>
> No. You can always undo the effect on buffer.

When it's customized to a width more than the window's width,
the user won't notice that the output was truncated and data lost.

We shouldn't cause data loss when unsuspecting users will customize this
option to increase the output width of async or remote shell commands.

There is already data loss in the output of async commands that are truncated.
A new option was intended to reduce data loss by allowing less truncation.

OTOH, currently there is no data loss in synchronous shell commands
that often are used to operate on the buffer's contents by using
shell-command-on-region where output is inserted to a file buffer and saved.

We shouldn't allow data loss in synchronous shell commands
at the cost of reducing data loss in async/remove shells.

There is no need for the new option to be consistent across
both synchronous and async/remote operations.

What we could do is to make clear the scope of the new option
in the documentation:

diff --git a/doc/emacs/misc.texi b/doc/emacs/misc.texi
index 7d7065a441..2f1a33e202 100644
--- a/doc/emacs/misc.texi
+++ b/doc/emacs/misc.texi
@@ -775,6 +775,13 @@ Single Shell
 displayed only when the command generates output, set
 @code{async-shell-command-display-buffer} to @code{nil}.
 
+@vindex shell-command-width
+  The option @code{shell-command-width} defines the number of display
+columns available for output of asynchronous or remote shell commands.
+A positive integer tells the shell to use that number of columns for
+command output.  The default value is @code{nil} that means to use
+the same number of columns as provided by the shell.
+
 @kindex M-|
 @findex shell-command-on-region
   @kbd{M-|} (@code{shell-command-on-region}) is like @kbd{M-!}, but
diff --git a/lisp/simple.el b/lisp/simple.el
index 160c433845..30d111a70b 100644
--- a/lisp/simple.el
+++ b/lisp/simple.el
@@ -3352,7 +3352,7 @@ async-shell-command-display-buffer
   :version "26.1")
 
 (defcustom shell-command-width nil
-  "Number of display columns available for asynchronous shell command output.
+  "Number of display columns available for asynchronous/remote shell command.
 If nil, use the shell default number (usually 80 columns).
 If a positive integer, tell the shell to use that number of columns for
 command output."



12