Writing pipeline commands in Eshell

Over the holydays I finally took some time experimenting with eshell, an interesting shell implementation for emacs. What I love most about it is having shell commands and lisp code together in one place. That can be a real game-changer. Instead of calling ielm, just use eshell for defining and calling lisp functions. And then using them in combination with shell commands and pipelines.

For example, just being able to call

$ dired .

(unfortunately, the path argument is not optional) or just

$ magit

from current directory is really convenient. In fact, I'm using dired so frequently that I've defined a shortcut called els:

(defun eshell/els (&optional arg)
  (dired
   (if (null arg) "." arg)))

After using these commands, q conveniently brings you back to your eshell buffer.

Eshell has some unusual redirection functions for stdout, like sending it to a buffer (ls > #<buffer name>) or to the emacs kill ring (ls > /dev/kill). An example, filtering the contents of the current directory and sending the output to a buffer. Then, setting that buffer to dired-virtual mode and displaying it:

$ ls -l | grep org > #<buffer orgs>
$ (with-current-buffer (get-buffer "orgs") (dired-virtual (eshell/pwd)))
$ switch-to-buffer #<buffer orgs>

And instead of calling emacsclient my-file, you can use the emacs command find-file my-file (or something like find-file-other-window).

What I also like is being able to directly use all different editing features inside of eshell buffers (or in other buffers); that is, just using the eshell buffer as an editable text. I'm trying to get to a workflow where I have shorter command outputs directly in the shell output while sending longer outputs into their own dedicated buffers, possibly displaying them in a second window.

For this, I want to have a function sending output to a buffer and setting that buffer to view-mode. However, writing a lisp function to handle stdin and stdout in eshell doesn't seem straightforward. Instead I first tried to implement a virtual device for eshell. Virtual devices are also targets for redirection, looking like a system device (say /dev/null) but only visible for eshell. Examples for virtual targets are /dev/kill and /dev/clip, which can be used to send shell output to clipboard and kill ring.

This kind of device is declared by means of customization, like so:

(custom-set-variables
 '(eshell-virtual-targets
   '(("/dev/view" my-eshell-to-view-buffer t))))

Now, on each redirection to /dev/view, eshell will call the lisp function given here. It receives an argument giving the mode (overwrite or append) and produces a lambda function, which will be called repeatedly to receive the input.

I tried a quick-and-dirty implementation:

(defun my-eshell-to-view-buffer (mode)
  (lexical-let ((my-buffer (get-buffer-create "*viewer*")))
    (with-current-buffer my-buffer
      (view-mode 0)
      (end-of-buffer)
      (when (eq mode 'overwrite)
        (erase-buffer)))
    (lambda (string)
      (with-current-buffer my-buffer
        (view-mode 0)
        (save-excursion
          (end-of-buffer)
          (insert string))
        (view-mode t)
        (switch-to-buffer my-buffer)))))

Now, doing something like ls -al > /dev/view in eshell results in another emacs buffer being displayed in view-mode and showing the text being piped in. Which is just like the more or less commands: you can scroll, search and q to return to your eshell.

The last step is an alias

alias more 'cat $1 > /dev/view'

and that's it! Both

$ more my.txt

and

$ ls -al | more

just work as expected—nice! So I'm using the system cat command as a helper redirecting piped data into my virtual device. Probably this can also be used as a template when implementing other lisp commands receiving piped input.

The other way would be piping a region into an eshell pipeline. To do this also for non-contiguous regions (like rectangles), I'm using a small helper:

(defun region-string ()
  (cl-labels
       ((nl-string (str)
          (if (string-equal (string-chop-newline str) str) (format "%s\n" str)
            str)))
     (apply #'concat
            (mapcar
             (lambda (region)
               (nl-string (buffer-substring-no-properties (car region) (cdr region))))
             (region-bounds)))))

The rest is easy:

(defun eshell/region (buffer)
  (with-current-buffer buffer (region-string)))

This defines a new command region, receiving a buffer as argument and outputting it's current region into the shell. So,

$ region #<buffer my-buffer> | grep text

would output the current region of my-buffer, filtered by the grep argument (a silly example, as you would do the filtering directly in that buffer..). But you get the point.

A complete roundtrip of input and output pipes would be like preparing a buffer with shell commands, selecting some of them and then calling

$ region "shell-commands" | sh | more

Of course, something similar could be done by just pressing C-u M-| in the first buffer. But again, you get the point.