Recently, I have been working on my personal project for building a Nix-based CI framework for Emacs Lisp packages. I started writing some Nix for building my home-manager configuration about a year ago, but the current project gives me better opportunities for learning Nix. We have a holiday in May in Japan, during which I have made progress in the project. Hopefully, I will be able to release it within a month or so. I have also gained insights on how to improve my home-manager configuration, which I want to rewrite someday.

This post is about how to apply an external formatter using reformatter.el, on a project-wide basis. The package usually works on a per-buffer basis, but with a wrapper command introduced in this post, it can be applied to all files of a certain type in a project. It does a simple thing, but as you work on more projects with similar tech stacks, it may make things slightly easier. Several other tips for configuring Emacs are introduced along the way. Hope they will be useful.

Problem

When you write Nix, you will need a formatter for the language. At present, nixfmt, which is available from Nixpkgs, seems to be a sensible choice. It is a command line program, so you will somehow need to integrate it into your Emacs workflow. For this purpose, you can use reformatter.el by Steve Purcell, which is a generic package for running external formatters (and linters) on Emacs buffers. You can configure nixfmt as follows:

(use-package reformatter
  :config
  (reformatter-define nixfmt
    :program "nixfmt"
    :mode t))

reformatter-define macro defines interactive functions for running a given formatter. You can use nixfmt-buffer to apply it on the current buffer, and nixfmt-region on an active region.

However, I have already written a bunch of Nix files, without applying the formatter to each file. Fortunately, there is no contributor for the project right now, so I don’t have to worry about merge conflicts at all, but I still have to apply the formatter. How can I do that?

Solution without Emacs

One solution would be to run the following shell command in the project root:

fd --color=never .nix | xargs nixfmt

This is obvious for anyone. It uses fd as a better alternative to GNU find. fd ignores hidden files by default, so you don’t have to worry about files inside a .git directory.

It even doesn’t need reformatter.

However, I don’t want to leave Emacs.

Solution from within/inside Emacs

Recently, I feel reluctant to run command line programs directly. Even though I definitely sometimes have to do that, it is a kind of cognitive burden and prefer avoiding it. Because this is Emacs, there is a way.

To apply a formatter on a project, I have written the following command:

(defun akirak/run-formatter-on-project ()
  (interactive)
  (require 'my/formatter)
  (require 'my/file/enum)
  (let* ((project default-directory)
         (files (akirak/project-files project))
         (alist (->> (-group-by #'f-ext files)
                     (-sort (lambda (a b)
                              (> (length (cdr a))
                                 (length (cdr b)))))
                     (-filter #'car)))
         (ext (completing-read "File extension: "
                               (-map #'car alist)
                               nil t)))
    (dolist (file (cdr (assoc ext alist)))
      (let (new-buffer)
        (with-current-buffer (or (find-buffer-visiting file)
                                 (setq new-buffer
                                       (find-file-noselect file)))
          (save-restriction
            (widen)
            (pcase (akirak/get-project-formatter project :mode major-mode)
              (`(reformatter ,name)
               (funcall (intern (concat name "-buffer"))))
              (_ (user-error "%s formatter" formatter)))
            (save-buffer))
          (let ((error-buf (get-buffer (format "*%s errors*" name))))
            (if (and error-buf
                     (> (buffer-size error-buf) 0))
                (progn
                  (switch-to-buffer (current-buffer))
                  (display-buffer error-buf)
                  (user-error "Error while applying the formatter"))
              (when-let (w (and error-buf
                                (get-buffer-window error-buf)))
                (quit-window nil w)))))
        (when new-buffer
          (kill-buffer new-buffer)))))
  (if (derived-mode-p 'magit-status-mode)
      (progn
        (message "Finished formatting. Refreshing the magit buffer...")
        (magit-refresh))
    (message "Finished formatting")))

This command does the following:

  1. Let the user choose a file extension in the project.
  2. Apply a formatter on all files matching the extension in the project.

If you encounter an error in one of the matching files, it aborts processing.

If you dispatch this command from inside a magit-status buffer, it refreshes the buffer after successful exit on all files.

That’s basically all, but it depends on several other functions in my configuration. I will introduce them as well in the following subsections.

Getting all files in a project

akirak/project-files is defined as follows in my/file/enum.el:

(defvar akirak/directory-contents-cache nil)

(cl-defun akirak/project-files (root &key (sort 'modified))
  (let* ((attrs (file-attributes default-directory))
         (mtime (nth 5 attrs))
         (cache (assoc (cons root sort) akirak/directory-contents-cache))
         (default-directory root))
    (if (or (not (cdr cache))
            (time-less-p (cadr cache) mtime))
        (let* ((items (apply #'process-lines
                             "rg" "--files"
                             "--color=never"
                             "--iglob=!.git"
                             "--iglob=!.svn"
                             "--hidden"
                             "--one-file-system"
                             (cl-ecase sort
                               (modified '("--sortr" "modified")))))
               (cell (cons mtime items)))
          (if cache
              (setf (cdr cache) cell)
            (push (cons (cons root sort) cell) akirak/directory-contents-cache))
          items)
      (cddr cache))))

(cl-defun akirak/clear-project-file-cache (root &key sort)
  (when-let ((cache (assoc (cons root sort) akirak/directory-contents-cache)))
    (message "Clearing cache for %s..." root)
    (setcdr cache nil)))

(provide 'my/file/enum)

It retrieves files recursively using ripgrep (rg) and caches the result in a variable. Like fd, rg can be used to find regular files, and it also supports sorting. With --hidden flag, it discover files in hidden directories (e.g. .github/workflows), but with the iglob option, it skips the .git directory. It respects .gitignore and other ignore files by default. I use this function to replace counsel-projectile or counsel-git with a custom Helm command.

You can clear the cache of a project by calling akirak/clear-project-file-cache function.

Also note that you can organise Emacs Lisp files hierarchically. If you put the file in a directory in load-path organised in a subdirectory, Emacs will find it with a require statement. As far as I remember, clemera found this technique. Thanks!

Choosing a formatter

akirak/get-project-formatter is defined as follows in my/formatter.el:

(require 'my/project)

(defcustom akirak/project-formatter-list nil
  "List of formatters to use in individual projects.")

(defun akirak/get-reformatter-formatters ()
  (cl-loop for sym being the symbols of obarray
           ;; Filter minor modes
           when (and (memq sym minor-mode-list)
                     (string-suffix-p "-on-save-mode" (symbol-name sym)))
           collect (string-remove-suffix "-on-save-mode" (symbol-name sym))))

(defun akirak/pick-mode-formatter (mode)
  (-some->> akirak/project-formatter-list
    (assoc mode)
    (cdadr)))

(cl-defun akirak/get-project-formatter (&optional project &key mode)
  (let* ((project (or project (akirak/project-root default-directory)))
         (mode (or mode major-mode))
         (formatter (-some->> akirak/project-formatter-list
                      (assoc mode)
                      (cdr)
                      (assoc project)
                      (cdr))))
    (or formatter
        (let* ((formatter (list 'reformatter
                                (completing-read (format "Formatter for %s in project %s: "
                                                         mode
                                                         (abbreviate-file-name project))
                                                 (akirak/get-reformatter-formatters)
                                                 nil t nil nil
                                                 (akirak/pick-mode-formatter mode))))
               (cell (assoc mode akirak/project-formatter-list))
               (subcell (cons project formatter)))
          (cond
           (cell
            (setcdr cell (cons subcell (cdr cell))))
           (t
            (add-to-list 'akirak/project-formatter-list (cons mode (list subcell)))))
          (customize-save-variable 'akirak/project-formatter-list
                                   akirak/project-formatter-list
                                   "Set by akirak/get-project-formatter")
          formatter))))

(provide 'my/formatter)

It retrieves a previously used formatter for the major mode in the project if any, or asks the user for one. The default choice is determined based on other projects. It saves each choice to custom-file, so it “learns” user’s preferences, in a stupid way. To reset your preference and use a different program, you can edit the custom variable.

It currently only supports reformatter, but it can be extended to support other formatting functions.

akirak/project-root is a function which retrieves the root of a project:

(require 'project)

;; Based on ibuffer-project.el.
(defvar akirak/project-roots-cache (make-hash-table :test 'equal)
  "Variable to store cache of project per directory.")

(defun akirak/clear-project-cache ()
  (interactive)
  (clrhash akirak/project-roots-cache))

(defun akirak/project-root (dir)
  "Return the project root of DIR with cache enabled."
  (pcase (gethash dir akirak/project-roots-cache 'no-cached)
    ('no-cached (let* ((project (project-current nil dir))
                       (root (and project (car (project-roots project)))))
                  (puthash dir root akirak/project-roots-cache)
                  root))
    (root root)))

(provide 'my/project)

It is a cached version of project.el. I have to cache project roots for performance, because I use Emacs inside WSL at work. This was an idea which I initially contributed to ibuffer-project, but Andrii Kolomoiets, its author and maintainer, improved the implementation.

The entire solution depends on some Emacs Lisp libraries which are not part of Emacs:

  • dash.el
  • f.el (only for f-ext)

Keybinding

I bind a key to the command in magit-status-mode-map, so I can dispatch the command from inside a magit-status buffer. This way, default-directory becomes the project root, and you can review the effect immediately. magit-status is a project dashboard in Emacs, and it is even extensible, like magit-todos does.

The actual configuration in my init.el is as follows:

(akirak/bind-mode :keymaps 'magit-status-mode-map :package 'magit-status
  "lf"
  (defun akirak/run-formatter-on-project ()
    (interactive)
    (require 'my/formatter)
    (require 'my/file/enum)
    (let* ((project default-directory)
           (files (akirak/project-files project))
           (alist (->> (-group-by #'f-ext files)
                       (-sort (lambda (a b)
                                (> (length (cdr a))
                                   (length (cdr b)))))
                       (-filter #'car)))
           (ext (completing-read "File extension: "
                                 (-map #'car alist)
                                 nil t)))
      (dolist (file (cdr (assoc ext alist)))
        (let (new-buffer)
          (with-current-buffer (or (find-buffer-visiting file)
                                   (setq new-buffer
                                         (find-file-noselect file)))
            (save-restriction
              (widen)
              (pcase (akirak/get-project-formatter project :mode major-mode)
                (`(reformatter ,name)
                 (funcall (intern (concat name "-buffer"))))
                (_ (user-error "%s formatter" formatter)))
              (save-buffer))
            (let ((error-buf (get-buffer (format "*%s errors*" name))))
              (if (and error-buf
                       (> (buffer-size error-buf) 0))
                  (progn
                    (switch-to-buffer (current-buffer))
                    (display-buffer error-buf)
                    (user-error "Error while applying the formatter"))
                (when-let (w (and error-buf
                                  (get-buffer-window error-buf)))
                  (quit-window nil w)))))
          (when new-buffer
            (kill-buffer new-buffer)))))
    (if (derived-mode-p 'magit-status-mode)
        (progn
          (message "Finished formatting. Refreshing the magit buffer...")
          (magit-refresh))
      (message "Finished formatting"))))

akirak/bind-mode is a custom keybinding definer created using general.el. It defines keybindings on C-, prefix, but the prefix can be changed later. You can substitute it for general-def with a proper key sequence.

Note that you can define interactive functions inside general-def forms. This is another technique I learned from clemera, and I highly recommend it in keybindings, function advices, and hooks.

I have also defined a command which applies a formatter to a single file or a region, which is a unified wrapper for a bunch of reformatter commands:

(akirak/bind-generic
  "lf"
  (defun akirak/run-formatter ()
    (interactive)
    (require 'my/formatter)
    (pcase (akirak/get-project-formatter)
      (`(reformatter ,name)
       (if (region-active-p)
           (funcall (intern (concat name "-region")))
         (funcall (intern (concat name "-buffer"))))
       (let ((error-buf (get-buffer (format "*%s errors*" name))))
         (if (and error-buf
                  (> (buffer-size error-buf) 0))
             (display-buffer error-buf)
           (when-let (w (and error-buf
                             (get-buffer-window error-buf)))
             (quit-window nil w)))))
      (_ (user-error "%s formatter" formatter)))))

This command reformats a region if any or the current buffer. Now you don’t have to define keybindings for each mode.

akirak/bind-generic is another custom general.el definer (on C-.), and you can substitute it for general-def instead.

Comparison with other methods

By default, reformatter defines a minor mode for applying a formatter on save, and it is suggested that you should activate the mode through .dir-locals.el and/or as file-local variables. This allows a fine-grained control of which formatter to apply on individual entities. The downside of this approach would be that you will have to share the formatter definition across your team or project contributors. You will probably have to package the formatter to distribute it.

On the contrary, my solution is more suitable for manual operations, scratches, and private use due to the following reasons:

  • It provides a single entry point. You don’t have to bind keys to different commands for different languages in different projects.
  • You can pick a formatter interactively from all of your available options, and your choice is persisted in custom-file.
  • If you are the only Emacs user in your work project, you can consistently use designated formatters without committing .dir-locals.el to your repository. It never distracts other latent users of Emacs opening the project.

In either way, you don’t have to remember the detailed usage of a formatter command, unlike calling a program from command line. This is an advantage of Emacs.

These solutions are not mutually exclusive. You can first try out a formatter interactively and then enforce it through .dir-locals.el.

Nonetheless, you would probably want to configure CI or Git hooks to enforce formatters if it’s a team project.

Conclusion

This post contains a considerable amount of code, but the gist of the entire solution is summarised into the following bullets:

  • Command line programs get things done quickly, provided that you are accustomed to using them.
  • reformatter works either on a buffer or on a region, and the former feature can be extended to project-wide by wrapping it in a function.
  • You can use ripgrep as an alternative to find. It respects .gitignore by default, which is convenient.
  • When you apply a formatter/linter on files in a project, you might dispatch it from inside a magit-status buffer and refresh it immediately. Magit-status is the project dashboard in Emacs.

The code is available as part of my Emacs configuration in the following locations: