Introduction

Welcome reader! This is a book about scripting with Clojure and babashka. Clojure is a functional, dynamic programming language from the Lisp family which runs on the JVM. Babashka is a scripting environment made with Clojure, compiled to native with GraalVM. The primary benefits of using babashka for scripting compared to the JVM are fast startup time and low memory consumption. Babashka comes with batteries included and packs libraries like clojure.tools.cli for parsing command line arguments and cheshire for working with JSON. Moreover, it can be installed just by downloading a self-contained binary.

Target audience

Babashka is written for developers who are familiar with Clojure on the JVM. This book assumes familiarity with Clojure and is not a Clojure tutorial. If you aren’t that familiar with Clojure but you’re curious to learn, check out this list of beginner resources.

Setting expectations

Babashka uses SCI for interpreting Clojure. Sci implements a substantial subset of Clojure. Interpreting code is in general not as performant as executing compiled code. If your script takes more than a few seconds to run or has lots of loops, Clojure on the JVM may be a better fit, as the performance on JVM is going to outweigh its startup time penalty. Read more about the differences with Clojure here.

Getting started

Installation

Installing babashka is as simple as downloading the binary for your platform and placing it on your path. Pre-built binaries are provided on the releases page of babashka’s Github repo. Babashka is also available in various package managers like brew for macOS and linux and scoop for Windows. See here for details.

Building from source

If you would rather build babashka from source, download a copy of GraalVM and set the GRAALVM_HOME environment variable. Also make sure you have lein installed. Then run:

$ git clone https://github.com/borkdude/babashka --recursive
$ script/uberjar && script/compile

See the babashka build.md page for details.

Running babashka

The babashka executable is called bb. You can either provide it with a Clojure expression directly:

$ bb -e '(+ 1 2 3)'
6

or run a script:

script.clj
(+ 1 2 3)
$ bb -f script.clj
6

The -e flag is optional when the argument starts with a paren. In that case babashka will treat it automatically as an expression:

$ bb '(+ 1 2 3)'
6

Similarly, the -f flag is optional when the argument is a filename:

$ bb script.clj
6

Commonly, scripts have shebangs so you can invoke them with their filename only:

script.clj
#!/usr/bin/env bb
(+ 1 2 3)

Usage

Typing bb help from the command line will print all the available command line options which should give you a sense of the available features in babashka.

Babashka v1.0.167

Usage: bb [svm-opts] [global-opts] [eval opts] [cmdline args]
or:    bb [svm-opts] [global-opts] file [cmdline args]
or:    bb [svm-opts] [global-opts] subcommand [subcommand opts] [cmdline args]

Substrate VM opts:

  -Xmx<size>[g|G|m|M|k|K]  Set a maximum heap size (e.g. -Xmx256M to limit the heap to 256MB).
  -XX:PrintFlags=          Print all Substrate VM options.

Global opts:

  -cp, --classpath  Classpath to use. Overrides bb.edn classpath.
  --debug           Print debug information and internal stacktrace in case of exception.
  --init <file>     Load file after any preloads and prior to evaluation/subcommands.
  --config <file>   Replacing bb.edn with file. Relative paths are resolved relative to file.
  --deps-root <dir> Treat dir as root of relative paths in config.
  -Sforce           Force recalculation of the classpath (don't use the cache)
  -Sdeps            Deps data to use as the last deps file to be merged

Help:

  help, -h or -?     Print this help text.
  version            Print the current version of babashka.
  describe           Print an EDN map with information about this version of babashka.
  doc <var|ns>       Print docstring of var or namespace. Requires namespace if necessary.

Evaluation:

  -e, --eval <expr>    Evaluate an expression.
  -f, --file <path>    Evaluate a file.
  -m, --main <ns|var>  Call the -main function from a namespace or call a fully qualified var.
  -x, --exec <var>     Call the fully qualified var. Args are parsed by babashka CLI.

REPL:

  repl                 Start REPL. Use rlwrap for history.
  socket-repl  [addr]  Start a socket REPL. Address defaults to localhost:1666.
  nrepl-server [addr]  Start nREPL server. Address defaults to localhost:1667.

Tasks:

  tasks       Print list of available tasks.
  run <task>  Run task. See run --help for more details.

Clojure:

  clojure [args...]  Invokes clojure. Takes same args as the official clojure CLI.

Packaging:

  uberscript <file> [eval-opt]  Collect all required namespaces from the classpath into a single file. Accepts additional eval opts, like `-m`.
  uberjar    <jar>  [eval-opt]  Similar to uberscript but creates jar file.
  prepare                       Download deps & pods defined in bb.edn and cache their metadata. Only an optimization, this will happen on demand when needed.

In- and output flags (only to be used with -e one-liners):

  -i                 Bind *input* to a lazy seq of lines from stdin.
  -I                 Bind *input* to a lazy seq of EDN values from stdin.
  -o                 Write lines to stdout.
  -O                 Write EDN values to stdout.
  --stream           Stream over lines or EDN values from stdin. Combined with -i or -I *input* becomes a single value per iteration.

Tooling:

  print-deps [--format <deps | classpath>]: prints a deps.edn map or classpath
    with built-in deps and deps from bb.edn.

File names take precedence over subcommand names.
Remaining arguments are bound to *command-line-args*.
Use -- to separate script command line args from bb command line args.
When no eval opts or subcommand is provided, the implicit subcommand is repl.

Running a script

Scripts may be executed from a file using -f or --file:

bb -f download_html.clj

The file may also be passed directly, without -f:

bb download_html.clj

Using bb with a shebang also works:

#!/usr/bin/env bb

(require '[babashka.curl :as curl])

(defn get-url [url]
  (println "Downloading url:" url)
  (curl/get url))

(defn write-html [file html]
  (println "Writing file:" file)
  (spit file html))

(let [[url file] *command-line-args*]
  (when (or (empty? url) (empty? file))
    (println "Usage: <url> <file>")
    (System/exit 1))
  (write-html file (:body (get-url url))))
$ ./download_html.clj
Usage: <url> <file>

$ ./download_html.clj https://www.clojure.org /tmp/clojure.org.html
Downloading url: https://www.clojure.org
Writing file: /tmp/clojure.org.html

If /usr/bin/env doesn’t work for you, you can use the following workaround:

$ cat script.clj
#!/bin/sh

#_(
   "exec" "bb" "$0" hello "$@"
   )

(prn *command-line-args*)

./script.clj 1 2 3
("hello" "1" "2" "3")

Current file path

The var *file* contains the full path of the file that is currently being executed:

$ cat example.clj
(prn *file*)

$ bb example.clj
"/Users/borkdude/example.clj"

Parsing command line arguments

Command-line arguments can be retrieved using *command-line-args*. If you want to parse command line arguments, you may use the built-in clojure.tools.cli namespace:

Babashka ships with clojure.tools.cli:

(require '[clojure.tools.cli :refer [parse-opts]])

(def cli-options
  ;; An option with a required argument
  [["-p" "--port PORT" "Port number"
    :default 80
    :parse-fn #(Integer/parseInt %)
    :validate [#(< 0 % 0x10000) "Must be a number between 0 and 65536"]]
   ["-h" "--help"]])

(:options (parse-opts *command-line-args* cli-options))
$ bb script.clj
{:port 80}
$ bb script.clj -h
{:port 80, :help true}

There is also the nubank/docopt library that is compatible with babashka.

Classpath

It is recommended to use bb.edn to control what directories and libraries are included on babashka’s classpath. See Project setup

If you want a lower level to control babashka’s classpath, without the usage of bb.edn you can use the --classpath option that will override the classpath. Say we have a file script/my/namespace.clj:

(ns my.namespace)
(defn -main [& args]
  (apply println "Hello from my namespace!" args))

Now we can execute this main function with:

$ bb --classpath script --main my.namespace 1 2 3
Hello from my namespace! 1 2 3

If you have a larger script with a classic Clojure project layout like

$ tree -L 3
├── deps.edn
├── README
├── src
│   └── project_namespace
│       ├── main.clj
│       └── utilities.clj
└── test
    └── project_namespace
        ├── test_main.clj
        └── test_utilities.clj

then you can tell babashka to include both the src and test folders in the classpath and start a socket REPL by running:

$ bb --classpath src:test socket-repl 1666

If there is no --classpath argument, the BABASHKA_CLASSPATH environment variable will be used. If that variable isn’t set either, babashka will use :deps and :paths from bb.edn.

Also see the babashka.classpath namespace which allows dynamically adding to the classpath.

The namespace babashka.deps integrates tools.deps with babashka and allows you to set the classpath using a deps.edn map.

Invoking a main function

A main function can be invoked with -m or --main like shown above. When given the argument foo.bar, the namespace foo.bar will be required and the function foo.bar/-main will be called with command line arguments as strings.

Since babashka 0.3.1 you may pass a fully qualified symbol to -m:

$ bb -m clojure.core/prn 1 2 3
"1" "2" "3"

so you can execute any function as a main function, as long as it accepts the number of provided arguments.

When invoking bb with a main function, the expression (System/getProperty "babashka.main") will return the name of the main function.

Preloads

The environment variable BABASHKA_PRELOADS allows to define code that will be available in all subsequent usages of babashka.

BABASHKA_PRELOADS='(defn foo [x] (+ x 2))'
BABASHKA_PRELOADS=$BABASHKA_PRELOADS' (defn bar [x] (* x 2))'
export BABASHKA_PRELOADS

Note that you can concatenate multiple expressions. Now you can use these functions in babashka:

$ bb '(-> (foo *input*) bar)' <<< 1
6

You can also preload an entire file using load-file:

export BABASHKA_PRELOADS='(load-file "my_awesome_prelude.clj")'

Note that *input* is not available in preloads.

Running a REPL

Babashka supports running a REPL, a socket REPL and an nREPL server.

REPL

To start a REPL, type:

$ bb repl

To get history with up and down arrows, use rlwrap:

$ rlwrap bb repl

Socket REPL

To start a socket REPL on port 1666:

$ bb socket-repl 1666
Babashka socket REPL started at localhost:1666

Now you can connect with your favorite socket REPL client:

$ rlwrap nc 127.0.0.1 1666
Babashka v0.0.14 REPL.
Use :repl/quit or :repl/exit to quit the REPL.
Clojure rocks, Bash reaches.

bb=> (+ 1 2 3)
6
bb=> :repl/quit
$

The --socket-repl option takes options similar to the clojure.server.repl Java property option in Clojure:

$ bb socket-repl '{:address "0.0.0.0" :accept clojure.core.server/repl :port 1666}'

Editor plugins and tools known to work with a babashka socket REPL:

  • Emacs: inf-clojure:

    To connect:

    M-x inf-clojure-connect <RET> localhost <RET> 1666

    Before evaluating from a Clojure buffer:

    M-x inf-clojure-minor-mode

  • Atom: Chlorine

  • Vim: vim-iced

  • IntelliJ IDEA: Cursive

    Note: you will have to use a workaround via tubular. For more info, look here.

pREPL

Launching a prepl can be done as follows:

$ bb socket-repl '{:address "0.0.0.0" :accept clojure.core.server/io-prepl :port 1666}'

or programmatically:

$ bb -e '(clojure.core.server/io-prepl)'
(+ 1 2 3)
{:tag :ret, :val "6", :ns "user", :ms 0, :form "(+ 1 2 3)"}

nREPL

To start an nREPL server:

$ bb nrepl-server 1667

or programmatically:

$ bb -e "(babashka.nrepl.server/start-server\!) (deref (promise))"
Started nREPL server at 0.0.0.0:1667

Then connect with your favorite nREPL client:

$ lein repl :connect 1667
Connecting to nREPL at 127.0.0.1:1667
user=> (+ 1 2 3)
6
user=>

Editor plugins and tools known to work with the babashka nREPL server:

The babashka nREPL server currently does not write an .nrepl-port file at startup. Using the following nrepl task, defined in a bb.edn, you can accomplish the same:

{:tasks
 {nrepl
  {:requires ([babashka.fs :as fs]
              [babashka.nrepl.server :as srv])
   :task (do (srv/start-server! {:host "localhost"
                                 :port 1339})
             (spit ".nrepl-port" "1339")
             (-> (Runtime/getRuntime)
                 (.addShutdownHook
                  (Thread. (fn [] (fs/delete ".nrepl-port")))))
             (deref (promise)))}}}

The babashka.nrepl.server API is exposed since version 0.8.157.

Debugging the nREPL server

To debug the nREPL server from the binary you can run:

$ BABASHKA_DEV=true bb nrepl-server 1667

This will print all the incoming messages.

To debug the nREPL server from source:

$ git clone https://github.com/borkdude/babashka --recurse-submodules
$ cd babashka
$ BABASHKA_DEV=true clojure -A:main --nrepl-server 1667

REPL server port

For the socket REPL, pREPL, or nREPL, if a randomized port is needed, 0 can be used anywhere a port argument is accepted.

Input and output flags

In one-liners the *input* value may come in handy. It contains the input read from stdin as EDN by default. If you want to read in text, use the -i flag, which binds *input* to a lazy seq of lines of text. If you want to read multiple EDN values, use the -I flag. The -o option prints the result as lines of text. The -O option prints the result as lines of EDN values.

*input* is only available in the user namespace, designed for one-liners. For writing scripts, see Scripts.

The following table illustrates the combination of options for commands of the form

echo "{{Input}}" | bb {{Input flags}} {{Output flags}} "*input*"
Input Input flags Output flag *input* Output

{:a 1} {:a 2}

{:a 1}

{:a 1}

hello
bye

-i

("hello" "bye")

("hello" "bye")

hello
bye

-i

-o

("hello" "bye")

hello
bye

{:a 1} {:a 2}

-I

({:a 1} {:a 2})

({:a 1} {:a 2})

{:a 1} {:a 2}

-I

-O

({:a 1} {:a 2})

{:a 1} {:a 2}

When combined with the --stream option, the expression is executed for each value in the input:

$ echo '{:a 1} {:a 2}' | bb --stream '*input*'
{:a 1}
{:a 2}

Scripts

When writing scripts instead of one-liners on the command line, it is not recommended to use *input*. Here is how you can rewrite to standard Clojure code.

EDN input

Reading a single EDN value from stdin:

(ns script
 (:require [clojure.edn :as edn]))

(edn/read *in*)

Reading multiple EDN values from stdin (the -I flag):

(ns script
 (:require [clojure.edn :as edn]
           [clojure.java.io :as io]))

(let [reader  (java.io.PushbackReader. (io/reader *in*))]
  (take-while #(not (identical? ::eof %)) (repeatedly #(edn/read {:eof ::eof} reader))))

Text input

Reading text from stdin can be done with (slurp *in*). To get a lazy seq of lines (the -i flag), you can use:

(ns script
 (:require [clojure.java.io :as io]))

(line-seq (io/reader *in*))

Output

To print to stdout, use println for text and prn for EDN values.

Uberscript

The --uberscript option collects the expressions in BABASHKA_PRELOADS, the command line expression or file, the main entrypoint and all required namespaces from the classpath into a single file. This can be convenient for debugging and deployment.

Here is an example that uses a function from the clj-commons/fs library.

Let’s first set the classpath:

$ export BABASHKA_CLASSPATH=$(clojure -Spath -Sdeps '{:deps {clj-commons/fs {:mvn/version "1.6.307"}}}')

Write a little script, say glob.clj:

(ns glob (:require [me.raynes.fs :as fs]))

(run! (comp println str)
      (fs/glob (first *command-line-args*)))

For testing, we’ll make a file which we will find using the glob function:

$ touch README.md

Now we can execute the script which uses the library:

$ time bb glob.clj '*.md'
/private/tmp/glob/README.md
bb glob.clj '*.md'   0.03s  user 0.01s system 88% cpu 0.047 total

Producing an uberscript with all required code:

$ bb uberscript glob-uberscript.clj -f glob.clj

To prove that we don’t need the classpath anymore:

$ unset BABASHKA_CLASSPATH
$ time bb glob-uberscript.clj '*.md'
/private/tmp/glob/README.md
bb glob-uberscript.clj '*.md'   0.03s  user 0.02s system 93% cpu 0.049 total

Caveats:

  • Dynamic requires. Building uberscripts works by running top-level ns and require forms. The rest of the code is not evaluated. Code that relies on dynamic requires may not work in an uberscript.

  • Resources. The usage of io/resource assumes a classpath, so when this is used in your uberscript, you still have to set a classpath and bring the resources along.

If any of the above is problematic for your project, using an uberjar is a good alternative.

Carve

Uberscripts can be optimized by cutting out unused vars with carve.

$ wc -l glob-uberscript.clj
     583 glob-uberscript.clj
$ carve --opts '{:paths ["glob-uberscript.clj"] :aggressive true :silent true}'
$ wc -l glob-uberscript.clj
     105 glob-uberscript.clj

Note that the uberscript became 72% shorter. This has a beneficial effect on execution time:

$ time bb glob-uberscript.clj '*.md'
/private/tmp/glob/README.md
bb glob-uberscript.clj '*.md'   0.02s  user 0.01s system 84% cpu 0.034 total

Uberjar

Babashka can create uberjars from a given classpath and optionally a main method:

$ cat bb/foo.clj
(ns foo)
(defn -main [& args] (prn :hello))
$ cat bb.edn
{:paths ["bb"]}
$ bb uberjar foo.jar -m foo
$ bb foo.jar
:hello

System properties

Babashka sets the following system properties:

  • babashka.version: the version string, e.g. "1.2.0"

  • babashka.main: the --main argument

  • babashka.file: the --file argument (normalized using .getAbsolutePath)

Data readers

Data readers can be enabled by setting *data-readers* to a hashmap of symbols to functions or vars:

$ bb -e "(set! *data-readers* {'t/tag inc}) #t/tag 1"
2

To preserve good startup time, babashka does not scan the classpath for data_readers.clj files.

Reader conditionals

Babashka supports reader conditionals by taking either the :bb or :clj branch, whichever comes first. NOTE: the :clj branch behavior was added in version 0.0.71, before that version the :clj branch was ignored.

$ bb -e "#?(:bb :hello :clj :bye)"
:hello

$ bb -e "#?(:clj :bye :bb :hello)"
:bye

$ bb -e "[1 2 #?@(:bb [] :clj [1])]"
[1 2]

Invoking clojure

Babashka bundles deps.clj for invoking a clojure JVM process:

$ bb clojure -M -e "*clojure-version*"
{:major 1, :minor 10, :incremental 1, :qualifier nil}

See the clojure function in the babashka.deps namespace for programmatically invoking clojure.

Project setup

bb.edn

Since version 0.3.1, babashka supports a local bb.edn file to manage a project.

:paths and :deps

You can declare one or multiple paths and dependencies so they are automatically added to the classpath:

{:paths ["bb"]
 :deps {medley/medley {:mvn/version "1.3.0"}}}

If we have a project that has a deps.edn and would like to reuse those deps in bb.edn:

{:deps {local/deps {:local/root "."}}}

If we have a main function in a file called bb/my_project/main.clj like:

(ns my-project.main
  (:require [medley.core :as m]))

(defn -main [& _args]
  (prn (m/index-by :id [{:id 1} {:id 2}])))

we can invoke it like:

$ bb -m my-project.main
{1 {:id 1}, 2 {:id 2}}

See Invoking a main function for more details on how to invoke a function from the command line.

The :deps entry is managed by deps.clj and requires a java installation to resolve and download dependencies.

:min-bb-version

Since version 0.3.6, babashka supports the :min-bb-version where the minimal babashka version can be declared:

{:paths ["src"]
 :deps {medley/medley {:mvn/version "1.3.0"}}
 :min-bb-version "0.3.7"}

When using an older bb version (that supports :min-bb-version), babashka will print a warning:

WARNING: this project requires babashka 0.3.7 or newer, but you have: 0.3.6

:tasks

Since babashka 0.4.0 the bb.edn file supports the :tasks entry which describes tasks that you can run in the current project. The tasks feature is similar to what people use Makefile, Justfile or npm run for. See Task runner for more details.

Task runner

Introduction

People often use a Makefile, Justfile, npm scripts or lein aliases in their (clojure) projects to remember complex invocations and to create shortcuts for them. Since version 0.4.0, babashka supports a similar feature as part of the bb.edn project configuration file. For a general overview of what’s available in bb.edn, go to Project setup.

The tasks configuration lives under the :tasks key and can be used together with :paths and :deps:

{:paths ["script"]
 :deps {medley/medley {:mvn/version "1.3.0"}}
 :min-bb-version "0.4.0"
 :tasks
 {clean (shell "rm -rf target")
 ...}
 }

In the above example we see a simple task called clean which invokes the shell command, to remove the target directory. You can invoke this task from the command line with:

$ bb run clean

Babashka also accepts a task name without explicitly mentioning run:

$ bb clean

To make your tasks more cross-platform friendly, you can use the built-in babashka.fs library. To use libraries in tasks, use the :requires option:

{:tasks
 {:requires ([babashka.fs :as fs])
  clean (fs/delete-tree "target")
  }
 }

Tasks accept arbitrary Clojure expressions. E.g. you can print something when executing the task:

{:tasks
 {:requires ([babashka.fs :as fs])
  clean (do (println "Removing target folder.")
            (fs/delete-tree "target"))
  }
 }
$ bb clean
Removing target folder.

Talk

Go here if you would like to watch a talk on babashka tasks.

Task-local options

Instead of naked expressions, tasks can be defined as maps with options. The task expression should then be moved to the :task key:

{:tasks
 {
  clean {:doc "Removes target folder"
         :requires ([babashka.fs :as fs])
         :task (fs/delete-tree "target")}
  }
 }

Tasks support the :doc option which gives it a docstring which is printed when invoking bb tasks on the command line. Other options include:

  • :requires: task-specific namespace requires.

  • :extra-paths: add paths to the classpath.

  • :extra-deps: add extra dependencies to the classpath.

  • :enter, :leave: override the global :enter/:leave hook.

Discoverability

When invoking bb tasks, babashka prints a list of all tasks found in bb.edn in the order of appearance. E.g. in the clj-kondo.lsp project it prints:

$ bb tasks
The following tasks are available:

recent-clj-kondo   Detects most recent clj-kondo version from clojars
update-project-clj Updates project.clj with most recent clj-kondo version
java1.8            Asserts that we are using java 1.8
build-server       Produces lsp server standalone jar
lsp-jar            Copies renamed jar for upload to clj-kondo repo
upload-jar         Uploads standalone lsp server jar to clj-kondo repo
vscode-server      Copied lsp server jar to vscode extension
vscode-version     Prepares package.json with up to date clj-kondo version
vscode-publish     Publishes vscode extension to marketplace
ovsx-publish       Publishes vscode extension to ovsx thing
publish            The mother of all tasks: publishes everything needed for new release

Command line arguments

Command line arguments are available as *command-line-args*, just like in Clojure. Since version 0.9.160, you can use babashka.cli in tasks via the exec function to deal with command line arguments in a concise way. See the chapter on babashka CLI.

Of course, you are free to parse command line arguments using the built-in tools.cli library or just handle them manually.

You can re-bind *command-line-args* to ensure functions see a different set of arguments:

{:tasks
 {:init (do (defn print-args []
              (prn (:name (current-task))
                   *command-line-args*)))
  bar (print-args)
  foo (do (print-args)
          (binding [*command-line-args* (next *command-line-args*)]
            (run 'bar)))}}
$ bb foo 1 2 3
foo ("1" "2" "3")
bar ("2" "3")

Terminal tab-completion

zsh

Add this to your .zshrc to get tab-complete feature on ZSH.

_bb_tasks() {
    local matches=(`bb tasks |tail -n +3 |cut -f1 -d ' '`)
    compadd -a matches
    _files # autocomplete filenames as well
}
compdef _bb_tasks bb
bash

Add this to your .bashrc to get tab-complete feature on bash.

_bb_tasks() {
    COMPREPLY=( $(compgen -W "$(bb tasks |tail -n +3 |cut -f1 -d ' ')" -- ${COMP_WORDS[COMP_CWORD]}) );
}
# autocomplete filenames as well
complete -f -F _bb_tasks bb
fish

Add this to your .config/fish/completions/bb.fish to get tab-complete feature on Fish shell.

function __bb_complete_tasks
  if not test "$__bb_tasks"
    set -g __bb_tasks (bb tasks |tail -n +3 |cut -f1 -d ' ')
  end

  printf "%s\n" $__bb_tasks
end

complete -c bb -a "(__bb_complete_tasks)" -d 'tasks'

Run

You can execute tasks using bb <task-name>. The babashka run subcommand can be used to execute with some additinoal options:

  • --parallel: invoke task dependencies in parallel.

    {:tasks
 {:init (def log (Object.))
  :enter (locking log
           (println (str (:name (current-task))
                         ":")
                    (java.util.Date.)))
  a (Thread/sleep 5000)
  b (Thread/sleep 5000)
  c {:depends [a b]}
  d {:task (time (run 'c))}}}
    $ bb run --parallel d
d: #inst "2021-05-08T14:14:56.322-00:00"
a: #inst "2021-05-08T14:14:56.357-00:00"
b: #inst "2021-05-08T14:14:56.360-00:00"
c: #inst "2021-05-08T14:15:01.366-00:00"
"Elapsed time: 5023.894512 msecs"

    Also see Parallel tasks.

  • --prn: print the result from the task expression:

    {:tasks {sum (+ 1 2 3)}}
    $ bb run --prn sum
6

    Unlike scripts, babashka tasks do not print their return value.

Hooks

The task runner exposes the following hooks:

:init

The :init is for expressions that are executed before any of the tasks are executed. It is typically used for defining helper functions and constants:

{:tasks
 {:init (defn env [s] (System/getenv s))
  print-env (println (env (first *command-line-args*)))
  }
 }
$ FOO=1 bb print-env FOO
1

:enter, :leave

The :enter hook is executed before each task. This is typically used to print the name of a task, which can be obtained using the current-task function:

{:tasks
 {:init (defn env [s] (System/getenv s))
  :enter (println "Entering:" (:name (current-task)))
  print-env (println (env (first *command-line-args*)))
  }
 }
$ FOO=1 bb print-env FOO
Entering: print-env
1

The :leave hook is similar to :enter but is executed after each task.

Both hooks can be overriden as task-local options. Setting them to nil will disable them for specific tasks (see Task-local options).

Tasks API

The babashka.tasks namespace exposes the following functions: run, shell, clojure and current-task. They are implicitly imported, thus available without a namespace prefix.

run

Tasks provide the run function to explicitly invoke another task:

{:tasks
 {:requires ([babashka.fs :as fs])

  clean (do
          (println "Removing target folder.")
          (fs/delete-tree "target"))
  uberjar (do
            (println "Making uberjar")
            (clojure "-X:uberjar"))
  uberjar:clean (do (run 'clean)
                    (run 'uberjar))}
 }

When running bb uberjar:clean, first the clean task is executed and the uberjar:

$ bb uberjar:clean
Removing target folder.
Making uberjar

The clojure function in the above example executes a clojure process using deps.clj. See clojure for more info

The run function accepts an additional map with options:

:parallel

The :parallel option executes dependencies of the invoked task in parallel (when possible). See Parallel tasks.

shell

Both shell and clojure return a process object which returns the :exit code among other info. By default these functions will throw an exception when a non-zero exit code was returned and they will inherit the stdin/stdout/stderr from the babashka process.

{:tasks
 {
  ls (shell "ls foo")
 }
}
$ bb ls
ls: foo: No such file or directory
Error while executing task: ls
$ echo $?
1

You can opt out of this behavior by using the :continue option:

{:tasks
 {
  ls (shell {:continue true} "ls foo")
 }
}
$ bb ls
ls: foo: No such file or directory
$ echo $?
0

When you want to redirect output to a file instead, you can provide the :out option.

(shell {:out "file.txt"} "echo hello")

To run a program in another directory, you can use the :dir option:

(shell {:dir "subproject"} "ls")

To set environment variables with shell or clojure:

(shell {:extra-env {"FOO" "BAR"}} "printenv FOO")

Other supported options are similar to those of babashka.process/process.

The process is executed synchronously: i.e. babashka will wait for the process to finish before executing the next expression. If this doesn’t fit your use case, you can use babashka.process/process directly instead. These two invocations are roughly equivalent:

(require '[babashka.process :as p :refer [process]]
         '[babashka.tasks :as tasks])

(tasks/shell {:dir "subproject"} "npm install")

(-> (process "npm install" {:dir "subproject" :inherit true})
    (p/check))

Note that the first string argument to shell it tokenized (broken into multiple parts) and the trailing arguments are not:

Correct:

(shell "npm install" "-g" "nbb")

Not correct (-g nbb within the same string):

(shell "npm install" "-g nbb")

Note that the varargs signature plays well with feeding *command-line-args*:

(apply shell "npm install" *command-line-args*)

Also see the docstring of shell here.

clojure

The clojure function starts a Clojure process using deps.clj. The interface is exactly the same as the clojure CLI. E.g. to evaluate an expression:

{:tasks {eval (clojure "-M -e '(+ 1 2 3)'")}}

or to invoke clj-kondo as a library:

{:tasks {eval (clojure {:dir "subproject"} "-M:clj-kondo")}}

The clojure task function behaves similar to shell with respect to the exit code, return value and supported options, except when it comes to features that do not start a process, but only do some printing. E.g. you can capture the classpath using:

(with-out-str (clojure "-Spath"))

because this operation doesn’t start a process but prints to *out*.

To run a clojure task in another directory:

{:tasks {eval (clojure {:dir "subproject"} "-M:clj-kondo")}}

current-task

The current-task function returns a map representing the currently running task. This function is typically used in the :enter and :leave hooks.

exec

See exec.

Dependencies between tasks

Dependencies between tasks can be declared using :depends:

{:tasks {:requires ([babashka.fs :as fs])
         -target-dir "target"
         -target {:depends [-target-dir]
                  :task (fs/create-dirs -target-dir)}
         -jar-file {:depends [-target]
                    :task "target/foo.jar"}

         jar {:depends [-target -jar-file]
              :task (when (seq (fs/modified-since -jar-file
                                             (fs/glob "src" "**.clj")))
                      (spit -jar-file "test")
                      (println "made jar!"))}
         uberjar {:depends [jar]
                  :task (println "creating uberjar!")}}}

The fs/modified-since function returns a seq of all newer files compared to a target, which can be used to prevent rebuilding artifacts when not necessary.

Alternatively you can use the :init hook to define vars, require namespaces, etc.:

{:tasks {:requires ([babashka.fs :as fs])
         :init (do (def target-dir  "target")
                   (def jar-file "target/foo.jar"))
         -target {:task (fs/create-dirs target-dir)}
         jar {:depends [-target]
              :task (when (seq (fs/modified-since jar-file
                                             (fs/glob "src" "**.clj")))
                      (spit jar-file "test")
                      (println "made jar!"))}
         uberjar {:depends [jar]
                  :task (println "creating uberjar!")}}}

It is common to define tasks that only serve as a helper to other tasks. To not expose these tasks in the output of bb tasks, you can start their name with a hyphen.

Parallel tasks

The :parallel option executes dependencies of the invoked task in parallel (when possible). This can be used to speed up execution, but also to have multiple tasks running in parallel for development:

dev         {:doc  "Runs app in dev mode. Compiles cljs, less and runs JVM app in parallel."
             :task (run '-dev {:parallel true})}       (1)
-dev        {:depends [dev:cljs dev:less dev:backend]} (2)
dev:cljs    {:doc  "Runs front-end compilation"
             :task (clojure "-M:frontend:cljs/dev")}
dev:less    {:doc  "Compiles less"
             :task (clojure "-M:frontend:less/dev")}
dev:backend {:doc  "Runs backend in dev mode"
             :task (clojure (str "-A:backend:backend/dev:" platform-alias)
                            "-X" "dre.standalone/start")}
1 The dev task invokes the (private) -dev task in parallel
2 The -dev task depends on three other tasks which are executed simultaneously.

Invoking a main function

Invoking a main function can be done by providing a fully qualified symbol:

{:tasks
  {foo-bar foo.bar/-main}}

You can use any fully qualified symbol, not just ones that end in -main (so e.g. foo.bar/baz is fine). You can also have multiple main functions in one namespace.

The namespace foo.bar will be automatically required and the function will be invoked with *command-line-args*:

$ bb foo-bar 1 2 3

REPL

To get a REPL within a task, you can use clojure.main/repl:

{:tasks {repl (clojure.main/repl)}}

Alternatively, you can use babashka.tasks/run to invoke a task from a REPL.

For REPL- and linting-friendliness, it’s recommended to move task code longer than a couple of lines to a .clj or .bb file.

Real world examples

Naming

Valid names

When running a task, babashka assembles a small program which defines vars bound to the return values of tasks. This brings the limitation that you can only choose names for your tasks that are valid as var names. You can’t name your task foo/bar for this reason. If you want to use delimiters to indicate some sort of grouping, you can do it like foo-bar, foo:bar or foo_bar.

Names starting with a - are considered "private" and not listed in the bb tasks output.

Conflicting file / task / subcommand names

bb <option> is resolved in the order of file > task > subcommand.

Escape hatches in case of conflicts:

  • execute relative file as bb ./foo

  • execute task as bb run foo

  • execute subcommand as bb --foo

Conflicting task and clojure.core var names

You can name a task similar to a core var, let’s say: format. If you want to refer to the core var, it is recommended to use the fully qualified clojure.core/format in that case, to avoid conflicts in :enter and :leave expressions and when using the format task as a dependency.

Syntax

Because bb.edn is an EDN file, you cannot use all of Clojure’s syntax in expressions. Most notably:

  • You cannot use #(foo %), but you can use (fn [x] (foo x))

  • You cannot use @(foo) but you can use (deref foo)

  • You cannot use #"re" but you can use (re-pattern "re")

  • Single quotes are accidentally supported in some places, but are better avoided: {:task '(foo)} does not work, but {:task (quote (foo)) does work. When requiring namespaces, use the :requires feature in favor of doing it manually using (require '[foo]).

Babashka CLI

In version 0.9.160 of babashka, the babashka CLI added as a built-in library together with task integration.

-x

For invoking functions from the command line, you can use the new -x flag (a pun to Clojure’s -X of course!):

bb -x clojure.core/identity --hello there
{:hello "there"}

What we see in the above snippet is that a map {:hello "there"} is constructed by babashka CLI and then fed to the identity function. After that the result is printed to the console.

What if we want to influence how things are parsed by babashka CLI and provide some defaults? This can be done using metadata. Let’s create a bb.edn and make a file available on the classpath:

bb.edn:

{:paths ["."]}

tasks.clj:

(ns tasks
  {:org.babashka/cli {:exec-args {:ns-data 1}}})

(defn my-function
  {:org.babashka/cli {:exec-args {:fn-data 1}
                      :coerce {:num [:int]}
                      :alias {:n :num}}}
  [m]
  m)

Now let’s invoke:

$ bb -x tasks/my-function -n 1 2
{:ns-data 1, :fn-data 1, :num [1 2]}

As you can see, the namespace options are merged with the function options. Defaults can be provided with :exec-args, like you’re used to from the clojure CLI.

exec

What about task integration? Let’s adapt our bb.edn:

{:paths ["."]
 :tasks {doit {:task (let [x (exec 'tasks/my-function)]
                       (prn :x x))
               :exec-args {:task-data 1234}}
         }}

and invoke the task:

$ bb doit --cli-option :yeah -n 1 2 3
:x {:ns-data 1, :fn-data 1, :task-data 1234, :cli-option :yeah, :num [1 2 3]}

As you can see it works similar to -x, but you can provide another set of defaults on the task level with :exec-args. Executing a function through babashka CLI is done using the babashka.task/exec function, available by default in tasks.

Libraries

Built-in namespaces

In addition to clojure.core, the following libraries / namespaces are available in babashka. Some are available through pre-defined aliases in the user namespace, which can be handy for one-liners. If not all vars are available, they are enumerated explicitly. If some important var is missing, an issue or PR is welcome.

From Clojure:

  • clojure.core

  • clojure.core.protocols: Datafiable, Navigable

  • clojure.data

  • clojure.datafy

  • clojure.edn aliased as edn

  • clojure.math

  • clojure.java.browse

  • clojure.java.io aliased as io:

    • as-relative-path, as-url, copy, delete-file, file, input-stream, make-parents, output-stream, reader, resource, writer

  • clojure.java.shell aliased as shell

  • clojure.main: demunge, repl, repl-requires

  • clojure.pprint: pprint, cl-format

  • clojure.set aliased as set

  • clojure.string aliased as str

  • clojure.stacktrace

  • clojure.test

  • clojure.zip

Additional libraries:

Check out the babashk toolbox and projects page for libraries that are not built-in, but which you can load as an external dependency in bb.edn.

See the build page for built-in libraries that can be enabled via feature flags, if you want to compile babashka yourself.

A selection of Java classes are available, see babashka/impl/classes.clj in babashka’s git repo.

Babashka namespaces

babashka.classpath

Available functions:

  • add-classpath

  • get-classpath

  • split-classpath

add-classpath

The function add-classpath which can be used to add to the classpath dynamically:

(require '[babashka.classpath :refer [add-classpath]]
         '[clojure.java.shell :refer [sh]]
         '[clojure.string :as str])

(def medley-dep '{:deps {medley {:git/url "https://github.com/borkdude/medley"
                                 :sha "91adfb5da33f8d23f75f0894da1defe567a625c0"}}})
(def cp (-> (sh "clojure" "-Spath" "-Sdeps" (str medley-dep)) :out str/trim))
(add-classpath cp)
(require '[medley.core :as m])
(m/index-by :id [{:id 1} {:id 2}]) ;;=> {1 {:id 1}, 2 {:id 2}}
get-classpath

The function get-classpath returns the classpath as set by --classpath, BABASHKA_CLASSPATH and add-classpath.

split-classpath

Given a classpath, returns a seq of strings as the result of splitting the classpath by the platform specific path separatator.

babashka.deps

Available functions:

  • add-deps

  • clojure

  • merge-deps

add-deps

The function add-deps takes a deps edn map like {:deps {medley/medley {:mvn/version "1.3.0"}}}, resolves it using deps.clj and then adds to the babashka classpath accordingly.

Example:

(require '[babashka.deps :as deps])

(deps/add-deps '{:deps {medley/medley {:mvn/version "1.3.0"}}})

(require '[medley.core :as m])
(m/index-by :id [{:id 1} {:id 2}])

Optionally, add-deps takes a second arg with options. Currently the only option is :aliases which will affect how deps are resolved:

Example:

(deps/add-deps '{:aliases {:medley {:extra-deps {medley/medley {:mvn/version "1.3.0"}}}}}
               {:aliases [:medley]})
clojure

The function clojure takes a sequential collection of arguments, similar to the clojure CLI. The arguments are then passed to deps.clj. The clojure function returns nil and prints to *out* for commands like -Stree, and -Spath. For -M, -X and -A it invokes java with babashka.process/process (see babashka.process) and returns the associated record. For more details, read the docstring with:

(require '[clojure.repl :refer [doc]])
(doc babashka.deps/clojure)

Example:

The following script passes through command line arguments to clojure, while adding the medley dependency:

(require '[babashka.deps :as deps])

(def deps '{:deps {medley/medley {:mvn/version "1.3.0"}}})
(def clojure-args (list* "-Sdeps" deps  *command-line-args*))

(if-let [proc (deps/clojure clojure-args)]
  (-> @proc :exit (System/exit))
  (System/exit 0))

babashka.wait

Contains the functions: wait-for-port and wait-for-path.

Usage of wait-for-port:

(wait/wait-for-port "localhost" 8080)
(wait/wait-for-port "localhost" 8080 {:timeout 1000 :pause 1000})

Waits for TCP connection to be available on host and port. Options map supports :timeout and :pause. If :timeout is provided and reached, :default's value (if any) is returned. The :pause option determines the time waited between retries.

Usage of wait-for-path:

(wait/wait-for-path "/tmp/wait-path-test")
(wait/wait-for-path "/tmp/wait-path-test" {:timeout 1000 :pause 1000})

Waits for file path to be available. Options map supports :default, :timeout and :pause. If :timeout is provided and reached, :default's value (if any) is returned. The :pause option determines the time waited between retries.

The namespace babashka.wait is aliased as wait in the user namespace.

babashka.signal

Contains the function signal/pipe-signal-received?. Usage:

(signal/pipe-signal-received?)

Returns true if PIPE signal was received. Example:

$ bb -e '((fn [x] (println x) (when (not (signal/pipe-signal-received?)) (recur (inc x)))) 0)' | head -n2
1
2

The namespace babashka.signal is aliased as signal in the user namespace.

babashka.curl

The namespace babashka.curl is a tiny wrapper around curl. It’s aliased as curl in the user namespace. See babashka.curl for how to use it.

babashka.process

The babashka.process library. See the process repo for API docs.

babashka.fs

The babashka.fs library offers file system utilities. See the fs repo for API docs.

babashka.cli

The babashka.cli library allows you to turn functions into CLIs. See the cli repo for API docs and check out the babashka CLI chapter on how to use it from the command line or with tasks.

Projects

Babashka is able to run Clojure projects from source, if they are compatible with the subset of Clojure that sci is capable of running.

Check this page for projects that are known to work with babashka.

Do you have a library that is compatible with babashka? Add the official badge to give some flair to your repo!

svg+xml;base64,PHN2ZyB3aWR0aD0iMTQiIGhlaWdodD0iMTQiIHZpZXdCb3g9IjAgMCAxNDcyIDE0NzIiIGZpbGw9Im5vbmUiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+CjxwYXRoIGZpbGwtcnVsZT0iZXZlbm9kZCIgY2xpcC1ydWxlPSJldmVub2RkIiBkPSJNOTk1LjkxMSAxMTkzLjg5QzExMTAuOTMgMTExMi4zOCAxMTg2IDk3OC4yMDYgMTE4NiA4MjYuNUMxMTg2IDU3Ny45NzIgMTAxNCAyNTQuNSA3MzYgMTEzQzQ2MC41IDI2MiAyODYgNTc3Ljk3MiAyODYgODI2LjVDMjg2IDk3OC4yMDYgMzYxLjA3IDExMTIuMzggNDc2LjA4OSAxMTkzLjg5QzQ3MS45ODMgMTE5NC40NCA0NjcuOTQ5IDExOTUuMTQgNDY0IDExOTZDMzc2LjYxMSAxMjE1LjA3IDI3MCAxMzU5LjA1IDI3MCAxMzU5LjA1SDUzNEM1ODAuOTcyIDEzNTkuMDUgNjI1LjYzMSAxMzQxLjQxIDY2MC45NTYgMTMyMS42MkM2NzguMTE1IDEzNDQuMzIgNzA1LjM0NCAxMzU5IDczNiAxMzU5Qzc2Ni42NTYgMTM1OSA3OTMuODg1IDEzNDQuMzIgODExLjA0NCAxMzIxLjYyQzg0Ni4zNjkgMTM0MS40MSA4OTEuMDI4IDEzNTkuMDUgOTM4IDEzNTkuMDVIMTIwMkMxMjAyIDEzNTkuMDUgMTA5NS4zOSAxMjE1LjA3IDEwMDggMTE5NkMxMDA0LjA1IDExOTUuMTQgMTAwMC4wMiAxMTk0LjQ0IDk5NS45MTEgMTE5My44OVoiIGZpbGw9InVybCgjcGFpbnQwX2xpbmVhcikiLz4KPGNpcmNsZSBjeD0iNzM1LjUiIGN5PSI4NTIuNSIgcj0iMzExLjUiIGZpbGw9IndoaXRlIi8+CjxwYXRoIGQ9Ik00NDMgNzUwLjQxN0M2NjIgNzY1Ljg2NiA3OTEuMzgyIDcxMi4zODggODgxIDU3OS44ODFDODgxIDU3OS44ODEgNzA4LjI5MyA1MjEuNjQ5IDYwMy4yNDQgNTc5Ljg4MUM0OTguNTUgNjM3LjkxNiA0NDMgNzUwLjQxNyA0NDMgNzUwLjQxN1oiIGZpbGw9IiNCNEI0QjQiLz4KPHBhdGggZD0iTTEwMzkuOTkgNzQ4LjI4MkM5MTQuODYyIDczNC43NDUgODAzLjI2NCA2OTQuMTM1IDc1MCA1NDFDNzUwIDU0MSA4NjAuNjMzIDU2Ni4yNzIgOTcyLjM1NSA2NjguMTAxQzEwODguMzkgNzczLjg1NyAxMDM5Ljk5IDc0OC4yODIgMTAzOS45OSA3NDguMjgyWiIgZmlsbD0iI0I0QjRCNCIvPgo8Y2lyY2xlIGN4PSI3MzUuNSIgY3k9Ijg1Mi41IiByPSIzMTEuNSIgc3Ryb2tlPSIjMEEwMDAwIiBzdHJva2Utd2lkdGg9IjQwIi8+CjxwYXRoIGQ9Ik03OTcgNzE5SDY3NEw3MDQgNzQ5VjgzOUM3MDguNSA4MDkuNSA3NjQuNSA4MDguNSA3NjcuNSA4MzlMNzc1IDc0OUw3OTcgNzE5WiIgZmlsbD0iYmxhY2siLz4KPHBhdGggZD0iTTM1MS4xMTcgNzU5QzM1MS4xMTcgNzM2LjkwOSAzNjkuMDI2IDcxOSAzOTEuMTE3IDcxOUg2NzYuMDE4QzcwMC4yIDcxOSA3MTguODUyIDc0MC4yOTIgNzE1LjY3IDc2NC4yNjRMNjkwLjkxIDk1MC43NjRDNjg4LjI3IDk3MC42NDYgNjcxLjMxNCA5ODUuNSA2NTEuMjU4IDk4NS41SDM5MS4xMTdDMzY5LjAyNiA5ODUuNSAzNTEuMTE3IDk2Ny41OTEgMzUxLjExNyA5NDUuNVY3NTlaIiBmaWxsPSJibGFjayIvPgo8cGF0aCBkPSJNNzU2LjAxMiA3NjQuMjY3Qzc1Mi44MjggNzQwLjI5NCA3NzEuNDgxIDcxOSA3OTUuNjY0IDcxOUgxMDgwQzExMDIuMDkgNzE5IDExMjAgNzM2LjkwOSAxMTIwIDc1OVY5NDUuMDk2QzExMjAgOTY3LjE4NyAxMTAyLjA5IDk4NS4wOTYgMTA4MCA5ODUuMDk2SDgyMC4zODFDODAwLjMyNSA5ODUuMDk2IDc4My4zNyA5NzAuMjQ0IDc4MC43MjkgOTUwLjM2Mkw3NTYuMDEyIDc2NC4yNjdaIiBmaWxsPSJibGFjayIvPgo8cGF0aCBkPSJNMTAyNCA5NTJWOTIxLjQ3SDEwMTMuNzFDMTAwNy4wMyA5MjEuNDcgMTAwMi41OCA5MTYuNzUyIDEwMDAuMzUgOTA5LjgxM0w5NTguMDY0IDc4Mi42OTdDOTU1LjI4MiA3NzQuNjQ5IDk1MS45NDQgNzY4LjgyIDk0Ny40OTIgNzY0LjM3OUM5MzkuMTQ2IDc1NS43NzUgOTI4LjI5NiA3NTMgOTE2LjA1NCA3NTNIOTAyLjdWNzg0LjkxOEg5MTEuNjAzQzkxOS4xMTUgNzg0LjkxOCA5MjQuOTU3IDc4Ny42OTMgOTI3LjQ2MSA3OTYuNTc1TDkzMy4zMDMgODE3LjExM0w4ODEgOTUySDkxOS4xMTVMOTUwLjU1MyA4NjMuMTg1TDk2Ny4yNDUgOTE2Ljc1MkM5NzMuNjQ0IDkzNy41NjggOTg0LjQ5NCA5NTIgMTAwOC45OCA5NTJIMTAyNFoiIGZpbGw9IndoaXRlIi8+CjxwYXRoIGQ9Ik01OTAgOTUyVjkyMS40N0g1NzkuNzA2QzU3My4wMjkgOTIxLjQ3IDU2OC41NzggOTE2Ljc1MiA1NjYuMzUyIDkwOS44MTNMNTI0LjA2NCA3ODIuNjk3QzUyMS4yODIgNzc0LjY0OSA1MTcuOTQ0IDc2OC44MiA1MTMuNDkyIDc2NC4zNzlDNTA1LjE0NiA3NTUuNzc1IDQ5NC4yOTYgNzUzIDQ4Mi4wNTQgNzUzSDQ2OC43Vjc4NC45MThINDc3LjYwM0M0ODUuMTE1IDc4NC45MTggNDkwLjk1NyA3ODcuNjkzIDQ5My40NjEgNzk2LjU3NUw0OTkuMzAzIDgxNy4xMTNMNDQ3IDk1Mkg0ODUuMTE1TDUxNi41NTMgODYzLjE4NUw1MzMuMjQ1IDkxNi43NTJDNTM5LjY0NCA5MzcuNTY4IDU1MC40OTQgOTUyIDU3NC45NzcgOTUySDU5MFoiIGZpbGw9IndoaXRlIi8+CjxkZWZzPgo8bGluZWFyR3JhZGllbnQgaWQ9InBhaW50MF9saW5lYXIiIHgxPSI3NTIiIHkxPSIxMTMiIHgyPSI3NTIiIHkyPSIxMzU5LjUyIiBncmFkaWVudFVuaXRzPSJ1c2VyU3BhY2VPblVzZSI+CjxzdG9wIHN0b3AtY29sb3I9IiNFNDFGMjYiLz4KPHN0b3Agb2Zmc2V0PSIxIiBzdG9wLWNvbG9yPSIjQjcwMDAwIi8+CjwvbGluZWFyR3JhZGllbnQ+CjwvZGVmcz4KPC9zdmc+Cg==

Pods

Pods are programs that can be used as a Clojure library by babashka. Documentation is available in the library repo.

A list of available pods can be found here.

Pod registry

Since bb 0.2.6 pods can be obtained via the pod-registry.

This is an example script which uses the fswatcher pod to watch a directory for changes:

#!/usr/bin/env bb

(require '[babashka.pods :as pods])
(pods/load-pod 'org.babashka/fswatcher "0.0.3")

(require '[pod.babashka.fswatcher :as fw])

(fw/watch "." prn {:delay-ms 5000})

(println "Watching current directory for changes... Press Ctrl-C to quit.")

@(promise)

Pods in bb.edn

Since bb 0.8.0 pods can be declared in bb.edn:

{:paths ["bb"]
 :pods {org.babashka/go-sqlite3 {:version "0.1.0"}}}

Given the file bb/my_project/db.clj:

(ns my-project.db
  (:require [pod.babashka.go-sqlite3 :as sqlite]))

(defn -main [& _args]
  (prn (sqlite/query ":memory:" ["SELECT 1 + 1 AS sum"])))

you can then execute the main function, without calling load-pod manually:

$ bb -m my-project.db
[{:sum 2}]

Style

A note on style. Babashka recommends the following:

Explicit requires

Use explicit requires with namespace aliases in scripts, unless you’re writing one-liners.

Do this:

$ ls | bb -i '(-> *input* first (str/includes? "m"))'
true

But not this:

script.clj:

(-> *input* first (str/includes? "m"))

Rather do this:

script.clj:

(ns script
  (:require [clojure.java.io :as io]
            [clojure.string :as str]))
(-> (io/reader *in*) line-seq first (str/includes? "m"))

Some reasons for this:

  • Linters like clj-kondo work better with code that uses namespace forms, explicit requires, and known Clojure constructs

  • Editor tooling works better with namespace forms (sorting requires, etc).

  • Writing compatible code gives you the option to run the same script with clojure

Child processes

by Michiel Borkent

There are several ways of creating child processes in babashka. Let’s start with the easiest one.

clojure.java.shell

A common way to shell out to another process is via clojure.java.shell:

user=> (require '[clojure.java.shell :refer [sh]])
nil
user=> (sh "ls")
{:exit 0, :out "README.md\ndist\ngh-pages\nscript\nsrc\n", :err ""}

As you can see the result of :out are the lines of text produced by ls. The :exit code was 0 and there was no output on stderr.

user=> (sh "ls" "foo")
{:exit 1, :out "", :err "ls: foo: No such file or directory\n"}

If we invoke ls with a non-existing file, there is error output, but not output on stdout and the :exit code is 1.

For more information on clojure.java.shell, read the API documentation

babashka.process

Creating a child process with clojure.java.shell/sh is convenient and sufficient in 90% of the scripts you’re going to write with babashka. For the remaining 10% you’re going to need something slightly more powerful. This is where the babashka.process library namespace comes in. The README and source code of this library is located here.

What if we wanted to start a Clojure pREPL in our script, send commands to it and read the evaluated output in a loop? Using clojure.java.shell/sh for this has the limitation that we can only send input and read output once per process. Using babashka.process which leverages the java.lang.ProcessBuilder and java.lang.Process classes gives us the extra power we need to overcome this limitation.

First we show how to use process to start a process that inherits stdout, stderr and stdin from the parent process. This is often useful when you want to hand over control to another process at the end of a script. The following code starts a pREPL process:

process/prepl_1.clj
#!/usr/bin/env bb

(require '[babashka.process :as p])

(defn io-prepl []
  (let [cmd ["clojure" "-M"
             "-e" "(require '[clojure.core.server :as s])"
             "-e" "(s/io-prepl)"]                           (1)
        proc (p/process cmd                                 (2)
                        {:inherit true                      (3)
                         :shutdown p/destroy-tree})]        (4)
    proc))

@(io-prepl)                                                 (5)
1 The command and its arguments
2 Creating the process
3 This option redirects stdout and stdin from the parent process to the child process.
4 This line will destroy the child process and all of it’s children processes before the parent process exits.
5 Dereferencing the process waits until it ends. If we would not include this, our script would end immediately.

When calling the script, we can interact with the pREPL

$ bb src/process/prepl_1.clj
(+ 1 2 3)
{:tag :ret, :val "6", :ns "user", :ms 5, :form "(+ 1 2 3)"}

Pressing ctrl-C or typing :repl/quit will make the script terminate.

What if we want to parse the output of the prepl and reformat it to our own likings? Instead of sending the output of the child process to stdout we will have to get a hold of it. We can grab the output by not using :inherit and getting the :out stream of the process. Similarly we will grab the :in stream of the process and handle reading ourselves.

process/prepl_2.clj
#!/usr/bin/env bb

(require '[babashka.process :as p])

(defn io-prepl []
  (let [cmd ["clojure" "-M"
             "-e" "(require '[clojure.core.server :as s])"
             "-e" "(s/io-prepl)"]
        proc (p/process cmd
                        {:shutdown p/destroy-tree})]
    proc))

(def prepl-process (io-prepl))

(require '[clojure.java.io :as io])

(def input-writer (io/writer (:in prepl-process)))        (1)
(def output-reader (java.io.PushbackReader.
                    (io/reader (:out prepl-process))))    (2)

(require '[clojure.edn :as edn])

(loop []
  (println "Type an expression to evaluate:")
  (when-let [v (read-line)]                               (3)
    (binding [*out* input-writer]                         (4)
      (println v))
    (let [next-val (edn/read {:eof ::EOF} output-reader)] (5)
      (when-not (identical? ::EOF next-val)
        (println (:form next-val)                         (6)
                 "evaluates to"
                 (:val next-val))
        (recur)))))
1 A writer that we will use to send input to the pREPL
2 A reader that reads from the pREPL output. pREPL returns EDN and clojure.edn/read needs a PushbackReader.
3 We read user input via clojure.core/read-line
4 Here we binding *out* to the input writer so we can use println to write to it.
5 Read the next EDN value.
6 Print the output using our custom formatting.

Here is an example run:

$ bb src/process/prepl_2.clj
Type an expression to evaluate:
(+ 1 2 3)
(+ 1 2 3) evaluates to 6
:repl/quit

Real world examples

Prior to babashka.process people were using raw ProcessBuilder interop. Here are some real world examples. These might get updated to babashka.process in the future.

  • The babashka repo uses ProcessBuilder to invoke git to make a commit after bumping the version. It redirects the output from git directly to stdout of the parent process.

  • babashka.curl uses ProcessBuilder to interact with the curl executable

  • deps.clj uses ProcessBuilder to interact with java, e.g. to start a Clojure REPL or compute a classpath using tools.deps

Recipes

Running tests

Babashka bundles clojure.test. To run tests you can write a test runner script. Given the following project structure:

.
├── src
│   └──...
└── test
    └── your
        ├── test_a.clj
        └── test_b.clj
test-runner.clj
#!/usr/bin/env bb

(require '[clojure.test :as t]
         '[babashka.classpath :as cp])

(cp/add-classpath "src:test")                        (1)

(require 'your.test-a 'your.test-b)                  (2)

(def test-results
  (t/run-tests 'your.test-a 'your.test-b))           (3)

(def failures-and-errors
  (let [{:keys [fail error]} test-results]
    (+ fail error)))                                 (4)

(System/exit failures-and-errors)                    (5)
1 Add sources and tests to the classpath
2 Require the test namespaces
3 Run all tests in the test namespaces
4 Extract failures and errors
5 Exit the test script with a non-zero exit code when there are failures or errors

Main file

In Python scripts there is a well-known pattern to check if the current file was the file invoked from the command line, or loaded from another file: the __name__ == "__main__" pattern. In babashka this pattern can be implemented with:

(= *file* (System/getProperty "babashka.file"))

Combining this with a conditional invocation of -main creates a script file that is safe to load at a REPL, and easy to invoke at the CLI.

#!/usr/bin/env bb

;; Various functions defined here

(defn -main [& args]
;; Implementation of main
)

(when (= *file* (System/getProperty "babashka.file"))
  (apply -main *command-line-args*))

This can be exceedingly handy for editing complex scripts interactively, while not being able to adjust how they are invoked by other tools.

Shutdown hook

Adding a shutdown hook allows you to execute some code before the script exits.

$ bb -e '(-> (Runtime/getRuntime) (.addShutdownHook (Thread. #(println "bye"))))'
bye

This also works when the script is interrupted with ctrl-c.

Printing returned values

Babashka doesn’t print a returned nil as lots of scripts end in something side-effecting.

$ bb -e '(:a {:a 5})'
5
$ bb -e '(:b {:a 5})'
$

If you really want to print the nil, you can use (prn ..) instead.

HTTP requests

For making HTTP requests you can use:

  • babashka.curl. This library is included with babashka and aliased as curl in the user namespace. The interface is similar to that of clj-http but it will shell out to curl to make requests.

  • org.httpkit.client

  • slurp for simple GET requests

  • clj-http-lite as a library.

  • clojure.java.shell or babashka.process for shelling out to your favorite command line http client

Choosing the right client

If memory usage is a concern and you are downloading big files, choose babashka.curl with :as :stream over org.httpkit.client since http-kit holds the entire response in memory at once. Let’s download a 200mb file with 10mb heap size:

$ bb -Xmx10m -e '(io/copy (:body (curl/get "http://ipv4.download.thinkbroadband.com/200MB.zip" {:as :stream})) (io/file "/tmp/200mb.zip"))'

With babashka.curl this works fine. However with org.httpkit.client that won’t work. Not even 190mb of heap will do:

$ bb -Xmx190m -e '(io/copy (:body @(org.httpkit.client/get "http://ipv4.download.thinkbroadband.com/200MB.zip" {:as :stream})) (io/file "/tmp/200mb.zip"))'
Sun Nov 08 23:01:46 CET 2020 [client-loop] ERROR - select exception, should not happen
java.lang.OutOfMemoryError: Array allocation too large.

If your script creates many requests with relatively small payloads, choose org.httpkit.client over babashka.curl since babashka.curl creates a curl process for each request.

In the future babashka (1.0.0?) may come with an HTTP client based on the JVM 11 java.net.http package that ticks all the boxes (async, HTTP/2, websockets, multi-part file uploads, sane memory usage) and is a suitable replacement for all of the above options. If you know about a GraalVM-friendly feature-complete well-maintained library, please reach out!

HTTP over Unix sockets

This can be useful for talking to Docker:

(require '[clojure.java.shell :refer [sh]])
(require '[cheshire.core :as json])
(-> (sh "curl" "--silent"
        "--no-buffer" "--unix-socket"
        "/var/run/docker.sock"
        "http://localhost/images/json")
    :out
    (json/parse-string true)
    first
    :RepoTags) ;;=> ["borkdude/babashka:latest"]

Core.async

In addition to future, pmap, promise and friends, you may use the clojure.core.async namespace for asynchronous scripting. The following example shows how to get first available value from two different processes:

bb -e '
(defn async-command [& args]
  (async/thread (apply shell/sh "bash" "-c" args)))

(-> (async/alts!! [(async-command "sleep 2 && echo process 1")
                   (async-command "sleep 1 && echo process 2")])
    first :out str/trim println)'
process 2

Caveat: currently the go macro is available for compatibility with JVM programs, but the implementation maps to clojure.core.async/thread and the single exclamation mark operations (<!, >!, etc.) map to the double exclamation mark operations (<!!, >!!, etc.). It will not "park" threads, like on the JVM.

Examples like the following may still work, but will take a lot more system resources than on the JVM and will break down for some high value of n:

(require '[clojure.core.async :as async])

(def n 1000)

(let [cs (repeatedly n async/chan)
      begin (System/currentTimeMillis)]
  (doseq [c cs] (async/go (async/>! c "hi")))
  (dotimes [_ n]
    (let [[v _] (async/alts!! cs)]
      (assert (= "hi" v))))
  (println "Read" n "msgs in" (- (System/currentTimeMillis) begin) "ms"))

Interacting with an nREPL server

Babashka comes with the nrepl/bencode library which allows you to read and write bencode messages to a socket. A simple example which evaluates a Clojure expression on an nREPL server started with lein repl:

(ns nrepl-client
  (:require [bencode.core :as b]))

(defn nrepl-eval [port expr]
  (let [s (java.net.Socket. "localhost" port)
        out (.getOutputStream s)
        in (java.io.PushbackInputStream. (.getInputStream s))
        _ (b/write-bencode out {"op" "eval" "code" expr})
        bytes (get (b/read-bencode in) "value")]
    (String. bytes)))

(nrepl-eval 52054 "(+ 1 2 3)") ;;=> "6"

Running from Cygwin/Git Bash

On Windows, bb can be invoked from the bash shell directly:

$ bb -e '(+ 1 2 3)'
6

However, creating a script that invokes bb via a shebang leads to an error if the script is not in the current directory. Suppose you had the following script named hello on your path:

#!/usr/bin/env bb
(println "Hello, world!")
$ hello
----- Error --------------------------------------------------------------------
Type:     java.lang.Exception
Message:  File does not exist: /cygdrive/c/path/to/hello

The problem here is that the shell is passing a Cygwin-style path to bb, but bb can’t recognize it because it wasn’t compiled with Cygwin.

The solution is to create a wrapper script that converts the Cygwin-style path to a Windows-style path before invoking bb. Put the following into a script called bbwrap somewhere on your Cygwin path, say in /usr/local/bin/bbwrap:

#!/bin/bash
SCRIPT=$1
shift
bb.exe $(cygpath -w $SCRIPT) $@

Make sure to fix your original script to invoke bbwrap instead of bb directly:

#!/usr/bin/env bbwrap
(println "Hello, world!")

Differences with Clojure

Babashka is implemented using the Small Clojure Interpreter. This means that a snippet or script is not compiled to JVM bytecode, but executed form by form by a runtime which implements a substantial subset of Clojure. Babashka is compiled to a native binary using GraalVM. It comes with a selection of built-in namespaces and functions from Clojure and other useful libraries. The data types (numbers, strings, persistent collections) are the same. Multi-threading is supported (pmap, future).

Differences with Clojure:

  • A pre-selected set of Java classes are supported. You cannot add Java classes at runtime.

  • Interpretation comes with overhead. Therefore loops are slower than in Clojure on the JVM. In general interpretation yields slower programs than compiled programs.

  • No deftype, definterface and unboxed math.

  • defprotocol and defrecord are implemented using multimethods and regular maps. Ostensibly they work the same, but under the hood there are no Java classes that correspond to them.

  • Currently reify works only for one class at a time

  • The clojure.core.async/go macro is not (yet) supported. For compatibility it currently maps to clojure.core.async/thread. More info here.

Contributing

Visit Babashka book’s Github repository and make an issue and/or PR.

License

Copyright © 2020-2021 Michiel Borkent

Licensed under CC BY-SA 4.0.