Introduction

Welcome reader! This is a book about scripting with Clojure and babashka. Clojure is a functional, dynamic programming language from the Lisp family which runs on the JVM. Babashka is a scripting environment made with Clojure, compiled to native with GraalVM. The primary benefits of using babashka for scripting compared to the JVM are fast startup time and low memory consumption. Babashka comes with batteries included and packs libraries like clojure.tools.cli for parsing command line arguments and cheshire for working with JSON. Moreover, it can be installed just by downloading a self-contained binary.

Target audience

Babashka is written for developers who are familiar with Clojure on the JVM. This book assumes familiarity with Clojure and is not a Clojure tutorial. If you aren’t that familiar with Clojure but you’re curious to learn, check out this list of beginner resources.

Setting expectations

Babashka uses sci for interpreting Clojure. Sci implements a substantial subset of Clojure. Interpreting code is in general not as performant as executing compiled code. If your script takes more than a few seconds to run or has lots of loops, Clojure on the JVM may be a better fit, as the performance on JVM is going to outweigh its startup time penalty. Read more about the differences with Clojure here.

Getting started

Installation

Installing babashka is as simple as downloading the binary for your platform and placing it on your path. Pre-built binaries are provided on the releases page of babashka’s Github repo. Babashka is also available in various package managers like brew for macOS and linux and scoop for Windows. See here for details.

Building from source

If you would rather build babashka from source, download a copy of GraalVM and set the GRAALVM_HOME environment variable. Also make sure you have lein installed. Then run:

$ git clone https://github.com/borkdude/babashka --recursive
$ script/uberjar && script/compile

See the babashka build.md page for details.

Running babashka

The babashka executable is called bb. You can either provide it with a Clojure expression directly:

$ bb -e '(+ 1 2 3)'
6

or run a script:

script.clj
(+ 1 2 3)
$ bb -f script.clj
6

The -e flag is optional when the argument starts with a paren. In that case babashka will treat it automatically as an expression:

$ bb '(+ 1 2 3)'
6

Similarly, the -f flag is optional when the argument is a filename:

$ bb script.clj
6

Commonly, scripts have shebangs so you can invoke them with their filename only:

script.clj
#!/usr/bin/env bb
(+ 1 2 3)

Usage

Typing bb help from the command line will print all the available command line options which should give you a sense of the available features in babashka.

Usage: bb [classpath opts] [eval opts] [cmdline args]
or:    bb [classpath opts] file [cmdline args]
or:    bb [classpath opts] subcommand [subcommand opts] [cmdline args]

Classpath:

  -cp, --classpath     Classpath to use. Overrides bb.edn classpath.

Evaluation:

  -e, --eval <expr>    Evaluate an expression.
  -f, --file <path>    Evaluate a file.
  -m, --main <ns|var>  Call the -main function from a namespace or call a fully qualified var.
  --verbose            Print debug information and entire stacktrace in case of exception.

Help:

  help, -h or -?     Print this help text.
  version            Print the current version of babashka.
  describe           Print an EDN map with information about this version of babashka.
  doc <var|ns>       Print docstring of var or namespace. Requires namespace if necessary.

REPL:

  repl                 Start REPL. Use rlwrap for history.
  socket-repl  [addr]  Start a socket REPL. Address defaults to localhost:1666.
  nrepl-server [addr]  Start nREPL server. Address defaults to localhost:1667.

Clojure:

  clojure [args...]  Invokes clojure. Takes same args as the official clojure CLI.

Packaging:

  uberscript <file> [eval-opt]  Collect all required namespaces from the classpath into a single file. Accepts additional eval opts, like `-m`.
  uberjar    <jar>  [eval-opt]  Similar to uberscript but creates jar file.

In- and output flags (only to be used with -e one-liners):

  -i                 Bind *input* to a lazy seq of lines from stdin.
  -I                 Bind *input* to a lazy seq of EDN values from stdin.
  -o                 Write lines to stdout.
  -O                 Write EDN values to stdout.
  --stream           Stream over lines or EDN values from stdin. Combined with -i or -I *input* becomes a single value per iteration.

File names take precedence over subcommand names.
Remaining arguments are bound to *command-line-args*.
Use -- to separate script command line args from bb command line args.
When no eval opts or subcommand is provided, the implicit subcommand is repl.

Running a script

Scripts may be executed from a file using -f or --file:

bb -f download_html.clj

The file may also be passed directly, without -f:

bb download_html.clj

Using bb with a shebang also works:

#!/usr/bin/env bb

(require '[babashka.curl :as curl])

(defn get-url [url]
  (println "Downloading url:" url)
  (curl/get url))

(defn write-html [file html]
  (println "Writing file:" file)
  (spit file html))

(let [[url file] *command-line-args*]
  (when (or (empty? url) (empty? file))
    (println "Usage: <url> <file>")
    (System/exit 1))
  (write-html file (:body (get-url url))))
$ ./download_html.clj
Usage: <url> <file>

$ ./download_html.clj https://www.clojure.org /tmp/clojure.org.html
Downloading url: https://www.clojure.org
Writing file: /tmp/clojure.org.html

If /usr/bin/env doesn’t work for you, you can use the following workaround:

$ cat script.clj
#!/bin/sh

#_(
   "exec" "bb" "$0" hello "$@"
   )

(prn *command-line-args*)

./script.clj 1 2 3
("hello" "1" "2" "3")

Current file path

The var *file* contains the full path of the file that is currently being executed:

$ cat example.clj
(prn *file*)

$ bb example.clj
"/Users/borkdude/example.clj"

Parsing command line arguments

Command-line arguments can be retrieved using *command-line-args*. If you want to parse command line arguments, you may use the built-in clojure.tools.cli namespace:

Babashka ships with clojure.tools.cli:

(require '[clojure.tools.cli :refer [parse-opts]])

(def cli-options
  ;; An option with a required argument
  [["-p" "--port PORT" "Port number"
    :default 80
    :parse-fn #(Integer/parseInt %)
    :validate [#(< 0 % 0x10000) "Must be a number between 0 and 65536"]]
   ["-h" "--help"]])

(:options (parse-opts *command-line-args* cli-options))
$ bb script.clj
{:port 80}
$ bb script.clj -h
{:port 80, :help true}

There is also the nubank/docopt library that is compatible with babashka.

Project setup

Classpath

When you want to organize babashka scripts like a regular Clojure project, you will need to set a classpath.

Babashka accepts a --classpath option that will be used to search for namespaces when requiring them. Say we have a file src/my/namespace.clj:

(ns my.namespace)
(defn -main [& args]
  (apply println "Hello from my namespace!" args))

Now we can execute this main function with:

$ bb --classpath src --main my.namespace 1 2 3
Hello from my namespace! 1 2 3

If you have a larger script with a classic Clojure project layout like

$ tree -L 3
├── deps.edn
├── README
├── src
│   └── project_namespace
│       ├── main.clj
│       └── utilities.clj
└── test
    └── project_namespace
        ├── test_main.clj
        └── test_utilities.clj

then you can tell babashka to include both the src and test folders in the classpath and start a socket REPL by running:

$ bb --classpath src:test socket-repl 1666

If there is no --classpath argument, the BABASHKA_CLASSPATH environment variable will be used.

Also see the babashka.classpath namespace which allows dynamically adding to the classpath.

The namespace babashka.deps integrates tools.deps with babashka and allows you to set the classpath using a deps.edn map.

bb.edn

Since version 0.3.1, babashka supports a local bb.edn file to manage a project.

Classpath

You can declare one or multiple paths and dependencies so they are automatically added to the classpath:

{:paths ["src"]
 :deps {medley/medley {:mvn/version "1.3.0"}}}
$ bb -e "(require '[medley.core :as m]) (m/index-by :id [{:id 1} {:id 2}])"
{1 {:id 1}, 2 {:id 2}}
:min-bb-version

Since version 0.3.6, babashka supports the :min-bb-version where the minimal babashka version can be declared:

{:paths ["src"]
 :deps {medley/medley {:mvn/version "1.3.0"}}
 :min-bb-version "1.0.0"}

When using an older bb version (that supports :min-bb-version), babashka will print a warning:

---
WARNING: this project requires babashka 1.0.0 or newer, but you have: 0.3.6
---
[source]

Invoking a main function

A main function can be invoked with -m or --main like shown above. When given the argument foo.bar, the namespace foo.bar will be required and the function foo.bar/-main will be called with command line arguments as strings.

Since babashka 0.3.1 you may pass a fully qualified symbol to -m:

$ bb -m clojure.core/prn 1 2 3
"1" "2" "3"

so you can execute any function as a main function, as long as it accepts varargs arguments.

When invoking bb with a main function, the expression (System/getProperty "babashka.main") will return the name of the main function.

Preloads

The environment variable BABASHKA_PRELOADS allows to define code that will be available in all subsequent usages of babashka.

BABASHKA_PRELOADS='(defn foo [x] (+ x 2))'
BABASHKA_PRELOADS=$BABASHKA_PRELOADS' (defn bar [x] (* x 2))'
export BABASHKA_PRELOADS

Note that you can concatenate multiple expressions. Now you can use these functions in babashka:

$ bb '(-> (foo *input*) bar)' <<< 1
6

You can also preload an entire file using load-file:

export BABASHKA_PRELOADS='(load-file "my_awesome_prelude.clj")'

Note that *input* is not available in preloads.

Running a REPL

Babashka supports running a REPL, a socket REPL and an nREPL server.

REPL

To start a REPL, type:

$ bb repl

To get history with up and down arrows, use rlwrap:

$ rlwrap bb repl

Socket REPL

To start a socket REPL on port 1666:

$ bb socket-repl 1666
Babashka socket REPL started at localhost:1666

Now you can connect with your favorite socket REPL client:

$ rlwrap nc 127.0.0.1 1666
Babashka v0.0.14 REPL.
Use :repl/quit or :repl/exit to quit the REPL.
Clojure rocks, Bash reaches.

bb=> (+ 1 2 3)
6
bb=> :repl/quit
$

The --socket-repl option takes options similar to the clojure.server.repl Java property option in Clojure:

$ bb socket-repl '{:address "0.0.0.0" :accept clojure.core.server/repl :port 1666}'

Editor plugins and tools known to work with a babashka socket REPL:

  • Emacs: inf-clojure:

    To connect:

    M-x inf-clojure-connect <RET> localhost <RET> 1666

    Before evaluating from a Clojure buffer:

    M-x inf-clojure-minor-mode

  • Atom: Chlorine

  • Vim: vim-iced

  • IntelliJ IDEA: Cursive

    Note: you will have to use a workaround via tubular. For more info, look here.

pREPL

Launching a prepl can be done as follows:

$ bb socket-repl '{:address "0.0.0.0" :accept clojure.core.server/io-prepl :port 1666}'

or programmatically:

$ bb -e '(clojure.core.server/io-prepl)'
(+ 1 2 3)
{:tag :ret, :val "6", :ns "user", :ms 0, :form "(+ 1 2 3)"}

nREPL

To start an nREPL server:

$ bb nrepl-server 1667

Then connect with your favorite nREPL client:

$ lein repl :connect 1667
Connecting to nREPL at 127.0.0.1:1667
user=> (+ 1 2 3)
6
user=>

Editor plugins and tools known to work with the babashka nREPL server:

The babashka nREPL server does not write an .nrepl-port file at startup, but you can easily write a script that launches the server and writes the file:

#!/usr/bin/env bb

(import [java.net ServerSocket]
        [java.io File]
        [java.lang ProcessBuilder$Redirect])

(require '[babashka.wait :as wait])

(let [nrepl-port (with-open [sock (ServerSocket. 0)] (.getLocalPort sock))
      cp (str/join File/pathSeparatorChar ["src" "test"])
      pb (doto (ProcessBuilder. (into ["bb" "--nrepl-server" (str nrepl-port)
                                       "--classpath" cp]
                                      *command-line-args*))
           (.redirectOutput ProcessBuilder$Redirect/INHERIT))
      proc (.start pb)]
  (wait/wait-for-port "localhost" nrepl-port)
  (spit ".nrepl-port" nrepl-port)
  (.deleteOnExit (File. ".nrepl-port"))
  (.waitFor proc))
Debugging the nREPL server

To debug the nREPL server from the binary you can run:

$ BABASHKA_DEV=true bb nrepl-server 1667

This will print all the incoming messages.

To debug the nREPL server from source:

$ git clone https://github.com/borkdude/babashka --recurse-submodules
$ cd babashka
$ BABASHKA_DEV=true clojure -A:main --nrepl-server 1667

Input and output flags

In one-liners the *input* value may come in handy. It contains the input read from stdin as EDN by default. If you want to read in text, use the -i flag, which binds *input* to a lazy seq of lines of text. If you want to read multiple EDN values, use the -I flag. The -o option prints the result as lines of text. The -O option prints the result as lines of EDN values.

*input* is only available in the user namespace, designed for one-liners. For writing scripts, see Scripts.

The following table illustrates the combination of options for commands of the form

echo "{{Input}}" | bb {{Input flags}} {{Output flags}} "*input*"
Input Input flags Output flag *input* Output

{:a 1} {:a 2}

{:a 1}

{:a 1}

hello
bye

-i

("hello" "bye")

("hello" "bye")

hello
bye

-i

-o

("hello" "bye")

hello
bye

{:a 1} {:a 2}

-I

({:a 1} {:a 2})

({:a 1} {:a 2})

{:a 1} {:a 2}

-I

-O

({:a 1} {:a 2})

{:a 1} {:a 2}

When combined with the --stream option, the expression is executed for each value in the input:

$ echo '{:a 1} {:a 2}' | bb --stream '*input*'
{:a 1}
{:a 2}

Scripts

When writing scripts instead of one-liners on the command line, it is not recommended to use *input*. Here is how you can rewrite to standard Clojure code.

EDN input

Reading a single EDN value from stdin:

(ns script
 (:require [clojure.edn :as edn]))

(edn/read *in*)

Reading multiple EDN values from stdin (the -I flag):

(ns script
 (:require [clojure.edn :as edn]
           [clojure.java.io :as io]))

(let [reader  (java.io.PushbackReader. (io/reader *in*))]
  (take-while #(not (identical? ::eof %)) (repeatedly #(edn/read {:eof ::eof} reader))))

Text input

Reading text from stdin can be done with (slurp *in*). To get a lazy seq of lines (the -i flag), you can use:

(ns script
 (:require [clojure.java.io :as io]))

(line-seq (io/reader *in*))

Output

To print to stdout, use println for text and prn for EDN values.

Uberscript

The --uberscript option collects the expressions in BABASHKA_PRELOADS, the command line expression or file, the main entrypoint and all required namespaces from the classpath into a single file. This can be convenient for debugging and deployment.

Here is an example that uses a function from the clj-commons/fs library.

Let’s first set the classpath:

$ export BABASHKA_CLASSPATH=$(clojure -Spath -Sdeps '{:deps {clj-commons/fs {:mvn/version "1.6.307"}}}')

Write a little script, say glob.clj:

(ns glob (:require [me.raynes.fs :as fs]))

(run! (comp println str)
      (fs/glob (first *command-line-args*)))

For testing, we’ll make a file which we will find using the glob function:

$ touch README.md

Now we can execute the script which uses the library:

$ time bb glob.clj '*.md'
/private/tmp/glob/README.md
bb glob.clj '*.md'   0.03s  user 0.01s system 88% cpu 0.047 total

Producing an uberscript with all required code:

$ bb uberscript glob-uberscript.clj -f glob.clj

To prove that we don’t need the classpath anymore:

$ unset BABASHKA_CLASSPATH
$ time bb glob-uberscript.clj '*.md'
/private/tmp/glob/README.md
bb glob-uberscript.clj '*.md'   0.03s  user 0.02s system 93% cpu 0.049 total

Caveats:

  • Dynamic requires. Building uberscripts works by running top-level ns and require forms. The rest of the code is not evaluated. Code that relies on dynamic requires may not work in an uberscript.

  • Resources. The usage of io/resource assumes a classpath, so when this is used in your uberscript, you still have to set a classpath and bring the resources along.

If any of the above is problematic for your project, using an uberjar is a good alternative.

Carve

Uberscripts can be optimized by cutting out unused vars with carve.

$ wc -l glob-uberscript.clj
     583 glob-uberscript.clj
$ carve --opts '{:paths ["glob-uberscript.clj"] :aggressive true :silent true}'
$ wc -l glob-uberscript.clj
     105 glob-uberscript.clj

Note that the uberscript became 72% shorter. This has a beneficial effect on execution time:

$ time bb glob-uberscript.clj '*.md'
/private/tmp/glob/README.md
bb glob-uberscript.clj '*.md'   0.02s  user 0.01s system 84% cpu 0.034 total

Uberjar

Babashka can create uberjars from a given classpath and optionally a main method:

$ cat src/foo.clj
(ns foo (:gen-class)) (defn -main [& args] (prn :hello))
$ bb -cp $(clojure -Spath) uberjar foo.jar -m foo
$ bb foo.jar
:hello

When producing a classpath using the clojure or deps.clj tool, Clojure itself, spec and the core specs will be on the classpath and will therefore be included in your uberjar, which makes it bigger than necessary:

$ ls -lh foo.jar
-rw-r--r--  1 borkdude  staff   4.5M Aug 19 17:04 foo.jar

To exclude these dependencies, you can use the following :classpath-overrides in your deps.edn:

{:aliases {:remove-clojure {:classpath-overrides {org.clojure/clojure nil
                                                  org.clojure/spec.alpha nil
                                                  org.clojure/core.specs.alpha nil}}}}
$ rm foo.jar
$ bb -cp $(clojure -A:remove-clojure -Spath) uberjar foo.jar -m foo
$ bb foo.jar
:hello
$ ls -lh foo.jar
-rw-r--r--  1 borkdude  staff   871B Aug 19 17:07 foo.jar

If you want your uberjar to be compatible with the JVM, you’ll need to compile the main namespace. Babashka does not do compilation, so we use Clojure on the JVM for that part:

$ rm foo.jar
$ mkdir classes
$ clojure -e "(require 'foo) (compile 'foo)"
foo
$ bb -cp $(clojure -Spath):classes uberjar foo.jar -m foo
$ bb foo.jar
:hello
$ java -jar foo.jar
:hello

System properties

Babashka sets the following system properties:

  • babashka.version: the version string, e.g. "1.2.0"

  • babashka.main: the --main argument

  • babashka.file: the --file argument (normalized using .getAbsolutePath)

Data readers

Data readers can be enabled by setting *data-readers* to a hashmap of symbols to functions or vars:

$ bb -e "(set! *data-readers* {'t/tag inc}) #t/tag 1"
2

To preserve good startup time, babashka does not scan the classpath for data_readers.clj files.

Reader conditionals

Babashka supports reader conditionals by taking either the :bb or :clj branch, whichever comes first. NOTE: the :clj branch behavior was added in version 0.0.71, before that version the :clj branch was ignored.

$ bb -e "#?(:bb :hello :clj :bye)"
:hello

$ bb -e "#?(:clj :bye :bb :hello)"
:bye

$ bb -e "[1 2 #?@(:bb [] :clj [1])]"
[1 2]

Invoking clojure

Babashka bundles deps.clj for invoking a clojure JVM process:

$ bb clojure -M -e "*clojure-version*"
{:major 1, :minor 10, :incremental 1, :qualifier nil}

See the clojure function in the babashka.deps namespace for programmatically invoking clojure.

Style

A note on style. Babashka recommends the following:

Explicit requires

Use explicit requires with namespace aliases in scripts, unless you’re writing one-liners.

Do this:

$ ls | bb -i '(-> *input* first (str/includes? "m"))'
true

But not this:

script.clj:

(-> *input* first (str/includes? "m"))

Rather do this:

script.clj:

(ns script
  (:require [clojure.java.io :as io]
            [clojure.string :as str]))
(-> (io/reader *in*) line-seq first (str/includes? "m"))

Some reasons for this:

  • Linters like clj-kondo work better with code that uses namespace forms, explicit requires, and known Clojure constructs

  • Editor tooling works better with namespace forms (sorting requires, etc).

  • Writing compatible code gives you the option to run the same script with clojure

Libraries

Built-in namespaces

In addition to clojure.core, the following namespaces are available in babashka. Some are available through pre-defined aliases in the user namespace, which can be handy for one-liners. If not all vars are available, they are enumerated explicitly. If some important var is missing, an issue or PR is welcome.

From Clojure:

  • clojure.core

  • clojure.core.protocols: Datafiable, Navigable

  • clojure.data

  • clojure.datafy

  • clojure.edn aliased as edn

  • clojure.java.browse

  • clojure.java.io aliased as io:

    • as-relative-path, as-url, copy, delete-file, file, input-stream, make-parents, output-stream, reader, resource, writer

  • clojure.java.shell aliased as shell

  • clojure.main: demunge, repl, repl-requires

  • clojure.pprint: pprint, cl-format

  • clojure.set aliased as set

  • clojure.string aliased as str

  • clojure.stacktrace

  • clojure.test

  • clojure.zip

Additional libraries:

See the projects page for libraries that are not built-in, but which you can load from source via the --classpath option.

See the build page for built-in libraries that can be enabled via feature flags, if you want to compile babashka yourself.

A selection of Java classes are available, see babashka/impl/classes.clj in babashka’s git repo.

Babashka namespaces

babashka.classpath

Available functions:

  • add-classpath

  • get-classpath

  • split-classpath

add-classpath

The function add-classpath which can be used to add to the classpath dynamically:

(require '[babashka.classpath :refer [add-classpath]]
         '[clojure.java.shell :refer [sh]]
         '[clojure.string :as str])

(def medley-dep '{:deps {medley {:git/url "https://github.com/borkdude/medley"
                                 :sha "91adfb5da33f8d23f75f0894da1defe567a625c0"}}})
(def cp (-> (sh "clojure" "-Spath" "-Sdeps" (str medley-dep)) :out str/trim))
(add-classpath cp)
(require '[medley.core :as m])
(m/index-by :id [{:id 1} {:id 2}]) ;;=> {1 {:id 1}, 2 {:id 2}}
get-classpath

The function get-classpath returns the classpath as set by --classpath, BABASHKA_CLASSPATH and add-classpath.

split-classpath

Given a classpath, returns a seq of strings as the result of splitting the classpath by the platform specific path separatator.

babashka.deps

Available functions:

  • add-deps

  • clojure

  • merge-deps

add-deps

The function add-deps takes a deps edn map like {:deps {medley/medley {:mvn/version "1.3.0"}}}, resolves it using deps.clj and then adds to the babashka classpath accordingly.

Example:

(require '[babashka.deps :as deps])

(deps/add-deps '{:deps {medley/medley {:mvn/version "1.3.0"}}})

(require '[medley.core :as m])
(m/index-by :id [{:id 1} {:id 2}])

Optionally, add-deps takes a second arg with options. Currently the only option is :aliases which will affect how deps are resolved:

Example:

(deps/add-deps '{:aliases {:medley {:extra-deps {medley/medley {:mvn/version "1.3.0"}}}}}
               {:aliases [:medley]})
clojure

The function clojure takes a sequential collection of arguments, similar to the clojure CLI. The arguments are then passed to deps.clj. The clojure function returns nil and prints to *out* for commands like -Stree, and -Spath. For -M, -X and -A it invokes java with babashka.process/process (see babashka.process) and returns the associated record. For more details, read the docstring with:

(require '[clojure.repl :refer [doc]])
(doc babashka.deps/clojure)

Example:

The following script passes through command line arguments to clojure, while adding the medley dependency:

(require '[babashka.deps :as deps])

(def deps '{:deps {medley/medley {:mvn/version "1.3.0"}}})
(def clojure-args (list* "-Sdeps" deps  *command-line-args*))

(if-let [proc (deps/clojure clojure-args)]
  (-> @proc :exit (System/exit))
  (System/exit 0))

babashka.wait

Contains the functions: wait-for-port and wait-for-path.

Usage of wait-for-port:

(wait/wait-for-port "localhost" 8080)
(wait/wait-for-port "localhost" 8080 {:timeout 1000 :pause 1000})

Waits for TCP connection to be available on host and port. Options map supports :timeout and :pause. If :timeout is provided and reached, :default's value (if any) is returned. The :pause option determines the time waited between retries.

Usage of wait-for-path:

(wait/wait-for-path "/tmp/wait-path-test")
(wait/wait-for-path "/tmp/wait-path-test" {:timeout 1000 :pause 1000})

Waits for file path to be available. Options map supports :default, :timeout and :pause. If :timeout is provided and reached, :default's value (if any) is returned. The :pause option determines the time waited between retries.

The namespace babashka.wait is aliased as wait in the user namespace.

babashka.signal

Contains the function signal/pipe-signal-received?. Usage:

(signal/pipe-signal-received?)

Returns true if PIPE signal was received. Example:

$ bb -e '((fn [x] (println x) (when (not (signal/pipe-signal-received?)) (recur (inc x)))) 0)' | head -n2
1
2

The namespace babashka.signal is aliased as signal in the user namespace.

babashka.curl

The namespace babashka.curl is a tiny wrapper around curl. It’s aliased as curl in the user namespace. See babashka.curl for how to use it.

babashka.process

The babashka.process library. See the process repo for API docs.

Projects

Babashka is able to run Clojure projects from source, if they are compatible with the subset of Clojure that sci is capable of running.

Check this page for projects that are known to work with babashka.

babashka.fs

The babashka.fs library offers file system utilities. See the fs repo for API docs.

Pods

Pods are programs that can be used as a Clojure library by babashka. Documentation is available in the library repo.

A list of available pods can be found here.

Pod registry

Since bb 0.2.6 pods can be obtained via the pod-registry.

This is an example script which uses bootleg as a pod to convert hiccup to HTML.

#!/usr/bin/env bb

(require '[babashka.pods :as pods])

(pods/load-pod 'retrogradeorbit/bootleg "0.1.9")

(require '[pod.retrogradeorbit.bootleg.utils :as utils])

(-> [:div
     [:h1 "Using Bootleg From Babashka"]
     [:p "This is a demo"]]
    (utils/convert-to :html))

Child processes

by Michiel Borkent

There are several ways of creating child processes in babashka. Let’s start with the easiest one.

clojure.java.shell

A common way to shell out to another process is via clojure.java.shell:

user=> (require '[clojure.java.shell :refer [sh]])
nil
user=> (sh "ls")
{:exit 0, :out "README.md\ndist\ngh-pages\nscript\nsrc\n", :err ""}

As you can see the result of :out are the lines of text produced by ls. The :exit code was 0 and there was no output on stderr.

user=> (sh "ls" "foo")
{:exit 1, :out "", :err "ls: foo: No such file or directory\n"}

If we invoke ls with a non-existing file, there is error output, but not output on stdout and the :exit code is 1.

For more information on clojure.java.shell, read the API documentation

babashka.process

Creating a child process with clojure.java.shell/sh is convenient and sufficient in 90% of the scripts you’re going to write with babashka. For the remaining 10% you’re going to need something slightly more powerful. This is where the babashka.process library namespace comes in. The README and source code of this library is located here.

What if we wanted to start a Clojure pREPL in our script, send commands to it and read the evaluated output in a loop? Using clojure.java.shell/sh for this has the limitation that we can only send input and read output once per process. Using babashka.process which leverages the java.lang.ProcessBuilder and java.lang.Process classes gives us the extra power we need to overcome this limitation.

First we show how to use process to start a process that inherits stdout, stderr and stdin from the parent process. This is often useful when you want to hand over control to another process at the end of a script. The following code starts a pREPL process:

process/prepl_1.clj
#!/usr/bin/env bb

(require '[babashka.process :as p])

(defn io-prepl []
  (let [cmd ["clojure" "-M"
             "-e" "(require '[clojure.core.server :as s])"
             "-e" "(s/io-prepl)"]                           (1)
        proc (p/process cmd                                 (2)
                        {:inherit true                      (3)
                         :shutdown p/destroy-tree})]        (4)
    proc))

@(io-prepl)                                                 (5)
1 The command and its arguments
2 Creating the process
3 This option redirects stdout and stdin from the parent process to the child process.
4 This line will destroy the child process and all of it’s children processes before the parent process exits.
5 Dereferencing the process waits until it ends. If we would not include this, our script would end immediately.

When calling the script, we can interact with the pREPL

$ bb src/process/prepl_1.clj
(+ 1 2 3)
{:tag :ret, :val "6", :ns "user", :ms 5, :form "(+ 1 2 3)"}

Pressing ctrl-C or typing :repl/quit will make the script terminate.

What if we want to parse the output of the prepl and reformat it to our own likings? Instead of sending the output of the child process to stdout we will have to get a hold of it. We can grab the output by not using :inherit and getting the :out stream of the process. Similarly we will grab the :in stream of the process and handle reading ourselves.

process/prepl_2.clj
#!/usr/bin/env bb

(require '[babashka.process :as p])

(defn io-prepl []
  (let [cmd ["clojure" "-M"
             "-e" "(require '[clojure.core.server :as s])"
             "-e" "(s/io-prepl)"]
        proc (p/process cmd
                        {:shutdown p/destroy-tree})]
    proc))

(def prepl-process (io-prepl))

(require '[clojure.java.io :as io])

(def input-writer (io/writer (:in prepl-process)))        (1)
(def output-reader (java.io.PushbackReader.
                    (io/reader (:out prepl-process))))    (2)

(require '[clojure.edn :as edn])

(loop []
  (println "Type an expression to evaluate:")
  (when-let [v (read-line)]                               (3)
    (binding [*out* input-writer]                         (4)
      (println v))
    (let [next-val (edn/read {:eof ::EOF} output-reader)] (5)
      (when-not (identical? ::EOF next-val)
        (println (:form next-val)                         (6)
                 "evaluates to"
                 (:val next-val))
        (recur)))))
1 A writer that we will use to send input to the pREPL
2 A reader that reads from the pREPL output. pREPL returns EDN and clojure.edn/read needs a PushbackReader.
3 We read user input via clojure.core/read-line
4 Here we binding *out* to the input writer so we can use println to write to it.
5 Read the next EDN value.
6 Print the output using our custom formatting.

Here is an example run:

$ bb src/process/prepl_2.clj
Type an expression to evaluate:
(+ 1 2 3)
(+ 1 2 3) evaluates to 6
:repl/quit

Real world examples

Prior to babashka.process people were using raw ProcessBuilder interop. Here are some real world examples. These might get updated to babashka.process in the future.

  • The babashka repo uses ProcessBuilder to invoke git to make a commit after bumping the version. It redirects the output from git directly to stdout of the parent process.

  • babashka.curl uses ProcessBuilder to interact with the curl executable

  • deps.clj uses ProcessBuilder to interact with java, e.g. to start a Clojure REPL or compute a classpath using tools.deps

Recipes

Running tests

Babashka bundles clojure.test. To run tests you can write a test runner script. Given the following project structure:

.
├── src
│   └──...
└── test
    └── your
        ├── test_a.clj
        └── test_b.clj
test-runner.clj
#!/usr/bin/env bb

(require '[clojure.test :as t]
         '[babashka.classpath :as cp])

(cp/add-classpath "src:test")                        (1)

(require 'your.test-a 'your.test-b)                  (2)

(def test-results
  (t/run-tests 'your.test-a 'your.test-b))           (3)

(def failures-and-errors
  (let [{:keys [:fail :error]} test-results]
    (+ fail error)))                                 (4)

(System/exit failures-and-errors)                    (5)
1 Add sources and tests to the classpath
2 Require the test namespaces
3 Run all tests in the test namespaces
4 Extract failures and errors
5 Exit the test script with a non-zero exit code when there are failures or errors

Main file

In Python scripts there is a well-known pattern to check if the current file was the file invoked from the command line, or loaded from another file: the __name__ == "__main__" pattern. In babashka this pattern can be implemented with:

(= *file* (System/getProperty "babashka.file"))

Shutdown hook

Adding a shutdown hook allows you to execute some code before the script exits.

$ bb -e '(-> (Runtime/getRuntime) (.addShutdownHook (Thread. #(println "bye"))))'
bye

This also works when the script is interrupted with ctrl-c.

Printing returned values

Babashka doesn’t print a returned nil as lots of scripts end in something side-effecting.

$ bb -e '(:a {:a 5})'
5
$ bb -e '(:b {:a 5})'
$

If you really want to print the nil, you can use (prn ..) instead.

HTTP requests

For making HTTP requests you can use:

  • babashka.curl. This library is included with babashka and aliased as curl in the user namespace. The interface is similar to that of clj-http but it will shell out to curl to make requests.

  • org.httpkit.client

  • slurp for simple GET requests

  • clj-http-lite as a library.

  • clojure.java.shell or babashka.process for shelling out to your favorite command line http client

Choosing the right client

If memory usage is a concern and you are downloading big files, choose babashka.curl with :as :stream over org.httpkit.client since http-kit holds the entire response in memory at once. Let’s download a 200mb file with 10mb heap size:

$ bb -Xmx10m -e '(io/copy (:body (curl/get "http://ipv4.download.thinkbroadband.com/200MB.zip" {:as :stream})) (io/file "/tmp/200mb.zip"))'

With babashka.curl this works fine. However with org.httpkit.client that won’t work. Not even 190mb of heap will do:

$ bb -Xmx190m -e '(io/copy (:body @(org.httpkit.client/get "http://ipv4.download.thinkbroadband.com/200MB.zip" {:as :stream})) (io/file "/tmp/200mb.zip"))'
Sun Nov 08 23:01:46 CET 2020 [client-loop] ERROR - select exception, should not happen
java.lang.OutOfMemoryError: Array allocation too large.

If your script creates many requests with relatively small payloads, choose org.httpkit.client over babashka.curl since babashka.curl creates a curl process for each request.

In the future babashka (1.0.0?) may come with an HTTP client based on the JVM 11 java.net.http package that ticks all the boxes (async, HTTP/2, websockets, multi-part file uploads, sane memory usage) and is a suitable replacement for all of the above options. If you know about a GraalVM-friendly feature-complete well-maintained library, please reach out!

HTTP over Unix sockets

This can be useful for talking to Docker:

(require '[clojure.java.shell :refer [sh]])
(require '[cheshire.core :as json])
(-> (sh "curl" "--silent"
        "--no-buffer" "--unix-socket"
        "/var/run/docker.sock"
        "http://localhost/images/json")
    :out
    (json/parse-string true)
    first
    :RepoTags) ;;=> ["borkdude/babashka:latest"]

Core.async

In addition to future, pmap, promise and friends, you may use the clojure.core.async namespace for asynchronous scripting. The following example shows how to get first available value from two different processes:

bb -e '
(defn async-command [& args]
  (async/thread (apply shell/sh "bash" "-c" args)))

(-> (async/alts!! [(async-command "sleep 2 && echo process 1")
                   (async-command "sleep 1 && echo process 2")])
    first :out str/trim println)'
process 2

Caveat: currently the go macro is available for compatibility with JVM programs, but the implementation maps to clojure.core.async/thread and the single exclamation mark operations (<!, >!, etc.) map to the double exclamation mark operations (<!!, >!!, etc.). It will not "park" threads, like on the JVM.

Examples like the following may still work, but will take a lot more system resources than on the JVM and will break down for some high value of n:

(require '[clojure.core.async :as async])

(def n 1000)

(let [cs (repeatedly n async/chan)
      begin (System/currentTimeMillis)]
  (doseq [c cs] (async/go (async/>! c "hi")))
  (dotimes [_ n]
    (let [[v _] (async/alts!! cs)]
      (assert (= "hi" v))))
  (println "Read" n "msgs in" (- (System/currentTimeMillis) begin) "ms"))

Interacting with an nREPL server

Babashka comes with the nrepl/bencode library which allows you to read and write bencode messages to a socket. A simple example which evaluates a Clojure expression on an nREPL server started with lein repl:

(ns nrepl-client
  (:require [bencode.core :as b]))

(defn nrepl-eval [port expr]
  (let [s (java.net.Socket. "localhost" port)
        out (.getOutputStream s)
        in (java.io.PushbackInputStream. (.getInputStream s))
        _ (b/write-bencode out {"op" "eval" "code" expr})
        bytes (get (b/read-bencode in) "value")]
    (String. bytes)))

(nrepl-eval 52054 "(+ 1 2 3)") ;;=> "6"

Running from Cygwin/Git Bash

On Windows, bb can be invoked from the bash shell directly:

$ bb -e '(+ 1 2 3)'
6

However, creating a script that invokes bb via a shebang leads to an error if the script is not in the current directory. Suppose you had the following script named hello on your path:

#!/usr/bin/env bb
(println "Hello, world!")
$ hello
----- Error --------------------------------------------------------------------
Type:     java.lang.Exception
Message:  File does not exist: /cygdrive/c/path/to/hello

The problem here is that the shell is passing a Cygwin-style path to bb, but bb can’t recognize it because it wasn’t compiled with Cygwin.

The solution is to create a wrapper script that converts the Cygwin-style path to a Windows-style path before invoking bb. Put the following into a script called bbwrap somewhere on your Cygwin path, say in /usr/local/bin/bbwrap:

#!/bin/bash
SCRIPT=$1
shift
bb.exe $(cygpath -w $SCRIPT) $@

Make sure to fix your original script to invoke bbwrap instead of bb directly:

#!/usr/bin/env bbwrap
(println "Hello, world!")

Differences with Clojure

Babashka is implemented using the Small Clojure Interpreter. This means that a snippet or script is not compiled to JVM bytecode, but executed form by form by a runtime which implements a substantial subset of Clojure. Babashka is compiled to a native binary using GraalVM. It comes with a selection of built-in namespaces and functions from Clojure and other useful libraries. The data types (numbers, strings, persistent collections) are the same. Multi-threading is supported (pmap, future).

Differences with Clojure:

  • A pre-selected set of Java classes are supported. You cannot add Java classes at runtime.

  • Interpretation comes with overhead. Therefore loops are slower than in Clojure on the JVM. In general interpretation yields slower programs than compiled programs.

  • No deftype, definterface and unboxed math.

  • defprotocol and defrecord are implemented using multimethods and regular maps. Ostensibly they work the same, but under the hood there are no Java classes that correspond to them.

  • Currently reify works only for one class at a time

  • The clojure.core.async/go macro is not (yet) supported. For compatibility it currently maps to clojure.core.async/thread. More info here.

Contributing

Visit Babashka book’s Github repository and read CONTRIBUTING.md on how to contribute.

License

Copyright © 2020-2021 Michiel Borkent

Creative Commons License

This book is licensed under a Creative Commons License.

Please note that because this is a No Derivatives license, you may not use this repository as a basis for creating your own book based on this one.