Clearly Biff is designed around deploying specifically to Digital Ocean on Ubuntu. However, the product I’m working on will need to run standalone/on-prem most of the time as it is a tool for small physician practices. Meanwhile our devops folks would like to create a staging/testing endpoint on AWS. So, any thoughts about how configuration and running the application might need to change to accommodate building and deploying a project as an uberjar (including assumptions that might actually break in this case) would be appreciated.
Sincerely,
Bob
Yep, DigitalOcean is the default at least. That’s certainly not a requirement, though. The past few weeks I’ve actually been working on an example repo for deploying Biff via an uberjar + Docker. See the diff–mainly there’s the Dockerfile, a build.clj file, and a couple changes to the application source to accommodate being built as a jar. I’ll probably clean up some of those changes and then merge them directly into Biff, but you can go ahead and use them any time. I’ve successfully deployed that to a couple different places (Fly.io and DigitalOcean App Platform) so the code should be reliable.
No need to actually deploy it with Docker yourself; you can just take the Uberjar bits and deploy on a regular VM if you prefer. For container-based deployments, you’d ideally have XTDB checkpointing configured, which I haven’t done yet. Otherwise the indexes will have to be built from scratch whenever a new container is started (may or may not be an actual problem for your situation).
If you aren’t deploying with Docker, you can also take a look at server-setup.sh. For non-Ubuntu distros, you can adapt that script (change the apt-get calls etc). I don’t have much experience with AWS, but you should be able to adapt that script to provision an EC2 instance, or just use whatever service AWS provides for running Docker images.
Lastly, for configuration–in config.edn, in addition to the :prod and :dev sections, you’ll likely need to add a :staging section where you can set different values for :biff/base-url and whatever other config options need to be different in the staging environment. And then however you deploy, you’ll need to make sure environment variables are set.
Hope that helps! Let me know if I can clarify anything further/if you run into any issues.
[edit: one small thing that does need to be changed in that example code is this part–the timestamps currently get formatted in a human-readable way instead of e.g. milliseconds-since-the-epoch]
Regarding the uberjar bits, I used your example project as the basis and when I run clj -T:build uberjar I get the following error:
Execution error (FileNotFoundException) at build/eval214$loading (build.clj:1).
Could not locate com/biffweb__init.class, com/biffweb.clj or com/biffweb.cljc on classpath.
This is reproducible on multiple machines. It seems so basic I can’t get my head around what might be wrong. All I modified in the build.clj file was the project namespace. It seems unable to find the biffweb components, but how is that possible?
Hmm… what does your deps.edn look like, or at least the :build alias portion of it? Is it exactly the same as in the example repo, i.e.:
:build {:extra-deps {io.github.clojure/tools.build {:git/sha "24f2894"
:git/tag "v0.9.5"
:git/url "https://github.com/clojure/tools.build.git"}}
:extra-paths ["build"]
:ns-default build
In particular–are you using :extra-deps as in the code snippet, or are you using :deps? (Also :extra-paths instead of :paths). If the latter, that would explain it. If I change :extra-deps to :deps, then I get the same error you posted.
In most build.clj examples, :deps is used. Biff is idiosyncratic because of its static files feature–the only way to include those static html files in the uberjar is to load the application code and call generate-assets!. If you aren’t using the :static plugin key anywhere, then another option would be to use :deps instead of :extra-deps, and edit build.clj so it doesn’t require com.biffweb. Remove the call to generate-assets!, and move the implementation of com.biffweb/sh into build.clj:
(ns build
(:require [clojure.java.shell :as shell]
...
(defn sh [& args]
(let [result (apply shell/sh args)]
(if (= 0 (:exit result))
(:out result)
(throw (ex-info (:err result) result)))))
I have exactly what you have (because I cut and paste it from your example!):
{:build {:extra-deps {io.github.clojure/tools.build {:git/sha "24f2894"
:git/tag "v0.9.5"
:git/url "https://github.com/clojure/tools.build.git"}}
:extra-paths ["build"]
:ns-default build}
Replacing the biff/sh with your recommendation now it complains about my application namespace - it can’t find that, either, and suggests maybe I’m not using underscores instead of dashes, but everything is kosher there, too. Here’s my build.clj code:
(ns build
(:require [clojure.tools.build.api :as b]
;;[com.biffweb :as biff]
[clojure.java.shell :as shell]
[com.my-practice-profile :as main]))
(defn sh [& args]
(let [result (apply shell/sh args)]
(if (= 0 (:exit result))
(:out result)
(throw (ex-info (:err result) result)))))
(def main-ns 'com.my-practice-profile)
(def class-dir "target/jar/classes")
(def basis (b/create-basis {:project "deps.edn"}))
(def uber-file "target/jar/my-practice-profile.jar")
(defn uberjar [_]
(b/delete {:path "target"})
(println (sh "bb" "css" "--minify"))
(main/generate-assets! {})
(b/copy-dir {:src-dirs ["src" "resources" "target/resources"]
:target-dir class-dir})
(println "Compiling...")
(b/compile-clj {:basis basis
:ns-compile [main-ns]
:class-dir class-dir})
(println "Building uberjar...")
(b/uber {:class-dir class-dir
:uber-file uber-file
:basis basis
:main main-ns}))
So I’m kinda puzzled at this point why it can’t find stuff in src. Here’s my whole deps.edn:
{:paths ["src" "resources" "target/resources"]
:deps {com.biffweb/biff {:git/url "https://github.com/jacobobryant/biff",
:sha "92d78a1",
:tag "v0.7.18"}
camel-snake-kebab/camel-snake-kebab {:mvn/version "0.4.3"}
metosin/muuntaja {:mvn/version "0.6.8"}
ring/ring-defaults {:mvn/version "0.3.4"}
org.clojure/clojure {:mvn/version "1.11.1"}
org.slf4j/slf4j-simple {:mvn/version "2.0.0-alpha5"}
;; auth and cryptography
buddy/buddy-core {:mvn/version "1.11.423"}
buddy/buddy-auth {:mvn/version "3.0.323"}
buddy/buddy-hashers {:mvn/version "2.0.167"}
buddy/buddy-sign {:mvn/version "3.5.351"}
;; ... more app-specific deps, elided ...
}
:aliases {:build {:extra-deps {io.github.clojure/tools.build {:git/sha "24f2894"
:git/tag "v0.9.5"
:git/url "https://github.com/clojure/tools.build.git"}}
:extra-paths ["build"]
:ns-default build}
:test {:extra-deps {lambdaisland/kaocha {:mvn/version "1.87.1366"}}
:main-opts ["-m" "kaocha.runner"]}}}
And the exact error message after the change you suggested:
PS C:\projects\my-practice-profile\app> clj -T:build uberjar
Execution error (FileNotFoundException) at build/eval214$loading (build.clj:1).
Could not locate com/my_practice_profile__init.class, com/my_practice_profile.clj or com/my_practice_profile.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name.
Full report at:
C:\Users\Bob\AppData\Local\Temp\clojure-9745053039766697039.edn
PS C:\projects\my-practice-profile\app>
Any suggestions appreciated.
Oh, I think it’s the alias: instead of clj -T:build uberjar you need to run clj -X:build uberjar (see the Docker file). Another difference from typical tools.build examples, for the same reasons as :deps vs. :extra-deps.
Oh wow. LOL. Terminal autocomplete fail. Thanks for catching that!
Yes that seems to be going better.
Can’t believe I didn’t see that. LOL.
haha glad it’s working now!
So just as a sanity test I tried running
java -jar target\jar\my-practice-profile.jar
And got the following error:
PS C:\projects\my-practice-profile\app> java -jar .\target\jar\my-practice-profile.jar
Error: Could not find or load main class com.my_practice_profile
Caused by: java.lang.ClassNotFoundException: com.my_practice_profile
But it seems the build.clj file establishes the main namespace where the -main function is defined. I hope this is also a stupid typo on my part. lol. If you see it, let me know. 
hm, haven’t seen that one yet. perhaps try listing the contents of the jar as another sanity check? java -tf target\jar\my-practice-profile.jar I believe.
did you add (:gen-class) to your app’s main namespace, in the ns form? (see the example repo)
Yeah I already figured this one out - just didn’t get a chance to post an update.
That’s what it was - missing (:gen-class). On to the next problem!
PS C:\projects\my-practice-profile\app> java -jar .\target\jar\my-practice-profile.jar
[main] INFO com.my-practice-profile - starting: com.biffweb$use_config@579cfcc4
[main] INFO com.my-practice-profile - starting: com.biffweb$use_secrets@22ea09b7
Secrets are missing. Run `bb generate-secrets` and edit secrets.env.
Odd because the directory in which I’m invoking the jar file contains the secrets.env jar I’ve been using in dev. But this raising the important question of how I would package all this up for the user on installation. So any insights about this aspect of the system startup (or how I might elide it - for example, automatically creating a basic secrets.env on startup, or otherwise make sure I set up the environment variables needed on installation) also appreciated.
Ah yes–when running the jar, it’s up to you to ensure environment variables are set. so for dev you can do source secrets.env; java -jar .... you could also do the same in prod, or some other method if you have another way to manage environment variables.
OK so that allows it to run (from WSL); however, now I get an error rendering a page:
HTTP ERROR 500 Not a file: jar:file:/mnt/c/projects/my-practice-profile/app/target/jar/my-practice-profile.jar!/public/css/main.css
URI: /
STATUS: 500
MESSAGE: Not a file: jar:file:/mnt/c/projects/my-practice-profile/app/target/jar/my-practice-profile.jar!/public/css/main.css
SERVLET: org.eclipse.jetty.servlet.ServletHandler$Default404Servlet-47543549
Powered by Jetty:// 10.0.7
But the build.clj include “target/resources” which does include the main.css in public/css after compiling. So I am not at all sure why that should be failing.
Have you made the changes from this file? Comparing 6f2b443..3a4b476 · jacobobryant/biff-docker2 · GitHub
Without those changes, the io/file call fails in a jar context.
And now a NEW problem! I am able to reproduce this on multiple machines, running Windoze and Linux (via WSL). Any call to any endpoint renders fine when running in normal dev mode, but when running the jar file:
apex@BobsLenovo:/mnt/c/projects/my-practice-profile/app$ source secrets.env; java -jar target/jar/my-practice-profile.jar
: command not found
[main] INFO com.my-practice-profile - starting: com.biffweb$use_config@15096b0e
[main] INFO com.my-practice-profile - starting: com.biffweb$use_secrets@3dedc8b8
[main] INFO com.my-practice-profile - starting: com.biffweb$use_xt@79e33802
[main] WARN xtdb.rocksdb - MALLOC_ARENA_MAX not set, memory usage might be high, recommended setting for XTDB is 2
[main] INFO xtdb.tx - Started tx-ingester
[main] INFO com.biffweb.impl.xtdb - Indexed {:xtdb.api/tx-time #inst "2023-11-28T13:41:50.594-00:00", :xtdb.api/tx-id 0}
[main] INFO com.my-practice-profile - starting: com.biffweb$use_queues@78f4be94
[main] INFO com.my-practice-profile - starting: com.biffweb$use_tx_listener@3d07c7aa
[main] INFO com.my-practice-profile - starting: com.biffweb$use_jetty@5ec30886
[main] INFO org.eclipse.jetty.server.Server - jetty-10.0.7; built: 2021-10-06T19:34:02.766Z; git: da8a4553af9dd84080931fa0f8c678cd2d60f3d9; jvm 21
[main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@187f107{/,null,AVAILABLE}
[main] INFO org.eclipse.jetty.server.AbstractConnector - Started ServerConnector@451431d2{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
[main] INFO org.eclipse.jetty.server.Server - Started Server@6728dc77{STARTING}[10.0.7,sto=0] @9236ms
[main] INFO com.biffweb.impl.misc - Jetty running on http://0.0.0.0:8080
[main] INFO com.my-practice-profile - starting: com.biffweb$use_chime@2c08b402
[main] INFO com.my-practice-profile - starting: com.biffweb$use_beholder@913a17a
[main] INFO com.my-practice-profile - System started.
[main] INFO com.my-practice-profile - Go to https://example.com
nREPL server started on port 45093 on host localhost - nrepl://localhost:45093
[qtp513545013-72] ERROR com.biffweb.impl.middleware - Exception while handling request
java.lang.IllegalArgumentException: Input byte array has incorrect ending byte at 24
at java.base/java.util.Base64$Decoder.decode0(Base64.java:880)
at java.base/java.util.Base64$Decoder.decode(Base64.java:570)
at java.base/java.util.Base64$Decoder.decode(Base64.java:593)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:167)
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:102)
at com.biffweb.impl.util$base64_decode.invokeStatic(util.clj:58)
at com.biffweb.impl.util$base64_decode.invoke(util.clj:57)
at com.biffweb.impl.middleware$wrap_session$fn__21392.invoke(middleware.clj:132)
at muuntaja.middleware$wrap_params$fn__20591.invoke(middleware.clj:52)
at muuntaja.middleware$wrap_format$fn__20595.invoke(middleware.clj:73)
at ring.middleware.keyword_params$wrap_keyword_params$fn__20959.invoke(keyword_params.clj:53)
at ring.middleware.nested_params$wrap_nested_params$fn__21009.invoke(nested_params.clj:89)
at ring.middleware.multipart_params$wrap_multipart_params$fn__21115.invoke(multipart_params.clj:171)
at ring.middleware.params$wrap_params$fn__21134.invoke(params.clj:75)
at ring.middleware.cookies$wrap_cookies$fn__20830.invoke(cookies.clj:214)
at ring.middleware.absolute_redirects$wrap_absolute_redirects$fn__21240.invoke(absolute_redirects.clj:47)
at ring.middleware.content_type$wrap_content_type$fn__20620.invoke(content_type.clj:34)
at ring.middleware.default_charset$wrap_default_charset$fn__21220.invoke(default_charset.clj:31)
at ring.middleware.not_modified$wrap_not_modified$fn__21204.invoke(not_modified.clj:61)
at ring.middleware.x_headers$wrap_x_header$fn__20637.invoke(x_headers.clj:22)
at ring.middleware.x_headers$wrap_x_header$fn__20637.invoke(x_headers.clj:22)
at reitit.ring$ring_handler$fn__18077.invoke(ring.cljc:328)
at clojure.lang.AFn.applyToHelper(AFn.java:154)
at clojure.lang.AFn.applyTo(AFn.java:144)
at clojure.lang.AFunction$1.doInvoke(AFunction.java:31)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at com.biffweb.impl.middleware$wrap_https_scheme$fn__21386.invoke(middleware.clj:111)
at com.biffweb.impl.middleware$wrap_resource$fn__21352.invoke(middleware.clj:62)
at com.biffweb.impl.middleware$wrap_internal_error$fn__21369.invoke(middleware.clj:80)
at ring.middleware.ssl$wrap_hsts$fn__21269.invoke(ssl.clj:105)
at com.biffweb.impl.middleware$wrap_ssl$fn__21397.invoke(middleware.clj:156)
at com.biffweb.impl.middleware$wrap_log_requests$fn__21380.invoke(middleware.clj:97)
at clojure.lang.Var.invoke(Var.java:384)
at com.biffweb.impl.misc$use_jetty$fn__18612.invoke(misc.clj:76)
at ring.adapter.jetty9$proxy_handler$fn__18522.invoke(jetty9.clj:75)
at ring.adapter.jetty9.proxy$org.eclipse.jetty.servlet.ServletHandler$ff19274a.doHandle(Unknown Source)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1378)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:463)
at ring.adapter.jetty9.proxy$org.eclipse.jetty.servlet.ServletHandler$ff19274a.doScope(Unknown Source)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1300)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
at org.eclipse.jetty.server.Server.handle(Server.java:562)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$0(HttpChannel.java:418)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:675)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:410)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:282)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:319)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)
at org.eclipse.jetty.io.SocketChannelEndPoint$1.run(SocketChannelEndPoint.java:101)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:894)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1038)
at java.base/java.lang.Thread.run(Thread.java:1583)
[qtp513545013-72] INFO com.biffweb.impl.middleware - 44ms 500 get /
[qtp513545013-67] INFO com.biffweb.impl.middleware - 6ms 200 get /js/main.js?t=Mon%20Dec%2018%2011:11:00%20EST%202023
[qtp513545013-66] INFO com.biffweb.impl.middleware - 6ms 200 get /css/main.css?t=Mon%20Dec%2018%2011:11:00%20EST%202023
[qtp513545013-73] INFO com.biffweb.impl.middleware - 2ms 200 get /img/glider.png
It looks like something getting borked in the base64 encoding process. Any idea what might be causing the “java.lang.IllegalArgumentException: Input byte array has incorrect ending byte at 24” error?
I think I’ve actually seen this one before! I believe it may have something to do with Windows line endings being added to the secrets. After running the jar, try evaluating (System/getenv "COOKIE_SECRET") from the repl. Does it have any whitespace on the end? Do yo uget the same error if you evaluate (biff/base64-decode (System/getenv "COOKIE_SECRET"))? If so, is the error fixed by (biff/base64-decode (str/trim (System/getenv "COOKIE_SECRET")))?
I remember someone else ran into a problem like that… I think they just saved their secrets.env file from within linux and it fixed the problem. Don’t quite remember. Anyway, if that is the problem and str/trim fixes it, I should have Biff call str/trim for you. In the mean time, you can use a custom use-secrets component to accomplish the same (put this in your main namespace):
(defn get-secret [ctx k]
(some-> (get ctx k)
(System/getenv)
str/trim
not-empty))
(defn use-secrets [ctx]
(when-not (every? #(get-secret ctx %) [:biff.middleware/cookie-secret :biff/jwt-secret])
(binding [*out* *err*]
(println "Secrets are missing. Run `bb generate-secrets` and edit secrets.env.")
(System/exit 1)))
(assoc ctx :biff/secret #(get-secret ctx %)))
(def components
[biff/use-config
use-secrets
...
(As for why this would trigger in the jar and not with a regular bb dev… no idea)
Makes sense. I’ll try it out.
I wish I didn’t have to target Windoze. Sigh.
1 Like
Yup, works now. Good catch! Thanks for the tip. I would have gotten around to guessing that sometime between forced retirement and never. lol.
Also you should make the changes to ui.clj you recommended for loading image and JS resources permanent too. they work fine in dev as well.
1 Like
Also you should make the changes to ui.clj you recommended for loading image and JS resources permanent too. they work fine in dev as well.
Yes, definitely–part of my current batch of WIP changes.