Biff updates and roadmap

As per Clojurists Together funding, I’m planning to spend about half of my Biff time over the next few months on writing documentation/content. The remaining time will be spent on code.

I cut a release for v0.7.9 a couple months ago; it has a few small tweaks. I’ve also technically made a v0.7.10 release, though I haven’t got around to publishing a release post on GitHub. That one modifies the template project so that in new projects, the middleware stack is copied into your project source. That way it’s easier to modify and debug.

Other than that, I’m currently modifying the bb deploy and bb soft-deploy commands so that they always use rsync to push your code to the server. They’ll fall back to git push only if rsync isn’t avaliable (e.g. if you’re on Windows without WSL). It occurred to me that there’s no real advantage to using git push if rsync is an option, and this will eliminate the need to deal with some git push-related inconveniences/bugs that have popped up.

Following that I’ll be sanding off a few rough edges that have come up on Slack lately, and then I’ll be doing a bunch of XTDB stuff. I still have not had a chance to play with v2 yet. I’d like to make a proof-of-concept fork/branch of Biff with that replacing v1. In the mean time before v2 is production-ready, I’ve thought it might be nice to revisit Biff’s transaction format and see if it could be made a little more ergonomic. That would also be a good chance to refactor the biff/submit-tx code, which is a bit hairy.

(I also wonder if some of this XTDB helper stuff might be worth releasing as a standalone library, so it can be used outside of Biff projects…)

Another thing I’d like to do is figure out a good story for materialized views in Biff, so queries can stay fast as your database grows in size. This has been the main pain point I’ve had in my own apps. A secondary search index(es) for XTDB might work well; that’s how the Lucene module works. It also wouldn’t hurt to check out Rama.

If I actually finish all that stuff and nothing else jumps to the head of the TODO list, the final things I have planned are to work on Yakread and Platypub. I’m planning to open-source Yakread and finish implementing a few features that I’d like for myself; at which point I’d move on to Platypub for a while and get that fully baked and approachable to potential contributors. Then I’ll circle back around to Yakread and make that approachable as well—in my initial pass I am planning to open-source Yakread, but I won’t spend much time trying to simplify things for contributors until after I’m satisfied with Platypub.

We’ll see how far I get into that before all my plans get reorganized :slightly_smiling_face:.