All articles
Blake Gentry

River UI as a packaged Go module

When we shipped River UI, we decided to ship it early even before we’d solved some important problems. We knew that a lot of River users would find a way to get value out of it even before it had any authentication options, whether by using modern zero trust HTTP proxies or via internal-only deployments.

We also knew we would want to quickly offer some better options, and that we didn’t want to find ourselves implementing and supporting every auth option under the sun. We were confident that an embeddable http.Handler would offer the flexibility to serve most use cases, and would make it possible to for developers to avoid deploying and operating yet another standalone HTTP service.

Making that work would require overcoming two big hurdles:

  1. While River UI exists as a Go backend API, its frontend is a React app written in TypeScript which has its own build pipeline. This is incompatible with the typical distribution of Go modules, which are bundled directly from repository source code without any opportunity to run additional build or compilation steps.
  2. Vite’s frontend tooling is optimized for static distribution with a hardcoded path prefix that must be supplied at build time, but which cannot normally be changed dynamically at runtime.

Today we’re releasing River UI v0.5.0, which overcomes these issues and allows for seamless embedding in an existing Go app as an http.Handler. Read on for more about how this works, or get started now with the docs.

Go module distribution

Historically, most Go code has been fetched by the Go tooling directly from the version control systems at the package’s source URL. The code in these repos needs to be fully ready-to-compile without additional tooling, except for libaries that users are required to have on their systems (such as C dependencies). This is still by far the most common way to distribute Go code today, though Go modules brought new possibilities.

Getting static non-Go files into a compiled Go binary was also a challenge for the first decade of Go’s existence until Go 1.16 brought us the embed package. Embed made it trivial to include static files into compiled Go apps, so long as they existed in a known hardcoded relative subdirectory of the module being built. But a JS project typically does not commit built assets to its repo, instead using a Node runtime with modules to build it.

For River UI to be usable as a Go module, it would need to ship with ready-to-use static assets or require each user to run a complex install and build pipeline prior to compiling anything that uses the module. We saw three ways of making this happen:

  1. Rewrite the frontend to use mostly-static files like plain unbuilt JavaScript, html/template, or HTMX which could exist in their embeddable form in our git repo. We have already built River UI’s frontend, don’t want to rewrite it, and plan to build even more stuff that takes advantage of the modern frontend.

  2. Commit the built JS, HTML, CSS, and image assets in dist to the River UI repo so embed can automatically include them. This goes against common conventions in the JS ecosystem. It would also bloat our Git repo with hundreds of KB or MB of files changing on every release, and would result in significantly larger Go module zip files than necessary (due to automatically bundling the raw frontend source and assets).

  3. Leverage an alternative option from the Go module system to distribute pre-packaged module zip files that contain built static assets, Go source code, and no unncessary bloat. While this sounds complex, it turns out we had already done all this work as part of shipping River Pro as a private Go module.

River UI v0.5.0 chooses option 3 and is available today via the Go module riverqueue.com/riverui.

How the Go module works

Since the launch of Go modules, many projects have opted for so-called “vanity names” made possible by Go’s module lookup via meta tag. Typically these are added to a static content site at the module’s URL and will direct the go tooling directly to an underlying public VCS like git:

<meta
  name="go-import"
  content="golang.org/x/mod git https://go.googlesource.com/mod"
/>

But it turns out there’s another option: serving bundled modules directly from a Go module proxy:

<meta
  name="go-import"
  content="example.com/gopher mod https://modproxy.example.com"
/>

The above mod directive tells the Go tooling to fetch the example.com/gopher module by communicating with the modproxy.example.com server using the GOPROXY protocol. That protocol allows for proxies to directly serve up their own modules as bundled .zip files which can be created through any build process.

This option is not merely less utilized—it’s virtually nonexistent in public contexts. I had actually never seen it used outside of enterprise Go proxies like Athens, at least not until launching River Pro.

As of v0.5.0, the riverqueue.com/riverui module is distributed in this fashion using our custom Go module proxy. And because of the tiered caching architecture of Go modules, the vast majority of users will be fetching these modules through the proxy.golang.org default module proxy without ever hitting our servers.

While this architecture is unconventional in the Go ecosystem as of today, I believe it could prove useful to a lot of other projects that might want to keep raw source files in their VCS repo, while also bundling built artifacts into their released modules. It’s also what you’d seee in most other language ecosystems’ package managers: at the time of each release, a release script bundles relevant files into an archive file that’s then distributed, rather than merely snapshotting the raw VCS layout. The notable difference is that with Go modules, this setup can be still function in a decentralized way (if you disable the default proxy or choose an alternative).

Dynamic paths in Vite

Frontend JS apps typically build into static assets that make them suitable for hosting on S3, GitHub pages, or direct from disk. Vite allows specifying a base path, but because the expectation is that it’ll produce static files it needs to know the path at build time. This doesn’t work for the use case with River UI and the pre-built assets setup discussed above; we want users bringing it into their application to be able to nest it at any subpath without having to rebuild the asests or even have the required Node build tooling installed.

This turned out to be significantly more difficult than I expected, particularly setting up a decent development experience for doing this within a Go app. In my extensive research, I was lucky to stumble upon exactly one example of this in the wild: Dozzle. Amir Raminfar had already pioneered exactly the approach I needed, allowing a Go server to dynamically render the root index.html file to inject the dynamic path prefix, while also letting Vite handle the rest of the static assets.

You should buy him a beer 🍻 if you’re happy to see these improvements—I may have given up on solving this problem if I hadn’t found his example.

Try it out

If you’ve held off on trying River UI because of the lack of authentication or because you wanted to run it with a path prefix, give it another shot!