Dependabot + private Go proxies: how they work and why it matters

GitHub's Dependabot now supports private Go proxies/registries in preview, which means it can finally update dependencies that live behind authenticated Go proxies (like River Pro). In fact, it wasn't just a matter of not being able to update private modules: using any private Go modules meant that Dependabot would break altogether in that project.

But what are Go module proxies? How do they work, and why does this matter? Read on to find out.

How Go module proxies work

Go resolves modules through a list of proxies set in GOPROXY according to the Go module proxy protocol. The default is https://proxy.golang.org,direct, meaning: try the public proxy first, and if it doesn't have the module (denoted by 404 or 410 status codes), fall back to fetching directly from the origin VCS ("direct").

If the public Go proxy doesn't have a module, it'll attempt to fetch it from origin in order to serve the user's request directly. If successful, it will cache the module ~forever (with a few exceptions) and use it to serve future requests, so that even if the underlying source module becomes unavailable, Go programs which depend on it can continue to function. If the public proxy fails to fetch the underlying module (say it's in a private GitHub repository) then it'll 404 to the user, allowing the local Go tooling to move on to the next proxy in the list or try to fetch it directly. This works well as a default, because even private dependencies on GitHub can be fetched successfully by a local go client in direct mode if the user already has local auth configured for GitHub.

Users can customize the GOPROXY list, adding custom public or private proxies, even removing the public proxy or direct options altogether. A proxy may also opt to serve only specific modules; if it returns 404s or 410s on the rest, they'll fall through to the next proxy in the list.

Large enterprises might use a private proxy like a self-hosted Athens as their first or only option, allowing them to not only centralize authentication, but also to play a gatekeeping role by rejecting unvetted dependencies using other status codes like 403.

Until writing this post, I hadn't even realized that there were two fallback modes for the GOPROXY list:

• List elements separated by commas (i.e. GOPROXY=a,b) receive the normal fallback behavior, with only 404 and 410 status codes causing the request to fall through to the next proxy in the list.
• List elements ending with a pipe character (i.e. GOPROXY=a|b) will fall back to the next proxy in the list on any error, including 5xx, timeouts, etc.

Private modules

There are two primary use cases for private modules:

1. A company's proprietary internal source code is typically kept private to its team. If these modules were public, the public Go proxy could fetch them and then forever make their source code available to the world.

   In most cases, the company's internal module names like github.com/example/apiserver are uninteresting and not likely to expose a lot about the company's business or internal architecture. Sometimes, however, they might reveal upcoming products, features, or other details the company doesn't want to share with the world. Imagine a module like your-company.example.com/new-feature-name including a name or other details the company doesn't want to share with the world.

2. A provider like River might offer a specific limited set of modules to paying customers as in the case of River Pro. In this case, the provider may not want to cache and serve the entire universe of public modules (even to its customers), but they also don't care if its module names are sent to other proxies (they are publicly documented, after all).

Why proxy order matters (and why cascading matters)

Let's look at how the Go toolchain fetches modules from proxies given a GOPROXY=https://proxy.golang.org,https://riverqueue.com/goproxy,direct.

When a public module is fetched from a public proxy, the flow is simple. The module name is sent to the proxy as part of the several request paths like /github.com/jackc/pgx/v5/@v/v5.0.0.info and if the public proxy already has that module available, it returns a 200 for each of those. If it doesn't know about that module or version, internally it will attempt to fetch it in direct mode so that it can not only serve it to the current user, but also cache it for future requests. After these 200 responses, the toolchain stops there and moves on to the next step.

For private modules, a public proxy should fail because the module is unavailable to the public proxy for one reason or another (authentication credentials required, internal networks only, etc.) so its own internal attempts to resolve the module in direct mode fail and it returns a 404.

The go tool interprets this 404 to mean that it must try the next proxy in the list and it proceeds to do so, this time making a request to River's private proxy for the same module. This time, River's private proxy will have the module available and will return a 200 for the request, completing the toolchain's quest for this module and allowing it to move on to the next step.

Finally, let's imagine a third scenario: a company's internal private module including the top secret new-feature-name with a GOPROXY=https://proxy.golang.org,https://your-private-proxy,direct. In this case, the go toolchain will first attempt to request the top secret module by name from the public proxy (which will fail), and then it will request it from the private proxy, ultimately falling back to direct mode if that too fails.

Oops, we've just sent the top secret new-feature-name to the public proxy! Depending on your exact use case and your organization's security posture and exactly which proxy you're using, this might not matter—or you might have just leaked the name of a secret government project to a hostile foreign government.

The point is that there are definitely cases where the GOPROXY order matters. Internal company proxies should come first in the list, and for truly sensitive use cases, you should consider using GOPRIVATE.

The GOPRIVATE and GONOPROXY environment variables

Go offers other options for more sensitive use cases: the GOPRIVATE environment variable, which acts as a default for GONOPROXY and GONOSUMDB. From the docs (emphasis added):

The GOPRIVATE or GONOPROXY environment variables may be set to lists of glob patterns matching module prefixes that are private and should not be requested from any proxy.

Any module names matching the patterns in GOPRIVATE or GONOPROXY will be skip the proxy resolution path entirely and go straight to fetching from version control. Most teams using private go modules can set GOPRIVATE=yourcorp.example.com to exclude all modules beginning with the yourcorp.example.com prefix from ever being fetched from a proxy or hitting the checksum database.

Check out the private modules docs for more scenarios and examples.

GONOSUMDB

The GONOSUMDB environment variable is used to disable checksum verification for private modules. This is a topic for another post, but in short, Go maintains a cryptographic checksum database for all public modules, and modules are (by default) verified against this database when fetched to prevent tampering.

Private modules can never be resolved publicly by the public proxy, so they'll fail any attempts to validate checksums against the checksum database. Those using private modules must set either GOPRIVATE or GONOSUMDB to disable checksum verification for private modules with a given module prefix.

Dependabot private Go proxy support

A long-running GitHub issue covers customer requests to add support for private Go proxies to Dependabot. As stated at the top, this deficiency not only meant that Dependabot couldn't update private modules, but it also meant that Dependabot couldn't function at all in Go projects which relied on private Go modules served through authenticated proxies.

This affected River Pro customers in particular, which meant that adopting River Pro meant giving up Dependabot (at least for Go dependencies in that project).

As of yesterday, that's been fixed with a preview release! With private proxy support, Dependabot can:

• Authenticate to a private Go proxy (like River's), resolve private modules, and propose PRs normally—even updating private dependencies.
• Be configured to resolve from proxies in a specific order, such as trying the public proxy first, then falling back to a private proxy for private non-sensitive modules (like the River Pro packages) — mirroring how many teams configure local GOPROXY cascading. Alternatively, a private internal proxy can be used first or even exclusively to ensure that private module names are not leaked.
• Respect GONOSUMDB so checksum verification doesn’t fail on private modules.

Quick start: Dependabot + River Pro

If you're using River Pro and want to use Dependabot to update your Go dependencies on a regular basis, configure two registries for Dependabot: the public Go proxy, and River's private proxy. Dependabot will try them in order, with the proper fallback behavior.

version: 2
registries:
  golang-proxy:
    type: goproxy-server
    url: https://proxy.golang.org
    username: ""
    password: ""
  riverpro-proxy:
    type: goproxy-server
    url: https://riverqueue.com/goproxy
    username: river
    password: ${{secrets.RIVER_PRO_SECRET}}
updates:
  - package-ecosystem: "gomod"
    directory: "/"
    groups:
      go-dependencies:
        update-types:
          - "minor"
          - "patch"
    registries:
      - golang-proxy     # prefer the public proxy first
      - riverpro-proxy   # fall back to River's private proxy
    schedule:
      interval: "weekly"

Finally, add a go.env to your repository root with a GONOSUMDB to disable public checksum validation of private modules. Dependabot will automatically respect any Go env settings in this file.

GONOSUMDB=riverqueue.com/riverpro

See also our docs on configuring Dependabot for River Pro.

Notes and caveats

• This capability is in preview; config keys and behavior may change. Track the discussion on the GitHub thread.
• In developer environments, it’s fine (and often helpful) to set GOPRIVATE for your organization’s module patterns when you intend to go direct. In CI/automation with a private proxy, it’s usually cleaner to rely on a private proxy plus GONOSUMDB.
• Avoid disabling checksums globally (GOSUMDB=off). Use the narrowest GONOSUMDB/GOPRIVATE patterns that cover only your private modules, and avoid broad path prefixes like GONOSUMDB=github.com.