Self-hosted npm registry: what to plan for

First package? Follow the publish and install quickstart →

Overview

This guide tells you what a self-hosted npm registry needs, from storage and TLS to credentials, backups and upgrades. It is for platform engineers and their managers, who must decide where private packages will live.

A self-hosted registry runs in infrastructure that your team operates. A self-hosted registry can be the correct choice when you need control over network placement, data location or the release schedule of the server software. Your team is then responsible for each step from a package upload to a successful install.

The guide first describes what a registry service must do. Then it gives a procedure to plan the service and the tests that prove a deployment. It also lists the usual failures and gives a procedure to move packages to a managed registry.

Before you start

Record the answers to these questions. The answers decide which deployment is correct for you.

• Who installs the packages: only your own developers and CI, or also external recipients?
• Where do the installs come from: an internal network only, or the internet?
• Which rules apply to the location of package files and to access records?
• How many engineers can operate the service, and who responds to a failure outside office hours?
• How long can installs and releases stop before the business is affected?

You also need an inventory of the current packages: names, scopes, versions, dist-tags and the projects that depend on them.

What a registry service must do

Package storage is only one part of a registry. The service must also speak the npm protocol, check credentials, serve TLS and keep the data safe. This section describes each part.

The npm protocol

npm, pnpm and Yarn use HTTP requests that the registry must answer exactly:

• GET /<name> returns the package document, with versions, dist-tags and tarball addresses.
• GET /<name>/-/<file>.tgz returns the tarball of one version.
• PUT /<name> publishes a version, with the tarball as a base64 attachment.
• /-/package/<name>/dist-tags reads and changes dist-tags.
• /-/whoami, /-/v1/login and /-/v1/search serve identity, login and search.

For a scoped name, npm encodes the slash as %2f, for example GET /@acme%2fsdk. Each component between npm and the registry must send that path without change.

Storage of metadata and tarballs

A registry keeps two kinds of data. The metadata holds the names, versions, dist-tags and integrity values. The tarballs hold the package files. Each version record points to one tarball and holds its SHA-512 digest.

The two kinds of data must stay consistent. If you restore the metadata without the tarballs, installs fail with 404. If you restore old metadata over new tarballs, a lockfile can request a version that the registry does not list. Make a backup of both kinds of data at the same time. Restore the two kinds of data together.

Published versions must be immutable. If the server lets a person overwrite a version, the integrity values in the lockfiles of your users no longer agree. Then npm stops each install with EINTEGRITY.

Authentication and authorization

A registry for private packages must refuse requests without a valid credential. It must also decide which credential can do which operation. Plan at least these controls:

• Separate credentials to publish and to install.
• Credentials limited to some packages or scopes.
• Rotation, expiry and revocation for each credential.
• Removal of all access when a person leaves the company.
• A record of who published which version and who downloaded it.

Many self-hosted servers keep users in a local file or connect to a directory service. External recipients need a different model: a read-only credential for exact packages that you can revoke when the contract ends.

TLS, DNS and proxies

npm sends credentials in the Authorization header, so the registry must use HTTPS. You need a DNS name, a certificate from a trusted authority, and automatic certificate renewal. If the certificate expires, every install fails at once.

Most deployments put a reverse proxy or a load balancer in front of the registry software. The proxy must keep the Authorization header and the encoded path. It must also accept publish bodies that are larger than the largest tarball.

The registry software writes tarball addresses into the package document. These addresses must use the public hostname and the https scheme. If the software writes an internal address, npm downloads the metadata but cannot download the tarball.

Public packages and dependency confusion

Many registry servers can forward requests for unknown names to npmjs.org, so that one registry address serves public and private packages. This function needs a strict rule. A name that you host must never resolve from npmjs.org.

Dependency confusion is an attack in which a person publishes a public package with the name of your private package. If your registry forwards that name upstream when the private package is absent, your build installs the attacker's code. Configure the server to refuse all forwards for your scopes. Make sure that an internal error gives an error, not a public package.

Plan the whole service

Before you start this procedure, make sure that you have the answers from Before you start. Make a decision for each step. Record each decision.

1. Select the registry software. Verdaccio is an open-source option for npm. General artifact managers also serve npm.
2. Select the storage for metadata and tarballs. Object storage is durable and lets you run more than one server.
3. Configure DNS and TLS with automatic renewal. Set an alarm for certificates that expire within 14 days.
4. Configure the reverse proxy. Keep the encoded path and the Authorization header.
5. Define the credential model: publish credentials, install credentials, rotation and revocation.
6. Configure the policy for public names. Block all upstream forwards for your scopes.
7. Schedule backups of metadata and tarballs at the same time.
8. Do a restore test on a separate server. Then repeat the test each quarter.
9. Configure monitors for publish errors, install errors, latency, disk space and certificate expiry.
10. Assign an owner for security updates and upgrades of the registry software.

This example shows the parts of an nginx configuration that affect npm. When proxy_pass has no path, nginx sends the request path without change, so %2f stays encoded. The default nginx body limit is 1 MB, which is too small for many publishes. The publish body is larger than the tarball, because npm encodes the tarball as base64.

server {
    listen 443 ssl;
    server_name npm.internal.example.com;

    # Accept publish bodies larger than the largest tarball.
    client_max_body_size 100m;

    location / {
        # No URI part: nginx keeps %2f in scoped names.
        proxy_pass http://127.0.0.1:4873;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Authorization $http_authorization;
    }
}

CAUTION: Configure the public URL in the registry software also. If the software does not know its public URL, it can write internal tarball addresses into the package documents.

Verify the result

Do these tests from a clean container on a network that your recipients use. A test on the server itself does not prove that DNS, TLS and the proxy work.

1. Publish a scoped test package, for example @acme/registry-test. npm prints the name and version.
2. Run npm view @acme/registry-test dist.tarball. The address must use the public hostname and https.
3. Install the package in a new project with an install credential. npm installs it with no error.
4. Publish a package that is larger than 10 MB. The proxy must not answer 413.
5. Publish the same version again. The registry must refuse the request.
6. Revoke the install credential.
7. Install the package again with the revoked credential. The registry must answer 401 or 403.
8. Request a scoped name that does not exist. The registry must answer 404 and must not return a public package.
9. Restore the last backup on a separate server.
10. Install the test package from the restored server. npm installs it with no error.

Troubleshoot

Security and maintenance

WARNING: Do not expose the administration interface or the storage bucket of the registry to the internet. An attacker with write access to storage can replace tarballs.

• Apply security updates to the registry software, the proxy, the servers and Node.js on a fixed schedule.
• Rotate install and publish credentials on a schedule, and at once when a person leaves.
• Keep backups in a different account or location from the primary storage.
• Keep access logs and publish records for the period that your audit rules require.
• Examine the list of users and credentials each quarter. Remove the entries that nobody uses.
• Monitor disk space. A full disk stops publishes and can damage metadata.

When a managed registry fits

If your main goal is to deliver private packages, a managed service can operate the infrastructure for you. PrivateNPM.com is a hosted private registry for npm, PyPI and files. It has shared scopes, private registries on custom domains, publisher keys and read-only download keys.

A custom registry domain changes the address that recipients use. A custom domain does not make the deployment self-hosted. Compare the total operations work with the subscription price: backups, upgrades, incident response and the time to build access rules for recipients. The plans page shows the limits of each plan.

Move packages to PrivateNPM.com

Before you start, create a scope or a private registry and a publisher key. The registry setup guide gives the steps. Keep the old registry available until all projects use the new one.

1. List the versions on the old registry: npm view @acme/sdk versions --json.
2. Download each tarball from the old registry: npm pack @acme/[email protected]. npm writes acme-sdk-1.0.0.tgz.
3. Examine publishConfig in each tarball. Remove an old registry address before you publish.
4. Change the scope line and the credential line in .npmrc to the new registry.
5. Publish each tarball, from the lowest version to the highest: npm publish ./acme-sdk-1.0.0.tgz.
6. Compare the dist-tags on the two registries with npm view @acme/sdk dist-tags. Correct them with npm dist-tag add.
7. Compare npm view @acme/[email protected] dist.integrity on the two registries. The values must be the same.
8. Update the .npmrc of each project, then update its lockfile.

npm writes the full tarball address into package-lock.json. By default, npm replaces only the npmjs.org host in those addresses. To make npm use the configured registry for each address, set replace-registry-host=always in .npmrc, or create the lockfile again.

NOTE: When the integrity values are the same on the two registries, the current lockfiles stay valid. Only the host in the tarball addresses changes.

Related guides

• What is a private npm registry?
• Create a private npm registry
• Verify a custom npm registry domain
• Cache public npm packages safely