A Hashicorp Boundary Installation Checklist

If you’ve come here after searching for an error message, see Errors That Might Have Led You Here.

I’ve been playing around with the excellent Boundary (0.5.0) from Hashicorp recently, which you can use as a software defined perimeter (SDP) within a zero-trust architecture (ZTA).

It’s early days for Boundary, the supporting docs and community resources. After reading through the docs & tutorials available on the official site, I thought an installation / troubleshooting / debug checklist was in order. As a distributed security system, there is a lot of configuration to get right and several possible failure modes if you mess up.

Clearly this isn’t going to be exhaustive, but hopefully it helps someone!

Nodes

Purpose Software
Client Boundary binary
Client Boundary desktop app
Controller Boundary binary
Worker Boundary binary
Load balancer for controller(s) Web server
Database Postgres

Network

Check these using netcat from *nix systems & telnet on Windows. You will need to start listener on the receiving end using nc -lvp {port}, if you haven’t started the Boundary services yet.

Path Port
Client -> LB 443
LB -> Controller 9200
Controller <-> Worker 9201
Client -> Worker 9202
Worker -> Target Target app port
Controller -> Postgres 5432

OS & Files

Assuming Boundary is being run via Systemd

Node Aspect Check Remediation
Controller & Worker boundary user1 grep boundary /etc/password See below
Controller & Worker boundary group2 grep boundary /etc/group useradd --system --no-create-home --shell /usr/sbin/nologin --user-group boundary
Controller & Worker Allow infinite memory locking (stops secrets leaking via memory paging) Check Boundary service file for LimitMEMLOCK=infinity Capabilities=CAP_IPC_LOCK+ep CapabilityBoundingSet=CAP_SYSLOG CAP_IPC_LOCK Add missing lines, daemon-reload and restart service.
Controller & Worker PKI file ownership3 Check boundary:boundary ownership on certificate cert & key chown boundary:boundary {the files}

Database

Aspect Check Remediation
boundary user is super user4 From psql shell, \du As postgres user in bash, createuser -P --interactive boundary
boundary database exists5 From psql shell, \l As postgres user in bash, createdb boundary
Allow external password authentication5, 6, 7 psql -U boundary -h 127.0.0.1 -d boundary In pg_hba.conf add host all all 0.0.0.0/0 md5
Postgres listening on external interface5, 7 ss -plnt should show 0.0.0.0:5432 In postgresql.conf amend listen_addresses = '*'

Environment Variables

These are optional, but might make things easier in some cases.

Name Should be
BOUNDARY_ADDR {Controller address}:{API port, usually 9200}
BOUNDARY_CACERT Path to a PEM-encoded CA cert file used to verify the Boundary server SSL certificate.
BOUNDARY_CAPATH Path to a directory of PEM-encoded CA cert files to verify the Boundary server SSL certificate.
BOUNDARY_CLIENT_CERT Path to the certificate for Boundary communication
BOUNDARY_CLIENT_KEY Path to the private key for Boundary communication
BOUNDARY_CLIENT_TIMEOUT Timeout in seconds (default 60)
BOUNDARY_TLS_INSECURE Enables or disables SSL verification (boolean)
BOUNDARY_TLS_SERVER_NAME Used to set the SNI host when connecting via TLS
BOUNDARY_MAX_RETRIES Number of retries for the HTTP client
BOUNDARY_TOKEN Add this token to the Boundary client
BOUNDARY_RATE_LIMIT Rate limiting for the client, unsure what value would be used
BOUNDARY_SRV_LOOKUP Enables the client to lookup the host through DNS SRV lookup (boolean)

Errors That Might Have Led You Here

Check for the reference number in the rest of this post

  1. boundary-worker.service: Failed at step USER spawning /usr/local/bin/boundary: No such process status=216/GROUP
  2. boundary-controller.service: Failed at step GROUP spawning /usr/local/bin/boundary: No such process status=217/USER
  3. Enter passphrase for /etc/boundary/pki/key.pem: Error initializing listener of type tcp: error loading TLS cert: EOF status=3/NOTIMPLEMENTED
  4. Exceptions arising from boundary database init, in particular relating to pgcrypto
  5. Unable to connect to boundary database during boundary database init
  6. Peer authentication failed when trying to authenticate as boundary, check this with psql not boundary database init
  7. Unable to connect to 5432 on Postgres from Controller. Confirmed issue if nc -v {postgres IP} 5432 from Controller does not work