Rotated secrets

A rotated secret holds the password of an account that already exists, on one of your hosts or database servers, and your Burrow gives that account a new password on a schedule, after a number of reads, or when you ask. On a host, its Ratel Node makes each change. On a database server, your Burrow makes it through an admin login it signs in with. Either way, your Burrow keeps a new password only once the account has it. Machines read the account and its current password like any other secret.

How a rotated secret works#

• Its value is the account's password. Every new password is a new version, so Version History lists each one and you can reveal any of them.

• The password changes where the account is. The node, or on a database server the admin login, sets the new password and checks the account took it before your Burrow keeps it. If the account doesn't take it, its previous one is set back.

• One secret per account. An account another secret rotates names that secret, and can't be picked again.

• Never edited by hand. You don't type a rotated secret's value, and an older version can't be put back: the account always has the newest one.

Before you start#

A Windows or SSH account is reached through the Ratel Node on its host, running ratel-node 0.5.0 or newer. Add the node on Nodes first. The node runs as SYSTEM on Windows and as root on Linux, so it needs no password of its own to change an account's.

An account on a database server is reached through an admin login your Burrow signs in with, the way a dynamic secret makes its logins. The admin login has to be allowed to change other accounts' passwords:

For example, a Redis admin login for a server with an ACL file:

ACL SETUSER rotator on >admin-password -@all +@connection +acl +info +config|get
ACL SAVE

And an MSSQL admin login on SQL Server 2022 or newer, with a user in app, the database its connection names:

CREATE LOGIN rotator WITH PASSWORD = '<admin password>';
GRANT ALTER ANY LOGIN, VIEW ANY CRYPTOGRAPHICALLY SECURED DEFINITION TO rotator;
USE app;
CREATE USER rotator FOR LOGIN rotator;

And a MongoDB admin login that can change the passwords of app's users, and nothing else:

use admin
db.createRole({ role: "rotate_app", privileges: [{ resource: { db: "app", collection: "" }, actions: ["viewUser", "changePassword"] }], roles: [] })
db.createUser({ user: "rotator", pwd: "<admin password>", roles: ["rotate_app"] })

And an Oracle admin login for the users of the pluggable database ORCLPDB1, made as SYS:

ALTER SESSION SET CONTAINER = ORCLPDB1;
CREATE USER rotator IDENTIFIED BY "<admin password>";
GRANT CREATE SESSION, ALTER USER TO rotator;
GRANT SELECT ON SYS.USER$ TO rotator;
GRANT SELECT ON SYS.DBA_USERS TO rotator;
GRANT SELECT ON SYS.DBA_PROFILES TO rotator;

Accounts it can rotate#

On a host, an account is found by its id, a Windows SID or a Linux uid. One that's renamed keeps its secret, and one that's deleted and made again under the same name is never taken for it. A PostgreSQL role is found by its id and its name together, so one that's renamed, or dropped and made again, takes a new secret. A MySQL or MariaDB account is found by its user and host, and a Redis user by its name, since those servers know them by nothing else: one that's renamed takes a new secret, and one made again under the same name is the same account to them. An MSSQL login is found by its SID and its name together, so one that's renamed, or dropped and made again, takes a new secret, and the same login on another replica of its availability group, under the same SID, is the same account. A MongoDB user is found by its userId and its name together, so one that's dropped and made again takes a new secret; MongoDB can't rename a user. An Oracle user is found by its id in its container and its name together: Oracle never gives a dropped user's id to another, so one dropped and made again takes a new secret. A common user has another id in each container, so its secret keeps the root's.

Create a rotated secret#

1. In an environment, select the + at the top right of the secrets table and choose Rotated.

2. On Account, name the secret and choose the kind of account. For a Windows or SSH account, pick the node: its host's accounts are listed. For one on a database server, enter its connection and select List roles or List users. Choose the account. One that can't be rotated is grayed out with the reason.

3. For a Windows or SSH account, leave Host empty for machines to connect to the host's own name, or enter the address they should use.

4. On Password & rotation, enter the account's current password, and select Check it to have it checked now. Or choose Have the node set a new one, or on a database server Have the admin login set a new one: the account is given a new password once the secret is saved, and its current one stops working.

5. Choose when it rotates and how its passwords are made, select Continue, check the review, and select Create rotated secret.

Saving checks the current password again where the account is. With Rotate once it's saved on, as it is to start with, the account is given a new password straight away, so the one you entered stops working. With it off, the secret keeps the password you entered until its first rotation.

When a password is turned down#

Windows services and scheduled tasks#

A Windows service that runs as the account starts with its password. Under Services, every service that runs as the account is ticked, and each one ticked is given the new password with every change. A service you untick keeps the old password and won't start again after a change.

Scheduled tasks that store the account's password are listed by name. They keep the old password: update them on the host, or run them as another account.

New passwords meet Windows' rules: at least three kinds of characters, at most 127 characters, and at least the host's minimum password length. When the host expires the account's password after a number of days, the form says so, so you can rotate more often than that.

SSH accounts#

An SSH account signs in with the password the secret holds. When the host's SSH server wouldn't take it, such as one that refuses passwords or only lets listed users in, the form says why before you save. The password still rotates on the host.

Accounts on a database server#

The connection is the one a dynamic secret's form asks for: Host and Port, the Database to connect to (not on Redis; on MongoDB, the database whose users it lists; on Oracle, the service of the pluggable database whose users it lists, or the root's for its common users), the Admin user and its Admin password, and Connection security, with the server's CA certificate when it has a private one. Machines read the same host, port and security.

Your Burrow checks a PostgreSQL role's password by signing in as it, as a machine would. It checks a MySQL or MariaDB account's, a Redis user's, an MSSQL login's, a MongoDB user's and an Oracle user's against the hash the server keeps, never by signing in, so a wrong one doesn't count toward locking the account. Before the secret is saved, your Burrow also tries the change without changing anything (on MongoDB and Oracle, which can't, it reads the admin login's privileges), so an account the admin login can't change is turned down then.

PostgreSQL and MariaDB are sent the hash they keep of a new password, so no statement carries it, unless the server checks passwords against a policy that needs to see them. MySQL is sent the password, which it masks in the statements it logs, and any secondary password is discarded first, so only the new one works. Redis is only ever sent the hash. MSSQL is sent the password, since SQL Server refuses a hash for a login under its password policy, and keeps it out of what it records: Extended Events, the plan cache, Query Store and audits. MongoDB is sent the password too, since it takes no hash, and writes it as xxx in its log and its error messages; its profiler doesn't record the change, and MongoDB Enterprise's audit log records only that the password changed. Oracle is sent the password, since a hash would pass by the user's profile's password function, and keeps it out of V$SQL; its unified audit trail records the change as IDENTIFIED BY *.

Redis users#

Redis keeps its users on each server apart, so a Redis user's password is changed on every server it lives on: the server the connection names and the replicas it lists, or every node of a cluster. A connection that names a replica leads to its primary. Each server is reached with the same admin login and security.

A change comes in two steps, so no client finds its password gone: the new password is added beside the user's own everywhere, and once every server has it and the secret holds it, the other passwords are taken off. A server that can't be reached holds the change back, and the user's old password keeps working everywhere meanwhile.

Each change is saved the way the server keeps its users: ACL SAVE to its ACL file, or CONFIG REWRITE to its redis.conf, which also writes any other setting changed since the server started.

Your Burrow checks the user after every change, every 5 minutes, and soon after the secret is read, at most once a minute. A check puts the password back on a server that lost it, such as one restarted from a file that didn't have it, and takes any other password, and nopass, off. It never makes a user that's missing: the secret shows Last check found a problem until the user is made again, and then its password is put back. While all is well, Status says when the user was last found in place on every server.

When you pick a user with more than one password, the form says so: once the secret is saved, its own is the only one that works.

MSSQL logins#

SQL Server keeps each replica's logins apart, so your Burrow changes a login on every replica of its availability group, signing in to each with the connection's admin login and security. It finds the replicas through the server the connection names, which can be the group's listener, and reaches each at its read-only routing address, or else at the host its endpoint names on the connection's port.

The change is made on every replica and committed only once all of them have it, the secondaries first and the primary last. A replica that can't be reached, or refuses the change, leaves every replica with the password it had: the change waits for it, or is turned down with the reason. A replica without the login is noted on the secret, and isn't given one.

A contained availability group keeps its logins in its own master and copies them to its replicas itself. Connect through the group's listener, or name one of its databases, and the login there is changed once.

A change keeps the login's own password policy and expiration settings, and clears MUST_CHANGE. A login whose password also expires under the server's policy has to rotate more often than the policy's maximum age; the form says so when you pick one.

Before the secret is saved, your Burrow changes the login's password on every replica in a transaction it rolls back, to prove the admin login may change it. A server audit records that attempt as a password change.

Rotated secrets change logins on SQL Server 2012 and newer. Azure SQL Database, Azure Synapse Analytics and Microsoft Fabric are turned away when you list their logins.

MongoDB users#

A MongoDB user belongs to a database, which is also where it signs in. The connection's Database names the database whose users your Burrow lists, and machines read it as database. A user is written with its database, such as app_user@app.

A replica set copies its users to every member itself, so your Burrow changes a user on the primary alone. A connection that names a secondary leads to the primary that member names, reached at the address the replica set gives it, with the connection's admin login and security. If the Burrow can't reach the primary there, the change waits.

A change counts once a majority of the replica set has it. A primary that can't get a majority to confirm a change within 5 seconds has made it all the same, so the change is unconfirmed, and sent again until a majority has it.

Behind mongos, the users are the cluster's, which its config servers keep. A shard's own users are reached by connecting to that shard.

A change keeps the user's own SCRAM mechanisms, so one that signs in with SCRAM-SHA-256 alone keeps only that. MongoDB can't try a change without making it, so before the secret is saved, your Burrow reads the admin login's privileges instead, which is what MongoDB checks a change against.

Rotated secrets change users on MongoDB 4.4 and newer, on servers you run. MongoDB Atlas, Amazon DocumentDB and Azure Cosmos DB are turned away when you list their users, and so is the $external database, whose users sign in with a certificate, LDAP or Kerberos.

Oracle users#

An Oracle user belongs to a container. A connection to a pluggable database's service, such as ORCLPDB1, lists that database's own users. One to the container database's root lists its common users (C##…), whose one password covers every container and is changed only in the root. Machines read the service as database. A user is written with its container, such as APP@ORCLPDB1.

RAC and Data Guard need nothing more: one dictionary holds the users, and a standby takes each change from the primary. A connection to a standby is turned away when you list its users: connect to the primary.

Oracle's password functions judge a change an admin makes too, so new passwords carry at least two of each kind of character you pick, as Oracle's strongest function asks, and are at most 30 characters, the most 19c and 21c take. When you pick a user, the form says what its profile's password function asks of a new password, and when its profile expires one. If the function turns a password down, the change shows Oracle's reason.

Under a profile with a gradual rollover (PASSWORD_ROLLOVER_TIME), Oracle lets the previous password sign in beside the new one until the rollover time is up. Your Burrow ends the rollover as soon as it changes the password, so only the new one works. A change also opens a user whose password has expired.

Oracle can't try a change without making it, so before the secret is saved, your Burrow reads the admin login's privileges and the user's state instead.

Rotated secrets change users on Oracle 19c and newer.

Schedules and passwords#

A read is any time the current password is handed out: a machine reading the secret, a member revealing it, or a share link showing it. Reading only the account's name doesn't count.

Each new password is random characters at the length you set, with at least one of each kind you pick (two, for an Oracle user): A to Z, a to z, 0 to 9 and symbols. It never contains the account's name.

To change the schedule or how its passwords are made, choose Rotation from the secret's row menu, as for automatic rotation. Switching rotation off leaves the account its password until someone rotates it, and switching it back on gives the account a new password straight away.

Read it from a machine#

A machine reads a rotated secret the way it reads a structured secret, with the CLI or an SDK:

On a database server, these are the fields a dynamic secret's login there gives, in the same order, so either can stand in for the other.

ratel secret get myapp/prod/BACKUP_ADMIN | jq -r .password

ratel run gives one variable per field: for a secret named BACKUP_ADMIN, BACKUP_ADMIN_HOST, BACKUP_ADMIN_USERNAME, BACKUP_ADMIN_PASSWORD and so on. A command started with ratel run --watch restarts when the account gets a new password, or the secret moves to another account. Until its first password is set, a read says so, and ratel run leaves the secret out.

Where its password stands#

A rotated secret's row shows its account after its name. While its password is anything but settled, a warning follows them in place of the rotation mark, red when the last change was refused or it needs attention and amber otherwise. Hover it for the state, what it means and why the last change didn't go through. While a change is on its way, the rotation mark turns instead. Configure, on the row menu, shows it in full on Status.

Rotate now and change it#

Choose Configure from the secret's row menu. Each tab saves on its own.

In the terminal, secret create --rotated and the rotated commands do the same; see Interactive CLI.

Delete a rotated secret#

Deleting a rotated secret leaves its account with the password it has now, so reveal it first if you'll need it. While the account is being given a new password, the secret can't be deleted, nor its environment or project: try again once the change is done.

Removing a node leaves the accounts on its host as they are. Their secrets stop rotating until you move them to another node.

Audit log#

Every change is recorded in your audit log, and a webhook can subscribe to any of these events:

Reads are recorded as for any secret, by the machine or member that read it.