Staging to production, one image

Before you start

The project is live in production, the way First deploy to a fresh VPS leaves it. You work on the machine that registered it, from the project directory, and your login may push images.

$ grim env:pull production
$ rm .env.production
$ grim auth:whoami

Staging here shares the server with production. It gets its own directory, database, vhost and certificate, so the two never touch. A second server works the same way: raise it and give it its token as in the first deploy, then use its alias wherever this page says vps1.

The local stack is up and the suite passes. The release in step four runs the tests before it builds.

The ritual, in order

Seven steps. The first three are done once and set staging up. The last four are the flow you repeat, and only the sixth touches production.

1. I

   Point the staging name at the server

   The certificate for staging is issued the moment the environment is registered, and Let's Encrypt has to find the server behind the name. Create the A record at your DNS provider and wait until it answers.

   $ dig +short A staging.shop.example.com
   203.0.113.10

   An empty answer means the record has not spread yet. Go on only when the address is there.

2. II

   Give staging a home

   Run grim server:add-project again from the project directory, with the staging domain and --env=staging. Production is not touched.

   $ grim server:add-project shop --host=vps1 --domain=staging.shop.example.com --env=staging

   It creates /opt/shop-staging, the database and user shop_staging, the vhost grim-shop-staging.conf, a certificate and a fresh .env with its own APP_KEY. In the summary the Environment row says staging, All envs lists production, staging, and Next is grim release staging. Read the SSL row before you go on. failed does not stop the run.

3. III

   Name both environments in grim.json

   grim.json probably carries deploy.host and deploy.domain from the first deploy. The top-level domain is production's and staging never inherits it. Without an entry of its own, staging takes its domain from your server registration in ~/.grim/servers.json, which a colleague's machine and the CI runner do not have. Move both keys under environments, one entry per environment, leave the rest of the deploy block as it is, and commit the file.

   "deploy": {
     "environments": {
       "production": { "host": "203.0.113.10", "domain": "shop.example.com" },
       "staging": { "host": "203.0.113.10", "domain": "staging.shop.example.com" }
     }
   }

   Set no path here. Production lives in /opt/shop and staging in /opt/shop-staging, and both are found without it. Then let grim env:pull prove that the file resolves staging.

   $ grim env:pull staging
   $ grep -E '^(APP_URL|DB_DATABASE)=' .env.staging
   APP_URL=https://staging.shop.example.com
   DB_DATABASE=shop_staging
   $ rm .env.staging

4. IV

   Release to staging

   grim release with the environment named picks the next tag, runs the suite, builds the image, pushes it and deploys it to staging. Always name the environment. Without an argument the target is production.

   $ grim release staging
   
   Releasing shop@v1.4.3
   …
   Deploy shop@v1.4.3 to 203.0.113.10 (staging)?

   Note the tag from the first line. It is also the Tag row of the deploy summary. The deploy runs the hooks from grim.json on staging, so the migrations meet a real MySQL here before they meet production.

   The first deploy to staging creates admin@staging.shop.example.com with a random password. It is shown once, in the Admin and Password rows of that deploy's summary. Copy it before the terminal scrolls away.

5. V

   Look at it

   Ask the site, read what the app logged while it started with grim logs, then click through the change in a browser. Staging has its own database, so what you create here stays here.

   $ curl -I https://staging.shop.example.com/api/v1/healthcheck
   $ grim logs app --remote=staging

   When something is wrong, fix it, commit and repeat step four. The next tag is v1.4.4, and that is the one that goes on.

6. VI

   Send the same tag to production

   grim deploy builds nothing. It tells production to pull the tag that staging is running and to restart on it.

   $ grim deploy production --tag=v1.4.3
   Deploy shop@v1.4.3 to 203.0.113.10 (production)?

   The server pulls the image, restarts the containers, waits for the app to report healthy and runs the same hooks, migrations included. ✓ Deployed shop@v1.4.3 to 203.0.113.10 is the last line.

7. VII

   Check production

   Look at the live site the same way you looked at staging.

   $ curl -I https://shop.example.com/api/v1/healthcheck
   $ grim logs app --remote=production

   When the release turns out badly, Roll back a deploy is the way back.

What you have now

Two stacks of one project, side by side, behind the same nginx and Traefik.

/opt/shop/            production, database shop
/opt/shop-staging/    staging, database shop_staging

Each has its own .env and its own .deploy-history. grim env:set, grim env:pull and grim env:push take the environment as their argument. The remote:* commands and grim server:auth take none and act on the environment that was registered first, which is production.

grim.json names both targets, so a colleague whose key the deploy user accepts can run steps four to seven without registering anything. From now on a release is grim release staging, a look, and grim deploy production --tag=<tag>.

When it does not work

Staging answers on the production domain

deploy.environments.staging.domain names the production hostname, or staging was released by a grim older than 1.0, which handed the top-level deploy.domain to every environment. Two stacks then claim one hostname. Fix deploy.environments.staging.domain, then deploy the current tag again to both environments so that each compose file is rendered with its own domain.

“Domain "staging.shop.example.com" is already served by another project's nginx vhost on 203.0.113.10:”

Staging used to be a project of its own, such as shop-staging. Retire it first with grim server:remove-project shop-staging --keep-ssl --keep-db, then repeat step two.

The staging site answers without a certificate

The SSL row said failed. The DNS record was not visible yet. Wait until dig shows the address, then run the certbot command that was printed under the summary.

“Image not found. Build it first:”

Production could not pull the tag. Compare it with the Releasing shop@… line from step four. A tag typed from memory is the usual cause.

“Environment 'staging' not configured for project 'shop'.”

grim logs --remote reads only ~/.grim/servers.json, and this machine never registered staging. Releases and the env:* commands work from grim.json alone. For logs, work from the machine that ran step two.

Questions

Can I put a password in front of staging?

Not with grim server:auth. It takes no environment and acts on the one registered first, so on this project it would lock production. Keep staging on a name nobody guesses, or protect it inside the app.

Does staging get a copy of production's data?

No. shop_staging starts empty and the first deploy migrates it. Moving data between the two is your own dump and import.