Goals

Catch 95% of prod issues within 1 hour of deploying to production

A Prod Issues Prevention (PIP) test is completed by the ops assistant upon request during the deployment process

A full regression test is done at the beginning and end of each day by Operations

This manual flow will be eliminated by automated testing in the future

Agreements

Operations, Engineering, Product, stakeholders, and Exec team agree to the following:

All pushes to prod, will be PIP tested by Engineering and Operations

An OTA update is a push to prod

All pushes to prod will be between 9am-6pm MST for feature work

Only outside of work hours exceptions are prod issue fixes

If there are any other exceptions then you need product, engineering, operations sign off

The triage process will be followed for bug reports

A bug is anything that is not a prod issue

Deploy Workflow

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/48baa102-cfc7-4c15-9161-e4277eb5b73a/Screen_Shot_2021-04-30_at_11.57.49_AM.png

Pushing Code to Prod Expectations:

  1. Follow Deploy Workflow
  2. Join Queue
  3. When it’s your turn post that you are pushing to prod or staging