Deploying in weekend or at night is a terrible idea in disguise of a good idea. What we used to do are:

- No deployment on Weekend

- No deployment on Friday

- No deployment after 4pm on Monday to Thursday

- Deployment is rolled out in stage: one server, 5%, 10%, 50%, 100% of servers.

- Rollback steps must be accompanied with deployment steps.

- Verification steps must be specified in the deployment ticket. Verification is done by QA or OPS, other than Dev.

- Common deployment and rollback steps are automated.

- Emergency deployment is an exception to the above but must take extra precaution to babysit the deployment process.

Stress level has gone down a lot and problems are resolved much faster once we have the above.