Production Outage Post-Mortem

Brittany: Hi team. We had a major outage today. The production database went down at 2:00 PM.
Steve: Yes, I noticed the downtime while I was testing the new UI. It was very slow.
Tom: I investigated the code, but I did not find the root cause of the crash.
Brittany: Let's look at the steps. First, our system administrator updated the server. Then, the database crashed because the programming language version was too old.
Steve: Although we hired a new developer last week, they did not touch this server.
Tom: Wait, I remember. Our intern, who started his internship yesterday, ran a heavy script on production.
Brittany: Ah! When he ran the script, did he fail to close the connection?
Steve: Yes, he did. I just discovered his active sessions on the server.
Tom: Okay, we have a clear clue. I tried to kill the sessions, and the database succeeded to restart.
Brittany: Great. So the mystery is solved. No more downtime!

Read again and choose the right answer

When did the production database go down?

• At 2:00 PM today.
• Yesterday morning.
• Last week on Friday.

What was Steve doing when he noticed the downtime?

• He was writing a SQL script.
• He was testing the new UI.
• He was hiring a new developer.

Who ran the heavy script on the production server?

• The system administrator.
• The new developer.
• The intern who started his internship yesterday.

How did Tom resolve the database downtime?

• He killed the active sessions of the intern.
• He updated the programming language.
• He changed the database provider.

Sources: photo by Mikhail Nilov on Pexels