Sometimes you're faced with a question that only has bad answers. A while ago one of my friends was faced with exactly one of those:

It turns out that the technology the team chose to use on the project was not scalable enough, 2 days before going into production.

They had several choices available:

1. Host the non scalable part on a separate server and hope for the best
2. Change the code behavior (requires lots of testing, in a rush, not a good idea!)
3. find a different technical solution (impossible at that time frame)

All these solutions suck. but there is more you can do:

• Consult with your whole team for other solutions. Explain them the situation and be open to hearing other solutions without taking them down.
• Consider each solution for the time it takes, amount of people needed and the risk involved
• Consider combining two or more solutions if it makes sense

Don't Panic

Whatever you do, especially if you're in a time rush - don't panic.

If you panic - chances are you will rush into a decision without thinking well about it, and might end up digging a bigger hole.

in this case going for option #1 and changing the code in parallel could be a better option than just choosing one of them.

I put out the fire - now what?

Now it's time to make sure you learn something from this horrible experience (and we've allbeen there) so that it does not recur.

Sit down with your team and try to gauge out the reasons why they think you came to this situation. For example

• "We could have done something about this 2 months ago but didn't"
• "We could have tested for scalability but didn't"
• "We didn't really understand that the requirement was scalability of this magnitude"
• "We wanted to do this early but never got to doing it due to lack of time"
• "It was lower on our priority list"

etc..

Make sure it does not turn into a blame fest - no names. always make sure people use the term "we" since as a team you are all responsible for this. If people try to be negative and say they had nothing to do with this failure remind them that as a team everyone is equally responsible to anything that comes out.

After you're done making that list, sit down with the team and make a list of things that you commit to start doing starting today. It will be your job as a team lead to make sure the team has the time and permission to achieve these things.

Try to not be the only one saying something. Always be the third or fourth person contributing to the ideas, to show people that you expect them to be part of the process.

Here is what such a list can be:

• "The team commit to doing performance testing of this application on a weekly basis"
• "The team will always ask about scalability requirements for all applications from now on, and are they a must to go live"

It could just be one item, but one that you can really commit to - make sure you are not lying to yourself.