The Elastic Leadership Book

 

 

        RSS Feed

Latest Entries


First time here?


Wednesday
Feb222017

A Critical Chain of Bus Factors

I've spoken before about the meaning of a bus factor. It's basically the idea that if there is only one person in your team that can do a critical part of the work, and that person gets hit by a bus, the team stops functioning.

(I've been asked to call this the "Lottery" factor... which is a nice twist on it).

It's not just developers who are susceptible to this. In any team, IT teams in particular, this seems to happen very commonly. In larger organizations, you might even have an infrastructure group and an "IT" group, and they both have different areas of responsibility, and in each group, there's only one person who can do some critical task. 

 

A chain of bus factors happens when you have bus factors depending on bus factors:

Your one developer who knows how to configure the pipeline can't test the changes because the agent is down. The one guy in IT who has access to the agent needs to reboot it, but does not have access. The one person who has access to reboot it (in the Infra team) is sick, so now there are three people waiting, and there is nothing in this earth that can help that situation. 

Here is another example of a critical bus factor chain:

  • Developer can't check in the one feature only they can code so the application is released,
  • because the source control is down, and the source control person can't fix it,
  • because the private cloud node is down, and the one person who can fix it, can't, 
  • because the certificate is expired, and the one person who can fix it, can't,
  • because the person who acquires certificates is away etc...

 

Steps to avoid this hell on earth may include:

  • Giving more access to other groups for the same resource
  • Sharing knowledge and responsibility with other people
  • pairing while working on critical things so knowledge sharing is guaranteed
  • Create a company policy of (2 bus factors or more) required.

 

Look around you at work right now, and realize how many bus factors you see, and are a part of. Each one slows down the system more and more. Each one causes the system to be more and more fragile. 

When you are a bus factor you might think:

"I need to work only on this one thing, otherwise I'll have too much to do, and so will everyone else on my team".

But realize that you are also creating more work for yourself this way:

Whenever someone needs work done on this topic, you will be the one person they turn to. You will be the pain point for every project that needs this type of worl from now on.

Sharing the burden also means alleviating the load, and letting more people finish their work without needing you.

Yes, there is a balance to be had, but remember that balance means not only "I won't give admin rights to the whole world" but it also means "I won't be the only one with the ability to do something".

How much time, money, pain and sufferingin the workplace is caused by people just waiting for people who are waiting for people who are waiting for people?

The answer will shock you! (OK, bad ending, but hey, great punch lines are not my job...)

 

Wednesday
Feb222017

Elastic Leadership Book Published

I'm happy (if a little late) to annoucne that "Elastic Leadership" has now been officially published by Manning.

You can grab the book here

Wednesday
Oct212015

Elastic Leadership Book: The re-publishing of "Notes to a Software Team Leader"

I'm happy to announce that I am re-publishing the "Notes to a Software Team Leader" as a new book under Manning publishing, re-titled "Elastic Leadership". 

The new book will contain a couple of new chapters, regrouping the "notes" chapters under common themes, and extra annotations from me on the external notes, to add where I think they fit into the elastic leadership framework.

The new chapters I will be adding:

 

 

I am thankful to Manning for publishing my second book after "Art of Unit Testing" 1st and 2nd editions, and hope that the book will end up in the right hands of the right readers who need it most.

Friday
Oct162015

Team Bus Factors: How to Reduce Them and How to Prevent Them

There was a discussion on HN asking about good tips for team leaders. So I thought I'd chime in a bit more deeply.

The following is a new chapter from the upcoming second edition of my book for software team leaders 

They’re all around you, and they are huge risks to your projects. Bus factors.

A bus factor can be defined as “How many people need to get hit by a bus for the project or team to stop functioning”.  A bus factor of 1 is the most risky.

If you’ve worked in software any length of time and I’d ask you “Do you know a person in your project that, if they had disappeared tomorrow, the project or team would get stuck?” it is very very likely that you can provide me with multiple names. 

These people (roles, really) are what I’ll call “bus factors” from now on. I had a friend who joked that “every successful software company is hiding one “Yuri” in the basement that does all the important stuff"..

Joking aside,  “Yuri” represents talent, a specialist, acquired from overseas perhaps, that is hard to reproduce. Many companies indeed do have multiple “Yuri”s.

And they are a huge risk.

Let’s break down some of the reasons they are a risk:

-       Having a single point of failure

-       Having a bottleneck that slows things to a crawl

-       Reducing moral/Avoiding moral boosting/job insecurity

-       Avoiding team growth

 

  • ·      Having a single point of failure

I once consulted at a large insurance provider One of the projects, with about 200 developers, had just one build consultant who had been working there for several years, but was still an outside consultant.  His job was to take care of the build and release. A release took a week. There was one release every month.

Everyone stayed out of each other’s way, and he mind his own business as well.

One day he got tired of working at the same place and same project and transferred to a different consulting gig at another place.The project (the copany’s main source of revenue) was unable to release software, at all.No bug fixes to production, no new features seeing the light of day. Nothing.

Huge money loss, as well as possible new customer loss. Everyone was running around like headless chickens, trying to fix this. Eventually they called the build consultant again, paid him 3 times his usual hourly to sit there for a couple of weeks, while they attached a mic and screen recording software to his machine, while he was doing two releases (fake and real one). They paid dearly for getting rid of the bus factor.

Lesson: The less people you have that know an important part of your business, the more you will pay when they move on.

  • ·      Having a bottleneck that slows things to a crawl

One of the large projects I consulted for had a huge problem with code quality and slow release cycles. When I paired with developers several things were very clear.

One developer that I paired with was working on a long function that was very unreadable and unmaintainable. He was just adding spaghetti code onto more spaghetti code.

When I asked him why he wouldn’t refactor his new code a little bit or just extract some stuff to an external method, his reply was “That’d be crazy. If I do that, I’m going to have to ask for a new code review for that code section, and those things take days until they're done by an architect.

There were only two architects that were allowed to do code reviews, and they were busy writing new code most of the time. So they had little time to review code changes and approve them. So getting code into even a regular dev branch would take days, which slowed the place down to a crawl.

It is harder to keep up with fixes, add changes quickly or even show demos and get feedback fast enough in a sprint cycle.

If you’ve read about the “Theory of Constraints”, then a bus factor is a type of a constraint, and thus everything will move as slowly as the slowest constraint.

 ·      Job insecurity/Reducing moral/Avoiding moral boosting

There’s also negative consequence to the bus factor person themselves.

I once consulted at a large software/hardware firm. The build process was managed by three people and they were very defensive about teaching other people how to use the build process (we’ll talk about job security in the next point regarding this).

One of them was adamantly against showing other people how the build worked, because he felt that this was his job. He was the specialist with 20 years of training on the job.

It can be said that being the only expert at something is a good way to have job security, but it is is very much a double edges sword.

An organization will usually seek to reduce the bus factor risk. A person who is a bus factor and avoid reducing the factor will usually have to take a forceful stand against the organization, effectively holding the project or team hostage since no one else can fill the role.

It is very likely for the organization to try to find the first “match” to come along to replace or augment that person’s job to reduce the bus factor. The more forceful he was originally, the less chance he has of staying onwards when someone else with the same or better skill set comes along.

It’s the same people hate buying from monopolies, and can’t way to jump ship with an adequate competitor comes along, even if it is just on principle.

 

 

  • ·      Preventing team growth

 

I’ve seen many people quit their jobs because they were’t learning anything new in them They got bored of doing the same work, of not being challenged, and they moved on to more challenging pastures.

 Bus factors are knowledge silos by definition. Knowledge silos that are not broken can lead to the “not my job” mentality (Go yourself a favor and google “No my Job Award” and see extreme versions of that mentality when it gets perpetuated by the organization itself.)

In that mentality, even people who want to learn and be challenged are going to have a hard time finding new challenges since they are expected to stay in their own little cube. That , in turn, makes for a team of specialists, or, to put it more bluntly, a team of bus factors.

One of the places I was consulting for once had an issue: the search server the product was using was failing and since it was setup by an external company, nobody on the team knew how to fix it. The people form the external consultancy were on a company vacation or something of that sort, and it took three days to fix.

Here’s how it got fixed: A developer called the expert on the phone,The expert told the developer exactly what to type into the terminal.The developer blindly typed it into the terminal , waited, then said “ok it seems to work now” and hung up. 

What a wasted opportunity to learn something that might really help solve this problem next time. Not even a “why did that command fix it?” or “Why did it fail? What can we do next time on our own to prevent this?”. Nothing.

This is what that mentality begets: lots of lost time, and people putting cubicles inside their minds, preventing learning. 

Removing Bus Factors

There are several ways that I’ve used in the past to remove existing bus factors. Some of them are slower, but more effective in the long run. Others are faster but are a bit less effective in the long run.

 -       Pairing

Ask the bus factor to pair up for at least 30 minutes a day with one other people in the team, and during that time, the less experiences person will do most of the hands on work, with the bus factor sitting next to them coaching and explaining what to do next.

Do this until the less experienced person knows how to accomplish one task without the bus factor’s help, then either move to a new task, or have the newly minted person stop pairing with the bus factor, and pair with some other member of the team to achieve the same goals.

The best way to learn is to teach, and by teaching, the new person will have learned the new task in a much deeper way.

 -       Make them in charge of teaching

Have the bus factor be in charge of a project that requires multiple people to accomplish tasks relating to the bus factor’s area of knowledge. Then make sure part of that project is the bus factor teaching others on how to accomplish this.

 -       Prevent them from working on area of knowledge

Ask the bus factor to not work hands on in their  area of knowledge for a day a week, while others try to take over. This might feel scary but it is a great way to get up and running fast.

 -       Apprentices

Assign a full time apprentice to that expert and make sure they pair as much as possible.

 Avoiding future bus factors

 You can’t always avoid bus factors from sprouting, but you can minimize the likelihood of it happening by:

-       Pairing

The more pairing your team does, the less likely it is that only one person knows how something works.

-       1-1 code reviews

it’s almost as good as pairing: Each code checkin gets personally reviews locally or via remove call (skype etc) with at least audio. This provides learning since it is meant to be a conversation.

-       Rotation (support, scrum master, build..)

Set a daily, weekly or bi weekly rotation on tasks that are bus factors or with new tasks to prevent them from being bus factors.

 -       Pushing people out of their comfort zone instead of asking the veterans to do it

If a new task comes up, select the person on your team with the least skills to accomplish it (assuming it is not high risk). If it is high risk, save that thought for a low risk task (don’t tell me everything is high risk!)

 Next, we’ll discuss survival mode, which , if you don’t take care of your bus factors, you can easily fall into.

 

Thursday
Jul092015

A Realistic HackerRank Developer Interview Question List

 

 

When I was looking for a new job it always puzzled me that companies looking for senior developers, team leaders, architects and Dev managers were asking me to fill out silly hackerrank style questions such as :

* How to effectively measure the diameter of a tree

* How to compute Prime factorials

* If two eggs are all you have and you have 100 story building...

Unless you are specifically looking for an algorithm related developers, can we stop this lunacy?

This is not the stuff 99.9% of developer and tech leads have to deal with in real life. Here are some questions that will tell you if that person can really help in your team. Unfortunately it's hard to "measure" the answers. But I'll try.

 

 

If I were building a "hackerrank" website:

 

1. You have a team of 12 people spread out across three different locations. It takes an hour to get to one location (A) from your house. 90 minutes (without traffic) to get to the second location (B) , and 3 hours (no traffic) to get to the third location (C). Each person in the team seems to be over committed: they are behind on things they haven't even started yet.

The people in site A have knowledge abouttechnology T and T2. The peoplein site B have 1 person who knows about technology T and all the rest know about a technology T1. The people in site C know about technology T but not about T1. 

There are 7 people in site A, 6 people in site B and 2 people in site C.

The project manager for the highest priority project is hounding you because a project with technology T is 2 months late. However, all the people who know  technology T are already busy working on solving this and everyone else can't help you from your group.

What is likely the root cause? what can you?

A. Root is cause is the project is demanding things that are unreasonable. Plow onwards and hope for the best.

B. Root cause is project is demanding things that are unreasonable. Tell that to the project leader and your manager. 

C. Root cause is there isn't enough cross training. Plow through and do some cross training when the fire is out.

D. Root is cause id there isn't enough cross training. Stop all other work by your group and do cross training so people can help the folks in site A get the job done.

E. A and D

F. B and D

G. Other (open ended)

 

 

2. The project manager is complaining about the amount of of bugs found in production, and you know that your code quality sucks (you just joined and took a look at the code). What questions must you be able to answer before starting to fix this?

A.  What's our deadline(s)?

B.  What other commitments do we have?

C. Why are people writing bad code?

D. Is bad code really the problem?

E. What's our code coverage?

F. B C and D

G. A D and D

H. B D and E

I. all of the bove

J. none of the above

K. Other (open ended)

 

3. How would you measure your success as a tech leader?

 

A. The quality of the code my people write

B. Our code works in production

C Our code works in production and is maintainable

D We live up to the targets provided

E It's fun to work in our team

F The project manager likes our outputs

G I am not a bottleneck for making decisions

 

 

4. You have a developer that no one wants to interact with because they are very aggressive. What do you do?

A. Fire them. No asshole rule applies.

B. Coach them for a few months and get them off the team if it doesn't work out.

C. Have 1-1 and tell them to behave better.

D. Ask the people that complained to talk to your developer about his/her behavior

E. other (open ended)

 

 

5. A developer sends you an email showing that he's being requested by various stakeholders to work on a project that is not top priority. What do you do?

 

A. Send a stern email to the stakeholders with CC to my manager

B. Ask developer to send email to stakeholders with CC to me, with the promise I will back him/her up with what he thinks is the right thing to do

C. Tell developer to find a way to support this in spite of time pressure

D. Tell developer to ignore that email

E. Find another developer to support this activity. If none are available use A or B

F. Find another developer to support this activity. If none are available use C or D

 

 

6. Which of the following design patterns would make it harder for you to follow SOLID design rules? Select all that apply.

A. Singleton

B. Chain Of Responsibility

C Composite

D Decorator

E (more pattern names here)

 

7. Write down as many uses for a blanket that you can think of within the next 60 seconds. Think out of the box.

 

 


Can you guess why I'm asking the last one?

 

 

 

I want to see how creative the person is. Less creative people might come up with 5 or 10. The really creative ones might find up to 20 in a minute. If you give them 5 minutes you might get 50 uses or more. Less creative folks might give up after 10 or less.