RSS Feed

Latest Entries


First time here?


Thursday
Mar272014

Signs you might be in survival mode

We had an interesting discussion in class on how one can detect you might be in survival mode. Here are some possible early warning signs:

 

  1. There is a perception or feeling of continuous urgency
  2. There is lack of context (why are we doing this? where are we heading? When is this expected? what are the expectations?)
  3. The team is forced into a state where they are always reacting to changes (key word here is forced because a self organizing team can choose to simply react with changes such as during  a continuous delivery flow). Not being able to choose whether to plan some things out.
  4. There is no time to reduce risk (technical debt, bus factors etc)
  5. The team is unable to deliver
  6. There is only time to take shortcuts
  7. A long period of stress
  8. Being non productive for a long period of time

 

Sunday
Nov032013

Technical Disobedience

Rachel's Suffering

In one of my first jobs as a programmer I joined a team working on a governmental project. The team was sitting at the code shop, not at the governmental place.

The coding shop I was working for as a coder at that time had some really good people working there. Many of them were suffering silently.

I remember this one specific coder working there. I'll call her Rachel because that's not her real name. Rachel was in her 40s, and she was an expert database person. She mastered SQL syntax, stored procedure optimizations, index optimizations and partitioning like nobody else at that office.

I remember the first time I came up to Rachel's desk to ask her something. She was hunched over the screen, like Golum staring at his precious ring. Her nose almost touching the screen.

I turned my face at the screen to see what was so interesting. For half a second, I didn't understand what I was seeing. It looked like very small, dead black ants, on a white background, all over her screen.

I squinted (I must have been 25 at the time) to try and see better. These were words.

"SELECT * FROM…"

There were well over a hundred lines of SQL code, in a tiny, oh so tiny font, all made to fit the small 15" screen.

"What the…" I muttered. Rachel smiled with pride. "Yeah, I know". She was proud of having been able to work on such a small screen with such a large amount of data.

She accepted the smallness of the screen as a god given commandment, and was just trying to do her best. She wasn't unhappy, or at least unaware of how unhappy she was.

I could not understand what she was smiling about. She has been working there for at least five years before I stepped through the door, and I tried to imagine her life as a developer, hunched over the code, trying to get just enough real-estate to be able to do their job. Focusing her eyes so many hours a day at such small text must not have made much good.

It was madness.

I looked around, and everyone around me had 15" monitors. A 15 inch monitor was not unheard of at the time, but most developers had at home nothing less than a 17" monitor, and some had 19". These were the old, bloated CRT screens, not the nice thin ones like we have today.

Breaking the Screen Rules

I had a 17" screen. Well, not at first. I started out with a 15" screen, but having had a 17" screen at home, I knew I couldn't go back to working on a 15" for long.

I asked my team leader if I could get a 17" screen. I heard back that there was a line of seniority to receive the screens. There were at least six people ahead of me. Rachel was number 8 on that list. New screen would arrive only when an older screen had broke, which was every couple of months.

I couldn't stand it. I just couldn't accept it. I had just finished reading "Mythical Man Month" and "The Pragmatic Programmer" and was all riled up about having good developer environments and tools that help rather than hurt.

So I took my 17" screen from home and brought it to work. I would work at home with a 15" and work at work with 17". So one morning I just showed up at the office with a big box containing a 17" screen and connected it to my computer. As I walked down the corridor I had to explain to everyone that I got it from home, so that nobody will feel like I cheated my way up to a 17" screen. How silly is it to have a screen size measure your "seniority" anyway?

I was cheating, though. I didn't go by the rules. I went around the rules, and made up my own rules. Nobody got hurt. But I got to work in an environment that was better than I had, and it helped me do better at my work.

I didn't care that it wasn't "my job" to take care of the tooling I was working on. Because it is my job to worry about the tools I work with.

If you don't worry about your technical environment, nobody else won't either, probably.

A year later I got my new screen, but it was old news. I had already been working with such a screen for a year, so I gave up my turn and gave it to the next in line after me.

The Creeping Whiteboard

As I reach back into my mind, I had behaved like this in other technical projects. Always trying to find a way to shorten, simplify or simply work around things that bother me.

In a recent (again, governmental) project, we had to use TFS to manage the tasks. It was a team of 7 developers. And the testers were using Mercury tools to manage their own tasks.

I was coaching the Scrum Master at the time and suggested that we also use a whiteboard to "you know, manage just the weekly tasks". This task board became more and more sophisticated, and the product owner liked it as well.

Soon, all work would revolve around the task board and its notes. People started to forget to put tasks in TFS, and nothing happened.

When I left the project six months later, the product owner asked me:

"Hey, am I crazy, or do we not really need TFS to manage our tasks anymore? we seem to be doing just fine with the whiteboard."

"Exactly" - I replied.

Win for common sense over company bureaucracy. The only reason this worked is because people felt it working. It's not enough to just discuss something and try to convince people. Show them what it feels like.

Technical Disobedience

I never had a name for this behavior that I seem to be repeating over and over again at various places. I do know that it helps move things and get shit done.

I know that I teach it in my book for software team leaders, to never accept the status quo for what it is and to realize you always have things you can do.

Then the phrase "Technical Disobedience" finally clicked into place when I saw this short film about Cuba and the "Technological Disobedience" the cubans had to come up with to survive in a society that had choked out any input of mechanical parts, new technology and new science.

The cubans had to deal with being in a very closed bubble, and had to make their own luck and their own innovations with the things they already had. So they reinvented old ideas, and gave them a twist. Old ventilator engines became bicycle engines, freezer engines. Cars and Bicycle would be combines to create half breed mutants that did what they couldn't do with the things they already had. get around.

And what about us developers?

We take so many things for granted, when we could be going so much farther and faster with just a little more courage to break and remake the small rules that govern our technical life.

We are all locked in our own little technical world; A technical kingdom with DOs and DON'Ts. From startups to "enterprise software houses" — we are all bound by some rules.

Those rules can be as simple as "we don't have the budget to do this", as annoying as "approving a budget for this will take three months", as frustrating as being unable to have test or staging environments due to oblivious "security restrictions" and as down right stupid as "we'd rather spend $20,000 on the wrong thing rather than get the right thing for $1000 because that's where the budgets are".

If you just accept it, you're stuck just like everybody else. If you realize that you can actually do small hacks in the way the system works, you can get things done that none in that organization ever thought possible.

Sometimes you just have to say "screw it" and try out something.
It's better to try than to continue to fail and sometimes it's better to ask forgiveness than permission.

It's actually pretty easy once you try it. Here are some other examples.

The Test Machine(s)

A project I was consulting for wanted to get things just right. The team even got an OK from management to try this new Kanban and Continuous Integration they have been hearing so much about.

So I introduced the team to the idea of Shipping Skeleton. You start with a shipping "hello world" application that gets deployed to test, staging and production with the click of a button, while also passing all the tests. then you start working on the actual code.

Great idea? Only in theory for that team. They tried, but they didn't even have test machines to deploy to and show a demo. To get new virtual test machines would take a month, and they wanted to start as quickly as possible without compromising on their new values. You had to go through a whole system of things to get something as precious as that, and the people responsible were always so damn busy, and in a totally different country.

This is a classic case of management saying one thing, and the system fully designed to say a totally different thing ("screw you, you ain't going nowhere so quickly").

So I went to management as asked their permission to stop using the company that was providing the virtualization services for that team. They were happy and said "go ahead. try other things. it will give us leverage with that 3rd party company come negotiations day. " So I urged the team to try and look into amazon EC2 or Azure machines. No such luck. Security would not let something as simple as that work. Too many loopholes.

So I went to master architect at the company asked him if there are any machines lying around that we could use for internal test and demo machines. He pointed me at the machine we had then used to show demos on the big screen TV, and that as long as it is not in use, it should be ok to use it. Bam, we had a machine under our control that we could at least do automated deploys and demos on. Better than nothing.

The unplanned team room and hardware

Me and a team once took over a non used project room along with a bunch of non used machines from an abandoned project. We had 4 test machines for load testing. it was a good project.

Summary

In software, it is far to easy to accept the system around you as status quo. It is harder to realize that software is soft, and that systems can be soft as well. There are many small rules you can try and hack around to get things done.

Innovation is part of your job, even if the system directs you otherwise. If you are not going to quit your job, do what you can to innovate in your own boundaries, and get things done by changing the rules just enough. That is why I write about continuous experimentation as part of your role as a team leader in the team leader manifesto.

Sunday
Oct202013

Definition: Survival Mode

A team is in "Survival" mode if they do not have time to learn and slowly practice new skills. Such a team is usually over committed and is too busy chasing fires instead of having enough time to hone and practice the skills that prevent the root cause of the fires in the first place.

For example, a team may be too busy fixing all the bugs that keep popping up, that they do not have time to learn and slowly practice design,  refactoring and automated testing of the code to prevent future occurrences.

Another example: A team is too busy deploying things manually and preparing for a release instead of spending the time to create a release and build chain.

Survival mode is usually patterned as a downward spiral. The less time you have to fix the root causes, the more time you will need to fix the fires that are caused, and then you have even less time to fix the root causes.

To get out of survival mode the team needs to:

  1. Remove over commitments
  2. Change current estimates to include quality in the estimates (usually an increase estimates on current work by 100%-500% is needed)

By taking these actions you can create a positive loop in which you fix root causes and then have less fires to fix so you have more time to fix more root causes etc.

Friday
Sep062013

Two Techniques to Avoid the Bus Factor In your Teams: Push and Pull

I spoke before on this blog about BUS FACTORS and how to get rid of them. A bus factor is calculated the the amount of people on your team or project who need to get hit by a bus before the project comes to a halt. There is something that only THEY know how to do.

I am revisiting this because there are many ways to get rid of it. Today I cover a couple. But first, a true story.

The Robot

In one of the projects I was consulting for a few years back, we had four teams and many bus factors relating to many things. One of them was related to the search technology the project was using at the time (sharepoint search…shudder!)

One day I saw the architect* running around like a headless chicken trying to figure out how to fix a problem. I asked him what’s wrong and he said “the search guy is out on vacation this week and not answering his phone. I have no idea how to fix this”.

It took a couple of hours until he found an external company that once provided these services to the project, and got them on the phone to ask if they knew how to fix this problem. Of course they did. They coached him on the phone what to type to reset some indexes.

I was sitting in the next seat listening to him typing word by word into the terminal, pressing enter, and then saying “ok now it works thanks”.

Then he hung up.

What a great loss of knowledge! Instead of using the opportunity to LEARN something about the problem, its causes and WHY what he typed fixed it, he simply made himself a remote controlled robot for the person on the other side of the phone.

I coached him to do the following next time:

When you find the situation has been solved, ask the following questions on the phone:

  •  
    • What exactly was the problem? (tag this type of problem in your mind. like a design patterns, we can have “problem patterns”)
    • What made you realize that was the problem? if you indeed realized it. (learn their thought process. learn to think like them)
    • What did the actions we just took mean? where can I learn the full syntax of those commands? (learn the solution pattern)
    • Who else around me knows what you just told me? (see how bad the bus factor really is. Maybe you just did not know who to ask? maybe you are the ONLY one who does not know how to do this?)

 

Pull Based Learning

What I coached him to do was what I call “PULL Based Learning”. Before I discuss that, let’s talk about why it is important to learn this pattern.

One of the best ways to remove bus factors, is by pairing. Pairing is an act of active learning where knowledge is constantly back and forth. A person asks a question “How did you do that?” . that is an act of deliberate learning.

But pairing is also hard to achieve in a culture where pairing was never a “thing” people do.  If you cannot get enough people to pair, or the bus factor is happening when a person from a DIFFERENT team knows something that your team replies on, it’s time to start encouraging active knowledge gathering, or active learning.

Active learning comes in two forms, or two directions:

  • Pull Based Learning: The person who does not know X asks another person who knows X to teach them about X in some way. They initiate a PULL of the information from the other person.
  • Push Based Learning: The person who knows about X asks a person who does not know about X to join them to learn about X. They initiate a PUSH of information towards the other person.

 

So pulling or pushing depends on who initiated the knowledge transfer. If you find that you have many bus factors, it is important that you teach and coach people how to work in BOTH directions:

  • When people who need to learn are too shy to ask for help, coach the more experienced people how to PUSH knowledge by asking others to join them in a task that will teach that person something new.
  • When the people who are the bus factors do not have many opportunities to share knowledge** coach the people with less experience to jump in when they see the bus factor doing something the less experienced person does not know how to do that affects them, and ask to join in for a few minutes to see how things are done.

Get Permission

This also means you might need to get an upper level manager to send an email or have a group meeting letting all developers know that “it is OK to do pairing,  ask for help, request someone to join you to learn new things – even if it means tasks take longer than estimated. it is OK to estimate tasks longer because you want to build these things into your iterations”.

In cultures where hierarchy matters a lot, this could be the beginning of the breakthrough you might be looking for – giving people permission to learn and teach is usually the thing most cited (by people I ask) as the reason why they are not doing it.

Bi-Directional Knowledge Handshake

Having only a single direction knowledge sharing culture is very ineffective.

In a project full of consultants, who love teaching, but also full of people who hate learning, knowledge will not be shared very often, and if so, with a tea spoon instead of a fire hose (which is usually what you need to start with in extreme cases)

In a project full of people wanting to learn, but having all the people with the knowledge be in a different building, company or country, mostly unavailable, knowledge will not be shared.

There has to be a knowledge HANDSHAKE for knowledge to actually flow through. otherwise we are talking about tear drops in the ocean.

 

Notes:

* Remember that having a non coaching architect is having a bottleneck

** Maybe you should think about increasing the amount of opportunities?

Thursday
Sep052013

What's your Position, Velocity, and Acceleration? by Neal Tibrewala

About Neal: Neal Tibrewala is the head of software development at L5 Software Group in Austin, TX

As a software consultant for 25 years, the thing that I've found the most useful is measuring to figure out where you are and where you're going as a team.

Let's take something like the number of known defects in a product. Most software projects track their defects using some kind of defect management tool.  Therefore, a team could know that they have, say 100 known issues with a certain product.   This is their "position".

Is this good?  Is this bad?  Is action required to address this fact? Often, someone will make a gut call at this point and say that it is either "fine" or "bad" and try to make a change.  I would argue that change is premature at this point.  As a team, having 10 defects or 100 doesn't tell you if you're in trouble.

Smarter teams will say:

"Aha!  We should track our defect rate instead!"  

Indeed, tracking your defect rate is the next step.  Take a certain fixed time period, say a scrum iteration, and track the number of defects fixed and new ones discovered to see the total number of open defects at the end of each iteration.  This is called the team's velocity.  In the example above, the team with only 10 defects could be accumulating new ones at a rate of 10 per iteration, and the team with 100 could be decreasing them at a rate of 10 per iteration.

Clearly that means the team that's bringing defects down instead of up is the team on the better course, right?   Maybe not.

The final step is to track your acceleration.  This is looking how the defect rate changes over time.  The team with a defect rate of +10 per iteration could be learning, thus, the following iteration be at +8 defects, then only +6, etc.  While the team at -10 per iteration could be slowing down, thus be fixing 8, then only 6, then -4, etc.  

This number is either plus or minus 2 defects per iteration per iteration (2 defects/iteration^2).  This is the team's acceleration, and this is the number that should be looked at most by the team's leader.

Learning will always impact position and velocity.  Spending time learning will slow a team down in the short term, but acceleration, measured over a long period of time, is the surest sign of a team that's learning or not.

A team that's not only able to adjust their position, but also control their velocity, is the kind of team that you most want to encourage. Changes that affect acceleration are powerful tools as they can have drastic impact on velocities over time.  Likewise, they can serve as early warning indicators as slowing velocity can be seen far before a project gets into visible trouble.