Devops recipe

I will try to explain what is my vision of the good way to organize a project following the Devops path.

  • Federate your team around the project goal and define measurable KPI to check you are on the right path to reach your objectives.
  • Define organize activities and how they are involve to reach your goal.
  • Define who is in charge of each activity and how the speak together.
  • Use the right tool for the right task to limit useless document writing and stupid actions.

Startup spirits

Devops approach is often associated with “Startup spirit” where small team does a lot of stuff together to achieve some amazing success stories. Usually developers think that in startup everything look easier than in big company full of process and constraints. Be the reality is a bit different; I have start to work on a startup 15 years ago. Yes communication between us was simple, the team was composed of 3 guys that do everything, architecture, design, development, exploitation, support. But I remember also the nights, weekend, holidays spent to work because we have an issue. And our startup failed, one of the problem was when we start to grow up we were not organize at all to work with more people, and our wonderful castle failed under is own weight.

Yes Startups have amazing success stories, but how much failed silently? Yes there are some good ideas to take from startup way of working, but just working with small team without organization is probably not one of them. The Devops approach is to analyze how reorganize the work on “big” structure to make it more efficient.

Is there a pilot on the activity?

A first requirement to implement Devops, is to understand the difference between an Activity and the Pilot of that activity. To have a service up and running you have a few jobs to do, you have to design the service, to develop it, to put it in production, and to keep it running. All those jobs have to be done, you can be alone to do all of them or have a full team of hundreds people each one in charge of a small part of each jobs. So each elementary job that has to be done is an Activity, people who execute an Activity are Pilot. An activity has input or requirement and produce output or delivery.

Two activities communicate together when one take as input the output of the other one. The protocol use to transfer the output to the input depends of the Pilot of each activity. If the pilot is the same for both activities, there is no need to formalize a lot the protocol. More the pilots are close to each other less you need to formalize the exchange protocol; the proximity will help people to work around the difficulties.

Don’t take the path of the dark side

So it could look easier and quicker to have team working under the same roof than a distribute team in different cities. But remember that searching the quicker and easier solution is the path to the dark side of the force. If you just guys under the same roof and let them work without organization, perhaps it will work for a time, like in startup, but it is not enough for a long tail solution.

Another current mistake is to aggregate the work of some pilots to reduce the weigh of the communication. For example asking the developer to deploy themselves their application on the production platform could look quicker and easier. But run team activities are not just to deploy stuff, they also have to keep it running. To do that they need to be aware of the change made on the platform, and how software work, if they are not in charge anymore of the deployment activities they will loose knowledge and will not anymore able to execute the other activities. And as for the “one roof” idea … you will fail.

One team, one project, one goal

What work well with small team or in startup, is not really that every one work under the same roof but that all people involve on the project are motivate to achieve the same objective. In sport most of the coach know that is better to have less skill players that play all together like one, rather having a full team of stars that try to make their own show. If development team has for objective to delivery a new function every two hour and run team was evaluate on the stability of the platform … that will not work well.

So the first thing to do is to define the main objective of your project that will allow you to federate all the people involve. But you also have to motivate them; so for that they will have to be able to check they are on the right path to achieve this goal. A good solution for that is to divide your main goal on smallest SMART objectives and to describe how those objectives are on the path to achieve the main one. Them you have to produce a dashboard to show the project is on the good path with the achievement of each measurable objective.

If your main objective is bullshit you can be sure that the project will produce a peace of shit!

The path of the activities start at the end

So now that we have a team federate around a shared goal, we have to see which activities you need to achieve your objectives. A good way to organize your project and to see how the activities work together is to start with the end. From your defined objectives, define the deliveries that you want to have produced at the end.

When you will have defined all the delivery involve reaching your goal, you can start to define which activities will produce them. With those activities you can define which inputs they need to execute their work. So now you have define a new list of deliveries.

And you continue recursively like that until your list contains only input not produce by your project.

Working that way, you know how each activity help to achieve your objective. If you do that on an existing project and you discover some activities not touched by this analyze you have to take time and ask your self: - Is this activity allowed to achieve an objective we have forgot? - Is this activity allowed to produce a needed delivery we have forgot? If the answer are no for both question, perhaps this activity is useless.

So at the end of this analyze, you have the list of the activities involve producing the deliveries you need to reach your objectives, and you know how they work together.

Pilot, Communication and Noise

Now you have shared goals and you have organized activities, you have to define who on your team will do what: who is the pilot of each activity. One way is to think “agile”. You have a team federate around a shared goal. Every man in this team has capabilities and expertise, so let them do what they think will be the most useful use of their knowledge for the project. Problem is that we are not in the Care Bears world. There is always some activity and port of activity work that is less funny. If someone has two tasks to do, he will do first the funny one. The worst case is the man who just gathers on activity doing just funny part of each one but never finishing the work.

So as often, the good path is not the one that seems easiest and quicker. You have to define clearly who is the pilot of each activity. Of course you have to take care of what they want, but you have to define who is in charge of each activity and what are its personals objectives and how they help reaching the project one.

But the problem is still how to regroup project activities to allocate them to a pilot. Two activities communicate together using deliveries. The output of one became part of the input of the other one. Each time one pilot delivers data to another one; the data have to be translating with possible loss of information and time. So first analyze is to check the cost of each translation and to try to regroup activities that help to reduce this cost. For example, development activity will deliver the software to packaging activity that will deliver it to deployment activity. If you allocate packaging activity to system integration; developer have to explain to system integrator how to deploy the software, then integrator have to explain to operator how to do it. If developer does packaging activity we just keep one translation.

But you have to be careful; some translation just adds noise, other one helps to improve the information. For example if the “customer” has the capabilities to do software architecture we could avoid a translation if we let him do it. But the activity of translating requirements into software architecture is not just a change of format. This activity is a negotiation between marketing requirements and developer constraints. If you are alone it’s difficult to take care of both side needs.

So you have to think how to reduce the noise on the data chain without altering the improvement of the information.

There is only one Lord of the Ring, only one who can bend it to his will.

An easy way to try to reduce noise is to use only one tool to share information between all project activities. If you ask good craftsman they always tell you to use the right tool for each job. Handymen that search on TV the miraculous tool that do everything are often more a “weekend Mr Fix-it” than a craftsman. Another way to said it it’s: “When you have a hammer in your hand, everything looks like a nail”. When you work on relational databases or REST API that have to be use by different kind of application, you know there is a big difference between the structure of the data and the way you present it to the client. Each activity has its own requirement on what information they need and how to display it. For example when communication team prepare events that will involve the platform of the project it could be useful for them to manage them with a kanban view, but for operational team they will need to show this events in calendar view to know when they can execute maintenance operations with low impact.

So the good way is to put the information in a shared information system having different tools to display the information for each activity. A good way to limit noisy translation is to only fill one time each data and to use application to display it the best way for the current activity.

Each time you use “paper document” you loose money

Each you use document like Word or Excel to exchange information between activities of your project, you loose money. For example if four activities produce a document with information another activity has to compute to produce a calculate view of those data. If you do it manually:

  • Errors could occurs
  • It introduce delay to produce the data
  • If formula change you have to update or forgot all previous produce view

If instead of a “paper document” you have fill those information in a SI, you can produce the views you want and dispose the right information immediately.

But you can think that managing those kind of SI cost a lot. It probably was right few years ago. It not true now. For example you can use oauth2.0 mechanism for managing user access to your SI, develop REST API to manage data using node.js and loopback framework to create quickly API from database model and use stuff like AngularJS to develop the UI.

The idea is not to develop perfect software you can put in the hands of every kind of users. The idea is to create a tool useful for your project and company. The great things working with API is that can change the server implementation and keeping the client UI or create new UI using same API. You can “prototype” whatever you want and industrialize the good part later.

When you have identity solution that allow to manage user, groups and role, creating an API to manage data and develop the UI could take less than 2 weeks, how many it will cost you to don’t do it?

Devops recipe

  • Federate your team around the project goal and define measurable KPI to check you are on the right path to reach your objectives.
  • Define organize activities and how they are involve to reach your goal.
  • Define who is in charge of each activity and how the speak together.
  • Use the right tool for the right task to limit useless document writing and stupid actions.