Integration
people over process in modern software development
In January 2019, Chris Coyier released the article The Great Divide.
He shows that we are facing some challenges by introducing new disciplines and sub-disciplines to the field of software development.
What do you need to learn, to call yourself a frontend developer? What even is frontend-development?
The difference between "front of the frontend" (UX, UI-Design, a11y) and "back of the frontend" (programming) shows a divide - that is not without cost.
A cost that needs to be addressed: with every specialization comes the need for integration.
I would argue, that this is mostly overlooked and it creates one of the biggest problems in modern software development.
In this article, i will describe a systemic problem at the very core of how we work,
and show a solution to resolve this problem - something you can use today.
System And Parts
A way to look at our situation, is through the lens of Systems Thinking.
Systems Thinking was pioneered by the work of Russell Ackoff.
We are dividing software development into parts (specializations). The assumption is, that if we improve every part separately, software development as a whole will improve. But this divide & conquer strategy is clarly not working as expected. Software development as a whole, can't be divided into independent parts. The parts are all interconnected and improving a part without improving the whole, will make the software development worse.
Example
When an architect designs a house, he can't design the rooms taken separately.
The rooms are all interconnected, and if he wants to make the bathroom bigger, he knows that this
has an impact on other rooms and the structure of the house itself.
For the best possible house, no room might be the best possible room.
Everything needs to be evaluated against the improvement of the whole, the interconnection of all parts.
Coordination
How can we be interconnected?
If we know what to build, with a clear plan from start to finish, we can assign work to every worker and coordinate how the finished work will be put together.
This might work for construction - but not so well for software development. In software development, we face a different world. A construct of thought and ideas, that is way more dynamic than a construction-plan and building materials.
Because of this, software development it is not only about doing things right - and on time. It is also about an ongoing quest to find out if we are doing the right thing. If our assumptions are correct. Otherwise we will still deliver results, but without value. Software without value.
To be interconnected, we need to add a layer on top of coordination.
This layer serves as a platform where we can share our understanding and check our assumptions on a human level.
A platform for human interactions.
agile
This is why we see the rise of agile software development.
At least, this is what i assumed.
"individuals and interactions over processes and tools" - sounds great, but what happened?
Something went wrong with agile. And i can tell you that James Coplien is very angry.
No, simply using an agile framework is not the answer.
Integration
agile or not, we need to find ways to bring different disciplines together.
A platform for cross-team interactions can be understood as an overlap of differnt disciplines.
A "uniting of things" - in other words: Integration
The difference between coordination and integration is the difference between a developer-handoff and a
human interaction between Designers and Developers.
developer-handoff:
A process, where a designer will hand results of this work to the developers.
Developers will take the designs and implement the necessary code.
The work is clearly divided and coordinated.
A Developer-Handoff is a warning sign that reads: "warning: missing interactions"
How can we design a system that will really put interactions before processes?
think, make, check
To check and recheck aussumptions is at the core of every agile framework.
It can be displayed as a think-make-check circle.
At first there is an idea that can result in requirements
requirements are used to create working software (make)
the software is then tested and checked for usability and added value.
I use the think-make-check circle to show the difference between cooordination and integration.
The Divide
Design and Testing will describe the software in terms of scenarios (seen on the left).
On the right side, developers will describe the software in terms of an executable software construct.
This is the duality. On one hand, we have a scenario based thinking that lets us define in a very human way how things
should work. This is in fact how we -as humans- try to make sense of the word that surrounds us.
Like: "the user is able to register by providing the needed information".
On the other hand, we write software, because we need something that is machine-executable. We add methods, classes, components, ..., to the software construct, so that it is capable of running our scenario. "we need to implement a new service, the form-component needs to include field x, ..."
This is the divide at the very core of how we work.
Problems
From a developer perspective, this duality creates a world of problems. Here are my top 3.
1. complecting scenarios
After adding functionality to the same part of our construct, we might fail to reason about it.
What scenarios have shaped this component?
Why is this if/else here?
If we are not able to reason about our code, how can we make confident changes?
Complecting scenarios will result in complexity.
2. enabling unwanted scenarios
When we implement a new feature, we add new functionality to parts of our construct -
but what unwanted scenarios we have we accidentally enabled, too?
Bob Martin tells me that i need "negative tests".
It is like the advice to "expect the unexpected" - those tests are not easy to find.
3. change of mindset
We want to increase the power of our software.
The wish is, that the software will be so well constructed, that it would be easy to enable possible future scenarios, or even possible future projects.
We focus on our powerful software - and less on the needs of our users.
Resolving
Different specializations not only need to be coordinated - they need to be integrated as well.
To enable integration between two disciplines, we need something that serves as a common ground.
In other words, it would be beneficial to talk the same language when we talk about software.
The goal is to keep an artifact during the whole process. From design over to development and testing.
Modelling languages for reactive systems, like Statecharts come to mind.