Thinking Like an Architect
Introduction
Building maintainable software requires more than just writing code, it demands a long-term perspective. This is one of the biggest differences between developing software for a college assignment and building systems in a professional setting. In industry, software often lives for years, and the cost of maintaining and evolving it frequently exceeds the initial cost of development. That’s why thoughtful software architecture is essential, it lays the foundation for systems that are not only functional today, but sustainable and adaptable in the future.
Working Software is the Minimum Bar
The Agile Manifesto states that working software is the primary measure of progress. This is true, but it is not the only measure of progress. Software that works as intended should be the minimum bar. It is the starting point, not the end goal. The end goal is working software that is maintainable, extensible, scalable, and secure. Creating such software requires a different mindset than just writing code.
Tactical vs Strategic Thinking
In order to succeed at creating maintainable software, you need to approach programming with a different mindset than you might be used to. You need to think about the big picture, not just the code in front of you. You need to think about how your code will be used, how it will be maintained, and how it will be extended. Many organizations don't understand the cost of rushed implementation and the benefits of taking the time to do things right. They just want things as fast as possible without considering the long-term consequences. Part of your role as an engineer is to help communicate these consequences to the business.
Tactical Thinking
Most junior software developers, and even some senior developers, think tactically. They focus on the code in front of them and don't think about the big picture. Their objective is complete the task at hand as quickly as possible. They want finish some feature or fix some bug and move on to the next task. This mindset produces working code, but it will make the codebase harder to maintain and extend in the long run.
The biggest problem with tactical thinking is that it's short sighted. It doesn't consider the long-term consequences of the decisions being made. If you are rushing to finish something then you aren't thinking about the system design or the consequences of the shortcuts that you are taking today. As we mentioned previously, complexity grows incrementally and is much easier to prevent than to remove. If you are taking a tactical approach for all of your work, or even worse, if everyone on your team is taking a tactical approach, then you are setting yourself up for failure. Complexity will rapidly grow unmanageable and your codebase will soon just be a big ball of mud.
Leaders without a strong technical background will often encourage this mindset. They want to see results quickly and don't understand the long-term consequences of rushed implementation. They don't understand that taking the time to do things right will save time and money in the long run.
Many organizations will have a developer or two who are the rock stars programmers of the team, who take tactical programming to the extreme. Ousterhout describes these sorts of developers as tactical tornados. They are the ones who can get things done quickly and are always the first to finish their tasks. They are the ones who are always working late and are always the first to volunteer for extra work. They are the ones who are always fixing the bugs that no one else can fix. They are the ones who are always the first to be promoted. These developers are often the ones who are taking a tactical approach to their work. They are the ones who are taking shortcuts and creating technical debt. They are the ones who are setting the team up for failure.
While leadership may see these developers as the most productive members of the team, they are actually the ones who are causing the most harm. Other developers on the team will end up wasting hours trying to understand their rushed solutions and more hours trying to clean up the mess that they left behind.
Strategic Thinking
The alternative is to take a more strategic approach to software development. Consider the long-term structure of the system as you are writing code. Think about how your code will be used, how it will be maintained, and how it will be extended. Consider the consequences of the decisions that you are making today. Consider the trade-offs that you are making.
The system you are building is probable going to live for years, often much longer than you will be working on it. Your most important job as a developer should be to make future extensions and updates as easy as possible. You should not just be trying to make code that works. Obviously you code has to work, but your primary goal should be to build a well designed system that is easy to maintain and extend. You need to be constantly examining the system design and looking for ways to improve it. Question every design decision you make, looking for better alternatives. Be willing to refactor your system to make it better any time you see an opportunity.
Inevitably this will require you to take more time to complete your tasks. Sometimes your initial design will be incorrect, and you'll feel like you've wasted time. But in the long run, taking the time to do things right will save you time and money. It will make your system easier to maintain and extend.
Working in a codebase where this mindset has been the norm is much more fun and rewarding than working in a codebase where the tactical mindset has been the norm. You will be able to make changes quickly and confidently. You will be able to add new features without fear of breaking existing functionality. You will be able to fix bugs without fear of introducing new bugs.
State of DevOps Report
The State of DevOps Report is an annual report that examines the state of software development in industry. The report is based on a survey of thousands of software developers and managers. The report consistently finds that organizations that take a strategic approach to software development are more successful than organizations that take a tactical approach. These organizations not only have higher quality software, but they also have higher productivity and are able to deliver software faster.
While not all organizations will support you taking a more strategic approach, the data is clear. The tradeoff between tactical and strategic programming isn't a tradeoff of time for quality. The choice is between building worse software more slowly or building better software more quickly. If you're struggling to convince your organization to take the time to work on design, then you should consider sharing some of the data from the State of DevOps Report.
The Architect's Mindset
The software architect needs to take strategic thinking to the extreme. They are responsible for crafting the overall design of the system, and for communicating that design to the rest of the team. As the system evolves, the architect is responsible for ensuring that the design remains consistent and that the system remains maintainable and extensible. The design will need to evolve as new requirements are discovered and as the system grows. The architect is responsible for ensuring that the design evolves in a consistent and maintainable way.
A good software engineer will also be considering many of these same topics. The biggest difference between a software engineer and a software architect is the scope of their design. The software engineer is responsible for the design of individual components or applications, while the software architect must also consider how their design will interact and communicate with other components and applications at the organization.
Up Front vs Emergent Design
It's tempting to try to make all of your most important design decisions up front. You want to make sure that you are building the system in the right way from the beginning. However, this is always a mistake for any large system. This the waterfall method, which has never worked to build good software.
At the start of the project is when you know the least about the system you are building. You don't know what the requirements are, you don't know what the constraints are, and you don't know what the trade-offs are. You can't possibly make all of the right decisions up front. The earliest decisions are also the ones that are the most expensive to change. By the time you realize that the design is flawed, the system and the design will be so tightly coupled that it will be very difficult to change.
The extreme alternative to this is the emergent design approach, often recommended by Agile development proponents who haven't actually understood the Agile Manifesto. In this approach, you don't make any design decisions up front. You just start writing code and let the design emerge as you go. You start with a small piece of functionality and build on it incrementally, making design decisions as you go, and refactoring your code as you learn more about the system. This approach is much more flexible and allows you to adapt to changing requirements. However, in practice this often leads to a poorly designed system. You need to be constantly examining the design of the system and looking for ways to improve it. You need to be willing to refactor your code to make it better any time you see an opportunity.
The biggest issue with this approach is that some design decisions are too expensive to change once you've made them. If you've built your system on a flawed foundation, then you will have to throw away a lot of work to fix it. Things like security, scalability, and performance are often impossible to add after the fact, and without them you have a system that cannot be used at all.
In my experience, the best designs come from a combination of these two approaches. You need to make some design decisions up front, but you need to be willing to change those decisions as you learn more about the system. You need to be constantly examining the design of the system and looking for ways to improve it, refactoring your code to make it better any time you see an opportunity.
Minimum Viable Architecture
When working with a new system, you should start with a minimum viable architecture. This is the simplest architecture that will allow you to build the system you need. You should start with a simple design and build on it incrementally, making design decisions as you go, and refactoring your code as you learn more about the system.
This approach is especially important for startups and other organizations that are working on new products. You don't know what the requirements are, you don't know what the constraints are, and you don't know what the trade-offs are. You can't possibly make all of the right decisions up front. You need to start with a simple design and build on it incrementally.
It's all too common to see startups decide to build microservices for their product, just because they think that's what they are supposed to do. They don't have the requirements or the constraints to justify building a microservices architecture, and they end up with a system that is much more complex than it needs to be. They would have been better off starting with a monolithic architecture and building on it incrementally.
Non Functional Requirements
The first thing I consider when designing a system is the non-functional requirements. These are the requirements that are not directly related to the functionality of the system, but are still critical for the system to be successful. These requirements include things like security, scalability, performance, and maintainability.
The reason I start with these requirements is because they often have a bigger impact on the overall design of the system than the functional requirements. For example, if you need to build a system where performance is critical, then you will need to design the system in a way that allows you to scale it horizontally with important operations all being done locally. If you need to build a system that is highly available, then you will need to design the system in a way that allows you to failover to another data center. If you need to build a system that is secure, then you will need to design the system in a way that allows you to encrypt data at rest and in transit. Each of these requirements will impact every piece of the system, and it's important to consider them up front.
The term non-functional requirements is a bit of a misnomer. These requirements are just as important as the functional requirements, and they are often more difficult to meet. I prefer to think of these as cross-cutting requirements. They are requirements that cut across the entire system and impact every piece of the system. While most functional requirements can be implemented in a handful of components, non-functional requirements often need to be considered by every component in the system.
Identifying the non-functional requirements is often left to the architect. Product owners and stakeholders can usually describe what they want the system to do, but rarely understand the limitations that the system will have to operate under. The architect is responsible for identifying these requirements and ensuring that the system is designed to meet them.
Behavioral Anti-Patterns
Certain patterns of thought or behavior can lead to poor software design. These are often referred to as anti-patterns. Anti-patterns are common solutions to common problems that are ineffective and may result in bad software design. Here are a few common anti-patterns that even experienced developers can fall into.
Golden Hammer
The golden hammer anti-pattern is when you use the same tool for every problem. This is often seen in developers who are experts in a particular technology or framework. They will try to use that technology or framework for every problem, even when it's not the best solution.
Cargo Cult Programming
Cargo cult programming is when you copy and paste code without understanding what it does. This is often seen in developers who are under pressure to deliver something quickly. They will copy and paste code from Stack Overflow or from other parts of the codebase without understanding what it does. This can result in a system that is full of bugs and is difficult to maintain.
Not Invented Here
The not invented here anti-pattern is when you refuse to use existing solutions to common problems. This is often seen in developers who are eager to be creative and solve problems. They will try to build their own solution to a common problem, even when there are existing solutions that are much better.
This is particularly common with developers who are prioritizing their own promotion over the success of the project. They want to build something new and exciting, even when it's not the best solution for the project. Building something new and exciting is much more likely to get you promoted than using an off-the-shelf solution, even when the off-the-shelf solution is the cheaper and better.
Premature Optimization
The premature optimization anti-pattern is when you optimize your code before you know where the bottlenecks are. They will spend hours optimizing a piece of code that is not a bottleneck, while ignoring the pieces of code that are actually slowing the system down.
When I was an intern at my first job, I once spent two week carefully improving a clustering algorithm I was working on. After all of that work, I had only reduced the runtime by 15 minutes for a 3 hour job. It turned out that the bottleneck was I/O, not the clustering algorithm. I could have saved myself a lot of time and effort if I had just profiled the code first.
YAGNI
YAGNI stands for "You Aren't Gonna Need It". This is the idea that you shouldn't build something until you actually need it. This is often seen in developers who are trying to predict the next feature they'll need. They will try to build a system that is much more complex than it needs to be, because they think that they will need the extra functionality in the future.
Technical Debt
The term technical debt was originally coined by Ward Cunningham, one of the signers of the Agile Manifesto. It is the consequence of software development decisions that result in prioritizing speed or releases over well-designed code. Just like financial debt, technical debt can be a useful tool when used correctly, but it can also be a burden that slows you down.
This can manifest as missing documentation, bad test coverage, inefficient designs, tightly coupled components, or incomprehensible package structure. All of these make the codebase less fun to work on and more difficult to maintain.
There are times when the rushed solution is the right solution. Many software engineering textbooks advocate for always seeking the perfect solution, but that's just not realistic or reasonable. There are times when you just need to get something out the door. The key is to be aware of the trade-offs that you are making and to be willing to pay down the debt when the time comes.
Tolerance for technical debt will vary from team to team and company to company. In general, engineers tend to have a much lower tolerance for technical debt than management or product owners. Engineers are the ones who have to work with the code every day, and they are the ones who have to deal with the consequences of the shortcuts that were taken. Management is often more concerned with getting things out the door as quickly as possible, and they don't understand the long-term consequences of rushed implementation.
Common Causes of Technical Debt
Robert Martin, the author of Clean Code, argues that technical debt should always be a result of a conscious decision. Laziness, ignorance, or unprofessionalism are sources technical debt in his opinion. I think this mindset is unreasonable. It doesn't leave room for engineers to be wrong, to work without complete information.
Broader definitions of Technical debt come from other authors. Steve McConnell, the author of Code Complete, split technical debt into two categories: intentional and unintentional. Intentional technical debt is when you make a conscious decision to take a shortcut. Unintentional technical debt is when you make a mistake. This is a much more reasonable definition. It allows for engineers to be wrong, to work without complete information.
Martin Fowler goes even further and splits technical debt into four categories: reckless, deliberate, inadvertent, and prudent. Reckless technical debt is when you take a shortcut without considering the consequences. Deliberate technical debt is when you take a shortcut with a plan to pay it down later. Inadvertent technical debt is when you make a mistake. Prudent technical debt is when you take a shortcut to meet a deadline, but you are aware of the consequences.
Regardless of what definition you're using, there are some common causes of technical debt that you should be aware of.
Financial Pressure
Financial pressure can result in staffing restrictions or limit your technical options. When cost is the main driver of a technical decision, tech debt is the likely result. Adopting tech debt can be a way to meet a deadline or to save money in the short term, but will cause issues later.
Tight Deadlines
Probably the most common cause of technical debt is tight deadlines. When you are under pressure to deliver something quickly, you are more likely to take shortcuts. This is especially true when the deadline is arbitrary or when the consequences of missing the deadline are not well understood.
Unclear Requirements
When major requirements change mid project, it can be difficult to keep the design of the system consistent. The initial architecture might have been great for the initial requirements, but it might not be able to adapt to the new requirements. This can result in a lot of technical debt as you try to shoehorn the new requirements into the existing design.
Inexperienced Engineers
Most junior engineers don't have the experience to understand the long-term consequences of their designs. They've never suffered the consequences of an inflexible design, so don't understand the benefits of a more careful approach. Input from more experienced engineers and mentors is critical to avoid technical debt.
Outsourcing Critical Decisions to Non-Stakeholders
When non-technical stakeholders are making technical decisions, you are likely to end up with technical debt. Non-stakeholders are less invested in the project and more likely to accept tech debt to meet short term goals. Studies have shown that if developers have a sense of ownership over the code, they are more likely to produce higher quality code.
Top Down Management
A top-down style of management results in engineers with less ownership in the project and less empowerment to make good decisions. This can result in engineers taking shortcuts to meet deadlines or to avoid conflict with management.
Consequences of Technical Debt
Technical debt itself is neither good or bad. It's a tool that can be used to meet deadlines or to save money in the short term. The problem is when technical debt is not managed properly. If you are not aware of the trade-offs that you are making, then you are setting yourself up for failure.
The biggest issue is when technical debt is allowed to accumulate without being managed. It can start to have significant consequences for the project and the company. The codebase will become harder to work on, and the cost of making changes will increase. The system will become more fragile and more bugs will be introduced with each change. Eventually this leads to reduced team momentum. If not addressed it will eventually lead to morale problems and developers leaving the team.
When Should You Take on Technical Debt?
Sometimes a shortcut is the right path. While I usually concentrate on crafting the best design I can, there are times when I just need to get the code finished. There are three main cases where I will take on technical debt.
Prototyping or POCs
If I'm building a prototype or a proof of concept, then my only goal is to finish as quickly as possible. I'm not concerned with maintainability or extensibility. I just want to show that the idea works. The purpose of a prototype is to learn, not to build a production system. Once I've learned what I need to learn, I'll throw the prototype away and start over with a clean slate. You should rarely be building on top of a prototype, the consequences of doing so are often disastrous.
Emergency Fixes
Sometime critical stuff breaks and you need to fix it quickly. If the system is down or if the bug is causing data loss, then you need to fix it as quickly as possible. You don't have time to consider the long-term consequences of the fix. You just need to get the system back up and running. Once the system is stable, you can go back and clean up the fix.
Exploration
Sometimes you need to explore a new technology or a new approach. You don't know if the technology is going to work, so you don't want to spend a lot of time building a perfect solution. You just want to build something quickly to see if it works. If it does work, then you can go back and build a better solution. If it doesn't work, then you can throw it away and start over.
Conclusion
Getting into the right mindset is the first step to building maintainable software. You need to think about the big picture, not just the code in front of you. Working software is where your journey starts, not where it ends. You need to think about how your code will be used, how it will be maintained, and how it will be extended and consider the long-term consequences of the decisions that you are making today.
You also need to know when to discard the rules and take a shortcut. Sometimes a shortcut is the right path. You need to be aware of the trade-offs that you are making and be willing to pay down the debt when the time comes.