Agile Architecture in Practice

This article is published as a complement to my article on Agile Architecture. It details our approach to agile architecture at Triple Dot Engineering including some specific tools and processes.

tl;dr

We like Draw.io for architecture diagrams. It's simple, easy for everyone, and free. The Draw.io extension for VSCode makes it easily accessible to engineers and the .drawio.png format can be embedded in markdown documents with no extra export step. This makes it fast to go from architecture changes to updated documentation.

We publish our docs with Docusaurus. There many similar tools but our team prefers it for its balance of simplicity and powerful features. We also like that it uses React.

Both our markdown docs and diagrams are stored in a /docs directory in the project they describe. The docusaurus server resides in /tools/docs-server and is configured to point to our /docs directory to render documentation.

---

Background on Architecture

Past approaches to architecture development fall into three categories:

• Formal and semi-formal architectures such as UML and SysML rely on heavy desktop-based modeling tools. While they have the benefit of an underlying data model, they come with a steep learning curve. It's worth noting that in an academic research context, these are frequently considered semi-formal or informal. Formal methods tend to refer to structured architecture description languages such as AADL.
• Informal architecture such as Visio, Diagrams.net, LucidChart, Gliffy, or similar diagrams that focus on visual communication without a strongly defined or easily accessible underlying data model.

There also other categories worth calling out such as Diagrams as Code (DaC), which emphasizes text-based method to create diagrams, allowing them to be easily version controlled with the potential to parse and analyze diagrams. This approach to diagramming comes with tedious overhead to create diagrams but is seeing increasing use. DaC tools such as Mermaid are now supported in Github Markdown.

Each of these approaches have clear benefits as well as unintended pain points or drawbacks.

Approach	Good things	Bad things
Formal approaches	Strong data models, good for analysis	Steep learning curve, successful analysis still requires consistent modeling
Informal approaches	Low learning curve, fast, effective visuals	Lack of underlying data model
Diagrams as code	Good for version control	Tedious to create diagrams

On Architecture and Architects

The more I do “architecture” the more I have grown to dislike the term “architect”. I dislike it because it implies that the role of the architect is somehow fundamentally different from that of a developer or engineer. It’s not. It might be a higher-level view of the system and it might require a better understanding of translating business needs into technical decisions, but that is part of engineering design at all levels of a system.

The architect is the ultimate design authority of the system. They settle technical debates when teams can’t agree, define critical interfaces, describe the overarching structure and behavior of a system, and engage with stakeholders to translate the business problem into a technical one.

But there is a role for this at every level of the engineering organization. On a small team, we might call them the tech lead, or perhaps a TPM (Technical Program Manager). For larger teams or a team of teams or a large system, we tend to call that person an architect.

Our Approach in Practice

In order create better architectures, we had to consider the both the producers and consumers of the architecture.

Software engineers are both a producer and consumer of architecture artifacts. Since they spend much of their time in a text editor or IDE, it was logical for us to try to build a workflow around that.

We began to focus our architecture efforts around the concept of making it easier for developers to engage with the architecture. We now store ALL of our architecture artifacts in the codebase alongside source code. The sample file structure below is similar to what we're using in practice.

.                                  # project root
├── docs/
│   ├── architecture               # architecture definition
│   ├── coding-standards           # ... other engineering documentation ...
│   ├── how-tos                    # ... other engineering documentation ...
│   ├── logbook                    # ... other engineering documentation ...
├── infr/                          # infrastructure as code
├── src/                           # source code
└── tools/
    └── doc-server                 # documentation renderer

In this case, everything that we consider architecture is stored in the /doc/architecture directory. This includes diagrams (in the /doc/architecture/diagrams directory) and documents (all written as Markdown files).

We have other documentation in the docs directory too. How-to articles to help share knowledge among the team, coding standards and version control strategy documents that define team practices, and a logbook where we informally capture engineering design decisions, trade-offs, prototyping, etc.

All of this is checked-in to Git and version controlled alongside our source code, tests, IaC source code (in /infr), and additional tooling.

•••

Diagrams first

Diagrams are the backbone of an architecture description. A clear picture can provide an understanding of the structure and interconnectivity of a system without the need for lengthy documents that never get read.

In our case, we use Henning Dieterichs' Draw.io integration for VSCode to create diagrams directly in our development environment. We also like Draw.io because it is free, open source, and easy to use. It can be used on the web at https://diagrams.net and has a desktop app available here.

We always use the .drawio.png extension because it allows us to embed the diagrams directly in documents without the need for an additional export step.

Data-Driven Draw.io

One concern I've heard about Draw.io is the inability to get data out of the diagram. This is a valid concern. However, it is possible.

Draw.io stores data in an XML format using MxGraph. It is possible to extract the XML from the PNG file and parse it to get the data. Here is a simple Python script that does this:

pngbytes = open('path/to/your/diagram.drawio.png', mode='rb').read()
png = pngbytes.decode('utf-8', errors='ignore')
decoded = unquote(png, encoding='utf-8')
match = re.search('<mxfile>.*</mxfile>', decoded)
mxfile = match.group(0)
print(mxfile)

I will be releasing more content on this topic later this year.

•••

Documents next

Diagrams alone rarely tell the whole picture. While they often need to be supplemented with written documents, those documents are read and updated infrequently. Use them, but use them only for necessary additional context.

Markdown files are the norm for something like this these days. We prefer this over an external wiki because it allows engineers to utilize the same tools they use to write code and allows the documentation to be version controlled. We'll talk more about publishing later.

A clear entrypoint

The documentation should have a clear entrypoint. A README file, index.md, or similar. This is the first document that a new engineer should read when joining the team. It should be a high-level overview and provide links to other documents.

Architecture Decision Records

Architecture Decision Records (ADRs) should be used to capture major architectural decisions or constraints. There are many tools available to support this. We use adr-tools.

ADRs should capture what decision was made, why it was made, and any alternatives that were considered. This helps drive consistent, long-term decisions that outlive the individual team members who were part of the decision making process.

•••

Architecture as code

Our entire architecture documentation exists in the same codebase it describes. It is version controlled and collaborative. Documents should exist alongside and be version controlled with code.

There's not much too this. Architecture as Code may imply that you need to use a diagrams as code tool like Mermaid. While you certainly can do this, I address in a previous article some of the limitations of diagrams as code tools.

•••

Minimize processes

Whenever possible, build your architecture tooling around existing engineering tooling. We write documents as Markdown using VSCode. For diagrams, we use a Draw.io plugin for VSCode so our engineers don't have to jump back and forth between tools. It also streamlines our publishing process.

While we like the integration of Draw.io with VSCode, it's not the only approach. For those who don't use VSCode, there is also the Draw.io desktop app or diagrams.net.

Other tools like LucidChart, Miro, or Visio are also viable options. As are more sophisticated modeling tools like MagicDraw, Enterprise Architect, or ArchiMate.

•••

Publish architecture

Leverage static site generators to make documentation more readable. Embed the static site build into the build process. Integrate this into existing CI/CD processes and deploy documentation sites internally when it makes sense.

Publishing consumable architecture is critical to ensuring it does its job. It should be easily accessible by all team members and key stakeholders.

In our case, we use Docusaurus to render our documentation. We set up a Docusaurus project in our /tools directory and configure its documentation plugin to point to our /docs directory. Other static site generators like Sphinx or Jekyll could also be used.

This approach would also work well with other tools like Azure DevOps Wikis or Github Pages.

•••

Summary

This approach is all about applying the same agile principles we use to build software to the architecture and documentation of that software. If we view architecture as a product, and consider our core agile values, we find that this approach is a natural fit.

We focus on the individuals and interactions of the team over a specific process or tool. We value living, up-to-date documentation over comprehensive documentation.

Frequently published, consumable architecture is more valuable to the team than a comprehensive up-front architecture with every detail defined. Our goal as architects is to enable the team to build the right product while thinking about all the complexities that meet the business need. We find that this approach helps us do that.