Architecture as code is not an easy task

This article is of a kind I don't like : it's a failed experience feedback. And it's written in english for one reason (and only one) : making sure Simon Brown (whom I don't know french reading level) reads and understands it.

And what failed is a project I developped to leverage (at least that what I thought when I started) Structurizr ability at providing a lightweight model usable and editable by Java.

But first ...

Context

I've discovered C4 and Structurizr some years ago (my external archive tells me it dates from '18). And I've immeditaly loved the way it allows someone to describe a software system architecture with an ease that UML never ever dreamed of. Not long after, I read the agile architecture documentation template provided by Simon and really felt in love (undoublty because I do believe humans are story people) with the way it tells the story of the software system studied using C4 and Structurizr.

So, when I was missioned as cross-team tech lead for three projects, I decided to use that tooling to produce all my documentation. And I was able, thanks to these tools and my old architect tool belt (PlantUML and Asciidoc) to produce architecture documents of a very good level with little effort. However, there were some Java classes used to extract informations from the environment (K8s deployment information) and from the build artifacts (maven modules as containers) that were too much specific to my mind.

Need is mother of invention

As a consequence, when the pandemic locked me at home with no more mission, I took the time to get my ideas together and

1. Write some french documentation explaining/paraphrasing agile architecture documentation
2. Create a maven artifact project containing smart tools allowing my colleagues - and myself - to easily produce high quality architecture documentation at low cost.

I immediatly met some accidental complexities that I solved through ... well ... shenanigans.

Asciidoc is not so easy

Figure yourself, generating asciidoc in Maven is not so simple.

Indeed, as Asciidoctor is not a Java port of asciidoc, but rather a wrapping of the Ruby code using JRuby, customizing that generation is quite complex. Typically, each component of the system must have a version that allows all the components to work together.So when you update JRuby, you have to check "religiously" that the asciidoct-maven-plugin still works, and that asciidoctor-diagram works.

Speaking of which I had to solve a very specific bug with what can only be described as a hack.

Chaining Java code is not so easy

Considering I wanted to build a Java model of the code prior to feeding asciidoctor-maven-plugin with PlantUML diagrams generated from that model, I had to

• Generate a basic architecture model of my application using Java code
• Augment that model with informations extracted from that model metadata
• Augment documentation with informations also extracted from that model metadata
• Generate an include chain allowing human-written documentation to include all that generated content
• Generate the correct diagrams

The augmentation part being modular (the curse of architects) to handle different cases, such as projects using GitHub or Gitlab, extracting containers from Maven and components from Spring/GWT/JavaEE components, I chose to use an old friend of mine, CDI (and in that particular case, JBoss Weld).

And, since all that code was to be run in maven, I had wrap that Java executable in a call to exec-aven-plugin, with all parameters sent as system properties (which worked well, but was quite ugly in the maven pom).

So, my Java code started a Weld container, loaded all CDI components, and orchestrated it. All in a maven build which job is ... well ... to orchestrate plugins, no ? So the concept was good, but the implementation was overly complex.

What was more annoying was that, since my maven build had to embed the exec-maven-plugin code and the asciidoctor-maven-plugin generation (which used asciidoctor-diagram and "some" configuration to handle niceties such as ability to send feedback to documentation author), the pom.xml file was of a rarely seen size (692 lines for the project documentation module).

A chain is as strong as its weakest link

But there is even worse. Acustomed to old french companies, which tend to despise external tools, I prefered to use C4-PlantUML to output diagrams instead of sending them to a structurizr workspace. It worked well, thanks to structurizr-plantuml module (which I contributed to), until the accident. Well, not a real accident, but more an incident. As you see in that question, I found a solution (the hack previously mentionned). But even after having solved that issue, generating the PlantUML diagram remains painfully slow.

To be honest, in my current mission, generating asciidoc routinely takes more than 2 minutes, mainly due to diagrams (yup, i've tested removing them). And, due to the accidental complexity of my pom, i never did the move from asciidoctor-diagram to asciidoctor-kroki (which is way faster, and accept ay more easily to generate diagrams using a locally unning docker image)

The final nail

I guess you see the end coming ... but, you know, I'm quite persistant. And I tried to fix that, but then came the final nail. This one took the form of an opportunity : replaying in an evening event the presentation I did in my company, which you can see below

And when I started working on it, I took a look at this intriguing Simon Brown tweet (because I was intrigued by the concept of structurizr-lite)

simonbrown Simon Brown 2 juil. 2021 à 09:57:18 A couple of people have asked whether Structurizr Lite can be used with the code-based client libraries. Absolutely! And since Lite is file-based, you don't need to use the API ... just export your workspace to a JSON file instead. More details at https://t.co/ITf3hiWrGx

And it was a blast : I was able to write my model in a cute DSL, and have all my diagrams rendered nicely. This was a very good presentation tool (I tested it on my colleague), but also a way to generate those diagrams that was undoubtly wayyyy better packaged than what I was building.

Time for introspection

I'm not good at introspecting as I tend to think i'm an awesome person, with genius ideas and the ability to deliver them flawlessly. But this time, the comparison was hard and I had to consider my frankenstein of a tool for what it was : a good idea, poorly delivered.

Isn't it anything to salvage ?

It took some time (and this article is the realization of this introspection), but I've finally understood that there were good and bad ideas.

Let me sum them up here

The good

• Provided I'm working on a monolith, I can assume maven modules are mapped to structurizr containers, and this helps producing immediate understanding of the project architecture
• Collecting informations from the maven model to augment the structurizr architecture model is really useful for developers : I don't have to indicate Structurizr it's a Spring project, as my code is already able to understand that by scanning dependencies
• In the same fashion, extracting GitHub conventionnal project files (README.md, CONTRIBUTING.md) is a great idea, as developers are used to fill those files with great content, that has its own place in architecture documentation
• Speaking of GitHub extraction, I played with the idea of using GitHub token to connect to GitOps repository to extract K8s deployment files and transform them into standard architecture documentation. This was poorly made, but already understandable AND useful

The bad

• As I already mentionned, build asciidoc content in maven, when using some plugins and wanting modern versions, is just a nightmare. The build file becomes very fast huge, and nobody can maintain a maven pom cumulating 5 profiles, 3 different plugin configurations - each using hundred of lines of XML). Beware : it's not a critic of Maven, more a citic of the non-optimal integration of Asciidoc in the maven way (yeah I'm a believer)
• Using another plugin system in a maven build is just plain stupid
• For the first time, I decided not to deliver my artifacts on maven central (due to the needed bureaucracy) but on Jitpack. It was a bad iead, since companies tend to maintain maven mirrors which mirror maven central (cool), possibly Spring repository (of no interest to me), but never Jitpack, which is way too volatile. So I couldn't download my artifacts in CI machines, and one of the promise of Structurizr (considering the architecture documentation as one of project by-products, always generated on build, and as a consequence alays up-to-date and available ina rtifact repositories)

The weird

• Being able to reuse architecture diagrams in reveal.js diagrams (thanks to asciidoctor-reveal) was quite funny, but was hardly used (which is quite normal, to my mind)
• I built a quite complex system to hide the Asciidoc content I generate, and I still can't decide if it was a genius idea, or the weirdest fuckery I've ever produced

What to do next

I tink this article quite sums up the next step : I should refocus on the parts of the project that adds value to Structurizr ecosystem, and give up the other fragments.

But, instead of immediatly giving a method (what I did in my first draft which I immediatly seen as wrong), I think I should first use agile architecture documentation lessons and redefine my goals and non-goals. So, what are my goals ?

• Use build-time analysis to enhance Structurizr models
• Use build-time analysis to also enhance produced documentation
• Integrate as much as possible in Structurizr ecosystem

And what are my non-goals

• Force users to choose between enhancements and conformance to Structurizr tooling
• Produce a virtual power plant, too complex to maintain
• Multiply the moving parts

I think I'm gonna try to use Hacktoberfest to rewrite my tooling in a smart way ...