Beyond Whiteboarding: Creating Expert System Architecture Diagrams

Expert system architecture diagrams (ESADs) are diagrams created to deliver long-term documentary value. Relative to casual “whiteboard” diagrams, they are more precise, comprehensive, and maintainable.

ESADs promote and demonstrate clear thinking about technical systems. Because they are long-term documentation, ESADs are useful both for onboarding new hires and as a reference for established employees.

In this post, we’ll look at the elements that make up ESADs (Intent, Precision, Comprehensiveness, Maintainability) and the tools and techniques used to create them.

Intent

The most important ingredient of ESADs is the intent of the author. ESADs generally do not happen organically. ESADs are the product of expertise, clear intentions, and specific goals.

Most system architecture diagrams you see online or hand-drawn on a whiteboard are created to be visual aids. They are designed to accompany an article, presentation, or ad-hoc design session delivered in person. They intentionally lack clarity, precision, and comprehensiveness; the accompanying article (or presenter) is responsible for filling in the details and making them clear.

ESADs, in contrast, are created to be documentation. They are meant to be valuable in their own right and aren’t tied to a larger work like an article or presentation. Furthermore, they are intended to provide long-term value. ESADs are designed to be clear, precise, and comprehensive.

The goals of ESADs can be summed up by the “prime directive” of expert system architecture diagramming:

When creating expert system architecture diagrams, always strive to truly inform your viewer rather than merely make an impression.

Precision

Visual-aid diagrams intentionally leave out details. “Handwaving” is expected, sometimes literally. Small inaccuracies are also usually acceptable. This is the case not only because the presenter (or accompanying article) can fill in the details but also for efficiency’s sake. Imprecise diagrams are cheap to create, after all.

In service of truly informing the viewer, ESADs should instead place a premium on precision. For instance, visual-aid diagrams commonly show databases as monoliths and omit details like individual tables. ESADs should reveal details like these.

Ideally, ESADs should include both high- and low-level views. If your diagramming tool allows (see Tools below), high- and low-level views can even be combined:

Beyond generally including more details, ESADs should follow these precision rules:

Label all arrows: An unlabeled arrow between two resources says “these two things are related…somehow.” This lack of specificity is not acceptable in an ESAD. In virtually all cases, arrows should have a label specifying the relationship between two resources.

Use named resources: Similar to the above, resources appearing in ESADs should be labeled with their name or identifier. It sounds obvious, but many visual-aid diagrams (like above) don’t do this. An icon isn’t enough nor is labeling a resource by its type (“Lambda”, “DynamoDB”). Resources should be referenced by name so the viewer knows what it actually is.

Don’t mix levels of abstraction: When adding detail and precision to a diagram, take care not to mix levels of abstraction. Low-level resources (like functions) shouldn’t be related to high-level resources (like databases). Instead, showing functions’ relations to database tables is both more balanced and more informative in this case:

Comprehensiveness

In addition to having high levels of precision, ESADs should be highly comprehensive. That is, they should show not only more detail than traditional visual-aid diagrams, but more detail from more points of view. With multiple points of view, ESADs are useful for a diverse range of audiences and use cases. These points of view are called perspectives.

Some visual-aid diagrams attempt to show all the ways resources in a system are related in a single perspective like in this example AWS diagram:

The above diagram tries to show run-time, deploy-time, security, and data egress concerns all at once. The result is a diagram that is vague to the point of nearly being meaningless. An ESAD of this system would instead split those four concerns into different perspectives:

Breaking a diagram into multiple perspectives allows each perspective to include more details with greater clarity. Together, the perspectives of an ESAD form a comprehensive whole. Read more about multiperspective diagrams here.

Note that perspectives can be used to document not only relations between resources, but their interactions as well. What is the difference? Relational perspectives show static relations between resources, while interaction perspectives show examples of how the same resources interact. A comprehensive ESAD should contain interaction perspectives of common and important data flow paths:

You can read more about relational and interaction perspectives in this post.

Finally, ESADs should include inline notes for the sake of comprehensiveness. These notes help explain, in prose, things the perspectives themselves cannot. A two- to three-sentence summary for each perspective is ideal:

Maintainability and Tooling

Diagrams created for a presentation, blog post, or whiteboard session aren’t typically made with long-term value in mind. They aren’t meant to change and therefore aren’t created for change. In many (most?) cases, changing such diagrams is so difficult that they are effectively immutable.

Diagrams created for documentation don’t have the luxury of not worrying about long-term value. The systems they document change; therefore, they must change to stay relevant. Maintainability is a critical requirement for ESADs.

Resolving to maintain system architecture diagrams is easy. Actually following through is hard. Willpower and process (for instance, dedicating a percentage of time to documentation maintenance) are important. Just as important, if not moreso, is choosing diagramming tools that make maintenance easier and, therefore, more likely to happen.

For the sake of maintainability, choose a diagramming tool that lets you define diagrams using code. Do not use drag-and-drop diagramming tools for creating ESADs.

Drag-and-drop tools are easy to learn, but diagrams created with them are extremely difficult to maintain. Updating a drag-and-drop diagram requires endless re-arranging of boxes and lines and often fiddling with arrows, labels, and font sizes. Because of these costs, drag-and-drop diagrams are rarely maintained and quickly lose relevance.

Furthermore, diagrams created with drag-and-drop tools generally cannot be diff’d or merged. Diagrams defined in code naturally allow for these things. Such diagrams are easily and naturally stored in version control systems.

As for specific tools, Ilograph defines diagrams using YAML. It also has first-class support for perspectives and combined high- and low-levels of detail. Structurizr, PlantUML, and Mermaid are other diagrams-as-code options.

Questions or comments? Please reach out to me @ilographs on Twitter or by email at billy@ilograph.com.

Share this article on Twitter

Share this article on Facebook

Share this article on LinkedIn