Readable Documentation

Good documentation is readable documentation, this guide will help you structure your technical documentation to get people to read it. When writing for busy engineers there are three things to keep in mind:

  • Everyone is in a hurry, so wants to be told exactly what to do next
  • Most deep read only as a last resort, or when curiosity strikes
  • Most readers need concrete examples first, and abstractions later

If you write knowing that readers will be skimming, your documentation will be more successful.

This means you will be repeating yourself a lot, you’ll find I’ll be doing the same below.

Why trust me?

I wrote Software Telemetry from Manning, which involved professional editing. My editor spent a good six months hammering these lessons into me again and again, and it absolutely made the final book far more useful as an educational guide. The lessons I learned writing Software Telemetry greatly improved my workplace documentation writing skills.

This is me, helping you.

And thank Marina Michaels, who helped me.

What this document does

This document will give you the fundamentals of how to structure readable documentation that will give readers the answers they’re looking for. Technical writing is a whole discipline, but the techniques here will give you a lot of improvement without having to understand the theory.

I will also give you guidance on how to encode the tips here into both Confluence and Markdown, which are quite common documentation standards. Asciidoc is somewhat more suited to the job of documentation than Markdown, but doesn’t have anywhere near the uptake; and if you’re using Google Drive or Office 365 you have a full style system to work with.

Figures and images are their own topic, so the second half of this document will be about how to best use them—and you really do need to use them, even if you loath the practice. More on that in Handling images and figures.

Opinion: Why I dislike Markdown for documentation

Due to how many extensions are needed to get Markdown usable for documentation tasks, no two Markdown implementations are the same. This creates incompatible documentation islands. Markdown that works for Github probably won’t work in the company homebrew knowledge-base without a lot of work.

But the big reason? WYSIWYG. Authors want to see what their documentation will look like while rendered. Do you write HTML raw, or do you use a visual editor and tweak code later? Unless your Markdown authoring tool has the exact same extensions and render-style as where you’re publishing—did the callout markup actually work?—that disconnect will slow writing down!

Documents have a job, and the big jobs for internal technical documentation are:

  • Tell someone how to do something, such as runbooks for responding to alarms or how to craft an API request against your service. Such documentation is mostly aimed at consumers of your service.
  • Explain the standard procedures for maintaining your product, such as how to take a shard offline for patching or executing a multi-region failover. Such documentation is intended to help new people on your team.
  • Explain a process for requesting something be done, which are less technical and more bureaucratic. Knowing which JIRA project to use and how to fill out the fields, among other things.
  • Provide new people on your team, and curious outsiders, the fundamental concepts of how your service works and operates. Unlike the previous three points, people reading architecture guides aren’t in a terrible hurry. More on this in When to go detailed.

Most of what you’re likely to write are documents that people in a hurry will be consuming. Hurrying people don’t like to read much.

What this document does not do

Long as this document is, this document isn’t a deep-dive. Technical writing is an entire discipline which improves with practice and feedback. This should get you started; but for the next level, you will need to engage with others and a few experts. Find your writing crew and share! You can help everyone find their own next level.

This document covers accessibility concerns in ways that can be addressed using base Markdown or Confluence feature, which is insufficient for fully accessible documentation. This is a tragedy, and is also a systemic limit on our ability to include all readers. Often, you the writer are at the mercy of the accessibility features of the HTML rendering system your document will be run through.

Organization basics

Readers often get to a document through searching, so structure your document to be legible to the search engines indexing your documentation.

  • Provide detailed headings. Headings are key metadata for a search engine to determine topicality. A heading of “Runbook” will show up in search results with that name, but not be that useful. A heading of “Restoring soft deleted accounts” is detailed and more likely to be clicked by someone in a rush.
  • Avoid walls of text. When you have a block of paragraphs larger than the reader’s screen, they’re more likely to skip the reading. Break up walls with figures, bulleted lists, and other breaks.
  • People like pictures. Readers tend to stop skimming when they get to an image, so you should use them. Many readers often start with the images, and only move to the text after they’re confused.

Because your readers are likely jumping to the middle of a document, you’ll need to write the document as if the reader hasn’t read much of anything else on your topic. There are some cases where you are safe to assume the reader has baseline information, more on that in When to go detailed. Repeating high level context in your details is usually called for, but link to your deep dives if you feel the temptation to get into nuances.

Creating useful headings

Headings are the previous line, the structural elements of a document. In HTML world, these are your <h1> and <h2> and so on tags, which give you a structured outline of your page. When search engines index an article in the document repository, heading text is important metadata affecting relevancy computation for search queries.

In short, you really do need to provide detailed headings. Consider the following bad example:

<h1>Standard operations</h1>
<h2>Restoring</h2>
<h2>Dropping</h2>
<h2>Splitting</h2>

If you don’t know what the above document is about; that’s my point, you can’t tell. Consider this next heading set, which goes into somewhat more detail:

<h1>Standard Atlas shard operations</h1>
<h2>Restoring reaped shards from soft-delete</h2>
<h2>Hard-dropping soft-deleted shards</h2>
<h2>Splitting shards as part of team splits</h2>

Seeing these headers in search results makes it far more clear what this indexed document covers, and how closely it matches the search criteria. The first bad example was probably a page in a directory called “Atlas” which you would hope the indexing engine would take in to account, but you’d be surprised. The second example leaves less to chance.

Warning

You can go overboard with this, keep header text to a single line. Consider the line-width of a git-commit message, that’s about as big as you want them to get.

Breaking up walls of text

Readers in a hurry don’t want to sift through more than a page full of paragraphs, so you need to break up your documentation. Generally speaking, if you’re stringing together many paragraphs you’re trying to teach some nuance that readers in a hurry don’t have time for. Keep such things for your deep dives; and even then, break up the walls when you can. More on writing for deep dives is in When to go detailed.

You have many tools to break up text-walls.

  • Bulleted lists, like this one. Same goes for numbered lists.
  • Callouts, like the Warning in the previous section and the Opinion in What this document does.
  • Hyperlinks, like the two already in this section. The change of color and style causes the eye to pause.
  • Tables, though they create significant layout complications with various screen geometries, especially when viewed on mobile.
  • Code blocks, which are probably your concrete examples. The changed style of a code block grabs the eye.
  • Images, because a picture is sometimes the repetition people need for learning. Also, they get people to stop and read.

The key component of the entire previous list is the change of style causes skimmers to slow down and parse what they’re looking at. People will read bullets because they’re short. Images convey a lot of information in a small space. Callouts highlight key information. Tables are small text with associations.

Most skimmers read text second. Get them to stop and read. Let’s go into the above in more detail.

Using bullets and numbers

Bullets are a base feature of Markdown, and even tools like Confluence will automatically turn a line starting with an asterisk (*) into a bullet. Similarly, numbered lists are a base feature of Markdown and Confluence will auto-start a numbered list if you begin a line with a number and a period.

The following is a Markdown example demonstrating both styles. Note that a line with a space is needed to break up the list and start a new style.

* Bullet point 1
* Bullet point 2
with line overflow
* Bullet point 3

Spaces to break the list and allow new style.

1. Numbered list item 1.
1. Numbered list item 2.

Similar text will start bullets and numbered lists in Confluence.

Warning

Keep bulleted and numbered lists to five or fewer items, six at the outside! Long lists get skipped. If you need a longer list, use leading emphasis like the list in the previous section. Leading emphasis lets skimmers parse the list at a glance and pick the line they want to read.

Using callouts

Callouts are those stand-out boxes in documentation that draw your eye to a key nugget. The below is an example from GitHub.

Warning-symbol in amber with 'warning' in text, and 'beware of the leopard' in black text below
Warning: beware of leopard

Callouts are great for catching the eye and getting people to slow down and read.

For Callouts we have to step outside of the Markdown standard. Markdown extensions that add support for callouts often coopt the block-quote style. The more common extensions encode callouts like the below, using a prefix line.

> [!WARNING]
> You can go overboard with this, keep header text to a single line. Consider line-width of a git-commit message, that’s about as big as you want them to get.

This would render a callout similar to the one in the Headings section. Here is how the above Markdown would look rendered by GitHub.

Warning-symbol in amber with 'you can go overboard with this,' and the remainder of the callout from earlier
The callout in 'bullets and numbers' rendered in Github style

In Confluence, the comparable features are found in Macros. Confluence has predefined styles for Info, Tip, Note, and Warning callouts. For callouts of other types, they have a Panel macro to define everything from the title text to box color. You can also encode macros while typing. Here is an example of a Warning callout using a Confluence macro.

{warning:icon=true}
You can go overboard with this,  keep header text to a single line. Consider line-width of a git-commit message, that’s about as big as you want them to get.
{warning}

Unlike some Markdown extensions, Confluence allows adding text to the tile of the callout. You saw an example of this in the opinion callout in What this document does. The Confluence markup for that callout would use the Panel macro.

{panel:title=Opinion: Why I dislike Markdown for documentation}
[Long explanation]
{panel}

Using tables

Tables are a bane of Markdown. No one likes how they’re encoded, which is why there are so many competing extensions to encode them. Authoring a table in Markdown really requires use of a WYSIWYG editor due to the alignment difficulties of keeping rows and columns straight in your head; tables are inherently visual, so you need a visual representation when creating them.

If you have to create a table in Markdown (please avoid it, you’ll be happier) you’ll need to check your local extensions for the exact syntax. Github uses a visual system of pipes and dashes to create a sort of table.

| Column header 1 | Column header 2 |
| --- | --- |
| Row 1/Col 1 | Row 1/Col 2|
| Row 2/Col 1 | Row 2/Col 2|

The Github syntax is tolerable for text blocks that are mostly similar, but are likely to create some huge text-lines in your document to avoid line-breaks. Again, the table extension used in your document repository may use a different syntax.

Confluence has a visual table editor, which is far more intuitive to use.

Using code blocks

Code blocks are where you give details on how to code something up. These are concrete examples and quite important to include to speed readers up in solving their problems.

Code blocks are now a first class citizen in most Markdown uses, even though the base standard didn’t have the concept of code fences. The following is my header examples put into a Markdown code fence, using HTML markup.

```html
<h1>Standard Atlas shard operations</h1>
<h2>Restoring reaped shards from soft-delete</h2>
<h2>Hard-dropping soft-deleted shards</h2>
<h2>Splitting shards as part of team splits</h2>
```

Confluence has similar. If you enter three backticks, Confluence will create a code block and give you a pop-up to select which language will be in the code block.

Using images

Images get people to slow down and read, I go into much more detail in Handling images and figures. Image markup matters due to how captions and alt-text are handled. Consider the following image from Chapter 1 of Software Telemetry.

Header titled 'example of a metrics system.' Three boxes left to right labeled Emitting Stage, Shipping Stage, and Presentation stage. In the emitting stage are two green boxes labeled Production Software and OS Monitoring (collectD). In the shipping stage are two light green boxes labeled StatsD Exporter (connected to Production Software) and Graphite Exporter (connected to OS Monitoring). Both green boxes are connected to a cylinder labeled 'prometheus storage'. In the Presentation Stage is a green box labeled Grafana Server, connected to the Prometheus cylinder.
Figure 4: Metrics system example, using multiple sources, storing in Prometheus, fronted by a Grafana server

Markdown does not do the above well. Per the markdown specification, inserting a figure is done using the following syntax:

![Mouse-over text](URL "Alt-text")]

Which should render HTML similar to:

<img src="URL" alt="Alt-text" title="Mouse-over text">

The bigger problem is that the above image has a caption, which has it’s own style. A style that standard Markdown (and Commonmark) doesn’t support. The straight HTML of the above image would be more accurately rendered as:

<figure>
  <img src="CH01_F04.png">
  <figcaption>Figure 4: Metrics system example, using multiple sources, storing in Prometheus, fronted by a grafana server</figcaption>
</figure>

This HTML block is using HTML5 elements and are quite well supported by screen readers. Markdown was first specified (2004) before HTML5 was created (2008) which is why base Markdown doesn’t have support for <figcaption>. Some Markdown extensions rewrite the ‘title’ to be a figcaption, so check with the maintainers of your document repository for which they do.

Confluence does this natively when inserting an image.

For more on handling images and figures, including what makes good figures, see Handling images and figures.

Concrete examples before abstract systems

Most readers need to start with a simple mental model of how a system works before you can start teaching all the important nuance. Consider the image example from the previous example, repeated here.

Header titled 'example of a metrics system.' Three boxes left to right labeled Emitting Stage, Shipping Stage, and Presentation stage. In the emitting stage are two green boxes labeled Production Software and OS Monitoring (collectD). In the shipping stage are two light green boxes labeled StatsD Exporter (connected to Production Software) and Graphite Exporter (connected to OS Monitoring). Both green boxes are connected to a cylinder labeled 'prometheus storage'. In the Presentation Stage is a green box labeled Grafana Server, connected to the Prometheus cylinder.
Figure 4: Metrics system example, using multiple sources, storing in Prometheus, fronted by a Grafana server

This figure is a concrete example of an existing metrics system. We have specific, named components listed, components readers are likely to have heard about before. The arrows tell the reader the direction metrics flow through the system. The dotted boxes add abstraction concepts, introducing the phrases Emitting, Shipping, and Presentation Stages. This figure is simplified with respect to the overall concept Software Telemetry is teaching, but forms an important foundation.

This and other figures work together to shape definitions and improve the reader’s mental model enough that the following figure immediately makes sense.

'Telemetry Pipeline Stages' in a box in the upper left. In descending green chevrons on the left are the Emitting Stage (top, captioned 'accepts telemetry from production systems and prepares it for use inside the telemetry pipeline'), Shipping Stage (middle, captioned 'processes, transforms, and ultimately stores telemetry for use in the Presentation Stage') and Presentation Stage (bottom, captioned 'presents telemetry to people to support decision-making, drawing on Shipping Stage storage.') On the right side are two descending bars. The upper one is tagged 'telemetry markup' with the caption 'Adding context related details to telemetry to improve understanding of what telemetry is telling you' which overlaps with the Emitting and Shipping Stages. The lower one is tagged 'telemetry enrichment' with the caption 'Transforming telemetry to bring out details embedded within it, such as deserializing JSON, or parsing strings'
Figure 14: Stages of any telemetry pipeline, including processing, markup, and transformation

Chapters 2 and onward reference this diagram a lot, because Chapter 1 was spent using concrete examples like Figure 4 to build up to the concepts.

Concrete examples allow you to teach the abstract systems.

But wait! I like abstractions before concrete examples!

My fellow nerd, I’m like you. However, we’re in the minority. I’m sorry. If it helps, this was one of the harder lessons I had to learn while writing the book. My editor was very patient. She was right.

When to go detailed

While most of the documentation you write will be of the explaining how to fix problems variety, once in a while you’ll need to write far longer volumes on how the architecture works. Such longer documentation is most useful for people new to your team, or who are attempting to onboard your system after a corporate restructuring.

Note

If you are the only person who really understands how something works, write it down. You will be amazed at how much better your own understanding gets after attempting to teach it. It’s better for you to go through this learning process when you have time to write a document than having to hand-hold a new person through learning it. This gives your new person a more coherent message, who will then ask better questions.

Writing the long explainer is a great way for you to remember all the rough edges and tech debt lurking in corners waiting to trap the unwary. I have done deep explainers and come away with a punch list of lint to get out of the way so I don’t have to explain why we do it that way. Such explainers are useful documents, and good link-targets from your break/fix documentation for understanding more about the constraints your system operates under.

Longer documents are also the most likely to rot over time. The long deep-dives need experts to write them, and experts are not great at tracking all the little changes that happen quarter to quarter and back-porting them into documentation. Make it a habit to refresh your long docs twice a year.

Another method for injecting context into a document mostly about “make the pain stop” is to put the added context in a section at the bottom. Perhaps name that section Why it works this way and spend a few paragraphs explaining the details of why. Don’t put this in with the steps, people will skip it. But hopefully enough people will wonder WTF and circle back for the context, and maybe be lured into reading your actual explainers!

Writing for the confused

Confused people can’t track complex sentence structure which means you need to reduce your use of pronouns. Consider the following example paragraph:

When travelling to Bennington you have to be careful of the roads. They’re tricky at the best of times, and a nightmare at the worst.

For prose, this short paragraph works quite well. Look at the second sentence, what does “they’re” refer to? Obviously, “they’re” refers to the roads. However, for a confused person “they’re” could refer to either “roads” or “Bennington.” Is Bennington a town, a person, a place? We don't know what Bennington is beyond the fact Bennington is capitalized.

The phrase I saw quite often in editing was “unclear antecedent,” which is what the previous quote has regarding the word “they’re.” Re-read my explanation again, and see how often I use words like “it”, “that,” and “they’re.” Under standard English grammar for prose, using pronouns like it and that allows you to avoid repetition. When writing for learning, repetition teaches.

  • Each sentence needs to make sense when taken individually, and out of the context of the paragraph.
  • You will repeat words a lot, way more than you’re initially comfortable with.
  • Confused people need repetition.

The next problem is concept stacking, how to order your concept and definition introductions so they build on top of each other as the reader gets through a document. Concept stacking matters more for long documents, not break/fix runbooks. In Concrete examples before abstract systems we saw an example of concept stacking in the form of two images.

The first image, Figure 4, was a concrete example, demonstrating a real telemetry system for gathering metrics with components labeled, and dotted boxes outlining the abstract concepts of Emitting, Shipping, and Presentation stages. The second image, Figure 14, was a mental model with those same stages, but the right side included two new concepts: markup, and enrichment. The concepts of the first allowed introduction of the second, which is concept stacking in practice.

Three practices will help you get the confused out of their unlearning state.

  • Know what your reader already knows, define the rest.
  • Define your terms on first use.
  • Use references to direct readers to more detailed definitions.

Know what your reader knows, define the rest

Before you set out to write, have a good idea of what knowledge your reader is likely to have.

  • For something like API documentation, assume readers know how REST or gRPC APIs work, and nothing more.
  • For something like alarm response runbooks for platform systems, assume readers have basic API understanding (they’re using your systems) and no understanding of nuance.
  • For team member onboarding documentation, assume they know nothing beyond basic citizen of the company stuff and need to get to expert in your stuff fast.

Warning

It is unsafe to assume people not on your team understand the acronyms and concepts you use every day. It is far too easy to work with the guts of a system every day for 18 months and assume the basics of your system are general knowledge.

They aren’t. You wish they were, but they aren’t.

Once you have a good idea what your reader is supposed to know, you can better write your documentation to avoid confusing them further. For documentation intended to be used by internal audiences, it’s safer to assume your knowledge baseline is just finished the onboarding process and projects. This onboarding baseline is likely less than most engineers, but should represent the common baseline for everyone; that’s why we call it onboarding.

For certain systems, you’re only ever used by people already using certain specific other systems. If your stuff is in this class, you can write as if people know the other things too; but you will need to include references to documentation for the upstream dependencies. Anyone attempting to onboard into your team will need to learn your system and the dependencies, but that can’t be helped.

Know who your reader is.

Define your terms on first use

When writing longer documents intended for deep-dives or an extended code-lab type exercise, you can take the time to define your terms in the document. Define the terms the first time you use them, and don’t be afraid to back-reference definitions when you refer to them later on.

For job to be done oriented documents like HOWTOs, alert runbooks, and API references, you may be better served by creating a definitions document that defines the terms and linking to those definitions from where they’re used in the job-oriented documents. Having a common definitions document lets you be more consistent in usage, which will help keep you honest about terms that tend to get overloaded. Overloaded terms are ambiguous terms, and ambiguous terms create confusion. Confused readers land in your Slack/Teams channels asking for hand-holding, and you quietly grumbling about people who don’t read the documentation.

Use references to direct readers to more detailed definitions

Using references for system-specific terms lets you be shorter in your sentences, while giving the reader the option to do a deep dive if they wish. The following example is a short definition for a term, with a link to where to find far more context.

This is an example of a resource attribute. For more on the OpenTelemetry attribute schema, see section 4.3.

Where this OpenTelemetry example is a text callback, you can also use a hyperlink if your system allows links (as I've used several times so far). Using links opens the door to broken-link detectors catching broken callback links, allowing you to fix the links.

References such as these are also a suggestion to readers who keep encountering them that perhaps they should spend the time to read the deep-dive. So much of what we do is persuading readers to do the reading they should be doing, but don’t want to.

Handling images and figures

Most people are visual learners, which means you need to include images and figures in your documentation. When my proposal for Software Telemetry was accepted and I learned that I had to create over 100 images I was aghast, I totally didn’t have that skill. 20 months later when I saw the first physical book, I had created over 260 images (including redrafts) and had strong opinions about what made a good image. A good image should teach a concept, but not work too hard.

Here is one of the first images I made for the book. This never hit publication—this also was before a rebrand of “metadata” to “telemetry”—but I kept this embarrassment around as an example of how I started.

Titled 'Technical organization teams and metadata usage'. Dark green boxes labeled SRE, DevOps, Operations, Customer Support, Software Engineering, plus Security and Compliance connect variously to light green boxes in the middle with names like Centralized Logging, Metrics, Observability, Distributed Tracing, and Continuous Integration. The visual layout is quite confusing.
Originally figure 1, doing way too much, in a confusing way.

This is terrible for many reasons, let me point them out:

  • No consistent direction of flow. What there exists is roughly ‘inward’
  • Lines cross. You never want to see lines crossing if you can avoid it, they confuse people
  • Default colors. It’s hard to see scaled down like this, but each of those boxes has a blue border on it, which was the default line-color of the tool I was using. The blue was not complimentary to the green hues of the rest of the figure.

The replacement figures were a series of similarly formatted figures, but with far more precision. This next image is better constructed, I’ll go over how that is after you see it.

At the top a yellow box with a black border labeled 'Software Engineering' and a row of dark green boxes below named Centralized Logging, Metrics, Distributed Tracing, and SIEM. There is a right-angle line connecting the Software Engineering box to everything except SIEM.
Figure 10, telemetry system styles used by Software Engineering teams

You can see some of the heritage, but this image is much cleaner:

  • Consistent flow. Flow is top down and never changes.
  • No crossing lines. This uses a bus layout to simplify the chart rather than using a star layout like the original. Right angles makes things feel cleaner.
  • Better color choices. The colors here pay more attention to contrast ratios. Where the original used white text in a light green box for some elements, which is not readable, this version appropriately uses white or black text depending on what the background color is.
  • Consistent color theme. All the colors in the book come from the same palate. Your documents will be much smaller, but using the same colors to mean the same things across your images will make your images better teaching tools.
  • Used with others like it. There were a series of figures formatted just like this one, but with the name of the team changed. The old Figure 1 was replaced with a series of four and was a much better teaching tool.

We’ll get into more about styling great images in How to style images and figures. First, you need to learn about when images are needed.

When to use images and figures

You need to use an image for several reasons:

  • To provide a visual example of a flow. Most people are visual learners, so having a visualization of flow better cements the concepts in their head.
  • To provide context about relationships. Knowing what links to which helps cement concepts.
  • To provide a visual mental model. Figure 14 from Concrete examples before abstract systems is one I used extensively in the book, you will make your own.
  • To hammer a point home harder than a callout would. Sometimes a concept is so important you need a callout and an image.

Manning’s style-guide requires a figure by any code examples! Their reasoning is that all code describes a flow, and you need to visualize flows. Remember, when Writing for the confused repetition teaches, and you need to provide methods to get readers out of the confused state. This next image is an example of a flow diagram for code.

A multi-stage flow-chart showing the initial stage, using redis_client.blpop. Diamond decision points are IF statements in code with Yes and No paths leading to actions. The end state is either restarting the flow chart or writing data to Elasticsearch.
Figure 3.6: Flow of an event accumulator loop to take advantage of bulk write

I needed to create figures such as 3.6 to comply with a style guide, you may need to do the same when writing something like a demo project to give people a way to experiment with your service. The more a reader needs detailed understanding about code, the more likely you should create a flow diagram for code.

This next example is a visual breakdown of a function call, which is sometimes called for when you’re in the weeds of explaining something.

A line of code, redis_client.xreadgroup, with four arguments. Annotation arrows and brackets point to each argument to define what each argument is for.
Figure 3.10: Components of a streaming read

This particular example is describing how to read from a Redis-based stream, where new events can come in at any time. The callouts are using thick lines and brackets due to the Manning style guide for figures of this sort, but point to components of a function-call and describe what each element does. The text should define what each of the annotations means. This same style can be used to provide matching clues. This next image is drawn from my Regular Expressions chapter and documents how a given string matches against an expression.

The top of the flow diagram is the text 'Account 1141 deleted from zone 37' which links to a rather longer regular expression. Arrows and brackets link components of the regular expression to matched elements. Below the regular expression is a gloss linking to each statement and decodes what text it is intended to match.
Figure 11.2: How an expression matches an example string, and what the resulting capture groups contain

This image is doing more work than the first example of a breakout, but Regular Expressions can be a tricky topic to cover so I needed to give visual examples for how matching works. Because Regular Expressions are code, Figure 11.2 is a flow diagram; but one having an example to work with.

If you are writing a demo project where people are supposed to get a specific result at the end, images of the above styles are useful ways to describe how the code is supposed to work, and what conditions the code checks along the way. If nothing else, your demo-project person can check their own code to make sure they’re catching all the cases they should be checking. If something goes wrong in your demo-project reader's attempt, diagrams like the above help them refine understanding of their own code and how your system works.

Sometimes, you need to underline an important concept in the most strenuous way possible, and images are a good way to get people to pay attention. The following is a warning about the penalties of mishandling personal and private information.

Titled 'The penalties for mishandling privacy and health-related data'. At the top is a green segment of pipe labeled 'telemetry pipeline' with a blue leak in the center and falling drops. The drops are labeled, 'Ban your use of that data', 'Great big fines', 'Public admission of guilt', 'Assign civil liability', and 'Increased regulatory scrutiny'.
Figure 16.1: The penalties for mishandling private data are severe, you should pay attention.

As this was the first figure of chapter 16, I needed to convince people to pay attention to the topic of data safety with regards to regulatory enforcement. Getting everything in that figure correct took me a couple of hours, but could save millions in enforcement actions if I did it right! Worth the time.

Now that we know why we need images and figures, let’s talk about how to effectively style them.

How to style images and figures

The introduction gave us some clues to style already:

  • Consistent flow. Flow lines should move consistently.
  • No crossing lines. Flow lines should not cross. When lines cross, people get confused. If you have to cross lines, ensure they’re styled quite differently; such as crossing a dashed line with a solid line.
  • Use contrasting shape and text colors. You should use black or white for text, whichever gives better contrast versus the shape color.
  • Use a consistent color theme. Using the same colors to mean the same things across your images will make your images better teaching tools.

Let’s next look at an example that uses annotations, directional arrows to describe components, which will let us cover a few more consistency topics.

Left side cloud labeled 'AWS CloudTrail' has two right angle lines leading to green 'S3 Bucket' and 'SNS'. Right angle lines lead from 'S3 Bucket' and 'SNS' to dark green 'Telemetry Saas Provider'. Numbered annotations point using dotted lines to various right angle black lines. '1: Audit logs enter bucket' points to the line linking Cloudtrail to S3. '2: Notice of new audit logs' connects Cloudtrail to SNS. '3: Receives notice' connects SNS to 'Telemetry SaaS Provider'. '4: Fetches new logs' connects S3 to 'Telemetry SaaS Provider'
Figure 2.13: Flow of logs to a SaaS-based telemetry provider from AWS CloudTrail

Figure 2.13 follows all of the directives we’ve seen already. There are several new things I want to draw your attention to:

  • Different line styles for flow vs annotation lines. I used straight, right-angle connectors for flow, and dotted, curved connectors for annotations. I wanted as little confusion as possible between the two styles.
  • Annotations use a different font than element labels. To make annotations even more visually distinct, use a different font! If your tools allow it, use different arrow-pointer styles to make it even more obvious.
  • Numbered annotations to describe the flow. The lines themselves look like two parallel flows, but the numbers tell the reader the order of operations.
  • Consistent colors. We see a return of the light green in this example, this time bordered by dark green.

After this image I wrote down the flow again, but used more words to clarify nuance. Also, screen-readers won’t be able to read inside images to describe a flow; meaning screen-reader using engineers will be better served through duplicating the information in text.

Remember: repetition teaches.

Let’s take another look at Figure 11.2 and pay attention to the use of flow direction and contrast.

Regular expression breakdown diagram linking the text 'Account 1141 deleted from zone 37' to a regular expression. Bolded text links text to its capture group such as linking '1141' to '(?<acct_id>\d+)'. At the bottom of the diagram is a box with capture groups and their text, such as 'acct_id = 1141'
Figure 11.2: How an expression matches an example string, and what the resulting capture groups contain
  • Flow is top-down. This is a consistent direction, and called for to keep the image within a column of text.
  • Bold text tracks Regular Expression capture groups. The example text at the top bolds the text that matches our capture groups, which links the concepts of the Regular Expressions with their text.
  • Arrows link Regular Expression capture groups to their extracted values. Arrows finish the association of text to Regular Expression, to the extracted value.
  • Crossing lines use different styles. This example has a crossing line, which I said to avoid! The lines being crossed are the straight annotation lines with the curved flow-lines, and they’re using different weights. The risk of confusing the lines is greatly reduced with all the differences.

This next example demonstrates use of flow, and style choices when you’re making a series of images.

Left to right diagram with a box titled 'Retention policy.' Dotted-boxes labeled 'Logstash shipping' and 'Beats shipping' link to dotted-box 'Logstash enrichment'. Dotted-box 'logstash enrichment' links to green cylinder with solid lines labeled 'Elasticsearch'. A right angle line links dark-green 'Kibana' to Elasticsearch'. Directly below 'Elasticsearch' is another cylinder titled 'Azure Cloud Storage' connected to Elasticsearch with curved lines, bearing annotations 'Backup' and 'Restores'.
Figure 18.2: How retention policies work with an ELK stack, with elements we need to know about, but don't need to talk about, using alternate styling
  • Alternate style for shapes not being discussed. In this case, earlier figures described what they were and why they were there. However, for this figure, we don’t need to discuss them but need to include them for consistency. I used an alternate line style, dashed instead of solid, and muted colors versus the standard used throughout to further make clear we don’t care about these elements right now.
  • Split flow to make a point. Our telemetry flow continues to use right-angle connectors, but our backup/restore flows use curved lined to make it clear there is an alternate flow in play.

A note about line styles

Most diagramming tools let you pick from a number of line styles from solid, to dotted, to dashed, to long-dashed, to dot-dashed, and so on. When building a multi-flow diagram, be careful with how many line styles you use! For best results, use at most three styles. Any more than that, and people will have a hard time telling the lines apart.

Image rules summary

When to use images:

  • To provide a visual example of a flow. Most people are visual learners, so having a visualization of flow better cements the concepts in their head.
  • To provide context about relationships. Knowing what links to which helps cement concepts.
  • To provide a visual mental model. Figure 14 from Concrete examples before abstract systems is one I used extensively in the book, you will make your own.
  • To hammer a point home harder than a callout would. Sometimes a concept is so important you need a callout and an image.

How to style images:

  • Consistent flow. Flow lines should move consistently.
  • No crossing lines. Flow lines should not cross. When lines cross, people get confused. If you have to cross lines, ensure they’re styled quite differently; such as crossing a dashed line with a solid line.
  • Use contrasting shape and text colors. You should use black or white for text, whichever gives better contrast versus the shape color.
  • Use a consistent color theme. Using the same colors to mean the same things across your images will make your images better teaching tools.
  • Different line styles for flow vs annotation lines. You want as much style difference as you can get between flow lines and lines coming from annotations.
  • Annotations use a different font than element labels. To make annotations even more visually distinct, use a different font! If your tools allow it, use different arrow-pointer styles to make it even more obvious.
  • Use numbered annotations to describe the flow. Numbers are intuitive, which helps learning.
  • When making a series of images, use muted colors and alternate line styles for elements not under discussion. Consistency means carrying over shapes from previous images, but styling them so they’re obviously not the topic will help learning.

Making internal documentation better as a practice

Improving documentation practices all comes down to what the tech industry believes is a full time occupation, and how much power they’re willing to invest in the documentation function. In an industry where Quality Engineers keep being moved from full time to contract, if not fully off-shored, treating internal documentation quality with the same technical rigor we use to assure code quality is quite rare.

You see, a high quality internal documentation function requires several things that aren’t generally present in most tech companies:

  • Technical leadership willing to sacrifice feature delivery velocity to make high quality internal documentation.
  • Product Management expectation that some sprint capacity shall be spent maintaining documentation, not on internal or revenue features. The industry has a hard enough time keeping up with tech debt and CI system rot.
  • Sufficient editorial staff to provide the documentation equivalent of code-review to level up overall documentation craft. The industry stopped hiring full time internal tech writers over a decade ago, moving them to contract work or off-shoring them, and tech writers are the people with the skills to do this editorial work. Also people like me who have a side gig writing books can do this, but making this my Day Job means I’ll be right there with the QA team when the budget crunch comes.
  • A document publication system that allows pre-update reviews, similar to the PR review cycle, but for documentation. Such a system needs to support pop-out comments, suggested edits viewable inline, and acceptance interfaces for individual line-edit suggestions. Documentation review needs to be fully rendered. Nearly all git-based tools don’t work for this.

Once you have the above you will have a truly top quality documentation function. I hope I get to see one like it some day. Maybe in the recovery after the AI bubble deflates.

History: what my technical writer friends tell me

The job title of Technical Writer is a vanishing one in tech companies in the United States. Around 2012-2014 the industry decided as a whole that internal documentation could be handled by engineers, and external documentation could also be handled by engineers with extra bits added by newly created Developer Relations people. The Technical Writers that used to be needed to create things like release notes and installation guides were let go, or replaced with 1099 contractors.

Certain functions like regulatory filings or documents used as part of compliance processes still need to be created. Such documentation is now written by people dedicated to those roles, or as part of short term contracts. SaaS companies largely got out of the business of professionally writing for technical audiences.

However, these same companies forgot (because it isn’t efficient) that they have large technical populations internally. The larger companies even have whole divisions, currently called Platform, with the duty to create technical platforms that revenue-teams need to build the product. These internal platforms need documentation too, and we no longer have the base talent to do the job in an expert way. No one is satisfied with the state of internal documentation.

The US-based Tech Writers of old found new jobs.

  • Some became engineers; since it was obvious that engineer was the only respected job in the industry, and being a good Tech Writer involves most of the skills of an engineer.
  • Some became Quality Engineers; because writing documentation is a great way to find problems, and that’s a transferable skill.
  • Some moved into the new DevRel jobs; because DevRel was still writing, and definitely about helping people. And helping people is what technical writing is all about.

The Pandemic recession of 2020-2021 and the collapse of the technical conference circuit that came with it ended a lot of DevRel jobs. Quality Engineers have seen a great off-shoring in the 2020-2025 era. For those former Tech Writers that chose the engineer’s path, they’re the engineers that get cranky about making sure customers have good documentation. A small few have gone into book writing, as the last bastion of full expertise tech writing in the industry.

For better or worse, Software Engineers need to make their own documentation and our software factories aren’t designed to foster that skill.