First, introduce The Diataxis framework ( https://diataxis.fr/ ) for documentation. It makes people think about documentation in a more structured way, and allows you to be more specific in the types of missing documentation. (High documentation cultures are often good with explanation but not tutorials, for example.)

Second, I would introduct the idea of a Documentation Portfolio. I have a review of Agile Documentation at https://www.ebiester.com/documentation/2020/06/02/agile-docu... and it speaks to another structure for how to build the documentation in a more reliable form and thinking more carefully about your audience for a particular type of documentation.

Not every issue needs to be solved by a butter robot.

Why not employ a technical writer/documenter/whatever job title you like, even as a temp, whose sole job is to sort out the mess of documentation you have and then to write new documentation as you move forward?

Primarily because it's a far, far more complicated job than that and you can't really hire someone off the street to do it effectively. Typically in a tech company a tech writer is going to know almost as much or more (after years of experience diving into every detail) about a given technology or application or API, and so that begs the question why not make twice as much working as a software developer and not have to sort out these types of messes?

Also job security. Anyone doing this work full-time is the first on the chopping block, and developers who are working on documentation tend to be perceived as lower status since they aren't delivering features.

But a different question is, why is no company trying to do this differently? Like, hiring one good tech writer to maintain the company documentation, and paying them as much as they pay the developers.

I once worked at a company - in a different domain - that made a conscious decision to make this kind of hire. It worked incredibly well, and I never understood why more companies didn't do it.

The context in my case was the Australian offices of a management consulting firm (BCG). The Melbourne and Sydney offices hired what were called "editors", brought on at the same grade as the consultants. Not editing as in correcting grammar. But helping the consultants improve the logic of the arguments in their slide decks: so they were logically consistent, easy to understand, and actually addressed the clients' issues. I was a junior consultant back then, and we were constantly pushed by our managers "have you seen Yvonne?" [the Melbourne editor] when preparing for major presentations.

I do like the sibling comment calling for a librarian. I imagine that would pay a ton of dividends if the librarian was motivated and got support.

There are three things that I think are preventing technical writing from being more widely valued:

1. Software companies tend not to distinguish between technical writers who are good at English vs. technical writers who are good at engineering, understand their audience, and can articulate complex ideas to that audience effectively.

2. Technical writers who are good at English make about half as much as technical writers with engineering skills, but they also muddy the hiring waters and drag salaries down for everyone else.

3. Most corporate-people think because they can type up a decent email they can write technical documentation themselves. They're usually wrong on both counts.

He assumes that "full understanding (into every detail) of what is being documented is needed" (as I put it). So, the new hire will never get it right 100%. he will both struggle and annoy others (to forever enlighten him), which is a fair point.

But it is not black or white. Others here have more positive experiences

This comment is absolutely true and many, of not most, companies fail to understand it. I think the problem stems from corporate-people thinking, "Why should I pay a writer when we all speak English (or whatever language) and can write it ourselves." And that's why so many companies have shitty documentation.

> ...so that begs the question why not make twice as much working as a software developer and not have to sort out these types of messes?

I was a software engineer for 30+ years and got completely burned out on it, so I left engineering to do technical writing. So far, I like it much better because I have far more control over my time. In my experience so far, the sorting-out-messes work is about the same in either field. Both jobs are pretty complex. I also make exactly the same as I did while working as an engineer.

I think the secret to not being first on the chopping block is to show you're delivering value to customers and internal teams. At least I attribute that to my survival through multiple layoffs so far.

How did you make this transition? Any credentials/certifications you needed? Did you transition within the same company?

So, to solve the first staleness part, there is only two ways: raise the documenter status, or make it somewhat possible (easier?) for someone else to do at least a part of it. are they both really that hopeless?

PS: to solve the second loss/discovery part, I think we are heading for that AI powered simple "unified search" experience.

Yes, there's a very good reason not to use metadata: it's extra work, and it's not very fun, just like writing docs for software. So people don't want to do it because it isn't "sexy" (and there aren't very good incentives to overcome people's reluctance to do that work).

Because of this, just like any job that people don't really want to do, you have to "bully and cajole" them into doing it.

If you don't include the right metadata, google won't rank you highly. If you include the right metadata, your content will get higher rankings and the snazzy preview cards on different social media platforms.

Metadata is good! There are structural incentives to be mediocre though.

Second, the whole point of AI NLP search (i.e., 2nd part = loss) is that it does not need metadata (which was the basis of the now mostly abandoned semantic web approach to KM).

1) Link to the documentation in your tools

2) Ensure everyone has edit access to the linked documents

Another idea might be, whenever a new feature is closed out, auto-allocate some percentage of its implementation time to documentation, and schedule an interview with a tech writer.

Library science is a popular area of study but the job market isn’t great and neither is the pay. Lots of people to choose from here even without poaching from existing libraries.

I disagree, I've been apart of a few companies that have done exactly this. I would suggest not presenting this argument as an absolutism.

You partly answered your own question: perhaps you pay the librarian/documentation writer too little? ;-)

Seriously: letting the documentation to be written by such a person won't be as much a cost-reducing measure, but instead mostly an approach to improve the documentation quality.

This isn’t the case for external documentation, that has to be more polished, needs sign offs and images and demos and stuff - tech writers can come in useful here

So even on those rare occasions when they do actually document something, the documentation tends to be pretty bare-bones and not very readable.

Good technical writers are worth their weight in gold.

If I'm working on a library, design docs and commented code is nice though.

They're not, actually, as is obvious if you actually read the links. They're human process approaches.

In this sense it was meant that the technological solution - auto documenter tools, etc - they're the butter robots. I was making the point that rather than just shim Yet Another Tool into the stack to do a single but important job (pass the butter / write the docs) perhaps give it to a human who will a) do a much better job and b) reduce the complexity of the stack.

Apologies for the confusion.

I'm surprised how this comment thread took off. Looks like there's plenty of support for AND against. I simply meant to make the point that for some problems humans do better than machines and it can be more efficient in the long term to look away from a tech solution to a human problem.

This is about prioritizing the documentation you write.

Who would supply that person with the information required to write the documentation?

In my opinion, companies should hire subject matter experts who can write rather than just someone with an English degree. I've fixed a lot of terrible documentation written by English majors with no engineering background.

But according to the network graph on GitHub, Diataxis seems to be more active, although both of them still receive updates.

divio/diataxis-documentation-framework is forked from evildmp/diataxis-documentation-framework, and evildmp is Daniele Procida, the creator of the latter repository and the maintainer of diataxis.fr.

Thanks so much for the link, I wish I'd had that chart ten years ago!

Adding a caption here for anyone on a screen-reader, before I give commentary on it:

  * X-axis: "serve our study" vs "serve our work"
  * Y-axis: "practical steps" vs "theoretical knowledge"
  * which gives 4 quadrants: how-to guides, tutorials, explanation, and reference

Also, I wonder if the relative distributions of each one could tell you something about the team's health — or even just the kind of work the team does?

For example, between two teams who are both documentation-heavy, what does it means if one team's docbase is 80% tutorials, whereas the other team's is 80% reference guides? It would be fascinating if anyone's already given thought to relative metrics/heuristics like this.

How-tos are an interesting case. I see a lot of informal how-tos in slack. However, how-tos also are the most likely to become stale because if they're needed often enough, they tend to become automated in part or whole. It is by its very nature transitory.

I see how it applies to documents which describe things as they are, but I'm curious how it would classify forward looking documents like technical designs, strategy and vision documents, roadmaps, and mission statements.

There's your problem. The only use case for Confluence is when you want to hide information, but credibly claim that it's documented.

The primary failure mode I see is when people just throw random documents into Confluence wherever convenient at time of writing and never go back to logically organize anything. One symptom of this is when key information is being recorded in a hundred different people’s “Personal Space”

Taking even half a day to organize the average Confluence makes a huge difference.

There isn't a universal note taking application that comes pre-organized for your team's use case. You have to put some work into any tool you use.

Most teams and companies aren't special snowflakes that need individualized organizations, and document hierarchies. There can be such a thing as sensible defaults that you customize or tweak later (no idea if Confluence ships with that - I've only ever seen Confluence installations in their already-screwed-up state). At the same time, an inexperienced user staring at a fresh Confluence install isn't going to get the organization correct right off the bat.

If you have to put in work upfront before the tool is even halfway useful, it better be really damn good after that. Confluence is not.

You would not believe how special some use-cases are, especially when you work with organisations that have highly regulated environments. I've seen anything from markdown files in a git repository being semiautomatically created in a Jenkins run to an organization having built essentially their own wiki software because nothing on the market fulfilled their need at the time (now 5 years later, they realise no-one uses that thing because it is just unintuitive). I have seen organisations that have no content oversight and some who had a whole department of "content czars", whose sole job it was to keep their documentation fresh and updated. I've seen organisations that had strict rules on approving each individual change, with complex approval workflows.

If you have never documented anything, Confluence may be overwhelming, but so will every tool that has "sensible defaults", because before too long, you will start hitting the envelope. Documentation software is not like a MacBook that you just buy and start using, you always need some level of customisation.

So, is Confluence damn good? No - there's a lot that could be improved. But from the mediocre solutions on the market today, it is one of the better choices.

I personally have never seen it work, but of course I've also only seen a handful of data points.

It's incumbent upon all users or members of the team to use the common tool along with agreed upon standards. Otherwise even if you wrote documentation in your own hemoglobin, no one would touch it either.

Some manager prob chose _________ as the tool for ticketing, documentation, etc not because it was good at ______, or _______ but because it fulfilled their action plan to have something, anything in place so that if the universe goes supernova, well some stuff was written down.

In my journey it seems that nobody is willing to criticize Edward Teach for the lousy treasure map he left, but rather we make fun of those who're still looking for his stuff.

That's the average wiki. It's a commons and a tragic one. To make docs work you have to treat it more like a codebase: clear ownership, standards, review processes, approvals, up front design, refactoring efforts etc.

Maybe true in large orgs.

But for smaller companies what I've seen is usually paralysis.

e.g. someone notes a problem (maybe just a typo) in the doc. Can they fix it within seconds? If instead they need to raise a ticket then most likely it ain't happening. They move on, and the next person experiences the same problem.

IMO the default should indeed be towards everyone committing at will. Yes that will result in the occasional snafu. Fix that when it happens. (obviously not good practice for the operating manual for a nuclear power plant - but for a <500 person Saas company it is).

Mandating a Jira ticket for simple typo fixes is overkill. But if you make it easy to create a PR directly on the documentation file, without leaving the tab, I don't see an issue. This is already a Github feature.

Overall the comments on this page fall into 2 camps, people who've tried it all and found what works is discipline and those who are still trying it all.

The default for Confluence is just that, everyone commits at will. There is no structure, tons of duplication, no standards when it comes to naming, formatting, audience, etc. I'm a huge fan of markdown/plain-text solutions, only because linters can be run that force you down one happy path. I don't believe Confluence has linters at all.

A ticket represents a process (otherwise it has no added value over git commit message) and thus creates much more work than a couple of seconds.

Yep, and that process also involves other people, to review/ approve the fix to the typo.

It then goes from being a few seconds of elapsed time and actual time (to just commit a fix to the typo) to taking hours, days or weeks of elapsed time and hours of actual time and forcing context switching on, and interrupting the workflow of, all of people involved.

It's hard enough to get technical minded people to contribute to a git (or style) based knowledge base.

Pick your poison I guess but I'm quite happy to have testers/BAs/directors/etc able to quickly jot down thoughts roughly than have it disappear into the ether.

A better solution might be that anyone can write the documentation, and there is a maintainer who constantly refactors the wiki to keep it legible. Makes sure the information is not duplicated, adds hyperlinks to things, etc.

Why do you hire software developers, instead of making software development everyone's responsibility? Is that because most people suck at software development? Well, most people suck at writing documentation, too.

It was a directory of files - I think plain text with a few wiki shortcuts, but might have been some sort of early Markdown.

The editing form was basically a text area on top of mercurial.

Similarly things like the edit log were basically dumping the mercurial output into html.

No clue how long it lasted, but it was still in regular use when I left in 2012.

Wrote the whole thing in a few afternoons.

Confluence and other wiki systems are clearly meant for longer-lived documentation and canonical information. You should link from or attach your working documents (spreadsheets, slide decks etc) to your wiki documentation for people to discover why certain decisions were taken. But if the wiki's discoverability is poor or it's not well-maintained or regularly reviewed, it's basically useless.

Protip: Use wildcards extensively. Differently from Google, Confluence search considers keywords entered to be limited by word delimiters. so foo matches only foo, not barfoo or footer. Use them with some wildcards, foo, and the search results starts to make sense.

It also weights titles incredibly heavy, which combined with the previous part led to me not even noticing it searched the document body for years.

link: https://cloudsearch.google.com

[1] https://github.com/kovetskiy/mark

There have been times I know a page exists and I even know the title/content of the page and yet still I am unable to find it via the search.

That describes Sharepoint more than Confluence.

That's it. No mystery.

(BTW, this would also be why company financial records would suck, if management decided to save money on accounting staff and have all employees just kinda do their own accounting for the company. I SAY HIRE A SCRIBE FOR EVERY TECHNICAL TEAM!)

This. Yes, having everything written down and searchable is definitely a good goal. However, in my experience, the people in most companies have very different skills and few are good writers. So it probably takes a lot of time to create an organization that has a good process for creating great documents, let alone to transform an existing organization which can do so.

Duplication can be useful. But the more of it you have, the greater the maintenance burden is. (The main exception is documentation that is not supposed to be kept up to date, like a daily journal or blog posts.) So I think it behooves people to be very careful about adding documentation. Because as you say, it can turn 1 problem into n problems.

The lingua franca of ideas is natural language.

Usually tickets are written, code is changed. Updating existing documentation is an afterthought at best.

Personally I prefer any formal or semi-formal documentation (e.g. Swagger) over a Confluence page any time of the day.

For example, one could structure things where the English documentation is the deliverable. The code merely serves to actualize the document. In this world, we would consider the act of writing documentation of paramout importance, whereas the code is an implementation detail.

I think software as a discipline is distinctly undiscipled about these sorts of concepts.

Documentation is useful when it's job is to help understand why the code is the way it is, what problems are trying to be solved, and what constraints the devs had.

I realized one day that the specs, tests and how-to markdown documentation I wrote all used the same examples.

From that I derived the idea to create a "spec" DSL that could both be run as a test and generate markdown (with screenshots, etc.) to make nice high level how-tos.

Cucumber has the same sort of idea but the DSL really isn't suitable.

Personally, I don't think documents have intent. People have intent. Sometimes they write down some words as an approximate way to communicate their intent in that moment. But if there is a conflict between the written document and the actual intent, it's the human intent that matters.

I also think that drift between written description and actual intent is good and desirable, because it usually means that the humans learned something about what actually needs to happen. So to the extent that documentation acts as leg irons for intent, I'm generally opposed to it. The faster our teams learn, the better the things we make.

Confluence has been the bane of my attempts in finding any relevant docs. Which one is the source of truth? Which one was a draft written by an overly eager to make a first impression, new employee (who is no longer with the company)? Don't even get me started on saving meeting notes to confluence.

These days, I maintain my own knowledge base on Obsidian. If there's ever any confusion or request for more information within the company, I copy-pasta the relevant note from my obsidian bank to whomever person or whichever confluence page they deem the source of truth.

Toplevel: - Work -- Job A: -- Daily notes -- Services -- Auth --- overview --- login flow -- Client -- Logger -- Job B: -- Daily notes -- Architecture -- node -- react -- etc

(edit: sorry about the formatting)

As the scope of your work expands, you add another sub-directory or file where necessary. Once it starts to grow in size, you can start making insightful connections via [[keyword]].

Furthermore, you can pretty much take this knowledge base with you, wherever you go, by uploading the vault file to your google drive and accessing it locally via SMB. Automatic save/backup.

Design docs for each and every feature has turned out not to scale for my current team. Larger, multi team features demand consolidated documentation, but for internal changes we rely on quick meetings as code reviews. Part of me misses the ceremony of the round table discussions, but the real difficulty is keeping track of why changes happen. Documenting processes and cross cutting concerns is a must have, but keeping track of all changes across quickly moving teams… it’s no surprise so many teams are just rife with tribal knowledge.

Are you saying this person should be hired/paid to store and retrieve the information being sought out?

I have worked with teams that focus so much time on design docs and insist that everything has to be documented. Pace of work is slow. Documentation and designs became obsolete due to shutting down of services, change in architecture, refactors.

The best form of documentation is code.

Documentation as code might be a good alternative path, using the same tools and processes as software development and embed documentation tasks into engineering workflows. More insights in https://about.gitlab.com/blog/2022/10/12/five-fast-facts-abo... how the GitLab technical writing team collaborates.

Note: GitLab team member here.

> Occasionally if you ask how to actually do a task in Slack someone will yell at you that you should have searched for a specific, obscurely named document in Google Drive, Confluence, or Github.

When I'm the person being asked and know of the doc in question, here's what I try to do instead: I ask where someone searched for it. Then I update that place to refer to the correct document (and do some light refresh on the doc as needed). This works whether or not they tried to look before asking. If they did, well, now the next person who does that will just find it. If they didn't, I'm making them look before getting an answer. Maybe they find it, maybe they don't, either way I'll help them in the end if they're willing to look.

What do you mean it can't find anything?

- Can't find "Technical Specification" in the "<Cool Project>" folder.

- Can't find "Tech Spec" (and vice versa)

- Can't find "<CoolProject>"

Is there some "enable real search functionality" checkbox I've missed, or am I just doing it wrong?

Search is keyword-based, like large-scale search is pretty much anywhere. Expecting "specifications" to match "spec" is expecting too much, same as expecting half your search to match a folder and the other half to match a file the folder is in.

The main thing to keep in mind is that search is content-based, not just filename. So instead, search for key terms you think are in the file, as opposed to focusing on folders/filenames. Start with one or two, then modify or add as necessary to narrow down.

(I currently work at Google, but not in Search or Drive / Docs...)

When I use Google to search the whole web, it DOES match. So how come it doesn't work when I search a small document?

And if that's ambiguous, then ask how often your company penalizes people for making the common but wrong choice versus the uncommon but wrong choice.

You may still not know how to find anything, but they will.

Like a lot of other things, this has suffered from computerization making it yet another small part of everyone's job (which also increases context switching, the amount of shit you need to know and keep track of, and generally makes jobs more stressful) rather than a specialty that's the main focus of a few workers.

The benefit (get to stop paying some employees) is easy to measure, while the harm is not.

Nicely said. I am getting really stressed out how complex things are becoming. It's already hard to keep up with git, Jira, Bitbucket, AWS, k8s, Helm, JS frameworks, databases and whatsoever. But then add hard to use document management systems and f...ed up processes that are mainly designed for management to get nice reports and not for productivity. Now you are a bad developer and you are a bad document management person because it's simply impossible to be good at all this stuff.

I am constantly preaching to management that we need specialized tech writers and specialized devs that are good at their respective. But I guess it looks cheaper to waste time and energy of engineers on stuff they aren't good at.

The exception is daily supply chain and accounting, which due to factors like urgency, multiple stakeholders per order, high pace of handover, external system integration, multilingual presentation requirements and nontechnical users we prefer a dedicated web based system with more of a real time focus with event hooks (eg. notification, translation, verification).

Nobody knows how to accomplish this.

Whatever the solution is, if one exists, I'm sure it involves a lot of work keeping documentation up to date.

Out of date doc is worse than no doc, because it makes you feel all warm and fuzzy right until you footgun.

How is this not inevitable if your goal is to always write things down? It seems like the way for document to be accurate is to keep the scope small and if you want everything in scope then it's going to contain a lot of outdated information.

There's no easy answer.

I mean, you're clearly not describing a high documentation culture, you're describing a culture that underinvests in intra-organizational communication.

I run a high documentation, low meeting culture by necessity (we operate in five time zones around world). Meetings vs docs is remarkably similar to the decision between paying for office space vs paying for occasional team retreats. If you run a fully remote company retreats are almost always a better use of your money than leasing office space. But you still need to pay for something.

Similarly with meetings vs. process documentation. If you're heavily remote and spread out I'd say you should cut down on meetings and being high documentation is the better choice. But again you still need to "pay" for something - you save time on meetings but you need to reinvest at least part of that time into writing documents.

Another bonus of documents is that they scale better than meetings. McDonald's doesn't deliver the same big Mac in every corner of the world by holding a lot of meetings. They have a book that goes out to all of their thousands of franchisees. When they want to add another thousand franchisees, they print more books.

If the documents are out of date my answer to my team is always "update them!" Anywhere that we're writing documents there are revision and discussion features so it's not like you can irrevocably screw something up, just improve it and let us know what you did. I do struggle with getting people to actually do it though.

Obviously this doesn't apply to public facing codebases. But trying to keep an internal codebase documented, other than fully finished self contained library level code, is a sisyphean task.

I'm not checking confluence, write it in a ".md" file in the repo if you want me to see it.

I think that high documentation can work BUT the company has to invest in it in the way that Digital Ocean or even Stripe has done outwardly.

1. Investment - You have to hire at least a few technical writers and librarians to provide training & cleanup functions.

2. Management buy-in - You have to budget for it and encourage it through day one communication (ie. New hire training) and consistently rewarding and recognizing people for getting it right.

A lot comes from tribal knowledge/who to talk to/etc.

People need to actually learn how to write and maintain(!) documentation, otherwise it’s just a huge chaos.

Rule 1: less (text) is more.

Would that place be the U.S. Department of Defense by chance?

Are people at the same job telling you to check 3 different sources for internal docs? Maybe that is the main issue. Put knowledge in one place.

More specifically: One place that is not Sharepoint.

Can all your engineers see all your other engineers' code? It's hard enough to get code to do what it says it does; I've very rarely seen documentation that's correct.