From Word Docs in a File to a Non-Linear Visual Knowledge Base using Markdown

Our team maintains a library of basic research about eLearning vendors we’ve worked with, or may explore in future. Until now, these files were stored in Microsoft Office docs. When I was asked to review and revise them, I decided to convert them into a non-linear, visual knowledge base using markdown files linked together in the Foam plugin for VS Code.
This post outlines my thinking about how and why you might want to turn a collection of information into a knowledge base like this.

Prior State

Our Instructional Design team created detailed research records about the learning design vendors we use and their capabilities, as well as others we may be aware of but haven’t worked with yet. In times when unique project requirements arise, we go back to this Vendor Research file to see who we know who could deliver great results for our stakeholders.

These files were developed as a series of Word docs, and my big frustration has always been that we can’t see through them to the precious data within. When we want to dig into the Vendor Research, it involves opening one Word doc after another to see who would be best for video development, for example. Each file includes a standardized checklist of various capabilities like “narration” or “motion graphics”, and you have to open each file to see what it says. They’re not searchable, so I usually just find myself looking at the names of the closed files and trying to remember what they say rather than bothering to dig into the data within.

For those new team members who were not present in 2018 during our big push to establish this research, I fear this information is not accessible or easy to consume for them either, so I want to improve the overall accessibility of the tool.

Envisioning a Better Alternative

I love modern tools like wikis, blogs, and database-driven knowledge bases because they allow you to freely alternate between writing content and organizing it. Modern applications can look at a folder full of markdown-formatted text files and display it as a blog, a wiki, or any number of other content management systems. Such a system enables you to connect content via tagging, interlinking, and make the whole thing searchable — a huge improvement over discrete files that don’t talk to each other.

I wanted to bring these organization tools to our Vendor Research docs, so I converted them to markdown format and opened them up in the personal knowledge base app Foam, a plugin for VS Code that’s designed to offer the same functionality as Roam Research from within VS Code, for free.

This flat file wiki database lets you link pages together via simple #hashtags — if that hashtag appears anywhere within a doc, it becomes a searchable tag across your knowledge base, connecting all docs where it appears. I converted all the table-based checkboxes about each vendor’s core capabilities to a simple hashtag, signifying that a vendor offers that service.

This enables Foam to generate this detailed “X-ray” graph view of all vendor records in the project, immediately surfacing all instances where each capability occurs.

From this view, we can click on a capability and see which vendor offers it,
Click on a vendor and see all the capabilities they offer,
And see a bird’s eye view of all vendors and their capabilities.

If the graph view is too chaotic for you, you can use the search bar to surface any tag or any word in any of the documents instantly.

Additionally, the appearance of one specified hashtag can be used to label records in the graph view — here, green nodes means it’s a “recommended” vendor, red means “not-recommended”, and grey means “unknown/ not worked with”. This enables us to see at a glance a detailed view of a vendor’s status.

Strengths of This Approach over the Word Doc System

Updating Vendor Capabilities

One strength of the new approach is the ability to continuously capture new vendor capabilities without a prescriptive list (which will always lag behind current understanding).

The checklist in the Word docs had to be preemptively populated in 2018 with a “complete” list of all the possible services a vendor might offer, but what about future capabilities, or those we failed to envision at the outset of the project?

The new visual markdown database enables a team member to just create a hashtag for a brand-new technology like, say, #Metaverse, and it would become a part of the overall navigation structure of the knowledge base.

This approach, called collaborative tagging or Folksonomy, enables groups of end-users to continuously organize a knowledge base like this from the bottom-up, not the top-down, and from the present-backwards rather than from the past-forwards.

The beauty of this is that it enables us to incorporate new information and organize it continuously. This type of on-the-fly categorization is common in Web 2.0 apps like Twitter, Instagram, and WordPress, allowing diverse individuals to collaboratively define an ever-evolving navigation structure for constantly renewing content.

Of course, the danger is that too much improvisation in tagging can lack standardization, leading to redundant tags (maybe with synonyms) that don’t capture multiple documents but only refer to the document where they appear.

To solve this, Foam opens a popup the moment it senses that you’re writing a #tag so you can choose from a list of the existing tags or write your own new ones if needed. I think this will enable us to keep the list of vendor capabilities current as we discover new vendors with new capabilities.

Additional Data about Vendor Status

One feature I added to the template (and all of the vendor records) is their status — whether we’ve worked with them or not and whether they’re recommended. This way, a vendor can either be

#vendor-status-workedwith or #vendor-status-notworkedwith
and
#recommended or #not-recommended

By combining these two tags, we can get a sense of the organizational relationship.

Based on these tags, we can automatically color-code vendors in the graph preview so we can see at a glance whether they are recommended or not. In the image below, green is “recommended” and red is “not-recommended”.

Additionally, I’ve added #in-billing-system so we can get a sense of whether they’d be ready for quick deployment on a project.

Tags Enable Further Customization

The beauty of this tag-based system is that future improvements can be made to our knowledge base, enabling new ways to organize the data as we discover the kinds of questions we want to ask of vendors.

The new scenario-based screening questions I developed this week might lead us to a strength of #coaching-SMEs or #adaptability that could be added to our database.

Caveat: Collaborator

I am in the process now of socializing this solution with my teammates, many of whom are not as experienced or enthusiastic about Markdown as I am. For those accustomed to working in Microsoft Office, this approach may feel scary and overly complex for such a simple and mundane task as tracking our vendors.

It’s an open question whether this approach will work with a diverse team. Though the plain markdown files can be sync’d in any desktop sync app (we use OneDrive), and can be edited with any markdown editor, offering this interactive graph view is only possible in a handful of applications. Foam is the one we’re using, but the commercial Obsidian editor offers some nice additional features to the graph that other similar solutions don’t match. Some of the screen captures above come from the original solution I developed in Obsidian, and will soon be replaced by the Foam/VS Code database. Stay tuned!

Liked this post? Follow this blog to get more.