Monthly Archives: May 2015

Concept Modeling is Hard

Concept Modeling is a method used to visually express understanding. It forces the author to indicate the explicit relationships between the concepts that make up a domain. I became interested in this approach back when Bryce Glass created a very visually compelling one to explain Flickr (10 years ago!).

Flickr User Model, v0.2
Bryce Glass, 2005

I’ve talked to Bryce about how to do these things and how hard it is, and I feel like every time I try to do one I fail and abandon it before I get to a place where it can be useful for anything.

Since starting at NPR three months ago, I’ve met lots of people with very different answers to questions about how NPR works. Not conflicting, but nuanced based on their role (and using very specialized language). I thought creating a Concept Model would be a good way to capture these perspectives and hopefully generate an artifact that could help people start conversations from a common base of understanding so they could more quickly dig in deeper into whatever topic they need to address.

This past week was Serendipity Days at NPR (often called hackdays/weeks in other places) a time for people to work on things that they wouldn’t normally work on, but that are useful to the collective. So I decided to try to get others to help create a Concept Model to explain NPR and how users relate to it. Being new to the domain (NPR) makes me ill equipped to explain it, but I thought I could facilitate the process and get smarter and more qualified folks to contribute the content I lack.

We started out with a free-listing exercise generating one concept per post-it of “things that make up the NPR ecosystem”, then clustered like-items by affinity. First inspection showed that most things were quite understandable among us: the relationship between NPR and local stations, the financial model behind NPR, even how NPR plays out in social media, but the bucket of things we labeled “content” was a complete mess. There was very little clarity around what an “episode” meant in the context of radio versus podcasts, for example. So we decide to dig into this area more deeply to make that understanding more explicit.

Freelisting followed by affinity clustering

It was unavoidable: get 3 information architects in front of a whiteboard discussing definitions and you will very quickly see a taxonomy conversation shape up. As we discussed the differences in concepts and their hierarchical relationship across radio, podcasts, articles, photo essays, apps and web apps, we needed more robust language and structure that a general mapping was not offering. We were really talking about the Content Model needs of the whole organization.

The blob

So I went back to the procedural model of how you make a Concept Model and wrote down statements to define what the terms meant. This helped disambiguate situations where one term meant different concepts and single concepts that had multiple terms depending on domain.

Concept statements

Interesting to note that this did not flow well at first. My focus question was “how does NPR work?” and my statements were broad and useless. I reframed the question to “how does NPR tell stories?” and all the above came out easily: concrete and specific. A big lesson learned about Concept Modeling regarding the usefulness of the focus question. (Later iterations it became “How does NPR investigates and tells stories?”)

Once I had a good set to work with, we started visually connecting them and making the implicit relationships visible by linking the post-its on the whiteboard and using verbs to express the relationships between them.

concept model fetus

This was very revealing. We found we were missing more granularity and after flailing for a while (but asking and generating lots of new and interesting questions), we realized that we had hit a dead-end talking about this in terms of abstractions. We kept going to look at the website to see how things were actually organized, or looking up glossaries people had made internally.

Veronica suggested we map out specific examples so we could get a really concrete sense of the relationships and then see if a pattern emerged from them so we could go back to abstractions. That was brilliant and we found new terms we had not even included like “a desk”, “a beat” or the actual people that make the content. This is relevant because we observed that most of the work is organized around people, so that had to be a central construct in the model.

example relationships Screenshot 2015-05-15 15.30.14 Screenshot 2015-05-15 15.30.43

By this point we had identified something more concrete to model, but also gathered a lot of new questions we could not quite answer. For example, if a “desk” is a group of journalists that pursue a specific beat (topic area), are all groups of journalists with a topical interest a desk? It turns out the answer is no, but what are they then? The thing about Concept Models is that you need to be precise, so you have to come up with an answer in order to include a concept like this. Presumption without definition does not play well, because after all, this is about expressing understanding and vagueness is NOT understanding.

We almost got off track with the Concept Model because we needed to understand the Content Model of all of NPR for this part, but it raised the issue of how a unified Content Model would benefit the various areas that share the same concepts and relationships. This is how far we got with this exercise given how much time we had to explore, but serendipitously, I happened to tweet about what we were doing…

Paul Rissen, Data Architect from the BBC, who shares our enthusiasm for this geekery wrote back and told us that the BBC had done a lot of work in this area (Content Modeling, not Concept Modeling) and was kind enough to Skype with us for an hour and share some of the definitions, relationships and terms the BBC uses to express its content ecosystem (thanks Paul!).

This was helpful because it offered a concrete AND complete model to talk about the same things, as well as it offered different language to show us which of our terms has general application and what is just NPR lingo.

This was a few hours of work and now I’ve decided to spend more time continuing to validate the relationships with other teams and people in other roles (editors, reporters, producers, etc) to document the Concept Map as I originally intended. In addition, I drew some other conclusions from this exercise:

  • Creating a Concept Model collaboratively was WAY more productive than attempting to interview people and distill that understanding on my own, as I tried before. The process of quickly asking and answering questions together got me much further than I could have on my own.
  • I knew Concept Models were damn hard to do, but learned that the visual representation of the relationships is relatively easy in comparison to gaining agreement around the meaning of things.
  • Even though I didn’t finish the whole thing, Dan and I did a show & tell to explain how we went about it and I received a lot of positive feedback, validating that doing this and having an artifact is indeed desirable and worth pursuing (which is why I’ll continue doing it).
  • It is very easy to abandon the whole thing to avoid this complexity and attempt to write a glossary instead. Except that glossaries are definitional by nature, while Concept Models are relational. And it’s in this relationship between things that deeper meaning and understanding emerges. So, I doubt I’ll create a glossary again, unless it’s in the service of creating a Concept Model, which has already proven more useful.
  • We came to realize that the shape and emphasis of certain concepts in the diagraming was a political statement. So much so that at one point we noticed that the way we did it was a publishing-driven perspective, whereas we wanted a journalism-driven perspective. I suspect the shape/tone of the focus question is partially responsible for this. It also made me wonder if there are assumptions or principles worth stating before starting to help frame this.
  • Throughout this exercise we repeatedly asked “are we trying to illustrate what is or what should we be?”, which was revealing. The point of the Concept Model is to make the complex clear, not simpler. At the same time, the process of understanding concepts, their relationships and disambiguating things forces you to create new language and normalizing existing language, so in a sense it can express a viewpoint that some may not see as “current state”. This simply shows that there is NOT a current shared mental model of that domain.

Thank you Veronica and Dan for your patience going along with my crazy ambition to map all the things (and Kate for stopping by and answering tons of questions!). Special shout-out to Paul Rissen for being so generous with his time and expertise.

I’ll likely write another post with more details as this work progresses.