Category Archives: Good Experiences

Concept Modeling is Hard

Concept Modeling is a method used to visually express understanding. It forces the author to indicate the explicit relationships between the concepts that make up a domain. I became interested in this approach back when Bryce Glass created a very visually compelling one to explain Flickr (10 years ago!).

Flickr User Model, v0.2
Bryce Glass, 2005

I’ve talked to Bryce about how to do these things and how hard it is, and I feel like every time I try to do one I fail and abandon it before I get to a place where it can be useful for anything.

Since starting at NPR three months ago, I’ve met lots of people with very different answers to questions about how NPR works. Not conflicting, but nuanced based on their role (and using very specialized language). I thought creating a Concept Model would be a good way to capture these perspectives and hopefully generate an artifact that could help people start conversations from a common base of understanding so they could more quickly dig in deeper into whatever topic they need to address.

This past week was Serendipity Days at NPR (often called hackdays/weeks in other places) a time for people to work on things that they wouldn’t normally work on, but that are useful to the collective. So I decided to try to get others to help create a Concept Model to explain NPR and how users relate to it. Being new to the domain (NPR) makes me ill equipped to explain it, but I thought I could facilitate the process and get smarter and more qualified folks to contribute the content I lack.

We started out with a free-listing exercise generating one concept per post-it of “things that make up the NPR ecosystem”, then clustered like-items by affinity. First inspection showed that most things were quite understandable among us: the relationship between NPR and local stations, the financial model behind NPR, even how NPR plays out in social media, but the bucket of things we labeled “content” was a complete mess. There was very little clarity around what an “episode” meant in the context of radio versus podcasts, for example. So we decide to dig into this area more deeply to make that understanding more explicit.

Freelisting followed by affinity clustering

It was unavoidable: get 3 information architects in front of a whiteboard discussing definitions and you will very quickly see a taxonomy conversation shape up. As we discussed the differences in concepts and their hierarchical relationship across radio, podcasts, articles, photo essays, apps and web apps, we needed more robust language and structure that a general mapping was not offering. We were really talking about the Content Model needs of the whole organization.

The blob

So I went back to the procedural model of how you make a Concept Model and wrote down statements to define what the terms meant. This helped disambiguate situations where one term meant different concepts and single concepts that had multiple terms depending on domain.

Concept statements

Interesting to note that this did not flow well at first. My focus question was “how does NPR work?” and my statements were broad and useless. I reframed the question to “how does NPR tell stories?” and all the above came out easily: concrete and specific. A big lesson learned about Concept Modeling regarding the usefulness of the focus question. (Later iterations it became “How does NPR investigates and tells stories?”)

Once I had a good set to work with, we started visually connecting them and making the implicit relationships visible by linking the post-its on the whiteboard and using verbs to express the relationships between them.

concept model fetus

This was very revealing. We found we were missing more granularity and after flailing for a while (but asking and generating lots of new and interesting questions), we realized that we had hit a dead-end talking about this in terms of abstractions. We kept going to look at the website to see how things were actually organized, or looking up glossaries people had made internally.

Veronica suggested we map out specific examples so we could get a really concrete sense of the relationships and then see if a pattern emerged from them so we could go back to abstractions. That was brilliant and we found new terms we had not even included like “a desk”, “a beat” or the actual people that make the content. This is relevant because we observed that most of the work is organized around people, so that had to be a central construct in the model.

example relationships Screenshot 2015-05-15 15.30.14 Screenshot 2015-05-15 15.30.43

By this point we had identified something more concrete to model, but also gathered a lot of new questions we could not quite answer. For example, if a “desk” is a group of journalists that pursue a specific beat (topic area), are all groups of journalists with a topical interest a desk? It turns out the answer is no, but what are they then? The thing about Concept Models is that you need to be precise, so you have to come up with an answer in order to include a concept like this. Presumption without definition does not play well, because after all, this is about expressing understanding and vagueness is NOT understanding.

We almost got off track with the Concept Model because we needed to understand the Content Model of all of NPR for this part, but it raised the issue of how a unified Content Model would benefit the various areas that share the same concepts and relationships. This is how far we got with this exercise given how much time we had to explore, but serendipitously, I happened to tweet about what we were doing…

Paul Rissen, Data Architect from the BBC, who shares our enthusiasm for this geekery wrote back and told us that the BBC had done a lot of work in this area (Content Modeling, not Concept Modeling) and was kind enough to Skype with us for an hour and share some of the definitions, relationships and terms the BBC uses to express its content ecosystem (thanks Paul!).

This was helpful because it offered a concrete AND complete model to talk about the same things, as well as it offered different language to show us which of our terms has general application and what is just NPR lingo.

This was a few hours of work and now I’ve decided to spend more time continuing to validate the relationships with other teams and people in other roles (editors, reporters, producers, etc) to document the Concept Map as I originally intended. In addition, I drew some other conclusions from this exercise:

  • Creating a Concept Model collaboratively was WAY more productive than attempting to interview people and distill that understanding on my own, as I tried before. The process of quickly asking and answering questions together got me much further than I could have on my own.
  • I knew Concept Models were damn hard to do, but learned that the visual representation of the relationships is relatively easy in comparison to gaining agreement around the meaning of things.
  • Even though I didn’t finish the whole thing, Dan and I did a show & tell to explain how we went about it and I received a lot of positive feedback, validating that doing this and having an artifact is indeed desirable and worth pursuing (which is why I’ll continue doing it).
  • It is very easy to abandon the whole thing to avoid this complexity and attempt to write a glossary instead. Except that glossaries are definitional by nature, while Concept Models are relational. And it’s in this relationship between things that deeper meaning and understanding emerges. So, I doubt I’ll create a glossary again, unless it’s in the service of creating a Concept Model, which has already proven more useful.
  • We came to realize that the shape and emphasis of certain concepts in the diagraming was a political statement. So much so that at one point we noticed that the way we did it was a publishing-driven perspective, whereas we wanted a journalism-driven perspective. I suspect the shape/tone of the focus question is partially responsible for this. It also made me wonder if there are assumptions or principles worth stating before starting to help frame this.
  • Throughout this exercise we repeatedly asked “are we trying to illustrate what is or what should we be?”, which was revealing. The point of the Concept Model is to make the complex clear, not simpler. At the same time, the process of understanding concepts, their relationships and disambiguating things forces you to create new language and normalizing existing language, so in a sense it can express a viewpoint that some may not see as “current state”. This simply shows that there is NOT a current shared mental model of that domain.

Thank you Veronica and Dan for your patience going along with my crazy ambition to map all the things (and Kate for stopping by and answering tons of questions!). Special shout-out to Paul Rissen for being so generous with his time and expertise.

I’ll likely write another post with more details as this work progresses.

Come work with me

I am looking for the right person join my team as Director of User Experience Design.

I am in the process of creating one integrated multi-disciplinary experience design practice (the organization used to have several separate compartmentalized/specialized departments). To become one team, I’ve consolidated the existing groups (40 people) and identified four main areas of oversight for our service so we can divide and conquer. For each of these areas, a director of UX design will oversee a team that will focus on a core aspect of our offering, developing subject matter expertise over time and establishing a long-term design vision.

This role has two core responsibilities: 1. To support and grow a team of talented UX people 2. To define and steward an experience vision for the aspect of the service they focus on.

In a year’s time this person will have taken a group of folks with information architecture, interaction design, content strategy, graphic design and other core skills and expertise, and successfully turned them into a team that acts as a unit.

They’ll have contributed to creating a work environment that fosters productive design practices, including training and practicing critiquing, presenting, storytelling, sketching and facilitation. The team will be capable of designing solutions that adequately translate into device-agnostic experiences employing a foundation of modular, responsive design.

Individuals on the team will have a clear picture of what their role responsibilities entail and what opportunities for growth, improvement and career advancement are available to them. They will be confident in the UX design director’s leadership and management skills, knowing they can be counted on to act in the best interest of the team and its members.

Executive leadership will trust the UXD director’s long-term design vision and have an understanding of how it aligns to the overall department and company-wide strategies and pursuits. That vision will be easily articulated by any member of the Experience Design team and used as a reference point to direct long-term design decisions.

The organization will have become accustomed to modeling approaches of varying fidelity as a method to explore design solutions and feedback cycles with users as a foundation for incremental improvements. This will signal a particular focus of the UX Design team on delivery over deliverables, solutions over documents.

Moreover, the quality of users’ experiences will be markedly improved by a concerted effort to establish a cohesive design system that unifies the service offering, addressing the core issues users experience. Given the breadth and depth of our offering, this will have been made possible through the establishment of a strong foundation of design standards and guidelines combined with a robust design practice and a team of individuals empowered and prepared to make decisions.

Are you that person? If so, please apply today.

Update: We are in the middle of updating our HR recruiting tool so if you have any difficulty with this process please email greg.killeen@marriott.com

Make it happen 2011

Next week I’ll be giving a talk and participating in a panel at the Design Management Institute’s Design/Management Thinking “Make It Happen” conference in Seattle. I’m excited about this event because they’ve framed it as:

We know quite well the value of Design to business, and Design Thinking to problem solving. But what remains a bit fuzzy for many organizations is the distance between thinking and doing—the proverbial gap between strategic intent and execution. Or, how to make it happen. This year’s design thinking conference will focus on closing the gap—and moving from design thinking to design doing.

What one actually does. I enjoy the conversations about design thinking but they tend to lead to a lot of hand waving and I have found many designers and specially young managers struggling to grasp just what it is they need to do (not just talk about) to produce the positive outcomes discussed in this context.

My talk, which could not have been more appropriately timed, will be a journey through my work at Comcast between 2004 and 2011. I’m going to talk about how the UXD practice was established, how it grew, changed and evolved over the years, and what impact it’s had in the company culture and products.

What aspects of this journey would YOU be interested in hearing about? DMI is recording the video for this session so you’ll have the opportunity to see it later in case you can’t make it to Seattle. Please let me know what points in this story you’d find most useful learning about or any questions you may have.

I’ll post a summary after I’m back. Thank you!