Category Archives: Bad Experiences

Quant before Qual makes no sense. But it does.

As I continue to explore how designers can make better informed decisions by leveraging information, the issue with number aversion is still #1. I talked about this already in my Interaction 10 presentation, but I’ve been digging deeper and have some other thoughts (check my presentation for some base assumptions).

If we agree that quantifiable data, specifically the ever popular web analytics, provide you with rich detail to tell you WHAT is happening, it is comforting to realize that it is the type of data gathering that we already do – design research – that provides the qualitative color to answer WHY said things are happening.

What I am finding, however, is that it is more valuable to START with the quantitative work and get to the WHATs and ask WHYs based on those findings, rather than trying to figure out WHYs in exploratory mode (even if the WHAT’s are going to emerge at one point or another in this quest).

My point is that it’s not sustainable as an approach. It’s inneficient to start digging deeper to answer the WHY questions if you don’t have a baseline of WHATs identified.

The problem is that it is not intuitive for designers to start where they are uncomfortable. We are super comfortable with qualitative approaches – they are our go-to tools because that’s what makes sense for design research. However, quantitative research instruments really help narrow stuff down, but they do require you to understand those pesky numbers in order to a) dig in and get to concrete answers and b) understand what it’s saying so you can ask “why”.

In short, WHATs before WHYs are more efficient than WHYs before WHATs, but that requires designers to start with unfamiliar tools to then apply familiar tools. If it was the other way around I think it would be much easier for designers to bridge both approaches and come out the other end with more useful insights.

In other words, since we don’t particularly feel an attraction to numbers (to put it lightly), why would we start there? It’s such a leap from how we think about problems that it is counter intuitive. I don’t believe designers reject the notion of starting with Quant approaches (WHATS) to expand with Qual approaches (WHYs), but it’s inherently counter-intuitive to think that way.

How can I help designers do this when it goes against their nature? That’s what I’m working on right now. More on this later.

Learning how to make UX decisions

I just had a great time recording a Userability Podcast where Jared Spool and Robert Hoekman answer my questions about how UX practitioners can learn to make good decisions about which methods to employ in their work.

[I’ll update this with a link once it’s published]

My question is an old concern about how new practitioners are being introduced to User Experience Design and Research practices by being fed a multitude of methods and not given much support about how to decide the right circumstances to use them.

It is not sufficient just to know how a certain method works. It is also not sufficient having used that method once or twice. What is it about our experience as practitioners that makes us better or worse decision makers? How do we choose to dedicate time and money to an 8-week long project to produce personas instead of a different approach?

What distinguishes the practitioners that not only choose methods and know how to apply them, but choose the methods that are most effective for a given problem?

A few years ago, Jared himself told me a story about an experiment where two distinct research teams (unaware of each other I believe) were given the exact same research goal and employed the same methodology to achieve it, and came up with different results and findings.

When that sort of thing happens, I wonder: Can we really trust our methods? But more importantly, if we accept that our methods are not really scientific and that we can’t really have a high level of confidence about the results we end up with, how do we choose one over another?

Somehow we just do. But some do better than others. Some do MUCH better than MANY others. If you have the opportunity to work with practitioners with enough experience and knowledge, you see excellent arguments for why to do A versus B for a given set of circumstances. So yes, only experience will help one make better choices, but everyone’s experiences are different. As a way to try to educate new practitioners we coach and mentor by teaching the methods and also giving advice such as “be flexible” and “don’t marry a particular process” and “figure out what kind of problem you are trying to solve first”, which are all excellent advice, but not strategic enough and often not practical enough that it can really help someone make a decision when they are faced with a new challenge.

Jared’s opinion is that our field is still too young and we haven’t yet been able to articulate the criteria we use in that decision-making process. I agree, however, it worries me that many think they are advancing in their practice because they know more, when in fact, they just learned new methods, but don’t really have the skills to assess risks, and benefits, between choosing one over another.

Being a runner gets you to the finish line, knowing which way to run wins the race. I really hope we become better equipped to pass on knowledge about how we make choices and why because, paraphrasing Jared, knowing a lot of recipes a restauranteur does not make.

Tune Deaf

Do you use conference calling service for your work? I am sure you do. It’s inescapable; whether you use it for remote team collaboration, sales pitches or anything else, you have experienced the music that comes up when you first call in and is waiting for the leader to join and start the call.

It’s bad. I have used a number of different services and they are all bad. So when my friend Kit Seeborg told me about her new start-up, BumperTunes, I thought, they could definitely help with the lousy quality music these services have to offer!

So, in the spirit of encouragement for Kit & team, who are really focused on the podcasting market rather than conference calling, I just wanted to share what I have to listen to between 5 to 10 times a week (sometimes multiple times in a day):

please-help-my-poor-corporate-ears.wav

Update: If you have other examples, please record and post here! It’s easy. On Windows, just go to Programs > Accessories > Entertainment > Sound Recorder (fire up your lovely conference call tune and hit record)

Spam out of control!

I thought getting unsolicited mail was bad, then emails were invented and SPAM was born, then blogs were invented and with them came SPAM comments. Is there an end to this? No, of course, now I have voicemail spam!

Since I listed my number of my chi.mp profile it’s been receiving the most ridiculous calls. Like the one above. Where do we go from here?

Discussion List Technology

When I started evaluating the IA Institute overall tech infrastructure I was not expecting the messiest part to be related to the various discussion lists we provide to the community. I was first surprised, now I’m annoyed.

The list software we use is Mailman, which is extremely popular and very good at one thing: delivering mail. I guess they chose a pretty appropriate name for it. Other than that, it’s pretty sucky.

My intention when I started to take a look at our discussion lists was to understand how extensible our technology was to support any future plans (indexing archives, subscribing to threads, integration list subscription with membership profile, RSS subscription, etc). What I’ve found is a messy legacy that needs to be at least normalized before we can think of expanding its capabilities.

Here’s a list of all the discussion lists we have:

  • aifia-announce -IA Institute announcements.
  • AIfIA-da -Om informationsarkitektur på dansk
  • Aifia-education -Discussion of IA education
  • AIfIA-es -Instituto para la Arquitectura de Información
  • Aifia-fr – IA discussion in French
  • Aifia-it – IA discussion in Italian
  • Aifia-ja – IA discussion in Japanese
  • Aifia-mentoring – AIfIA Mentoring Initiative
  • Aifia-metrics – Towards standard methods and metrics for evaluating IA
  • AIfIA-nl – IA discussion in Dutch
  • AIfIA-pt – IA discussion in Portuguese
  • Aifia-tools – Discussion list for the AIfIA Tools initiative
  • Advisors – IAI Advisors
  • Arqinf -Lista de Discusión sobre Arquitectura de la Información
  • Board – Board of Directors
  • Directors – IAI Board of Directors
  • Eastcoastretreat – New Challenges Retreat list
  • eiaproject – Higher Education in IA Working Group
  • EnterpriseIA – Enterprise IA Discussion List
  • iai-aunz – Australia New Zealand Region IA Discussion List
  • iai-jobs -IA Institute Job Newsletter
  • Iai-Members – IA Institute Members Discussion List
  • Iai-Mentoring – IAI Mentoring Discussion List
  • Iai-Newsletter – IA Institute Newsletter
  • IAI-pt – Lista de Discussão AI-pt
  • iai-translations – IAI Translations Discussion List
  • Localgroups – local IA groups
  • Management – IAI Management
  • Meta IAI – Meta List
  • Secondlife – IA Institute Second Life Discussion List
  • Test – yes, it’s what you are guessing
  • Ux-Management – UX Management Discussion List
  • From this list it should be easy to tell that we (the IA Institute) have not been big on naming conventions. I created some of these lists at one point or another as I volunteered in different initiatives, but I didn’t even know all of them were out there. I would love to be able to go to the IAI website and just know what’s available (right now the site shows a partial list).

    Some of these lists, I am sure, are dead. But somebody forgot to pull the plug. Also, between managing subscribers and moderating discussions, there is this horrible thing called the discussion list interface. Mailman as I said before is good at one thing and that’s not its user interface. It’s impressively adequate in terms of multi-lingual support and is flexible enough that you can customize presentation to fit your website (We have tried before), but if you don’t have a standard way to to do in an organization with such high volume, this mess is inevitable.

    If it’s not clear from the rant above, many lists still have our old organization name (Asilomar Institute for Information Architecture) and are hosted at ibiblio.org, which provides free discussion lists. Another issue: We host our site and systems on Dreamhost. Their Mailman implementation doesn’t allow me to go directly and finagle with the lists directly (like merge archives or modify the code) so I have to ask them to do it, which means any changes may take a while.

    Now that I’ve bitched about the current situation, here’s what I believe needs to happen:

  • Get rid of lists we don’t need to maintain.
  • Evaluate if an alternate software to Mailman is a better fit for our organization
  • Create some basic guidelines for starting discussion lists
  • Migrate ibiblio discussion lists to iainstitute.org
  • Merge archives of lists that should be consolidated
  • Notify subscribers about any plans
  • Do you have experience with discussion lists? Drop me a note if you have any advice or suggestions. I’m particularly interested in systems that have discussion lists associated with member/profile management associated with other services. Anyone has experience with Drupal; any Drupal modules for discussion lists?