RAW NOTES FROM THE AIFIA IA LEADERSHIP SEMINAR Official Seminar Archive >> http://aifia.org/calendar/000053.php Some Photos From the Seminar http://noisebetweenstations.com/personal/photo/ia_summit_2003/ia_summit_2003.html ____________________________________________________________ SECTION I Peter Morville. "Strategy, Architecture and Tomorrow’s Web." NOTES by Andrew Hinton have to be a futurist borderlands/edges of IA what interest me have to have some insights, good guesses about where things are going with business strategy Michael Porter: talks about strategy about choosing to perform activities differently, or to perform different activities than rivals. What are our unique strengths as an organization? Don't just run around ocpying best practices. In-depth case study of Vanguard John Bogle, founder Vanguard Early in its history, Vanguard establieshed "a mutual structure without precedent in the industry - a structure in which the funds would be operated solely in best interests of shareholders. Strategy follows Structure Started with Strict Cost Control, spreads out to every level of detail Strategic Planning --> Plans Executed (10%) --> Realized Strategy Unrealized Emergent Strategy 90% Strategy 90% Mintzberg Encourages us to embrace lots of different definitions of strategy. Plan: a direction, guide, course of action Pattern: consistency in behavior over time Position: locating specific products in specific markets Perspective: ways of doing things ("the HP Way") Ploy: Specfific maneuver to outwit Sterling/Gibson Futurism doesn mean predicing an awesome wonder, rather it means recognizing and describing a small apparent oddity that is destined to become a great commonplace (sterling) Apophenia: the spontaneous perception of connections and meaningfulness in unrelated things (Gibson -- in Pattern Recognition) The present is unevenly distributed. (US life expectancy all time high, going backwards in Africa) A welath of information creates a poverty of attention. Herbert Simon, Nobel Laureate Economist Among very experienced users, the Itnernet now ranks higher than books, tele, radio, newsp, and magaz, as an imp source of informationl. ucla information report, jan 2003 Consumer Web Watch While information is often assoc with usability the comments here show how inf struct has implications for credibility. Sites that were easy to navigate were top 3 1. Design Look 2. Information Design/Structure 3. Information Focus Credibility = really important area. When to Trust Your Gut -- hayashi, Feb 2001, HBR Exec's routinely rely on intuitions your mind continuously processes info you're not aware of so it's not all about facts, figures... at very high levels (executives and us) make decisions by gut instincts Intertwingling of physical and virtual worlds... morville ambient devices jeffrey huang: case study at harvard on the Swisshouse -- buildings becoming portals into online environments as well wristwatch with everything... gps tech, safety lock/key fob, tracks their location Smart Mobs: Rheingold Future/present Thumb Tribes: sms text mssging mapping geography into an online environ? maybe not think it'll look more like social network analysis behind the scenes knowledge of who's linking to whom/what pretty convinced it'll look more like ... amazong page of customer reviews will rely on lables/words shows a winamp screen (or another ? RM?) with and without words... how useless it is without words still will have search/browse picking on jared: yeah, today in many instances search still sucks secureid can't be found at RSA but it can be found (did you mean securid?) at Google!! ____________________________________________________________ SECTION II Victor Lombardi. "The (Unfulfilled) Promise of Content Management Systems." NOTES by: ____________________________________________________________ SECTION III Rashmi Sinha. "Building Bridges Between Practice and Research." NOTES by: Erin Malone Why special user research methods for I-rich domains Important to understand information representations in a user's min There are a few guidelines Borrow methods from anthropology - culture - semantic domains - psychology Project vision Develop a portfolio of methods to probe user's representation and needs Reduce time for user research Reuse the data for research and personas Reuse results Shouldn't have to start from groundzero everytime - borrow from other's research as well Much research is prescriptive - these methods suggest don't prescribe Methods to empower the designer to allow them to make decisions Reliable methods - can be duplicated by designers 2 key questions User categorization - Scope Structure Differences between groups User Information Need Categories - structures of the mind Why do people categorize - Cognitive economy - Luria's case - mind of a mnemonic - could remember anything as long as there was a 4second gap between items - couldn't remember people though Allow generalization from past experience - by treating something as a kind we can use what we have learned from other examples Facilitate communication - describing something as a kind of thing, provide details of the \thing A level of abstraction is made easier Structure of semantic memory Is this important for IA Direct use - when user cat. Informs an IA Indirect use - good for broad understaning Context is important - getting whole scheme right is more important thans specifics Important to remember Categorization is not static - people are good at learngin new cateogries. Context and right examples - people will pick it up Should interfaces always reflect user categories faithfully? - No Different groups have different perceptions so no scheme is going to be perfect for everyone Business propositions, brand etc affects the scheme The structure of memory and how categorization fits in Implicit and explicit memory affect categorization poerceptions Short term / working memory limitations Categorization is implicit Terms - categorization and semantic memory used interchangeably Structure of Semantic memory Semantic distance Hierarchical structure Graded structure - some members more prototypical than others Semantic organziation builds up user experience Not all categories follow these rules - goal directed categories Things to wear in winter Individual differences for categorization Culture, background etc affect thinking Even though there are differences - there will be enough consensus and consistency to design for the 80% - even across culture Degree of closeness between items might vary between people but we recognize the idosyncracies Semantic distance is the basis of categorization At root of all categorication is the questions - how far is a from b Semantic distance can be plotted as a similarity matrix Methods Free-listing Free listing exercise is to explore the doamin scope and boundaries Goals- Explore boundaries Gain familiarity with user vocabulary Use as a precursor to card sorting Some items start repeating Some items occur earlier than others For each item - # of times mentioned, the number of particpants who mentioned it, average ranking Ranks Plot items based on frequency 3 concentric circles core middle periphery Similarity from co-occurance If two people mention the same items then you can assume a similarity Crude similarity metric Can compare how two groups percieve the same domain Can compare two domains Can find respondants with greater domain familiarity - can help with audience segmentation Recursive free-listing to explore concept structure Categorization methods Goal: to understand the overall scheme Method is open card sort - no categories specified Options Provide total number of categories or not (create between 8 and 12 - example) other wise you have splitters and lumpers Successive card sorts Helps determine hierarchies Ask for labels for each grouping Category Allocation Goal - to understand goodness of existing IA and labels Method - closed card sort Give them labels and user assigns item in category - don't all ow misc. Useful for Understanding user cats when cat labels are given Refine existing schemes Options Allow items to belong in multiple categories Provide cat descriptions rather than category labels Category Verification Goasl - test good ness of existing scheme Method - speed verification tests Create a nav scheme - test the scheme by asking users to find items form various levesl Utility called "Classify" =- step two Useful for comparing tow designs Card sorting and alternatives Cluster analysis Advantages Suggests structural solution Dis Prescriptive Alternative Multidimensional scaling Quadrant grid Hints at solutions and shows direct relationships But leaves structure up to the designer Similarity maps are easy to understand Helps identify which dimensions are most important Cluster analysis and MDS are complementary Infer data from a card sort with users Card sort to see groupings Put together correlation matrix to see where items that are similar appear together Then do cluster analysis and grid - to infer behaviors or attributes of categorization Alternatives to cards sort Purpose of card sort How to structure website to help users find info - fast, remote, online survey type (could also have user on the phone) Do it like a survey For each item put an X for which group they think the item belongs to Then ask them to label each group Each item occupies a row Each category is a row Can be done in excel as well Options with card sorting and what to expect in the future Options Should each item be in only one category Should card sorts be done individually or with groups? Group card sorts give energy and enthusiasm Group card sorts can overshadow indivdual differences Combining card sorting with iportance ratings Future Difference user groups, culture, individual differences - how much consensus is there Stakeholde analysis Reuse of domain concept maps Analysis of sorting data Identifying dimensions of facets Method 1, 20 questions User tries to guess what the item is by asking questions about aspects or facet of the object Differentiating items Any two items in a domain - identify their differences Items need to be complex enough to have multiple aspects to them to provide dimensions and facets Understanding User needs and creating personas Are some needs more important than others? Can users be differentiated into groups based on these needs Can this form the basis of personas Persona as User Archetypes who represent needs and goals for many other users Current method Interview stakeholdes Find patterns Pick a nugget and build person around it Problems Interviews not economical No tight coupling between user reseach and person Would 2 desings using same research create same presona Method works well when you have time, resources and skilled persona designers Market segmentation techniques - is this useful Identify clusters of people Questions focus on product concept Cluster on demographic High conceptual level What is needed is a method that Grounds in realityu Focused on needs of users Complementary Two people can derive similar personas from same research Concrete series of steps Creating personas (get Rashmi to write this up for B&A) Preliminary phone interviews - exploration of domain, identify questions that are needed for next stage. Get acquainted with user vocabulary Survey of user needs Online Broad and shallow look Tie response to specific instances of behavior Ask about scenarios of usage General data analysis to broadly understand user needs Importance of various factors Relationships between factors - examining relationships through Numeric correlation matrix Examine relationships through a shaded correlation matrix - excel will do this Identify User archetypes Features that are highly related will form part of the same constellation or group of needs Factor analysis - techniques that identify groupings Use user archetypes as candidate personas ____________________________________________________________ SECTION IV Louis Rosenfeld & Karen McGrane. "Selling Information Architecture." NOTES by Kirsten Swearingen Note: we should focus on selling solutions, not IA Agenda: " Techniques for making the case (45 mins) " Role-playing exercises (45 mins) Background " Lou has been selling solo IA consulting services for several years. Before that, at Argus. " Karen has been at Razorfish for past 5 years. Past couple years, much more sales-lots of presentations. Heartening to hear about the importance of explaining what IA is in a way that business leaders will understand. Techniques Technique # 1. ROI case Razorfish anecdote: back in 2000, we brought in a new CEO, an MBA. Made big speech about explaining to clients the ROI. Founder said, "We don't talk about ROI at Razorfish." Shortly after that, he got shuffled off. Recently, tried complex analyses of ROI on giant Excel spreadsheets. People still didn't buy it. Clients value money (and customer experience and brand). Translate whatever you want to do into how they're going to make money or how are they going to reduce costs. Rhetorical tool. Reduce cost of customer service or sell more products. Don't use numbers if you can't back them up. Begin by talking about the kinds of returns company might receive. Then talk to people who can give you numbers to back you up. Q. How many of you are using ROI cases? About five people raised their hands. Aud. Depends on what you call an ROI case. Aud: I sell to senior management-need numbers to back me up. Aud: I use benchmarking in combo with ROI to show change in performance. I'm not a big fan of ROI, but you need to at least be able to talk the numbers. Depends on who you're communicating with. I'm a big fan of telling stories about pain. Pain is the friend for us all. Getting in someone's head and exposing horrible truths of their jobs, their lives. Technique #2 Narrative case studies (could be print or web page) and storytelling (face-to-face). Have some stories with some horrible arc in the beginning. For example, worked on an intranet of 50,000 pages. We started with printed material and we dumped it on the Intranet and it didn't work. People connect emotionally to this sort of story. You can do ROI in combo with these types of things. " Establish actors " Portray pain " Describe solution " Describe results When to do it? Early in cycle, though too long to "open" a prospect Audience: Here's an example: we used a cartoon showing doors you open to get to info. There were doors with descriptive labels. Arc of something bad, something good. This was very effective. Humorous. Recognizable. It also shows that Info Architects do have senses of humor. Story needs to appeal to 2 people: who you're telling it to and yourself. You have to be able to do what you're doing. Nice thing about stories in any format-person probably does have some experience in storytelling. We're already programmed cognitively to respond to them. Karen: Clients love to hear stories about other clients. Makes them more comfortable about the problems they're facing, they're not the only ones, shows that you are credible since you have successfully solved the problem. Space between you and audience. Website doesn't close space very effectively. Trying to sell IA when they're used to consulting. Story is more comfortable for everyone involved. Technique #3 : Therapeutic sales. I'm an information therapist. Clients are undergoing extreme information pain. Get them one on one on the couch. Or work with them in a group-helps them realize "I'm not the only one." Create a safe environment. Must have some level of trust-so don't try this on Day One. Give them building blocks. Get people speaking the same language about info and info pain. Give them a book or something to make them see that this is REAL. May require a brief "What is IA?" seminar. Give them terms to discuss things like IA. Then they talk about their specific situation and their specific pain in a common language, your language. This helps you in writing a proposal. Timing: not at the opening, after opening. Good to do once you're on the clock. Give them a couple case studies, then shut up and listen to what's going on with them. May need someone to break the ice. Need to tell something a little embarrassing about your own (or your organizations) (or a past client) info pain. Dish a little dirt. Audience: useful as external consultant but also useful for internal work. Group therapy from within. Audience: I use 3 minute or less PowerPoint "info pain" movies. Explain issues. Appeal to their logic and their frustration. You, being logical, will look for it this way, but it doesn't work. It can be fixed. Don't forget to remind people that there is HOPE. It's tough but not impossible. Problems can be solved. Audience: People may want to grasp onto best practices. But people think their own problem is totally unique. Listen to their individual troubles-repeat back in person and in a proposal. Closing the gap. Use their terms. Show you've been listening. These first 3 techniques are not mutually exclusive. Triangulate and use them together. Technique #4. Competitive Analysis. Identify competitors-talk about other best practices. Competitive feature sets . Companies implementing similar functionality. Do as part of requirements gathering process. Gather all the things they might want to do and identify which are the ways to go. Example: we don't want to do just what everyone else does. Is there anyone out there doing what we do? Or doing it particularly well? Reverse-engineering business strategy from websites. Look across sites, what relative emphasis different sites have. Relationship of company and customers. What are they trying to do? Where should our company fit in the mix? Compare and contrast approaches to architecture, navigation, and page layout Aud: Sell what competitors aren't doing. Where they're failing, unmet opportunities. Sometimes this makes clients feel better about opportunities. Aud: Seems like way to get buy-in for particular arch approach. How could this be employed to encourage practice of info arch itself? Yes, I do that-show different approaches to problems. Example: financial services. Show them different approaches by using existing sites rather than prototyping. Walk them through scenarios. Helps them understand pros and cons, feel how it worked. Quicker way to discuss the options. Other competitors do things differently. Made them feel like she understood their business. Don't claim industry expertise you don't have. But competitive audit is a good way to build your expertise in the area. Show that you know how others are doing things. Aud: We have educated clients. Then client uses competitive info to pick apart design. Because diesgners didn't use what we put together in comp analysis. It's a double-edged sword. Aud: CEO at Washingtonpost.com was fixated on NYT online. Used this to convince him that search had a problem. Competitors can be a great way to get them to hear you. When you can't say things directly, using your IA terminology-it's a clear way to get it across. Let other companies make your mistakes for you. Many large sites have done extensive testing. You don't have to reinvent the wheel. Users spend most of their time on other people's sites. Comp analysis should inform decisions but not dictate them. Shouldn't be the sole reason for your rationale. Q. Have you ever had competitive analysis blow up on you? When clients say, we just have to be where everyone else is. Aud. Sometimes we get, "We really don't care what the other guys are doing." Carpet manufacturers in North Georgia get weird when you bring up their competitors. They've been shooting each other for years. Aud: We use it afterwards. Our goal was to do better than these sites or than our old site. And look how well we did. Calms people down. Aud: We're on a teeter-totter. Users vs. management. What we find through user research vs. what's out there through competitive analysis-they may conflict. Need to find some balance point. Aud: It really is selling a solution that involves info. architecture. Don't try to sell IA. People are into things that fix their problems. Aud: Differs depending on which level of the company you're talking to. Lower levels are interested in rolling out a product. Upper levels: doesn't help to talk about features. Talk about process instead. Differences that get created by choosing different solutions. Technique #5: Eating your own dog food. You're working with someone inside the company, trying to sell up the chain. Can you get them near the website and make them use it? Do so in a one-on-one, non-threatening situation. This is powerful when you can manage it. Q. How many of you ever get to do this? Anyone as a consultant? Several have done it internally. Only one has done it as a consultant. Can get away from you if you have CEO use the site. They may come in with a long list of bugs for you. Aud: or get CEO in the observation room when doing a usability test. This can get out of hand too. They hear what they want to hear. All of these things have their pros and cons. Aud: Teams write down core user tasks for product. Now do them on your product. All sorts of obstacles are uncovered. Now do them on your competitor's site(s). Simple method to make them open their eyes. Another variant is to show highlight tapes on usability testing. Technique # 6: User testing & diagnostics. Watching users struggle can be a persuasive argument. Use as a design tool, as a persuasion tool to help convince of need for changes, as a diagnostic tool, and an evaluation tool (using metrics for success) Ways to do this: " Decision makers behind glass or feed of live testing (pros: makes problems seem important. Cons: can be difficult to manage expectation. May jump to conclusions or get very upset. Have moderator talk to them during breaks and debrief afterwards. Ask them to sit on what they have seen and wait til the findings have been analyzed) " Video highlights-less time consuming for execs, benefits you because you can be selective to persuade, BUT requires video edititng skills, may be costly (it's worth it) " Write debrief document. Can take a systematic approach to problems and solutions. Gives you weight of something that seems substantial. Lacks emotional appeal. Q. Are you all using this? Aud: Looking for ways people are alike and how they are different. Good to help them focus on difference between opinion and problem. If they don't like yellow vs. can't complete task or wouldn't use this site again. Aud. Senior managers get upset when they observe. They say "These are the stupidest people I've ever seen." Then "They're asking the wrong questions." And finally "Here's the list: 15 changes I want to see done tomorrow." This is an exaggeration. Rest of audience: (laughing) No, it's not! Aud: These are take-charge people. They want to solve the problem, to tell you what to do to address the complaints. You have to convince them that you need to read between the lines, don't just do whatever the customer says. Aud: Tactically, one thing I've found very effective is combining 2 and 3-quicktime movie inside a Powerpoint presentation. Keeps emotional resonance of users but can also send link out to the company. Better case. 100 people can watch it at their leisure instead of 3 people behind the glass. Aud: we take each section, a column of user comments that relate to the data or the feature. Especially useful when dealing with a 3rd-party vendor. "It's not a nice to have, it's a NEED to have." They're trying to do it as quickly as possible. Aud: I require them to observe 2 users or not observe at all. Because they might see one freak user who goes right to advanced search every time. Tell them "You're really busy-I'll just send you the report." Technique #7 User Feedback Users who call in may be a little too invested in the software-not necessarily representative. Aud: senior managers tend to think that only problem we have is the one that users called in about. Allying with customer service. Get valuable perspectives from people who deal with problems every day. Example: power company encouraging people to use online billing. Created huge requirements for CSR. They had lots of info on what wasn't working. Warning: reduced costs may come in the form of reduced salaries. (Even if the CSRs benefit from easier work flow.) Counting complaints / hate-mail. All sites have feedback mechanisms. Get quantitative and qualitative data. Spontaneous. Cons: numbers not entirely accurate. Surveys. Qual and quant data. Easy to gather. May integrate with log analysis. Aud: can see people eating their own dog food. CSRs trying to use their own sites. Aud: They're really happy to have people ask them their opinions. Aud: Get friendly with trainers who teach people how to use site. "People always do X when I try to teach them this." Invaluable information. Technique 8: Log analysis. Lots of people in audience are using this. Log analysis can tell you the WHAT. The actual data of usage. Nice complement to user testing. Can't really get the WHY from logs. Server logs, search logs, customer service logs-how to put these things together? So you're not looking at individual actions, but in the context of a whole user session. Get logs in standard format. Assemble data from many companies and analyze sessions. Informs metrics and is informed by metrics. You aleady have the data. Q. How many of you can't get the data from the IT people? Aud: they're scared to share. Webtrends report that is meaningless, no context. We burn hours trying to explain why server logs are the way they are. Not created for our purposes. Make IT people nervous. Should make you nervous too-need to look at total user experience. Example, people search in combination with browsing. There's not really good software out there-reports are designed with someone else in mind. We need tools for this. Aud: When companies are aware of why they have a website, it's easier to measure success. Get in at strategy phase. Info is part of corporate strategy before any design is happening. Interactive Exercise 2 teams for each scenario. One trying to sell, one being sold to. Spend 10 minutes figuring out what kind of case you'd want to build. Spend 15 minutes arguing back and forth. Team A: "Proposers" Team B: "Decision-makers" Bottom-line focused. Scenario #1: Media/entertainment group Disconnect in our group. Boss thought I was a consultant not an employee Traffic log analysis. Found that we were losing a lot of people-coming to home page and then bailing. Listened to client's position. We're just a home.com site. Concerned about corporate community and stakeholders, not customers. Most of our customers use Google. We don't need to act as place that filters them. 3 million visitors, half of them didn't move on. Stories of customers looking for products in different spaces. Pinocchio video and figuirine. Pointed out that we're not supporting cross-selling. Not supporting corporate strategy. Told story about cousin not being able to find things. Media mentions of the site, esp. negative mentions. This is incredibly powerful where I work. People couldn't use same shopping cart on different sites. Take away some of the metadata concerns, not worry about structure of the site. Allow them to have better user experience. Scenario 2: redesign of company's website Disconnect between what customer thought they needed. Thought they needed to change graphics and nav. But really needed to focus on search-based on user research. Had to do both to satisfy customer. Customer wanted us to share in some of the risk. Decrement on our fee if we don't meet goals, or increment if we exceed goals. We're new project managers, don't know what site is supposed to be, don't know why it needs to be redesigned. We didn't make this decision. Oh that's really nice, but we don't care unless we get some benefit from it. Put some metrics on it. Wanted to be sure we could be involved in requirements gathering. Also wanted to get boss' input on redesign. Plan for this. Scenario #3: someone wants a job as IA in a user experience group A few of our methods worked, a few failed. Since we already work here, we showed that we knew the company, knew the problems behind the scenes. Too abrupt so it failed. Positive; position would be horizontal and help coordinate across all team members. Push strategic goals from the top. Didn't work well in this situation. Manager thought interaction designer already did this. Wanted to know what skills they are missing. Didn't expect them to tell us what's wrong with our org. Good job framing themselves as a liaison. But we still didn't know what we were missing: skills of library scientist. They ran out of time so didn't make their case. We're still on the fence. Not worth fighting over the job title. Hire on as interaction designer and then show the additional things you're doing. But we weren't qualified to be interaction designer. Easier for a manager to add a head than to create a whole new role. But then again, it's less powerful-we already have one of those, why do we need another? Jargon was used-as manager, I didn't understand why this terminology was used. Use language that already exists. NOTES by: Andrew Hinton selling ia nobody should try to sell IA should sell solutions, but that's another story lou rosenfeld, karne mcgrane agenda: techniques for making the case, then role-playing exercises karen: now in sales situations more often... every week at least once i'm having to explain what we do in a way that they understand. techn 1: the roi case a way of justifying the value of proposed changes according to business value clients value money money and related terms... cust exp, high quality, brand but, really about reducing cost, and making money translate what you can do into terms they can understand. usually measured in dollars, but the concept can be extended to include intangibles value isn't in coming up with numbers to plug into dept goals it's a rhetorical tool... for talkinga bout what you want to do or why it is valuable, and translate that into something... don't use numbers if yuo can't back them up. people are skeptical of numerical claims if you don't have firm numbers, even addressing the types of returns companies can achieve is a good way to demonstrate your thinking. audience: benchmarking in combo with ROI, middle vs senior mgmt. lou: not a big fran of ROI, it's all horseshit but depending on whom you're selling to, must be able to at least talk numbers Technique 2: narrative case studies and storytelling Goals: -portrya a situation or story with which a client or prospect can identify -demonstrate the success of the featured solution Typical story arc: - establish actors - portray pain and stories of past failures and failed solutions - introduce featured solution (i.e. IA) describe how it works/what happens need some stories handy, some horrible arc in the beginning, namely -- we had an intranet that came out of some printed material, dumped it on intranet and didn't work, etc. Can see in their faces if you're connecting, emotionally. - portray positive results (qualitative and quantitative) Audience & Context anybody found it useful to tell stories? experience doing it? - used a cartoon to tell about a site that was a person sitting at a desk, and several doors, unclear which door to use to get which information, all kinds of advisors standing there telling him what to do... in other panel, different, the advisors were waiting to listen effective, because humorous, and clear some ppl aren't going to take to stories very well esp if they'r emore driven by numbers and you have to be able to tell it well whoever you're talling it to probably does have some experience with it potential clients love to hear stories about other clients... one of the things that helps is that you mention a past client has faced X and that helps them feel more comfortable about their own problems. Space... face-to-face situation more effective... closes distance between storyteller and audience emotionally engaging Technique 3: Therapeutic Sales Goal: help participatns get in touch with their "information pain" try to get them on the couch couple of ways... get someone to sit with you and talk about what is going on, and also the group approach, and they hear each other. like group counseling Get them on the couch: safe environment, but have to have established some level of trust... not in the first moments of meeting Give them building blocks ... give them a seminar, a book , something... to help them realize this is real, others are doing it. Give them some language. then you're getting them to talk about their specific situation in a combination of their language, specific to their industry, and yours, the lingo of IA. You've converted them somewhat, and you've learned some of their setting and their vernacular. Let them talk: shut up and listen! Highly dependent on your own skills as a listener/therapist (start with an hour and a half seminar type thing, something provocative... have to break the ice, then shut up and get them to talk) Ice breaking is key: someone has to 'go first' Can be a good internal tool as well. I make 3 min or less powerpoint animation thing movies, and explain to them in their logic + frustration. Could be with her own internal help system she works with. Being a logical person, try typing in X and see it doesn't work...then explain that it can be fixed, with time, money, etc. Hope... gottta hold out hope to them. Can't repeat "it isn't easy" so much that it becomes perceived as hopeless. If you show you've been listening by using their vernacular, it means a lot to them. Technique 4: Competitive Analysis identify competitors (or other companies successfully facing similar challenges) sometimes use as part ofprocess to show we understand the space, but usually part of the requirements gathering The me-too argument for proposed changes: "everybody else is doing it" but need to refine that. List of 5, 6 or more competitors... or other companies implementing similar functionality and trying to implement. reverse-engineering business strategy. not every site is the same, the goal though is to stystemically look across the space, see how different sites choose to organiza themselves. compare and contrast approaches to arthictecture, nav system design, and page layout in a lot of cases, you can talk about where competitors are failing, or unmet opportunities, gaps out there - is there a way this technique can be employed to encourage the practice if IA itself... I do that a lot more in illustrating and comparint to clients different approaches to doing different things. one of the risks... things that have imploded... we've educated the client thru competitive, etc, user experience, nav schema, what ind is doing, and after we go into development, the extended team may not have read the assessment that was done, and they use the assessment to pick apart the prototypes... internally: wash post, ceo was fixated on NYT website.. couldn't think about anything else, took it as an opportunity to explain to him about why our navigation confses users... if you try to talk IA it puts him asleep, but NYT got his attention. If somebody's fixated on competitors it's a great way in. Useful for educating yourself about an industry Lets other companies make your mistakes for you -- many large sites have done extensive testing "Users spend most of their time on other people's sites" Don't commoditize yourself out of the market - competitive analysis should inform decisions but not dictate them we're going out and donig user studies, profling, good research ... on other side, mangmt side, yeah but our competitors did such and such... if that's such a good thing, why aren't our competitors doing it. jess: it really is selling a solution that involves IA...when you're doing competitive analysis. you don't say "and it will involve card-sorting"... gets back to selling a solution, it's been detrimental to sell IA for me... but pepole are realy into things that fix their problems. zap: sometimes ppl are incented on inmplementing certain features... if we can say "we can do a site that provides these features" it works... but with sr mgmt it can be bad to do taht, seen as a commodity -- at higher levels, have had more success talking about process to getting to solutions : using >>> to collect nonverbal requirements... we can collect a lot of things that pepole noremally canot express verbally... Technique 5: Eating your own dog food Goal: help site owners understand the value of good IA by making them use their own poor IA... make them use it context: use one on one with decision makers and, when possible, sr managers timing best at the end of a long design cycle or when you believe a new cycle needs to begin; helpful at instantiating continual improvement may end up riding a runaway pony- something like that is getting the exec in the observation room watching their users. but that can get out of hand too... they can hear what they want to hear; perhaps show highlight tapes... but more palpable if in person borrow 1-3 exercises from task analysis studies a rare opportunity but more decision-makers are eating their own df anyway Technique 6: user testing and diagnostics - like eating yr own df watching actual users stuggle can be persuasive identify key tasks people wouldp erform on the site and ask rep users to complete them use as a diagnostic tool to identify problem areas define metrics for success and use as an evaluation tool to ensure quality having decisionmakers behind the glass...or a streamed feed of live testing pros: seeing it live makes problems seem important cons: can be difficult to manage expectations video highlights: pros: less time-consuming; can cut together relvant clips cons: requires video editing skills and equipment; costly written debrief doc: pros: can take a systematic approach to rpboems and solutions cons: lacks emotion of observing real users describing usability testing... everybody's gonna see it differently, but seeing how people like thigns or if they don't work, find when you have lots of people. and read behind the face of it and look into the underlying cause... also need to distinguish between opinion and problem. sr mgrs: in any user testing... saw exact same reactions from 3 diff mgrs: these are the stupidest people, cross arms and sulk, then "they're asking the wrong questions"... here's the list... i wanna see it done tomorrow. maybe combine in a little quicktime movie, and controlling what's going on, and still getting emotional impact, can stick on a fileserver or something. send link to 60 ppl in company Technique 7: getting user feedback - asking users what they say, getting their stories/discussion/feedback thing is that spontaneous feedback could be discounted... and somebody might also say the only problem they have is this one thing, not worth messing with. Allying with customer service: - pros, get valuabel perspective from people who deal with problems every day; roi if you can fix the problems - cons: aforementioned roi may bcome at expense of allies counting complaings/hate mail: - pros: quantitative - cons: numbers not entirely accurate, fanatics tend to write power compnay, trying to get customer sto use online billing, but didn't save them that much if the cust svc reps now have to manage more calls becasue ppl didn't know how to use the site ro were confused, reps ahd to be trained on how to explain things, how to support site, and esp if they're trying to maintain 5 different sites. When you talk about reduced costs, that reduced costs often come in salaries... talking with ppl in cust svcs is great but it may mean they get cut... Surveys: pros: quantitative and qualitative data; easy to gather; may integrate with log analysis cons: not always accurate reflection of real nuymbers. some freeform, some structured. relatively easy to gather. intersted in ways you can integrate that with log analysis, etc. rather than just asking opinions after the fact, being able to analyze along with real data of activities. cust svc: good place to find if ppl eating own dog food too... watching them use our own site. another grp to get friendly with is the trainers, who use your web based app. Technique 8: log analysis (both server and search) Goals: - portray how ewell existing existing site addresses real user needs - uncover problems and gaps in content, interface, and IA - determine specific ares of improvement (not selling IA here, but components thereof) Context: generally as aprt of ongoing design process -0 identify problem areas (eg where users get stuck) that IA improvements could address... ]what happens during the context of a WHOLE session - beneficial to designers and developers, less so to managers Benefits: large volumes of quantitative data based on real use typicall already they should also make you nervous, though, because only a piece of a much larger experience. must take with big grain of salt. ppl do searching in combination with browsing, and need to look at several logs at once. not really good tools out there for log analysis... don't have the flexibility we want. these reports are designed with someone else in mind. Problems: too much data, tools not always great incomplete picture of usage bucause a lot o finterp is involved, your anlaysis ... getting in at the strategy phase... very important. easier to sell IA then. Scenario exercise: 3 of them. Scenario Discussion Scenario 1 -- dotcom of big media site, dotcom home page linked to diff biz unit sites... Sony Team b: decision makers [SEE SLIDES FOR DESCRIPTIONS] Summary, Scenario 1: talked a bout a few diff things... did some traffic log analysis, found we were losing a lot of ppl who were bailing on homepage, couldn't find what looking for, if looked at logs on subsites, etc, compared, we lost ppl then. Our first argument. Even a step back, listened to their position .. we're just a home.com site, our primary audience is corp gov community and staek holders... we think most of our users use google to get to where they need, not helping filter or help users. but out of 3 million users 1.5 didn't move beyond homepage division into silos... against users wants, and corp strategy. cross-selling. media mentions ... media mentions of the sites, negative. another t hing was ppl can't use same shopping cart on diff sites. making a play for taking away some of the boring meta data sand search from that and l3et those ppl focus on branding, but give them better consistency and user experience. Scenario 2: New prj mgr put out rfp that she'd adapterd form another project, consultants wanted to propose a diff approach incorporating redesinging the search . proposer: disconnect between customer's expectations of what they needed, and the real root cause. they thaught they needed to change nav and graphics. what they really needed was affect use of searchability. we sould change the nav capability in order to more effectively ofcus on an improved search engine... if thre's a big disconnect between what they ask for and what we want to deliver, it won't make em happy. Also they wanted to share in some of the risk, so if our new approach didn't help them meet targets, there'd be an appropriate decrement of our fee...but if exceeds, we mutually gained. decisionmaker: new proj mgr was we don't know wat it's suposed to be, want to do good job on this new project. that's really nice, but i don['t actually care how you will make it better unless it gives us some financial benefit, makes us look good to our bosses. we weren't actually the ones who decided it needed redesign... we wanted to make sure we could be invovled in the porcess, and that imp ppl in my company also got their input . Scenario 3: wants a job and involved creating user exp position somewhere that has other roles started with thinking that (since already worked at co), we could make up aproblem for them, this vp didn't talk to that vp, needed someone who would communicate, but that failed because they responded by what do you mean? strength would be as interpreter between designers, marketers, it, etc. also tried to push idea of info arch as someone who could push strategy, the vp's strat goals... didn't work well in this situation, weren't very confvincing. typical of mid mgmt, didn't know what they didn't know. what skills they thaought missing, what bring to team... weren't aware they would have this tack of telling us what's wrong with our organizaztion already... did do a good job of framing as liason with good horizontal skills, didn't know what we were missing, bringing up skill so flib scientist... we have user researcher, designer, etc. why are we missing this? on the fence still as to if tetting their job. why apply as an IA if you could apply under diff title and do the same stuff? according to info we got, we weren't qualified for interactiond esign... but maybe we could try to make that case? Easier for mgmt to add another person to an existing role, than to create new role... Most convincing was the coordination among the positions was important. Some ppl threatened discussing strategic issues... if it's your job and it feels someone is threatening you... (losing her logic here) wodke: some jargon was used... could never get hands around why lib sci was important, for example... caught up on the terms. when tryin gto create a new role, use ordinary language or their vernacular.