End-User
Surveys for Design Diagnosis
Issue addressed
“We’ve done lots of surveys, but how can we use them to make solid decisions about the design and direction of our product? Are other methods more effective for our project's specific needs?”
Your challenge: Get the best data for your project needs
It's easy to collect end-user satisfaction ratings about your product. Anyone can write a questionnaire, use it to collect data, and draw conclusions. However, it is also easy to collect data of dubious quality, and even easier to collect data that cannot be used to meaningully evaluate and improve interactive design.
First, I advise whether a questionnaire/survey is indeed the best method that will serve the specific requirements of your project. Sure they're easy to outsource or do yourself and the results are easy to understand, but for feedback regarding interactive products, self-report data is only the "tip of the iceberg:" people simply aren't that good at self-reporting on relevant details of their past experiences and behaviors. Chances are, there is a behaviorally-based method that will better serve your your project's needs.
Where surveys are appropriate, I write content that will provide high-quality data that can be used to drive design decisions for your product. This not only requires an understanding of the design process, but also some serious psychology, including:
- psychometrics and psychological scaling theory
- respondent sampling procedures, including targetted invitations and incentives
- questionnaire construction (content, response alternatives, item ordering, skip patterns)
- delivery methods (telephone, online questionnnaire, event-triggered pop-up)
- data file formats that enable effective analysis, including parametric and non-parametric statistical analysis and keyword-parsing of comment data
In addition, the usability of the survey session itself can dramatically affect the validity and quality of results.
Don't base decisions on misleading data!
I was a university professor with doctorate-level education and experience in the fundamentals of questionnaire/survey design and statistics. I have completed several large-scale surveys for software-development projects, with input from thousands of end-users. I also know what sort of data surveys will give you, and where other methods will better serve specific needs.
Contact me to discuss your end-user survey requirements, or to discuss where surveys should fit within a comprehensive user-centered design strategy.
Last update: May 07, 2002
(c) 2002 Don Hameluck Usability Consulting Inc.
