Questionnaire is a method used for collecting data; a set of written
questions, which calls for responses on the part of the client; may be
self-administered or group-administered. Questionnaires are an inexpensive way
to gather data from a potentially large number of respondents. Often they are
the only feasible way to reach a number of reviewers large enough to allow
statistically analysis of the results.
A well-designed questionnaire that is used effectively can gather
information on both the overall performance of the test system as well as
information on specific components of the system. If the questionnaire includes
demographic questions on the participants, they can be used to correlate
performance and satisfaction with the test system among different groups of
users. It is important to remember that a questionnaire should be viewed as a
multi-stage process beginning with definition of the aspects to be examined and
ending with interpretation of the results.
Every step needs to be designed carefully because the final results are
only as good as the weakest link in the questionnaire process. Although
questionnaires may be cheap to administer compared to other data collection
methods, they are every bit as expensive in terms of design time and
interpretation. The steps required to design and administer a questionnaire
include:
1. Defining the Objectives of the survey
2. Determining the Sampling Group
3. Writing the Questionnaire
4. Administering the Questionnaire
5. Interpretation of the Results
This document will concentrate on how to formulate objectives and write
the questionnaire. Before these steps are examined in detail, it is good to
consider what questionnaires are good at measuring and when it is appropriate
to use questionnaires. What can questionnaires measure? Questionnaires are
quite flexible in what they can measure, however they are not equally suited to
measuring all types of data.
We can classify data in two ways, Subjective vs. Objective and
Quantitative vs. Qualitative.
When a questionnaire is administered, the researchers control over the
environment will be somewhat limited. This is why questionnaires are
inexpensive to administer. This loss of control means the validity of the
results is more reliant on the honesty of the respondent. Consequently, it is
more difficult to claim complete objectivity with questionnaire data then with
results of a tightly controlled lab test.
For example, if a group of participants are asked on a questionnaire how
long it took them to learn a particular function on a piece of software, it is
likely that they will be biased towards themselves and answer, on average, with
a lower than actual time. A more objective usability test of the same function
with a similar group of participants may return a significantly higher learning
time.
More elaborate questionnaire design or administration may provide
slightly better objective data, but the cost of such a questionnaire can be
much higher and offset their economic advantage. In general, questionnaires are
better suited to gathering reliable subjective measures, such as user
satisfaction, of the system or interface in question. Questions may be designed
to gather either qualitative or quantitative data. By their very nature,
quantitative questions are more exact then qualitative.
For example, the word "easy" and "difficult" can
mean radically different things to different people. Any question must be
carefully crafted, but in particular questions that assess a qualitative
measure must be phrased to avoid ambiguity.
Qualitative questions may also require more thought on the part of the
participant and may cause them to become bored with the questionnaire sooner.
In general, we can say that questionnaires can measure both qualitative and
quantitative data well, but that qualitative questions require more care in
design, administration, and interpretation.
When to use a questionnaire?
There is no all-encompassing rule for when to use a questionnaire. The
choice will be made based on a variety of factors including the type of
information to be gathered and the available resources for the experiment. A
questionnaire should be considered in the following circumstances.
a. When resources and money are limited. A Questionnaire can be quite inexpensive to
administer. Although preparation may be costly, any data collection scheme will
have similar preparation expenses. The administration cost per person of a
questionnaire can be as low as postage and a few photocopies. Time is also an
important resource that questionnaires can maximize. If a questionnaire is
self-administering, such as a e-mail questionnaire, potentially several
thousand people could respond in a few days. It would be impossible to get a
similar number of usability tests completed in the same short time.
b. When it is necessary to protect the privacy of the
participants. Questionnaires
are easy to administer confidentially. Often confidentiality is the necessary
to ensure participants will respond honestly if at all. Examples of such cases
would include studies that need to ask embarrassing questions about private or
personal behaviour.
c. When corroborating other findings. In studies that have resources to pursue other
data collection strategies, questionnaires can be a useful confirmation tools.
More costly schemes may turn up interesting trends, but occasionally there will
not be resources to run these other tests on large enough participant groups to
make the results statistically significant. A follow-up large scale
questionnaire may be necessary to corroborate these earlier results
Characteristics of a Good Questionnaire
• Questions worded simply and clearly, not ambiguous or vague, must be objective
• Attractive in appearance (questions spaced out, and neatly arranged)
• Write a descriptive title for the questionnaire
• Write an introduction to the questionnaire
• Order questions in logical sequence
• Keep questionnaire uncluttered and easy to complete
• Delicate questions last (especially demographic questions)
• Design for easy tabulation
• Design to achieve objectives
• Define terms
• Avoid double negatives (I have not any money)
• Avoid double-barreled questions (this AND that)
•
Avoid loaded
questions ("Have you stopped beating your wife?")
No comments:
Post a Comment