SPD1088: English for Academic Studies II

Supplementary material: Research skills – Questionnaire design

Writing a workplace report involves researchers deciding on the method(s) which they intend to use to gather data (e.g. by analysing statistics, conducting interviews, carrying out laboratory experiments). One of the most frequently used methods of collecting data for business reports is the questionnaire survey. This method is also often used by university students and teachers when gathering primary data for academic reports and papers (e.g. the final-year project). Given the importance of questionnaire surveys in business and academic research, we have prepared this resources section in order to introduce you to some of the key principles of questionnaire design.

Conducting a questionnaire survey

TASK 1

The process of designing and administering a questionnaire generally consists of eight stages. These stages are listed below, but they are in the wrong order. Put them in the correct order by writing the letters in the spaces provided below. (1 = the first stage in the process)

(a) Revise the questionnaire on the basis of the results of the pilot.

(b) List the specific information that you want to find out.

(d) Pilot the questionnaire on a representative group of subjects.

(e) Draft and sequence the questions.

(f) Administer the final version of the questionnaire.

(g) Establish the topic, purpose and scope of the research project.

(h) Work out the overall structure of your questionnaire.

1. ______ 2. ______ 3. ______ 4. ______

5. ______ 6. ______ 7. ______ 8. ______

The organisation of a questionnaire

Questionnaires generally include the following sections:

1. Information about the topic and purpose of the project and instructions about how to fill in the questionnaire.

2. Demographic data about the subjects (e.g. age, gender).

3. Various types of questions designed to elicit the information required by the researchers.

4. An ethics statement, which may include information for participants about how their information will be used and how their rights and privacy will be protected.

PART 1

This paragraph tells the subjects how to fill in the questionnaire.

The second sentence explains how the information will be used.

Title of the project.

Name of the organisation conducting the survey.

The first sentence explains the purpose of the questionnaire.

Hong Kong Polytechnic University

English Language Centre

Workplace communication in Hong Kong

This questionnaire is designed to provide information about language use in the professional workplace in Hong Kong. The information that you provide will help the English Language Centre to design business English courses and materials which meet the communication needs of practitioners in a range of professions serviced by the PolyU.

When answering this questionnaire, please reflect on your language use in your current job. If you are currently between jobs, please reflect on your language use in your last job. Please mark all your answers on the red answer sheet provided.

All information collected will be regarded as strictly confidential. Thank you for your participation.

This encourages the subjects to provide accurate answers. Here is an annotated example of the first section:

PART 2

This paragraph tells the subjects how to fill in the questionnaire. The second section of a questionnaire provides researchers with a profile of their subjects. This is done by asking classification questions. The type of information sought will depend on the purpose of the project. In the example on the next page (which is taken from the questionnaire introduced above), the researchers are particularly interested in classifying the subjects according to their rank and the type of company or organisation they work for. We may assume that an important objective of the project is to compare language use at different levels and in different employment sectors.

Personal Information

Please circle the appropriate letter.

1. Gender: A) Female B) Male

2. Years of work experience:

A) < 5 B) 6 – 10 C) 11 – 15 D) 16 – 20

E) 21 – 25 F) 26 – 30 G) > 31

3. Rank in current job: A) Junior B) Middle C) Senior

4. Type of company / organisation: A) Government

B) Government-related

C) Private company

5. If you work for a private company, is it A) a HK-owned company

B) a China-owned company

C) a foreign-owned company

PART 3

The third section of a questionnaire aims to elicit the specific information required by researchers. Questions tend to fall into two categories: attitudinal (i.e. what people think) and behavioural (i.e. what people do). As you will see below, researchers use a variety of techniques to elicit this type of information.

Question types

Researchers use a mixture of fixed-response questions and open-ended questions to gather data about their subjects’ attitudes and/or behaviour. Fixed-response (or closed) questions ask the subjects to choose between a limited number of options (e.g. by circling a number on a scale ranking factor to determine importance).

Researchers tend to use fixed-response questions more than open-ended questions in questionnaires. Scales and ranking questions are perhaps the most common question types within the category of fixed-response questions.

Scales are generally used to measure opinions, attitudes, feelings and habits. A verbal rating scale asks a respondent to choose a word on a scale (which usually has five points). Here is an example of a verbal rating scale. This is taken from the questionnaire about workplace communication introduced above. As you can see from the instructions, the subjects are asked to indicate which language they use when writing in the workplace.

Look at the types of written texts in the list below. Indicate which language you use when writing each text type at work by circling the appropriate number on the scale.

Text Types Always English
Usually English Sometimes English, Sometimes Chinese
Usually Chinese Always Chinese Letters 1 2 3 4 5 Memos 1 2 3 4 5 Faxes 1 2 3 4 5 E-mails 1 2 3 4 5 Reports 1 2 3 4 5 Instructions 1 2 3 4 5 Minutes 1 2 3 4 5

A numerical rating scale asks respondents to give a numerical score out of four, five or six. At each end of the scale is a descriptor:

Strongly disagree 1 2 3 4 Strongly agree Very dissatisfied 1 2 3 4 Very satisfied Highly unlikely 1 2 3 4 Highly likely Poor 1 2 3 4 Excellent

Ranking questions are used when researchers want to determine degrees of importance. In the following example, respondents are asked to rank five factors relating to job motivation.

Indicate the importance you attach to the following ‘job motivators’ by writing the appropriate number (1-5) in the space provided. (1 = most important, 5 = least important) _____ Recognition for good work _____ A say in decision making _____ A good chance for promotion _____ Good pay and benefits _____ Fair treatment from the company

Open-ended questions give respondents the freedom to express their own ideas on the issue under investigation. These questions generally appear at the end of a questionnaire. Here is an example from the questionnaire about workplace communication.

Please answer the following questions:
What are the main factors which influence you to write workplace documents in English? ……………………………………………………………………………………………. … ……………………………………………………………………………………………. What are the main factors which influence you to write workplace documents in Chinese? ……………………………………………………………………………………….………. … ……………………………………………………………………………………………

TASK 2

Discuss and note down the advantages and disadvantages of fixed-response and open-ended questions.

QUESTION TYPE	ADVANTAGES	DISADVANTAGES
Fixed-response
Open-ended

Questionnaires should be clear and easy to complete. Here is a checklist of points to consider when designing a questionnaire. Read the checklist and then try to apply these ideas in Exercises 3 and 4 on the following pages.

CHECKLIST FOR QUESTIONNAIRE CONSTRUCTION
FORMAT
Is the questionnaire easy to complete? Are the types of questions used suitable? Are the items in the questionnaire properly numbered? Is the title clearly shown? Is the purpose of the questionnaire clearly stated? Are there clear instructions about how the questionnaire should be returned? Are the instructions clear? Are there smooth transitions between sections?
QUESTION CONTENT
Is the question necessary? Do the respondents have the information or experience necessary to answer the question? Is the question specific? Will the respondent give the information that is asked for?
QUESTION WORDING
Is the question clear and easy to understand? Is the wording biased? Is the question likely to be objectionable to the respondent in any way? Does the question adequately express the alternative when one is available? Would a more personalised wording of the question produce better results?
QUESTION SEQUENCE
Will the question lead to rejection by the respondents because it comes too early or too late? Will the answer to the question be influenced by the content of the preceding questions? Are the questions put in a logical order?

TASK 3

Before writing a questionnaire it is useful to be aware of some of the common mistakes which researchers make when writing questions. Here are some questions from a draft questionnaire investigating the language training needs in a company.

What is wrong with each question?
How would you improve them?

1. Age: 2. Educational background: [ ] 17 – 18 [ ] Secondary [ ] 18 – 20 [ ] Form 7 [ ] 21 – 25 [ ] Technical Institute [ ] HKCEE

3. Mother tongue: [ ] English [ ] Chinese

4. Please rate your ability in the following areas:

Excellent Very good Quite good Fair Poor Speaking

Grammar

Writing

Please comment on the reasons why our company is not doing enough to improve staff’s English proficiency.

6. Which aspects of English do you think our staff need most help with? Please rank the following from 1 to 12.
[ ] grammar [ ] word processing skills [ ] speaking [ ] listening [ ] handwriting [ ] report writing [ ] writing [ ] reading [ ] spelling [ ] vocabulary [ ] pronunciation [ ] paragraph writing

TASK 4

Design a questionnaire to find out some attitudes and/or behaviour of PolyU SPEED students. You should choose your own topic. Here are some suggestions:

Mobile phones – compare two major brands
Fast food shops in Tsim Sha Tsui – compare their services
Students’ reading habits – identify the types of book students need
Students’ spending habits – identify students’ problems with financial management
Students’ online habits – identify any possible health problems

Consider all the points mentioned in this section.

1. Decide the topic and purpose of your survey.

2. Consider what demographic information you need.

3. Decide which types of questions are most suitable to get the information you require: fixed-response, open-ended, a mixture of both.

4. Draft a questionnaire (including all three sections) and pilot it with a small group of students. Adjust any questions which do not work well.

5. Create a final version of the questionnaire.