Research Instruments of Data Collection

Research instruments include the following: questionnaires, interview schedules, observation and focus group discussions.

In formulating research instruments the researcher should ensure the following:

a)  The objectives of the study are clear. This will assist the researcher to anticipate the type of information needed.

b)  The population sample. The researcher should be aware that some types of instruments are unsuitable to some groups of people due to factors such as the literacy level, profession and culture. A researcher should determine the literacy level of the study population in advance. For the illiterate, interview and focus group discussions should be used. The type of language that will be used (either English or Chinyanja) will depend on the literacy level of respondents.

c)  Geographical distribution. The span of the study dictates the type of instrument to be used. A countrywide study may require the use of postal questionnaires and telephone interviews.

d)  A researcher should be careful about the questions he/she asks. A researcher should do the following:

• Begin with a few interesting but non-threatening questions.
• Avoid vague questions, for instance, “What do you like?”
• Keep the language simple.
• Limit each question to a single idea. Ensure each item included has a specific purpose.
• Only include questions that are directly relevant to the study.
• Have a logical sequence.
• Do not put the key questions at the end of the questionnaire. It is best to have them in the middle.
• Avoid emotionally charged words.
• Avoid leading questions, for example, “Do you think students riot because they are unfairly treated?”
• Avoid acronyms and abbreviations.
• Consider the order of questions related to each topic.
• A researcher should vary closed and open ended questions. Closed questions give the respondent a set of choice or options. Open-ended questions are free response type questions. They allow the respondent to answer in their own words.

e)   Check the consistency of answers. It may be beneficial to ask the same question again using different wording. This ensures validity.

Questionnaires

In a research situation, a questionnaire can be said to be a carefully designed instrument for collecting information or data in relation to the specifications of the research questions and hypotheses. This instrument is used to elicit written responses from the subjects of the research through a series of questions or statements put together with some specific objectives in mind. It can be used to ascertain facts, opinions, beliefs, attitudes, practices, etc.

Questionnaire is the most common instrument used in research, it is used like a test and is constructed for specific purposes. It is also used for the assessment of students’ personal-social adjustment and interest with regards to different issues. You need to think of a specific study and design before you determine whether it will be appropriate for you to use a questionnaire.

For instance, when you require to obtain data on the distribution of a group of subjects in relation to such factors as gender, state of origin, state of residence, qualification, experience on a job, age, socio-economic status or to provide information for assessing certain situations such as the availability of workshop equipment, laboratory facilities, facilities in a state, school, or the extent of implementation of a certain school programme etc.

It can be used to obtain information on the feelings and perceptions of a group of people towards certain issues such as the ODL system of education, or the perception of the problems or their attitude towards the problems associated with the use of information and communication technologies in our educational system.

Questionnaires can be used without direct personal contact with the respondents i.e. without the help of an interviewer. They are self- administered and should be filled in by respondents themselves. This can be done either by distributing the questionnaire and collecting after it has been filled out, or by mailing it and asking the respondent to send it back. The latter is called a mail questionnaire which is definitely a non- personal method of gathering data.

The ultimate power of the questionnaire as a research instrument is in that it can be used to gather data over a large sample. The various advantages questionnaires include the following:

• Information can be collected from a large sample and diverse regions.
• Confidentiality is upheld.
• Saves on time.
• Since they are presented in paper format there is no opportunity for interviewer bias.

However they have their disadvantages in that:

• Response rates can be quite low.
• There is no direct contact so the researcher cannot deal with any misunderstanding.
• There is no opportunity to ask for further information related to answers given.
• No clear reason can be given for incomplete responses.

Steps in Formulating a Questionnaire

The following are essential in the formulation of a questionnaire:

a)  Reflection – Before formulating a questionnaire it is important for the researcher to reflect on the aim and objective of the study. The researcher should reflect on the type of response expected.

b)  Formulation of questions – The researcher should write down questions related to each stated objective. While constructing the questions the researcher should begin with a few interesting but non­-threatening questions. The researcher should only include questions that are relevant to his/her study. The researcher should keep the key questions in the middle.

c)  Pilot – After constructing the questionnaire, the researcher should try it out on a small sample of the population. While piloting the researcher should address the following questions:

• Are the questions measuring what they are supposed to measure – the researcher should analyse each answer and see if it is supplying the appropriate information.
• Is the wording clear? The researcher should analyse the responses to find out if there was any confusion in the way questions were interpreted by all the respondents.
• Do the questions provoke a response? If some questions have been omitted, the researcher should find out why.
• Is there researcher bias? The researcher will analyse whether the questions asked were skewed towards certain issues more than others.

d)  Evaluation – After piloting and making the necessary amendment the researcher should carry out an evaluation of the revised questions. This includes finding out if the questions are clear and specific, where the key questions are placed and if the balance of questions is correct.

Qualities of an Effective Questionnaire

You have seen that the questionnaire is a very popular method of data collection in education and behavioural sciences. The reason can be attributed to the relative ease and cost-effectiveness with which it is constructed and administered to large samples when compared to other methods. To serve its purposes very effectively, the questionnaire you construct must be characterised by the following features:

• : The instrument should be relevant to the purpose of the research. It should elicit all the information necessary for answering the research questions and testing the hypotheses. It should also consider the background and experience of the respondents.
• The instrument should be able to yield responses that are consistent. The responses of a group of people to the instrument on two different occasions should be as close as possible on these occasions.
• The instrument should as much as possible be usable. It should not be too long or so bulky. The conditions for the administration and the method for interpreting the data elicited should be fairly simple and easy.
• Both the instructions and the items should be clear enough to avoid possible misinterpretations. You should note once again that a good questionnaire should not contain ambiguous items or instructions.
• The responses from a good questionnaire must be easily quantifiable. It should be easy to assign numerical values or figures to such responses in a manner that is systematic.
• All the items should be legible without tiny characters. The words should be properly spaced with appropriate side margins. Always use the computer to type your questionnaire so as to produce very neat and legible instrument.

Advantages and Disadvantages

You have noticed that in every section of this unit, one or more advantages or disadvantages may have been mentioned. Let us now summarise them briefly.

Advantages

• It is economical in terms of time and cost;
• It can be administered to variety of people;
• It is easy to administer and even to distant respondents;
• It can be used to get information on non-cognitive constructs such as kindness, stress, anxiety etc.;
• Great percentage of students or respondents can be reached at a time.

Disadvantages

• There may be low percentage return which may lead to the distortion of the findings;
• There may be misunderstanding or wrong responses if the item s are not clear or the instructions are misleading;
• Wrong or negative or faked answers may be given if the instrument is too lengthy or if it is intruding on the respondents’ private life;
• It is not suitable for the illiterates, semi-illiterates, and children.

Interviews as Research Instruments

Interview can be regarded as a face-to-face interaction in which oral questions are posed by an interviewer to draw out responses from the interviewee. An interview of whatever type involves direct personal contact with the participant who is asked to answer questions.

A method of getting people to express their views broadly on a certain issue is the non -scheduled interview which consists of asking respondents to comment on widely defined issues, those interviewed are free to expand on atopic to focuses on some aspects, to relate to their own experiences, etc. The interviewer will only intervene to ask for clarification or further explanation, but not to give directives or to confront the interviewee with probing questions. Usually no time limit is fixed for completing interviewing.

The interview as a method of data collection passes through four major phases. These phases overlap and interact. They are preparation, establishment of rapport, the question- answer and the recording phases.

Preparation Phase

The degree of success in an interview is dependent on how well you have prepared for it. The preparation stage is when you take decisions on the mode of recording the responses. You need to check the recording instruments for validity and reliability. You need to trial-test the instruments to ensure that they are in good working conditions.

For instance, if you are going to use a tape recorder, you have to see that the batteries are good and tested. If you need to use gifts to express gratitude or to establish rapport, you need to make the correct selection.

Again, during this stage, you will ascertain the cultural background of your subjects, so that appropriate salutations may be used, appreciate wears may be selected in advance. You also need to ascertain the biographical data of your subjects so as to address them appropriately. Your questions should be derived from well-defined hypotheses or research questions.

They have to be edited in terms of such issues as appropriateness, length, relevance, palatability, clarity, simplicity, precision, language and the conceptual levels. You have to validate the questions through expert validation and trial-testing. After the validation, you would review the questions thoroughly, memorise them, if possible and rehearse it to mastery level in the sequence in which they will be asked.

Rapport Phase

For you to collect valid data, you need to establish a cordial atmosphere for the interview. There is no rule for doing this, but as a mature investigator, you have to survey the situation and evolve appropriate strategies. You may follow any of these suggestions:

• Courteously seek permission from the appropriate authority. In doing this, you should provide information about the objectives of the study and the nature of the interview.
• Give notice to the interviewee and book appointment for date, time and venue. The venue should be comfortable in terms of sitting, ventilation, lighting and decoration. The venue should be noise-proof.
• Take note of your appearance. It should be appropriate, neat, and inoffensive. Consider your dental and body deodorisation and any other aspect likely to irritate your subject.
• Your first contact with the interviewee must be friendly, pleasant and courteous. Use appropriate salutation and address them with the correct title. Pay some compliments, but not flatter. This will make them more relaxed.
• You need to be relaxed too. Introduce yourself briefly and modestly and also introduce the problems which are the focus of the interview. You may need to start by asking the first few minutes for a short conversation to enable you and your subject to relax. You also need to assure them that their responses will be treated confidentially and only for the research purposes.

Otherwise, they might be suspicious and unwilling to respond to the questions freely.

Question-Answer Phase

This is where you have to make use of your skills and expertise to make the session more permissive, flexible and interactive. You have to keep the interviewee, interested and responsive till the end of the interview.

In doing this, you have to devise appropriate strategies, but you have to be pointed and business-like and not to wonder aimlessly. When you ask a starter question, follow it up with prodding and probing questions to get comprehensive information. Where you need to use pictures or related objects, drafts, drawings etc. use them for probing to elicit further explanations and reasons for earlier statements. But you should try to avoid interviewer bias by being as non-directive as possible. In all, you have to be relaxed, not tensed up or nervous throughout the duration of the interview.

Recording Phase

Recording is a very important aspect of interview. You must try to comprehensively record information from an interview as unobtrusively as possible. Recording may be done in any of the three methods.

• Mental Note

If you want to commit the responses into memory, you can be sure; it removes apprehension on the part of the respondent and therefore increases the rapport. But you have to appropriately assess yourself in terms of memory retention before using mental notes. This is because any information forgotten is as good as missing and may not be easily reconstructed.

• Written Note

It is advisable to use written records when there are too many questions and responses. But extensive writing is likely to excite or offend respondents. If this happens the rapport and validity of the responses will be reduced. If you are versed in shorthand, it can be an advantage in written notes. You can also use structure interview schedule where alternative responses are provided. Here, there is minimum writing, but you have to make allowance for unanticipated responses.

(c) Taped Records

In order to solve all the problems of memory loss and that of extensive writing, tape recording of interview is the answer. It removes strain from the interviewer. It can be replayed and transcribed at leisure. But you have to bear at the back of your mind that audio and video recording instruments can go faulty in the process. You must take care of this at the planning stage. Other problems like instrument reactivity should also be taken care of.

Recording can also frighten or excite the subject. This may channel away his/her attention and bring about distortion in his/her behaviour. As far as possible, conceal your recorders. You can use micro recorders, remote censors or pick up buttons when available.

Interviews are basically in two types. These are the face-to-face interview and the telephone interview. You have to note that whether it is face-to-face of telephone interview you are using, they have the same characteristics. The only difference between face-to-face and telephone interview is that in telephone interview no one sees the other. The advantages of the telephone interview are that it is cheaper, especially if the respondents are very far away, and the interviewee is shielded from the influence of the interviewer. Interviews can also be classified in the following categories:

Unstructured Interviews

In this approach to interviewing, the researcher has some idea in mind of the topics to be covered and may use some sort of topic list as a reminder. There is minimal control over the order in which topics are covered and over respondents’ answers. In unstructured interviewing, neither the specific questions to be asked nor the range or type of possible answers are pre-defined. They are informal and con­versational. The aim is to get the informants to open up, and the researcher should stimulate an informant to produce more information.

This approach allows the interviewer to be responsive to individual differences and situational characteristics. This approach builds on observation. It is useful in studying sensitive topics such as sexuality or political topics. To effectively achieve the aims of an open-ended interview, one must ask a whole series of secondary questions such as:

• What do I want to get out of these interviews?
• With whom am I going to conduct these interviews?
• How do I know they will talk to me?
• How many interviews must I do?

As much as possible, test your methods in advance. Because there is no set format for conducting these interviews, each interview is unique. This makes it difficult to systematise and analyse data.

Advantages

Unstructured interviews have the following advantages:

• They are flexible. This is because there are no pre-defined questions. This allows the respondents to freely respond to an issue. The researcher can therefore gather a lot of information.
• The respondent feels part of the team since no rigidity is displayed. He/she can therefore freely participate in the research.
• Since it is a free response in a relaxed atmosphere situation, the answer given are more reliable.
• It allows the interviewer to be responsive to individual differences and situational characteristics.
• This instrument is useful for studying sensitive topics such as sexuality and abortion.

Disadvantages

Unstructured interviews have the following disadvantages:

• They are time consuming since a respondent can dwell on one issue.
• They are not systematic as a respondent can comment on issues in a haphazard way. A respondent can comment on issues already discussed.
• If the researcher is not careful, it can get out of control, with the respondent getting too emotional or personal.
• Irrelevancies can be displayed by the respondent.
• Since there is no set format for conducting these interviews, it is difficult to systematise and analyse data.

Semi-structured Interviews

These interviews are based on the use of an interview guide. This is a written list of questions or topics that need to be covered by the interview. There are several types of semi-structured interviews.

1. Focused interviews – This intensively investigates a particular topic. They aim at gaining a complete and detailed understanding of the topic.
2. Case studies – The purpose of case studies is to collect comprehensive, systematic and in-depth information about particular cases of interest.

Advantages

Semi-structured interviews have the following advantages:

• They are flexible. This is because they consist of both open and closed-ended questions.
• In-depth information is gathered by closed ended questions.
• By using both the open and closed-ended approach, the researcher gets a complete and detailed understanding of the issue under research.

Disadvantages

Semi-structured interviews have the following disadvantages:

• They can be time consuming due to the open-ended questions.
• Analysis of data may be problematic.
• The respondent may be cautious of the answers given in close- ended questions.

Structured Interviews

These involve subjecting every informant in a sample to the same stimuli, for instance, asking each informant similar questions, as in the case of a survey.

Advantages

Structured interviews have the following advantages:

• The reliability of the information gathered is high. This is because each informant is subjected to similar questions with the others.
• It gives in-depth information about particular cases of interest to the researcher. This is because the researcher seeks information on specific issues.
• It is systematic. Researchers intensively investigate a particular issue before moving to the next.
• It is time-saving since the respondents simply answer what has been asked by the researcher.
• The researcher gets a complete and detailed understanding of the issue from the respondent.
• It is comprehensive and systematic since questions are formulated before the interview.
• The data collected is quantifiable.

Disadvantages

Structured interviews portray the following disadvantages:

• The rigidity displayed by the researcher can affect the responses given. The respondent may feel as if he/she is under investigation and is being probed. This may affect the response. Some of the respondents may become hostile.
• It is too formal. Since the researcher does the questioning and the respondent simply answers, the respondent may be too cautious in the answer given. The respondent may give answers he/she thinks are acceptable or will impress the researcher.
• The researcher may miss out on some important points that are not included in the questions formulated.

Focus group discussions

This is a special type of group in terms of its purpose, size, composition and procedures. A focus group is usually composed of 6-8 individuals who share certain characteristics, which are relevant for the study. The discussion is carefully planned and designed to obtain information on the participants’ beliefs and perceptions on a defined area of interest. Special predetermined criteria are used in selecting focus group participants. This includes the following:

1. The topics to be discussed are decided beforehand.
2. There is a predetermined list of open ended questions.
3. Focus relies on discussion among participants about the topics presented.

This method requires thorough planning and training of group moderators. Focus groups should usually be composed of homogeneous members of the target population, for instance, similar in age, education level, gender, profession. Focus group discussions can produce a lot of information quickly and are good for identifying and exploring beliefs, ideas or opinions in a community.

However, the researcher has less control over the flow of the discussion and results are hard to analyse. Focus group discussions are used to assess needs, develop intervention, test new ideas or programmes or improve existing programmes.

Observation

Although a seemingly straight forward technique, observation must be pursued in a system way, following scientific rules, if usable and quantifiable data are to be obtained. It is a tool that provides information about actual behaviour. Direct observation is useful because some behaviour involves habitual routines of which people are hardly aware. Direct observation allows the researcher to put behaviour in context and thereby understand it better. Observation can be made of actual behaviour patterns. Forms of observation include the following:

Simple observation/non-participant observation

This is the recording of events as observed by an outsider e.g. an observer placed at a road junction can observe traffic and record numbers of cars passing or pedestrians crossing the road, the speed of the cars, number and cause of accidents. But this method has some weakness. People who feel they are being observed might change their behaviour, become uneasy or stop activities altogether.

Participant observation

To avoid this indirect interference with participant observation can be used. In this case the observers hide the real purpose of their presence by themselves becoming participants. They join the community or group under investigation as one of its members, sharing in all activities. Becoming an insider allows deeper insight into the research problem since one enjoys the confidence of the participants and shares their experiences without disturbing their behaviour.

The weakness of this method is that researches risk losing their objectivity. Being directly involved with people and their daily concerns for an extended period of time may predispose one to be effectively engaged and thus lose detachment from people and events. Also because notes might have to be taken down secretly or form recall, inaccurate information may be recorded.

Participant observation is clearly a very demanding method of gathering data and might involve extended periods of residence among respondents. For this reason a modified participant observation methods restricted to participation in only major events, such as village meetings or ceremonies are often preferred.

• The third type of observation is that done under laboratory conditions, mainly used in psychology. It is hidden observation of the behaviour of one or more persons in a room with one -way windows or false mirrors. In this case, although the negative aspects of a laboratory experiments are present, the distorting factors associated with simple or participant observation are removed.

The following points are to be noted about observation:

1. Observations serve clearly formulated research purposes thus observations must be planned systematically, specifying what and how to observe.
2. Observations should be recorded in a systematic, objective and standardised way
3. Observation should be subjected to control in order to maintain a high level of objectivity i.e. many observers should be able to record the same phenomena or events.
4. Observation as a method of data collection has some major limitations that need to be noted; not only is it costly and time consuming, but it cannot be applied to many aspects of social life. One cannot observe attitudes or beliefs for example; neither can one easily observe phenomena spread over a long period of time (like career of a politician).

Standardised tests

Standardised tests of one sort or another are used in most educational research studies. A researcher will frequently use standardised tests to measure one or more of the variables in a study. It is important that one gets as much information as possible about the tests to be used in the study. In some cases no suitable instrument exists to measure the variables of the study.

In that case, the researcher will have to design their own instrument. One can look at the instruments (such as questionnaire forms), which have been used in similar studies and modify these for use in his/her own study. There are many different types of tests that one might consider for use in their study. Some of the most commonly used types of tests for educational research are:

• Achievement tests.
• Personality tests.
• Aptitude tests, including tests of academic aptitude (intelligence tests).

Characteristics of standardised tests are:

Validity – the validity of a test is a measure of how well a test measures what it is supposed to measure. The examiner’s manual or technical manual for most tests will have information on the validity of the test.

Reliability – reliability is a measure of how consistent the results from a test are. If you administer a test to a subject twice do you get the same score on the second administration as you did on the first? The reliability of the test is the answer to this question.

Experimental techniques

Laboratory experiment is the most controlled method of data collection. It stimulates certain characteristics a natural environment but, only as much as it does not affect the control and the manipulation of independent and other variables at stake.

The main distinction between this method and all the others is that only the laboratory experiment allows the manipulation of one variable at a time. In order to study its effects on other variables all conditions are standardised: the presentation of the experiment and the testing situation, the environment, the laboratory, are the same for all participants in a study.

The instructions are rigorously the same given to the participants by an experimenter who avoids any personal interference with them. The instruments are strictly the same, they are produced under copyright to avoid the slightest modification. The recording and the evaluation of the results are systematically done and usually quantitative measurements are obtained.