Procedures

CHEQ coordinates the ordering and use of Brief Feedback questionnaires. Please fill out the appropriate fields in the Request Form and CHEQ will forward the appropriate questionnaires to you.

Once the Brief Feedback questionnaires have been completed and returned CHEQ will process them and generate a statistical summary of the responses. This is then sent to you plus any untranscribed written comments from students.

Results

After processing your Brief Feedback evaluation, CHEQ will then send you a report that presents the results of your evaluation. You now need to think about:

Interpreting results
Considerations in interpreting results

Interpreting results

This section gives you some guidelines on how to interpret your report. 

The top section of each page of your report (shown in Figure 1) has a header that identifies you and your department, details of the class and subject to which the questionnaire was administered, and the number questionnaires completed. 

Considerations in interpreting results

Figure 1: Top section of the Report

Below this header, a panel of boxes (shown in Figure 2) presents the response analyses for individual questions in the order that they appear on the questionnaire (the image below is Q6 from the Brief Teaching Feedback Questionnaire). 

Figure 2: Panels or boxes in the response analysis of each item in the questionnaire

Each panel shows:

• The question to which the data refer
• the number of students who gave each of the seven possible responses (five `answers´ plus the two `cannot answer´ possibilities)
• the percentage who gave each possible response (calculated over the number answering the question)
• the total number of students who responded to the question
• a graphical representation of those percentages
• a median response, and
• a semi-interquartile range (SIR). 

You should obtain a reasonable idea of the strengths and weaknesses that students identify, simply by looking at the distribution of responses in the graph. With this understanding, the percentages, `cannot answer´ responses and the median and semi-interquartile range add more information about how students´ responses are distributed. 

The median is an indication of the `central point´ in a distribution of responses. It is the scale value that divides the top 50% of the responses from the bottom 50% (the `cannot answer´ responses are excluded). The higher the median for any particular question, the more that students have responded towards the `All´ or `Almost all´ end of the scale.

The semi-interquartile range (SIR) is an indication of the spread of responses in a distribution. Most SIRs will be between about 0.4 and 0.8. The smaller the SIR for any one question the more that students have clustered around one or two responses. A small SIR value means that the students are in agreement on the question.

In summary, what is the ideal for which you should be looking?

• Most of the enrolled students completing the questionnaire
• Very few students leaving a question blank, or opting for a `cannot answer´ response
• The clear majority of responses sited at the one or two positions at the `All´ or `Almost all´ end
• Small percentages of responses sited outside those one or two positions 

Considerations in interpreting results

The reliability of your response data is adversely affected when the ratio of actual responses to the total number of students enrolled is low. In the header, check the total number of students enrolled and the number of questionnaires completed. For each individual question, check that most of the students completing the questionnaire also responded to the question (not including the two `cannot answer´ choices). This figure might be less than the number completing the questionnaire. 

If the number completed is less than about 60% of the number enrolled then your responses might not be representative of the perceptions of the students who have experienced your teaching. In the sample graphic, 100 students are enrolled in the subject. Of these, 95 students completed the questionnaire, and only 86 answered this question. It could be that those students not completing the questionnaire never come to class anyhow, but the point is that you need to consider representativeness. 

Assuming that you have enough students responding to the questionnaire, that those students are a reasonable proportion of the number enrolled, and that they have experienced sufficient of your subject or teaching, how do you interpret each panel? Is it possible to infer quite specific or targeted feedback from the questionnaire responses?

The short answer here is that it is not. That would be to expect something of the questionnaires for which they were not designed. Both the Teaching Feedback and the Subject Feedback questionnaires are deliberately brief and general. This is so that they can be used as vehicles for quick and frequent feedback from students, without restriction to any particular class format. For those instances where you need detailed feedback from your students you should use either a MonQueST questionnaire or a more purpose designed form with which CHEQ can assist).

Given then that the responses here indicate student perceptions over more broadly defined aspects of class teaching or of subject organization and operation, how can some meaning be extracted from those responses?

First, take a close look at the questionnaire questions. Remind yourself of what it was that your chosen questionnaire asked of your students. Look particularly at the descriptions that accompanied or defined each question. These descriptions were what the students were instructed to think about as the bases for making their individual responses. So, you in turn should also use those descriptions to think back on your teaching, to prompt yourself when considering to what the responses might be referring, in the context of your subject.

Next, look over any written comments that the students might have offered. These comments can often elaborate on the reasons behind individual students' responses. Read them carefully, and use them as potential prompts for your own reflections.

Finally, you might discuss the questionnaire results with a few students. Twenty minutes spent asking them to offer possible reasons for the response patterns, and suggestions as to what would have improved the classes or the subject for them, can be time very fruitfully spent. If you would prefer, the Higher Education Development Unit (HEDU) staff would be happy to assist or `mediate´ in such a discussion session.