The Administration of Student Evaluations of Teaching (SETs) at the KSB
Originally designed for formative purposes only, Student Evaluations of Teaching (SETs) have long since taken on the additional purpose of providing summative information about teaching performance from the students’ perspective. In addition to instructors’ use of data for teaching improvement, student evaluations are also used in all sorts of administrative contexts, contributing to decisions about tenure and promotion, salary, and teaching awards. Indeed, so important are the data from SETs that Kelley School policy (April 1999) mandates their administration in all classes: “Without exception, every class section is independently evaluated.” Yet despite the ubiquity and importance of SETs at the Kelley School, the process for administering them remains a mystery to most faculty. This article will attempt to explain the process of gathering data, organizing the forms, and processing the data so that faculty may receive reports a few weeks after the conclusion of the semester.
The process begins already during the first week of the semester when requests for information are sent out to various administrative assistants at both the Indianapolis and Bloomington campuses. The data retrieved from the Registrar must be maintained and manipulated manually so that each instructor has a unique identifying class number (formerly section number) for the course or session taught. In cases such as the MBA core (X504), for example, fictive class numbers must be created lest there be one class number for several instructors. Because data arriving from the Registrar can often be incomplete or corrupt, further work is required such as seeking accurate information, entering it manually, and cross checking for accuracy. Once the database is worked into shape, a first batch of reports is sent to departmental administrative assistants for checking and then labels are printed for each course that needs to be evaluated before the eleventh week of the semester. Once the packets have been assembled by support staff in Facility Operations & Services, they are ready for student to fill out.
At mid-semester a second data pull from the Registrar is performed in order to ensure the most accurate information regarding enrollments and course cancellations. This more current information is used to run final reports and labels for the end-of-the-semester evaluations – those that comprise the great majority of student evaluations. Once students have completed the evaluations they are returned to BU 141 where each evaluation form is inspected by support staff in Facility Operations & Services to ensure the correct class number has been filled in by the students and to check for bubbles not filled in completely or for forms completed in ink. These must be gone over with a #2 pencil so that their data will be read by the scanning device in Franklin Hall.
When this process is complete the forms are collected and stored in BU 141until the Wednesday of finals week, a date set to allow evaluations to arrive from the Indianapolis campus, whose semester continues a day later than Bloomington. Once collected, the thousands of forms are sent to Franklin Hall for machine scanning, a process that typically takes an entire day’s work and is therefore subject to delays because of the high traffic in the scanning room at the end of the semester.
Once the scanning is complete, the data are returned to the Kelley School for processing where the raw data are worked into reports that may be easily read and interpreted by the end user: the instructor, deans, department chairs, and program chairs. The reports are printed, duplicated, and inserted, along with the reports of school-wide statistics, back into the packet containing the original evaluations so that instructors may view the reports alongside students’ qualitative comments before they are finally returned to the instructor.
Although thousands of student evaluations are processed each semester at the Kelley School, this rather complicated system allows the School to collect the students’ perspective on the sum total of its teaching, quite a feat considering the number of students and courses that must be evaluated!