ASSESSMENT OF TEACHING AT OSU: We Need You!
Good Input = Good Output
by
Faculty Senate Advancement of Teaching Committee
Academic Year, 2000/2001

 

1) AOT Committee's premise: Assessment of teaching at OSU serves two primary functions:

  • to offer faculty feedback for the purpose of professional development;

  • to provide administrators on-going assessment of faculty teaching performance.

2) AOT is proposing a significant change in the tools used to assess your teaching. We ask you to do two things:

  • familiarize yourself with the brief proposal below.


Introduction:
The purpose of this faculty forum paper is to solicit faculty opinion about changing the current form used to assess teaching. The current form (widely used across most departments) provides information about student ratings of instruction. Both faculty and administrators have voiced concerns about the current form, questioning its focus (several questions contain more than one assessment factor); its validity (do questions measure what they were intended to?) and the appropriateness of the ways the information on it is used (statistical measurement requires responses across a minimum of seven, similar classes when decisions about faculty personnel matters are being made). In addition, concerns about interpretation and use of data have been expressed by faculty and administrators.

The Faculty Senate Advancement of Teaching Committee is suggesting contracting with the Office of Educational Assessment (OEA) at the University of Washington. The OEA has developed a series of forms that, while similar in format to the form OSU currently uses, have been designed to separate the developmental role and administrative role of assessment, to accommodate different types of class formats (lecture, laboratory, discussion, seminar, etc), and to certify their validity and statistical reliability.

Background:
Faculty sometimes express skepticism about teaching assessment at OSU, its validity and purpose, and its multiple uses for instructional development, P&T considerations, and/or to provide accountability to the public.

During Fall term 2000, AOT Committee members invited colleagues from the Honors College, Forestry, and Extension to describe their current assessment procedures and to suggest improvements in student evaluation of teaching at OSU. A major concern focused on whether the current OSU form has been tested for validity and statistically analyzed for reliability. These discussions led the AOT to research state-of-the-art teaching assessment practices recommended by Centers dedicated to the science of teaching assessment (Office of Educational Assessment at the University of Washington; the Idea Center at Kansas State University). In addition, selected members of the AOT committee read faculty generated white papers on evaluation of teaching from Northwestern University, Indiana University, Pennsylvania State University, and Virginia Tech. The AOT committee believes that the series of eleven scannable forms developed at the University of Washington would provide OSU a more flexible format and more accurate data in the assessment of teaching than we now have.

Proposed Evaluation Forms available from the Office of Educational Assessment at the University of Washington:
The Instructional Assessment System (IAS) is a program to collect and summarize student ratings of instruction at the postsecondary level. It is used to assess about 11,000 courses annually at the UW as well as being used at about 30 other universities. Eleven machine-readable course evaluation forms (reflecting various course formats and including open ended comment sections on the back) are used to assess teaching. As with OSU's current form, student responses are anonymous and "comment sheets" are forwarded to faculty, along with the scanned data, after final examinations. The forms also contain the sorts of demographic data available on OSU's current form. In addition, IAS is developing a distance learning assessment form.

These IAS forms differ from OSU's current form in that they are tailored to class format or types. Forms A through E address five course types: lecture-discussion (Form A), large lecture (Form B), seminar discussion (Form C), problem solving (Form D), and skill acquisition (Form E). Forms F, G and H were added for quiz sections taught by graduate teaching assistants (Form F), lecture courses emphasizing assignments and a textbook (Form G), and laboratory sections (Form H). Form I was developed to assess distance learning formats. Form J is for clinical rotations and studio work. Form X emphasizes student outcomes and teaching behaviors rather than methods of instruction.

In addition, forms separate responses into those used for developmental purposes and those used for administrative purposes (the first four questions on each form assess global characteristics of teaching and are considered as valid measures of teaching performance across disciplines and colleges for P&T or accountability purposes). The rating system uses a six-point Likert scale ranging from excellent to very poor.

Questions on each assessment form are divided into 3 categories:

1) General Assessment Items. The first four items on each form provide an overall rating of the course. Students are asked to rate the course as a whole, the course content, the instructor's contribution to the course, and his or her effectiveness in teaching.
2) Form-Specific Items. The focus of each course evaluation form is defined by items 5 through 22 relating to instructional processes (Forms A through I) and/or outcomes (Form X). A reduced set of items (5-14) is found on Form J.
3) Academic Demand Items. All forms, with the exception of Form J, contain five items (23-27) that ask students to indicate their expected grade in the course relative to other courses, the intellectual challenge of the course, the amount of effort they put into the course, the amount of effort necessary to succeed, and their involvement in the course. Four additional items (28-31) ask students to record the number of hours per week spent on the course, the number of valuable hours, their expected grade, and how the course fits into their academic program.

 

The back of the evaluation forms provide space for instructors to add up to 35 course specific questions. The responses will be in scannable format.

A separate non-scannable "comment sheet" asks four open-ended questions. Students are asked whether they found the course to be intellectually challenging, which aspects contributed most to learning, which detracted from learning, and what suggestions for improvement they might have. These sheets are simply returned to the instructor along with the summary reports for the scanned data.

Sample Forms: [Please use your menu bar above to return to this page; clicking "back" on the forms below will take you into the University of Washington site]

Form A is designed for small lecture/discussion courses. Items emphasize the clarity and quality of information transmitted, as well as the nature of the interaction between instructor and student.

Form B is designed for large lecture classes, with little or no in-class interaction between instructor and student. Items strongly emphasize the quality of course organization and information transmitted.

Form C is designed for seminar discussion classes which include a minimal amount of formal lecturing by the instructor. The items emphasize quality of discussion as well as course organization and interest level.

Form D is designed for those classes whose purpose is the teaching of problem-solving or heuristic methods. Clear explanations, dealing with student difficulties and quality of problems are emphasized.

Form E is designed for those classes which are skill oriented and in which students get "hands on" experiences related to future occupational demands. Such classes include clinical nursing, art studio, social-work field experience, etc.

Form F is designed for quiz sections. These are usually taught by graduate teaching assistants, in conjunction with a lecture section taught by a regular faculty member . Items focus on the ability of the quiz section instructor to interact with students and provide clear and useful explanations.

Form G is designed for use in large lecture classes (such as those in math) which rely heavily on homework problems and a textbook. Emphasis is on the instructor's ability to communicate with students, and the value of assigned problems and readings.

Form H is designed for lab sections generally taught in conjunction with classes in the physical sciences. Items emphasize the instructor's ability to introduce meaningful questions, assist students, and deal with unexpected problems.

Form I is designed to be used in distance learning (correspondence) courses. Items relate to the instructor's responsiveness and the quality of support material.

Form J is designed to evaluate instruction provided through clinical experience rather than traditional academic coursework. Such courses are often found in the health professions or the arts. Items focus on the instructor's ability to provide information, stimulate learning, and demonstrate skills.

Form X is designed to be used across all course types. It includes a reduced set of items relating to general educational processes and a unique set intended to assess educational outcomes.

The back of all IAS Forms (Forms A - J, and X) are identical and permit individual instructors to query students on any subject they think is appropriate to the course.

Comment Sheets solicit responses to four open-ended questions. Students are asked whether they found the course to be intellectually challenging, which aspects contributed most to learning, which detracted from learning, and what suggestions for improvement they might have. Students can answer additional, instructor-generated questions on the back of the IAS form.

 

NOTE: Efforts are currently underway to also include an assessment form that more closely suits the needs of Extension Service faculty.
Proposed action on diversity question:

AOT Committee members searched numerous university and Centers for Teaching Excellence and found no questions relating to diversity at present. Based on standard assessment and testing criteria associated with testing for validity and reliability, Committee members strongly encourage use of an open-ended question added to assessment questions for student response. An example might be "The instructor encouraged discussion and respect for diverse views, values, and beliefs with regard to gender, race, religion, sexual orientation, disability, and other forms of difference."