The U.S. Commission on the Future of Higher Education (2006) has expressed concerns about the quality of higher education. As a response to accountability demands, such as those of the Commission, several institutional-level assessment efforts have been undertaken, such as administration of standardized tests and surveys of student engagement. Nonetheless, universities and colleges have often used student ratings as a means of evaluating courses and instruction—because they are very practical. According to meta-analyses, only a few items on typical course evaluations have been found to be related to student achievement. To address these concerns, Frick, Chadha, Watson, Wang and Green (2008) developed a new course evaluation instrument that consists of nine student rating scales of teaching and learning quality (TALQ). Five TALQ scales measure First Principles of Instruction, which were derived from a synthesis of instructional theories by Merrill (2002). Other TALQ scales measure student perceptions of successful engagement (academic learning time) and learning progress. The present study examines (1) within-class agreement among students on TALQ items that measure instructor-level variables and (2) the dependability of TALQ scale scores, since these psychometric properties have not been previously addressed.
The TALQ was administered near the end of the semester to 464 students in 12 classes taught by 8 professors at a large Midwestern university. Measures of within-class agreement on TALQ items were found to be satisfactory on 27 out of 30 TALQ items in 9 of the 12 classes. Results of two-facet and three-facet generalizability study designs revealed that TALQ scale scores were dependable for student ratings of overall course and instructor quality, satisfaction, learning progress and on 3 First Principles of Instruction (activation, application and integration). Two of the First Principles scale scores were found to be less dependable (authentic problems and instructor demonstration), as well as student academic learning time. Based on these findings, change in 3 items are suggested. Changes in 5 items on the academic learning time scale are also recommended. Future validation studies are recommended that investigate the use of the modified TALQ scales for improving the quality of teaching in postsecondary education.
|Advisor:||Frick, Theodore W.|
|Commitee:||Bichelmeyer, Barbara A., Delandshere, Ginette, Peng, Chao-Ying Joanne|
|Department:||School of Education|
|School Location:||United States -- Indiana|
|Source:||DAI-A 70/12, Dissertation Abstracts International|
|Subjects:||Educational evaluation, Instructional Design, Educational technology, Higher education|
|Keywords:||College student, Course evaluations, First principles of instruction, Generalizability theory, Improving taching in higher education, Instructional theory, Teaching quality|
Copyright in each Dissertation and Thesis is retained by the author. All Rights Reserved
The supplemental file or files you are about to download were provided to ProQuest by the author as part of a
dissertation or thesis. The supplemental files are provided "AS IS" without warranty. ProQuest is not responsible for the
content, format or impact on the supplemental file(s) on our system. in some cases, the file type may be unknown or
may be a .exe file. We recommend caution as you open such files.
Copyright of the original materials contained in the supplemental file is retained by the author and your access to the
supplemental files is subject to the ProQuest Terms and Conditions of use.
Depending on the size of the file(s) you are downloading, the system may take some time to download them. Please be