Fall 2012 Political Science Assessment Report


Scope of Assessment

Two courses covered in this general education assessment report are Critical Thinking: Politics and Law (POL 1000); and American National Government (POL 1010). The first has one section, while the second comprises eight sections, seven of which are covered in this report. The one section missing is section 01, taught by Professor Phillip Beverly, who, in spite of timely notices to all instructors reminding them to conduct the tests, submitted no pre-test or post-test for his course.


General Education Assessment Outcomes

The assessment outcomes for Critical Thinking: Politics and Law (POL 1000) are as follows:

    1. To be able to find information, evaluate it critically in terms of reliability, and use it appropriately in thinking and writing.
    2. To understand and be able to apply some of the basic methods, questions, and vocabularies of the humanities and the social sciences.
    3. To understand the role of creativity in problem-solving, addressing issues and concerns, and generating new knowledge.
    4. To apply analytical skills to the humanities and social sciences.
    5. To have an exposure to and appreciation of philosophical, religious, ethical, political, and scientific ideas of diverse cultures.

The assessment outcomes for American National Government (POL 1010) are as follows:

    1. To apply the basic methods, questions, and vocabularies of the humanities, mathematics, the natural sciences, and the social sciences.
    2. To apply analytical skills, including mathematical reasoning, to the natural sciences, social sciences, and the humanities.
    3. To demonstrate knowledge of philosophical, religious, ethical, political, and scientific ideas of diverse cultures.

 

Method of Assessment

To evaluate the realization of the above outcomes, the political science teaching faculty developed for each course an assessment instrument, made up of questions that tap into the criteria enumerated above, administered at the beginning of the semester as pre-test and the same material at the of the semester as post-test. The assessment for the Critical Thinking course (POL 1000) consists of a series of six short, comprehensive-reading-type passages, each of which is then followed by a range of multiple-choice questions designed to test student understanding of that read material. Altogether there are 20 such multiple-choice questions. The assessment for the American National Government course (POL 1010) consists of plain multiple-choice questions, unlike the critical thinking course, not preceded by any reading exercises, that test knowledge in American national government and Illinois politics. Altogether there are 25 questions, 19 of which are on American national government and 6 of which are questions on Illinois politics.


Presentation of Data Results and Interpretations


POL 1000-01, Critical Thinking: Politics and Law

Pre- Test   Post-Test
N Raw Score Median Range   N Raw Score Median Range Mdn. Gains
29 291 10 14-4   23 222 10 16-3 0%

 

POL 1010, American National Government

Pre-Test   Post-Test
Section N Raw Score Median Range   N Raw Score Median Range Mdn. Gains
03 8 88 11 15-6   4 40 12 16-10 1
04 13 113 9 12-16   15 183 13.5 19-8 4.5
05 21 205 10.5 15-6   12 136 12 15-8 1.5
06 12 108 9 13-4   11 110 9.5 12-8 1.5
07 25 213 9 15-5   23 250 10.5 16-4 1.5
08 10 89 9.5 12-8   8 82 10 13-7 .5
61 11 114 10.5 13-6   13 158 12 18-8 1.5
All Secs 100 930 9.5 15-5   86 959 12 19-4 2.5

 

The results summarized above generally demonstrate improved student learning. The term generally is used advisedly because of the figures from the critical thinking class which registered a zero median gain, meaning that, between the pre-test and post-test, learning remained the same unchanged. The result departs unfortunately from information in the three-year cycle trend for this course which, in spring 2012, posted a 25% post-test increase over the pre-test number. It is also somewhat puzzling given that, unlike American national government, discussed next below, there is only one section of this course, only one instructor, one set of textbooks, and last, but by no means least, the instrument is based on reading passages of the kind to be expected in a critical thinking course.

With respect to the American national government, all the sections recorded improved learning, measured in median gains, with section 4 posting the highest gain, 4.5, and section 8 least at .5. Collectively, taken as a whole, the sections posted a healthy median gain of 2.5. These are positive developments, compared, for example, to the picture from last semester, which was mixed in the sense that some sections posted increases over the pre-test numbers while others showed decreases. Note, however, that this semester, unlike spring 2012, compliance in the administration of the pre-tests and post-tests was better. For example, in the last semester, three sections were incomplete in that they have only pre-tests without complementary post-tests; and there were scrantons which could not be used because they were done in ink rather than in HB pencils necessary for machine grading. None of these problems occurred this semester, thanks to better compliance, and, where, save for Dr. Beverly, every other faculty member also participated in the assessments.

Decision-Making Using Findings

While the assessment instruments of the two courses under review generally provide reliable indication of what our students are learning and not learning, to achieve enhanced student learning, there is need for the political science faculty to go over the results in this report in an attempt to stimulate discussion around the findings presented above, including questions consistently missed on post-tests.


Demonstrated Improved Learning

See Part IV above, especially the discussion related to interpretation in the two paragraphs following the tables.


Publicizing Student Learning

This report and its entire contents will be disseminated through various avenues that include publication in the website of the philosophy and political science program.


Accomplishments and Challenges

The assessment accomplishments of the last season are manifold and include less problems with the administration of the instruments (necessary for complete and accurate report), more progress toward our goal of 100 percent faculty participation in the assessments, and a report that, however imperfect, provide generally reliable indication of what our students are learning or not learning. Unfinished business or lingering challenges include the need to achieve more consistency across all sections of the American national government, and, for all two courses, the need to increase the level of attainment in terms of the assessment scores.


Minutes of Department/Program Meeting

Minutes of program meetings related to the assessment are included here. (See below)

Fall 2012


Conclusion

Overall, the assessments for the two courses here under review continue to indicate that there is general improvement from the beginning of the semester, less so though critical thinking which showed no change in learning, based on comparison of the median scores in the pre-test and post-test.

The rest of the comment here is devoted to American national government. Political science faculty members do not "teach to the instrument." However, there is need for discussion around the particular questions that are consistently missed or show inadequate improvement. Because students no longer need to take the course to satisfy the requirement for the Illinois Constitution Exam, some faculty members no longer integrate materials on the Illinois Constitution in teaching the course. However, to improve performance on Illinois questions that students miss often (e.g., voting and election, spending and the budget, and procedures for amending the Illinois constitution), faculty members may elect to interject an Illinois context into some of their lectures on American national government.


PDF icon View the Summary of Results and Trend Data for General Education Assessment Findings