Physics Program Assessment (Summary and Examples)

We assess the following in the Physics Program (1) student scientific reasoning skills and problem solving skills, (2) student conceptual understanding of various physics topics from the introductory level to the advanced level, and (3) student views about physics, the learning of physics and how students view themselves as learners.
The following are the specific program objectives and student outcomes.

Program Objectives
1. Train students in the discipline of physics so that they are competitive locally, nationally, and internationally for positions in teaching, industry, government, and academia.
2. Help students develop strong conceptual and quantitative knowledge about physics.
3. Contribute to the growth of scientific and educational knowledge through faculty and student research and peer mentoring.

Student Outcomes
a. Analyze a description, question, problem or task and be able to identify the underlying physics principles, concepts and scientific reasoning skills involved.
b. Apply the underlying physics principles and scientific reasoning skills to a diverse set of qualitative and quantitative questions, problems and tasks.
c. Communicate ideas, concepts, solutions, and experimental results.
d. Implement different types of experimental techniques.
e. Demonstrate expert-like attitudes and expectations toward science.
f. Develop coherent and logical solutions to a variety of problems and tasks.

 

Diagnostic Instruments (Force Concept Inventory - Physics I) and Brief Electricity and Magnetism Assessment - Physics II

Diagnostic results from the Force Concept Inventory (FCI) and the Brief Electricity and Magnetism Assessment (BEMA) are given each semester in our Introductory Physics courses: Physics 1510, 2110, 1520, 2220.   The test is administered as a pretest and as a posttest. The pretest is typically given during the 1st week of class and the posttest is typically given during the last week of class.

Prior to implementing the reform-based materials, gains in the calculus-based physics class at CSU on the FCI were typically in the range of scores that Hake reports for students in traditional classes (Hake, 1997). In Fall 2004, when we began using the new materials, we observed an improvement from earlier results in the calc-based physics classes. Researchers must always be careful in interpreting results from multiple-choice diagnostics. We have observed that students are often able to form robust sets of knowledge as a result of the new instructional materials we are using at CSU but have difficulty activating these sets of knowledge when confronted with certain tasks.

Student construction of isolated sets of knowledge is discussed in DiSessa, (1988), Sabella, (1999), Redish, (2003) and Sabella and Redish (2007.)

The following published results are from the traditional and reformed classes at the University of Maryland and the City College of NY.

University of Maryland (trad. Instr. 5 courses)	49%	58%	.18
University of Maryland (tutorial instr. 5 courses)	50%	67%	.35
City College NY (trad. instr.)	40%	53%	.23
City College NY (using innovative curriculum)	39%	65%	.43

Results from CSU are typically better than the results from these schools in the traditional instructional environment but not as good as those of the tutorial or reformed classes. The following data show the results from the calculus introductory physics classes at CSU.In most calculus-based courses students at CSU have gains of roughly 30%. In some instances, class gains have been as high as 45% and as low as 19%.

Attitudes and Expectations

Maryland Physics Expectations Survey (MPEX) and Colorado Learning Attitudes about Science Survey (CLASS)
The MPEX and CLASS look at student attitudes and expectations toward physics and science. Students are asked to state whether they agree/disagree with the statements on a five point Lickert scale. The questions touch on topics in different clusters such as Reality, Math, Effort, Independence, Coherence, Sense Making, Connections, and Concepts. The responses provided by the students are compared to the responses given by experts (scientists, physics professors, etc.)

One comparison that we can make regarding student attitudes in the introductory physics classes involves a comparison between the traditional course and the modified instruction courses from the recent years. In one algebra-based modified course student attitudes improved (53% → 55%) whereas in the traditional course attitudes veered away from the experts (54% → 38%). In the calc-based courses, using modified instruction we see attitudes remaining roughly the same or decreasing slightly (54% → 50%) and (54% → 54%).  In most CSU classes attitudes and expectations seem to get closer to those of experts.

Students nearing the end of their physics majors at CSU see a much larger percentage agreeing with experts. As an example in one year students tended to agree with the experts on 76% of the statements, indicating that as students progress through the program they develop more expert like attitudes.

Online Assessment Tools

The Physics Program uses an online diagnostic delivery system which has greatly enhanced the type of assessment we can do in our courses.   The system called LASSO is part of a project from the Learning Assistant Alliance (CSU is a member institution.) LASSO allows us to look at performance based on gains and the effect size (Cohen's d) and generates sophisticated reports that we can use to improve instruction. For more information on the LAA and LASSO see: https://learningassistantalliance.org/public/lasso.php. 

Example: The following capture is from a recent physics II course report and shows significant growth from the pre to the post test as measured by the pre/post %'s and effect size.

 

 

Student Thinking and Reasoning

To improve student thinking and reasoning skills we incorporate diverse problem solving, conceptual and experimental tasks into our physics courses throughout the curriculum.   Below are some comments from students:

• Once we figured out the problem I learned a lot. I learned to take my time, to look at what is given, to expect the unexpected, not to give up, to draw a picture to go along with the problem, plan a strategy, and to make sure I understand the problem. I learned to develop these techniques with the other problems we did in class.
• In my science and society class we were studying population patterns in Indiana and we had to test the numbers we got to see what kind of information we could obtain. Because of my physics ...  class I took my time to analyze the data and looked at everything that was given without jumping to conclusions or assumptions. Those skills also helped me a lot with my test taking skills. I used all the strategies I learned to answer test questions. I read the questions, took note of what was given, drew a picture to represent the problem, did not rush and did the problem. In the questions that I did not know how to answer I simply did something.
• Physics ... has taught me a lot of skills and different ways of looking at problems. Since day one [it] ... has helped me expand the way I think ... one of the main strategies I learned was to be patient when solving a problem.

Instructional Reform in Physics

The Physics Program has been involved in a number of instructional reform efforts to improve student learning physics. CSU’s Physics courses are informed by the results of Physics Education Research. CSU has had over six NSF  grants since 2004 to revise the introductory algebra and calculus-based sequences. These revisions include the implementation of interactive PowerPoint lectures (CSU), research based laboratories (NMSU, Buffalo State, CSUF, CSU), clicker question sequences (CSU, OSU), TIPERS (Hieggelke, Maloney, Kanim, Okuma), and prelectures (UIUC). Each of these components are based on PER. More recently we have incorporated Learning Assistants into many of our STEM classes as a result of support from CSU, an NSF-IUSE grant and a Department of Education PBI grant. More information on the CSU LA Program can be found at: http://msscsu.wixsite.com/csu-la-program. 

Reforming and revising our physical space with state of  the art classrooms (https://www.aps.org/units/fed/newsletters/fall2012/sabella.cfm)  and incorporating research-based  instruction has helped us create an extremely active classroom where students are involved in sense making.

 

Curriculum Map

	1510,1520, 2110, 2220 (Intro Phys)	2330 (Physics III)	2700 (Sophomore Lab, Electronics)	3110, 3150, 3250, 3450 (Advanced Theory)	4550, 4850 (Advanced Labs)	4905 (Thesis)
Analyze a description, question, problem or task and be able to identify the underlying physics principles, concepts and scientific reasoning skills involved.	K, A	K, A	K, A	K, A	S	S
Apply the underlying physics principles and scientific reasoning skills to a diverse set of qualitative and quantitative questions, problems and tasks.	K, A	K, A	K, A	A	A	S
Communicate complex ideas, concepts, solutions, and experimental results.	K			A	S	S
Implement different types of experimental techniques.	K		A		A	S
Demonstrate expert-like attitudes and expectations toward science.	K					S
Develop coherent and logical solutions to a variety of problems and tasks.	K	K	K	A	A	S