The concept of the transition from inputs to outputs being a 'black box' is one that exists even outside the realm of teaching. Implementation of policies has been widely hypothesized and theorized within the field of Public Administration, but it has mostly been evaluated through a process of summative, or end-product, assessment that only evaluates products rather than the process.
While there has been increasing literature in various fields on the importance of formative evaluation, which can help to catch problems as they develop rather than only learning about them after the fact, it seems that there has been increasing focus in the actual field on summative assessments. We have become hyper-focused on end products, whether they are grades, standardized test scores, or other arbitrary numbers that are deemed telling of the successes of implementation. They do nothing to tell of the quality of the inputs or what factors during the implementation process caused the success or failure of implementation, leading to outputs that meet or fail to meet expectations.
Perhaps one of the reasons why we have become increasingly focused on output measures rather than process measures is because of the costs involved in measuring the process at multiple points rather than measuring outputs at a single point. There may also be a lack of desire to have one's process evaluated because a lack of willingness to have their process critiqued. It's certainly something that's worth some degree of consideration in the discussion of evaluating summative and formative assessments.
Subscribe to:
Post Comments (Atom)
"Perhaps one of the reasons why we have become increasingly focused on output measures rather than process measures is because of the costs involved in measuring the process at multiple points rather than measuring outputs at a single point."
ReplyDeleteNot sure I agree, as if one waits until the end (output), measures, and finds an error, it is very expensive to fix at that point. Having multiple assessment points allows you to find and correct errors early before they become expensive (as in "failed the course").
Oh yes, I wholeheartedly agree that is the case (sorry, I don't think I was completely awake when I wrote this). It seems to me, from having dealt with organizations on this matter, that evaluation in general is often considered superfluous and unnecessary (this is more in terms of program evaluation than individual evaluation in terms of grade), so organizations will implement cost-saving measures on evaluation even if it means increasing costs of the project as a whole (short-term vs. long-run thinking).
ReplyDeleteLet me offer two points for consideration. First, I think the cost savings comes in when people do not evaluate at all, evaluate poorly in order to get superficial data, or hide the results until they get what they want (e.g. Toyota, Phillip Morris, the War on Terrorism). So it isn't expensive until and unless it is public! Second, I may have inadvertently artificially treated formative and summative assessment as mutually distinct. For instance. When I give a multiple choice test, I have the folks at the library run an item analysis on each question so I can determine if the test (or test questions) were fair. If not, I make changes. Secondly, I encourage students to come to my office and review the test. The grade still stands unless they can make a compelling argument why they answered a certain question one way or another. Also, upon reviewing the test and study habits with me, we can often come up with a better method for preparing for an upcoming test. Faculty evaluations can operate the same way (ideally). At the end of the year a faculty member sits down with their chair to review the progress over the year. Suggestion are offered and modifications are implemented.
ReplyDeleteOn the other hand, a body of formative assessment data can be compiled into a "learning portfolio" that can collectively serve as a portion of your final grade. Individually, they were not graded, collectively they can serve as a learning portfolio.