How to Give Objective Grades on Subjective Material

This article originally appeared in the winter 2004 edition of Communication: Journalism Education Today.

AN OVERVIEW

By Candace Perkins Bowen
with a little help from the other ASNE High School Journalism Institute instructors at Kent State —
H.L. Hall, Susan Hathaway Tantillo and John Bowen

Megan is a photographer who spends every weekend covering all the school’s sporting events, but she often appears to be doing “very little” during class.

Shawn can take the results of a survey and turn them into an infographic that tells a complicated story at a glance, but his editor has to double-check his spelling on the most simple words.

And Kara grasps the potential impact of a discussion at the school board meeting and can search out sources to get facts she needs, but deadlines? She doesn’t know the meaning of the word.

When it comes to grading these and all the other staffers in a production class, what’s a teacher to do? Ultimately, each needs the A, B, C, D or F in the proper blank on the report card, but how can anyone compare — much less evaluate — such diverse learning styles and contributions to the publication? If grading a “traditional” class is hard, grading those in a production class often seems impossible.

That’s probably why new advisers look for the perfect instrument — a grading form they can adopt to solve all their problems. Those attending the American Society of Newspaper Editors High School Journalism Institute at Kent State the last four summers have been no different. Many wanted us to simply hand them a form they could use …. and move on to the next topic.
But it doesn’t work that way.

Some teachers do fine with a system that tabulates student contributions to class and to the publication, giving point values to everything from making deadlines to mailing tear sheets to advertisers. With a handy calculator — or a math brain far better than mine — they add up the points and assign a grade. Other teachers do equally well with a much more subjective approach that includes instruments such as student self-evaluations, input from editors, string books and reflective journals.

In other words, there is no “right” way to grade any class, much less a production one. However, the ASNE instructors at Kent State came up with a check list of characteristics of a good grading system that might be useful to teachers of all temperaments and organizational abilities. Consider these guidelines:

• Find a method that works for you. If you know you won’t keep up with point totals for every little activity, don’t develop something requiring that. If you get uneasy when you can’t quantify your results, don’t settle for something that requires too much of your opinion. Probably the best system has some of each, but do tailor it to your style.
• Weight the grading so what’s important to you means a great deal in the final results. That means if you believe making deadlines is paramount for journalists, make sure those who miss them suffer appropriately. If you want students to learn teamwork, find some way to emphasize that in the grading system. This seems like a given, but it requires articulating the mission or goals of this class before you come up with the way to grade students.
• Explain your system up front. Be sure students — and even their parents — understand how it works and what they need to achieve to earn a certain grade — or to avoid a bad grade. Supplying rubrics for individual activities — news stories, features, other types of writing, page design, photos, etc. — can be useful.
• Keep individual differences in mind. It’s hard to put the square pegs in round holes — much less make writers, photographers, designers and ad salespersons accomplish exactly the same goals so….
• Provide opportunities for students to set their own goals — and evaluate how well they have done meeting them.
• Offer frequent feedback. No one wants to wait six weeks to find out how to write better leads … and meanwhile has had to write three more, making the same mistakes every time.
• Allow ways for students to improve their grades if they are not pleased with how they are doing. The opportunity to improve relates to feedback, too, because students can’t be surprised with their grades during the last week of class and expect to make up for lost time.
• Realize that even the best systems from the most experienced advisers need tweaking and downright revamping from time to time. Consider that part of the process as you seek the seemingly elusive perfect way to grade production classes.

TAKING THE MYSTERY OUT OF GRADING
By Mary Lou Nagy

Teachers in the English department at my school often use the phrase, “Why reinvent the wheel?” when speaking of materials used in classes. They are willing to share lesson plans, handouts or videos to help save time when we are all teaching many of the same units. Precious time can go toward teaching rather than toward writing a new test on To Kill a Mockingbird.

The same should be true in my journalism class, but it is not. Every year I try to reinvent the wheel for grading advanced journalism, my newspaper production class.

I have solved the mystery fairly well in my Introduction to Journalism classes by using AP style quizzes, story grades, presentations, projects and journal writing. These are concrete scores, mostly using the point system, that show how I arrived at their final grades.

Newspaper class is a whole different challenge. Let’s look at the possible choices for grading.

Strategy 1: writing, meeting deadlines, editing, photography, layout, advertising

Strategy 2: communication, teamwork, initiative, cooperation, working independently, quality of writing, going beyond the call of duty, responsibility and organization.

Strategy 3: attendance, listening, taking notes, keeping handouts and using them, interview skills, phone etiquette.

All of those, in some percentage or another, should go into the grading of a newspaper staff. How do we sort it all out? What do we concentrate on? How do we address those students who are upset because Ted (who goofs off frequently in class but is organized, looks for additional work to complete and always stays after school) gets a better grade than Marcia (who gets her work done in class but loses other people’s stories, doesn’t return phone calls and constantly complains about everything and everyone)?

Two years ago, after fighting this battle for 13 years, I finally turned the corner and came up with a system that works for me. My criteria boiled down to this: It had to be manageable for me and have a quick turn around time for the students. There had to be definite lines in the sand, that when a student crossed them, the grade would suffer. The mystery had to be removed. For years I was so unsure of the best way to grade that I would only give occasional hints if a student asked what they were getting. At the last minute of the marking period, I would have to decide what to do.
I looked at programs of award-winning advisers who utilize point systems so complicated that I would need an accountant to figure it all out. Many advisers let the students grade themselves or one another. Some use rubrics. Others use checklists.

I decided that I needed one or two concrete items for which I can show a score — when and if parents or administrators want justification for a student’s low grade.

When setting up any system, first start with deadlines. Be clear what is due and when. An easy deadline to check first is questions for their interviews. Ask them to have 15 questions for five or 10 points.

After that, give points for turning in the first draft. They either have it or they don’t. Yes, there is always the computer malfunction or the other 99 bazillion extenuating circumstances. Make it clear that they have to turn in some evidence of work at those times no matter what. It is an all or nothing on the first deadline. You can adjust the number of points if the student has a legitimate excuse or you know the coach is terrible about sending over the scores. Flexibility is a given in this job.

I can’t emphasize enough how critical it is to keep on top of this responsibility. Holding them to the fire on deadlines with an all or nothing grade (five for on time, zero for five minutes late) will gain the staff’s respect and grades will be in the book. Deadlines are real world — and parents who work know how critical it is to produce results on time at their jobs.
Giving a weekly quiz on copyediting or grammar is a great way to have a concrete grade in the book. It also has application to their work. Some advisers use current event quizzes. I just use Associated Press Stylebook quizzes. Once students see quiz scores affect their grades, they will start taking quizzes seriously.

I also use a three-ring binder divided into five sections. The first section is for a copy of all their work. From a caption to a story to a page layout, it is all included. They may print a copy of the story and hole punch it or cut it out from the edition and mount it. In addition, there should be a form where all the work that was assigned is available so it is clear whether they completed what they signed up to do.

Next is a calendar. It must have large enough spaces for them to write in detail what they did during the production period. I used to let my editors get away with writing “Lay out” or “Reporting” when what I really wanted was “Met with Mr. Green for 20 minutes on the cross country story; typed up notes, talked with Sharon about layout, checked messages.”

The next section is advertising, which includes contacts they made and/or contracts that were sold. I have forms for both that they include. My staff has a requirement to sell $250 of ads each semester. They sign a sheet that says they will do this so I grade according to effort. Some people try hard but simply can’t sell. Fortunately, we receive calls from advertisers, and I give those leads to the students who struggle.

The next section is the reflective letter and critique of the edition. This is their opportunity to tell me what happened during production that I may not have noticed, such as they ran an errand for someone or did extra work outside of class. They may talk about their frustration of not finding any advertisers or talk about a problem with a staff member that I can address at another time.
The last section is for handouts, forms, or articles that I give them from time to time. If the information on a handout is crucial, this can form the basis for a quiz.

I assign a point value to the binder as well as to completing it on time. I read the reflective letter carefully and use a checklist for the other info. It is helpful to return the binders to students within a few days.

Don’t worry if each year the grading system needs tweaking. Using deadlines as the foundation, the rest of the system will fall into place. When there is no more guessing about grades, both the adviser and the students will relax and have more time to work on the publication.

SELF ASSESSMENT
Student develop professional finesse
as they critique their own performance
By Janis Hatfield

Usually students, especially younger ones, do not want to evaluate their performance as journalists – at least not at first. After all, students who critique their own work cannot escape responsibility for the grade they receive. And often students arrive armed with the handy notion that grades reflect little more than the whim of a teacher.

But sooner or later, if not in the first grading period, then certainly by the second and succeeding quarters, students who “self-assess” begin to see the direct relationship between their work and their grades. Soon almost all staff members recognize that they are being treated as responsible young adults and begin to fulfill that expectation.

Consequently, they are more likely to do good work and to meet deadlines. The students know that they earned the grades they received. The grade is not a mystery. It is not a surprise. The grade reflects their degree of commitment, the quality of their work and how much they have learned.

The system magnifies in the students all those best qualities that any journalism experience encourages. Its greatest merit is that it requires that students be accountable for their own performance.

This is the way students assess their work in my journalism class. Near the end of the grading period, they must submit a packet that contains four essential documentations:

1. PERSONAL LOG
Because this class involves both direct instruction and production lab, class activities vary widely. On production days every student may be performing a different task — for example, setting up or conducting an interview, keying in an article, conducting Internet research, scanning sports photos, conferencing, working on layout, drawing a cartoon or using Illustrator. In the manual, the students have a blank calendar. Each day they record, on this calendar, a brief description of their journalism activity, in or out of class.

2. COPIES OF WORK
The packet includes photographs, text, tables, sidebars, etc. If the work has not been published, the student submits revised printouts. All drafts, transcripts of interviews, research and other documentation are also included

3. SUMMARY OF LEARNING
Students are given guidelines for the narrative about learning experiences during the grading period. At a minimum, the narrative should address these questions:

• What was your most important new learning experience?
• Of what achievement were you the most proud?
• What improvements have you noticed in your habits, methods or work in general?
• What did you struggle most with?
• What do you hope to learn next grading period?

4. COPY OF SELF-ASSESSMENT
The Self-Assessment is composed of two equal parts. In the first, students assess the work they have completed. This is their production score.

a. Production
Students at my high school may use journalism once to meet their English requirement. Moreover, parents and administrators expect all journalism students to write. Therefore, all students, regardless of their skill level, must write.

Less ambitious and less confident students, or those who have not yet had much training may limit their ambitions and choose smaller assignments. Weaker writers often elect to write short articles or briefs and assume other responsibilities. However, a quick calculation shows that a student who writes only two small articles may, in fact, earn as much as 90% for production.
The penalty for not meeting deadline is appropriately stiff. The system is intended to encourage students to (1) choose manageable assignments, (2) do their best work the first time and (3) meet deadlines. Conscientious students are rewarded fairly, and those who don’t take their staff responsibilities seriously are also treated fairly.

For documentation, the editors and I also post a performance chart in the publication room. On it, we record the dates when drafts and revisions are completed.

The second part of the self-assessment is a holistic self-evaluation in which students must take a good look at both their contribution to the newspaper staff and to their learning.

b. Self-evaluation
After students have gathered all the evidence of their work, examined their logs, written their narratives and assessed their production, completing the self-evaluation is simple. It also provides an almost foolproof reflection of the students’ work. Not surprisingly, I almost always agree precisely with the students’ self-assessment. If students err, it is usually because they assess their work too harshly.

Although I reserve the right to override a student’s self-evaluation, I have never lowered the grade. Occasionally, I have given a more generous assessment.
And that’s not to say that quizzes, exercises, graphic notebooks and other required assignments don’t count. They do. Occasionally, if it is a large assignment, I may announce how a grade can be raised or lowered for “Growth/Learning” on the self-evaluation.

We also record quiz grades and other miscellaneous assignment scores on the performance chart. Students allude to these grades in their narrative and consider them in calculating the score for “Growth/Learning” on the self-evaluation.

Because I do not subjectively evaluate the quality of the student writing, cutthroat competition is not a problem. Also, staff members tend to be more cooperative with one another.

Since I have used this system, the sweetest reward has been that students do better work when they are largely responsible for assessing the quality of their work. The positive benefits are enormous. Everyone wins. Most important, the students who work harder in all areas of producing a newspaper simply learn more. They also, some for the first time, learn that grades reflect and enhance learning.

The adviser has less stress because it is simple and easy to assign grades that are authentic. In fact, because of the nature of the beast and the constant give and take, teachers usually have a good idea of what grade our journalism students deserve. This system merely provides documentation to support that grade (without using a cumbersome point system) and assures that the students, too, understand the validity of the grade. Most important, they know what they can do to improve it.

Parents, whether they are pleased or displeased with the grade, have to admit that it is valid because they can access the documentation prepared by the students themselves. And finally, the higher quality of the work each student performs enhances the publication.

This system has evolved during the decade that I have been teaching journalism. It is as authentic and fair as a grading system can be. It accurately reflects the student achievement and learning and gives every student, regardless of his or her skills or talent, the same opportunity to excel. Most important, it genuinely promotes learning — something that cannot be said for all grading methods

RUBRICS

Provide structure to grading in workshop-based class
By Nancy Freeman • http://ssp-stl.org

Rubrics. The first time the head of my department told me I needed to write rubrics for every one of my journalism story styles, I groaned. What a waste of time and energy, I thought. Simply another case of bureaucratic nonsense that impinges on the real business of teaching and advising a student newspaper.

I have always considered myself lucky to teach in a workshop-based journalism class, a place where I had the freedom to do small group and one-on-one work with my 60+ staff members. As a result, I had never felt a need to codify a formal set of rubrics for each different type of story.

We had a style manual that included a host of journalism style rules reporters were expected to follow, and we talked in class about what went into different types of stories. Students received individual feedback in conferences and on rough drafts.

Plus, I already taught them about narrative leads, the differences between an editorial and a personal column, the elements of writing a good review, the hazards of passive voice and the purpose of a solid nut graph — and oh, so much more — so why did I need to write rubrics?

During the next several months, as I crafted and recrafted a set of rubrics for nine different types of stories, I discovered that writing and using rubrics has enhanced my teaching by making me clarify my expectations in specific ways.

First, I had to determine the categories I was going to use for each story type. Each rubric has nine categories. I originally thought the categories would be the same for all story types, but they actually have some variation. After all, categories for reviews are not necessarily the same categories I want to use to evaluate feature stories or hard news.

Generating these categories became interesting fodder for journalism room discussions about what was essential for different stories and helped editors and staff to come to heightened awareness of the various nuances that each type of story had.

While the categories varied to some degree for different types of stories, there were also certain similarities. All stories needed an effective and appropriate lead. The description of what that lead might entail might vary from one type of story to the next, but it was necessary in some form for all.

On the other hand, some categories apply only to certain types of stories. Specialized stories, such as news, features and sports, needed a category entitled Adequate Use of Primary Sources because these are stories that are strongly based on reporting and interviewing skills, but it would make no sense to include that category in a rubric for movie reviews or personal columns, which are based upon one’s own opinions and experiences.
I found that all stories required categories that covered effective organization and flow (clarity of story structure), clear focus (unity of content), writing style (covering voice and diction), journalism style, grammar and mechanics and formatting of final product. Every teacher will have to determine what categories to use and how to weight them as he or she creates rubrics.

Next, I had to determine what would qualify as exemplary work in each of those categories. Where was I really setting the bar? How high did I really want the students to jump and was I communicating that to them clearly?

This was harder than I thought, but in some ways easier as well.

I decided to set the exemplary bar high. It is difficult for new students to get that mark in every category. I wanted to reserve the highest mark for the best story by my most sophisticated writers. But a student can miss the exemplary mark in a few categories and still get an A on a story, a factor that makes it a little easier to keep the expectation high for each category. Also, students have other opportunities to earn points.

On my rubrics, the highest a student can get in a category is an 11, which seldom happens and the lowest he or she can get is a 5, which also seldom happens. I decided when I created the rubric to avoid awarding any points lower than 5 because I would never give a student anything lower than a 50 percent on a story that was original work and was turned in on time. My 99-point rubric is a reminder to both my students and to myself that no story is perfect and no paper is perfect. I always want them to think that they should try a little harder to improve.

Deciding what qualifies as an exemplary and poor in each category was fairly easy, and explaining the high and the low end of these rubrics is always the clearest. As in most cases in life, the middle ground is always a bit trickier to define. Each semester now, I tweak the rubrics a bit as I refine my expectations clarify what qualifies as excellence.

Once created, having these rubrics has simplified many areas of my life as a teacher. I use them to teach a concept like the various types of review writing along with model stories. We look at a model for a story. Then we “grade” it according to the rubric. We discuss the techniques the writer uses and whether the story fits into the parameters set out. Having a rubric in their hands gives new writers specific standards to comprehend and a guide to follow. I also tell them that every story will not fit into a cookie cutter mold, but rubrics still allow great flexibility within certain standards. Now even experienced writers will come in and ask for a rubric for a story that they have not done for a while.

Also, editors use rubrics as they coach writers in story conferences. They circle areas on the rubric that correspond to weaknesses in the story and areas where the story needs to be fleshed out as well as areas where the story is already strong. This is faster than writing everything out by hand, but it still gives writers specific feedback. The process helps to produce stronger, well-developed final drafts.

The specificity of the form also is a great aid in grading. I use it to give quick and easy feedback to students on their final grades on a story. It has definitely reduced the number of hours I spend grading — a result that makes the rubric time worthwhile. And best of all, constructive evaluations help students improve their journalism skills.

SAMPLE FORMS

These forms are not meant to be any kind of definitive guide to producing grading materials for the production class. They are, however, sample materials provided by instructors all over the nation. Many have other materials they will be willing to share. Contact them directly using the e-mail addresses provided. The copyright for these materials remains with the contributing author.

The Spread: An 11-step Checklist • From Rachel Engelhardt

35mm Photo/Caption Rubric • From Betsy K. Ahlersmeyer

Digital Photo/Caption Rubric • From Betsy K. Ahlersmeyer

Layout Rubric • From Betsy K. Ahlersmeyer

Yearbook Writing Rubric • From Betsy K. Ahlersmeyer

Editor in chief • From April Ryan

Copy Editor • From April Ryan

Advertising/Business Manager • From April Ryan

Citizenship • From April Ryan

Advanced Journalism • From Logan Aimone

Stories • From Sandy Gaul

Self-assessment • From Janis Hatfield

Evaluating photographs • By Jack Zibluk