June 6, 2024

Assessing and Grading in Competency-Based Learning and Leveraging GenAI for Support

My favorite part of our ninth grade student interviews was during their discussion of “Growth Orientation.”  This was part of the grading process my team and I designed. We broke this particular competency down into seven parts: purpose, resourcefulness, planning, time management, motivation, risk taking, and reflection. What students had to say during this part was almost always insightful and substantive.

We started with purpose, which we defined as a “long term goal, connected to personal interest, that positively impacted others.” Most of our ninth grade students fell into the “progressing” category for this one. They would say that they were actively exploring interests and considering long term goals related to those interests, and how those goals might positively impact others. All the while, they would be tasked with providing evidence of the steps they were taking to find that elusive purpose.

We’d then move to resourcefulness. Did they know how to find answers to the questions they had, or were they reliant on teachers to provide the answers for them? Did they generate their own questions? My repeated line with my students was that if they still needed me at the end of the year, then we had both failed.

The middle chunk was especially enlightening, particularly for students who struggled to get work done on time. Some students would say that this was a planning problem, others would say that they made plans, but they couldn’t stick to them, and others would say they had plans, and they could stick to them for a time, but then they would lose motivation midway through a semester. For others, they would tell me that they had done the work, but they weren’t happy with it, so rather than take a risk at getting negative feedback, they just wouldn’t turn it in.

These are all very different issues that require different strategies to address them, and yet how often are students who don’t turn in work reduced to being called “lazy.”

To argue for their level of mastery in each of these strands, students would often use evidence from Student Directed Learning Time, a block of time each week where students, under our supervision, took ownership of their learning.

As for reflection, by the end of their talk on these first six strands, teachers and students would clearly see how things were progressing.

This type of experience and the demands leading up to it give students a better understanding of themselves. It also allows teachers a broader range of opportunities to better understand their students and to offer them feedback to assist their growth.

Assessment of, as, and for Learning
Traditional alpha-numeric grading systems, which assess only course content and narrow skill sets, are painfully reductive in comparison. In a competency-based system, even if students perform well on a given assessment, demonstrating mastery in several of the Communication strands, they might have struggled with strands of Growth Orientation. Perhaps they didn’t take sufficient risk. Or in their Citizenship...perhaps they were not being particularly gracious giving or receiving feedback during writing workshop.

Focusing on competencies and student-led interviews, in particular, enhanced my effectiveness at personalized learning. I believe GenAI has the potential to further improve upon this practice, and I will highlight those areas below. But before that, I’d like to elaborate on how this grading process worked in the school we created it for, and how I adapted it for a more rigid system.

School A
I started working at this school immediately after my experience with the IB. I had enjoyed working with the IB, and I felt its best-fit grading process was a game changer for keeping students motivated until the end and getting them to take necessary risks throughout the year. The problem that I find with the IB, though, is that there is too much language to make it meaningful to students. Four criteria in six subject groups, 10 Learner Profile characteristics, and five ATL, each with a page or more of descriptors. That can lead to a lot of dusty language hanging on the walls.

Most of our 9th grade students in that school came from one of two environments: a gradeless, competency-based middle school or a high stakes testing school. Our team’s job was to build a bridge into the high school that accounted for these differences, while preparing students for the rigor of content-heavy AP exams in the later grades. And we needed to make this work with a traditional percentage grading system.

We addressed this challenge by building competencies based on the school’s ‘Portrait of the Graduate.’  We also borrowed from the IB Learner Profile and Approaches to Learning. We broke that language down as best we could into the following competencies: Communication, Growth Orientation, Citizenship, and Thinking. We further divided each competency into 5-7 strands and created 'I can' descriptors for each, assessing them on a mastery scale–Mastery, Almost There, Progressing, and Not There Yet. Each strand had a corresponding point total to their level of mastery, and the total score for each of these strands was the grade out of 100.  It was clear but complicated enough that we stopped talking about grades and started talking much more about each of the competency strands.

We determined grades entirely on four competency interviews, and it was the responsibility of each student to collect portfolio evidence to argue for their level of mastery in each strand. It was a reasonable compromise for all parties. If students asked what their grade was, we kept telling them that their score was the total of their best argument with evidence for each competency strand. With Canvas as our LMS, its mastery gradebook allowed us to keep track of the individual assessments, giving us data on when to push or intervene. Students and parents also had access to this data.

School B
Transitioning to School B required some compromise. It is a larger, national school that offers the IB’s DP program for students in grade 11 and 12. As this is a national system where students are ranked according to their grades, the grading process can be quite rigid, and students and parents tend to be hyper focussed on their scores.

At the same time, the school is committed to a competency-based focus. The challenge, then, is how to integrate these competencies into a system with heavy grade pressure.
In grading environments such as these, tension tends to run high, and anything new can feel like it will crush everything beneath it. Unlike the previous school, where we made the competencies the course’s criteria, grades here are attached to existing criteria. There are also strict percentages to determine average grades and heavy moderation of grades between classes.

The only flexibility was the 'Effort and Participation Score. It was only 10 percent, but in a grade heavy environment, that was still enough to get my students’ attention. So I took the competency-based structure I’d previously created and adapted it to the school’s competencies. In this case, I removed the numbers from each strand and created a rubric emphasizing risk-taking and honest reflection, characteristics that are rare in high-stakes assessment environments. I then made these competencies the core of the course, highlighting weekly strands and adapting the curriculum to focus on these.

The effect? It worked. Students' reflections were excellent and had a strong impact on their performance in future classes. Administrators and colleagues took notice, and there are discussions about expanding this into other areas next year. Most importantly, we found meaningful ways of getting language off the walls and into the minds and experiences of our students.

The easiest thing to have done this year would have been to drop this approach, but that’s really not a possibility for me anymore now that I see the impact of such work. In this case, we reached a reasonable compromise, and I’m excited to build on this next year.

Leveraging GenAI to Assist with Assessment:
I’ve had some early successes integrating GenAI into this process, and I’m looking forward to having a little more time this summer to explore how else I might do this. I welcome recommendations.

Reflections and Practice Interviews:
I uploaded the competencies into a GPT along with documentation on habit loops and other tools I use in class. Students can interact with this GPT in a number of ways. They can identify a strand they are struggling with and get some help. The tool will then ask the student to describe their experience with that strand. The student might then be prompted to recognize how habits might impact this, and the specific cue that is triggering the problem behavior. Having gone through this reflection, they can get help creating a new SMART goal that they can work with on their own or with me during Student Directed Learning Time. Students can also do an interactive self-assessment on the competencies or do a practice interview, receiving immediate feedback on their performance.
​
It works with cooperative students...

​And those who are less inclined…

​Feedback
“Virtual Mr. Pultz” is performing well. As it is trained on the rubrics, resources, and banter I provide, students can use him to get feedback in the style that I usually give. I’ve played around a bit with having it grade student writing, but as Leon Furze and others have reported, there are inconsistencies that make this unworkable…at least for now.
Simulations, Tools, Games, Exploration

Both the Cross-Cultural Navigator and the Negotiation Tool, which I adapted from Ethan Mollick’s prompt, are performing well. Students are using these for novel opportunities to practice for their speaking exams. I’ve also used Student Directed Learning Time to have students create their own GPTs. They created a fun murder mystery role-playing game that takes the names of all the participants, assigns them roles in a murder investigation, and plays out the drama. They see holes in the design, and we discuss how we might adjust the prompt to eliminate that hole. Students can use evidence from these experiences to argue for mastery in a number of our competencies.

Next Time:
Next week I’ll wish everyone a nice summer break and conclude this series with the article I’m most excited about: Giving Students the Space and Structure to Surprise You. In that piece I’ll also discuss some of my students’ thoughts on GenAI.