This week I gave a presentation to my college about assessment in the age of AI. Here's a written version of what I shared.
AI eats all the homework
Before the talk, I took the fundraising campaign analysis assignment from my nonprofit course and handed it to Claude. This assignment is a bit like a law school exam, requiring my students to look over a proposed campaign and look for any legal pitfalls related to the Federal tax code and other regulations. It’s annually the lowest average score in any of my classes because students consistently overlook issues.
With nothing but the assignment PDF, it scored 74% on my rubric. Seems like I’m safe, right? Keep teaching as I normally do? Nope. I also gave it the assignment plus my class slides—the same materials every student has—and it scored 100%. I cannot in good conscience require an assignment that AI can ace in a matter of minutes.
I can’t stress this enough. No homework is safe now.
But the problem isn't cheating
Like most professors, my first instinct might be exasperation over the ease of cheating. But that's the wrong thing to worry about.
As I wrote in another post, students are quite capable of finding excuses to have AI do their homework, for at least these reasons: the class isn't needed for their career, everyone else is doing it, something has to give in a heavy semester, and they'll use AI every day of their working lives anyway. The last one, notably, is true!
You see, the problem isn't that students can just have AI do the assignment. The problem is that I'm expecting them to do something AI can already do for them. Of course, they need to learn the nature of the legal dangers still, but not any further than enough to ask Claude Opus to assess their situation for them. Otherwise, it would be like me expecting them to learn to code before they can write an email.
If you’re freaking out on their behalf for being told to trust AI legal advice, can I remind you again of the perfect score Claude got on my hardest assignment of the year? And it’s only getting smarter.
All of this is to say that we educators have to make two urgent changes: choose what the world actually needs our students to learn, and measure in a way that ensures they've learned it.
What to choose
A colleague in the session at this point asked how to know what AI is good at doing. Given the jagged frontier of AI models, it’s quite hard to guess at what they’ve mastered.
The answer, of course, is simply to have a frontier model do every assignment your students do. Where it does the job expertly, you may not need to teach that anymore—at least not the same way. That's uncomfortable, for sure, especially to those of us who have taught the same thing for years. But doing otherwise means we’re just assigning busywork.
Where AI goes subtly wrong is the interesting part. The mark of expertise is, and will always be, nuance. The nuance is what we then teach. It’s foolhardy, though, to assume the models won’t have that covered soon enough. I say again, they are only getting smarter.
How to measure
Homework is now mostly just for practice. Practice and assessment (i.e. grades) should have always been distinct, even if we faculty designed assignments poorly enough to blend them. AI has now forced the separation. Grade homework for completion, let students use whatever they want, and move the real measurement to modalities that survive AI.
This summer I hired a couple of RAs to help me make my classes AI-native, to help my students have a class that encourages actual learning over delegating to AI. They did a remarkable job identifying all kinds of good ideas. I’m making a lot of changes, at least as many as I can pull off before classes start on Wednesday!
Because the range of what faculty teach and how we teach it varies widely, here are the ideas distilled into four principles:
- Presence. You watch the work happen: in-class exams and quizzes, oral work, recorded team discussions.
- Judgment. Students evaluate instead of produce: critique AI output, find planted flaws, defend valid criticism.
- Process. Students show their work by turning in how they did it: AI chat transcripts, drafts with visible revision, handwritten things.
- Particularity. Students operate with information the model doesn't have: data students gathered themselves, live client projects, things that happened in your classroom and attentive students captured.
I don’t have confidence that the specific examples above are durable, but I do have confidence in the principles themselves. Indeed, I think they are the same principles that reflect the future of professional success. They will improve the world with their presence and judgment, ensuring a sound process that weighs the particulars.
Cheating is missing the point
Preventing every chance of cheating has never been possible, at any point in the past. The real work we do is helping students want to learn—building courses where the point of the learning is compelling enough that not learning it feels like a loss to them.
I’ll finish by saying I’m convinced that managing AI agents will very soon be a professional qualification. It’s worthwhile right now having students work with the paid AI tools and assigning work that requires directing them well. That itself requires a refined judgment they acquire through practice. And, if you’re a teacher reading this, maybe consider honing the skill for yourself, too.