"Can AI Ace Your Exam? A Penn Law Experiment," by Prof. R. Polk Wagner
A very interesting preprint; here’s the Abstract:
I report a controlled experiment that ran a frontier large language model through eleven real University of Pennsylvania Carey Law School final exams in Spring 2026 and sent the machine’s answers out for blind grading alongside the work of enrolled students. Each exam was answered twice: once with no course materials in the model’s context (the “no-materials” arm, relying on the model’s training alone) and once with the course syllabus and an outline supplied (the “with-materials” arm). Three findings emerge.
First, the model is broadly competitive: across the ten exams returned with grades, it passed every one, sat at or above the class median on every one, and reached the top decile on seven. On two exams, both scored on an open, non-saturated scale, it outscored every enrolled student.
Second, the benefit of supplying course materials is real but concentrated and uneven. It is large on multiple-choice components, small to absent on essays, and on one exam the materials helped the machine’s multiple-choice score while hurting its essay—so the gain vanishes when the two are averaged into a single number. The subject of the exam matters as much as its format.
Third, faculty graders detected the AI submissions at a high rate, but mostly through signals that are easy to remove: formatting artifacts, identification-number placement, even a reversed quotation-mark glyph. The detection that survives those fixes is content-based—two independently generated answers that read as though written by the same hand, and a voice that does not match time-pressured student work. One grader was confident enough in detection to decline to grade the AI answers at all.
I read these results less as a verdict than as a measurement problem: when a model that never took the course can top the curve, it is no longer clear that the time-limited, high-stakes final is measuring what we have long assumed it measures—mastery of this course—rather than fluency in a genre the machine commands. That genre was always a contrivance built for grading, not a faithful model of practice, so the machine’s success says less about lawyering than the headline suggests; but it says a good deal about how much weight the instrument can still bear.
The urgent ask the experiment surfaces is intentionality: that faculty be clear about what each assessment is meant to measure, and recognize that AI is widening an already-real gap between the law-school exam and the practice it stands in for. The responses will range—from rethinking assessment top to bottom, to narrower adjustments, to simply naming the mismatch and grading with it in view—and reasonable colleagues will differ
The post “Can AI Ace Your Exam? A Penn Law Experiment,” by Prof. R. Polk Wagner appeared first on Reason Magazine.
Source: https://reason.com/volokh/2026/11/19/can-ai-ace-your-exam-a-penn-law-experiment-by-prof-r-polk-wagner/
Anyone can join.
Anyone can contribute.
Anyone can become informed about their world.
"United We Stand" Click Here To Create Your Personal Citizen Journalist Account Today, Be Sure To Invite Your Friends.
Before It’s News® is a community of individuals who report on what’s going on around them, from all around the world. Anyone can join. Anyone can contribute. Anyone can become informed about their world. "United We Stand" Click Here To Create Your Personal Citizen Journalist Account Today, Be Sure To Invite Your Friends.
LION'S MANE PRODUCT
Try Our Lion’s Mane WHOLE MIND Nootropic Blend 60 Capsules
Mushrooms are having a moment. One fabulous fungus in particular, lion’s mane, may help improve memory, depression and anxiety symptoms. They are also an excellent source of nutrients that show promise as a therapy for dementia, and other neurodegenerative diseases. If you’re living with anxiety or depression, you may be curious about all the therapy options out there — including the natural ones.Our Lion’s Mane WHOLE MIND Nootropic Blend has been formulated to utilize the potency of Lion’s mane but also include the benefits of four other Highly Beneficial Mushrooms. Synergistically, they work together to Build your health through improving cognitive function and immunity regardless of your age. Our Nootropic not only improves your Cognitive Function and Activates your Immune System, but it benefits growth of Essential Gut Flora, further enhancing your Vitality.
Our Formula includes: Lion’s Mane Mushrooms which Increase Brain Power through nerve growth, lessen anxiety, reduce depression, and improve concentration. Its an excellent adaptogen, promotes sleep and improves immunity. Shiitake Mushrooms which Fight cancer cells and infectious disease, boost the immune system, promotes brain function, and serves as a source of B vitamins. Maitake Mushrooms which regulate blood sugar levels of diabetics, reduce hypertension and boosts the immune system. Reishi Mushrooms which Fight inflammation, liver disease, fatigue, tumor growth and cancer. They Improve skin disorders and soothes digestive problems, stomach ulcers and leaky gut syndrome. Chaga Mushrooms which have anti-aging effects, boost immune function, improve stamina and athletic performance, even act as a natural aphrodisiac, fighting diabetes and improving liver function. Try Our Lion’s Mane WHOLE MIND Nootropic Blend 60 Capsules Today. Be 100% Satisfied or Receive a Full Money Back Guarantee. Order Yours Today by Following This Link.

