One professor's reflections on the end of an era, as AI tools such as ChatGPT have murdered the student essay (RIP). Here's why that threatens the future of human cognition—and how to save ourselves.
When I was studying history at the University of Vienna many years ago, our final exam was in the form of an hour-long Q&A discussion group of 3-4 students led by the professor. The quality of the answers and insights determined a large part of the ultimate grade for the course, with levels such as unsatisfactory - satisfactory - completely satisfactory - excellent - outstanding.
There are trade-offs with everything. In one of my Masters-level courses this year, 90% of the students spoke English as a second language and so, for some, their written English was much better than their spoken English. Take home essays were a great equalizer in this way, because having extra time smoothed out a lot of disadvantages...but now there's no way of doing them, so it's back to experimenting with other forms of in-person assessment, I think.
When l was teaching Art Appreciation (would meet the Gen.Ed. arts requirement), the first assignment given had 3 parts:
1. Students had to generate a description of Marc Chagal's I and the Village using AI. It was obvious the prompts they used was deficient.
2. Next, students had to use the description and analysis process of Feldman's method of art criticism to do the same task with the same painting (his method involves very deep LOOKING in the description and analysis).
3. Write an essay comparing the 2.
I got some interesting results.
I also took great pains to give writing/looking prompts that couldn't be done with Ai, like " Your budget is unlimited and you are buying a piece of art for your best friend, that matches their personality. Describe them and why the piece you chose fits that description visually." I found they had a very hard time writing (never mind the spelling and grammatical errors). The biggest complaint in my course evaluations were l made them write too much for a 100 level course (they only had 1 writing assignment a week).
I think you are correct that so few young people read ANYTHING; all is for immediate gratification (why read when you can see the movie?!?), and it absolutely interferes with any development of creative and critical thinking, and as we're seeing with this administration, seeing and understanding consequences for actions beyond the immediate result (i.e.entire departments gone, now there'sno ine to predict hurricanesaccurately, etc.). I'm not sure how to change that.
Lisa, thank you for such a brilliant classroom response to AI. I would be very interested in reading more about your experiences with these assignments - you should do a full post about this!
I got so frustrated with my students’ writing skills (or lack thereof) that we ended up spending a week going over Feldman's method of art criticism (mostly the looking and analysis parts), for all of the good it did; and this at a very expensive, local private university around here. I wondered how half of them got accepted in the first place (money). I didn't want to think of my students as lazy, entitled rich kids who never learned to think, so l quit (I was doing adjunct work after retirement to keep me busy), it just wasn't worth the battle.
I'm gleefully guilty of 2 things: 1.Not using my brain's limited memory space for remembering phone numbers.
2. Instantly stopping whatever I am doing to enrich my brain and soul with the writings/thoughts of Brian Klaas.
I am always grateful for your deep dives into the causations and consequences of humanity's ugly tendencies and shallow trends... Followed by inspiring reasons for us to resist them, and persist on our higher roads.
I blame multiple-choice testing. Guessing. I’m good at it, but essays brought out studying and over-studying. I knew lists I never needed, just in case it was on the test. I condensed, compared/contrasted, even studied the professor and what He/She seemed to think was important (I called it ‘professorology’).
There is an author somewhere (sorry can't remember the reference, but it's not my idea) who calls this performatively 'doing school,' which is not the same thing as learning.
When I lived in Egypt and taught SAT prep to high school students, one thing I was trained in was in how to teach students the technique of guessing which multiple choice answer was most likely to be correct.
I’m reminded of a certain professor I had in grad school (engineering) in the late ‘80s. Homework wasn’t graded, but the exams were all oral — Mano a mano. He was much feared. Most students avoided his classes. Professor S ran into one of my classmates on campus a full month after the oral final of one class and insisted he take the exam again, on the spot, because he’d done so poorly the first time. That classmate dropped out by the end of that day. True story. I took everything Professor S offered and had him on my thesis committee. He insisted on a mano a mano session with me before the oral prelim before the entire committee. Arrows were flung and arrows were deflected. Back. Forth. For one hour. Then two. And then…he seized up at one of my answers and smirked that he’d gotten me. Insisted I was wrong. I stood my ground. He started yelling. WRONG! WRONG! Told me to pick up a book and find the answer. I said no. Not wrong. Fifteen minute standoff and I saw a change come over his face as he realized HE was wrong. He shouted, “GAMMA! Just say GAMMA!” So I said gamma and we moved on. I’d passed. Take that, ChatGPT.
I remember taking the first AI course my university offered over 50 years ago. Programming in LISP - attempting to craft something that was “artificially intelligent.” My Greek CS/philosophy professor waxing on about what was to come. After class a small group would retire to the ‘Knollwood Tavern’ to discuss the class and speculate on what the future held. The consensus was “not in our lifetime” - but today - reality. The tavern is gone as are some of my classmates but we are in a new reality and as I look at what is to come I find it frightening. Brian thanks for your thoughts.
What a lovely trip down memory lane. I shudder to think what could be possible in 50 years from now. Some of it beyond wonderful, much of it dystopian, I suspect.
Love your work Brian and this essay. As a lawyer and expert in the field of risk management, I can validate that we need more critical thinking. Human nature is largely unchanged and can be manipulated through technology. That leaves us with our norms and institutions and as they continue to grow in complexity, we need more people (not less!) skilled with critical thinking and language as they are the essential tools for managing uncertainty.
This is a heartrending eulogy. I remember sweating it out during each of my semesters in college and grad school, how much I benefitted from those old timey blue books and term papers. As I've matured, I've come to appreciate how much this process helped mature my thinking. But one thing that might be considered is the focus on grading essays and papers, which distorts the purity of this process. Wouldn't a pass fail, with maybe some 'gold stars' for merit, help take this unnecessarily daunting pressure off? It might also be useful to do some meta-thinking, so as to fully recognize what's being lost: the ability to contemplate what you've read, a subtle relational process between writer and reader. Inherently creative, it brings forth new understandings. The value of contemplation was more clearly understood in earlier eras, I think, as a religious-philosophical process called lectio devina that contributed to insight. There is something about effort which opens the mind, and the subsequent combining of received and self-generated ideas. Sad that this 'alchemy' will be lost because adolescents will fail to grasp this value while rebelling against bending the knee to the judgment of those who have gone before them.
I've long been bemoaning the disconnect ChatGPT has between putting words on paper/pixel and THINKING. Writing, even a grocery list, involves thinking and choices. The choices range from what order to use introduce things to what word you want amongst synonyms that has the exact connotation you want to convey. It takes practice, and churning out even chunks of AI prose deprives one of that practice. Coming up with examples to make your point and particularly choosing an apt analogy are critical to understanding how thinking works.
I haven't seen much of my grandkids' writing. But I DO know that they simply do not read unless something is an assignment. They have, as Lisa Lieberman says, no idea what they are missing. Isn't ironic that as AI grabs more and more into its large language model, kids are refusing to look at the models right there available to them in the library: Alexandria or local store front.
I read so many books as a kid and it was the most thrilling thing for me. The big event - kids who grew up in the US in the late 80s and 90s will know what I'm talking about - was the Scholastic Book Fair, and I would always get to pick out a book. I hope those days are not gone and that the pendulum swings back -- often moments of tech overreach cause people to crave traditional means of connection, but there can be a lot of bad that happens before the realization takes place.
Me too. I've been going to the Queen Anne Public Library for 76 years. Every Saturday through grade school my parents dropped me off while they went shopping. Return 5 books, go home with 5 new ones.
I've been a bookworm all my life. Sadly my eyes now prevent me from reading my Dead Tree Books, but Kindle has jumped into the breach. I still have a copy of the first book I bought myself (with my allowance)--the Diary of Anne Frank.
Thanks, Nigella! It's certainly not too late...so hopefully there are more of these wake up calls happening in different sectors. *Everyone* I know in academia has the same experience I did; I just put it into an essay...and we are adapting, even though there is plenty of reason to be depressed about it all. (Despite my often dark subject matter, I'm a chronic optimist about humanity, I'm afraid).
Yes! Once again, delighted to engage with your thoughtful, insightful reflections, Brian. Ah, shall we of the Blue Book exam generation not also spare a moment to appreciate the value of the handwritten essay challenge to the student's learning process, (in spite of the professor's challenge to decipher the handwriting!).
We are voracious readers and so are our children. They tried to pass it on to the next generation with varying results.
One of our daughters has her Ph.D and teaches at the university level. She has a firm "No AI" rule in her classroom and has noticed a lot of "cheating" on writing assignments by students using AI. She has some apps that can check for AI generated text which she says work tolerably well. She runs down a few citations from every paper to see if they are real. There's only so much she can do, though. She also home schools using a secular designed program. From the results I've observed, it definitely is teaching them critical thinking, observation, etc.
When I was studying history at the University of Vienna many years ago, our final exam was in the form of an hour-long Q&A discussion group of 3-4 students led by the professor. The quality of the answers and insights determined a large part of the ultimate grade for the course, with levels such as unsatisfactory - satisfactory - completely satisfactory - excellent - outstanding.
There are trade-offs with everything. In one of my Masters-level courses this year, 90% of the students spoke English as a second language and so, for some, their written English was much better than their spoken English. Take home essays were a great equalizer in this way, because having extra time smoothed out a lot of disadvantages...but now there's no way of doing them, so it's back to experimenting with other forms of in-person assessment, I think.
Yes. Giving students a time to reflect and revise strengthens their thinking and writing in many ways.
Yes, I could predict my grade on a law school exam according to the type of test.
Brilliant piece. Thank you.
Thank you for reading and saying!
When l was teaching Art Appreciation (would meet the Gen.Ed. arts requirement), the first assignment given had 3 parts:
1. Students had to generate a description of Marc Chagal's I and the Village using AI. It was obvious the prompts they used was deficient.
2. Next, students had to use the description and analysis process of Feldman's method of art criticism to do the same task with the same painting (his method involves very deep LOOKING in the description and analysis).
3. Write an essay comparing the 2.
I got some interesting results.
I also took great pains to give writing/looking prompts that couldn't be done with Ai, like " Your budget is unlimited and you are buying a piece of art for your best friend, that matches their personality. Describe them and why the piece you chose fits that description visually." I found they had a very hard time writing (never mind the spelling and grammatical errors). The biggest complaint in my course evaluations were l made them write too much for a 100 level course (they only had 1 writing assignment a week).
I think you are correct that so few young people read ANYTHING; all is for immediate gratification (why read when you can see the movie?!?), and it absolutely interferes with any development of creative and critical thinking, and as we're seeing with this administration, seeing and understanding consequences for actions beyond the immediate result (i.e.entire departments gone, now there'sno ine to predict hurricanesaccurately, etc.). I'm not sure how to change that.
Lisa, thank you for such a brilliant classroom response to AI. I would be very interested in reading more about your experiences with these assignments - you should do a full post about this!
I got so frustrated with my students’ writing skills (or lack thereof) that we ended up spending a week going over Feldman's method of art criticism (mostly the looking and analysis parts), for all of the good it did; and this at a very expensive, local private university around here. I wondered how half of them got accepted in the first place (money). I didn't want to think of my students as lazy, entitled rich kids who never learned to think, so l quit (I was doing adjunct work after retirement to keep me busy), it just wasn't worth the battle.
I was once told by a repairman that he had given up studying to be a mechanical engineers because all the answers were on line.
I replied that in my career as an electrical engineer that I had determined that the key in engineering was not the answer, it was the question.
Gertrude Stein, on her deathbed: " What is the answer?"
She recieived no reply. She followed up with:
" In that case, what is the question?"
Answers are nice. Our humanity comes from the Question.
Your writings always make me think.
Mission Accomplished!
I'm gleefully guilty of 2 things: 1.Not using my brain's limited memory space for remembering phone numbers.
2. Instantly stopping whatever I am doing to enrich my brain and soul with the writings/thoughts of Brian Klaas.
I am always grateful for your deep dives into the causations and consequences of humanity's ugly tendencies and shallow trends... Followed by inspiring reasons for us to resist them, and persist on our higher roads.
I blame multiple-choice testing. Guessing. I’m good at it, but essays brought out studying and over-studying. I knew lists I never needed, just in case it was on the test. I condensed, compared/contrasted, even studied the professor and what He/She seemed to think was important (I called it ‘professorology’).
There is an author somewhere (sorry can't remember the reference, but it's not my idea) who calls this performatively 'doing school,' which is not the same thing as learning.
When I lived in Egypt and taught SAT prep to high school students, one thing I was trained in was in how to teach students the technique of guessing which multiple choice answer was most likely to be correct.
I’m reminded of a certain professor I had in grad school (engineering) in the late ‘80s. Homework wasn’t graded, but the exams were all oral — Mano a mano. He was much feared. Most students avoided his classes. Professor S ran into one of my classmates on campus a full month after the oral final of one class and insisted he take the exam again, on the spot, because he’d done so poorly the first time. That classmate dropped out by the end of that day. True story. I took everything Professor S offered and had him on my thesis committee. He insisted on a mano a mano session with me before the oral prelim before the entire committee. Arrows were flung and arrows were deflected. Back. Forth. For one hour. Then two. And then…he seized up at one of my answers and smirked that he’d gotten me. Insisted I was wrong. I stood my ground. He started yelling. WRONG! WRONG! Told me to pick up a book and find the answer. I said no. Not wrong. Fifteen minute standoff and I saw a change come over his face as he realized HE was wrong. He shouted, “GAMMA! Just say GAMMA!” So I said gamma and we moved on. I’d passed. Take that, ChatGPT.
I remember taking the first AI course my university offered over 50 years ago. Programming in LISP - attempting to craft something that was “artificially intelligent.” My Greek CS/philosophy professor waxing on about what was to come. After class a small group would retire to the ‘Knollwood Tavern’ to discuss the class and speculate on what the future held. The consensus was “not in our lifetime” - but today - reality. The tavern is gone as are some of my classmates but we are in a new reality and as I look at what is to come I find it frightening. Brian thanks for your thoughts.
What a lovely trip down memory lane. I shudder to think what could be possible in 50 years from now. Some of it beyond wonderful, much of it dystopian, I suspect.
Love your work Brian and this essay. As a lawyer and expert in the field of risk management, I can validate that we need more critical thinking. Human nature is largely unchanged and can be manipulated through technology. That leaves us with our norms and institutions and as they continue to grow in complexity, we need more people (not less!) skilled with critical thinking and language as they are the essential tools for managing uncertainty.
This is a heartrending eulogy. I remember sweating it out during each of my semesters in college and grad school, how much I benefitted from those old timey blue books and term papers. As I've matured, I've come to appreciate how much this process helped mature my thinking. But one thing that might be considered is the focus on grading essays and papers, which distorts the purity of this process. Wouldn't a pass fail, with maybe some 'gold stars' for merit, help take this unnecessarily daunting pressure off? It might also be useful to do some meta-thinking, so as to fully recognize what's being lost: the ability to contemplate what you've read, a subtle relational process between writer and reader. Inherently creative, it brings forth new understandings. The value of contemplation was more clearly understood in earlier eras, I think, as a religious-philosophical process called lectio devina that contributed to insight. There is something about effort which opens the mind, and the subsequent combining of received and self-generated ideas. Sad that this 'alchemy' will be lost because adolescents will fail to grasp this value while rebelling against bending the knee to the judgment of those who have gone before them.
Blue books are coming back.
Happyface emoji!!
I've long been bemoaning the disconnect ChatGPT has between putting words on paper/pixel and THINKING. Writing, even a grocery list, involves thinking and choices. The choices range from what order to use introduce things to what word you want amongst synonyms that has the exact connotation you want to convey. It takes practice, and churning out even chunks of AI prose deprives one of that practice. Coming up with examples to make your point and particularly choosing an apt analogy are critical to understanding how thinking works.
I haven't seen much of my grandkids' writing. But I DO know that they simply do not read unless something is an assignment. They have, as Lisa Lieberman says, no idea what they are missing. Isn't ironic that as AI grabs more and more into its large language model, kids are refusing to look at the models right there available to them in the library: Alexandria or local store front.
I read so many books as a kid and it was the most thrilling thing for me. The big event - kids who grew up in the US in the late 80s and 90s will know what I'm talking about - was the Scholastic Book Fair, and I would always get to pick out a book. I hope those days are not gone and that the pendulum swings back -- often moments of tech overreach cause people to crave traditional means of connection, but there can be a lot of bad that happens before the realization takes place.
Me too. I've been going to the Queen Anne Public Library for 76 years. Every Saturday through grade school my parents dropped me off while they went shopping. Return 5 books, go home with 5 new ones.
I've been a bookworm all my life. Sadly my eyes now prevent me from reading my Dead Tree Books, but Kindle has jumped into the breach. I still have a copy of the first book I bought myself (with my allowance)--the Diary of Anne Frank.
Brilliant, though depressing, piece
Thanks, Nigella! It's certainly not too late...so hopefully there are more of these wake up calls happening in different sectors. *Everyone* I know in academia has the same experience I did; I just put it into an essay...and we are adapting, even though there is plenty of reason to be depressed about it all. (Despite my often dark subject matter, I'm a chronic optimist about humanity, I'm afraid).
Yes! Once again, delighted to engage with your thoughtful, insightful reflections, Brian. Ah, shall we of the Blue Book exam generation not also spare a moment to appreciate the value of the handwritten essay challenge to the student's learning process, (in spite of the professor's challenge to decipher the handwriting!).
We are voracious readers and so are our children. They tried to pass it on to the next generation with varying results.
One of our daughters has her Ph.D and teaches at the university level. She has a firm "No AI" rule in her classroom and has noticed a lot of "cheating" on writing assignments by students using AI. She has some apps that can check for AI generated text which she says work tolerably well. She runs down a few citations from every paper to see if they are real. There's only so much she can do, though. She also home schools using a secular designed program. From the results I've observed, it definitely is teaching them critical thinking, observation, etc.
I am also sad that I don’t remember any phone numbers - not even my kids! Love your Bankman-fried reference and comment.