One professor's reflections on the end of an era, as AI tools such as ChatGPT have murdered the student essay (RIP). Here's why that threatens the future of human cognition—and how to save ourselves.
When I was studying history at the University of Vienna many years ago, our final exam was in the form of an hour-long Q&A discussion group of 3-4 students led by the professor. The quality of the answers and insights determined a large part of the ultimate grade for the course, with levels such as unsatisfactory - satisfactory - completely satisfactory - excellent - outstanding.
There are trade-offs with everything. In one of my Masters-level courses this year, 90% of the students spoke English as a second language and so, for some, their written English was much better than their spoken English. Take home essays were a great equalizer in this way, because having extra time smoothed out a lot of disadvantages...but now there's no way of doing them, so it's back to experimenting with other forms of in-person assessment, I think.
I'm gleefully guilty of 2 things: 1.Not using my brain's limited memory space for remembering phone numbers.
2. Instantly stopping whatever I am doing to enrich my brain and soul with the writings/thoughts of Brian Klaas.
I am always grateful for your deep dives into the causations and consequences of humanity's ugly tendencies and shallow trends... Followed by inspiring reasons for us to resist them, and persist on our higher roads.
When l was teaching Art Appreciation (would meet the Gen.Ed. arts requirement), the first assignment given had 3 parts:
1. Students had to generate a description of Marc Chagal's I and the Village using AI. It was obvious the prompts they used was deficient.
2. Next, students had to use the description and analysis process of Feldman's method of art criticism to do the same task with the same painting (his method involves very deep LOOKING in the description and analysis).
3. Write an essay comparing the 2.
I got some interesting results.
I also took great pains to give writing/looking prompts that couldn't be done with Ai, like " Your budget is unlimited and you are buying a piece of art for your best friend, that matches their personality. Describe them and why the piece you chose fits that description visually." I found they had a very hard time writing (never mind the spelling and grammatical errors). The biggest complaint in my course evaluations were l made them write too much for a 100 level course (they only had 1 writing assignment a week).
I think you are correct that so few young people read ANYTHING; all is for immediate gratification (why read when you can see the movie?!?), and it absolutely interferes with any development of creative and critical thinking, and as we're seeing with this administration, seeing and understanding consequences for actions beyond the immediate result (i.e.entire departments gone, now there'sno ine to predict hurricanesaccurately, etc.). I'm not sure how to change that.
I’m reminded of a certain professor I had in grad school (engineering) in the late ‘80s. Homework wasn’t graded, but the exams were all oral — Mano a mano. He was much feared. Most students avoided his classes. Professor S ran into one of my classmates on campus a full month after the oral final of one class and insisted he take the exam again, on the spot, because he’d done so poorly the first time. That classmate dropped out by the end of that day. True story. I took everything Professor S offered and had him on my thesis committee. He insisted on a mano a mano session with me before the oral prelim before the entire committee. Arrows were flung and arrows were deflected. Back. Forth. For one hour. Then two. And then…he seized up at one of my answers and smirked that he’d gotten me. Insisted I was wrong. I stood my ground. He started yelling. WRONG! WRONG! Told me to pick up a book and find the answer. I said no. Not wrong. Fifteen minute standoff and I saw a change come over his face as he realized HE was wrong. He shouted, “GAMMA! Just say GAMMA!” So I said gamma and we moved on. I’d passed. Take that, ChatGPT.
I blame multiple-choice testing. Guessing. I’m good at it, but essays brought out studying and over-studying. I knew lists I never needed, just in case it was on the test. I condensed, compared/contrasted, even studied the professor and what He/She seemed to think was important (I called it ‘professorology’).
There is an author somewhere (sorry can't remember the reference, but it's not my idea) who calls this performatively 'doing school,' which is not the same thing as learning.
When I lived in Egypt and taught SAT prep to high school students, one thing I was trained in was in how to teach students the technique of guessing which multiple choice answer was most likely to be correct.
I remember taking the first AI course my university offered over 50 years ago. Programming in LISP - attempting to craft something that was “artificially intelligent.” My Greek CS/philosophy professor waxing on about what was to come. After class a small group would retire to the ‘Knollwood Tavern’ to discuss the class and speculate on what the future held. The consensus was “not in our lifetime” - but today - reality. The tavern is gone as are some of my classmates but we are in a new reality and as I look at what is to come I find it frightening. Brian thanks for your thoughts.
What a lovely trip down memory lane. I shudder to think what could be possible in 50 years from now. Some of it beyond wonderful, much of it dystopian, I suspect.
This is a heartrending eulogy. I remember sweating it out during each of my semesters in college and grad school, how much I benefitted from those old timey blue books and term papers. As I've matured, I've come to appreciate how much this process helped mature my thinking. But one thing that might be considered is the focus on grading essays and papers, which distorts the purity of this process. Wouldn't a pass fail, with maybe some 'gold stars' for merit, help take this unnecessarily daunting pressure off? It might also be useful to do some meta-thinking, so as to fully recognize what's being lost: the ability to contemplate what you've read, a subtle relational process between writer and reader. Inherently creative, it brings forth new understandings. The value of contemplation was more clearly understood in earlier eras, I think, as a religious-philosophical process called lectio devina that contributed to insight. There is something about effort which opens the mind, and the subsequent combining of received and self-generated ideas. Sad that this 'alchemy' will be lost because adolescents will fail to grasp this value while rebelling against bending the knee to the judgment of those who have gone before them.
Thanks, Nigella! It's certainly not too late...so hopefully there are more of these wake up calls happening in different sectors. *Everyone* I know in academia has the same experience I did; I just put it into an essay...and we are adapting, even though there is plenty of reason to be depressed about it all. (Despite my often dark subject matter, I'm a chronic optimist about humanity, I'm afraid).
Love your work Brian and this essay. As a lawyer and expert in the field of risk management, I can validate that we need more critical thinking. Human nature is largely unchanged and can be manipulated through technology. That leaves us with our norms and institutions and as they continue to grow in complexity, we need more people (not less!) skilled with critical thinking and language as they are the essential tools for managing uncertainty.
I've long been bemoaning the disconnect ChatGPT has between putting words on paper/pixel and THINKING. Writing, even a grocery list, involves thinking and choices. The choices range from what order to use introduce things to what word you want amongst synonyms that has the exact connotation you want to convey. It takes practice, and churning out even chunks of AI prose deprives one of that practice. Coming up with examples to make your point and particularly choosing an apt analogy are critical to understanding how thinking works.
I haven't seen much of my grandkids' writing. But I DO know that they simply do not read unless something is an assignment. They have, as Lisa Lieberman says, no idea what they are missing. Isn't ironic that as AI grabs more and more into its large language model, kids are refusing to look at the models right there available to them in the library: Alexandria or local store front.
I read so many books as a kid and it was the most thrilling thing for me. The big event - kids who grew up in the US in the late 80s and 90s will know what I'm talking about - was the Scholastic Book Fair, and I would always get to pick out a book. I hope those days are not gone and that the pendulum swings back -- often moments of tech overreach cause people to crave traditional means of connection, but there can be a lot of bad that happens before the realization takes place.
Me too. I've been going to the Queen Anne Public Library for 76 years. Every Saturday through grade school my parents dropped me off while they went shopping. Return 5 books, go home with 5 new ones.
I've been a bookworm all my life. Sadly my eyes now prevent me from reading my Dead Tree Books, but Kindle has jumped into the breach. I still have a copy of the first book I bought myself (with my allowance)--the Diary of Anne Frank.
Yes! Once again, delighted to engage with your thoughtful, insightful reflections, Brian. Ah, shall we of the Blue Book exam generation not also spare a moment to appreciate the value of the handwritten essay challenge to the student's learning process, (in spite of the professor's challenge to decipher the handwriting!).
I wholeheartedly agree that writing/thinking/note-taking/research/reading and seminars are crucial to the development of intellect and mind. I have seen medicine going down the tubes for similar reasons (excuse the awkward metaphor).
When I was studying history at the University of Vienna many years ago, our final exam was in the form of an hour-long Q&A discussion group of 3-4 students led by the professor. The quality of the answers and insights determined a large part of the ultimate grade for the course, with levels such as unsatisfactory - satisfactory - completely satisfactory - excellent - outstanding.
There are trade-offs with everything. In one of my Masters-level courses this year, 90% of the students spoke English as a second language and so, for some, their written English was much better than their spoken English. Take home essays were a great equalizer in this way, because having extra time smoothed out a lot of disadvantages...but now there's no way of doing them, so it's back to experimenting with other forms of in-person assessment, I think.
Yes. Giving students a time to reflect and revise strengthens their thinking and writing in many ways.
Yes, I could predict my grade on a law school exam according to the type of test.
Brilliant piece. Thank you.
Thank you for reading and saying!
I'm gleefully guilty of 2 things: 1.Not using my brain's limited memory space for remembering phone numbers.
2. Instantly stopping whatever I am doing to enrich my brain and soul with the writings/thoughts of Brian Klaas.
I am always grateful for your deep dives into the causations and consequences of humanity's ugly tendencies and shallow trends... Followed by inspiring reasons for us to resist them, and persist on our higher roads.
I was once told by a repairman that he had given up studying to be a mechanical engineers because all the answers were on line.
I replied that in my career as an electrical engineer that I had determined that the key in engineering was not the answer, it was the question.
Gertrude Stein, on her deathbed: " What is the answer?"
She recieived no reply. She followed up with:
" In that case, what is the question?"
Answers are nice. Our humanity comes from the Question.
Your writings always make me think.
Mission Accomplished!
When l was teaching Art Appreciation (would meet the Gen.Ed. arts requirement), the first assignment given had 3 parts:
1. Students had to generate a description of Marc Chagal's I and the Village using AI. It was obvious the prompts they used was deficient.
2. Next, students had to use the description and analysis process of Feldman's method of art criticism to do the same task with the same painting (his method involves very deep LOOKING in the description and analysis).
3. Write an essay comparing the 2.
I got some interesting results.
I also took great pains to give writing/looking prompts that couldn't be done with Ai, like " Your budget is unlimited and you are buying a piece of art for your best friend, that matches their personality. Describe them and why the piece you chose fits that description visually." I found they had a very hard time writing (never mind the spelling and grammatical errors). The biggest complaint in my course evaluations were l made them write too much for a 100 level course (they only had 1 writing assignment a week).
I think you are correct that so few young people read ANYTHING; all is for immediate gratification (why read when you can see the movie?!?), and it absolutely interferes with any development of creative and critical thinking, and as we're seeing with this administration, seeing and understanding consequences for actions beyond the immediate result (i.e.entire departments gone, now there'sno ine to predict hurricanesaccurately, etc.). I'm not sure how to change that.
I’m reminded of a certain professor I had in grad school (engineering) in the late ‘80s. Homework wasn’t graded, but the exams were all oral — Mano a mano. He was much feared. Most students avoided his classes. Professor S ran into one of my classmates on campus a full month after the oral final of one class and insisted he take the exam again, on the spot, because he’d done so poorly the first time. That classmate dropped out by the end of that day. True story. I took everything Professor S offered and had him on my thesis committee. He insisted on a mano a mano session with me before the oral prelim before the entire committee. Arrows were flung and arrows were deflected. Back. Forth. For one hour. Then two. And then…he seized up at one of my answers and smirked that he’d gotten me. Insisted I was wrong. I stood my ground. He started yelling. WRONG! WRONG! Told me to pick up a book and find the answer. I said no. Not wrong. Fifteen minute standoff and I saw a change come over his face as he realized HE was wrong. He shouted, “GAMMA! Just say GAMMA!” So I said gamma and we moved on. I’d passed. Take that, ChatGPT.
I blame multiple-choice testing. Guessing. I’m good at it, but essays brought out studying and over-studying. I knew lists I never needed, just in case it was on the test. I condensed, compared/contrasted, even studied the professor and what He/She seemed to think was important (I called it ‘professorology’).
There is an author somewhere (sorry can't remember the reference, but it's not my idea) who calls this performatively 'doing school,' which is not the same thing as learning.
When I lived in Egypt and taught SAT prep to high school students, one thing I was trained in was in how to teach students the technique of guessing which multiple choice answer was most likely to be correct.
I remember taking the first AI course my university offered over 50 years ago. Programming in LISP - attempting to craft something that was “artificially intelligent.” My Greek CS/philosophy professor waxing on about what was to come. After class a small group would retire to the ‘Knollwood Tavern’ to discuss the class and speculate on what the future held. The consensus was “not in our lifetime” - but today - reality. The tavern is gone as are some of my classmates but we are in a new reality and as I look at what is to come I find it frightening. Brian thanks for your thoughts.
What a lovely trip down memory lane. I shudder to think what could be possible in 50 years from now. Some of it beyond wonderful, much of it dystopian, I suspect.
This is a heartrending eulogy. I remember sweating it out during each of my semesters in college and grad school, how much I benefitted from those old timey blue books and term papers. As I've matured, I've come to appreciate how much this process helped mature my thinking. But one thing that might be considered is the focus on grading essays and papers, which distorts the purity of this process. Wouldn't a pass fail, with maybe some 'gold stars' for merit, help take this unnecessarily daunting pressure off? It might also be useful to do some meta-thinking, so as to fully recognize what's being lost: the ability to contemplate what you've read, a subtle relational process between writer and reader. Inherently creative, it brings forth new understandings. The value of contemplation was more clearly understood in earlier eras, I think, as a religious-philosophical process called lectio devina that contributed to insight. There is something about effort which opens the mind, and the subsequent combining of received and self-generated ideas. Sad that this 'alchemy' will be lost because adolescents will fail to grasp this value while rebelling against bending the knee to the judgment of those who have gone before them.
Blue books are coming back.
Happyface emoji!!
Brilliant, though depressing, piece
Thanks, Nigella! It's certainly not too late...so hopefully there are more of these wake up calls happening in different sectors. *Everyone* I know in academia has the same experience I did; I just put it into an essay...and we are adapting, even though there is plenty of reason to be depressed about it all. (Despite my often dark subject matter, I'm a chronic optimist about humanity, I'm afraid).
Love your work Brian and this essay. As a lawyer and expert in the field of risk management, I can validate that we need more critical thinking. Human nature is largely unchanged and can be manipulated through technology. That leaves us with our norms and institutions and as they continue to grow in complexity, we need more people (not less!) skilled with critical thinking and language as they are the essential tools for managing uncertainty.
I've long been bemoaning the disconnect ChatGPT has between putting words on paper/pixel and THINKING. Writing, even a grocery list, involves thinking and choices. The choices range from what order to use introduce things to what word you want amongst synonyms that has the exact connotation you want to convey. It takes practice, and churning out even chunks of AI prose deprives one of that practice. Coming up with examples to make your point and particularly choosing an apt analogy are critical to understanding how thinking works.
I haven't seen much of my grandkids' writing. But I DO know that they simply do not read unless something is an assignment. They have, as Lisa Lieberman says, no idea what they are missing. Isn't ironic that as AI grabs more and more into its large language model, kids are refusing to look at the models right there available to them in the library: Alexandria or local store front.
I read so many books as a kid and it was the most thrilling thing for me. The big event - kids who grew up in the US in the late 80s and 90s will know what I'm talking about - was the Scholastic Book Fair, and I would always get to pick out a book. I hope those days are not gone and that the pendulum swings back -- often moments of tech overreach cause people to crave traditional means of connection, but there can be a lot of bad that happens before the realization takes place.
Me too. I've been going to the Queen Anne Public Library for 76 years. Every Saturday through grade school my parents dropped me off while they went shopping. Return 5 books, go home with 5 new ones.
I've been a bookworm all my life. Sadly my eyes now prevent me from reading my Dead Tree Books, but Kindle has jumped into the breach. I still have a copy of the first book I bought myself (with my allowance)--the Diary of Anne Frank.
Yes! Once again, delighted to engage with your thoughtful, insightful reflections, Brian. Ah, shall we of the Blue Book exam generation not also spare a moment to appreciate the value of the handwritten essay challenge to the student's learning process, (in spite of the professor's challenge to decipher the handwriting!).
I am also sad that I don’t remember any phone numbers - not even my kids! Love your Bankman-fried reference and comment.
Cogent arguments, Benjamin ;)
I wholeheartedly agree that writing/thinking/note-taking/research/reading and seminars are crucial to the development of intellect and mind. I have seen medicine going down the tubes for similar reasons (excuse the awkward metaphor).
Yeah that "Benjamin" is awesome, right? LoL!