Readings on AI vs critical thinking and writing?
May 30, 2023 1:46 PM   Subscribe

Every teacher I know is dealing with an avalanche of essays written by ChatGPT or Grammarly. There are lots of ways to detect cheating and redesign assignments to make it harder, but beyond just punishment and deterrence, it would be nice to convince students that doing your own writing, thinking your own thoughts, has value and is worth doing! Please recommend me some articles that teachers could assign that address this issue in the context of AI.
posted by Beardman to Education (14 answers total) 17 users marked this as a favorite
 
It requires a certain level of development on the part of the student, but this thread by CW Howell describes having the students get the AI to write an essay, and then grading it themselves for accuracy, specificity, etc. I imagine that would be much more effective than reading an article, but if you do want to point them at one, find a good writeup of Steven A. Schwartz submitting ChatGPT-generated court cases and, at minimum, getting his entire firm yelled at by a judge.
posted by sagc at 1:50 PM on May 30, 2023 [5 favorites]


Response by poster: Yeah, exercise ideas are welcome too! Assigning readings would just be one part of the approach.
posted by Beardman at 2:01 PM on May 30, 2023


GPT-4's own announcement page says it's relatively bad at AP English exams and has a graph showing how much worse it is on those tasks. If the students can't see that themselves, maybe they'll believe what is essentially product guidance. On Twitter, Camestros Felapton has been posting some examples of ChatGPT not reasoning about simple things like the spelling of CAT and the spelling of MISSISSIPPI. I don't know if he's using GPT-4 but maybe it's something students would understand as a failure mode relevant to English lit--i.e. if it doesn't really inspect words and reflect on them, what's it going to do with a poem it has never seen before. As for why that matters, there's Shelley's answer and everything that cites it.
posted by Wobbuffet at 3:12 PM on May 30, 2023


This is a different perspective on ChatGPT, more from the perspective of librarians and higher ed, and I think it's quite helpful: "Getting a Grip on ChatGPT."

They also link to these other articles:
"What ChatGPT Can't Teach My Writing Students"
"Making AI Generative for Higher Education"

But honestly, I'd share this one with students without much intro or comment, and let them learn some of the lessons this way: "Here's What Happens When Your Lawyer Uses ChatGPT."

Another teacher that I know is asking students to do more writing in class, and she says it's been a great idea for a host of reasons.
posted by bluedaisy at 3:46 PM on May 30, 2023 [5 favorites]


FWIW, throwing some very recent poems at ChatGPT, what it does by default is summarize them for main ideas, but there are many online resources aimed at teaching students why summary isn't analysis. Also, if exercise ideas are useful, Greg Ulmer's relatively weird assignments in Internet Invention hold up because so many of them essentially ask students to relate texts to personal experiences that might not be worth the bother of fictionalizing. The assignments in Scholes, Comley, & Ulmer's Text Book are more conventional but some are comparable.
posted by Wobbuffet at 3:50 PM on May 30, 2023 [2 favorites]


There are lots of interesting pieces here, and I'll just add a *self-link* to a piece I wrote on why large language models fundamentally don't know what is right or wrong. I think the problem with students is partly overconfidence in the results, so making them doubt and critically assess the oracle is valuable.
posted by BlackLeotardFront at 5:00 PM on May 30, 2023 [1 favorite]


Students (and parents) think that the point of education is credentials or grades, not learning. They believed therefore that the person who worked the least was the winner, not the loser. Because of that, I always had my English students write their essays in class, otherwise they would plagiarize or their parents would, and the kids who actually did their own writing often didn't do as well grade-wise, even if they were learning more. When I taught college, I was lucky to be teaching a practicum where their papers had to be based in up-close, prolonged observation in their practicum sites (where I also spent time and so could tell when they made something up).

Not a big fan of homework anyway.
posted by Peach at 6:55 PM on May 30, 2023 [3 favorites]


This Twitter thread is relevant and fascinating:
So I followed @GaryMarcus's suggestion and had my undergrad class use ChatGPT for a critical assignment. I had them all generate an essay using a prompt I gave them, and then their job was to "grade" it--look for hallucinated info and critique its analysis. *All 63* essays had

hallucinated information. Fake quotes, fake sources, or real sources misunderstood and mischaracterized. Every single assignment. I was stunned--I figured the rate would be high, but not that high.

The biggest takeaway from this was that the students all learned that it isn't fully reliable. Before doing it, many of them were under the impression it was always right. Their feedback largely focused on how shocked they were that it could mislead them. Probably 50% of them

were unaware it could do this. All of them expressed fears and concerns about mental atrophy and the possibility for misinformation/fake news. One student was worried that their neural pathways formed from critical thinking would start to degrade or weaken. One other student

opined that AI both knew more than us but is dumber than we are since it cannot think critically. She wrote, "I’m not worried about AI getting to where we are now. I’m much more worried about the possibility of us reverting to where AI is."
posted by Artifice_Eternity at 7:26 PM on May 30, 2023 [6 favorites]


Ah -- I see the C.W. Howell thing was mentioned in the first comment.
posted by Artifice_Eternity at 7:30 PM on May 30, 2023


The Shakespeare Unlimited podcast had a recent episode that might be of a little interest, that they titled "Artificial Intelligence Goes to English Class". There is a full transcript of the 37 minute episode at their webpage.
posted by gudrun at 8:39 PM on May 30, 2023


This ACOUP article made a good case for the problems with ChatGPT.
Thus the last and most important thing I am trying to train is not the form of the essay nor its content, but the basic skills of having a thought and putting it in a box that we outlined earlier. Even if your job or hobbies do not involve formal writing, chances are (especially if your job requires a college degree) you are still expected to observe something real, make conclusions about it and then present those conclusions to someone else (boss, subordinates, co-workers, customers, etc.) in a clear way, supported by convincing evidence if challenged. What we are practicing then is how to have good thoughts, put them in good boxes and then effectively hand that box to someone else. That can be done in a formal written form (the essay), in informal writing (emails, memos, notes, Slack conversations), or verbally (speeches, but also arguments, debates and discussions). The skills of having the idea, supporting it with evidence, organizing that evidence effectively to be understood and then communicating that effectively are transferable and the most important skills that are being practiced when a student writes an essay.

Crucially – and somehow this point seems to be missed by many of ChatGPT’s boosters I encountered on social media – at no point in this process do I actually want the essays. Yes, they have to be turned in to me and graded and commented because that feedback in turn is meant to both motivate students to improve but also to signal where they need to improve. But I did not assign the project because I wanted the essays. To indulge in an analogy, I am not asking my students to forge some nails because I want a whole bunch of nails – the nails they forge on early attempts will be quite bad anyway. I am asking them to forge nails so that they learn how to forge nails (which is why I inspect the nails and explain their defects each time) and by extension also learn how to forge other things that are akin to nails. I want students to learn how to analyze, organize ideas and communicate those ideas.

What one can immediately see is that a student who simply uses ChatGPT to write their essay for them has simply cheated themselves out of the opportunity to learn (and also wasted my time in providing comments and grades). As we’ve seen above, ChatGPT cannot effectively replace the actual core tasks we are training for, so this is not a case where the existence of spinning jennies renders most training at hand spinning obsolete. And it certainly doesn’t fulfill the purpose of the assignment.
posted by TheophileEscargot at 10:19 PM on May 30, 2023 [7 favorites]


I'll just add a *self-link* to a piece I wrote

I think it's a good piece, but I take issue with the conclusion:
for the present this is a useful mental model to keep in mind: The AI wants you to believe it and will say anything to improve its chances.
It's so hard not to think of these things in animist ways, but for the sake of our own safety I really think we need to remain rigorous about doing so.

The AI does not want you to believe it, because sparkling autocorrect is not built to want things any more than it's built to know them.

The entity with a stake in having you ascribe meaning to the output of a chatbot is the designer and/or operator of the chatbot, not the bot itself. We must pay attention to the men behind the curtain.
posted by flabdablet at 3:43 AM on May 31, 2023 [9 favorites]


Please recommend me some articles that teachers could assign that address this issue in the context of AI.

I think you might find value in having your students learn just what it is that ChatGPT actually does, and to that end you might want to have them spend a little time playing with ChatGPT's amoebic ancestor.
posted by flabdablet at 3:48 AM on May 31, 2023


Could you give them all the same essay generated by ChatGPT to critique?
posted by gottabefunky at 1:48 PM on May 31, 2023


« Older Higher Ed Under Fire   |   What can I do to prepare for aftermath of knee... Newer »

You are not logged in, either login or create an account to post comments