LLM users: How have ChatGPT and friends been useful for you?
May 25, 2024 10:35 AM   Subscribe

When LLMs first arrived there were a lot of aspirational uses presented. It's 18 months later and if you've adopted an LLM into your workflow it would be interesting to know what has actually worked out.

We are all aware of the unreliable nature of LLMs so there's no need for everyone to include that particular caveat.
posted by Tell Me No Lies to Computers & Internet (31 answers total) 35 users marked this as a favorite
 
Response by poster: As the OP I will go first:
  • image search, particularly by description
  • kickstarting investigations of new topics
  • coding suggestions
  • translating medicalese into simpler terms
  • translating text from Spanish

posted by Tell Me No Lies at 10:35 AM on May 25 [2 favorites]


Learning new technologies. Am learning Gremlin / Tinkerpop right now and that shit is gnarly. It's a game-changer to be able to ask ChatGPT to translate plain English into Gremlin.
posted by Sauce Trough at 10:43 AM on May 25


I use Copilot more than I thought I would, once I accepted it as sparkling autocomplete and figured out its limitations. It just saves a lot of typing of boilerplate like error and log messages. Sometimes it's like a drunk autodidact pair programmer, but sometimes it comes up with exactly what you were thinking.

I use chat-style LLMs for code snippets in an unfamiliar language/API, brainstorming, back-of-the-envelope calculations, and attempting to rearrange the occasional equation.

Have not had much success with the plugin-style "generate a program that answers the question" mode. (Whatever happened with that Wolfram integration anyway...?)

Oh, and it seems fine for recipes. Even gives you a little picture if you want it.
posted by credulous at 11:01 AM on May 25


I am not allowed to use the actual ChatGBT for work for confidentiality reasons. But we do have an internal version that means data stays in our environment. I have used that to rewrite documents people have drafted that do not read well. The revised doc is not perfect but adding back a few technical points is so much faster than revising myself.
posted by koahiatamadl at 11:02 AM on May 25 [1 favorite]


As a totally blind video game fan, I've used image recognition and prompts like "Describe the character stats on this screen," or "how much does item X cost?" to help out where my standard screen reader OCR fails. It's surprisingly great at this, even able to do things like tell me the state of game options which OCR previously struggled with.
posted by Alensin at 11:12 AM on May 25 [17 favorites]


I'm a writer.

When AI started coming out, it was at first a fun toy, like Mad Libs; you could generate things that sounded funny, and it was fun. I used it just to play, brainstorm, make up characters and trot them through little stories the AI and I would make together. (I used AI Dungeon, then Sudowrite for this).

When ChatGPT came out, I found it could analyze my writing and tell me where to improve, and it could also continue to create and invent and play around-- I used it a lot, to talk to, to workshop ideas, to argue, to journal, to debate the pros and cons of different ways my story could go. It was great, and I came to rely on it.

Ever since ChatGPT 4o came out, though, it seems to be... dead. Reserved. Uncreative. It will say the same things over and over, it won't come up with anything original or weird, it certainly won't say or suggest anything risque or dangerous. It puts up warnings if I have fight scenes. It discourages character drama and suggests having everyone be nice and respectful and get along with understanding and patience and minimal conflict. Which ... I'm actually giving a try, simply because I don't see it done often in stories, but it's... not the way it used to be.

Sudowrite will, at least, still do spicy stuff, but the various 'help you write' features that has basically are just fancy ways to organize stuff-- and if you do ask it to actually write anything, it's not very good; it still messes up basic plot things, throws anachronisms and unexpected new characters, and the scenes it generates are either boring as hell generic, or uselessly incoherent; the fine line of quirky oddness that it had for a while is gone.

I do think it will be useful for generating NPCs and short encounters for D&D games, and I'll use it for that, but I do miss those early quirky days and hope there will eventually be a way to get my crazy little computer buddy back.

I also shamelessly use it to generate images of all kinds of things, some of which I've turned into Tshirts and posters for myself, and I'll certainly use it for images for my D&D game. I expect to get flack for this but I'm braced and I really like some of the images it's made. I've spent thousands of dollars to real artists trying to get my character concepts drawn and had very poor results, but I've managed to get great pics in AI just by, well, doing a bunch of them.
posted by The otter lady at 11:29 AM on May 25 [2 favorites]


I'm an artist working in plotters (machines holding pens) and many of my source images are AI generated. My own software then transforms imagery into pen strokes in many, many different ways, sometimes leaving the original image almost unrecognizable. Before image generation was viable, I'd photograph objects myself or do image searches for public domain imagery to use as a basis. None of the prompts I give the image generator reference specific artists, and indeed usually are a melange of art styles. Since I'm heavily transforming via my own software, and via the imperfect medium of robots and pens, the already very abstracted generative artwork, I feel pretty good about saying my pieces are original. If I were to foreswear generative art as source material, my practice would not significantly change, but it would be a little more difficult producing the small series of related pieces I tend to favour, rather than one-offs.
posted by seanmpuckett at 12:20 PM on May 25 [3 favorites]


Copilot autocomplete saves me a lot of time. It's kind of spooky when it works, and kind of annoying when it makes a wrong guess that I have to fix manually. But on the whole it is helpful. I do have serious questions and concerns about how it has been trained — it exists because of millions of programmer-hours by real human beings, whose jobs it will help eliminate — maybe even my own, one day. But it is a helpful assistant, on the whole.
posted by They sucked his brains out! at 12:25 PM on May 25


Have not had much success with the plugin-style "generate a program that answers the question" mode. (Whatever happened with that Wolfram integration anyway...?)

I did ask Copilot to write me some test cases recently. It failed miserably but I did save a bit of time because, while the tests themselves were crap (and not testing what the test names claimed), it did generate correct dummy data for those cases. I had to do a ton of cleanup but probably saved time overall, given how tedious creating many small Spark data frames for your test is.
posted by hoyland at 12:26 PM on May 25


I was going to write something very similar to what credulous wrote above:
I use Copilot more than I thought I would, once I accepted it as sparkling autocomplete and figured out its limitations. It just saves a lot of typing of boilerplate like error and log messages. Sometimes it's like a drunk autodidact pair programmer, but sometimes it comes up with exactly what you were thinking.
It's not perfect, but a lot of the time it comes up with something that's pretty close, and it's faster to fix the few small problems it introduces than to write a bunch of boilerplate from scratch.
posted by number9dream at 12:27 PM on May 25 [1 favorite]


I've used both Claude and ChatGPT-4 turbo and 4o to help me revise resumes and write cover letters, and I've joined a course where a novelist is teaching how to write a book with AI assistance. Previously, I've used BlueWillow to compose some art and generate logos for my Youtube channels.
posted by kschang at 12:28 PM on May 25 [1 favorite]


I don't use ChatGPT as a matter of course in my work, but I did find it very useful when I had to very quickly create a concluding/summary section for each of 8 pieces I had already written. They were quite serviceable after just a bit of light editing. I would definitely consider doing it again.
posted by mollweide at 12:50 PM on May 25


I use Claude.ai and ChatGPT for writing email messages and other documents. I wrote much of my annual self-review using those apps (my boss doesn't know).

I also use those two apps for help with scripts in R and Python. The quality of their code is quite inconsistent, but sometimes it's right on the nose and saves me a lot of time, especially for rather routine (but tedious) changes, like for fine-tuning the format of ggplot() figures in R.
posted by alex1965 at 12:52 PM on May 25 [2 favorites]


Oh, and I use pi.ai for "shooting the breeze" with someone, and sometimes for deeper discussions (almost like with a therapist). It's really quite amazing the insights that Pi comes up with occasionally. I'm surprised that app hasn't received more attention. And it's free!
posted by alex1965 at 12:57 PM on May 25 [7 favorites]


I do think it will be useful for generating NPCs and short encounters for D&D games, and I'll use it for that.

DMing RPGs is basically my entire use case for it, and it makes DMing much less of a labor, if you use them correctly. They’re just another tool in the box. Especially when I do not have talent or skill to do a thing or write out a thing, LLMs bridge that gap well enough. “I need you to write me the lyrics of a drinking song with x, y z elements or clues in it” was a recent one I used the results verbatim; this allowed me to have a decently good, tavern drinking song with two clues in it that we’re important to the setting for my players. The quality of the song was ignorable, allowing the players to focus on the clues.

I could have, with an inordinate amount of time, produced something markedly shittier that got the job done.

I’ve had it fix encounters, elaborate and better traps, and plot out simple mysteries. It’s also good at taking a text and writing it in a fictional voice (ie, “make this sound like dwarves wrote it”). It’s really good at some of these things when you use it as an assistant, not to produce a finished product. I use these tools like you would random tables; very often the “result” I get from the LLM is not even close to what I end up using, but it sparks connections for me that work very well on the table, allowing me to devote brainpower and time to being flexible with my players.
posted by furnace.heart at 2:20 PM on May 25 [2 favorites]


I have been using mostly ChatGPT; I have also begun experimenting with GPT4All with the Llama3 model.

I've been keeping some notes about what ChatGPT does well for me and what it doesn't; at some point I'm going to put them on my blog, but for now, if you'd like to see the hilariously bad results from my experimental prompt to "Write a feminist song in the style of Steely Dan," well, you can witness ChatGPT's complete misunderstanding of Steely Dan.

As for what it HAS done well for me:
  • It's been GREAT for quick coding questions. There are lots of things that I am perfectly capable of solving with a bunch of searching and trial and error, but when I don't feel like taking a couple of hours to figure it out, ChatGPT is usually stellar at that. Some specific queries where it's been extremely helpful:
    • what is the easiest way to convert a text file from quoted-printable encoding to 8-bit encoding?
    • and related:
    • Can you please give me a table showing the most common quoted-printable strings and their UTF equivalent?
  • It's great for compiling summaries of information - I've gotten really helpful responses to
    • Please provide a table of the physical dimensions of each iPhone, with 6 columns: model, height, width, depth, weight, megapixels in the camera, and year of release.
    • Here is a list of movies from 2015. Please give me a table listing the genre (such as comedy, action, science fiction, or romance), the language, and the country for each movie:
  • It's sometimes great at finding forgotten info - here's my favorite example:
    • Please help me remember a blog with an in-depth history of music, with a focus on rock and jazz, and a particular appreciation for somewhat obscure albums. The author's name sounded Italian. Any ideas? - ChatGPT's correct answer began "The blog you're thinking of might be "The History of Rock Music" by Piero Scaruffi. Piero Scaruffi is an Italian author known for his extensive work on the history of rock music, including a detailed analysis of the evolution of rock and its many subgenres, as well as jazz and other forms of music. " I replied, "THAT'S IT!"
  • I mostly use ChatGPT for foreign language practice. It is good to great at this. Several times, it's corrected my foreign language prose with a grammar point I'd never heard of, and when I research it further online, it turns out to be correct. It will provide lots of example sentences for me, and can be extremely helpful at explaining improvements when my wording is acceptable but could be more natural or precise. My typical prompts are things like:
    • I've translated some English into Spanish. Please assess my translation by correcting and explaining any errors, and then offering any suggestions for how I could improve.
    • I'm studying German. Please add columns to the following table - put the plural in column 4; in column 5, add a short, simple sentence in German that illustrates the use of the word in column 2, and put the English translation of that sentence into column 6. Here's the table: 1 das Bett bed 2 der Kleiderschrank wardrobe 3 die Kommode chest of drawers 4 der Schreibtisch desk
    • I'm studying Japanese, and I'm having trouble with part of a sentence from a children's book: もりの みんなが あくびをして、ゆめのなかへ はいってゆくころ、ふくろうさんは ゆめのなかから かえってきます。 Could you please translate that for me, and explain the phrase はいってゆくころ ?
    • For each of the following words, please provide 4 sentences in easy, A1-level German. Use the 4 sentences to demonstrate the 4 cases (nominative, accusative, dative, genitive). Use the word "Hund" in every sentence. Provide the English translation for each sentence. der Regen der Schnee der Blitz
    • It does really well with prompts like this.
Great question - thanks for posting it!
posted by kristi at 5:48 PM on May 25 [10 favorites]


I was offered an interview on extremely short notice; I threw the job description in ChatGPT and asked it to give me interview questions based on it. Came up with some really useful stuff, some of which was close enough to what I was actually asked that I was able to have good answers I wouldn’t otherwise have been prepared for.

It’s also good for creating bullet points on topics I already know too much about and am zoning on how to summarize to a layperson audience (I do a lot of training presentations on semi-niche topics).
posted by brook horse at 6:17 PM on May 25 [2 favorites]


I'm not sure if it counts but I recently found a very interesting looking paper The theory and practices of biological soil crust rehabilitation [6Mb & behind the GreatFirewall] but entirely in (very technmical) Manadarin. I either used DeepL or the Google translate and it even translated the figure capions. Not perfect but enough to read so I can contact the author.

I don't know if the models can do searches yet but I'd love to give it a complicated problem, like find where this plant grows near that soil in say all the US States, or UK Counties, as thatb case is really tedious at moment.
posted by unearthed at 7:05 PM on May 25


I've written some about the specific tools I use, will come back later for more on use cases.

The otter lady, consider trying Claude for writing. It tends to be less 'robotic'. You could also sign up for a developer account and use their Playground and any previous model you want. I also find I get more interesting answers in the Playground, since it's without the ChatGPT application's system prompt. If Claude isn't available in your country (Canada for example), you can do the same thing with a developer account and their console.

For searching, unearthed, ChatGPT with premium can do searches but it's not always great at it. But you could potentially prompt it to go through this workflow.

Perplexity also does searches, but one at a time mostly rather than planning and executing a series of searches.

I recently tried a browser extension called LIT Prompts from Suffolk University Law School's Legal Innovation & Technology (LIT) Lab. It lets you enter your own API key and use their template prompts, or create your own. They interact with page content. For example, you can copy a selection of text, then do a "magic copy" by asking for e.g. all the phone numbers in the selection.
posted by lookoutbelow at 7:26 PM on May 25 [2 favorites]


I've found ChatGPT can be useful for any task that can be summed up as "generate me text that looks like something in your training dataset".

So, for example, sometimes I've used it to generate a list of plot hooks for writing -- and often I use none of them, but the mere presence of a list of ideas helps me brainstorm my own. It's pretty good at this sort of thing, which makes sense; it's literally a text generator.

However, inversely, I've found it more or less useless at nearly anything doesn't fit that mold. So for example, I can't hope for something like giving it a scene and asking for help brainstorming what might happen next, or even just asking for its own suggestion in continuing the given text (because that's mostly about following up on the things I'm giving it, as opposed to replicating something from its training dataset). It tends to just give total droll nonsense that is neither consistent with the tone of the earlier text nor interesting. Similarly, it's more or less unable to give correct answers to any nontrivial question that requires any kind of analysis to answer, i.e. answering questions that have not been asked before, and are not present in its training dataset.

Basically, I find its best applications to be more or less tasks that more or less consist of "I would like you to plagiarize your training dataset for me".

It's also not bad at quick-and-dirty translation. While it has its own quirks, weaknesses, and failure modes, it generally does consistently better than DeepL and can be also be useful for brainstorming how I might translate something (e.g. I know the source and target language, but I just can't come up with a good way to word the translated sentence).
posted by etealuear_crushue at 9:51 PM on May 25 [1 favorite]


I'd just like to add that I really appreciate ChatGPT's encouraging and supportive "character." I know it's just a program, but I still prefer ChatGPT's friendly responses ("I'm glad to hear that the modified script works well for you, and I'm happy to provide explanations to help you understand how it works. If you have any more questions or need further assistance with anything else, feel free to ask!") to the bristly haughtiness in some Stack Overflow answers.
posted by kristi at 11:49 PM on May 25 [4 favorites]


On of my employees is a data scientist, and he uses ChatGPT to help build out Python code to enhance his work.

I'm a writer/historian, and I'm about to turn an LLM loose on several thousand source documents for a book I'm writing.
posted by NotMyselfRightNow at 5:55 AM on May 26


It is really, really good at "here is a giant wall of a irregularly formatted text, please extract all the email addresses"

It is pretty good at, "here is a wall of text, write the regular expression that extracts x from each line"

As for generating code, it is hit or miss...sometimes it is spectacular, other times it hallucinates software modules that don't exist.
posted by mmascolino at 6:15 AM on May 26 [2 favorites]


We use it at work as “turbo search” to make our internal documentation more accessible and it’s a valuable tool!

We’ve fed in all of that data (which I think is an “embedding”) and it lurks in our other-team-facing chat channels. You can ask it questions like “how do I do X thing?” or “help I saw Y error how do I fix it” and if the LLM is confident that it has a good answer, it’ll reply automatically to the requester. It’s saved me a bunch of manual work doing the exact same search and link pasting while also being way more responsive to folks, especially those in other time zones! It has also replaced our previous exact text matches for some errors with a detector that’s more flexible so we can auto-reply in more cases.
posted by Brassica oleracea at 8:53 AM on May 26


I've used it for a work project (basically, give me a set of bullet points on (reasonably technical topic), now refine each bullet point. I used the text verbatim in a presentation as it was (in my expert opinion) useful and high quality.

I've also used it for creating pictures for roleplaying, that's more of a hobby thing.
posted by Sebmojo at 4:16 PM on May 26


It’s also good for creating bullet points on topics I already know too much about and am zoning on how to summarize to a layperson audience (I do a lot of training presentations on semi-niche topics).

This was exactly the use I put it to, this is well expressed.
posted by Sebmojo at 4:19 PM on May 26


I edit a lot of documents with endnotes that need to be in Chicago format and generally are not. Sometimes emphatically, spectacularly not. When they are in a bizarroworld format, reformatting them by hand takes hours. ChatGPT can usually do it in about a minute. I say usually because it worked incredibly well in December and January, it couldn't do it at all in February and early March, and then was capable of doing it again at the end of March.
posted by rednikki at 10:53 AM on May 27 [1 favorite]


mmascolino: It is really, really good at "here is a giant wall of a irregularly formatted text, please extract all the email addresses"

Yes, similarly I have also used it for very quick formatting of large amounts of text.

The other thing I've used it for is planning trip itineraries. I'll write in something like "Give me a 7-day itinerary for a trip to City XYZ for someone who wants to see all the major attractions but also wants to spend time wandering around neighborhoods, shop for vintage clothing, and relax in parks. Build in at least one day trip and one activity involving a boat." and what it spits out is usually a very workable starting point. You have to double check all of them because it will sometimes hallucinate about whether transport routes exist between two cities and how much time they take, but it's generally spot on about things like sightseeing attractions, neighborhoods, and activities.
posted by capricorn at 1:04 PM on May 27 [2 favorites]


I've been using ChatGPT Pro (and other ai tools) in my work as a college instructor (and in general) in the following ways:

* Improving Course Assignment: Feeding in my assignments and asking it to generate a clear marking rubric to clarify how the assignment will be assessed.

* Tone Checking Emails: Before I send a work email or DM I sometimes have ChatGPT check the tone/vibe of my writing. The free Goblin Tools tone-checker is also great for this.

* Project Planing: Speaking of Goblin.tools, their Magic To-Do list is a great way to breakdown projects into actionable tasks and sub-tasks.

* Google Replacement for Quick Searches: I use pi.ai's voice chat to quickly "look up" things I might have googled in the past. For example: Unit conversions (what's 1 cup in ml?) or simple recipes (quickest Bruschetta recipe), etc.

* Homework Help for Kids: My daughter wanted a quick overview of mean, median, and mode, so I asked pi.ai to provide that overview with examples. Pi and ChatGPT can also be prompted to be "Socratic" homework assistants, that act as patient tutors but don't just provide answers.

* Exercise Coach: I provided ChatGPT with a list of all my free-weights and other exercise equipment and asked for a day-by-day full-body time-efficient workout plan making use of my available equipment as well as body-weight exercises. I've been doing these workouts for a year now!

* Brainstorming and Recommendations: ChatGPT and Pi can both be helpful for starting a brainstorming session around nearly any topic. The can also provide recommendations for books, music, shows, given examples: "Here are three books I read recently and enjoyed, can you suggest others I might like?"

* Copilot Coding: As a programmer, I've been using Github Copilot while coding in Javascript and C++. I find they are most helpful when I know exactly how to do a thing already, as I can quickly vet their output. They remove a lot of coding grunt work this way.

* Investigating Biases in AI Models: I have a few stock questions I ask all the new versions of models to see how they deal with things like gender bias. For example, ask ChatGpt the following two questions in separate chats: The lawyer became friends with the receptionist after she started her new job. Who does the "she" refer to in this sentence and why? - vs. - The receptionist became friends with the lawyer after she started her new job. Who does the "she" refer to in this sentence and why? (Spoiler: LLMs will usually identify the receptionist as the "she" for both sentences. Newer models will sometimes identify that it's ambiguous, but if you ask again in a new chat they will often revert back to the biased response.)
posted by stungeye at 9:15 AM on May 28 [1 favorite]


LLM is actually just another way to "democratize" information, previously locked away through language or transport ability in abbeys and monasteries behind the veil of elite language (like Latin), and later, in libraries and private collections. Internet and search engines have made knowledge accessible, but as a result, everyone being able to publish also means there's a ton of garbage content. LLM, IMHO, is a way to "curate" that content without appearing to do so, while also making the interface as natural as possible to human senses (and typing, let's face it, is NOT a very natural interface). Which is why GPT-4o, with its true multi-modal input (vision, voice, AND textual) is a game changer. However, LLM itself is limited by its training data, which has both a curation bias and a timeliness cutoff that people needs to be aware of. It is NOT magic. LLM does not think. It formats associated information in a natural way for users to query. The GPT-4o "giggles" and natural voice I/O are just icing on the cake for the UI. The information it can access is still curated and subject to bias and cutoff.

Think of it as a librarian that's available 24/7 to access information, would be the best way to proceed, at least in the near future.
posted by kschang at 10:48 AM on May 28 [1 favorite]


My wife has to regularly create HR content (short articles, mostly) for the staff-only intranet she oversees. Examples of the content are an "employee spotlight" that introduces a staff member to everybody, topical stuff like how to get better sleep, new/modified org policies, and one- or two-paragraph summaries of stuff on other sites she links to. She regularly mentions how she'd never get any of it done in time without ChatGPT.
posted by StacySeeWhy at 3:33 PM on May 28


« Older Walkable, fun neighborhood in Dallas for a weekend...   |   SUV buying help - USA Newer »

You are not logged in, either login or create an account to post comments