What’s a quick and dirty way to train LLM on existing datasets?

March 22, 2024 8:26 AM

At work I often get varied requests with tight deadlines. As in “get this case study done today.” Or “create a presentation on this.” We usually have a large corpus of material that follow a similar pattern and language. I know how to take a model and train a dataset, but is there a way I can skip that upload 300 documents, write a one pager outline and prompt it to make it sound like the rest?

I could go months without doing anything like this or get a bunch of requests on a Friday for something that is essentially marketing material maybe one person will read. To be honest these things all look AI generated. I’d rather do a one page outline and say “make it sound like this” more or less. There’s only so many ways I can say transforming a business to deliver on sustainable customer needs before I go crazy. Anyone have a good trick? If I upload a couple of examples to ChatGPT and ask it to transform my outline is that good enough? I’m sure I’m not the first to do this. I have no qualms being honest what I’m doing to anyone I just don’t want to fool around with creating a dataset, training, then have it not work.

posted by geoff. to Technology (10 answers total)

Wouldn't a template or an editable master doc (throw in all the verbiage ever, edit down a copy as needed) be a vastly simpler fix?
posted by humbug at 8:40 AM on March 22 [1 favorite]

Given that there is time pressure, you likely won't have time to check it carefully. This is a recipe for potential disaster. You only need to have it hallucinate once.
posted by praemunire at 8:42 AM on March 22 [10 favorites]

AFAIK you don’t need to specifically train LLMs to do this sort of thing, you can throw a new corpus of input text at an existing model and have it summarize/rephrase it for you. Paid ChatGPT-3 would probably do a decent job of it.
posted by Jon Mitchell at 9:03 AM on March 22

I have fiddled around with what I think you're asking for by doing something like this with ChatGPT:

Please take the below information and convert it into the format specified.

INFORMATION:
Nvidia -
Revenue Increase -- US$60.92 billion (FY 2024)
Operating income -- Increase US$32.97 billion (FY 2024)
Net income -- Increase US$29.76 billion (FY 2024)
Total assets -- Increase US$65.73 billion (FY 2024)
Total equity -- Increase US$42.98 billion (FY 2024)
Number of employees -- 29,600 (FY 2024)

FORMAT:
In the final quarter of 2023, Alphabet Inc. demonstrated impressive growth in its financial metrics. Revenue jumped to $86.31 billion during this period, reflecting a 13% surge compared to the same quarter in the preceding year. Despite economic fluctuations, Alphabet's adept revenue generation strategies continued to drive significant growth. Furthermore, the company's total revenue for the year reached $307 billion, representing a solid 9% increase from 2022. Additionally, Alphabet's operating income witnessed a notable rise, reaching $23.7 billion, indicative of efficient operational management and strategic financial planning.

Getting an output like:

In the fiscal year 2024, Nvidia showcased remarkable financial performance across various metrics. The company's revenue surged to an impressive US$60.92 billion, marking a substantial increase from the previous fiscal year. Operating income also experienced a notable uptick, reaching US$32.97 billion, reflecting strong operational efficiency and strategic initiatives. Furthermore, Nvidia's net income saw a significant boost, climbing to US$29.76 billion. This impressive growth trajectory is indicative of Nvidia's robust market position and effective financial management strategies. Moreover, the company's total assets expanded to US$65.73 billion, underlining its growing economic footprint. Additionally, total equity reached US$42.98 billion, highlighting sustained value creation for stakeholders. Amidst this growth, Nvidia maintained a workforce of 29,600 employees, further reinforcing its commitment to talent retention and organizational development.

Data copied and pasted from the Nvidia wiki page & paragraphs from a Nasdaq page on Google.
posted by gregr at 9:05 AM on March 22

I think the issue, if I understanding it correctly, is that the asker wants the corpus of text inform style/tone ("make it sound like this") but not content. The one-pager outline is the actual content to be transformed into a ppt or case study or what have you.

I have found LLMs to be okay at changing style like "make this sound professional" or "can you make this sound more informal" or change format "put this into narrative format" but if my understanding of the request is correct, I don't know that it will be easy to train an LLM on tone. But it depends exactly what you mean by this question.
posted by misskaz at 9:07 AM on March 22

Oh, and I would make sure if there are numbers in the input information, that they are in a unit that makes sense "$60.92 billion" and not "60,922 (mil. US$)", so you're not relying on ChatGPT to do conversion or math. Similarly, scan through the output, looking for percent signs or numbers that you didn't give it. I wouldn't trust that ChatGPT will do any math correctly.
posted by gregr at 9:14 AM on March 22

This sounds like a use case for a custom GPT. You define how you want it to behave with some instructions, upload up to 20 documents to act as data sources/examples, and then you can start querying it.

See https://chat.openai.com/gpts
posted by SNACKeR at 9:43 AM on March 22

Yes, you can absolutely do this, I did this with ChatGPT4 ~9ish months ago and it worked then; i would assume it's probably improved since then. I used something like "you are a professional [role title here] who writes [whatever it is you write]. I will give you an outline and several examples of previous work, and I would like you to create a new doc based off the outline, using the same style and tone as my previous work. Do you understand?" (Apparently the do you understand is important here).

Then ChatGPT will usually re-iterate the request you gave it, and say something like "I am waiting for your outline and examples". After which I'd reply "here are some previous examples": [paste here] and then "here is the outline for this new document: [paste]"

It should spit out something quasi usable. If you don't like it you can always say "please re-write paragraph 2 so that it is [whatever it should be like instead, or whatever changes you want]". Or you can always ask ChatGPT in the original ask to give you 2 or 3 drafts so you can pick the one you like.

There are also a ton of examples of ChatGPT prompts for many use cases available if you start googling around, you might be able to find something that works even better.
posted by cgg at 9:55 AM on March 22 [2 favorites]

Response by poster: Yeah so what I needed was the tone and maybe some filler content since the request was so vague. This felt like “we don’t know what we need” kind of vague request.

I took about 5 case studies and put them in a text file and attached them to ChatGPT4. I edited the source files to remove quotes and make it sound like it was one paragraph. I removed numbers.

Then I wrote the case study in plain english and told it to use the tone of the attached files. Another 15 minuteS of asking it to rephrase some sentences or paragraphs. Literally “rephrase this.” Then 30 minutes of manually editing it.

Turned out really well but not at all what they wanted. In fact it turned out so well I’m impressed. I didn’t have to think about changing it to consulting speak basically.

That wasn’t at all what my boss wanted but I figured he didn’t know what he wanted. It took me maybe 1.5 hours total and if I did it without ChatGPT I’d be thinking about using phrases like increasing organizational transparency.

He canned the idea but I’m glad I didn’t spend all day on it. Nothing to do with the quality just didn’t know what was wanted.

I might create a model for everything I’ve been asked to do over the year since they’re all over the place and people are bad at explaining what they want. Fill it with data get a first draft out quickly and basically rewrite if it gets accepted. I’m finding LLMs are good at feeling out things. I don’t know about most people but my job is vague and I’ll get odd requests. I have no problem writing things out but crafting it to the company tone is difficult and this helped a lot.

Thanks all!
posted by geoff. at 2:24 PM on March 22 [3 favorites]

"I’m finding LLMs are good at feeling out things. I don’t know about most people but my job is vague and I’ll get odd requests." This is absolutely a great approach to LLMs! I wish more people saw it that way rather than "do my work for me."
posted by Mo Nickels at 7:24 AM on March 23