Are we here yet with AI?

January 28, 2023 6:33 AM

After playing with ChatGPT a bit like everyone else, I'm wondering about use applications. I'd like to point an app at a folder of documents and have the ai be able to answer questions about the text. Is this currently possible without programming and ai training knowledge, etc? Is it possible at all?
posted by dave78981 to Computers & Internet (7 answers total)

I have tried to use GPT for grant applications. The big problem is that it tries to be an editor, not an expander. GPT doesn't seem to learn from the addition of information, so it's not able to extrapolate on points. I'm working on finding a series of comparable grant applications I can do a GPT/human test case on.
posted by parmanparman at 9:11 AM on January 28, 2023

There are some resident MeFites with more domain knowledge here, but I’ve dabbled with NLP enough to give an educated layperson’s response:

We probably need more info about what kind of questions you’re asking about the text. Like, you don’t need anything as complex as machine learning to tell you the frequency of each word in the documents, and there are established mathematical models for finding patterns/tuples that can help with more complex questions, out of which many of the more complex and sophisticated models have sprung. But in terms of actually getting meaningful synthesis or summarizing of information, this is a much harder question than it sounds. Lots of “AI” is able to give the appearance of doing this. For some purposes that’s plenty. For other purposes there are surmountable qualitative factors that mean a machine learning answer will get you halfway there, or save you time. For still others, we’re still not there and may never get there.
posted by aspersioncast at 9:56 AM on January 28, 2023 [3 favorites]

I'm learning Wolfram Mathematica. One of the problems for a new user is that there are relatively fewer users of this software compared to say Matlab or R. So if I have some problem, I rarely find the answer by googling, and sites such as stackexchange or the wolfram community page are of help only occasionally. I tried using ChatGPT to generate some mathematica code, and while I know that the code itself cannot be trusted, it often points me in the right direction because it uses functions that I hadn't come across before. So I'm thinking of using it as a brainstorming tool.
posted by dhruva at 11:00 AM on January 28, 2023

Some technical people have been tinkering with using GPT-3 this way but I don't think it's been boiled down into a product or application yet.
posted by BungaDunga at 1:26 PM on January 28, 2023 [1 favorite]

This class of models does what is called "in context learning" like what you're asking (answers to prompts will change based on contextual text). It is analogous to subtly changing the model in response to the inputs. Yes, you can ask it to summarize or do other things with existing text. Depending on what you're asking, it might not do much, example

Can you re-write this story in the style of hunter s thompson?
In olden times when wishing still helped one, there lived a king whose daughters were all beautiful, but the youngest was so beautiful that the sun itself, which has seen so much, was astonished whenever it shone in her face. Close by the king's castle lay a great dark forest, and under an old lime-tree in the forest was a well, and when the day was very warm, the king's child went out into the forest and sat down by the side of the cool fountain, and when she was bored she took a golden ball, and threw it up on high and caught it, and this ball was her favorite plaything.

Gives

In the wild, drugged-out days of yore, when the act of wishing still held some semblance of power, there roamed a king. A king with daughters, all of them beautiful in their own right. But his youngest, ah, she was a sight to behold. The sun, that old sonuvabitch, who's seen it all in his time, would stop and stare in disbelief every time she'd step out into the light.

There was a forest, dark and ominous, that lay just beyond the castle walls. And in that forest, beneath the gnarled branches of a lime tree, there was a well. When the days got hot as hell, the king's little princess would venture out into the forest to escape the heat and sit by the side of that cool, refreshing well. And when the boredom set in, she'd take her golden ball and toss it up into the sky, catching it with the grace of a true champion. That ball was her everything, her ticket to a world of fun and adventure.

I can also ask follow up questions, that are specific to the text

Did the king have sons or daughters?
In the original story it is stated that the king had daughters.

If you have a specific thing that you want it to figure out, and you can give examples before the unknown, that often helps. No retraining. There are multiple companies in this space, and we will likely see consumer facing products very shortly. An advantage the OpenAI has comes from hiring people to interact with the model to re-train it, but that is something that can be done very quickly if you have desperate VC money to burn.

There is a great deal of work that will go into making more automatic applications, because it takes some prompting to get it to use "internal logic" that will get it to the right answer. Example:

Did the king live in a desert?
It is not specified in the original story that the king lived in a desert. It is mentioned that there was a great dark forest close to the king's castle and that the princess went to a cool fountain in the forest when she was bored.
Are forests often next to deserts?
Forests are not typically found next to deserts, as they have different climate and soil requirements. Forests typically require a lot of rainfall and cool temperatures, while deserts have very little rainfall and high temperatures. However, in some specific regions, such as Mediterranean, there are areas that are characterized by a mix of forest and desert-like vegetation, called Mediterranean forests, woodlands, and scrub.
So is it likely that the king lived in a desert?
It is not likely that the king lived in a desert based on the description provided in the original story, which mention of a great dark forest close to the king's castle, and a cool fountain where the princess went to escape the heat. Deserts are characterized by very little rainfall and high temperatures, which is not consistent with the presence of a forest and a cool fountain.

posted by a robot made out of meat at 1:32 PM on January 28, 2023 [2 favorites]

People are definitely using GPT-3 to provide a natural language interface to reference documentation, but I believe right now you'd need to write code.

Here are three examples I've seen:

* How to implement Q&A against your documentation with GPT3, embeddings and Datasette by Simon Willison
* Ask My Book
* HoustonAI

Also, Simon Willison's article has some notes about the effectiveness of this -- as you might expect, the answers aren't always accurate.
posted by oranger at 2:30 PM on January 28, 2023 [2 favorites]

Someone just sent me this link a couple of weeks ago — a searchable database of AI tools. I haven't looked through it carefully yet but maybe it has something of interest!
https://www.futuretools.io/?fbclid=IwAR18BqTIXGq9MeYM2qz1NIcoixwUg2Z067Outg98b5RFIjhI9Ot1J3BRFzE
posted by lavenderflowers at 6:24 PM on January 28, 2023