Generative AI for design students

December 9, 2024 2:46 AM

I'm putting together some material for my Graphic Design students about using generative AI. I've found some good basic explainers about LLMs as in this video by Grant Sanderson. Is there a similar video or article that explains how Image generating AI like MidJourney works? That's suitable for not-very-computer-literate students?

I would also like to broaden their understanding of the ethical implications of using Generative AI including:

Impact on the environment. Is the power usage of AI a concern because "if we create a demand by using it, there will be more of it" or does creating an image with Generative AI use a lot of power? For example, if my students are using MidJourney to create a design, does that use a lot of electricity?

I have a vague idea that there is an aspect of exploitation of labour, by rich countries of people living in poor countries involved in generative AI. Is this the case?

I've seen stories about artists' work being scraped without their consent, particularly by Adobe. Where can I learn more about this?

posted by Zumbador to Technology (12 answers total)

Here is a semi-technical but not super math heavy video on how diffusion based image generators work:

https://www.youtube.com/watch?v=1CIpzeNxIhU

Impact on the environment. Is the power usage of AI a concern because "if we create a demand by using it, there will be more of it" or does creating an image with Generative AI use a lot of power? For example, if my students are using MidJourney to create a design, does that use a lot of electricity?

Energy usage varies by model, but since you can run Stable Diffusion locally on an RTX 3070, that at least isn't using more energy than playing an AAA video game for the same amount of time. It's the training that uses a lot of energy. So it's more the former than the latter.

I have a vague idea that there is an aspect of exploitation of labour, by rich countries of people living in poor countries involved in generative AI. Is this the case?

Not that I'm aware of.

I've seen stories about artists' work being scraped without their consent, particularly by Adobe. Where can I learn more about this?

If anything, Adobe is the one player who isn't doing this, or at least doing this the least. They are doing all their training on stock images they own the copyrights for. Other generators are training their models on image data pulled from all publicly available sources, i.e., the Internet.
posted by justkevin at 5:42 AM on December 9, 2024 [2 favorites]

Regarding Adobe, you might be thinking of the recent furor over their them updating their terms of use in a way that seemed to imply they were giving themselves many more rights to users' work than before, including for training generative AI models. They claim this isn't the case and are updating the wording.

The important thing is that this isn't about Adobe - this happened when artists were already upset about their work being used to train other major generative AI models. There are two major concerns for artists specifically:

(a) These generators take artwork without permission or compensation to train their models

(b) And then these artists can't get work because people use images produced by these generators instead of hiring artists

There is a tremendous, massive amount of hatred for these generators among the artist community; many feel that they are stealing their work and then stealing their livelihood. It hasn't helped that some executives at some of these companies have said as much re: stealing their livelihood, that these artists should be put out of work.

I don't have specific references for you to read about this as most of this has been gathered by being in communities of artists online. I'm just mentioning it because I think there's a disconnect, that the general public doesn't know just how angry and scared a lot of illustrators are because of AI image generation. It's definitely something you should look into so you can discuss with your students and isn't specific to Adobe.
posted by Kutsuwamushi at 6:05 AM on December 9, 2024 [5 favorites]

Besides electricity, there is the issue of AI's water consumption. That link goes to an article written out of a university center, and it like pretty much all other reporting I've seen on this admits that it's really hard to know exactly how bad the problem is currently or how much it might improve once the models get more efficient. But, it is a negative factor that I'd least introduce to your students.
posted by coffeecat at 6:16 AM on December 9, 2024 [1 favorite]

Yes, my understanding is that the foundation of all image generation AI was established by scraping the web for any and all available images to train on before anyone had any notion that they might have to protect their artwork from such a thing. Then those images are remixed to produce instant on-demand "original" art that provides a tempting alternative to paying an artist for commercial projects.
posted by space snail at 6:18 AM on December 9, 2024

It's not specific to image generation but The Conversation just posted a new article about energy consumption by data centers related to AI.

Concerns about exploitation of third world countries may be related to the parts of the processes where humans are required to either filter data sets or train the tools by providing feedback that is used to fine tune the models or tools built on top of the models. For examples, people may be needed to view potentially problematic images (e.g., violence, pornography) so they can be removed from the training data.
posted by ElKevbo at 6:43 AM on December 9, 2024 [2 favorites]

I have a vague idea that there is an aspect of exploitation of labour, by rich countries of people living in poor countries involved in generative AI. Is this the case?

‘It’s destroyed me completely’: Kenyan moderators decry toll of training of AI models. "Employees describe the psychological trauma of reading and viewing graphic content, low pay and abrupt dismissals"

These are approximately the same people that companies like Facebook employ to perform moderation on their platforms, so it's not exactly novel to AI.
posted by BungaDunga at 8:17 AM on December 9, 2024 [5 favorites]

the water use thing may be a bit overblown:

This puts the Post piece in some better context. They calculate that training Microsoft’s GPT-3 required 700,000 liters of water—or just over half of one acre-foot. If 10 percent of working Americans were to use GPT-4 every week, that would require 435,235,476 liters per year—or about 353 acre-feet, or about 0.003 percent what Nebraskan farmers use. Or looking through the other end of the telescope, diverting just 10 percent of Nebraskan irrigation water would be enough for every single American to use GPT-4 about 24 times per day.

posted by BungaDunga at 8:24 AM on December 9, 2024 [1 favorite]

The shoggoth and blurry JPEG metaphors remain the best way to wrap a layperson's head around this class of technology. (They ostensibly focus on text generation but the insight is portable.) Also possibly appropriate for your audience is the article The Dark Forest and Generative AI.

This answer adapted from previously.
posted by daveliepmann at 8:42 AM on December 9, 2024 [4 favorites]

After Google fired the co-lead of their Ethical AI team, Timnit Gebru, she founded the Distributed AI Research (DAIR) Institute to address many of the questions you're asking. Most of these resources are either work by DAIR or stuff I found via their website, podcast, and so on:

• Zombie Trainers and a New Era of Forced Labor is a fantastic overview of the labour exploitation involved in training AI:

  "We need to question how data collection, and other AI labor, is happening in the first place. Lack of transparency around data collection, and the creation of AI systems at large, has given way to an invisible form of forced labor. A new type of "Zombie Trainer" has emerged. These are people who work as data labelers, content moderators, or image data collectors without their knowledge. Captive audiences, like refugees, children, prisoners, and low wage workers are all Zombie Trainers, unaware of the hidden tasks they perform, the new industries they're building, or the communities being harmed in the process. To be sure, corporate leaders will say that Zombie Trainers are not unpaid laborers because they are children or prisoners and therefore are not entitled to payment, or they are already being paid for the jobs they were hired to perform."

  It also contains links to other articles:
  • Refugees help power machine learning advances at Microsoft, Facebook, and Amazon:

    "The same economy of clicks determines the fates of refugees across the Middle East. Forced to adapt their sleeping patterns to meet the needs of firms on the other side of the planet and in different time zones, the largely Syrian population of Lebanon’s Shatila camp forgo their dreams to serve those of distant capitalists. Their nights are spent labeling footage of urban areas — house,” “shop,” “car” — labels that, in a grim twist of fate, map the streets where the labelers once lived, perhaps for automated drone systems that will later drop their payloads on those very same streets. The sites on which they labor are so opaque that it is impossible to establish with any certainty the precise purpose or beneficiaries of their work."

  • Millions of Workers Are Training AI Models for Pennies


• The Data Workers' Inquiry invites data workers— the workers that help produce data for intelligent systems including AI— to lead inquiries into their respective workplaces. Examples of their work:
  • "If I Had Another Job, I Would Not Accept Data Annotation Tasks": How Syrian Refugees in Lebanon Train AI (PDF Link)
  • Impact of Remotasks’ Closure on Kenyan Workers, includes audio interviews with Kenyan data labelers.
• AI Art and its Impact on Artists (PDF Link) is a good article that has some technical jargon, but check out its "Impact on Artists" section:

  "The proliferation of image generators poses a number of harms to artists, chief among them being economic loss due to corporations aiming to automate them away. In this section, we summarize some of these harms, including the impact of artists’ styles being mimicked without their consent, and in some cases, used for nefarious purposes. We close with a discussion of how image generators stand to perpetuate hegemonic views and stereotyping in the creative world, and the chilling effects of these technologies on artists as well as overall cultural production and consumption."

• Mystery AI Hype Theatre 3000 (PeerTube Video) is the DAIR podcast where "linguist Emily M. Bender and sociologist Alex Hanna break down the AI hype, separate fact from fiction, and science from bloviation. They're joined by special guests and talk about everything, from machine consciousness to science fiction, to political economy to art made by machines."
  • Episode 4: Is AI Art Actually 'Art'? (October 2022) (PeerTube Video)
  • Episode 43: AI Companies Gamble with Everyone's Planet (feat. Paris Marx) (October 2024) (PeerTube Video) mentions other articles:
    • Google's former CEO Eric Schmidt says they should abandon climate goals and chase AI.
  
  
  • Episode 19: The Murky Climate and Environmental Impact of Large Language Models (November 2023) (PeerTube Video)
    • The 'invisible' materiality of information technology (May 2020) (PDF Link):

      "THERE ARE SIGNIFICANT material impacts from extracting, processing, maintaining, and ultimately disposing of the materials used to support information technology, as well as from producing the energy used both by the devices in operation, as well as in their production and disposal. Yet these material impacts are largely invisible and receive substantially less attention than discussions about the technical aspects and benefits of information technology. We use the term materiality to encompass all of these aspects and more—a comprehensive accounting of the ways in which information technology impinges on the physical world."

      (Not directly about AI, but I think a useful framework for understanding the material impact of technology.)
    • TED Talk: AI is dangerous, but not for the reasons you think:

      "That cloud that AI models live on is actually made out of metal, plastic, and powered by vast amounts of energy. And each time you query an AI model, it comes with a cost to the planet. Last year, I was part of the BigScience initiative, which brought together a thousand researchers from all over the world to create Bloom, the first open large language model, like ChatGPT, but with an emphasis on ethics, transparency and consent. And the study I led that looked at Bloom's environmental impacts found that just training it used as much energy as 30 homes in a whole year and emitted 25 tons of carbon dioxide, which is like driving your car five times around the planet just so somebody can use this model to tell a knock-knock joke. And this might not seem like a lot, but other similar large language models, like GPT-3, emit 20 times more carbon. But the thing is, tech companies aren't measuring this stuff. They're not disclosing it. And so this is probably only the tip of the iceberg, even if it is a melting one."

I think you have a chance to make a really dynamic and engaging class on this topic. Best of luck!
posted by ftrtts at 9:36 AM on December 9, 2024 [9 favorites]

Regarding Adobe specifically, maybe you're thinking of Adobe’s ‘Ethical’ Firefly AI Was Trained on Midjourney Images (April 2024) (Archive Link), which was discussed on MAIHT3000 Episode 33: Much Ado About 'AI' 'Deception' (PeerTube Video):

"When this first came out, I think both of us flagged that there were a lot of details that were somewhat available in the Adobe details about this system, but it was not in any way a consented system based on those details. Although there are some mechanisms for consent, it still doesn't mean that it has all of the training data agreed upon by the people producing it.

This is just another example of that. It's consent or it's lack-of-consent laundering. If there's a public domain image that was generated from non-consented data, that doesn't mean it's ethically a good thing to use. It still means that you're not employing consent mechanisms."

(It's not about content theft, but there's also this story from November 2023: Adobe Stock is Selling AI-Generated Images of the Israel-Hamas Conflict)
posted by ftrtts at 10:02 AM on December 9, 2024 [1 favorite]

I follow the work of Jo Wood [Jo's webpage] at City Uni. London who has been writing about generative landscape imagery, I found his Beyond the walled garden. A visual essay in five chapters [openreview.net - 21.3Mb .pdf] in a Twitter post Fells that never were. The peer reviewers comments add to the post!

It looks interesting but what it does not do (afaict) is produce a representation of a landscape/scene over a real contour model base - at which point I lost interest, as it would seem to be only capable of pretty (but shallow) pictures (some citations of Beyond the walled garden look like they may be heading that way). If I could get it to 'paint' over real terrain I might be able to produce more variations* of a landscape approach (very useful to game out land planning and legal scenarios) - but these systems do not seem to work over a real underlying reality.

But here's the rub - if a client paid me I could produce more iterations from my own creativity (but each iteration is probably $3000 if I do it using my brain and training). What CLIP is doing is stealing other's prior creativity making our 'creations' less costly if using CLIP (or any other of these systems), and impoverishing creators (and curator/owners) of by taking their 'inspiration' with no payment. This is unlicensed use of an artist's creation and is illegal - apparently not so when it is 'diffuse' and large companies are doing the stealing...

Jo's Beyond the walled garden does cover how this works to a useful level, unfortunately the nuts and bolts of what is really going on is here DiffusionCLIP *: Text-guided image manipulation using diffusion models [openreview.net - 8Mb .pdf] which may as well be Greek except for the frequent occurrence of the word sampling though the text - the only way these systems can be built is to sample millions of images of human-created artwork.

* Contrastive Language-Image Pretraining (CLIP) with diffusion
posted by unearthed at 11:02 AM on December 9, 2024 [2 favorites]

which may as well be Greek except for the frequent occurrence of the word sampling though the text - the only way these systems can be built is to sample millions of images of human-created artwork

the models themselves certainly can't exist without being fed millions of real images.

However, that's not what "sampling" in that paper is referring to. They have a pre-trained model that's essentially static, a file within which the information about the images is embedded (this is the outcome of training), smeared across several gigabytes of data. When you sample from a model you are just kind of running it and seeing what it spits out, not pulling up the original images. These model files are much smaller than the datasets they're trained on so they rarely encode a copy of any single input image.
posted by BungaDunga at 11:55 AM on December 9, 2024