AI Scams & Shams: Scaling the wrong ideas

AI enables people to launch ventures more easily. Many are shoddy businesses, scams, & low value enterprises. Let's look at a few.

A lot of people ask What can AI do? I prefer to ask What can people do with AI?

And the answer is people can do a lot more, faster, and with less resources.

I’ve written about the positive potential of generative AI, especially in filmmaking. Today I’m writing about the negative potential. Specifically how AI can multiply small acts of short-sighted greed.

Most ventures require resources– people, equipment, time, money. And getting resources requires a reputation, some track record of pro-social behavior and past success. These requirements often disqualify people on the basis of character or circumstance— shy people, non-native speakers, misunderstood eccentrics have all historically failed to get a shot. But these requirements also barred scammers and lazy opportunists. As much as gatekeepers are the enemies of progress, they do a decent job filtering out nutjobs and con artists.

Generative AI is exciting because it’s a back door. While making a movie traditionally required a Kubrickian ego to rally the support of people and institutions, that’s no longer the case. Now everyone with a computer gets a shot.

But everyone means everyone. Including people with bad ideas. Everyone gets a digital workforce to carry out their whims. Just as online scams were a major subplot of the internet, scaling bad ideas will be major subplot of the generative AI boom.

Stanley Kubrick spent hours on the phone cajoling studio executives to get his movies made. Now he’d spend that time prompting models.

Projects of Dubious Value

So what are people doing with AI? They’re launching enterprises all the time, all over the world, in every field. People are writing books, building AI assistants, creating comics, launching services, and more.

I observe the activity in this space closely. I enable a lot too through my work. But not all of it is well thought-out. I’ve seen a handful of what could generously be called low-effort projects of dubious value in pursuit of profit. Other people might call these “scams”. I don’t think that quite captures what’s going on.

Whatever we call them, these projects are clarifying some accelerating trends for me. The macro trend is that AI lets people execute on ideas with minimal outside help. It’s exciting when the project is a film or business. Less exciting when it’s an ill-advised cash grab. 

Here are a few patterns I’ve noticed.

• AI can make stuff really easily

• AI can act instantly

• AI makes high-effort, low-return work worthwhile

• AI lowers the value of images and muddies their signal

• People associate ‘polished images’ with ‘legitimacy’ and this is a vulnerability.

• AI entrepreneurs underestimate the difficulty of bridging digital success into the physical world.

Today I want to write about these by detailing three, shall we say, Projects of Dubious Value and Minimal Effort. Each involves someone executing quickly on a poor idea to make money.

AI Authorship & Knockoffs

First let’s talk about AI-generated books. Many writers are pissed about this. They’ve spent a lifetime building up a professional ‘moat’ of expertise. But suddenly strangers can recycle their knowledge and generate books in less time than it takes to cook a frozen pizza. This topic is being debated all over the internet. Generally, I don’t think it’s a big problem until you combine it with SEO hijacking and mild identity theft.

My uncle Ted Gioia writes books about jazz. He’s popular enough that algorithms probably recommend his books to people who like jazz. And people probably google phrases like ‘Gioia jazz book’. He’s been writing books for 30+ years now, and been reading, researching and listening for his entire life.

Then Frank Gioia started writing books about jazz. At an astonishing rate. And with similar cover art and titles. He was even co-authoring with Ted Alkyer (who sounds very similar to another established jazz writer, Frank Alkyer).

Ted’s “The History of Jazz” on the left. The AI knockoff on the right.

What’s going on here? Well, I can hazard a pretty good guess. 

Someone rewrote Ted’s book with AI, then published it under a slightly different title with the intention of confusing people and algorithms into mistaken purchases.

Here’s an excerpt Frank Gioia’s book: 

Feel the rhythm of "The History of Jazz Music," a soul-stirring exploration into the rich tapestry of jazz from its humble beginnings to its status as a global phenomenon. This book is an invitation to swing through the epochs of musical innovation, where improvisation meets inspiration, and every page sings with the spirit of jazz.

Despite the fact the blurb confuses the title, there are obvious GPT-isms here. Dear readers, I am warning you to be suspicious of the word “tapestry” from hereon.

This book-recycling hustle wouldn’t be worth the effort normally. Manually paraphrasing a book is a time-consuming task with low expected return. But AI makes it easy enough to be worthwhile. All Frank Gioia had to do was copy the book’s text, paste it section-by-section into an LLM, and give it instructions like “Rewrite the following book section using new words but keeping the same information.”

I suspect Ted wasn’t his only target. Frank Gioia probably programmatically scraped and rewrote hundreds of popular titles under false names. From what I can tell, Frank’s jazz book was a financial wash. It was removed from Amazon fairly quickly, though you can still see it sitting with zero reviews on WalMart. But across a couple hundred or thousand titles, Frank probably got enough hits to make his score.

Generating books with AI is an unfortunately common idea among AI impresarios. It’s not inherently evil, but it is low value if you aren’t adding substantial new inputs. And it speaks to a certain casual attitude towards quality. In the next example we’ll see what happens when the same attitude is brought to real world projects.

Candy-Coated False Advertising

Scottish AI entrepreneur Billy Coul had already published over 12 AI-generated genre novels when he decided his next project would be hosting a live ticketed Willy Wonka Wonderland event in Glasgow. He didn’t have the resources or experience to put on a live event, but he did know how to create marketing materials with AI. So that’s what he did.

And it worked! Flocks of parents bought $35 tickets to his live experience.

The Willy Wonka Chocolate Experience website (all art is AI-generated)

But when attendees arrived, they found a sparsely decorated warehouse that was barely Wonkified. Quarter-cups of juice sat on folding tables with bargain-rate oompa loompas frowning over them. The event was one disappointment after another that resulted in angry parents demanding refunds and a social media revenge mob pilloring the Wonka imitator. People were furious at this poor excuse for a Willy Wonka Wonderland.

Now you could argue Mr. Coul wasn’t misleading anyone. That marketing is just marketing. False advertising has been around forever! But there’s a reason this incident felt different.

Photographs of the actual event.

Marketing is an indirect signal as much as an explicit message. If an event has polished marketing, we assume the organizers have the resources to deliver on the promise. Even if the marketing stinks, if it looks expensive people think “This is real”. The medium is the message.

People are very good at picking up signals from media. Within seconds we can tell a TikTok is a TikTok and a b-movie is a b-movie. People are so aware of this that filmmakers often aim for production value over narrative value. They want it to look expensive, because that’s the signal they care about sending. And traditionally, it’s hard to make stuff look expensive! So it’s not a bad signal to send.

But generative AI scatters this signal. The majority of people haven’t internalized this idea yet. When parents see a Wonka Wonderland advertised with custom marketing materials, they assume it’s real and there’s real effort behind it. But in reality, the Imitation Wonka probably spent mere minutes generating the marketing art.

His mistake came when he brought the same casual effort to the physical event planning. I don’t think Mr. Coul deliberately under-delivered a crap event. AI could help him with digital marketing. It just couldn’t help with physical event planning.

One of these images cost $50,000 to produce, the other took about 25 seconds.

Let me pull out the takeaway here. Polished images are no longer a signal of competence or resources. False advertising has always been a problem, but it still required effort and thus still held a signal. Now it’s too easy. As I’ve written about before tens of billions AI images have been generated. Very soon, most images we see will be AI-generated. This is a major update we need to make to our perception.

Obituary Attention Traps

Ranking first in google search has always been a dark art. Now it’s gotten darker. 

Obituaries are an odd genre of necessary writing. Whenever someone dies, their obituary is high-demand, very suddenly. In an attention economy, that’s valuable.

Recently, there’s been a flood of low-quality, error-prone obituaries for very commonplace people. Many people report googling a deceased friend’s name and finding obituaries on strange websites that seem to say “a lot without really saying anything at all”. The consensus is they’re AI-generated. This is the rare AI development where my dad actually informed me (Sidenote: 2 out of 3 items in this article were reported to me by older relatives).

The business model is as follows: wait for public death announcements from mortuaries, programmatically scrape publicly available data about the deceased person, then instantly spin up a low-quality article. If you’re first, people will stumble into your website and their attention is yours to monetize.

The resulting traffic from a single obituary is too small to be worth anything. But at scale, it’s enough to be significant. 10,000 obituaries a day starts to make some kind of economic sense. Deceased names and life details are essentially as fertilizer for clickbait creation pipelines. 

In the past, if you were gonna hunt attention, you would hunt big. Now, you can hunt small game. It makes sense to might write an obituary for a small town Mr. Smith because you can do it for every Mr. Smith across every town simultaneously.

The larger trend here is opportunistic news farming. We can expect a lot more of it. When there’s predictable news items, AI content will get there first. Poorly and with many errors. But it will be first.

Retune Your Perception

There are plenty of other misuses of AI. Many are more sophisticated and malicious: AI voice clones of loved ones, AI cat-fishing, the list goes on. Today I just focused on how it multiplies lazy short-term thinking.

We’ll likely see a lot more of this stuff. We can expect:

• Effort put into areas where it previously wasn’t worth the effort. The cost of playing is a lot lower.

• Visual polish will no longer be a very good signal. Marketing in general will be a less clear signal.

• A lot of disasters will occur as digital success tries to bridge into physical world success.

It will be weird, but people will adjust. And as the old signals stop being useful, we will find new signals. If you have other examples that fit with this article, feel free to add them to the comments.