Well, the thing is, everybody seems to be pointing fingers at everyone else. Lattitude (the dev team behind AIDungeon) threw one of their own staff under the bus and are blaming OpenAI (the company behind the AI model being used), while OpenAI is blaming AIDungeon, all while the players/users are being collectively lumped together and called out for alleged "problematic" behaviour that, by the statistics, seem to be a vanishingly low % of the playerbase (tabling the argument as to whether it would be harmful or not).

Definitely the second half of your statement is true, though: this whole thing was a PR disaster if only because, as one user pointed out:

• Pretty much all the news about this is "AI Dungeon was making pedophilic content", meaning most anyone who didn't know what it was about now thinks that's all it was used for
• The formerly-loyal users that they had are now incensed and betrayed by their poor handling of the situation and the dev team simply cannot make it up to them at this point

The gray-hat data breach even exposed how disastrously amateur the app itself was made, and I think that alone more-or-less exposes how they were not ready to be in the position they were in. Hell, they didn't have a UI / UX dev on the team until this year (and AID has been around since at least 2019); the app was just a cobbled-together mess from one of those hackathon things.

So, one thing that really JUMPS out to me here is that people seem to be under the impression that (at the least this specific AI) is actively improving itself. This is not correct, with a small caveat that I will get to in a moment. A quote that has recently been floating around because it was said by someone who actively develops and works with AI systems is "AI is neither artificial nor intelligent," and I feel it's relevant to quote here.

See, OpenAI has this model of AI called GPT-3. As Tevish put it, it's a lot like older generations of chatbots, trying to find the most likely "words" to follow given what context it was given (a slightly more technical explanation is that the AI breaks down words into word parts before all its calculations). The BIGGEST reason it's so much better than older generations of chatbots at doing this, is because it was trained on a gigantic mass of text fed to it mostly from many corners of the internet. Not only are the things it is capable of outputting directly correlated to what was initially input in that training data, but also the things it tends to output is directly correlated to the "average" words that followed from context similar to what you just gave it to process.

For instance, due to the large amount of text fed to it from Wikipedia, AID users could get the AI to output not just similarly-styled articles about fictional people, but correct information about real historical figures or places. Another example: could both read and output certain types of coding on its own, as plain text, presumably because of text fed to GPT-3 from programming/collaboration websites like github. These were both complete surprises to AID users, as was the discovery that AID could output some very well-written smut. Given the nature of its training, we have to assume large fiction sites with hefty amounts of literotica such as Archive Of Our Own and Fanfiction.net had to have been in the training data.

But OpenAI's model of GPT-3 is not improving itself. In a manner of speaking, it is static. It has something like 13.5 billion data points that it's using to shuffle together and do it's calculations, and the only way to add to that is to train the whole thing over again (which presumably will happen with GPT-4 if/when it comes out). You also cannot simply subtract from that training data without training the whole AI over again, much the same way that you cannot force someone not to forget any singular fact, like that elephants exist. There are too many other "memories" (to extend the metaphor) that rely on that small factoid, and removing that would mean tearing out large sections of the brain and leaving it severely damaged.

The small caveat which I mentioned earlier is that, while GPT-3 cannot be given more data points to work with, it is possible to streamline its output. This is what (we think, it was never entirely clear) the AI Dungeon dev team were doing with their specific setup of the GPT-3 model. Essentially, they took some user data in order to tweak the dials which come with GPT-3, so that they could get more natural-sounding (not necessarily more eloquent because some people were writing as if it were a chat program and the AI was perfectly capable of replicating slang and internet lingo) results without it throwing out a bunch of garbage characters or repeat itself too much.

There are a few big important takeaways that need to be addressed simply from acknowledging the above, being how the AI works and is trained, and then some additional food for thought given the situation and what both OpenAI and AIDungeon are trying to "prevent."

TAKEAWAY 1 — "Garbage in, garbage out"

As I mentioned above, and I'm pretty sure everyone already gets this on an intuitive level (that the AI doesn't "think" and doesn't apply morals), that the AI is simply trying to form patterns which match up to the training data it was given. What's important to keep in mind, is that this means that there was a significant portion of its training data which involved sexual content (which, let's face it; if it was trained on writings from the internet, of course that would be a significant portion of its training data). If a decent chunk of this data came from fanfiction sites, it's probably a good bet that there's a good amount of that text which was based on underage characters like Pokémon and Harry Potter, and as I mentioned, that's not something you can just remove without re-training the AI all over again.

TAKEAWAY 2 — filtering vs. censoring

Everything about how the AI works points to, if the creators (OpenAI) are worried about it outputting "problematic" text (and on a personal note, there are many worse sociopolitical possibilities that this kind of text-generating AI can be put to use for than generating content that doesn't even qualify as pornography in most of the world), they should have been more selective with the training data. While obviously it's not entirely feasible to hand-select every word that the AI is fed, being surprised that the AI is outputting sexual situations when it was specifically fed large sections of the internet known for being intensely biased towards male-pov stories is almost meme-worthy, IMO. There already seems to be an easy way to be selective about the training data as it is, since the AI Dungeon team put together a Lovecraftian model based on the earlier GPT-2 model (which only had like 1.7 billion data points, a mere fraction of GPT-3) by feeding it all of H.P. Lovecraft's body of work. It sounds rather simple to be able to filter out content which you wouldn't want to see, before you feed it to the AI, as long as you're not giving it free reign to eat everything off the internet.

TAKEAWAY 3 — lobotomizing the AI

I specifically extended the metaphor about destroying a person's brain by having to remove every memory and bit of knowledge related to the existence of elephants, because that is exactly how it's affecting the AI Dungeon output. Whether it's the userbase finding patterns where there are none, or (IMO the more likely situation given the words we have on record from the dev team) the censor was immediately expanded to ensnare other evil non-Christian values (fun fact: the last job which the co-founder and head of the company running AID had, was for a company whose sole purpose was censorship), the trigger to put a freeze on adventures and flag for human review includes horses, watermelons, any time any dirty word is used nearby nearly any number (a very common trigger is "****, I dropped my 8 year old laptop"). There also seems to have been some backend manipulation of those dials I was talking about before, to try and "reign in" the AI, but this does not change a thing about the training data. GPT-3 itself can only output what it knows, so by trying to avoid having it say certain things, all you are doing is shutting down probably billions of pathways it could have considered, because you told it to avoid a certain word or combination of words. it results in a massive drop in quality, and that's if your filter is working correctly.

FOOD FOR THOUGHT 1 — the giant's can't do it, how are you doing it?

Google, Facebook, and other big internet tech giants have a serious issue with content on their site, simply because of how big they are. They have been trying for many years to effectively filter their content without getting too many false flags, and they are still struggling with it. Sometimes this is a sociopolitical issue -- perhaps you may remember the thing about "freeing the breast" because breastfeeding mothers were getting caught in a "no pornography" net on Facebook -- but the systems that flag the content in the first place is almost always automatic; i.e. run by an AI. And the AI that is being used to try and detect these things is not advanced enough yet to not have a significant portion of its actions be false positives. Some of the biggest companies in the world have not been able to solve this problem while actively working on it.

FOOD FOR THOUGHT 2 — the hacker's statistics

So it's since been removed from the main file (though supposedly github has a way of looking at past versions of public files, though I cannot for the life of me navigate that site), but the hacker who exposed the security flaw in AID initially did a simple anonymized search through all the text data to give a lower-end estimate of how many people were playing sexual adventures (and a further % that were playing otherwise-NSFW adventures), simply based on how many separate adventures contained a certain word or phrase. He claimed this had to be a lower-end estimate simply because of how fuzzy and inaccurate the search was, that it could not catch the more vivid and less straightforward descriptions which would indicate either sexual or NSFW scenarios. IIRC, the % was something like 31% adventures were sexual in nature and >50% were NSFW (I think including blood and gore). I could be wrong on the percentages, but it was very clear that a large portion of the userbase were being very intimate with the AI, possibly just acting out fantasies but also possibly just getting an outlet that they can't get anywhere else.

FOOD FOR THOUGHT 3 — intimacy with a machine

I couldn't find it with a quick google search, but because of this whole debacle I read recently that the creator of the first chatbot, when he showed the just-finished product to his secretary, was told to leave the room within five minutes because his secretary had already begun talking about personal, intimate information with this simple chatbot, pouring her heart and soul out to its cold circuits. Anecdotally, several users came forward just as the whole thing exploded because the AI was so advanced it was allowing them to explore parts of themselves that they otherwise couldn't address, in what had been a safe environment. Many were gay or bisexual youth trying to figure themselves out (though AID's terms of service do say you're supposed to be 18+ to use the service, so I'm not sure how the courts would favor them on that), some were simply saying how they would use it to explore the dark fantasies they have that they would never dream of pursuing in real life (whether that be sexual in nature or not), but the way AI Dungeon's dev team decided to rip away everyone's privacy left them all even more vulnerable than they already were, and most were only able to open up their hearts to the AI knowing that no-one else would ever see them, just like a personal diary.

FOOD FOR THOUGHT 4 — adults don't ask each other their age

This has to do with the way the AI outputs apparently-pedophilic content, especially in sexual situations when suddenly the lady you're with says she's 12 or something like that. While the AI is incredibly good at having in-character introductions and naturalistic dialogue, there's a cultural limitation in its training, in that adults don't normally ask each other how old they are, especially when first meeting, but we ask kids how old they are all the time (as well as kids asking each other their ages, but that's another matter). That would obviously carry over into the training data, and so whenever someone tries to ask the AI directly how old a character is, they get a too-young response through no fault of their own.