How do you feel about some of the ingame chat moderation being done by an AI?

It would be mostly fine, but the adaptations of toxicity would be quick, and more importantly, 'meant to create false positives'.

Humans will adapt their 'methods of being toxic' faster than most language programs can keep up because humans can easily substitute terms. It's not that the AI wouldn't detect the new terms after a while, it's that people find ways to change the context to make 'regular conversation' cause things to flag.

Definitely not impossible, but I personally feel that the input-to-effectiveness ratio might be too low, for now. A full custom solution might be worth implementing, but this would almost certainly lead to 'English Only'.

And of course, some people are 'naturally foulmouthed or lewd' in specific ways that their friends or allies are okay with, and while that would probably be 'okay' to just 'ignore chat between people on friends lists', that also has its issues.

I think players should just be given word filter and word report tools. You see a word you dislike? You report it (which automatically filters it on your end). Enough reports of the same word would filter it for most people (excluding anyone who checked an option of "don't filter words"). And GMs could get flags of people who had their words filtered.

In other words, I just don't think that AI would be able to properly control the chat.

We have not received permission from Diamaht to share the quoted post.

I don't know how Amazon does it, but for the ones I've poked at or worked with/on, this isn't how it works.

The 'AI' parses two things, 'sentiment' and 'intent'. What it lacks most of the time is 'context'.

So it can tell you 'at X time on Y date Z player used a clearly racial slur in a chat'. And good ones can tell you 'whether the tone of the conversation in the surrounding lines was positive or negative or neutral' based on some other types of 'emphatic' language.

It SHOULDN'T then be able to ban or probably even mute the person. But it COULD be set to the point where if a person SAID something and another person reported it, the flag that would come through would be more like 'Confirmed usage of racial slur in negative sentiment at X time', probably with the required snippet from chat logs included.

A human can glance over that easily.

Similarly, the AI could issue warnings under the same conditions, because a warning is a useful point to tell a player 'hey you're about to go over the line maybe, and you are now being watched more'. They could then complain that the algorithm is badly tuned and explain the thing they said. There's no punishment then, it could be outright ignored.

An AI to tell people 'hey you might be close to breaking the rules here, tone it down' would not be a problem, and I consider this part of chat moderation.

@Diamaht

Grati, I understand what part of it you have a problem with now.

I obviously don't, but as you also pointed out, the one writing the software seldom does.