Question

Best practices for dealing with AI Spam?

  • 22 April 2024
  • 8 replies
  • 46 views

Userlevel 2

Hi all,

We’re definitely starting to see spam ramp up in our community and while I’ve taken every proactive measure I can think of… there’s got to be more!

We’re still seeing a lot of spam sneak through the Akismet filter - mostly because spammers are getting crafty by generating AI ‘answers’ and sneaking links in. Or sometimes they just post something that is clearly spam (example: a scantily clad woman with an inapproriate title) and Akismet is like “Looks great. Post!” 😫

Things we’ve done:

  • Actively use the Akismet spam filter properly by tagging spam as such and removing through the spam page
  • Liberally banning users who exhibit any spam like behaviors including suspicious posts
  • Setup a Zap that alerts us with a Slack message for any post that includes a link so we can review

Things we can’t do:

  • Permanently turn on pre-moderation. It kills engagement in our community and raises redundant posting significantly.

Any ideas? What I really wish we had was a Zapier trigger that would allow us to mark something as spam using API. I’m fairly certain we could use AI and some general guidelines to catch more posts and quickly get them out of public view.

Thanks!


8 replies

Badge

So we were faced with the same issue. We decided to turn on Settings → Registration Rules → Registration Approvals. It takes some effort to approve them but we’re fortunate that our member roles are basically customer, partner, employee and registered guest. The only role we don’t approve without further review is registered guest. The others are not going to spam us as they have been verified in our CRM. For the registered guest role we check the email and user name in Google. Anything suspicious comes up we ban them. We have a notice on the main page of the community letting everyone know they need their registration approved before they can post or reply and please allow up to 24 hrs. I include my email there so people can reach me in case we ban a legitimate account. We have been doing this for a few months and ZERO spam, no one has reached out to me with issues. We approve about 30-40 registrations a day and ban 3-4 a week. BTW - Gmail addresses make up about 90% of our spammers. Unfortunately, many of our legit users have Gmail addresses so we couldn’t ban that domain.

Userlevel 2

Hey @Chris Hackett! That’s an interesting approach. I’ll have to think on it a bit more. In our situation, the only people who can register for community are those with a product account. But like a lot of SaaS companies, we’ve got a free version (well, free trial and then into free basic) - so you’d think that would cut down on spam. But unfortunately our product registration process is really quick and easy, so the spammers barely slow down.

That also means that everyone trying to register for our community is technically in our CRM, however between country codes, domains, and such we could possibly come up with a bit of a risk score and decision. And yes, Gmail is at least 90% of our spam too but it’s also a significant portion of our legit customers.

Thanks for the idea!

Userlevel 6
Badge +5

Any ideas? What I really wish we had was a Zapier trigger that would allow us to mark something as spam using API. I’m fairly certain we could use AI and some general guidelines to catch more posts and quickly get them out of public view.

That API operation would be awesome. 

 

It inspired this less than ideal workaround Zap combo. I did something kinda similar to create auto-delete / auto-unmark-answers for my community (if reported by the original poster and includes a keyword in the reason). 

Objectives:

  1. get the spammy post removed ASAP
  2. get it in a queue to be officially marked as spam

Zap 1 

Trigger: Specific emoji added to a slack message (needs to be a public channel i think)

Action: Report topic/reply, include SPAM as the reason

  • You’d have to do paths for each topic type
  • Which means you may also need another step or 2 to get the topic type from the URL or something like that

Zap 2

Trigger: Reported post

Filter: Reason includes/equals SPAM

Actions:

  1. Delete the topic/reply
    • Same path branching requirement for question vs convo vs article, etc
  2. {Moderator Alert depending on your workflow}
    1. Slack alert to moderator with link to post in control view so they can mark as spam
    2. Add a moderation label that puts it into a control view of posts that need to have the offending post marked as spam and then clear the moderation label. 
Userlevel 2

@DannyPancratz I’m not seeing the “Report Topic/Reply” action anywhere. Can you elaborate on this bit?

Userlevel 6
Badge +5

@jillian.bejtlich ah, yes, it’s not a prebuilt Zap action, but it can be done via the API. I do a lot of automating in Zapier based on the API, since Zap actions are limited. 

Each topic type has it’s own API operation, which makes it a bit trickier. 

 

If you go this route (or any similar Zapier automation using API operations), use the API Request (Beta) action. That action was built by Zapier and the setup is a bit more intuitive and user friendly IMO. I haven’t used the Gainsight built Api Action action in quite some time; it works too, but it was buggier and tricker to setup from my past experience. 

Userlevel 2

Oh that makes sense @DannyPancratz. I was suspecting that was the case, but just wanted to make sure I wasn’t missing the obvious.

So now I’m wondering if I could modify the workflow to be something like:

  1. Trigger: New Post (Gainsight)
  2. Action: TBD (ChatGPT) - Need to dig into this one a bit more. Ideally asking ChatGPT to determine if it was written with AI and/or includes spam would be best. I’ve explored this before with some succes, but it’s been about a year!
  3. Path: No spam = Stop Workflow | Spam = Continue Workflow
  4. Action: API Request - Report (Gainsight)

Definitely something to look into. Better yet, I’d love to see some built in functionality around this!

Userlevel 6
Badge +5

@jillian.bejtlich Oooo that’s an interesting solution too. I haven’t done anything with ChatGPT and Zapier, but I know I need to get there eventually. 

Userlevel 2

It’s pretty darn powerful @DannyPancratz. I let it loose (in just one forum loaded with disclaimers and fail safes) at Zapier and while it wasn’t perfect, it did some really phenomenal stuff around summaries, tagging, and even initial answers.

I think what I’ll probably do before setting up any fancy Zaps is running all the spam we do get through the free version of ChatGPT and see how dialed in I can get it. If the proof of concept works, then a pilot is the next step.

Reply