Dealing with Spam

  • 19 August 2019
  • 7 replies
  • 257 views

Userlevel 6
Badge +1

Fighting spam is a continuous battle. We are constantly putting in effort to improve our tactics and stay ahead of spammers. Here are the 3 main tactics you can apply to your community in the battle against spam:

  1. Spam Filter
  2. Rate Limiting
  3. Registration Approval

1. Spam Filter


The Spam Filter protects the community from Spam messages. The Spam Filter uses Machine Learning to automatically detect spam messages. However, to be able to learn what is spam and not, the filter needs to be fed with data.

⚠ Once we’ve activated the Spam Filter for you, you’ll need to mark lots of posts as spam before it starts to automatically detect whether a post contains spam (the exact number is different for each community, but usually it must be fed tens of posts).
💎 The Spam Filter uses machine learning: the more information you feed it, the smarter it becomes. This includes unmarking non-spam posts that the Spam Filter marked as spam by mistake. It requires a sustained and committed moderation effort for the Spam Filter to become effective.

 

Tip: We sometimes see that users incorrectly are being detected as spammer due to the way they write their posts (e.g. a lot of code and urls), or due to a vpn which they use that our spam detection knows is often being used for spam. This can be prevented by not using such a vpn provider or by giving the user the primary role “Super user” (which automatically excludes them from the spam check).


How To Mark A Topic As Spam

  1. Go to ControlContent →  Moderation Overview.
  2. Select one or more topics, then press ‘More’ and select ‘Mark as Spam’.
  3. The topic will now be removed from the community.

How To Mark A Reply As Spam

  1. Go to ControlContent Moderation OverviewClick a topic.
  2. Find the reply that you want to mark as spam, click ‘Mark As’, and then choose ‘Spam’.
  3. The reply will now be removed from the community.

How To Unmark A Post As Spam

  1. Go to ControlContentModeration Spam.
  2. Select one or more posts that you want to unmark, then click ‘Not Spam’.
  3. The post will now be published on the community.

 

⚠ When you mark a member’s post as spam, that member will be automatically banned from the community. Take care not to accidentally ban a legitimate member!
🎯 We recommend that you frequently check the spam folder in Control and unmark any valid posts that the Spam Filter marked as spam by mistake (also, did you know that content that isn’t Spam is referred to as Ham?).
💎 Members whose posts are marked as spam by the Spam Filter aren’t automatically banned. This is in case the spam filter accidentally marks a valid post that isn’t actually spam. We recommend occasionally checking the spam folder and banning these members.

 

2. Rate Limiting


Rate limiting applies when specific users or actions exceed a defined limit. We've developed the following rate limits:

  • Post Topic limit on userid: New users, younger than 10 days, can only post 3 topics per day (measured per userid)
  • Post Topic limit on IP: post from different users on the same IP is limited to 3 accounts per IP within 10 days
  • Limit on failed registration: visitors from one ip might not do more than 5 tries per hour to register a user
  • Limit on failed login: visitors from one ip might not do more than 5 tries per hour to login a user
  • Limit on password request: visitors from one ip might not do more than 5 tries per hour to request a new password

Rate limiting is enabled on all communities by default. If you would like to disable it, let us know (support@insided.com)

 

3. Registration Approval


‘New user registration approval’ allows moderators to approve new user registrations. When this feature is enabled a moderator must approve or deny a registration before the user can post content on the community.

How to enable moderator approval for new registrations

  1. Go to Control Settings Registration Rules
  2. Enable ‘Registartion approvals’
  3. Hit save changes, and you're done!

New community users will see a “Membership requires approval from moderator” message when they activated their account. At this stage the user is part of the community but he/she can’t create topics or reply yet.

Moderators need to approve a registration before the user can create content. Approving new registrations is a simple 3 step process:

  1. Go to Control Users Users Overview
  2. Filter on user role: 'Waiting for moderator approval'
  3. Open a user profile page and click 'Approve registration'

Upon approval the user will receive an email that his registration is approved, at this stage he/she can create topics and reply.

 


7 replies

Hi all,

We’ve just finished cleaning up massive amounts of spam on our Community. We used the mark as spam button for this, but we’d like to know if there is anything we can do that prevents the spam instead of cleaning it up afterwards?

The topics especially pop up in the evening, so we aren’t there to ban the users when they start, which results in an immense amount of spam to be cleaned up in the morning. Additionally, our Superusers (and other customers) are getting quite irritated by all the spam, because they can’t really locate the topics that are from actual customers.

Would something like a Captcha log-in on the platform help with this?

 

Here is a screenshot of a page of marked spam:

 

Userlevel 7
Badge +1

Hi Brian,

sorry to hear that you are facing this issue at the moment, of course I get that this has a negative impact on the general user experience. Especially if this happens outside of office hours and in large quantities it becomes quite annoying.

So as soon as you hit the [mark as spam] button the system should learn how this content is differing from other, regular content on the community. In this specific example I could imagine that topic titles which contain “[1080p]” or “[2020]” will automatically put in quarantine and not be published, as this is something that regularly is not put into the title of a topic on your community. So let’s hope that by this the system will learn and prevent future (similar) spam to be published.

About detecting spammers early:

We also have thought about securing registration or login via a captcha-code, however this has a few downsides: Regular users will be hindered by this, which could result in people not registering. Next to that, non-robotic spam (which most of nowadays spam on our communities is really) will not be stopped by this as the spammer will easiliy tick that box. Also, it is a third-party tool and that would mean we/you would have to include it in e.g. terms & conditions etc.

What we have done is we have implemented something to detect automatic spammers in a different way so that they are not making it to the community in most of the cases: A hidden checkbox (dynamic honeypot) which spambots will use and automatically prevent those from registering. Hence in our view a captcha is not necessarily the best solution or necessary in our opinion.

What I would like to know: Do you see a specific pattern in the email addresses these spammer use? Maybe they all come from a certain (unusual) email provider, or from a domain of a certain country (e.g. “.ru”)? If this is the case, we could also look into blocking these specifically on your community. During the registration our spam protection checks with a list of known spam accounts anyways, however possibly such an individual blacklist could help you keeping them out in the future as well.

Hi Julian,

Thank you for the detailed response. We checked the e-mail addresses but they are all over the place from different sources and domains (private domains, Gmail, etc.) so it might be a botnet type thing. If it continues we’ll button down the hatches for now with manual approval until it stops. 

There seems to be another issue of certain users, including two of our superusers, being banned or losing their ‘role’ as well. I don't know if this related to maybe the anti-spam measures? We've forwarded the accounts to your colleague Tom by e-mail just in case.

Userlevel 7
Badge +1

Thanks for the swift reply. Yeah, if they are registering from gmail or other domains it is tricky, hopefully you will not have to use more drastic measurements to prevent this from happening in the future.

Usually (as described in the starting post of this article) users with the role “Super user” should be excluded from the spam detection, however it could be that some action has triggered the system to cause this. As far as I know Tom is already investigating, I hope this will be resolved quickly!

Userlevel 4
Badge +3

We’ve had two other bits of feedback to improve this spam mechanic:

 

  1. Links: prevent/flag any ‘newbie’ rank links in content. once they progress to a more regular (trusted) rank, allow users to post links more easily. We’d need some domains whitelisted though. 
  2. As an alternative: allows a certain rank (super users) to ‘hide’ anything they think is suspicious, as part of that flagging. So the potentially malicious link is not public for the time it takes for the ‘flagged’ folder to be looked at. 

 

Thoughts on these? Shall I raise as an idea?

 

 

Userlevel 6
Badge +4

@timcavey I believe the second bullet point might be covered by this idea:

The idea basically is: If multiple users flag the same content, it automatically gets quarantined until it was reviewed.

Userlevel 4
Badge +3

Nice one, @bjoern_schulze that’s cool feature that is just as good. :sunglasses:

Reply