2024-02-08 User bot spam spanning several days

Summary

From the 5th of Feburary 2024, a user going by the name “z**” began attacking various Revolt servers with automated bot spam.

The attacker appears to be an individual operating from a residential IP in Italy, utilising VPNs for their actual attacks.

It appears that the attacker’s motivation is purely for attention, as such their name has been censored and we ask that you avoid mentioning them.

Our response has been limited due to a combination of unfortunate factors:

1. We are still working on building tools for the moderation team to adjust the anti-spam systems on the fly.
2. It fell outside of usual operating hours.
3. The only way^†^ to tackle spam from registered accounts is:
   • Through direct database access and scripting. (most effective but due to 2. was delayed by several hours)
   • Through on-platform bots (limited per-server, requires development time + effort by server owners)
   • Through platform moderation / other automation using existing tools Directly tackling hundreds of accounts through the admin panel is quite cumbersome currently and I wouldn’t expect any staff member to be able to deal with this directly, but in theory this is possible.

^†^ this applies before further mitigations and tools are put into production, see Mitigations below for more information!

That’s all to say, there’s no straightforward way to block spam especially when the attacker has significantly more time to try to work around any sort of filters we put in place. Ideally we would have the option for human intervention at any hour but we’re yet to have any tools that can help with that (this is being worked towards).

We apologise for any inconvenience and we are working towards improving our response, transparency, and tools.

Mitigations

Currently deployed in production:

• Significantly harsher account creation rules (with dynamic scaling based on threat level)
• New users may no longer ping others in public servers for the first 12 hours of their account being created.
• Realtime ‘Bot Shield’ which detects and stops unusual activity from new users.

Shelved for now:

• An experimental realtime message anti-spam As per our Discover guidelines, all servers listed on Discover are automatically enroled into this new anti-spam. We hope to provide more customisation with regards to it in the future.

  The anti-spam will ban users if either:

  • A trained classifier for spam message features returns a positive (sticking to simple rules for now) NB. No personally identifying or user-generated content is present within the training dataset.
  • A simple set of rules for spam messages returns a positive

Future work:

• Interface for platform moderation team to adjust account creation rules.
• Interface for platform moderation team to quickly select hundreds of accounts based on given queries and ban them.
• Automatic re-training of spam classifier based on new spam messages that have been collected.

Timeline and Data

3rd - 7th Feburary Multiple attacks took place over several days (end times signify response):

• 5th Feburary 04:40-09:00 UTC: ~ 900,000 messages, ~ 200 accounts
• 6th Feburary 05:30-10:00 UTC: ~ 1,695,000 messages, ~ 150 accounts
• 7th Feburary 05:40-07:30 UTC: ~ 500,000 messages, ~ 185 accounts

(Evening of) - 6th February Additional account registration rules were deployed, but these did not prove to be effective enough.

7th Feburary The anti-spam was developed from scratch, deployed and all servers on Discover were enroled (as per guidelines).

01:07-06:06 UTC - 8th Feburary

Another attack started overnight, additional account registration rules were put in place at various times.

No accounts reached the platform and hence the anti-spam is yet to be tested against the bot accounts.

12:06 UTC - 8th February This post is written up.

I’ve just discovered a bug with the anti-spam which has unintentionally banned 12 new (registered within past 24 hours) users:

• All users have been unbanned and have had their account restored.
• All users have been sent an apology and notice that their account was momentarily banned.

01:09-08:25 UTC - 9th Februrary Another unmitigated attack, multiple issues cropped up:

• Certain account creation rules were temporarily on hold, (which would’ve at least blocked the first wave of accounts, attacker could still have opportunity to adapt).
• Anti-spam service appeared to have disconnected from the events server. It’s likely the connection just dropped and it did not re-establish, however the solution appears simple.
• A couple bugs cropped up in the service, these have been triaged and will be solved.

On the positive side of things:

• After restarting the anti-spam service, all spam accounts that were created in this time period were all swiftly banned (within 1 minute).

22:00-01:40 UTC - 9-10th Feburary

• Deployed additional account creation rules.
• The experimental anti-spam has been replaced with “Bot Shield”. A more detailed write-up will be provided later.
• Patches are being deployed to production Revolt to mitigate disruptance to users. You will not wake up to any mention spam from fresh bot accounts.