Report Brigading: Exploiting reporting systems in automated systems and self-moderated communities

بِسْمِ ٱللَّٰهِ ٱلرَّحْمَٰنِ ٱلرَّحِيمِ

Entities like YouTube (as part of Google) strive to automate everything, including reporting systems. Other entities, like Discord (a gaming chat platform), make communities self-moderate. In both cases, this reduces the need for staff to deal with reports/abuse/moderation and thereby increases profits.

I will now explain how to manipulate/game these reporting systems to achieve various goals.

Some reasons why reporting systems are manipulated by various threat actors

Many times I see exploits being detailed, but the motivations are never explained. Primarily, the first reason to conduct such attacks is due to money. If your streaming/video channel is getting fewer views than a competing 'vlogger', then that means you are getting less ad-derived revenue. You could attempt positive changes like new content strategies, or alternatively resort to negative tactics and attack your competitors. This negative approach to competitors is also seen in other technological areas where IP theft is rife (chip design, biotechnology, medical technology, etc.)

The second reason is driven by our own biases, dislike and hatred for the 'other' and is as old as humanity itself. These attacks are based on racial, ethnic, nationalistic, religious, intra-religious, cultural and other lines.

Gathering sizeable numbers to legitimize the attack

My assumption is that YouTube and other platforms are aware of 'brigading' (which is the usage of large numbers of people/bots to carry out various activities), but this is still an important vector in your attack. With enough non-bot-like users reporting from various geographies, the reports will be taken more seriously. While it is unclear if YouTube's AI self-reporting is 'smart' enough to spot this, it is relevant on various other platforms like: Twitch, Discord, Twitter, Reddit etc.

So gather your group, identify your target and then exploit the reporting systems:

The Discord/Reddit case

Discord uses a system where users create 'servers' (essentially communities) and have to self-moderate these communities. A similar model exists on Reddit ('subreddits').

I start with Discord/Reddit due to the need to explain a certain aspect of the attack process when you cannot report legitimate violations.

The reporting can be done using 2 different options:

1. Report legitimate violations: the targets here are copyright, pornography, racism, terrorism, homophobia and discrimination in various forms. The vagueness of the ToS of these entities enables attackers to be as broad as they desire

2. Infiltrate, incite, bait and report: this type of attack is the more effective one (¹). In this attack, you join your target community, embed yourself as sincere, start to slowly chorus with the slightly more extreme-minded individuals, ensure that your ToS violations target the 'other' that the community hates (like a right-wing group against LGBT+), gradually increase the incitement and then begin mass-reporting your own and other discussions focused on the ToS violations once they are severe enough. If the target community has no hatred for the 'other', then copyright, pornography and other non-discrimination methods can be used so long as they violate ToS

¹ Infiltration/Incitement is a known attack method used by various government intelligence agencies to incite marginalized groups and drive them to commit serious crimes or at least use their own incitement as evidence of the agenda of these groups.

The YouTube case

Platforms with content providers require a slightly different strategy. Here, you cannot infiltrate, incite or report content creators for what commenters say or do. The ToS violations still apply, but you will target victims on 2 levels:

1. Ad-demonetization: There are many documented cases of demonetization happening, as documented here in 2018. Exploitation here can happen via abusing the licensing system of content and hitting channels with copyright strikes (like a 'fails' channel publishing videos that they did not create)

2. Repeated strikes lead to bans: The same DMCA/copyright strikes can be targeted here, but more fruitful reporting would be against content that is not "age-restricted". YouTube and other platforms need to appeal to mainstream advertisers and as such, child-friendly and socially/politically favourable content will be prioritized. At the same time, any content crossing these boundaries would be ripe for reporting.

How to fix

The problem won't be fixed due to the sheer volume of labour needed to moderate content. Even when moderation is prioritized, the effects can be chilling to these workers.

Stay tuned for more infosec articles in the near future!