What is the feedback bot?

The Feedback Bot, also known as the "NewTubers Bot" is a bot I have been (occasionally) working on for a while now. Its sole goal is to help improve the state of the "feedback" channel in the NewTubers Discord server.

Its general approach is to autonomously judge a feedback posts text based on various weighted criteria, such as structure, specificity, effort, and context. Each of these areas are consolidated into a single score, which corresponds to 3 rating levels:

1. "Good" - Posts that are reasonably detailed and are indicative of someone who genuinely wants to improve their content in some way. Marked with a ✅ emoji.
2. "Needs Detail" - Posts that are not necessarily bad, but are lacklustre as far as proper context goes. These posts often just need a little extra detail or direction for feedback givers to fully help. Marked with a ⚠️ emoji.
3. "Low-effort" - Posts that are detected to come off as one of those "giv me views pls" type posts which are not looking for real feedback, and are instead looking to gain an extra few views. These are marked with a ❌.

If a post is edited, it is re-judged by the bot, and the post's initial rating can be updated automatically.

The judgement system uses zero LLM or ML for processing, and has a manually typed set of phrases and criteria to flag posts in certain ways. The bot is frequently tuned by a set of multipliers (the weights mentioned earlier) to make certain good/bad features of a post sway the score more.

The truth is that most feedback posts in the server (at least when I started this project) are awful, with people posting crappy videos (think fully AI videos, poorly edited IShowSpeed clips, unedited Minecraft gameplay) with zero additional detail other than "watch me vid pls".

As someone who is passionate about helping others improve at content creation, I find this bot to act as a good first line of defense to separate the slop from the creators genuinely trying to better their craft.

Example of a post that needs additional details.

Behind the scenes

Outside the publically visible "ratings" that the bot provides, there is a little bit of extra stuff that happens behind the scenes.

NewTubers moderators (and myself) have permission to override or manually change a post's rating. This is to diminish the number of false-negatives that may harm a user's experience. Additionally, for updating and debugging purposes, posts are relayed to a private Discord text channel where I (Ben) can monitor the bot's performance.

For the sake of transparency, here is what the logs look like:

Once a "human judgement" occurs, the post's content and ratings are stored in a local, on-site database for statistical and debugging purposes.

Privacy

Data Processing and Logging

The majority of data used throughout the bot exists within the bot's running memory, or Discord itself. Usernames are only temporarily stored in memory for Discord usage, such as to speak to a user directly (i.e. "Hello @User!").

Posts that are saved for debugging purposes, and may be read solely by the bot's author (myself) as a means of judging the bot's performance, and making changes if there are too many "unagreed with" posts. Usernames and links are scrubbed, so that only the conversational parts of the posts are kept. This database is cleared out on a weekly basis or once the bot's weights have been adjusted.

No data is sent to any third-party whatsoever.

Opting Out

Autonomous interactions with the bot are solely constrained with the Discord server's #feedback channel, as is considered as a major feature of using it. If a user does not want any of their messages to be processed by the bot, they can simply choose to avoid using the channel in question.

The feedback channel is a completely optional part of the server, and is only used by a small minority of it's 20,000+ members.