Every political thread is chock full of people being angry and unreasonable. I did some data mining, and most of the hate is coming from a very small percentage of the community, and the rest of the community is very consistent in downvoting them.
The problem is that even with human moderators enforcing a series of rules, most of those people are still in the comments making things miserable. So I made a bot to do it instead.
!santabot@slrpnk.net is a bot that uses an algorithm similar to PageRank to analyze the Lemmy community, and preemptively bans about 1-2% of posters, that consistently get a negative reaction a lot of the time. Take a look at an example of the early results. See how nice that is? It’s just people talking, and when they disagree, they say things like “clearly that part is wrong” and “your additions are good information though.”
It’s too early to tell how well it will work on a larger scale, but I’m hopeful. So, welcome to my experiment. Let’s talk politics without all the abusive people coming into the picture too. Please come in and test if this thing can work in the long run.
You must log in or register to comment.
I’m not interested in being polite to people who want to take away rights, promote discrimination, and try to overthrow elections.
The idea sounds great in theory, but seems like a bad idea with the massive rise in fascism.
I completely agree with you on that. “Pleasant” might have been a misleading way for me to frame the community. As far as the bot is concerned, you’re free to be as unfriendly to fascists as you want.
As a matter of fact, part of what I think is wrong with the current moderation model is the emphasis on “civility.” I think you should be allowed to be unfriendly.
I’ll give an example: I spent some time talking with existing moderators as I was tweaking and testing the bot, and we got in a discussion about two specific users. One of them, the bot was banning, and the other it wasn’t. The moderator I was talking with pointed it out and said that my bot was getting it backwards, because the one user was fine, and the other user was getting in arguments and drawing a lot of user reports. I looked at what was going on, and pointed out that the first user was posting some disingenuous claims that were drawing tons of hate and disagreement from almost the entire rest of the community, that would start big arguments that didn’t go anywhere. The second user was being rude sometimes, but it was a small issue from the point of view of the rest of the community, and usually I think the people they were being rude to were in the wrong anyway.
The current moderation model leaves the first user alone, even if they want to post their disingenuous stuff ten times a day, and dings the second user because they are “uncivil.” I think that’s backwards. Of course if someone’s being hostile to everyone, that’s a problem, but I think a lot of bad behavior that makes politics communities bad doesn’t fit the existing categories for moderation very well, and relying on volunteer moderators who are short on time to make snap judgements about individual users and comments is not a good approach to applying the rules even as they are.
So come in and be impolite to the fascists. Go nuts. You don’t have to be pleasant in that sense. In fact, I think you’ll probably have more freedom to do that here than in other communities.
Then that defeats the purpose of your community dude.
I was kind of like rooting for you, but it just seems like from what you said here that you’re only gonna allow people to be rude to whatever party. It is that the majority people on Lemmy don’t like. Isn’t there already enough of that here ? How is that any different from what’s already going on in the political communities?
Who is anyone to say who’s a fascist and who isn’t?
You’re just going to take their word for it, as if they’re some certified expert and shit? “I don’t like what you said, therefore i deem you a this or a that”
I mean, honestly, if you want to be real, (which I doubt you are gonna be)
Sorry, I’m not saying that to be an asshole, but you should be fair and equal to both sides, and you’re not.
You’re doing exactly what Lemmy is already doing.
Why is it that some of you moderators and admins can’t just be equal without letting your feelings dictate who is right and who is wrong?
Assess both sides under the SAME scrutiny, even if you don’t like something. I mean, really who, even wants to be a part of a discussion like this?
Its not productive or educational, it’s just a bunch of politically obsessed, angry People, throwing fits because somebody said that they’re a this or a that and don’t agree with their politics.
You guys really need to just stop bullshitting and just be honest with what your true intentions are.
Anybody who’s worth a dam, who could add anything of value to this community doesn’t want to waste their time with shit like this.
I was kind of like rooting for you, but it just seems like from what you said here that you’re only gonna allow people to be rude to whatever party. It is that the majority people on Lemmy don’t like.
You’re absolutely right to worry about this. This was one of my biggest concerns when I was setting it up. Lemmy already has a definite community vibe and consensus opinions to go with it, and I think censoring the “opposition” opinion is one of the quickest routes to turning any political community into a useless circle-jerk. Most lemmy.ml communities are like that.
My goal was to set the parameters broadly enough that people who disagree with the community are allowed to say whatever they want, but still strict enough that people who are outright jerks in any big fraction of their comments get removed. The current tuning bans about 1.4% of the community. You’re still not banned. I don’t think limiting it to 98.6% of the community will create too much of a circlejerk. There’s only one user that I’m aware of that is banned, for which I disagree with the ban, and I talked to them for a while, and sent them some detailed examples of what the bot concluded about their posting. I concluded by saying that while I disagree with silencing them, I think amending the way they present their posts will help the bot’s conclusions about them, and also for the same reason get their point across more effectively to any person who’s reading them. The huge amount of downvotes they’re getting doesn’t necessarily mean they are wrong, but it does mean most people are putting them in that bottom 1.4%, which is a problem if they want to convince anyone or accomplish anything.
It helps that I sympathize with some viewpoints that are unpopular, so I get it if someone wants to have the right to speak their mind without some person looking over their shoulder deciding if they’re allowed to, or if they’re being civil enough about each individual comment. You’re right. That’s ridiculous.
You’re just going to take their word for it, as if they’re some certified expert and shit? “I don’t like what you said, therefore i deem you a this or a that”
Absolutely not. Part of what came through over and over again while I was tuning the bot, and looking over mod decisions to contrast with it, was that a lot of times the moderators are coming in and making snap judgements that are far less complete and accurate than can be gotten from looking at what the whole community consensus thinks is a problem.
You’re doing exactly what Lemmy is already doing.
Why is it that some of you moderators and admins can’t just be equal without letting your feelings dictate who is right and who is wrong?
Assess both sides under the SAME scrutiny, even if you don’t like something. I mean, really who, even wants to be a part of a discussion like this?
This is the algorithm. It’s not going to be clear what it’s doing, since it’s not commented well and it would be complicated to understand even if it were, but surely you can see that there is no “if my_llm_thinks_is_fascist:” block in it or anything.
Like I said, you’re not banned, as of the current parameters. Part of the idea is to give people the freedom to come in and say what they want, instead of having an overworked mod decide by hand on the spot what is disinformation, what is incivility, what sources are reliable and not, important and not trivial decisions like that. I don’t know how to duplicate for you the time I spent looking over what the conversations really look like, how to draw the line so that the people everyone thinks are clearly bad actors are removed, but the people who are simply unpopular or have a minority opinion are welcome, but that’s what I tried to do.
One way to cut to the chase: Just try it. Come in, say some political opinions, see if it works. The bans are mostly static based on past behavior, so as long as you’re not posting porn or KKK flyers or something, I think you’ll be fine.
If it’s something outside the realm of politics I will probably moderate it by hand. I’m not trying to offer a blanket “free speech safe space” for racism or anything else that anyone feels like posting. Sorry. If you want that, you can go to Twitter. It’s up to you of course, but I think that this is a step closer to what you’re saying here that you want, not a step away from it.
Huh? I’m an adult, I can handle someone being rude. That’s no issue.
Don’t get me wrong, I’m all for respect and civility, but discussing politics is so… dull. Can’t we talk about something juicier? Like the latest fashion trends or which celebrity is serving looks on the red carpet? You know I’m all about spreading love and positivity, but save the politics for someone else, darling! Now, tell me, have you seen the newest collection from Versace?
Lol
I think any experiment that could potentially filter out bad faith participants is at least worth a try. I participate in political discussions pretty infrequently, but when reading them I often see users jumping in with a ridiculous viewpoint that they are completely unwilling to discuss or hear any flaws about. That’s not conversation, that’s trying to shout others down, and I will be interested to see if that kind of behavior gets caught by your bot.
I know exactly what you mean. If I had to pick one type of comment that the bot is designed to ban for, those are them. It turns out to be pretty easy to do, too, because the community usually downvotes those comments very severely, even if the current moderation rules allow them even when someone does them 20 times a day.
Pick a name of someone you’ve seen do that, search the modlog on slrpnk.net, and I think you will find them banned by Santa. And, if they’re not, DM me their username, because there might be some corner case in the parameter tuning that I have missed.
Disturbing.
Algorithmically censored and sanitised political speech.
Thats gonna be a no from me, frankly Id like to see such a community banned for the harm it’s going to cause. Its bad enough we have that nonsense on other social media.
Really cool idea! I agree on the whole most people are incredibly nice and will go out of their way to explain their reasoning. But a small, loud group seems to crop up in political discussions. It’s interesting because it’s not always they they don’t know they’re being rude, but rather they know and are proud of it because of their beliefs.
It was remarkable, when I started looking at it, how small the population of users is that seem to be causing almost all of the problems. It was also remarkable how little the existing moderation approach is doing to rein them in.
Interesting concept
I know this will ring hollow, considering I am (predictably) on the autoban list, but:
I don’t know how this isn’t a political-echochamber speedrun any%. People downvote posts and comments for a lot of reasons, and a big one (maybe the biggest one in a political community) is general disagreement/dislike, even simply extreme abstract mistrust. This is basically just crowdsourced vibes-based moderation.
Then again, I think communities are allowed to moderate/gatekeep their own spaces however the like. I see little difference between this practice and .ml or lemmygrad preemptively banning users based on comments made on other communities. In fact, I expect the same bot deployed on .ml or hexbear would end up banning the most impassioned centrist users from .world and kbin, and it would result in an accelerated silo-ing of the fediverse if it were applied at scale. Each community has a type of user they find the most disagreeable, and the more this automod is allowed to run the more each space will end up being defined by that perceived opposition.
Little doubt I would find the consensus-view unpalatable in a space like that, so no skin off my nose.
I looked at the bot’s judgements about your user. The issue isn’t your politics. Anti-center or anti-Western politics are the majority view on Lemmy, and your posts about your political views get ranked positively. The problem is that somehow you wind up in long heated arguments with “centrists” which wander away from the topic and get personal, where you double down on bad behavior because you say that’s the tactic you want to employ to get your point across. That’s the content that’s getting ranked negatively, and often enough to overcome the weight of the positive content.
If Lemmy split into a silo that was the 98.6% of users that didn’t do that, and a silo of 1.4% of users that wanted to do that, I would be okay with that outcome. I completely agree with your concern in the abstract, but that’s not what’s happening here.
The problem is that somehow you wind up in long heated arguments with “centrists” which wander away from the topic and get personal
I’m not surprised I was identified by the bot, but it’s worth pointing out that ending up in heated arguments happens because people disagree. Those things are related. If someone is getting into lots of lengthy disagreements that are largely positive but devolve into the unwanted behavior, doesn’t that at least give legitimacy to the concern that dissenting opinions are being penalized simply because they attract a lot of impassioned disagreement? Even if both participants in that disagreement are penalized, that just means any disagreement that may already be present isn’t given opportunity to play out. Your community would just be lots of people politely agreeing not to disagree.
I have no problem with wanting to build a community around a particular set of acceptable behaviors -I don’t even take issue with trying to quantify that behavior and automating it. But we shouldn’t pretend as if doing so doesn’t have unintended polarizing consequences.
A community that allows for disagreement but limits argumentation isn’t neutral - it gives preferences to status-quo and consensus positions by limiting the types of dissent allowed. If users aren’t able to resolve conflicting perspectives through argumentation, then the consensus view ends up being left uncontested (at least not meaningfully). That isn’t a problem if the intent of the community is to enforce decorum so that contentious argumentation happens elsewhere, but if a majority of communities utilizes a similar moderation policy then of course it is going to result in siloing.
I might also point out that an argument that is drawn out over dozens of comments and ends in that ‘unwanted’ behavior you’re looking for isn’t all that visible to most users; if you’re someone who is trying to avoid ‘jerks’ then I would think the relative nested position/visibility of that activity should be important. I’m not sure how your bot weighs activity against that visibility, but I think even that doubt that brings into question the effectiveness of this as a strategy.
Again, not challenging the specific moderation choices the bot has made, just pointing out the problem of employing this type of moderation on a large scale. As it has been employed in this particular community is interesting.
Do you mind if I give some examples? What you’re saying is valid in the abstract, but I think pointing out concrete examples of what the bot is reacting to will shed some light on what I’m talking about.
You’re free to provide examples, but like I said it’s not the specific moderation choices that are the problem, it’s using public sentiment as a core part of that determination.
Here are examples of things you got positive rank for, politics and argumentation:
- https://midwest.social/comment/10357782
- https://midwest.social/comment/10246105
- https://midwest.social/post/13347310
Here are examples of things you got negative rank for, not directly political interpersonal squabbling:
- https://midwest.social/comment/10412423
- https://midwest.social/comment/10413376
- https://midwest.social/comment/10432272
Maybe this is harsh, but I think this is a good decision by the bot. The first list is fine. Most of your political views are far from unpopular on Lemmy. The thing is that you post a lot more of the squabbling content than the political content. You said you’re being unpleasant on purpose, don’t plan to stop, and that people should probably block you. I feel okay about excluding that from this community.
If in the future you change your mind about how you want to converse, you can send a comment or DM. We can talk about it, make sure you’re not being targeted unfairly, but in the meantime this is completely fair.
I already said I don’t take issue with any one decision, I care about the macro social implications.
I made this system because I, also, was concerned about the macro social implications.
Right now, the model in most communities is banning people with unpopular political opinions or who are uncivil. Anyone else can come in and do whatever they like, even if a big majority of the community has decided they’re doing more harm than good. Furthermore, when certain things get too unpleasant to deal with on any level anymore, big instances will defederate from each other completely. The macro social implications of that on the community are exactly why I want to try a different model, because that one doesn’t seem very good.
You seem to be convinced ahead of time that this system is going to censor opposing views, ignoring everything I’ve done to address the concern and indicate that it is a valid concern. Your concern is noted. If you see it censoring any opposing views, please let me know, because I don’t want it to do that either.
At first I read it as “peasant”
Disappointed I read it wrong
I’ve already declined two reports requesting that I take moderator action against content that’s people directly going out into their community and helping get things done, because that is “not politics.” People definitely seem to want their mods to be vigorously engaged in enforcing the boundaries on the stuff people are allowed to say.
As far as my take on it, we can have overlap between the peasant politics and the pleasant politics. The community was for the latter, but the former sounds great, too.
I did some data mining
Can you share the model?
The code for the bot is open source. It’s not an AI model. It’s based on a classical technique for analyzing networks of relative trust and turning them into a master list of community trust, combined with a lot of studying its output and tweaking parameters. The documentation is sparse, but if someone is skilled in these things they can probably take a few hours to study it and its conclusions and see what’s going on.
If you’re interested in looking at it for real, I can write some better documentation for the algorithm parts, which will probably be necessary to make sense of it beyond the surface level.
Thanks you, I’m personally more interested on the statistics used on the parameter searching, but given that is python I’m checking out to see what can I learn.
I added an explanation of the details of how it works to the source file that implements the main rank algorithm. The math behind it is not simple, but it’s also not rocket science, if you have some data science abilities and want to check it out.
Don’t let the python fool you. It is not simple python. I’ll try to add some comments later on to make it more clear what’s going on.
For tuning parameters, it was complicated. Mostly, I did spot-checks on random users at different ranking levels, to try to check that the boundary for banning matched up pretty well with what I thought was the boundary of an acceptable level of jerkishness. That, combined with deeper dives into which comments had made what contributions to the user’s overall rankings. And then talking with existing moderators, looking over the banlists, and bringing up users where they thought the bot was getting it wrong. There were a lot of corner cases and fixes to the parameters to fix the corner cases. Sometimes it was increasing SMOOTHING_FACTOR to make users more equal in rank with each other, when we found some user that was banned because of one bad interaction with some high-rank person who downvoted them. Sometimes it was changing parameters to change how easy it is to overcome a few negatively-ranked postings by being generally positive with the rest of your postings. There are always users for which the right answer is a matter for debate or opinion, but as long as the bot isn’t making decisions that are clearly wrong, I think it’s doing pretty well.
You can look over some places where I talked with people about the bot’s opinion of their user, in this post and this post. I don’t want to publicly do those breakdowns for people who haven’t agreed to have it done to them, but that might give you an idea of how the tuning went. What I did to tune the parameters was the same type of thing as I showed in those comments, just a whole lot more of it.
Have you prepared for downvote manipulation by bots? Quora incentivizes it by treating downvoted answers differently, so now the site may have as many bots as people.
It’s difficult. A downvote from an account with no history does nothing. Your bot has to post a lot of content first to attract upvotes from genuine accounts. Then once you’ve accumulated some rank, you can start giving upvotes or downvotes in bulk to the accounts you want to manipulate. It’s impossible to completely prevent that, but you have to do it a lot to have an impact.
I think this model is more resistant to trickery than it would seem, but it’s not completely resistant. I do expect some amount of trickery that will then need counter-trickery. On the other hand, the problem of tricking the system also exists in the current moderation model. You don’t have to outwit the system to get your content posted or ban your enemy if it’s trivial to flood the comment section with your content from alt accounts and drown them out instead. I don’t know for sure that something like that is happening, but it wouldn’t surprise me if that was one reason why there are so many obnoxiously vocal people.
What are its rules? I don’t see anything in the sidebar.
It’s in the sidebar:
Post political news, or your own opinions or discussion. Anything goes. No personal attacks, no bigotry, no spam. Those will get a manual temporary ban.
I’m not sure whether this will have the intended results, but I’m interested in this experiment!
deleted by creator
Why is Nixon the thumbnail for it? Especially as it’s futuramas version who is anything but pleasant
Look at him, he’s so happy.
Maybe it should be Bernie smiling, instead? I didn’t want to be openly partisan.
The point is if you want a pleasant community don’t use any polarizing figures
Use clip art of some debate podiums or something?
I’m interested and curious to see what happens with such a setup. I wonder if I will end up on the ban list as a result of mostly participating in the conservative community and being contrary.
You’re not banned or even close to it. The ban list is surprisingly lenient in terms of people’s differing political views. You have to habitually make enemies of a lot of the people in the comments, one way or another, with a big fraction of what you post. Most people don’t do that, wherever on the political spectrum they might fall.
Whether that’s a good idea or not remains to be seen. I had some surprises today.
