Reddit Refugee RIP

  • 3 Posts
  • 14 Comments
Joined 1 year ago
cake
Cake day: June 12th, 2023

help-circle





  • FYI, this is due to a confluence of issues.

    • We are the largest instance with the highest active user count - and by a good margin.
    • We are dealing with a premature software base that was not designed to handle this load. For example, the way the ActivityPub Federation queues are handled are not conducive to high volume requests. Failed messages stay in queue for 60 seconds before they retry once, and if that attempt fails it sits in queue for one hour before attempting to retry. These queued messages sit in memory the whole time. It’s not great, and there isn’t much we can currently do to change this, other than to manually defederate from ‘dead’ servers in order to drop the number of items stuck in queue that are never going to get a response. Not an elegant solution by any means, and one we will go back and address when future tools are in place, but we have seen significant improvement because of this.
    • We have attempted contacting Lemmy devs for some insight/assistance with this, but have not heard back yet, at this time. Much of this is in their hands.
    • We were able to confirm receipt of our federation messages (from lemmy.world) to other instance admins instances at lemm.ee and discuss.as200950.com. As such we do know that federation is working at least to some degree, but it is obviously still in need of some work. As mentioned above, we have reached out to the Lemmy devs, who are instance owners of Lemmy.ml, to collaborate. I cannot confirm if they are getting our federation at this time. Hopefully in coming Lemmy releases this becomes easier to analyze without needing direct server access to both instances servers.

    As you can see, we are trying to juggle several different parameters here to try and provide the best experience we can, with the tools we have at our disposal. You may consider raising an issue on their GitHub about this to try to get more visibility to them from affected users.




  • Just so you know this has been reported and is a known issue (happens to me on the browser as well). Presumably, it is due to Lemmy’s use of websockets for API requests, which has its issues for this particular application.

    This has been addressed in version 0.18.0 as, among quite a few other changes, an HTTP API is now in place for API requests which will (should) resolve this issue.

    Lemmy.world is currently on back end version 0.17.4 which still uses websockets. You can see this at the bottom of every page. The reason for this is that version 18 removed the ability to use Captcha for account registration. Given the size of the instance and its rapidly growing user base, and the fact that we are still accepting registrations, we simply must have the ability to leverage Captchas to mitigate the bot spam signups, which has been an issue.

    We have tested the 0.18.1rc build (non-official release) in order to try and get this new functionality as soon as possible, but as you can see in the post 18.1 needs some time to bake still - at least for large instances.

    This is definitely being discussed among the admin/mod team and we will be upgrading as soon as we realistically can while providing a stable - usable - experience.