I am currently getting signed out every minute from lemmy.world. This is not a client side cache issue. I tested making API calls from the command line (with curl
) with no cache and the issue still occurs. One call I get the correct response, the next I get a 400 telling me im not signed in.
I’m primarily testing with the https://lemmy.world/api/v3/user/unread_count api endpoint. I’m not sure if this issue occurs with all endpoints.
Reproduction steps:
- Get a lemmy.world JWT token for your account using your desired method (eg. postman).
curl https://lemmy.world/api/v3/user/unread_count?auth={JWT_TOKEN_HERE}
- Note the 400 error. If you do not get an error repeat step 2.
Edit
This issue only seems to affect lemmy.world so a temporary workaround is to use a different instance for the time being.
Just a quick statement from the admins team to say that we are aware of the issue and yes we are looking into this.
Thank you @idunnololz@lemmy.world for the elaborate report and everyone else for their patience while we try to sort this one out!
Edit: Lemmy was upgraded to 0.18.2
Thank you for making a statement about it!
Thank you for all that you do for this place. I am consistently amazed at how quickly y’all are able to resolve issues.
o7
Sounds like lemmy.world runs on 2 instances and the requests are being loadbalanced between those two. That and that the jwt secret is different between those two instances causing one to accept and the other to reject
This is also my theory. I think you’re right on the money here. They probably rotated secrets from yesterday’s hack and forgot to restart both servers.
Does anyone know who can contact the server admins?
Yell real loud in all caps
REAL LOUD
Yes I have the issue on Liftoff - can’t even log in again. Alt time for now!
Ahhh thanks for that! I took could not stay logged in, annoyingly id find out when I attempted a post/reply, “not logged in”. Using Connect or website.
I installed jerboa, logged in ok and here I am. If it goes bad too then I dunno…
Okay, so how do we get this fixed? Any way to get admin attention? I think @Spaltovic@lemmy.world is probably correct about the cause.
This is happening in the connect app and I thought I was going crazy.
Making a new post is a nightmare. I wish the submit button would time out in these instances so you can try again. Right now I’m having to copy and paste into a new tab and hope for the best (but fail, 5 times and counting).
Just add me to the list. Jerboa seems to stay logged in for about 75% of interactions.
Yeah. Lemmy.world is currently unusable on the desktop. I don’t have that problem in Memmy. Growing pains but I hope the problem will be fixed soon. Do anyone know if one of the mods in North America are aware of the problem?
I was having trouble in liftoff and the browser. Cleared data and cache from liftoff thinking maybe something got messed up there and now I can’t even log back into my .world account 🤷♂️ I’ll hang here for a bit I guess.
Same here, can’t log in again via Liftoff.
I’m choking in desktop browser and in liftoff. Jerboa seems ok. It’s weird to me how different clients react differently, I’m not sure how they interact differently.
I was running into this across both my accounts on lemmy.world. Changing my password seems to have resolved it both on the web and in Mlem.
I’m having the same issue… might switch to an alt or something for now.
FWIW, I can confirm I’m having this issue as well. The load balancing hypothesis seems sound given the behavior I’m seeing. Definitely making lemmy.world pretty much unusable at this point.
Same problem for me it seems, dunno if I’ll even be able to comment. Refuses to stay logged in.
From my tests, it’s almost perfectly a 50/50 whether any API requests you make will yield a 200 (success) or a 400 (not signed in). If you perform an action that takes 3 API requests, your chances of succeeding is (1/2)^3 or 1/8 because only 1 request needs to fail in the chain for the entire action to fail. So, as long as you make single API actions you can maximize your success rate :D
What’s an example of something that would take more than one API request?
Signing in. Most websites/apps will probably also grab your unread count, and maybe even your subscription feeds.
Another example is checking your inbox. Lemmy actually has 3 inboxes: mentions, replies and PMs. A lot of websites/apps bundle these three so they will need to check all 3 inboxes via 3 API calls.
Smells like two instances behind the load balancer, one is fine with the JWT, one is not.
Seems like spamming actions also gets it to work eventually. It’s a pain in the arse though lol. I made some alt accounts on other instances, but I’m lazy and don’t wanna rebuild my subscription feed if I don’t have to, so hopefully it gets fixed at some point.
I’m getting this too, even after clearing cookies and logging in again. I’ve seen it on multiple devices (Android phone, Linux desktop with Chrome).
Thaaats what’s going on