Hey all, For context, I am self-hosting a lot of different applications on a headless server and some of them are just scripts scraping content or doing automatic tasks.
I’ve been wondering for some time about how I should handle errors and how to be notified when they happen.
Currently, I need to connect to the server and read my logs to know that something has gone wrong. But I would like to get near instant notifications on my phone or computer.
I know I could send emails, try to use signal-cli or other messaging services but some of you might have better ideas.
Should I use one of these or is there some great technology I have never heard of?
Use ntfy.sh. It’s open source and has a free server.
Disclaimer: I made it ;-)
This looks brilliant! I love how simple the sending API is.
It had occurred to me that I could do something like this, but I’m happy you did all the hard work for me (especially the always-on feature of the Android app).
Wow really nice project! Do you work on this full time?
Thanks. I don’t work on it full time, no. It’s a side gig project I’ve been doing for a year and a half. I recently added paid plans to get a little side income, but it’s not really taken off. Likely because the free tier is too generous hehe.
Fair enough! Do you cover your costs for it? I see you’ve got live stats - what’s your monitoring stack?
This is just really cool, I’d love to build an actually useful service like this and have it at least pay for itself but so many things are so daunting! (Payments, SRE, having a nice front end, …)
I do cover the costs yes, through donations and the paid plans.
It’s definitely fun to do some things, but others are daunting indeed. I do, however, learn a lot. I have learned a lot that I was able to reuse elsewhere. All that is priceless.
Oh great ! I will look into it, it seems very cool.
Didn’t know grafana provided this kind of feature. I don’t think i’m going to use it but it’s really good to know. Thanks !
Prometheus has alerting and that’s really the preferred system for sending the alerts, IMO.
I use Seq. You can use it for free for personal use and it has a docker image available. I can use their libraries to send logs my scripts create to Seq, you can ingest logs from other docker containers and syslog, and set it up to send emails for certain log message patterns.
Also is very easy to install and set up. It’s just an exe in windows and docker container elsewhere.
You could setup a status monitoring system and then configured it to send out messages for critical alerts. For example, I’m using Vigil to monitor my services and it’s configured to send email alerts when something is down and then sms alerts when things are still down for too long (in case I didn’t read the email).
You’re right ! I didn’t think about status page tools… It might be overkill for my setup but I will definitely look into it. Thank you.
Might be overkill but you could setup and run a Jenkins server to run your scripts and use one of the many notification plugins that are available.
