Don't get me wrong, I'm happy you posted this. If there are unvoiced concerns or issues, here is a good place to discuss them.
I'm not as active on lemmy as I'd like, so I'm likely not aware of issues unless they're raised somewhere like this.
For potential meta issues such as age verification, I'll be seeking input from the community when they arise and information around them is known and understood.
The volume of activity from LW reduced, and it also looks like they made some config changes on their side... at a guess more federation worker processes, allowing an almost constant stream of activity to be sent to AZ.
When the LW lag issue was previously discussed, my main constraint was (and still is) time. Longer term it would have been a financial concern.
Either way, the issue has cleared up now due to a number of factors, none of which required time or finances on our part. I call that a win.
Worth noting, LW have still not upgraded their instance to the version that introduced multiple sending threads to a remote instance. If their volume of traffic were to increase substantially again, we could "lag" again.
I'm happy to hear feedback, but tbh you should think of AZ as a benevolent dictatorship rather than a democracy. So the analogy to a company AGM isn't quite right 😃
Sure.
Finished.
A minor Christmas miracle... AZ is no longer behind on LW content:
Hopefully this stays the case :)
Can turn it off and find out. I've done this a few times, each time I've ended up turning it back on.. I figure better brief downtime at a low usage time, than random downtime that may require intervention to resolve.
I'll give as much notice as possible, whatever ends up happening.
All good questions, that I don't have any answers to. Plenty of time to figure it out... I'll probably be looking at the larger players to see where this goes, and what clarifying information the government put out next year.
This is our toaster, we call him Mr Toot... for obvious reasons.
Another week... another set of nerd graphs!
Some of the variations in CPU usage in the last week correlate with upgrades to the lemmy software. If you look closely, you can see a drop in CPU usage starting late on Thursday, this was when I upgraded to 0.18.1-rc.10 which includes some major DB query optimisations.
The spikey memory utilisation between the 28th of June and the 4th of July were due to a lemmy software issue. It resulted in huge resource consumption for varying amounts of time, before returning to normal. This specific issue seems to have been resolved in one of the upates this week.
The unusually high traffic levels between the 1st and 4th were caused by a large community icon.. 14MB. Thankfully, Cloudflare saved us by caching this file and saved us ~800GB of traffic on this file alone before it was shrunk to a more appropriate size 🙂
As expected, disk space is being used up at a relatively consistent rate. The drop on the 28th was when cached object storage images were deleted due to a server reboot that was required when adding 2 vCPUs. I have a scheduled job that runs every hour to delete cached objects older than a threshold. As the disk used by the cache increases (or the disk gets closer to being full), I'll decrease the age threshold.
Cloudflare caching is saving us a lot of egress traffic. It is saving us less since the 14MB "icon" was shrunk, however this shows the value in the service. Even now, it is saving ~40GB of egress traffic per day.
Everything is looking pretty good right now, thanks mostly to the ongoing improvements in lemmy making things run more smoothly. In the next week or so I'll be upgrading the storage on the VPS to cater for the ever increasing database. Admins are already discussing ways we can manage the increasing storage requirements, I'm optimistic we'll have good solutions before we hit a VPS scaling limit.
As always, happy to answer any questions.
If you're posting as an uninvolved third party about an event you're interested in, it's not an ad.
As for spam, if you post a ridiculous volume about such events it could be construed as spamming. So long as you're manually posting this shouldn't an issue.
Basically, don't be trying to sell anything or abusing the AZ site.