jerry

joined 2 years ago
MODERATOR OF
 

My apologies for how long the site was down. Just about everything that could go wrong did, but I think we are working now...

 

In an effort to reduce costs due to the exchange rate and declining donations, I’m going to be consolidating Fedia.io onto another existing server so I can decommission and return the server that it is running on. I’ll be turning off unauthenticated access to content again because that drastically reduces the bandwidth/compute load that Fedia uses. I am not sure on timing yet - likely around 2PM ET/6PM UTC. I’ll plan for an hour downtime but I’m hoping it should only be a few minutes if I do things correctly.

[–] jerry@fedia.io 1 points 2 weeks ago

No kicking on my side. I’ll see if there was anything going on in the logs yesterday.

[–] jerry@fedia.io 4 points 1 month ago

@BeAware@mementomori.social fedia.io was being swamped with crawlers from thousands of IPs causing the site to grind to a halt and periodically crash. I had to limit access to only logged in users while I try to sort out a better way to manage all those crawlers.

@Blaze@lemmy.dbzer0.com

 

I have some time to babysit the server now and so reenabled anonymous access. I've also removed the prior ASN blocks, but may add those back in as needed based on various AI datacenter crawling.

[–] jerry@fedia.io 4 points 1 month ago (1 children)

I understand. I have tried hard to make fedia.io work - it’s been far and away the most challenging app I’ve managed (note: the problems are all legacy kbin issues, the mbin team has been nothing but amazing). I am stuck in a difficult position - the site isn’t useful if I keep it locked down like it is now, and the site is super slow/requires constant attention if I make it open. I’ll have to assess my options and decide what the future for fedia is

[–] jerry@fedia.io 2 points 1 month ago (1 children)

Apologies for the delay, but this is fixed now

[–] jerry@fedia.io 1 points 1 month ago

Ohh - that is possible. I will check when I get back to my computer.

[–] jerry@fedia.io 6 points 1 month ago

I will add that to the donation page

[–] jerry@fedia.io 3 points 1 month ago

You and the mbin team continues to amaze me. Thank you so much!

[–] jerry@fedia.io 9 points 1 month ago (2 children)

It’s an application level ddos. Blocking anonymous access helped a bunch, but I am still getting about 5-10 login requests per second from hundreds of different IPs

[–] jerry@fedia.io 8 points 1 month ago

Thanks. Just trying to give people some alternatives

[–] jerry@fedia.io 10 points 1 month ago

We think it’s a csrf prevention measure in the php symphony library that creates a lot of database calls.

[–] jerry@fedia.io 7 points 1 month ago (1 children)

Not really. We have to accept incoming connections from thousands of other fediverse instances that would be blocked by that.

 

Hi all. Fedia.io has for a long time been subject to ddos attacks, including many that are "accidental", caused by myriad scrapers constantly hammering the site. I gave up on trying to play whack-a-mole with blocking them based on IP address (they do not honor robots.txt and do not use a conspicuous user agent string) since I was inadvertently blocking some legitimate users. So, I've restricted access to the content of fedia.io to only those that are logged in. That will mean we don't show up in search engines and whatnot, which for some will considered a good thing and will likely cause others to leave.

There is a remaining problem related to the login form. Calls to the login page are breathtakingly expensive, computationally speaking, and so I also have a script that monitors unusual numbers of calls to that form and blocks at the firewall any offenders. I strongly suspect I'm catching some legitimate users with this too, and so I continue to try to tune it, but it's maddening, y'all.

These issues have been causing performance problems for everyone (despite the fedia.io app running on a dedicated 96 core, 256GB server with nvme disks), and became unavailable for certain people that accidentally tripped various thresholds. I'm hoping most of this is resolved now.

Thanks for the patience.

 

I just saw this: https://every.to/p/the-disappearance-of-an-internet-domain

I have no idea if it's real, but if it is, that will be most unfortunate

 

testing federation

 
2
2 way cat scan (media.fedia.io)
 
view more: next ›