Db0

I see tankies keep trying to argue with people about “Actually Existing Socialist” states like USSR and China and try to argue with me or others about how “they were good actually”. It’s bad enough when most of their arguments are whataboutism, but it grinds my gears when I hear then prattle on about all the statistically significant material improvements the life of the people received. It’s like listening to a terminally-liberal prattle on about how “statistically, the life quality actually increased under capitalism”.

Why is this bothering me so much? Because tankies completely suppress the freedom aspects of those states. Sure the improvements in life quality in those nations improved compared to the feudal/agrarian societies they had before, much like liberal capitalism also improved those same metrics.

But the freedom of the populace barely improved improved whatsoever because that freedom is anathema to authoritarian regimes. When anarchists talk about our ideal society, we mean both positive and negative freedom together together. It’s not enough if your health expectancy is increased and infant mortality is reduced, if you have to constantly fear the secret police knocking on your door. It’s not enough to have food on your plate, when the state determines what you can create and where you can work. It’s not enough to get a free car and internet, if your family member got shipped to the concentration camp for criticizing the movement leaders online.

Tankies explicitly avoid this. They are desperate to argue that “authoritarianism is not a thing actually”, hilariously and endlessly promoting the worst socialist essay ever written to justify this. But authoritarianism is very much the crux of the problem here. A society with a hierarchical structure like capitalism or marxism-leninism (i.e. state capitalism) can never be good. It might be better than other states, but it will only get worse and worse as power concentrates to fewer hands and the grip of authority tightens the more control slips through their fingers.

We keep seeing this historically both in liberal and ML states. Clearly material quality of life is not enough to justify the system, or even be stable long-term, when actual human liberty is the sacrifice for it.

Someone made the comment about the assassination of the United Healthcare CEO and mentioned that the clients to private security forces are going to skyrocket. This is true.

It made me think of how much companies like these have profited from lobbying the government to remove their social contributions (i.e. taxes) while also being directly responsible for destroying those social safeties themselves.

The companies constantly lobby the government to reduce or remove their tax burden while retaining their state protections. But they don’t recognize that the more their actions erode the life of the working class, the more the social contract people accept to not take matters into their own hands is discarded.

As such, you start seeing things like assassinations and kidnappings, which in turn force the rich to use the money they saved by not paying taxes, to pay for private security instead.

This naturally leads to a more and more polarized society where the rich live in increasingly isolated and defended enclaves, while the proles live outside in slums and favelas. Sometimes directly next to each other, as this iconic photos from Sao Paolo exemplifies.

Of course, eventually, the dissolution of the social contract is going to make even this insufficient. More and more wealth will need to be used merely to protect their life and property, once the state has been sufficiently defunded, until at some point, you own private security will be either so powerful as to become a de-facto state, or they will turn themselves against the rich and claim their wealth for themselves.

Under capitalism, the taxman always gets their due.

I’ve had the ActivityPub plugin active on this blog for a while now and it’s been happily federating to mastodon for just as long. However it never worked on lemmy, and I always assumed it was just not set for it and was primarily focused on microblogging since lemmy was not even mentioned in the supported software.

This was until one of the lemmy developers contacted me, having been informed by a member of our lemmy instance that dbzer0.com was not properly configured for lemmy. I was perplexed of course because I didn’t really do any customization on the wordpress plugin whatsoever. I just used whatever defaults it came with.

Through some back and forth between the developers and me, I eventually started experimenting with the plugin settings, trying to see if any of them would make it behave in a way that lemmy could understand, until one of the options finally did the trick.

As a result, this WordPress blog is now happily existing as a lemmy community [email protected]

’tis a bit of a silly community name, but it works.

Unfortunately the previous posts on this blog are not retrieved automatically, so you won’t be able to see them or their comments in the community, but one can search for a blog url in lemmy and it will discover it and open it for comments. Any comments posted there should also appear as comments under the posts here which is pretty neat!

Example

So if you’re on lemmy or piefed, just visit its community from your own instance and subscribe to it, and new blogposts will appear directly in your lemmy feed. I love apub!

Many thanks to both pferfferle (the apub plugin developer) and the lemmy developers who looked into this!

leave a comment (from lemmy?) to let me know what you think.

Quote of the day: Snakes

Liked the modern Vikings series? Then maybe check this out for hilarious accuracy:

As the God of Alanis Morissette would have it, the man famous for his snake-proof pants gets his naked ass bit to death by snakes.

Grayson Del Faro

via Rusty Bertrand

From the first moment I first went online in 1996, forums were the main place to hang out. In fact the very first thing I did was join an online forum run by the Greek magazine “PC Master” so I could directly to my favourite game reviewers (for me it was Tsourinakis, for those old enough to remember).

Whoever didn’t like the real-time nature of the IRC livechat, forums were all the rage and I admit they had a wonderful charm for the upcoming teenager who wanted to express themselves with fancy signatures and some name recognition for their antics. Each forum was a wonderful microcosm, a little community of people with a similar hobby and/or mind-frame.

BBcode-style forums took the web 1.0 internet by storm and I remember I had to juggle dozens of accounts, one for for each one I was interacting with. Basically, one for each video game (or video game publisher) I was playing, plus some Linux distros, hobbies, politics and the like. It was a wonderful mess.

But a mess it was, and if the dozens of accounts and constant context switching barely enough to handle for an PC nerd like myself, I can only imagine how impenetrable it was for the less tech-savvy. Of course, for people like me this was an added benefit, since it kept the “normies” out and avoided the “Eternal September” in our little communities.

However the demand for places accessible for everyone to discuss was not missing, it was just unfulfilled. So as soon as Web 2.0 took over with the massive walled gardens of MySpace, Facebook, Twitter and so on, that demand manifested and the ability for anyone to create and run a forum within those spaces regardless of technical competency or BBcode knowledge, spawned thousands of little communities.

Soon after Digg and then Reddit came out, and after the self-inflicted implosion of Digg, Reddit along with Facebook became the de-facto spot to create and nurture new async-discussion communities, once they added the functionality for everyone to create one and run it as they wanted.

But the previously existing BBcode forums still existed and were very well established. Places like Something Awful had such strong communities that they resisted the pull of these corporate walled gardens for a long time. But eventually, they all more or less succumbed to the pressure and their members had an exodus. What happened?

I’m not a researcher, but I was there from the start and I saw the same process play out multiple times in the old forums I used to be in. Accessibility and convenience won.

There’s a few things I attribute this to.

1. The executive costs to create a new forum account is very high. Every time you want to join one, you need to go through making a username (often trying to find one that’s not taken, so now you have to juggle multiple usernames as well), new password, captchas, email verifications, application forms, review periods, lurker wait times and so on. It’s a whole thing and it’s frustrating to do every time. Even for someone like me who has gone through this process multiple times, I would internally groan for having to do it all over again.
2. Keeping up to date was a lot of work. Every time I wanted to keep up to date with all my topics, I had to open new tabs for each of my forums and look at what’s new is going on. The fact that most of the forums didn’t have threaded discussions and just floated old discussions with new replies to the top didn’t help at all (“thread necromancy” was a big netiquette faux-pas). Eventually most forums added RSS feeds, but not only were most people not technical enough to utilize RSS efficiently (even I struggled), but often the RSS was not implemented in a way that was efficient to use.
3. Discoverability was too onerous. Because of (1) Many people preferred to just hang out in one massive forum, and just beg or demand new forum topics to be added for their interests so they wouldn’t have to register, or learn other forum software and interact with foreign communities. This is how massive “anything goes” forums like Something Awful started, and this also started impacting other massive forums like RPGnet who slowly but surely expanded to many more topics. Hell almost every forum I remember has politics and/or “out of topic” sections for people to talk without disrupting the main topics because people couldn’t stop themselves.
   And where the forum admins didn’t open new subject areas, the bottom-up pressure demanded that solutions be invented in the current paradigm. This is how you ended up with immortal threads, thousands of pages deep for one subject, or regular mega-threads and so on. Internet life found a way.
4. Forum admins and staff were the same petty dictators they always were and always will be. Personality cults and good ole boys clubs abounded. People were established and woe to anyone who didn’t know enough to respect it, goddammit! I run into such situations more than once, even blogged about it back in the day. But it was an expected part of the setup, so people tolerated it because, well what else will you do? Run your own forum? Who has the time and knowledge for that? And even if you did, would anyone even join you?

And so, this was the paradigm we all lived in. People just declared this is how it had to be and never considered any proper interactivity between forums as worth the effort. In fact, one would be heavily ridiculed and shunned for even suggesting such blasphemous concepts

That is, until Facebook and Reddit made it possible for everyone to run their own little fief and upended everything we knew. By adding forum functionality into a central location, and then allowing everyone to create one for any topic, they immediately solved so many of these issues.

1. The executive cost to join a new topic is very low. One already has an account on Reddit and/or Facebook. All they have to do is press a button on the subreddit, group they want to join. At worst they might need to pass an approval, but they get to keep the same account, password and so on. Sure you might need to juggle 1-3 accounts for your main spaces (Reddit, Facebook, Discord), but that’s so much easier than 12 or more.
2. Keeping up to date is built-in. Reddit subscriptions allows one a personalized homepage, Facebook just gives you your own feed, discord shows you where there’s activity and so on. Of course the corporate enshittification of those services means that you’re getting more and more ads along masquerading as actual content and invisible algorithms are feeding you ragebait and fearbait to get you to keep interacting at the cost of your mental and social health, but that is invisible for most users so it doesn’t turn them off.
3. Discoverability is easy. Facebook randomly might show you content from groups you’re not in, shared by others. Reddit’s /r/all feed showed posts from topics you might not even know existed and people are quick to link to relevant subreddits. Every project has its own discord server link and so on.

The fourth forum problem of course was and can never be solved. There will always be sad little kings of small sad little hills. However solving 1-3 meant that the power of those abusing their power as moderators was massively diminished as one could just set up a new forum in a couple of minutes and if there was enough power abuse, whole communities would abandon the old space and move to the new one. This wasn’t perfect of course, as in Reddit, only one person could squat one specific subreddit, but as seen with successful transitions from /r/marijuana to /r/trees, given enough blow-back, it can certainly be achieved.

And the final cherry on top is that places like Reddit and discord are just…easier to use. Ain’t nobody who likes learning or using BBcode on 20-year-old software. Markdown became the norm for a reason due to how natural it is to use. Add to that less restrictions on uploads (file size, image size etc) and fancier interfaces with threaded discussions, emoji reactions and so on, and you get a lot of people using the service instead of trying to use the service. There are of course newer and better forum software like the excellent Discourse, but sadly that came in a bit too late to change momentum.

So while forums never went away, people just stopped using them, first slowly but accelerating as time passed. People banned just wouldn’t bother to create new accounts all over again when they already had a Facebook account. People who wanted to discuss a new topic wouldn’t bother with immortal mega-threads when they could just join or make a subreddit instead. It was a slow-burn that was impossible to stop once started.

10-15 years after Reddit started, it was all but over for forums. Now when someone wants to discuss a new topic, they don’t bother to even google for an appropriate forum (not that terminally enshittified search engines would find one anyway). They just search Reddit or Facebook, or ask in their discord servers for a link.

I admit, I was an immediate convert since Reddit added custom communities. I created and/or run some big ones back in the day, because I was naive about the corporate nature of Reddit and thought it was “one of the good ones”, even though I had already abandoned Facebook much earlier. It was just so much easier to use one Reddit account and have it as my internet homepage, especially once gReader was killed by Google.

But of course, as these things go, the big corporate gardens couldn’t avoid their nature and eventually once the old web forums were abandoned for good and people had no real alternatives, they started squeezing. What are you gonna do? Set up your own Reddit? Who has the time and knowledge for that? And even if you did, would anyone even join you?

Nowadays, I hear a lot of people say that the alternative to these massive services is to go back to old-school forums. My peeps, that is absurd. Nobody wants to go back to that clusterfuck I just described. The grognards who suggest this are either some of the lucky ones who used to be in the “in-crowd” in some big forums and miss the community and power they had, or they are so scarred by having to work in that paradigm, that they practically feel more comfortable in it.

No the answer is not anymore an archipelago of little fiefdoms. 1-3 forbid it! If we want to escape the greedy little fingers of u/spez and Zuckeberg, the only reasonable solution is moving forward is activitypub federated software.

We have already lemmy, piefed, and mbin, who already fulfill the role of forums, where everyone can run their own community, while at the same time solving for 1-3 above! Even Discourse understood this and started adding apub integration (although I think they should be focusing on threadiverse interoperability rather than not microblogging.)

Imagine a massive old-school forum like RPGnet migrating to a federated software and immediately allow their massive community access to the rest of the threadiverse without having to go through new accounts and so on, while everyone else gets access to the treasure trove of discussions and reviews they have. It’s a win-win for everyone and a loss for the profiteers of our social media presence.

Not only do federated forums solve for the pain points I described above, but they add a lot of other advantages as well. For example we now have way less single points of failure, as the abandonment of a federated instance doesn’t lose its content which continues living in the caches of the others who knew about it and makes it much easier for people to migrate from one lemmy instance to another due to common software and import/export functionalities. There’s a lot of other benefits, like common sysadmin support channels, support services like fediseer and so on.

These days, I see federated forums as the only way forward and I’m optimistic of the path forward. I think Reddit is a dead site running and the only way they have to go is down. I know we have our own challenges to face, but I place far more trust in the FOSS commons than I do in corporate overlords.

It’s been cooking for a while but we can now officially announce that the Flux.1-schnell is finally available on the AI Horde!

Flux is one of the most exciting Generative AI text2image models to come out this year, from a team of ex-stability.ai developers, and seemingly consumed all the attention of the GenAI enthusiasts overnight. It’s a very powerful model but as a downside it requires a significantly more powerful PC to run than the more popular SDXL models were until now.

The model available on the horde is primarily the fp8 compact version we took from civitAI, as it simplifies the amount of downloads we have to juggle.

I was really eager to offer the flux.1-dev version as well, as it has a lot more LoRas available and is a bit more versatile, but sadly its license contains some requirements which do not appear to allow a service like the AI Horde to provide it, even though it’s a completely free service for everyone. However we have reached to the Black Forest Labs via email to ask for clarification or exception for this and will let you all know if we hear back.

To use it, head over to Artbot or Lucid Creations and simply select the Flux.1-Schnell fp8 (Compact) model for your generation. However keep in mind that this model is quite different from the Stable Diffusion models you’re used to until now, so you need to adjust your request as following to get good results:

• Set sampler to k_euler
• Set steps between 4 and 8 (4 is enough for most images)
• Set cfg to 1

Also keep in mind that the model won’t use the negative prompt. Instead it benefits massively from using native speech to describe what you want to draw instead of a tag-based approach.

If you are running a dreamer worker make sure you check our instructions in our discord channel on the best settings to run flux. This is a big model, so GPU with 16G-24G VRAM are the best for running it at a decent speed and we could use all the help we can get.

If you are making integrations with the AI Horde, make sure you use the flux branch of the image reference repository until it’s merged into main on the end of the month, if you’re using it to retrieve model requirements.

Along with flux, tazlin has done some amazing work on adding the latest version of comfy and improving the stability and speed of the worker. I mean, just look at this changelog! This also greatly improves our support for AMD cards. They might not be as fast as nvidia, but they should work!

Finally we’ve added some improvements on the horde itself to allow slower workers to offer models. If you have an older GPU which often gets timed out and put on maintenance on the Horde due to speed, you can now set yourself as an extra_slow_worker which will extend your TTL and will be used by things like automated bots, or apps like that sweet AI Wallpaper Changer.

Finally, I’ve also extended our deployments ansible collection so that if you use a Linux system, you can easily deploy any number of reGen workers, even multiple in the same server to take advantage of multiple GPUs. It will even deploy the AMD drivers for you if you want it. With this I am continuing to extend the tools to allow more people to run the AI Horde infrastructure on their own.

We hope the existence of flux on the Horde will allow unlimited creativity from people who want access to the model but don’t have the hardware to run it. Now more than ever, people with mid-range GPUs can offer what they can run, such as SDXL or SD 1.5 models, and in turn, benefit from others offering the larger models like flux and we all benefit through mutual aid!

Enjoy!

A couple days ago, someone posted on /0 (the meta community for the Divisions by zero) that the incoming federation from lemmy.world (the largest lemmy instance by an order of magnitude) is malfunctioning. Alarmed, I started digging in, since a federation problem with lemmy.world will massively affect the content my community can see.

As always my first stop was the Lemmy General Chat on Matrix where I asked the lemmy.world admins if this appears to be something on their end. To their credit both their lead infra admin and the owner himself jumped in to assist me, changing their sync settings, adding custom DNS entries and so on. Nothing seemed to help.

But the problem is must still be somewhere in lemmy.world I thought. It’s the only instance where this is happening and they upgraded to 0.19.3 recently, so something must have broken. But wait, this didn’t start immediately after the upgrade. Someone pointed out this very useful federation status page, which kinda point that the problem is only on lemmy.world.

Not quite, other big instances like lemmy.ml and lemm.ee were not having any issues with federation with lemmy.world (even though 2 dozen others like lemmy.pt were), and they are as big if not bigger than lemmy.dbzer0.com. A problem originating from lemmy.world cannot be possibly affecting only some specific instances. To make matters worse, both me and lemmy.ml are using the same host (OVH), so I couldn’t even blame my hosting provider somehow.

So obviously the main culprit it somewhere in my backend, right? Well, maybe. Problem is, none of the components of my infrastructure were overloaded, everything sitting between 5-15% utilization. Nothing to even worry about.

OK, so first I need to make sure it’s not a network issue somehow specifically between me and lemmy.world specifically. I know OVH gave me a bum floating IP in the past and were completely useless at even understanding that their floating IP was faulty, so I had to stop using it. Maybe there’s some problem with my loadbalancers.

Still, I’m using haproxy, which is nothing if not fast and rock solid. So I didn’t really suspect the software. Rather, maybe it’s a network issue with the LB itself. So first thing I did is double the amount of Loadbalancers in play, by setting my DNS record to point to my secondary LB at the same time. This should lessen the amount of traffic hitting my LB and even take them at a completely different VM, and thus point if the problem is on the haproxy side. Sadly, this didn’t improve things at all.

OK so next step, I checked how long a request takes to return from the backend after haproxy sends it over. The results were not good.

I don’t blame you if you cannot read this, but what this basically says is that a request hitting a POST on my /inbox, took between 0.8 and 1.2 seconds. This is bad! This is supposed to be a tiny payload to tell you an event happened on another instance, it should be practically instant.

Even more weird, this is affecting all instances, not just lemmy.world. So this is clearly a problem on my end, but it also confused me. Why am I not having troubles with other instances? The answer came when I was informed that 0.19.3 added a brand new, special new federation queue.

You see, the old versions of lemmy used to send all federation actions over as soon as they received them. Fire and forget style. This naturally lead to federation events being dropped due to a myriad of issues, like network, downtimes, gremlins etc. So you would lose posts, comments and votes, and you would (probably) never realize.

The new queue added order to this madness, by making each instance send its requests serially. A request would be sent again and again until it succeeded. And the next one would only be sent if the previous one was done. This is great for instances not experiencing issues like mine. You see, at this point, I was processing 1 incoming federation request per second approximately, while lemmy.world was sending around 3. Even worse, I would occasionally timeout as well by exceeding 10 seconds to process, causing 2 more seconds or wait time.

Unlike lemmy.world, other federating instances to mine didn’t have nearly as much activity, so 1 per second was enough to keep up to sync with them. This explained why I seemingly was only affected by lemmy.world and nobody else. I was somewhat slow, but only slow enough to notice if the source had too much traffic.

OK, we know the “what”, now we needed to know the “why”.

At this point I’m starting to suspect something is going on my Database. So I have to start digging into stuff I’m really not that familiar. This is where the story gets quite frustrating, because there’s just not a lot of admins in the chat who know much about the DB stuff of lemmy internals. So I would ask a question, or provide logs, and then had to wait sometimes hours for a reply. Fortunately both sunaurus from lemm.ee and phiresky were around, who could review some of my queries.

Still, I had to know enough sql to craft and finetune those queries myself and how to enable things like pg_stat_activity etc.

Through trial and error we did discover that some insert/update queries were taking a bit too much time to do their thing, which could mean that we were I/O bound. Easy fix, disable synchronous_commit, sacrificing some safety for speed. Those slow queries went away, but the problem remained the same. WTF?!

There was nothing else clearly slow in the DB, so there was nothing more we could do there. So my next thought was, maybe it’s a networking issue between my loadbalancers and my backend. OK so I needed to remove that from the equation. I set up a haproxy directly on top of my backend which would allow me to go through the loopback interface and have 0 latency. For this I had to ask the lemmy.world admins to kindly add lemmy.dbzer0.com directly to their /etc/hosts file so they alone would hit my local haproxy.

No change whatsoever!

At this point I’m starting to lose my mind. It’s not networking between my LB and my backend, and it’s not the DB. It has to be the backend. But it’s not under any load and there’s no errors. Well, not quite. There’s some “INFO” logs which refer to lost connections, or unexpected errors, but nobody in the chat seems to worry about them.

Right, that must mean the problem is networking between my backend and my database, right? Unlike most lemmy instances, I keep my lemmy DB and my backend separated. Also, the DB has a limited amount of connections and lemmy backend itself limits itself to a small pool of connections. Maybe I run out of connections because of slow queries?

OK let’s increase that to a couple thousands and see what happens.

Nothing happens, that’s what happens. Same 1 per second requests.

As I’m spiraling more and more towards madness, and the chat is running out of suggestions, sunaurus suggests that he adds some extra debugging to lemmy and I will run that to try and figure out which DB query is losing time. Great idea. Problem is, I have to compile lemmy from scratch to do that. I’ve never done that before. Not only that, I barely know how to use docker in the first place!

Alright, nothing else I can do, got to bite that bullet. So I clone the lemmy backend and while waiting for sunaurus to come online, I start hacking at it to figure out how to make it compile a docker lemmy backend from scratch. I run into immediate crashes and despair. Fortunately nutomic (one of the core devs) walked by and told me the git commands to run to fix it, so I could proceed in cooking my very first lemmy container. Then nutomic helped me realize I don’t need to set up a whole online repo to transfer my docker container. The more you know…

Alright, so I cooked a container and plugged it onto a whole separate docker infra, which is only connected to the lemmy.world loadbalancer, so I can remove all other logs from anything but federation requests. So far so good.

Well, not quite, unfortunately I forgot that the “main” branch of lemmy is actually the development branch and has untested code in there. So when I was testing my custom docker deployment, I migrated my DB to whatever the experimental schema is on main. Whoops!

OK, nothing seemingly broke. Problem for a different day? No, just foreshadowing.

Finally sunaurus comes back online and gives me a debug fork. I eagerly compile and deploy it on prod and then send some logs to sunaurus. We were expecting we’d see 1 or 2 queries that were struggling, so maybe a bad lock situation somewhere. We did not expect we’d see ALL queries, including the most simple query such as lookup a language, take 100ms or more! That can’t be good!

Sunaurus connects the dots and asks the pertinent question: “Is your DB close to the Backend, geographically?”

Well, “Yes”, I reply, “I got them in the same datacenter”. “Can you ping?” he asks.

OK, I ping. 25ms. That’s good right? Well, in isolation, that’s great. When it’s not so great is when talking about backend-to-DB communication! This like 1000s km distance.

You see, typically a loadbalancer just makes one request to the backend and gets one reply, so a 25ms roundtrip is nothing. However a backend is talking to the DB a lot. In this instance, for every incoming federation action the backend does like 20 database calls, to verify and submit. Multiply each of these by 25+25 roundtrip and you got 1000ms extra before any actual processing on the DB!

But how did this happen? I’m convinced all my servers are in the same geographic area. So I go to my provider panel and check. Nope, all my server BUT the backend are in the same geographic area. My backend happens to be around 2000 km away. Whoops!

Turns out, when I was migrating my backend back in the day I run into performance issues, I failed to pay attention to that little geographic detail. Nevertheless It all worked perfectly well until this specific set of circumstances where the biggest lemmy instance upgraded to 0.19.3 which caused a serial federation, which my slow-ass connection couldn’t keep up. In the past, I would just get flooded by sync requests by lemmy.world as they came. I would be slow, but I’d process them eventually. Now, the problem became obvious.

Alright, it’s time to put up my sleeves and it’s migrate servers! Thank fuck I have everyone written in Ansible as code, so the migration was relatively painless (other than slapping Debian 12 around to let me do fucking docker-compose operations with python, goddamnit!)

A couple of hours later, I had migrated my backend to the same DC as the Database, and as expected, suddenly my ingestion rate for federation actions was in the order of 50ms, instead of 1000ms. This means I could ingest closer to 20 actions per sec from lemmy.world and it was getting just 3/s new from its userbase. Finally we started catching up!

All in all, this has been a fairly frustrating experience and I can’t imagine anyone who’s not doing IT Infrastructure as their day job being able to solve this. As helpful as the other lemmy admins were, they were relying a lot on me knowing my shit around Linux, networking, docker and postgresql at the same time. I had to do extended DB analysis, fork repositories, compile docker containers from scratch and deploy them ad-hoc etc. Someone who just wants to host a lemmy server would give up way earlier than this.

For me, a very stressing component was the lack of replies in the chat. I would sometimes write pages of debug logs, and there was no reply from anyone for 6 hours or more. It gave me the impression that nobody had any clue what to do to help me and I was on my own. In fact, if it wasn’t for sunaurus specifically, who had enough Infrastructure, Rust and DB chops to get an insight out of where it was all going wrong, I would probably still be out there, pulling my hair.

As someone hosting a service like this, especially when it has 12K people in it, this is very scary! While 2 lemmy core developers were in the chat, the help they provided was very limited overall and this session mostly relied on my own skills to troubleshoot.

This reinforced in my mind that as much as I like the idea of lemmy (or any of the other threadiverse SW), this is only something experts should try hosting. Sadly, this will lead to more centralization of the lemmy community to few big servers instead of many small ones, but given the nature of problems one can encounter and the lack of support to fix them if they’re not experts, I don’t see an option.

Fortunately this saga ended and we’re now fully up to sync with lemmy.world. Ended? Not quite. You see today I realized I couldn’t upload images on my instance anymore. Remember when I started the development instance of lemmy by mistake from main? Welp that broke them. So I had to also learn how to downgrade a lemmy instance as well. Fortunately sunaurus had my back on this as well!

To spare some people the pain, I’ve sent a PR to the lemmy docs to expand the documentation for building docker containers and doing troubleshooting. My pain is your gain.

This also gave me an insight about how the federation of lemmy will eventually break when a single server (say, lemmy.world) grows big enough to start overwhelming even servers who are not badly setup like mine was. I have some ideas to work around some of this so I plan to a suggestion on how to become more future proof, which would incidentally prevent the same issue which happened to me in the first place.

In the meantime, enjoy the Divisions by zero, which as a result of the migration should now feel massively faster as well!

I visit Reddit all the time. And I visited Digg before that. In fact I was hooked to this mode of operation since Digg. Suffice to say, something about link aggregation tickles my ADHD brain just right.

However with the recent blackout of a big part of reddit, I decided to start my own Lemmy instance and use Lemmy primarily instead through it. Since I’ve started this experiment, I feel any urge to visit Reddit for my “fix” less and less. I have some thoughts about that.

In Twitter Vs Mastodon AKA “micro-blogging”, The value was in the specific people one followed which made it way harder to switch services because one was help back by other people. I.e. the people kept each other locked-in. Similar to how Facebook keeps everyone locked-in their walled garden because it’s the only social media their parents and grandparents managed to learn to use.

In Reddit however, the value is all about the specific forums, or “subreddits”, in lingo. The specific people one was talking to, never really mattered. What was important was the overall engagement general sense of shared-interest. This has always been the core strength of Reddit, and its early pioneers like Aaron Schwartz understood that.

This is why the minimalist reddit of old, managed to dethrone Digg when the latter decided that its core principles wasn’t user-curated content, but linkspam. The people who migrated into Reddit made what it is today, by creating and nurturing their communities over years.

Any beneficial actions by reddit itself have been either following what the community was already doing (such as adding CSS options or on-boarding the automoderator bot), or forced by bad optics, such as when they were forced to finally ban /r/coontown, /r/fatpeoplehate, /r/jailbait (which their current CEO moderated btw) etc.

The community and the people who run the subreddits have always had to make the minimalist options allowed to them work. They had to develop their own tools and enhancements, such as RES, and Moderator Toolbox, while Reddit couldn’t even provide much requested functionality to counter the known abuses of cross-subreddit raiding.

Instead, Reddit focused on adding useless features nobody asked for like NFT. On the usability, the new look was their push to take the site more towards generic social media network, with friends, follows, awards and avatars, and instead of focusing on their core product: Link aggregation and discussions.

In fact, any action they took, was laser focused on social-media lock-in and extracting wealth and adding features which people didn’t care for, which is why most third party apps simply ignored all that stuff.

Through all this, their valuable communities kept fighting against reddit management’s pushes so that they could do what was right, even if some lost that fight, like /r/AMA which became but a shadow of its former self when the cowardly owners fired their low-level employee leading its success, and scapegoated their then female CEO for it.

Eventually though something had to give, and reddit seems to have realized that their users are too stubborn to simply accept the new paradigm they designed for them where they watch more ads, buy more reddit gold and get addicted to NFTs. And 3rd party apps enabled users to use the valuable part of reddit and skip the enshittification all too easily.

So they had to go. And here we are.

Unfortunately for reddit, since the core value of reddit has always been the links, and the discussions around said links, instead of specific people and a social network around them, it is stunningly easy to jump ship. It doesn’t take a lot to keep a community going on Lemmy instead of Reddit. All it needs is a handful of dedicated people to keep finding and posting links, and the discussions and memes will easily follow.

I don’t need to know that I know the links are coming from Gallowboob, in fact, I never cared who posted the links or started the discussions. Reddit has had the “friends” feature for close to a decade now, and I have “friended” less than a handful of people. There’s literally nothing holding me and most people back except our existing routines.

There is of course still a lot of momentum in reddit communities, and a lot of mods who really don’t want to lose their status. Nevertheless, I’m finding I’m not actually missing much by staying exclusively on lemmy atm and I see a lot of people are realizing the same thing increasingly fast. The finality of the loss of major apps like Apollo, RIF and Sync has already been the final nail for a lot of people.

This exodus might already be unstoppable unless reddit completely capitulates and goes back on their API plans. But I don’t hold my breath on this.

Feel free to come and hang out at the Divisions by zero lemmy instance btw. We’ll do fun things!

One of the big problems we’ve been fighting against since I created the AI Horde was attempts to use it to generate CSAM. While this technology is very new and there’s a lot of question to answer on whether it even is illegal to generate CSAM for personal use, I erred on the safe side and made it a rule from the start, that the one thing that is going against the AI Horde, is such generated content without exceptions.

It’s is not a overstatement to say I’ve spend weeks of work-hours on this problem. From adding capabilities for the workers to set their own comfort level through a blacklist and a censorlist and a bunch of other variables, to blocking VPN access, to the massive horde-regex filter that sits before every request and tries to ascertain from the prompt sent whether it intends to generate CSAM or not.

However the biggest problem is not just pedos, it’s is stupid, but cunning pedos! Stupid because they keep trying to use a free service which is recording all their failed attempts without a VPN. Cunning because they keep looking for ways to bypass our filters.

And that’s where the biggest problem lied until now. The regex filter is based on language which is not only flexible about the same concept, but very frustratingly, the AI is capable of understanding multiple typos of various words and other languages perfectly well. This strains what I can achieve with regex to the breaking point, and led to a cat&mouse game where dedicated pedos kept trying to bypass the filter using typos and translations, and I kept expanding the regex.

But it was inherently a losing game which was wasting an incredible amount of my time, so I needed to find a more robust approach. My new solution was to onboard image interrogation capability to the worker code. The way I go about this is by using image2text, AKA image interrogation. It’s basically AI Model which you feed an image and number of words or sentences and it will tell you how how well each of those words is represented in that image.

So what we’ve started doing is that every AI Horde Worker will now automatically scan every image they generate with clip and look for a number of words. Some of them are looking for underage context, while some of them are looking for lewd context. The trick is detecting one, or the other context is OK. You’re allowed to draw children, and you’re allowed to draw porn. It’s when these two combines that we filter goes into effect and censors the image!

But this is not even the whole plan. While the clip scanning on its own is fairly accurate, I further tweaked my approach by taking into account things like the value of other words interrogated. For example I noticed that when looking for “infant” in the generated image pregnant women would also have a very high rating for it, causing the csam-filter to censor out naked pregnant women consistently. My solution was then to also interrogate for “pregnant” and if the likelihood of that is very high, adjust the threshold to hit infant higher.

The second trick I did was to also utilize the prompt. A lot of pedos were trying to bypass my filters (which were looking for things like “young”, “child” etc) by not using those words, and instead specifying “old”, “mature” etc in the negative prompt. Effectively going the long route around to make Stable Diffusion draw children without explicitly telling it to. This was downright impossible to block using pure regex without causing a lot of false positives or an incredible amount of regex crafting.

So I implemented a little judo-trick instead. My new CSAM filter now also scans prompt and negative prompt for some words using regex and if they exist, also slightly adjusts the interrogated words based on the author intended. So let’s say the author used “old” in the negative prompt, this will automatically cause the “child” weight to increase by 0.05. This may not sound by a lot, but most words tend to variate from 0.13 to 0.22, so it’s actually has a significant chance to push a borderline word (which it would be at a successful CSAM) over the top. This converts the true/false result of a regex query, into a fine-grained approach, where each regex hit reduces the detection threshold only slightly, allowing non-CSAM images to remain unaffected (since the weight of the interrogated word would start low) while making more likely to catch the intended results.

Now the above is not the perfect description of what I’m doing, in the aim of keeping things understandable for the layperson, but if you want to see the exact implementation you can always look at my code directly (and suggest improvements 😉 ).

In my tests, the new filter has fairly great accuracy with very few false positives, mostly around anime which makes every woman look extraordinarily young as a matter of fact. But in any case, with the amount of images the horde generates I’ll have plenty of time to continue tweaking and maybe craft more specific filter for the models of each type (realistic, anime, furry etc)

Of course I can never expects this to be perfect, but that was never the idea. No such filter can ever catch everything, but what my hope is that this filter, along with my other countermeasures like the regex filter, will have enough of a detection rate to frustrate even the most dedicated pedos off of the platform.

Image by anarchosyn via Flickr

As an Epicurean, I require very little to be content: Food, Shelter, Friends and the absence of pain. All these things have always been generally easy to achieve and as such they are what each person should be able to have. The fact that so many do not is a telling problem of the disfunctionality of our society.

One could ask: “As an Epicurean, why do you care what others have? After all, if you can achieve a state of ataraxia why should you care if others do the same?”. This is really a moral issue and should be looked in this light.

The question is, how do I go from the descriptive “My only needs are those which bring me in ataraxia” to the prescriptive “Everyone should be able to fulfill the needs that bring them in a state of ataraxia“. To go there, we first need to look at my reasons for doing so.

1. The more people that desire that others are achieving ataraxia, the more likely it is that I will be able to achieve and sustain it through their combined efforts.
2. Achieving ataraxia allows people to work on achieving the rest of their desires. Since I’m trying to make it so that one of those desires is that everyone is achieving ataraxia, then this helps spread this desire as well as happiness which comes from being in this state.

From these we can see that I have reasons to promote the desire (i.e. it is considered good) that people should be able to fulfill the needs that bring them in a state of ataraxia. It becomes a moral value.

So how does this lead to Communism? Well, Communism has the ideological proposition that everyone should be producing according to their abilities and receiving according to their needs. By itself, the second part of the sentence is not very descriptive as anyone can claim the most extraordinary things as needs. However through the lenses of Epicurism, the needs transform to something objective: The things one needs to be in a state of ataraxia.

Communism then conflates exactly with the moral value I have reached via Epicurism. Each of us should be striving to the best of our abilities to help others fulfil their needs. And since the needs one has on average ((adding the cost of medicine which are more resource intensive but also much smaller in production scale than food)) are the very basic and most easy to create, the effort we would require from each of us for this to be achieved would be minimal.

Of course Communism is more than a ideological proposition. It also proposes the way a society would be organized (Classless & Staleless) which also follow from Epicurism since authority and inequality either lead to emotional pain or to the increased cost of basic needs, making them opposed to the moral value I explained above.

Now to be accurate, I never really moved towards the libertarian socialist quadrant because I looked at the subject philosophically, but rather because intuitively, for someone with an Epicurean mindframe, the concepts of Anarchism/Socialism/Communism fit very well to my moral values.

Only later did it occur to me how much one leads to the other and the dialectic relationship between them. As much as the Epicurean subconsiously espouses the libertarian socialist mindframe, so does the consistent libertarian socialist require an Epicurean thinking to avoid sliding into authoritarianism or crass individualism (ie Capitalism)

Related articles by Zemanta

• Desire Utilitarianism and Objective Moral Relativism – Part I (atheistethicist.blogspot.com)
• Poor children at higher risk for stunted growth, Montreal team finds (cbc.ca)