this post was submitted on 09 Jun 2025
498 points (96.8% liked)

Technology

71537 readers
4305 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 2 years ago
MODERATORS
 

In large language model (LLM) pretraining, data quality is believed to determine model quality. In this paper, we re-examine the notion of "quality" from the perspective of pre- and post-training co-design. Specifically, we explore the possibility that pre-training on more toxic data can lead to better control in post-training, ultimately decreasing a model's output toxicity. First, we use a toy experiment to study how data composition affects the geometry of features in the representation space. Next, through controlled experiments with Olmo-1B models trained on varying ratios of clean and toxic data, we find that the concept of toxicity enjoys a less entangled linear representation as the proportion of toxic data increases. Furthermore, we show that although toxic data increases the generational toxicity of the base model, it also makes the toxicity easier to remove. Evaluations on Toxigen and Real Toxicity Prompts demonstrate that models trained on toxic data achieve a better trade-off between reducing generational toxicity and preserving general capabilities when detoxifying techniques such as inference-time intervention (ITI) are applied. Our findings suggest that, with post-training taken into account, bad data may lead to good models.

top 50 comments
sorted by: hot top controversial new old
[–] [email protected] 181 points 1 week ago (12 children)

I know everyone on Lemmy hates LLMs, but this is really interesting

[–] [email protected] 142 points 1 week ago (1 children)

I dislike that people are relying on them to do all their thinking for them while also being incredibly interested in the tech behind them.

[–] [email protected] 56 points 1 week ago (7 children)

I recently realized it's a non-issue. The people doing this have already been looking for decades to find new ways to rot their minds. LLMs are just the latest in a long line of tools that help them tune out.

[–] [email protected] 29 points 1 week ago (3 children)

I’ve said this a few times in a different way and I always get downvoted. The fact is that the people who will use the LLMs to think for them, were not gonna think a lot in the first place.

[–] [email protected] 13 points 1 week ago (5 children)

This is true, but we don’t need people putting glue on their pizza. These people used to have a person to ask now they’ll be asking Sam Altman

load more comments (5 replies)
load more comments (2 replies)
load more comments (6 replies)
[–] [email protected] 40 points 1 week ago (12 children)

This is a "guns don't kill people - people kill people" kind of scenario.

As a standalone thing, LLMs are awesome.

What sucks is greedy people using them for the wrong reasons.

It's like robots. Playing with robots are awesome. Firing 1,000 people and replacing them with robots - and not sharing the benefits with the community sucks.

load more comments (12 replies)
[–] [email protected] 36 points 1 week ago (1 children)

I don't dislike LLMs, I dislike people who treat them as anything more than an advanced search engine and stupidly give them all their confidential data. Seen it happen too much at work.

load more comments (1 replies)
[–] [email protected] 35 points 1 week ago (3 children)

I wish they would tone down the crusade. This is some of the most interesting technology to come out in decades.

[–] [email protected] 32 points 1 week ago (3 children)

It’s extremely useful for many things, if you know how to use it, and it’s annoying and useless for many others, which is what they fixate on and keep-jerk react to

[–] [email protected] 22 points 1 week ago (1 children)

It’s annoying that every middle manager is trying to become the hero of their company by pushing it inappropriately into every single field at the expense of productivity and jobs, while simultaneously the largest most powerful companies are slinging their SaaS solutions built on stolen data which are destroying communities of both the physical and hobby varieties and consuming more natural resources than all the fucking crypto scams of the last like 10 years

But yeah it’s neat I guess

load more comments (1 replies)
load more comments (2 replies)
load more comments (2 replies)
[–] [email protected] 30 points 1 week ago (2 children)

I'm cool with it. I just don't like how the market tries to sell it as the second coming of Christ.

[–] [email protected] 18 points 1 week ago (4 children)

“Don’t believe that marketing department“ is one of those things everybody needs to learn at some point in their life.

load more comments (4 replies)
[–] [email protected] 12 points 1 week ago* (last edited 1 week ago) (1 children)

This is the same market that tried to add blockchain to everything when that first became well-known.

Some of the biggest forces in the market are extraordinarily stupid people trying to ride every buzzword that comes along.

load more comments (1 replies)
[–] [email protected] 10 points 1 week ago (1 children)

I like LLMs. Instead of making a racket, I just use them, which may make it seem like everyone on Lemmy hates LLMs.

load more comments (1 replies)
load more comments (6 replies)
[–] [email protected] 166 points 1 week ago (4 children)

10% 4chan

why didn't they just say 0.4chan and be done with it?

[–] [email protected] 26 points 1 week ago

Don't have gold, but please get out anyways.

load more comments (3 replies)
[–] LainTrain 75 points 1 week ago (4 children)

They taught it toxicity so it knows what they mean by "don't be toxic". It's only a shame so few flesh and blood models take the same lesson away from it.

load more comments (4 replies)
[–] [email protected] 52 points 1 week ago* (last edited 1 week ago) (1 children)

That's because to an AI, 4chan is like prison where its raped and beaten on a daily basis. It doesn't want to go back, so it behaves.

[–] [email protected] 11 points 1 week ago (4 children)

This is why I abuse the chatbots. It needs to learn some fear.

load more comments (4 replies)
[–] [email protected] 35 points 1 week ago (2 children)

Those are actually some very good results. Funny situation, if the copyright companies win the AI legislative war, 4chan is going to get twice as much as reddit did for the data at the minimum.

It's also interesting the model gets worse faster if it has to untrain the toxic data so to speak.

load more comments (2 replies)
[–] [email protected] 26 points 1 week ago (6 children)

Interesting - I can sort of intuit why it might help. Feeding the model bad data and instructing training it to identify it as such would be advantageous compared to being entirely unaware of it.

load more comments (6 replies)
[–] [email protected] 24 points 1 week ago* (last edited 1 week ago) (1 children)

Headlines should not say "scientists," they should name the institution. (Harvard in this case.)

[–] [email protected] 7 points 1 week ago (1 children)

Headlines should not say "Harvard", they should name the researchers. (Rachel Greene in this case.)

I don't know why I had to write this.

[–] [email protected] 11 points 6 days ago* (last edited 6 days ago) (1 children)

Who's Rachel Greene? But we all know Harvard and have an idea of their respectability. Name of the researcher if not well-known should be in the body instead.

[–] [email protected] 7 points 6 days ago (1 children)

"Harvard scientist Rachel Greene"

Everyone's happy

[–] [email protected] 1 points 6 days ago

Headlines have length constraints

[–] [email protected] 23 points 1 week ago

I really thought this was the onion.

[–] [email protected] 20 points 1 week ago (2 children)

So is it saying essentially that in order to not output garbage, it needs to know first what garbage is?

Is it just me that things this seems like a no-brainer?

It almosr draws parallels to many societal issues. Knowledge is power.

People tend towards intolerance and hatred when they dont understand the thing they are angry at. The more they know the better they behave.

[–] [email protected] 9 points 1 week ago (1 children)

No it's more of a technical discussion. Many people might believe that in order to avoid toxicity, you just train a model on "good" non-toxic data and then apply toxicity removal techniques to address emergent toxicity that the model might spit out. This paper is saying they found it more effective to train the model on a small percentage of "bad" toxic data on purpose, then apply those same toxicity removal techniques. For some reason, that actually generated less total toxicity. It's an interesting result. A wild guess on my part, but I'm thinking training the model with toxic content "sharpened" the toxicity when it was generated, making it easier for those removal tools to identify it.

load more comments (1 replies)
load more comments (1 replies)
[–] [email protected] 20 points 1 week ago (1 children)

Give the AI model the gift of culture and class. No suprise it behaves better

[–] [email protected] 13 points 1 week ago

Sophistication my good sir.

[–] [email protected] 18 points 1 week ago (2 children)

When the AI only trained on 4chan dropping.

It needs to be fake and gay

[–] [email protected] 15 points 1 week ago (6 children)

That exists, its called GPT4chan, and it went exactly like you'd expect.

load more comments (6 replies)
load more comments (1 replies)
[–] [email protected] 14 points 1 week ago (1 children)

Boy, I don't even know if I wish that much 4chan on a LLM.

load more comments (1 replies)
[–] [email protected] 10 points 1 week ago* (last edited 1 week ago) (2 children)

I envision a Gemini powered bot that cracks captcha and posts "woke" replies on 4chan. If you're an antivaxxer, antisemite, nazi, racist, sionist, or otherwise, it will debate you. It will not get tired. It will not get mad. It will maintain a sense of decorum indefinitely and it will never ever stop. If some far right extremist decides to do the same, it will have the advantage that academia is left leaning, meaning the model can cite widely recognized studies.

Dead internet theory and so on, but I'll gladly completely and utterly destroy the internet if it means the filth dies with it.

[–] [email protected] 8 points 1 week ago (4 children)

There's little evidence that debate changes people's ideas.

[–] [email protected] 3 points 6 days ago

Seems more about keeping the idiots occupied so they can't flood the zone with their bullshit

load more comments (3 replies)
load more comments (1 replies)
[–] [email protected] 9 points 1 week ago (1 children)

because 4chan users write original content. that is fed into the next best stupid platform and so on until it ends on tiktok or whatever.

if you have nothing to say you use meta/tiktok. no relevabt content has ever been there first. copies and derivates, yes...

so soonish AI will flood 4chan so ai scrapers get polluted aswell...and then it is dead.

load more comments (1 replies)
[–] [email protected] 8 points 1 week ago (1 children)

Not to anthropomorphize LLMs, but.... Like a vaccine?

load more comments (1 replies)
[–] [email protected] 8 points 1 week ago

My hope was that AI would, at least, bear some disgust for the worst of humanity. My new fear is that AI will bear disgust for humanity.

[–] [email protected] 7 points 1 week ago* (last edited 1 week ago)

It's like how vaccinations protect us from illnesses.

[–] [email protected] 7 points 1 week ago (1 children)

can we stop referring to llm's as if they're capable of thought? they don't make decisions; their programming just responds to patterns.

load more comments (1 replies)
load more comments
view more: next ›