brucethemoose

joined 1 year ago
[–] [email protected] 32 points 16 hours ago* (last edited 16 hours ago) (2 children)

"all-in-one API" that will allow agencies to connect their systems to models from OpenAI, Google, and Anthropic

This is a huge red flag to me.

It means:

  • They are ignorant of existing APIs and standards that are exactly this. Uh, MCP or OpenAI API? Which everyone already uses?
  • They have zero interesting in models they can host themselves, or from cheaper providers. Or, heaven forbit, finetune for their own work.
  • They have zero interest in actually useful tools. Like, say, SGLang's cached hosting and fast fill-in-the-blanks formatting which is perfect for say, processing government forms.

In other words, it's just full corporate AI Bro capture. And shitty.

One of the consequences is that it will be very bad, unfortunately.

[–] [email protected] 4 points 16 hours ago* (last edited 16 hours ago)

To be fair, Axios didn't rag on Iran here. They didn't do any kind of labeling, just stating what happened: Israel bombed a live Iranian news station.

That's what news is supposed to be. The opinion columns are where terms like 'Zionist regime' goes. Same with whatever you'd want to call Iran's govt, but news is just the event as it comes, maybe with context.

Bombing a live news station, and clarifying that there was no warning, speaks for itself. More context of Israel's many other crimes would be better, but Axios has a very-brief writing style.

141
submitted 23 hours ago* (last edited 22 hours ago) by [email protected] to c/[email protected]
 

Video is linked. SFW, but keep your volume down.

[–] [email protected] 2 points 1 day ago* (last edited 1 day ago)

What about the used market? The Nvidia 1080/1080 TI or AMD 6000/7000 series is not too bad.

[–] [email protected] 3 points 1 day ago* (last edited 1 day ago)

5090 is kinda terrible for AI actually. Its too expensive. It only just got support in pytorch, and if you look at 'normie' AI bros trying to use them online, shit doesn't work.

4090 is... mediocre because it's expensive for 24GB. The 3090 is basically the best AI card Nvidia ever made, and tinkerers just opt for banks of them.

Businesses tend to buy RTX Pro cards, rent cloud A100s/H100s or just use APIs.

The server cards DO eat up TSMC capacity, but insane 4090/5090 prices is mostly Nvidia's (and AMD's) fault for literally being anticompetitive.

[–] [email protected] 3 points 1 day ago

It seemed like horrible value at the time, but in hindsight a 4090 was not the worst investment, hah.

[–] [email protected] 2 points 1 day ago* (last edited 1 day ago)

One issue is everyone is supply constrained by TSMC. Even Arc Battlemage is OOS at MSRP.

I bet Intel is kicking themselves for using TSMC. It kinda made sense when they decided years ago, but holy heck, they'd be swimming in market share if they used their own fabs instead (and kept the bigger die).

I feel like another is... marketing?

Like, many buyers just impulse buy, or go with what some shill recommended in a feed. Doesn't matter how competitive anything is anymore.

[–] [email protected] 9 points 3 days ago* (last edited 3 days ago)

TBH we should be rioting in front of big news headquarters.

In front of Facebook. Twitter. Google. They're not too far from LA. Employees should feel scared about enabling all this.

None of this shit matters if algos ignore it, deliberately. Trump can do whatever TF he wants because the media ecosystem built around him is so profitable and engaging, and covering it milquetoast (or worse) is so profitable, not because absolutely everyone obeys him for no reason.

We should make that unprofitable.

[–] [email protected] 44 points 4 days ago (18 children)
[–] [email protected] -1 points 5 days ago (4 children)

A condom is not going to get elected.

[–] [email protected] 4 points 5 days ago* (last edited 5 days ago) (7 children)

To be fair, Newsom is sorta right, but Newsom's also kinda slimey, self-serving and angling for a presidential run. I don't want him to be the next Dem pres candidate...

Also, Trump has always been Trump. Stupid sounding hot air is his thing, way before politics.

[–] [email protected] 2 points 5 days ago* (last edited 5 days ago) (2 children)

Uhh... Tool calling is built into thier tokenizers, but Ollama/Langchain just ignore them because they're spagetti abstractions. To be blunt, langchain and ollama are overhyped, buggy junk trying to reinvent wheels.

For any kind of STEM work, I'd run Llama Nemotron 49B exl3 via TabbyAPI, which exposes a generic openai endpoint anything can use:

https://huggingface.co/nvidia/Llama-3_3-Nemotron-Super-49B-v1

https://huggingface.co/turboderp/Llama-3.3-Nemotron-Super-49B-v1-exl3/tree/3.0bpw

Nemotron models freaking rock at anything STEM-adjacent, and I can get squeeze in 48K+ context on 24GB VRAM (depending on your cache quantization settings).

Otherwise GLM-4 is very good at tool calling, as is Qwen3, and you can more confortable run them as GGUFs if you don't want to leave the llama.cpp ecosystem, or exl2s if you have specific trouble with exl3 in TabbyAPI.

[–] [email protected] 11 points 5 days ago* (last edited 5 days ago)

Pretty sure that’s not how it works. Water is mixed with a soup of stuff the moment it goes in your body, and our digestive system/diet is not as simple as osmotic pressure pushing water into cells (and somehow pushing other substances out?) if that’s what you’re getting at.

 

In a nutshell, he’s allegedly frustrated by too few policies favorable to him.

 
  • The IDF is planning to displace close to 2 million Palestinians to the Rafah area, where compounds for the delivery of humanitarian aid are being built.
  • The compounds are to be managed by a new international foundation and private U.S. companies, though it's unclear how the plan will function after the UN and all aid organizations announced they won't take part
 

Qwen3 was apparently posted early, then quickly pulled from HuggingFace and Modelscope. The large ones are MoEs, per screenshots from Reddit:

screenshots

Including a 235B/22B active and a 30B/3B active.

Context appears to 'only' be 32K unfortunately: https://huggingface.co/qingy2024/Qwen3-0.6B/blob/main/config_4b.json

But its possible they're still training them to 256K:

from reddit

Take it all with a grain of salt, configs could change with the official release, but it appears it is happening today.

23
submitted 1 month ago* (last edited 1 month ago) by [email protected] to c/[email protected]
 

This is one of the "smartest" models you can fit on a 24GB GPU now, with no offloading and very little quantization loss. It feels big and insightful, like a better (albeit dry) Llama 3.3 70B with thinking, and with more STEM world knowledge than QwQ 32B, but comfortably fits thanks the new exl3 quantization!

Quantization Loss

You need to use a backend that support exl3, like (at the moment) text-gen-web-ui or (soon) TabbyAPI.

 

"It makes me think that maybe he [Putin] doesn't want to stop the war, he's just tapping me along, and has to be dealt with differently, through 'Banking' or 'Secondary Sanctions?' Too many people are dying!!!", Trump wrote.

 

The U.S. expects Ukraine's response Wednesday to a peace framework that includes U.S. recognition of Crimea as part of Russia and unofficial recognition of Russian control of nearly all areas occupied since the 2022 invasion, sources with direct knowledge of the proposal tell Axios.

What Russia gets under Trump's proposal:

  • "De jure" U.S. recognition of Russian control in Crimea.
  • "De-facto recognition" of the Russia's occupation of nearly all of Luhansk oblast and the occupied portions of Donetsk, Kherson and Zaporizhzhia.
  • A promise that Ukraine will not become a member of NATO. The text notes that Ukraine could become part of the European Union.
  • The lifting of sanctions imposed since 2014.
  • Enhanced economic cooperation with the U.S., particularly in the energy and industrial sectors.

What Ukraine gets under Trump's proposal:

  • "A robust security guarantee" involving an ad hoc group of European countries and potentially also like-minded non-European countries. The document is vague in terms of how this peacekeeping operation would function and does not mention any U.S. participation.
  • The return of the small part of Kharkiv oblast Russia has occupied.
  • Unimpeded passage of the Dnieper River, which runs along the front line in parts of southern Ukraine.
  • Compensation and assistance for rebuilding, though the document does not say where the funding will come from.

Whole article is worth a read, as it’s quite short/dense as Axios usually is. For those outside the US, this is an outlet that’s been well sourced in Washington for years.

 

Seems there's not a lot of talk about relatively unknown finetunes these days, so I'll start posting more!

Openbuddy's been on my radar, but this one is very interesting: QwQ 32B, post-trained on openbuddy's dataset, apparently with QAT applied (though it's kinda unclear) and context-extended. Observations:

  • Quantized with exllamav2, it seems to show lower distortion levels than nomal QwQ. Its works conspicuously well at 4.0bpw and 3.5bpw.

  • Seems good at long context. Have not tested 200K, but it's quite excellent in the 64K range.

  • Works fine in English.

  • The chat template is funky. It seems to mix up the and <|think|> tags in particular (why don't they just use ChatML?), and needs some wrangling with your own template.

  • Seems smart, can't say if it's better or worse than QwQ yet, other than it doesn't seem to "suffer" below 3.75bpw like QwQ does.

Also, I reposted this from /r/locallama, as I feel the community generally should going forward. With its spirit, it seems like we should be on Lemmy instead?

 

So I had a clip I wanted to upload to a lemmy comment:

  • Tried it as an (avc) mp4... Failed.
  • OK, too big? I shrink it to 2MB, then 1MB. Failed.
  • VP9 Webm maybe? 2MB, 1MB, failed. AV1? Failed.
  • OK, fine, no video. Lets try an animated AVIF. Failed. It seems lemmy doesn't even take static AVIF images
  • WebP animation then... Failed. Animated PNG, failed.

End result, I have to burden the server with a massive, crappy looking GIF after trying a dozen formats. With all due respect, this is worse than some aging service like Reddit that doesn't support new media formats.

For reference, I'm using the web interface. Is this just a format restriction of lemmy.world, or an underlying software support issue?

 

53% of Americans approve of Trump so far, according to a newly released CBS News/YouGov poll conducted Feb. 5 to 7, while 47% disapproved.

A large majority, 70%, said he was doing what he promised in the campaign, per the poll that was released on Sunday.

Yes, but: 66% said he was not focusing enough on lowering prices, a key campaign trail promise that propelled Trump to the White House.

44% of Republicans said Musk and DOGE should have "some" influence, while just 13% of Democrats agreed.

1
submitted 4 months ago* (last edited 4 months ago) by [email protected] to c/[email protected]
 

Hey, I have nothing to do with CachyOS or this Lemmy community, but just wanna say I love this distro.

It's everything annoying about Arch Linux (to me) fixed, more convenient, and objectively fast as heck. I distro hopped for a long time, but have zero inclination to switch after finding CachyOS. I hardly need to tweak anything. It's all optimal out of the box! And how many other distros offer their own AVX2/AVX512 packages by default?

...I haven't even reinstalled CachyOS on my main PC for almost two years. I can't say that for Ubuntu, Fedora, PopOS, or (heaven forbid) Manjaro, all of which are more ostensibly stable yet always seem to break, or get behind on fixes I need. Kionite was too finicky with the whole immutable thing. Garuda Linux was OK, but more bloated, and not nearly as "optimally preconfigured" as CachyOS.

40
submitted 5 months ago* (last edited 5 months ago) by [email protected] to c/[email protected]
 

Here's the Meta formula:

  • Put a Trump friend on your board (Ultimate Fighting Championship CEO Dana White).
  • Promote a prominent Republican as your chief global affairs officer (Joel Kaplan, succeeding liberal-friendly Nick Clegg, president of global affairs).
  • Align your philosophy with Trump's on a big-ticket public issue (free speech over fact-checking).
  • Announce your philosophical change on Fox News, hoping Trump is watching. In this case, he was. "Meta, Facebook, I think they've come a long way," Trump said at a Mar-a-Lago news conference, adding of Kaplan's appearance on the "Fox and Friends" curvy couch: "The man was very impressive."
  • Take a big public stand on a favorite issue for Trump and MAGA (rolling back DEI programs).
  • Amplify that stand in an interview with Fox News Digital. (Kaplan again!)
  • Go on Joe Rogan's podcast and blast President Biden for censorship.
view more: next ›