projectmoon

cross-posted from: https://lemm.ee/post/56114125

I wanna fly away, on a unicorn, to discover a land of freedom and light...

This image was made using ComfyUI with Stable Diffusion 3.5 Medium. I can't tell you the exact prompt, because I asked OLMo2 via Open WebUI to make me a picture of a rainbow unicorn galloping through outer space.

I've been working on keeping the OSM tool up to date for OpenWebUI's rapid development pace. And now I've added better-looking citations, with fancy styling. Just a small announcement post!

Update: when this was originally posted, the tool was on 1.3. Now it's updated to 2.1.0, with a navigation feature (beta) and more fixes for robustness.

Over the weekend (this past Saturday specifically), GPT-4o seems to have gone from capable and rather free for generating creative writing to not being able to generate basically anything due to alleged content policy violations. It'll just say "can't assist with that" or "can't continue." But 80% of the time, if you regenerate the response, it'll happily continue on its way.

It's like someone updated some policy configuration over the weekend and accidentally put an extra 0 in a field for censorship.

GPT-4 and GPT 3.5 seem unaffected by this, which makes it even weirder. Switching to GPT 4 will have none of the issues that 4o is having.

I noticed this happening literally in the middle of generating text.

See also: https://old.reddit.com/r/ChatGPT/comments/1droujl/ladies_gentlemen_this_is_how_annoying_kiddie/

https://old.reddit.com/r/ChatGPT/comments/1dr3axv/anyone_elses_ai_refusing_to_do_literally_anything/

Current situation: I've got a desktop with 16 GB of DDR4 RAM, a 1st gen Ryzen CPU from 2017, and an AMD RX 6800 XT GPU with 16 GB VRAM. I can 7 - 13b models extremely quickly using ollama with ROCm (19+ tokens/sec). I can run Beyonder 4x7b Q6 at around 3 tokens/second.

I want to get to a point where I can run Mixtral 8x7b at Q4 quant at an acceptable token speed (5+/sec). I can run Mixtral Q3 quant at about 2 to 3 tokens per second. Q4 takes an hour to load, and assuming I don't run out of memory, it also runs at about 2 tokens per second.

What's the easiest/cheapest way to get my system to be able to run the higher quants of Mixtral effectively? I know that I need more RAM Another 16 GB should help. Should I upgrade the CPU?

As an aside, I also have an older Nvidia GTX 970 lying around that I might be able to stick in the machine. Not sure if ollama can split across different brand GPUs yet, but I know this capability is in llama.cpp now.

Thanks for any pointers!

Not sure if this has been asked before or not. I tried searching and couldn't find anything. I have an issue where any pictures from startrek.website do not show up on the homepage. It seems to only affect startrek.website. Going to the link directly loads the image just fine. Is this something wrong with lemm.ee?

For the past few days, the android app has been very slow. The app itself loads fine and is responsive, but it takes many seconds to load messages, sometimes up to 30 seconds. At first I thought it was a blip, but it's been going on for a few days now. Anyone else have this problem?

Edit: clearing cache in the app settings (not system settings) fixed it.

Erum við að læra íslensku? Er það áætlun til að læra tungumálið?

This has probably already been asked before, but:

The magazines of kbin federate as Lemmy communities, but is the microblog section of a kbin magazine accessible via Lemmy?

Assuming you haven't already.