OK, the new local llm hotness is Deepseek from CN. But you know this unless you've been dead for the last week. LLM devs I follow say it is awesome.
Talks in US congress about banning it ...so I figured it is worth a look. Besides, it is touted as the best at coding.
I downloaded on my average-spec'd laptop.
I followed these instructions
https://dev.to/extinctsion/how-run-deepseek-locally-with-ollama-4kbd
Pro: Seems to run pretty fast.
Cons: Runs on Terminal. Crap! I HATE terminal!!
Too verbose.
>Terminal
So it needs a decent UI. I'd seen Open WebUI mentioned as a good interface for local LMMs.
Crap2! Open WebUI uses Docker for install, and I don't have Docker.
Install Docker (even though I know next-to-nothing about using it). It looks like it is working.
OK, so now....
How to Install and Use Ollama WebUI on Windows
https://www.gpu-mart.com/blog/how-to-install-and-use-ollama-webui-on-windows
Copied and pasted command in Step 2 into Terminal
How to Install and Use Ollama WebUI on Windows
https://www.gpu-mart.com/blog/how-to-install-and-use-ollama-webui-on-windows
Bingo! I have a ChatGPT-like UI on FF.
With OWui you can load multiple AIs and then select which one you want to query via dropdown, I've added
Llama 3.2 3B 2.0GB
download command: "ollama run llama3.2" [*NO* space in llama3.2]
+
reference:
ollama/README.md at main · ollama/ollama · GitHub
https://github.com/ollama/ollama/blob/main/README.md#quickstart
I just went through the thread on the fitness site and it seems like at this point people are mostly talking about Ollama and Deepseek there as well.
And some discussion of RAG
https://github.com/athina-ai/rag-cookbooks
Some discussion of what you can run locally and I saw this comment RE Deepseek:
QuoteYes, you won't be able to run the full model locally (unless locally for you looks more like a supercomputer :blush:) but you'll probably find a distilled version that can work for you locally.
He pointed people to this, which I assume is old news for you since you're already up and running
https://medium.com/@isaakmwangi2018/a-simple-guide-to-deepseek-r1-architecture-training-local-deployment-and-hardware-requirements-300c87991126#2007
>> Bingo! I have a ChatGPT-like UI on FF.
Okay... that feels like the future.
I think the main thing to add is an audio interface so you can say, "Computer" and have it respond with "Working"
>audio interface so you can say, "Computer" and have it respond with "Working"
Now you KNOW I always mute speakers and -just to be sure- have a FF extension that defaults all tabs to mute.
But I did notice OWui has a microphone icon next to the text input field --And- it posts a "thinking" blurb when it starts working on a query. Maybe it says it outloud? If so, that'd be pretty close, hhh.
+
>you'll probably find a distilled version that can work for you locally
Correct. That Llama 3.2 is only 2gb --ain't no way that's the full monty.
I asked DS to create an abstract image of NYC rush hour traffic. It went through the details of what the image should include but no image shown. Upon further questioning, it said it could only do simple images in html.
++
Looks like you can import/export a library of prompts BUT it has to be in json. Screw that, I've already tested the OWui FF page and TMky works. I've already built a text prompt cheatsheet and I'll load it in background. Easier to access and copy a prompt than their library.
>> Now you KNOW I always mute speakers
But how will know when it's saying, "I'm sorry Bob, I can't do that." ???
RC the first three posts sounds like you are Indiana Jones looting some underground lair. heh.
After I posted, I thought of editing the title to "Deepseek: Hold my beer. I'm going in!" in keeping with my other Hold My Beer posts like so
https://th3core.com/talk/hardware-technology/w10-hold-my-beer/msg37661/#msg37661
>Jones
I had such a sketchy grasp of WTH I was doing it *was* pretty adventurous.
The bulk of the local LLM community keeps pointing to /r re specs and such
newest submissions : LocalLLaMA
https://old.reddit.com/r/LocalLLaMA/new/
====
Senator Hawley Proposes Jail Time for People Who Download DeepSeek
https://www.404media.co/senator-hawley-proposes-jail-time-for-people-who-download-deepseek/
>> Jail
"Every dollar and gig of data that flows into Chinese AI are dollars and data that will ultimately be used against the United States,"
Uh... remind us how many dollars you sent them and how many gigs of data is your local install sending to China?
>how many gigs of data is your local install sending to China?
No where near as much as I've unknowingly sent to the NSA.
BTW, I have run it with wi-fi off. No internet required. Of course, there is always the chance that they've hidden a 'delayed sentience' routine and it'll soon wake up, flip a switch, and start phoning home. Even though it is open source and has been picked apart down to the last semi-colon.
+
DeepSeek: The countries and agencies that have banned the AI company's tech | TechCrunch
https://techcrunch.com/2025/02/03/deepseek-the-countries-and-agencies-that-have-banned-the-ai-companys-tech/
Running in background when not open?
Slight latency while browsing and battery warm & draining quickly.
>banning
Well, banning usage by US govt agencies seems like an obvious no-brainer. Can't say I disagree with that.
>Running in background when not open?
Yup. Using taskmaster, I found ollama_llama_server.exe using about 70% of cpu and VmmemWSL 15%. I ended the .exe but couldn't end VmmemWSL. However, with the .exe stopped it dropped back to <2%.
It's a known problem.
Stop Ollama · Issue #690 · ollama/ollama
https://github.com/ollama/ollama/issues/690#issuecomment-1998454215
>banning
DeepSeek rated too dodgy down under: Banned from Australian government devices
https://www.theregister.com/2025/02/05/australia_deepseek_ban/
Well....
DeepSeek limits model access due to overwhelming server demand
https://www.engadget.com/ai/deepseek-limits-model-access-due-to-overwhelming-server-demand-151339342.html?src=rss