« previous next »
Pages: [1] 2

Author Topic: Local AI: Deepseek r1:1.5b running now  (Read 1029 times)

rcjordan

  • I'm consulting the authorities on the subject
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 17149
  • Debbie says...
    • View Profile
Local AI: Deepseek r1:1.5b running now
« on: February 02, 2025, 11:37:35 PM »
OK, the new local llm hotness is Deepseek from CN.  But you know this unless you've been dead for the last week. LLM devs I follow say it is awesome.

Talks in US congress about banning it ...so I figured it is worth a look.  Besides, it is touted as the best at coding.

I downloaded on my average-spec'd laptop.

I followed these instructions
https://dev.to/extinctsion/how-run-deepseek-locally-with-ollama-4kbd

Pro:  Seems to run pretty fast.
Cons: Runs on Terminal.  Crap! I HATE terminal!!
         Too verbose. 
Logged

rcjordan

  • I'm consulting the authorities on the subject
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 17149
  • Debbie says...
    • View Profile
Re: Local AI: Deepseek r1:1.5b running now
« Reply #1 on: February 02, 2025, 11:48:33 PM »
>Terminal

So it needs a decent UI.  I'd seen Open WebUI mentioned as a good interface for local LMMs.

Crap2! Open WebUI uses Docker for install, and I don't have Docker.

Install Docker (even though I know next-to-nothing about using it).  It looks like it is working.

OK, so now....
How to Install and Use Ollama WebUI on Windows
https://www.gpu-mart.com/blog/how-to-install-and-use-ollama-webui-on-windows

Copied and pasted command in Step 2 into Terminal

How to Install and Use Ollama WebUI on Windows
https://www.gpu-mart.com/blog/how-to-install-and-use-ollama-webui-on-windows

Bingo!  I have a ChatGPT-like UI on FF.

Logged

rcjordan

  • I'm consulting the authorities on the subject
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 17149
  • Debbie says...
    • View Profile
Re: Local AI: Deepseek r1:1.5b running now
« Reply #2 on: February 03, 2025, 01:33:00 AM »
With OWui you can load multiple AIs and then select which one you want to query via dropdown,  I've added

Llama 3.2    3B    2.0GB    

download command: "ollama run llama3.2" [*NO* space in llama3.2]

+
reference:
ollama/README.md at main · ollama/ollama · GitHub
https://github.com/ollama/ollama/blob/main/README.md#quickstart
« Last Edit: February 03, 2025, 01:38:06 AM by rcjordan »
Logged

ergophobe

  • Inner Core
  • Hero Member
  • *
  • Posts: 9824
    • View Profile
Re: Local AI: Deepseek r1:1.5b running now
« Reply #3 on: February 03, 2025, 01:57:21 AM »
I just went through the thread on the fitness site and it seems like at this point people are mostly talking about Ollama and Deepseek there as well.

And some discussion of RAG
https://github.com/athina-ai/rag-cookbooks

Some discussion of what you can run locally and I saw this comment RE Deepseek:

Quote
Yes, you won’t be able to run the full model locally (unless locally for you looks more like a supercomputer :blush:) but you’ll probably find a distilled version that can work for you locally.

He pointed people to this, which I assume is old news for you since you're already up and running
https://medium.com/@isaakmwangi2018/a-simple-guide-to-deepseek-r1-architecture-training-local-deployment-and-hardware-requirements-300c87991126#2007

>> Bingo!  I have a ChatGPT-like UI on FF.

Okay... that feels like the future.

I think the main thing to add is an audio interface so you can say, "Computer" and have it respond with "Working"
Logged

rcjordan

  • I'm consulting the authorities on the subject
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 17149
  • Debbie says...
    • View Profile
Re: Local AI: Deepseek r1:1.5b running now
« Reply #4 on: February 03, 2025, 02:30:09 AM »
>audio interface so you can say, "Computer" and have it respond with "Working"

Now you KNOW I always mute speakers and -just to be sure- have a FF extension that defaults all tabs to mute.

But I did notice OWui has a microphone icon next to the text input field   --And- it posts a "thinking" blurb when it starts working on a query.  Maybe it says it outloud? If so, that'd be pretty close, hhh.
Logged

rcjordan

  • I'm consulting the authorities on the subject
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 17149
  • Debbie says...
    • View Profile
Re: Local AI: Deepseek r1:1.5b running now
« Reply #5 on: February 03, 2025, 02:42:47 AM »
+
>you’ll probably find a distilled version that can work for you locally

Correct.  That Llama 3.2 is only 2gb --ain't no way that's the full monty.


I asked DS to create an abstract image of NYC rush hour traffic. It went through the details of what the image should include but no image shown.  Upon further questioning, it said it could only do simple images in html.
Logged

rcjordan

  • I'm consulting the authorities on the subject
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 17149
  • Debbie says...
    • View Profile
Re: Local AI: Deepseek r1:1.5b running now
« Reply #6 on: February 03, 2025, 02:52:59 AM »
++

Looks like you can import/export a library of prompts BUT it has to be in json.  Screw that, I've already tested the OWui FF page and TMky works.  I've already built a text prompt cheatsheet and I'll load it in background. Easier to access and copy a prompt than their library.
Logged

ergophobe

  • Inner Core
  • Hero Member
  • *
  • Posts: 9824
    • View Profile
Re: Local AI: Deepseek r1:1.5b running now
« Reply #7 on: February 03, 2025, 03:32:59 AM »
>> Now you KNOW I always mute speakers

But how will know when it's saying, "I'm sorry Bob, I can't do that." ???
Logged

Brad

  • Inner Core
  • Hero Member
  • *
  • Posts: 4318
  • What, me worry?
    • View Profile
Re: Local AI: Deepseek r1:1.5b running now
« Reply #8 on: February 03, 2025, 06:45:55 AM »
RC the first three posts sounds like you are Indiana Jones looting some underground lair. heh.
Logged

rcjordan

  • I'm consulting the authorities on the subject
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 17149
  • Debbie says...
    • View Profile
Re: Local AI: Deepseek r1:1.5b running now
« Reply #9 on: February 03, 2025, 02:44:55 PM »
After I posted, I thought of editing the title to "Deepseek: Hold my beer. I'm going in!" in keeping with my other Hold My Beer posts like so

https://th3core.com/talk/hardware-technology/w10-hold-my-beer/msg37661/#msg37661

>Jones

I had such a sketchy grasp of WTH I was doing it *was* pretty adventurous.
Logged

rcjordan

  • I'm consulting the authorities on the subject
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 17149
  • Debbie says...
    • View Profile
Re: Local AI: Deepseek r1:1.5b running now
« Reply #10 on: February 04, 2025, 12:43:57 AM »
The bulk of the local LLM community keeps pointing to /r re specs and such

newest submissions : LocalLLaMA
https://old.reddit.com/r/LocalLLaMA/new/

====
Senator Hawley Proposes Jail Time for People Who Download DeepSeek
https://www.404media.co/senator-hawley-proposes-jail-time-for-people-who-download-deepseek/
Logged

ergophobe

  • Inner Core
  • Hero Member
  • *
  • Posts: 9824
    • View Profile
Re: Local AI: Deepseek r1:1.5b running now
« Reply #11 on: February 04, 2025, 03:20:35 AM »
>> Jail

“Every dollar and gig of data that flows into Chinese AI are dollars and data that will ultimately be used against the United States,”

Uh... remind us how many dollars you sent them and how many gigs of data is your local install sending to China?
Logged

rcjordan

  • I'm consulting the authorities on the subject
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 17149
  • Debbie says...
    • View Profile
Re: Local AI: Deepseek r1:1.5b running now
« Reply #12 on: February 04, 2025, 12:12:26 PM »
>how many gigs of data is your local install sending to China?

No where near as much as I've unknowingly sent to the NSA.

BTW, I have run it with wi-fi off. No internet required.  Of course, there is always the chance that they've hidden a 'delayed sentience' routine and it'll soon wake up, flip a switch, and start phoning home.   Even though it is open source and has been picked apart down to the last semi-colon.
« Last Edit: February 04, 2025, 12:15:38 PM by rcjordan »
Logged

rcjordan

  • I'm consulting the authorities on the subject
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 17149
  • Debbie says...
    • View Profile
Re: Local AI: Deepseek r1:1.5b running now
« Reply #13 on: February 04, 2025, 05:21:40 PM »
+

DeepSeek: The countries and agencies that have banned the AI company's tech | TechCrunch

https://techcrunch.com/2025/02/03/deepseek-the-countries-and-agencies-that-have-banned-the-ai-companys-tech/
Logged

rcjordan

  • I'm consulting the authorities on the subject
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 17149
  • Debbie says...
    • View Profile
Re: Local AI: Deepseek r1:1.5b running now
« Reply #14 on: February 04, 2025, 10:35:57 PM »
Running in background when not open?

Slight latency while browsing and battery warm & draining quickly. 
Logged
Pages: [1] 2
« previous next »