Author Topic: LM Studio - Discover, download, and run local LLMs  (Read 390 times)

rcjordan

  • I'm consulting the authorities on the subject
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 17149
  • Debbie says...
    • View Profile
LM Studio - Discover, download, and run local LLMs
« on: December 20, 2024, 11:09:34 PM »
https://lmstudio.ai/


found here:
https://www.makeuseof.com/why-use-meta-llama-instead-chatgpt/

I’ve Started Using Llama 3.2, and I Wish I'd Switched Much Sooner


ergophobe

  • Inner Core
  • Hero Member
  • *
  • Posts: 9824
    • View Profile
Re: LM Studio - Discover, download, and run local LLMs
« Reply #1 on: December 25, 2024, 02:47:54 AM »
>> Chat with your local documents

That might be the thing I was looking for in the question about email. If it works for the task, it solves the thorny problem of all the personal info that is in customer emails (just the email addresses, since I promise them that their email will never be sold, shared or given away)

>>RAM: LLMs can consume a lot of RAM. At least 16GB of RAM is recommended.

My neighbor wondered why in the world I thought I needed 16GB of RAM when I bought this machine and I said, "Well, I don't right now, but I keep my machines a long time." I didn't plan on it running LMs.


It reminds me about how every so often, you have this feeling, "I won't need a more powerful computer than this ever because I don't care about gaming and do the same stuff I've been doing for years,"  which is almost guaranteed to precede a paradigm shift where you want to do new stuff.

I remember my brother buying a massive 500MB hard drive so he would never need to upgrade it again. Then Windows came out and was adopted by his employer and 500MB was instantly too small.

Then I get into photography and realize that I want to manipulate 32-bit HDR images, but that's it. I don't play games. I don't edit video. This computer is all I need.

Then I realize, no, wait, I need a computer that can run a Medium LM with agents that connect to an LLM for harder questions... and now I need a supercomputer (and, BTW, the nice but 3yo laptop on my desk is many orders of magnitude bigger than the Cray 1, the "supercomputer that won the Cold War" so by that measure we all have supercomputers today)

According to ChatGPT

1. Processing Power
Cray-1: Peak Performance: ~160 megaflops (million floating-point operations per second).
Intel Core i7 (modern versions): Peak Performance: Modern i7 processors can achieve hundreds of gigaflops to several teraflops (trillions of floating-point operations per second).
The i7 is roughly 10,000 to 100,000 times faster in terms of raw computational power.

2. Memory
Cray-1: Memory: 8 megabytes (MB) of high-speed memory.
Intel Core i7: Memory: Supports gigabytes to terabytes of memory with much higher bandwidth
« Last Edit: December 25, 2024, 02:52:54 AM by ergophobe »