Recent searches

No recent searches

Search options

Only available when logged in.
eupolicy.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
This Mastodon server is a friendly and respectful discussion space for people working in areas related to EU policy. When you request to create an account, please tell us something about you.

Administered by:

Server stats:

206
active users

eupolicy.social: AboutProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.4.3

#localLLM

2 posts2 participants0 posts today
Nicolas MOUART<p>Here some tests I made last year on a RPI4 4GB. My plan was to run a small RAG on it, but no. I think it is usuable as text parser tho, but I figure out lately that 9 tok/s is kind of the minimum for interaction. It works, but it is very slow, super efficient tho. That's why I won't buy a RPI 5, it is pointless for modern tiny home server applications. RPI4 was the best. When SBC vendors start fitting a fan, that's not the same range of computing, isn't it?</p><p><a href="https://mastodon.social/tags/RPI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>RPI</span></a> <a href="https://mastodon.social/tags/LocalLLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LocalLLM</span></a> <a href="https://mastodon.social/tags/RAG" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>RAG</span></a> <a href="https://mastodon.social/tags/experiments" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>experiments</span></a></p>
Nicolas MOUART<p>I know Microsoft is not the greatest company in the world when it comes to OS but, weirdly enough, Phi 3.5 is really good. I like when LLMs say I don't know, a breath of fresh air, it is less common that one might think.</p><p>NB: <a href="https://mastodon.social/tags/streamlit" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>streamlit</span></a> <a href="https://mastodon.social/tags/Python" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Python</span></a> library is quite good for prototyping nothing fancy, just the bare minimum but stable. It is not designed for loading and unloading large files tho, but that's ok.</p><p><a href="https://mastodon.social/tags/LLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LLM</span></a> <a href="https://mastodon.social/tags/dev" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>dev</span></a> <a href="https://mastodon.social/tags/LocalLLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LocalLLM</span></a></p>
Osma Suominen<p>AGI is just around the corner!</p><p>I'm learning to use DSPy with GEPA (Genetic-Pareto) prompt optimization. In GEPA a larger "teacher" LLM adjusts the prompt for a smaller "student" LM to perform a specific task as well as possible. The teacher will try many different prompts and evaluate the outcome, in my case the quality of a metadata extraction task.</p><p>The larger model (GPT-OSS 120B) just added this to the prompt for the smaller model (Gemma 3 4B):</p><p>&gt; Good luck! 🎯</p><p>😅</p><p><a href="https://sigmoid.social/tags/LLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LLM</span></a> <a href="https://sigmoid.social/tags/LocalLLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LocalLLM</span></a> <a href="https://sigmoid.social/tags/DSPy" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>DSPy</span></a> <a href="https://sigmoid.social/tags/GEPA" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>GEPA</span></a></p>
James House-Lantto (He/Him)<p><a href="https://itsfoss.com/android-on-device-ai/" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">itsfoss.com/android-on-device-</span><span class="invisible">ai/</span></a></p><p>Running Local LLMs on an android, because: reasons.</p><p>Some of the apps discussed:<br>- MLC Chat<br>- SmolChat<br>- Google AI Edge Gallery</p><p>the article also discusses which specific LLM's were tested:<br>- Google’s Gemma 3n (2B)<br>- Meta’s Llama 3.2 (1B/3B)<br>- Microsoft’s Phi-3 Mini (3.8B)<br>- Alibaba’s Qwen-2.5 (1.8B)<br>- TinyLlama-1.1B</p><p><a href="https://mastodon.social/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AI</span></a> <a href="https://mastodon.social/tags/LLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LLM</span></a> <a href="https://mastodon.social/tags/LocalLLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LocalLLM</span></a> <a href="https://mastodon.social/tags/Android" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Android</span></a></p>
Nicolas MOUART<p>Important: Grok did not understand this, made misleading/incorrect calculations (<a href="https://mastodon.social/tags/Grok" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Grok</span></a> is not a <a href="https://mastodon.social/tags/LocalLLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LocalLLM</span></a> by any means), even if it is this system which actually pointed me to this thread (well, I clicked on it). So Yan LeCun, who should know better in any case about this field, appears to be right when he said earlier this year that <a href="https://mastodon.social/tags/LLMs" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LLMs</span></a> were dead. As of why the whole tech industry continues investing in this, is a mystery to me. <a href="https://mastodon.social/tags/AIbubble" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AIbubble</span></a> ?</p><p><a href="https://www.newsweek.com/ai-impact-interview-yann-lecun-llm-limitations-analysis-2054255" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://www.</span><span class="ellipsis">newsweek.com/ai-impact-intervi</span><span class="invisible">ew-yann-lecun-llm-limitations-analysis-2054255</span></a></p><p><a href="https://mastodon.social/tags/AIslop" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AIslop</span></a></p>
Nicolas MOUART<p>Manufacturers are all over the AI market (read <a href="https://mastodon.social/tags/genAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>genAI</span></a>), to sell solutions for <a href="https://mastodon.social/tags/LocalLLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LocalLLM</span></a>, finding new ways to keep people from upgrading their HW, introducing: unified memory! <br>Now your computer is basically a <a href="https://mastodon.social/tags/GPU" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>GPU</span></a> (it's called an <a href="https://mastodon.social/tags/APU" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>APU</span></a> now). It's expensive, but <a href="https://mastodon.social/tags/cooling" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>cooling</span></a> looks good on this <a href="https://mastodon.social/tags/miniPC" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>miniPC</span></a> at least. Forget about fixing this <a href="https://mastodon.social/tags/motherboard" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>motherboard</span></a> on your own too, unless you are into <a href="https://mastodon.social/tags/microelectronics" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>microelectronics</span></a>.</p><p>Framework Desktop Teardown and Review<br><a href="https://youtu.be/D7BehyQVVbU" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">youtu.be/D7BehyQVVbU</span><span class="invisible"></span></a></p><p><a href="https://mastodon.social/tags/ewaste" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ewaste</span></a> <a href="https://mastodon.social/tags/environment" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>environment</span></a></p>
Nicolas MOUART<p>Qwen3-4B-Instruct-2507-Q8_0 (without "reasoning") is actually okay (for its size and relatively speaking; it doesn't speak French fluently). As per itself, it is not "a live verification system".</p><p><a href="https://mastodon.social/tags/LLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LLM</span></a> <a href="https://mastodon.social/tags/interview" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>interview</span></a> <a href="https://mastodon.social/tags/genAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>genAI</span></a> <a href="https://mastodon.social/tags/localLLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>localLLM</span></a> <a href="https://mastodon.social/tags/review" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>review</span></a> <a href="https://mastodon.social/tags/ontology" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ontology</span></a></p>
Sigismund Ninja<p>The first non-research libre LLM?</p><p>## Key features<br>- Fully open model: open weights + open data + full training details including all data and training recipes<br>- Massively Multilingual: 1811 natively supported languages<br>- Compliant Apertus is trained while respecting opt-out consent of data owners (even retrospectivey), and avoiding memorization of training data</p><p><a href="https://news.epfl.ch/news/apertus-a-fully-open-transparent-multilingual-lang/" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">news.epfl.ch/news/apertus-a-fu</span><span class="invisible">lly-open-transparent-multilingual-lang/</span></a></p><p><a href="https://mastodon.nu/tags/llm" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>llm</span></a> <a href="https://mastodon.nu/tags/localllm" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>localllm</span></a> <a href="https://mastodon.nu/tags/opensource" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>opensource</span></a> <a href="https://mastodon.nu/tags/gnu" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>gnu</span></a> <a href="https://mastodon.nu/tags/freesoftware" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>freesoftware</span></a></p>
☮ ♥ ♬ 🧑‍💻<p>“In my opinion the ggml stack (e.g. <a href="https://ioc.exchange/tags/llama" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>llama</span></a>.cpp and <a href="https://ioc.exchange/tags/whisper" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>whisper</span></a>.cpp) has made the biggest impact in making local <a href="https://ioc.exchange/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AI</span></a> possible by a large margin," said <a href="https://ioc.exchange/tags/GeorgiGerganov" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>GeorgiGerganov</span></a>. If you're doing any client-side inferencing these days, he's likely responsible for at least some of it. <a href="https://ioc.exchange/tags/Ggml" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Ggml</span></a> is his low-level library for running machine learning models on different types of hardware.</p><p>Gerganov also maintains llama.cpp, which is a bedrock package for running <a href="https://ioc.exchange/tags/LLMs" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LLMs</span></a> on <a href="https://ioc.exchange/tags/hardware" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>hardware</span></a> with different capabilities. It supports CPUs, but also takes advantage of <a href="https://ioc.exchange/tags/GPUs" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>GPUs</span></a> if you have them.</p><p><a href="https://ioc.exchange/tags/Ollama" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Ollama</span></a>, one of the most popular <a href="https://ioc.exchange/tags/CLI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>CLI</span></a> platforms for running your own LLMs, is a developer layer built atop llama.cpp. It offers single-line installation of over 200 pre-configured LLMs, making it easy for <a href="https://ioc.exchange/tags/LLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LLM</span></a> <a href="https://ioc.exchange/tags/developers" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>developers</span></a> to get up and running with their own local generative AI.”</p><p><a href="https://ioc.exchange/tags/LocalLLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LocalLLM</span></a> &lt;<a href="https://www.theregister.com/2025/08/31/local_llm_opinion_column/" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://www.</span><span class="ellipsis">theregister.com/2025/08/31/loc</span><span class="invisible">al_llm_opinion_column/</span></a>&gt;</p>
Paul<p>Context engineering is an interesting beast on <a href="https://chaos.social/tags/LocalLLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LocalLLM</span></a>|s: While the commercial <a href="https://chaos.social/tags/LLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LLM</span></a>|s offer 200k to 1 MILLION tokens, locally we’re limited to 32k at most, possibly a lot less (my experiments showed around 10k for an 8B model on a 16GB M1 before RAM was full, if I recall correctly).</p>
Demor<p>I tested a self-hosted coding assistant called Dyad (open-source, free).<br>Started with a static site, ended up asking it to rebuild one of my own web projects (3D map in Three.js).<br>Even tried running Ollama 3 locally on my PC 😅.</p><p>Full video and write-up here → Normal Coding Is Officially Dead</p><p><a href="https://mastodon.social/tags/SelfHosted" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SelfHosted</span></a> <a href="https://mastodon.social/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AI</span></a> <a href="https://mastodon.social/tags/WebDev" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>WebDev</span></a> <a href="https://mastodon.social/tags/Coding" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Coding</span></a> <a href="https://mastodon.social/tags/OpenSource" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>OpenSource</span></a> <a href="https://mastodon.social/tags/localLLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>localLLM</span></a></p>
Markus Eisele<p>Protect user privacy while analyzing emotions. Learn how to build a fully local sentiment analysis service using Quarkus, LangChain4j, and Ollama—no API keys, no external dependencies. <a href="https://mastodon.online/tags/LocalLLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LocalLLM</span></a> <a href="https://mastodon.online/tags/DevOps" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>DevOps</span></a> <a href="https://mastodon.online/tags/Java" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Java</span></a> </p><p><a href="https://www.the-main-thread.com/p/sentiment-analysis-java-quarkus-langchain4j-local-llm" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://www.</span><span class="ellipsis">the-main-thread.com/p/sentimen</span><span class="invisible">t-analysis-java-quarkus-langchain4j-local-llm</span></a></p>
Markus Eisele<p>Ollama v0.10.0 is here! Major highlights:</p><p>- New native app for macOS &amp; Windows<br>- 2-3x performance boost for Gemma3 models <br>- 10-30% faster multi-GPU performance<br>- Fixed tool calling issues with <a href="https://mastodon.online/tags/Granite3" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Granite3</span></a>.3 &amp; Mistral-Nemo<br>- `ollama ps` now shows context length<br>- WebP image support in OpenAI API</p><p><a href="https://github.com/ollama/ollama/releases/tag/v0.10.0" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">github.com/ollama/ollama/relea</span><span class="invisible">ses/tag/v0.10.0</span></a></p><p><a href="https://mastodon.online/tags/Ollama" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Ollama</span></a> <a href="https://mastodon.online/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AI</span></a> <a href="https://mastodon.online/tags/LocalLLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LocalLLM</span></a> <a href="https://mastodon.online/tags/OpenSource" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>OpenSource</span></a></p>
Nicolas MOUART<p>This is a Ficaria verna (formerly Ranunculus ficaria L.), or lesser celandine or pilewort (as per Wikipedia and other sites).</p><p>I have tested gemma 3 4b-it-q4_0 multimodal vision model for a while: it is not accurate, and can't be trusted. <br>For instance, it thinks it is a Ranunculus acris (it isn't). It's really hit and miss with this model. I guess it could still be useful to provide some clues or vocabulary.</p><p><a href="https://mastodon.social/tags/taximony" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>taximony</span></a> <a href="https://mastodon.social/tags/localLLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>localLLM</span></a> <a href="https://mastodon.social/tags/NLP" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>NLP</span></a> <a href="https://mastodon.social/tags/multimodal" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>multimodal</span></a> <a href="https://mastodon.social/tags/flower" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>flower</span></a> <a href="https://mastodon.social/tags/privacy" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>privacy</span></a> <a href="https://mastodon.social/tags/review" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>review</span></a> <a href="https://mastodon.social/tags/English" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>English</span></a> <a href="https://mastodon.social/tags/photography" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>photography</span></a></p>
Nicolas MOUART<p>Hopefully, people using GPTs will fact check answers, and won't act as mere schochastic parrot repeaters. <a href="https://mastodon.social/tags/humanoversight" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>humanoversight</span></a> </p><p>Q: How do you know that your answer is correct?<br>A: Remember that while my goal is always accuracy and relevance within the constraints of available data up to early-September 2021, human oversight or further research may be necessary for critical applications where absolute certainty matters most.</p><p><a href="https://mastodon.social/tags/Microsoft" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Microsoft</span></a> <a href="https://mastodon.social/tags/LLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LLM</span></a> <a href="https://mastodon.social/tags/localLLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>localLLM</span></a> <a href="https://mastodon.social/tags/GPT" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>GPT</span></a> <a href="https://mastodon.social/tags/accountability" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>accountability</span></a> <a href="https://mastodon.social/tags/promptTips" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>promptTips</span></a></p>
Nicolas MOUART<p>I wrote my own LLM chatbot experiment to test below Q8 LLMs on a computer without GPU <a href="https://mastodon.social/tags/environment" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>environment</span></a> <a href="https://mastodon.social/tags/EveryLittleHelp" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>EveryLittleHelp</span></a> </p><p><a href="https://mastodon.social/tags/Microsoft" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Microsoft</span></a> Phi 3.5 is the best so far at French, in my humble opinion. They did a really good curating job compared to others. Obviously not for serious business (<a href="https://mastodon.social/tags/hallucination" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>hallucination</span></a>), that's for data center LLMs I guess. <br>It is sad to see where the tech industry goes these days. I doubt customer will be king for very long at this pace.</p><p><a href="https://mastodon.social/tags/localLLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>localLLM</span></a> <a href="https://mastodon.social/tags/LLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LLM</span></a> <a href="https://mastodon.social/tags/review" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>review</span></a> <a href="https://mastodon.social/tags/customerservice" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>customerservice</span></a> <a href="https://mastodon.social/tags/IA" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>IA</span></a> <a href="https://mastodon.social/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AI</span></a></p>
Nami Sunami 🟥<p>Just updated <a href="https://akademienl.social/tags/janai" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>janai</span></a>, and oh my it's improved a lot. With the Jan nano model, the token speed is at 47 tokens/sec on my mac. Pretty impressive. </p><p><a href="https://jan.ai/" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">jan.ai/</span><span class="invisible"></span></a></p><p><a href="https://akademienl.social/tags/localllm" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>localllm</span></a> <a href="https://akademienl.social/tags/localai" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>localai</span></a></p>
Nicolas MOUART<p>Errata: the NVIDIA RTX A2000 6GB can reach up to 200 to/s ! On an 1.5B model. Not bad, @ &lt;70W for the GPU, and less than 140W total for the build (old deprecated HW), given that this kind of useless benchmark is promoted everywhere by 'pro"/-paid- tech enthusiasts.</p><p><a href="https://mastodon.social/tags/nvidia" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>nvidia</span></a> <a href="https://mastodon.social/tags/LLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LLM</span></a> <a href="https://mastodon.social/tags/GPU" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>GPU</span></a> <a href="https://mastodon.social/tags/benchmark" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>benchmark</span></a> <a href="https://mastodon.social/tags/LocalLLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LocalLLM</span></a></p>
Debby ‬⁂📎🐧:disability_flag:<p><span class="h-card" translate="no"><a href="https://fosstodon.org/@system76" class="u-url mention" rel="nofollow noopener" target="_blank">@<span>system76</span></a></span> <br>I love <a href="https://hear-me.social/tags/LLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LLM</span></a>, or as they're often called, <a href="https://hear-me.social/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AI</span></a>, especially when used locally. Local models are incredibly effective for enhancing daily tasks like proofreading, checking emails for spelling and grammatical errors, quickly creating image descriptions, transcribing audio to text, or even finding that one quote buried in tons of files that answers a recurring question.</p><p>However, if I wanted to be fully transparent to <a href="https://hear-me.social/tags/bigtech" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>bigtech</span></a>, I would use Windows and Android with all the "big brotherly goodness" baked into them. That's why I hope these tools don't connect to third-party servers.</p><p>So, my question to you is: Do you propose a privacy-oriented and locally/self-hosted first LLM?</p><p>I'm not opposed to the general notion of using AI, and if done locally and open-source, I really think it could enhance the desktop experience. Even the terminal could use some AI integration, especially for spell-checking and syntax-checking those convoluted and long commands. I would love a self-hosted integration of some AI features. 🌟💻 <br><a href="https://hear-me.social/tags/OpenSource" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>OpenSource</span></a> <a href="https://hear-me.social/tags/Privacy" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Privacy</span></a> <a href="https://hear-me.social/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AI</span></a> <a href="https://hear-me.social/tags/LocalModels" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LocalModels</span></a> <a href="https://hear-me.social/tags/SelfHosted" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SelfHosted</span></a> <a href="https://hear-me.social/tags/LinuxAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LinuxAI</span></a> <a href="https://hear-me.social/tags/LocalLLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LocalLLM</span></a> <a href="https://hear-me.social/tags/LocalAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LocalAI</span></a></p>
Winbuzzer<p>Ollama Local LLM Platform Unveils Custom Multimodal AI Engine, Steps Away from Llama.cpp Framework</p><p><a href="https://mastodon.social/tags/Ollama" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Ollama</span></a> <a href="https://mastodon.social/tags/MultimodalAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>MultimodalAI</span></a> <a href="https://mastodon.social/tags/LocalLLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LocalLLM</span></a> <a href="https://mastodon.social/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AI</span></a> <a href="https://mastodon.social/tags/ArtificialIntelligence" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ArtificialIntelligence</span></a> <a href="https://mastodon.social/tags/MachineLearning" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>MachineLearning</span></a> <a href="https://mastodon.social/tags/VisionModels" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>VisionModels</span></a> <a href="https://mastodon.social/tags/OpenSourceAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>OpenSourceAI</span></a> <a href="https://mastodon.social/tags/LLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LLM</span></a> <a href="https://mastodon.social/tags/AIEngine" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AIEngine</span></a> <a href="https://mastodon.social/tags/TechNews" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>TechNews</span></a> <a href="https://mastodon.social/tags/LocalAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LocalAI</span></a></p><p><a href="https://winbuzzer.com/2025/05/16/ollama-local-llm-platform-unveils-custom-multimodal-ai-engine-steps-away-from-llama-cpp-framework-xcxwbn/" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">winbuzzer.com/2025/05/16/ollam</span><span class="invisible">a-local-llm-platform-unveils-custom-multimodal-ai-engine-steps-away-from-llama-cpp-framework-xcxwbn/</span></a></p>
TrendingLive feeds
About