eupolicy.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
This Mastodon server is a friendly and respectful discussion space for people working in areas related to EU policy. When you request to create an account, please tell us something about you.

Server stats:

223
active users

#llamacpp

0 posts0 participants0 posts today
Thomas Renkert🦞<p>Ich durfte gestern beim "Heidelberger Informatiklehrkräftetag" (<a href="https://hcommons.social/tags/HILT" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>HILT</span></a>) Lehrer:innen der <a href="https://hcommons.social/tags/Informatik" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Informatik</span></a> zeigen, wie man sein eigenes <a href="https://hcommons.social/tags/LLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LLM</span></a> GRATIS finetunen kann - in unserem Fall <a href="https://hcommons.social/tags/Llama3" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Llama3</span></a> und <a href="https://hcommons.social/tags/Phi3" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Phi3</span></a> - mit der <a href="https://hcommons.social/tags/MacgAIver" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>MacgAIver</span></a> Pipeline. </p><p>Das Ziel ist dabei nicht (nur), Geld zu sparen, sondern mit einfachen Mitteln für mehr <a href="https://hcommons.social/tags/KIMuendigkeit" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>KIMuendigkeit</span></a> zu sorgen, indem Schüler:innen einen kleinen Blick unter die Motorhaube von Sprachmodellen werfen können.</p><p>Dabei helfen uns u.a. <a href="https://hcommons.social/tags/ollama" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ollama</span></a>, <a href="https://hcommons.social/tags/openwebui" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>openwebui</span></a>, <a href="https://hcommons.social/tags/llamafactory" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>llamafactory</span></a> und <a href="https://hcommons.social/tags/llamacpp" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>llamacpp</span></a>, aber auch Grundwissen aus den Geisteswissenschaften (bzw. den Fächern Deutsch, Englisch, Geschichte...), um zu einem tiefergehenden Verständnis von <a href="https://hcommons.social/tags/genAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>genAI</span></a> und seinen Umgang mit unserem "kulturellen Archiv" zu gelangen.</p><p>Es hat Spaß gemacht!</p><p>Für künftige MacgAIver-Ideen kann man dieser Gruppe folgen: <span class="h-card"><a href="https://a.gup.pe/u/macgaiver" class="u-url mention" rel="nofollow noopener" target="_blank">@<span>macgaiver</span></a></span> </p><p><a href="https://hcommons.social/tags/fediLZ" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>fediLZ</span></a> <a href="https://hcommons.social/tags/lernenmitKI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>lernenmitKI</span></a> <a href="https://hcommons.social/tags/KI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>KI</span></a> <a href="https://hcommons.social/tags/edubw" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>edubw</span></a> <a href="https://hcommons.social/tags/bildung" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>bildung</span></a> <a href="https://hcommons.social/tags/Heidelberg" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Heidelberg</span></a> <a href="https://hcommons.social/tags/education" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>education</span></a> <a href="https://hcommons.social/tags/ai" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ai</span></a> <a href="https://hcommons.social/tags/aieducation" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>aieducation</span></a> <a href="https://hcommons.social/tags/genAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>genAI</span></a> <a href="https://hcommons.social/tags/Informatikunterricht" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Informatikunterricht</span></a> <a href="https://hcommons.social/tags/Medienkompetenz" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Medienkompetenz</span></a> <a href="https://hcommons.social/tags/InformatikEdu" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>InformatikEdu</span></a> <a href="https://hcommons.social/tags/AILiteracy" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AILiteracy</span></a></p><p><span class="h-card"><a href="https://a.gup.pe/u/education" class="u-url mention" rel="nofollow noopener" target="_blank">@<span>education</span></a></span> <span class="h-card"><a href="https://a.gup.pe/u/edutooters" class="u-url mention" rel="nofollow noopener" target="_blank">@<span>edutooters</span></a></span></p>
Stuart Gray<p>Interesting to see Google adopting Context Caching for their online LLM Gemini:</p><p><a href="https://ai.google.dev/gemini-api/docs/caching" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">ai.google.dev/gemini-api/docs/</span><span class="invisible">caching</span></a></p><p>And their claimed benefit is to save users the financial costs of reprocessing tokens in large contexts.</p><p>I've wanted this *exact* feature in local/offline LLM tools. Money isn't the only cost it saves on, it also save compute resources on low end devices.</p><p>The *only* local LLM tool I know that does it is llama.cpp, and it's been effectively dropped from the server app.</p><p><a href="https://mastodonapp.uk/tags/LLMs" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LLMs</span></a> <a href="https://mastodonapp.uk/tags/LlamaCpp" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LlamaCpp</span></a></p>
Ferdinando Simonetti <p><span>[ A story from Martin Thissen on Medium ]<br>Read this story from Martin Thissen on Medium: </span><a href="https://medium.com/@martin-thissen/mixtral-8x7b-on-your-local-computer-free-gpt-4-alternative-e3cb301984e3" rel="nofollow noopener" target="_blank">https://medium.com/@martin-thissen/mixtral-8x7b-on-your-local-computer-free-gpt-4-alternative-e3cb301984e3</a> <a href="https://misskey.social/tags/Mixtral" rel="nofollow noopener" target="_blank">#Mixtral</a> <a href="https://misskey.social/tags/LLM" rel="nofollow noopener" target="_blank">#LLM</a> <a href="https://misskey.social/tags/LlamaCPP" rel="nofollow noopener" target="_blank">#LlamaCPP</a></p>
Stuart Gray<p>It looks like "Control Vectors" might be the next big feature for self-hosted LLMs - and support for them has just been added to Llama.cpp!</p><p>For LLMs, they're probably closer to realising what LoRAs are for text-to-image.</p><p>"Control vectors are an easy-to-train (~60s on a 4090 for a 7B parameter model) way to modify the behavior of an LLM without finetuning or inference-time prompting..."</p><p><a href="https://vgel.me/posts/representation-engineering/" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">vgel.me/posts/representation-e</span><span class="invisible">ngineering/</span></a></p><p><a href="https://github.com/ggerganov/llama.cpp/pull/5970" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">github.com/ggerganov/llama.cpp</span><span class="invisible">/pull/5970</span></a></p><p><a href="https://mastodonapp.uk/tags/LLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LLM</span></a> <a href="https://mastodonapp.uk/tags/LlamaCpp" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LlamaCpp</span></a> <a href="https://mastodonapp.uk/tags/ControlVectors" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ControlVectors</span></a> <a href="https://mastodonapp.uk/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AI</span></a> <a href="https://mastodonapp.uk/tags/LoRA" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LoRA</span></a></p>