HGPU group<p>RDMA Point-to-Point Communication for LLM Systems</p><p><a href="https://mast.hpc.social/tags/CUDA" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>CUDA</span></a> <a href="https://mast.hpc.social/tags/RDMA" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>RDMA</span></a> <a href="https://mast.hpc.social/tags/LLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LLM</span></a> <a href="https://mast.hpc.social/tags/Package" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Package</span></a></p><p><a href="https://hgpu.org/?p=30339" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">hgpu.org/?p=30339</span><span class="invisible"></span></a></p>
#cuda
3 posts3 participants0 posts today
Red Hat integriert Nvidia CUDA in Enterprise-Linux und OpenShift
Red Hat verteilt künftig Nvidias CUDA-Toolkit direkt über seine Plattformen. Das soll die Bereitstellung GPU-beschleunigter KI-Anwendungen vereinfachen.
heise online · Red Hat integriert Nvidia CUDA in Enterprise-Linux und OpenShift
OpenAI의 구글 TPU 검토에 NVIDIA가 긴급 대응한 진짜 이유
OpenAI가 구글 TPU 검토를 계기로 NVIDIA와 1,000억 달러 파트너십을 체결한 배경과 AI 인프라 시장의 권력 구조 변화를 분석합니다. CUDA 생태계의 강력함, TPU/Trainium 같은 대안 칩들의 가격 경쟁력, 그리고 OpenAI의 멀티 클라우드 전략이 시사하는 AI 인프라 시장의 미래 전망을 다룹니다.
Collective Communication for 100k+ GPUs
hgpu.org · Collective Communication for 100k+ GPUsThe increasing scale of large language models (LLMs) necessitates highly efficient collective communication frameworks, particularly as training workloads extend to hundreds of thousands of GPUs. T…
Tutoring LLM into a Better CUDA Optimizer
hgpu.org · Tutoring LLM into a Better CUDA OptimizerRecent leaps in large language models (LLMs) caused a revolution in programming tools (like GitHub Copilot) that can help with code generation, debugging, and even performance optimization. In this…
VectorWare – from creators of `rust-GPU` and `rust-CUDA`
Link: https://www.vectorware.com/blog/announcing-vectorware/
Discussion: https://news.ycombinator.com/item?id=45683153
www.vectorware.com · Announcing VectorWareWe are building the first GPU-native software company. Today we are sharing the thesis, people, and partners behind it.
Beyond the #AIslop, there is the #socialmediaSlop, with so many people repeating fake news like they just don't care...
It is kind of weird to me that #Alibaba/#Qwen, the Chinese equivalent of #Amazon/#AWS, recommands #vLLM for its VL model, thus #pytorch, therefore #CUDA ie #Nvidia?
Probably not for long, I tooted something about a new Chinese company (#Cambricon) developing its own software for parallel computing. Is it stable like software provided by Orange Pi?
I tested Qwen/Qwen3-VL-4B-Instruct quickly (full version, no GGUF) using #CUDA, so that there are no doubts about quality loss with quantization (see result in desc on the left picture)
It is quite interesting :
- Using device_map=auto in transformers demonstrates very well the bandwidth issue: process is going back and forth between CPU and GPU, it's slow on my setup (old system)
- it uses lots of ram for text encoding
- it was unable to distinguish all 137 animals
Nvidia DGX Spark: Erste Tests zeigen enormes Potenzial für lokale KI-Entwicklung
Die ersten Testberichte zum Nvidia DGX Spark deuten darauf hin, dass dieser Mini-PC die Erwartungen an lokale KI-Computer verändern könnte. Wir fassen für euch zusammen, wie sic
https://www.apfeltalk.de/magazin/news/nvidia-dgx-spark-erste-tests-zeigen-enormes-potenzial-fuer-lokale-ki-entwicklung/
#KI #News #Tellerrand #CUDA #DGXSpark #GraceBlackwell #KIEntwicklung #KIHardware #LokaleKI #MiniPC #Nvidia #Test
Apfeltalk Magazin · Nvidia DGX Spark: Erste Tests zeigen enormes Potenzial für lokale KI-EntwicklungDer Nvidia DGX Spark bringt mit hoher Flexibilität und leisen Betrieb frischen Wind in lokale KI-Entwicklungssysteme.
The Deepseek-OCR would run on it obviously, because there is OCR in the name, thus support #CUDA (joke alert).
"Inference using Huggingface transformers on NVIDIA GPUs. Requirements tested on python 3.12.9 + CUDA11.8："
It's open-source!, 2025.
Slogan: "if you can't catch the mouse on camera, we reimburse you".
it is not a RPI, it is an OCR Grapevine℠, with space between USB ports to accomodate extremely diverse USB vendors design packagings, with -angled- slots.
And the modern standards, USB-C, OClink, micro-SFP+, all 10Gbps+
It is for surveillance/security companies to plug more AI capable cameras with realtime cognitive bias; copy books at the library on-the-go, etc
Thesis: Compiler and Runtime Systems for Generative AI Models
hgpu.org · Compiler and Runtime Systems for Generative AI ModelsGenerative AI (GenAI) workloads have rapidly become the predominant data center GPU workload. However, designing efficient GPU kernels for GenAI presents significant challenges due to two central f…