Blog

🔥 Check out this insightful post from Hacker News 📖 📂 **Category**: 📌 **What You’ll Learn**: Executive Summary PFG-1 "Sophon" is a unified training-and-inference die on a 750 mm², 32-tier 2D Transition-Metal Dichalcogenide (TMD) Monolithic 3D (M3D) platform. Weights, gradients, and optimizer state reside in on-die 2T0C 2D-TMD gain-cell DRAM; because the array is fully read-write, the same silicon executes BF16 forward/backward training passes and serves low-batch decode at the compute-bound rate. Compute is pure digital Compute-In-Memory (CIM): each 256×256 DRAM subarray tile pairs a binary sense amplifier with an 8-level adder tree, driven by a 500 MHz bit-serial activation…

💥 Discover this insightful post from Hacker News 📖 📂 **Category**: ✅ **What You’ll Learn**: There’s something to be said for cigarettes. Most of it, of course, is bad. Cigarettes smell awful. The stench lingers and invades long after the cigarette is gone. Cigarettes, of course, also kill people. Lung cancer from a lifetime of smoking (mostly cigarettes, but also pipes) killed my father. It’s kind of amazing that fire from a cigarette left smoldering as he fell asleep didn’t kill him, the way it has so many others. The anti-smoking messaging I got in school as a kid was…

🔥 Discover this insightful post from Hacker News 📖 📂 **Category**: 📌 **What You’ll Learn**: [Submitted on 13 Jan 2024 (v1), last revised 9 Nov 2024 (this version, v2)] View a PDF of the paper titled Knowledge Distillation of Black-Box Large Language Models, by Hongzhan Chen and 5 other authors View PDF HTML (experimental) Abstract:Given the exceptional performance of proprietary large language models (LLMs) like GPT-4, recent research has increasingly focused on boosting the capabilities of smaller models through knowledge distillation (KD) from these powerful yet black-box teachers. While leveraging the high-quality outputs of these teachers is advantageous, the inaccessibility…

✨ Discover this trending post from Hacker News 📖 📂 **Category**: ✅ **What You’ll Learn**: Hello you fine Internet folks,Here at ISC 2026 in Hamburg, Germany, we got the 67th TOP500 list where there was a surprise awaiting us. That surprise being a new Number 1 Supercomputer on the TOP500.The new number one Supercomputer on the TOP500 list is the LineShine Supercomputer in Shenzhen, China. This is the first Chinese submission to the TOP500 in 9 years and they came in swinging with a massive CPU-only system.Starting with the specifications of the CPU that powers the LineShine system, the LX2.The…

💥 Check out this insightful post from Hacker News 📖 📂 **Category**: ✅ **What You’ll Learn**: A GPT-2-class language model built entirely from scratch in C/CUDA — no PyTorch, no autograd, no ML libraries. The forward and backward passes are written and verified by hand, and the whole training pipeline lives in this repo: a hand-written byte-level BPE tokenizer, pretraining on a books + web corpus, and supervised fine-tuning into a chat model (RLHF/DPO planned). It runs on CPU (libm + OpenMP) for a small showcase model, and a full from-scratch CUDA engine — cuBLAS matmuls, a hand-written FlashAttention, validated…