👋 Welcome to LeVanLoi' miscellaneous articles
Xin chào! Mình là Lê Văn Lợi. Đây là trang blog cá nhân với tựa đề "LeVanLoi miscellaneous articles" ghi chép lại các bài viết linh tinh.
A Review of DeepSeek Models
2025-03-25 · Chengen Wang (University of Texas at Dallas - chengen.wang@utdallas.edu), Murat Kantarcioglu (Virginia Tech - muratk@vt.edu)
DeepSeek-V3 and DeepSeek-R1 are leading open-source Large Language Models (LLMs) for general-purpose tasks and reasoning, achieving performance comparable to state-of-the-art closed- source models from companies like OpenAI and Anthropic—while requiring only a fraction of their training costs. Understanding the key innovative techniques behind DeepSeek’s success is crucial for advancing LLM research. In this paper, we review the core techniques driving the remarkable effectiveness and efficiency of these models, including refinements to the transformer architecture, innovations such as Multi-Head Latent Attention and Mixture of Experts, Multi-Token Prediction, the co-design of algorithms, frameworks, and hardware, the Group Relative Policy Optimization algorithm, post-training with pure reinforcement learning and iterative training alternating between supervised fine-tuning and reinforcement learning. Additionally, we identify several open questions and highlight potential research opportunities in this rapidly advancing field.
Remember Me
2025-03-17 · Margaret Mead
Short funeral poem by Margaret Mead, ideal for a eulogy. The words are a message of remembrance and love in times of grief.
