GitHub Hot Board

GitHub Hot Board · Depo arşivi

YeonwooSung/llm-architecture-from-scratch

Implementing LLM architectures from scratch in PyTorch (Bu açıklama depodan gelir.)

Veri tarihi October 22YeonwooSung

En iyi sıra

#100

Kara Atlar

1.1K

Toplam yıldız

4

Toplam fork

1

Son 30 günde sıralama günü

Proje durumuLisans: bilinmiyorSon push: bilinmiyorAçık issue (PR dahil): bilinmiyor

2026-10-25’ten beri toplanıyor; bilinmiyor = henüz toplanmadı

Rozeti README’ne koy · YeonwooSung/llm-architecture-from-scratch[![Rozeti README’ne koy](https://git.vaiox.xyz/api/badge/YeonwooSung/llm-architecture-from-scratch.svg)](https://git.vaiox.xyz/tr/repo/YeonwooSung/llm-architecture-from-scratch)

Badge · 任何仓库都可以挂 —— 未上榜的仓库显示灰章,上完榜会自动点亮。

<a href="https://git.vaiox.xyz/tr/repo/YeonwooSung/llm-architecture-from-scratch"><img src="https://git.vaiox.xyz/api/badge/YeonwooSung/llm-architecture-from-scratch.svg" alt="Rozeti README’ne koy" height="20"></a>

Proje belgeleri

LLM architecture from scratch

Trendshift

Implementing LLM architectures from scratch for deep understanding of modern LLM architectures.

Implemented

Model Model Size Modality Normalization Positional Encoding Attention Mixture of Experts
GPT-2 XL 1.5B Text LayerNorm Learned Absolute Multi-Head Self-Attention (causal) ❌
Llama3-8B 8B Text RMSNorm RoPE Grouped-Query Attention (causal) ❌
Llama 3.2-3B 3B Text RMSNorm RoPE Grouped-Query Attention (causal) ❌
OlMo 2 7B 7B Text RMSNorm RoPE Grouped-Query Attention (causal) ❌
Phi-4 14B Text RMSNorm RoPE Grouped-Query Attention (causal) ❌
DeepSeek v3 671B Text RMSNorm RoPE Multi-head Latent Attention ✅
Gemma 3 27B Text + Image RMSNorm RoPE Sliding window + GQA ❌
xLSTM 7B Text RMSNorm RoPE mLSTM ❌
Mistral 3.1 Small 24B Text + Image RMSNorm RoPE Sliding window + GQA ❌
Llama 4 Maverick 400B Text RMSNorm RoPE + NoPE GQA ✅
Qwen3 32B Text RMSNorm RoPE Grouped-Query Attention (causal) ❌
Qwen3 (235B-A22B) 235B A22B Text RMSNorm RoPE Grouped-Query Attention (causal) ✅
SmolLM3 3B Text RMSNorm RoPE + NoPE Grouped-Query Attention (causal) ❌
Kimi K2 1T Text RMSNorm RoPE Multi-head Latent Attention ✅
GPT OSS 120B Text RMSNorm YaRN RoPE Grouped-Query Attention (causal) ✅
GLM 4.5 355B Text RMSNorm RoPE Grouped-Query Attention (causal) ✅

In progress

  • Grok 2.5 (270B)
  • Qwen3 Next (80B-A3B)

Mevcut sıralamalar

Son 30 gün grafiği

Toplam yıldız

Sıra grafiği

← Sıralamalara dön

GitHub Hot BoardZaman MakinesiDiscover