SLM 101

A concept lab for engineers: train and run a tiny language model entirely on your machine. Offline-first Python with pytest. No API key or cloud inference for the core path.

What you will learn

  1. Corpus and tokens – load text, build a character vocabulary, make training pairs.
  2. Tiny model – next-character prediction from local weights you can inspect.
  3. Train locally – one pass over the corpus; write a checkpoint under artifacts/.
  4. Generate locally – reload the checkpoint and continue a prompt with no network.
  5. Eval harness – fixed cases and checkers; pytest is the regression suite.
  6. Design rule – when weights help vs editable canon/RAG (digital twin language packs come next).

Quick start

git clone https://github.com/iammikek/slm-101.git
cd slm-101
docker compose up --build

That trains, writes artifacts/model.json, and runs evals. No local Python required. Or use a venv: pip install -e '.[dev]' then python -m slm101 train.

Core tests pass with no secrets. Design rule: the checkpoint is yours and local; nothing in the core path calls a hosted API.