Neural CTMC decouples discrete diffusion into timing and direction
A new parameterization for discrete diffusion models separates when and where tokens jump, aligning training with mathematical structure.
Source: arxiv/cs.LG · Jingyuan Li, Xiaoyi Jiang, Fukang Wen, Wei Liu, Renqian Luo, Yi Zhu, Zuoqiang Shi, Pipi Hu · open original ↗
Neural CTMC splits discrete diffusion reverse process into exit rate and jump distribution, matching Poisson process fundamentals.
- — Existing discrete diffusion models treat the reverse rate matrix as one unit; Neural CTMC splits it into two components.
- — Exit rate network learns when to jump; jump distribution network learns where to jump in token space.
- — ELBO training objective factors into Poisson KL for timing and categorical KL for direction, decoupling optimization.
- — Theoretical proof shows conditional surrogate preserves gradients and minimizers of marginal reverse-process objective.
- — Framework handles masked and GIDD-style noise schedules within the same decomposed structure.
- — Uniform forward process with Neural CTMC outperforms mask-based methods on OpenWebText without special masking.
- — Pretrained weights released on Hugging Face for reproducibility and downstream use.
Astrobobo tool mapping
- Reading Queue Add the full arXiv paper to your reading queue, prioritizing Sections 2–3 (CTMC background and Neural CTMC formulation) before diving into proofs.
- Knowledge Capture Sketch a diagram showing how a CTMC decomposes into Poisson timing and categorical direction, then note how Neural CTMC parameterizes each separately.
- Focus Brief Summarize the key insight: single reverse-rate matrix → exit rate + jump distribution. Capture why this reduces optimization burden.
Frequently asked
- A CTMC models state transitions over continuous time. It is fully determined by two quantities: a Poisson process that governs jump timing (when transitions occur) and a categorical distribution that governs jump direction (which state to jump to). Neural CTMC exploits this mathematical structure by training separate networks for each, rather than learning a monolithic rate matrix.
cite ▸
APA
Jingyuan Li, Xiaoyi Jiang, Fukang Wen, Wei Liu, Renqian Luo, Yi Zhu, Zuoqiang Shi, Pipi Hu. (2026, April 20). Neural CTMC decouples discrete diffusion into timing and direction. Astrobobo Content Engine (rewrite of arxiv/cs.LG). https://astrobobo-content-engine.vercel.app/article/neural-ctmc-decouples-discrete-diffusion-into-timing-and-direction-4100aa
MLA
Jingyuan Li, Xiaoyi Jiang, Fukang Wen, Wei Liu, Renqian Luo, Yi Zhu, Zuoqiang Shi, Pipi Hu. "Neural CTMC decouples discrete diffusion into timing and direction." Astrobobo Content Engine, 20 Apr 2026, https://astrobobo-content-engine.vercel.app/article/neural-ctmc-decouples-discrete-diffusion-into-timing-and-direction-4100aa. Based on "arxiv/cs.LG", https://arxiv.org/abs/2604.15694.
BibTeX
@misc{astrobobo_neural-ctmc-decouples-discrete-diffusion-into-timing-and-direction-4100aa_2026,
author = {Jingyuan Li, Xiaoyi Jiang, Fukang Wen, Wei Liu, Renqian Luo, Yi Zhu, Zuoqiang Shi, Pipi Hu},
title = {Neural CTMC decouples discrete diffusion into timing and direction},
year = {2026},
url = {https://astrobobo-content-engine.vercel.app/article/neural-ctmc-decouples-discrete-diffusion-into-timing-and-direction-4100aa},
note = {Astrobobo rewrite of arxiv/cs.LG, https://arxiv.org/abs/2604.15694},
}