Rudy Labs
Research on small models, local training and the tools around them.
The research arm of Aeryx Industries. We publish the papers, the weights and the code.
Models
-
Icon
Questions about machine learning research and security, including papers newer than the base model's January 2025 cutoff.
Papers
-
Compiler-Validated Semantic Editing for Weak Local Coding Models
Coding agents edit source code as text. Small locally runnable models fail at it. Diff formats break, identifiers get guessed, and errors surface only after files change. ago is a semantic edit protocol for Go. The agent queries a typechecked workspace and submits mutations from…
-
Differential Attention at Small Scale: A Paired, Cross-Stack Reproduction with No Generalization Benefit
The Differential Transformer reports consistent gains over standard attention at 3B parameters and 1T tokens. We test whether the mechanism helps at the scale a single laptop can reach. We reimplement differential attention in MLX on Apple Silicon with custom Metal kernels,…