Actualités Mistral et développements LLM - 20 août 2026

L'article examine si le cache KV dans les grands modèles linguistiques fonctionne dans un espace vectoriel de haute dimension, en se concentrant sur son rôle durant l'inférence et les processus de stockage et de récupération.

I'm trying to make a small experimental LM by combining a bunch of ideas I found in different papers. I know this sounds like I threw half the recent LM literature into a blender, but I'm trying to see if the pieces can actually work together. The main idea is a Loop Language Model, where the same model is run multiple times instead of just making the network deeper and deeper. Right now I'm using 4 loops. text input ↓ same transformer ↓ loop 1 ↓ loop 2 ↓ loop 3 ↓ loop 4 The interesting part is that the model can learn to decide that it doesn't need all 4 loops and exit early. What paper

I have a big database of local court decisions with a legal sentence which is like a paragraph summary of the doc. I've been tinkering with FTing for many days now, all results inconclusive never beating base except for a highly specific task where the eval was built around a specific task of extracting passages from the text, where it seemed to match the gold, but as I've found out it is unusable and a lot of padding and generalizations which I need to actually eliminate. So the question is, can a 26B model even be fine-tuned to produce those complex legal principle/headnote of a court deci

​ I'm working on a system to estimate whether code committed to a repository was generated with AI coding tools. My current approach is based on Git/commit-level signals such as AI-related commit trailers, commit metadata, LOC changes, number of files changed, addition/deletion patterns, etc. The problem I'm running into is confidence and calibration. For example, a commit containing 500+ new lines isn't necessarily AI-generated. A developer can also modify or remove the metadata that would make an AI-assisted commit identifiable. Once the code leaves the IDE and reaches Git, much of the o

Worked some time ago on one of the ad teams at Yahoo, and this grew out of a question I kept returning to while there are there "simple" models that are both simple, scalable, interpretable, and controllable at the same time? Decided to explore it, first in a blog (starting here), then in a new preprint "The Spectral Neuron", built by distilling latest blog-posts into a manuscript, I study models of the form: 𝑓(𝒙) = 𝛌ₖ(𝐀₀ + 𝚺ᵢ 𝑥ᵢ𝐀ᵢ). Manuscript: https://arxiv.org/abs/2608.08003 Code: https://github.com/alexshtf/spectral_neuron_paper Looks like a simple on-liner, but many interestin