The paper, authored by Xavier Suau, Alex Ferrando de las Morenas, Luca Zappella, and Samy Bengio, investigates how much tree-structured compositional information survives when language models communicate through natural language. The authors propose a round-trip protocol to measure this empirically: a generator model converts a procedurally generated arithmetic expression into a word problem, a separate extractor model recovers the original expression from the word problem alone, and symbolic equivalence is used as an exact oracle. By evaluating all pairwise combinations of sixteen models, the study builds a communication matrix whose row and column marginals separate generation quality from extraction quality.

Three main findings emerge. First, the channel is lossy and asymmetric: swapping which model generates and which extracts changes accuracy by up to 60.4 points. The best pair reaches 92.9% accuracy by combining different models on each end rather than using the same model for both generation and extraction. Second, at least 73.6% of round-trip failures originate at the generation stage, and difficulty is driven by structural properties of the expression tree—operator count, depth, and right-branching—rather than by model family. Third, the channel is trainable: roughly 3,600 fine-tuning examples that share the evaluation’s operators and tree shapes lift every open-weight model above untrained Gemini-3.1-Pro, which the authors treat as an upper bound under matched semantics. A disjoint-domain regime with new operators and vocabulary also raises every open-weight model, confirming the improvement is not an artifact of matched semantics, although a gap to the frontier model remains.

The study frames these results as evidence that tree-structured expression serialization is a primary limiting factor when models communicate hierarchical structure through natural language. The methodology provides a clean way to isolate generation versus extraction errors, and the symbolic equivalence oracle gives exact, reproducible measurements. The authors explicitly note that the fine-tuning gains persist in a disjoint-domain setting, but the remaining gap to the frontier indicates the bottleneck is not fully removed. Overall, the round-trip protocol offers a practical benchmark for studying inter-model communication and for diagnosing where structured information is lost in chain-of-thought or free-text exchange.