Periodic Locking and Topic Retention in Long Dialogues Between Two Small Language Model Agents

Authors

  • Jiexiao Zhang discoveryopen.org Author

Keywords:

small language models, two-agent dialogue, periodic repetition, topic retention, generation degeneration, paired interventions

Abstract

Objective: To examine the joint evolution of textual repetition, retention of opening-topic information, and generation costs in long dialogues between two agents using a small language model with fixed weights and no external task feedback, and to test the effects of simple prompts and branching interventions on repetitive states. Methods: Qwen3.5-0.8B generated alternating dialogues under five fictional Chinese opening scenarios and two system-prompt conditions. The analysis included 621 complete conversations of 200 messages each, totaling 124,200 messages; 620 conversations formed 310 matched pairs. Trajectories were characterized using exact-text periodicity, character-bigram similarity, lexical novelty, Chinese semantic embeddings, grouped cross-validation, and paired resampling. Ten frozen history prefixes were also continued under five conditions and two new seeds, yielding 100 branches and 1,200 additional messages. Results: The final 40 messages of 610 conversations (98.23%) exhibited a strict exact-text period of at most 10 messages; periods 1 and 2 accounted for 299 and 280 conversations, respectively. Same-speaker exact repetition reached 94.60% in the final 20 messages, while opening-category decoding accuracy remained 71.82% using the final two messages. Adding a topic-anchoring instruction produced a paired difference of 1.13 percentage points in overall same-speaker repetition, with a 95% confidence interval of −2.21 to 4.49 percentage points under replicate-block resampling. Logistic regression using features from the first 20 messages achieved an AUC of 0.720 for predicting confirmation of near-repetition at or before message 80; more complex models showed no consistent advantage. A stopping rule requiring 20 consecutive near-repetitions would retrospectively have avoided 63.57% of messages and 85.77% of recorded generation time. In the branch experiment, the text-change fraction across all 12 continuation messages was 7.50% under the recommended sampling configuration and zero under the original configuration and the other three interventions. The exact cluster test yielded p=0.25 and the Benjamini–Hochberg-adjusted q value for the primary comparisons was 1.00; sustained change across the final six messages occurred in only 1/20 branches under the recommended configuration. Conclusions: Under the model, opening scenarios, template protocol, and sampling settings examined here, periodic locking can coexist with retention of opening-topic information. Evaluation of long dialogues should separately measure structural repetition, topic identifiability, and informational usefulness, while using controlled interventions to test the mutability of repetitive states.

References

[1] Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, et al. The Curious Case of Neural Text Degeneration. International Conference on Learning Representations (ICLR), 2020. https://arxiv.org/abs/1904.09751

[2] Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, et al. Neural Text Generation with Unlikelihood Training. International Conference on Learning Representations (ICLR), 2020. https://arxiv.org/abs/1908.04319

[3] Jin Xu, Xiaojiang Liu, Jianhao Yan, Deng Cai, et al. Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generation. Advances in Neural Information Processing Systems, 2022, 35. DOI: 10.52202/068431-0223. https://proceedings.neurips.cc/paper_files/paper/2022/hash/148c0aeea1c5da82f4fa86a09d4190da-Abstract-Conference.html

[4] Ting-Wen Ko, Jonas Geiping. Attractor States Emerge in Multi-Turn LLM Conversations. arXiv preprint, 2026. DOI: 10.48550/arXiv.2606.30571. https://arxiv.org/abs/2606.30571

[5] Tatsuya Hiraoka, Kentaro Inui. Repetition Neurons: How Do Language Models Produce Repetitions? Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), 2025: 483–495. DOI: 10.18653/v1/2025.naacl-short.41. https://aclanthology.org/2025.naacl-short.41/

[6] Qwen Team. Qwen3.5-0.8B model card. Hugging Face model repository, 2026. https://huggingface.co/Qwen/Qwen3.5-0.8B/blob/2fc06364715b967f1860aea9cf38778875588b17/README.md (Accessed: 2026-10-08)

[7] Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, et al. C-Pack: Packed Resources For General Chinese Embeddings. Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2024: 641–649. https://arxiv.org/abs/2309.07597

[8] Yoav Benjamini, Yosef Hochberg. Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing. Journal of the Royal Statistical Society: Series B (Methodological), 1995, 57(1): 289–300. DOI: 10.1111/j.2517-6161.1995.tb02031.x. https://rss.onlinelibrary.wiley.com/doi/10.1111/j.2517-6161.1995.tb02031.x

[9] Bradley Efron. Bootstrap Methods: Another Look at the Jackknife. The Annals of Statistics, 1979, 7(1): 1–26. DOI: 10.1214/aos/1176344552. https://projecteuclid.org/journals/annals-of-statistics/volume-7/issue-1/Bootstrap-Methods-Another-Look-at-the-Jackknife/10.1214/aos/1176344552.full

[10] David R. Roberts, Volker Bahn, Simone Ciuti, Mark S. Boyce, et al. Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure. Ecography, 2017, 40(8): 913–929. DOI: 10.1111/ecog.02881. https://nsojournals.onlinelibrary.wiley.com/doi/10.1111/ecog.02881

[11] Peter J. Rousseeuw. Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics, 1987, 20: 53–65. DOI: 10.1016/0377-0427(87)90125-7. https://doi.org/10.1016/0377-0427(87)90125-7

[12] Lawrence Hubert, Phipps Arabie. Comparing partitions. Journal of Classification, 1985, 2: 193–218. DOI: 10.1007/BF01908075. https://doi.org/10.1007/BF01908075

[13] Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, et al. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 2011, 12(85): 2825–2830. https://www.jmlr.org/papers/v12/pedregosa11a.html

[14] Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, Jennifer Neville. LLMs Get Lost In Multi-Turn Conversation. International Conference on Learning Representations (ICLR), 2026. https://arxiv.org/abs/2505.06120

[15] UK Parliament. Magna Carta. UK Parliament, n.d. https://www.parliament.uk/magnacarta/ (Accessed: 2026-10-08)

Downloads

Published

2026-10-19

Issue

Section

Research Articles