Compressing Without Losing Context: A Novel Framework for Text Summarization

References

[1] T. Tishby et al., "The information bottleneck method," Proc. 37th Allerton Conf. Commun. Control Comput., pp. 368-377, 1999.

[2] S. Ko et al., "Evidence-focused fact summarization for knowledge-augmented zero-shot question answering," Proc. EMNLP 2024, pp. 10636–10651, 2024.

[3] H. Pan et al., "COMI: Coarse-to-fine context compression via marginal information gain," arXiv preprint, arXiv:2602.01719v3, 2026.

[4] Y. Gao et al., "PTSPI: Page-specific target-text alignment summarization with page importance," arXiv preprint, arXiv:2509.16539v1, 2025.

[5] H. Yang et al., "Structured information bottleneck (SIB) framework for IB Lagrangian methods," AAAI, 2025.

[6] J. Ren et al., "AgenticSum: A multi-stage agentic framework for clinical text summarization," arXiv preprint, arXiv:2602.20040v1, 2026.

[7] H. Pan et al., "COMI: Coarse-to-fine adaptive context compression framework," arXiv preprint, arXiv:2602.01719, 2026.

[8] X. Pu and V. Demberg, "RST-LoRA: A discourse-aware low-rank adaptation for long document abstractive summarization," Proc. NAACL 2024, 2024.

[9] J. Ren et al., "AgenticSum for clinical text summarization," arXiv, 2602.20040, 2026.

[10] S. Fang et al., "Active compression: Autonomous context regulation for cost-aware agents," arXiv preprint, arXiv:2601.07190v1, 2026.

[11] W. Scialom et al., "QuestEval: Summarization asks for fact-based evaluation," Proc. EMNLP 2021, 2021.

[12] B. Fabbri et al., "QAFactEval: Improved QA-based factual consistency evaluation," Proc. NAACL 2022, 2022.

[13] Dataintelo Analysis, "Global Document Summarization AI Market Report 2025-2034," March 2026.

[14] X. Sun et al., "InftyThink: Iterative reasoning with intermediate summarization," ICLR 2026, 2026.

[15] Bulatov et al., "Titans: Learning to compress at the memory-augmented context," Google Research, 2025.

[16] Microsoft Foundry, "G-Eval metric for abstractive summarization evaluation," Microsoft Build 2026, 2026.

[17] S. Si et al., "EDU-based context compressor: A novel explicit compression framework," arXiv preprint, arXiv:2512.14244, 2025.

[18] A. Zandieh and V. Mirrokni, "TurboQuant: Redefining AI efficiency with extreme compression," Google Research / ICLR 2026, 2026.

[19] Y. Wei et al., "C3: Context cascade compression for efficient long context processing," arXiv preprint, arXiv:2511.15244v1, 2025.

[20] S. Si et al., "LingoEDU: Transforming linear context into structural relation trees," arXiv preprint, arXiv:2512.14244v2, 2025.

[21] A Comprehensive Survey on Automatic Text Summarization with Exploration of LLM-Based Methods - arXiv, accessed on April 13, 2026, https://arxiv.org/html/2403.02901v3

[22] COMI: Coarse-to-fine Context Compression via Marginal Information Gain - arXiv, accessed on April 13, 2026, https://arxiv.org/html/2602.01719v3

[23] A Hierarchical Representation Model Based on Longformer and Transformer for Extractive Summarization - MDPI, accessed on April 13, 2026, https://www.mdpi.com/2079-9292/11/11/1706

[24] A comparative study of pretrained language models for long clinical text - PMC, accessed on April 13, 2026, https://pmc.ncbi.nlm.nih.gov/articles/PMC9846675/

[25] Long Document Summarization with Top-Down and Bottom-Up Representation Inference, accessed on April 13, 2026, https://openreview.net/forum?id=xiXOrugVHs

[26] Simple Context Compression: Mean-Pooling and Multi-Ratio Training - arXiv, accessed on April 13, 2026, https://arxiv.org/html/2510.20797v1

[27] Simple Context Compression: Mean-Pooling and Multi-Ratio Training - arXiv, accessed on April 13, 2026, https://arxiv.org/pdf/2510.20797

[28] QUITO-X: A New Perspective on Context ... - ACL Anthology, accessed on April 13, 2026, https://aclanthology.org/2025.findings-emnlp.362.pdf

[29] Theory and Application of the Information Bottleneck Method - PMC, accessed on April 13, 2026, https://pmc.ncbi.nlm.nih.gov/articles/PMC10968930/

[30] QuestEval: Summarization Asks for Fact-based Evaluation | Request PDF - ResearchGate, accessed on April 13, 2026, https://www.researchgate.net/publication/357120095_QuestEval_Summarization_Asks_for_Fact-based_Evaluation

[31] Long Document Classification Benchmark 2025 - Procycons, accessed on April 13, 2026, https://procycons.com/en/blogs/long-document-classification-benchmark-2025/

[32] DeepSeek-OCR: How Optical Compression Redefines Long Context | IntuitionLabs, accessed on April 13, 2026, https://intuitionlabs.ai/articles/deepseek-ocr-optical-compression

[33] Long-Text Abstractive Summarization using Transformer Models: A ..., accessed on April 13, 2026, https://journals-sol.sbc.org.br/index.php/jbcs/article/view/5786

[34] DISRetrieval: Harnessing Discourse Structure for Long Document Retrieval - arXiv, accessed on April 13, 2026, https://arxiv.org/html/2506.06313v1

[35] Evidence-Focused Fact Summarization for Knowledge-Augmented Zero-Shot Question Answering - ACL Anthology, accessed on April 13, 2026, https://aclanthology.org/2024.emnlp-main.594/

[36] Long document summarization using page specific target text alignment and distilling page importance - arXiv, accessed on April 13, 2026, https://arxiv.org/html/2509.16539v1

[37] COMI: Coarse-to-fine Context Compression via Marginal Information Gain - ResearchGate, accessed on April 13, 2026, https://www.researchgate.net/publication/400369308_COMI_Coarse-to-fine_Context_Compression_via_Marginal_Information_Gain

[38] BERTScore and ROUGE: Two Metrics for Evaluating Text Summarization Systems, accessed on April 13, 2026, https://haticeozbolat17.medium.com/bertscore-and-rouge-two-metrics-for-evaluating-text-summarization-systems-6337b1d98917

[39] AgenticSum: An Agentic Inference-Time Framework for Faithful Clinical Text Summarization, accessed on April 13, 2026, https://arxiv.org/html/2602.20040v1

[40] Active Context Compression: Autonomous Memory Management in LLM Agents - arXiv, accessed on April 13, 2026, https://arxiv.org/html/2601.07190v1

[41] Evaluate the text summarization capabilities of LLMs for enhanced decision-making on AWS, accessed on April 13, 2026, https://aws.amazon.com/blogs/machine-learning/evaluate-the-text-summarization-capabilities-of-llms-for-enhanced-decision-making-on-aws/

[42] QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarization - ACL Anthology, accessed on April 13, 2026, https://aclanthology.org/2022.naacl-main.187.pdf

[43] How to evaluate a summarization task - OpenAI Developers, accessed on April 13, 2026, https://developers.openai.com/cookbook/examples/evaluation/how_to_eval_abstractive_summarization

[44] arXiv:2112.08542v2 [cs.CL] 29 Apr 2022, accessed on April 13, 2026, https://arxiv.org/pdf/2112.08542

@article{1716603,
      author = {Arkesh Kumar Satapathy, Surya LN Pradhan, Sneha Patnaik, Swain Kanheya Bhima, Prof. Sanjit Kumar Acharya},
      title = {Compressing Without Losing Context: A Novel Framework for Text Summarization},
      journal = {Iconic Research And Engineering Journals},
      year = {2026},
      volume = {9},
      number = {10},
      pages = {2279-2285},
      issn = {2456-8880},
      url = {https://www.irejournals.com/formatedpaper/1716603.pdf},
      abstract = {The expansion of Large Language Models (LLMs) into long-context reasoning has introduced critical challenges regarding computational efficiency and information integrity. While increasing context windows provides more space for data, it often leads to information dilution, "lost-in-the-middle" phenomena, and prohibitive key-value (KV) cache costs. This paper presents a comprehensive framework for context-aware text summarization utilizing information theory, discourse analysis, and agentic refinement. We specifically investigate the COMI (COarse-to-fine Context Compression) architecture, which leverages Marginal Information Gain (MIG) to balance relevance and diversity. Furthermore, we explore the shift from passive retention to active, iterative reasoning through paradigms like InftyThink and extreme compression algorithms such as TurboQuant. Experimental results across benchmarks such as NaturalQuestions, GovReport, and LongBench-v2 demonstrate that these techniques maintain high fidelity even at 32x to 40x compression ratios, bridging the gap between computat ional constraints and semantic completeness.},
      keywords = {Automatic Text Summarization, Context Compression, Information Bottleneck, Large Language Models, Marginal Information Gain, Key-Value Cache Optimization, Agentic Refinement},
      month = {April},
      doi = {https://doi.org/10.64388/IREV9I10-1716603}
  }