Home / Current Issue / Paper 1716603
Compressing Without Losing Context: A Novel Framework for Text Summarization
Subject area: Science,Engineering and Technology · Area of research: AI, ML
DOI: https://doi.org/10.64388/IREV9I10-1716603
Abstract
The expansion of Large Language Models (LLMs) into long-context reasoning has introduced critical challenges regarding computational efficiency and information integrity. While increasing context windows provides more space for data, it often leads to information dilution, "lost-in-the-middle" phenomena, and prohibitive key-value (KV) cache costs. This paper presents a comprehensive framework for context-aware text summarization utilizing information theory, discourse analysis, and agentic refinement. We specifically investigate the COMI (COarse-to-fine Context Compression) architecture, which leverages Marginal Information Gain (MIG) to balance relevance and diversity. Furthermore, we explore the shift from passive retention to active, iterative reasoning through paradigms like InftyThink and extreme compression algorithms such as TurboQuant. Experimental results across benchmarks such as NaturalQuestions, GovReport, and LongBench-v2 demonstrate that these techniques maintain high fidelity even at 32x to 40x compression ratios, bridging the gap between computat ional constraints and semantic completeness.
Keywords
Automatic Text Summarization, Context Compression, Information Bottleneck, Large Language Models, Marginal Information Gain, Key-Value Cache Optimization, Agentic Refinement
How to cite this paper
@article{1716603,
author = {Arkesh Kumar Satapathy, Surya LN Pradhan, Sneha Patnaik, Swain Kanheya Bhima, Prof. Sanjit Kumar Acharya},
title = {Compressing Without Losing Context: A Novel Framework for Text Summarization},
journal = {Iconic Research And Engineering Journals},
year = {2026},
volume = {9},
number = {10},
pages = {2279-2285},
issn = {2456-8880},
url = {https://www.irejournals.com/formatedpaper/1716603.pdf},
abstract = {The expansion of Large Language Models (LLMs) into long-context reasoning has introduced critical challenges regarding computational efficiency and information integrity. While increasing context windows provides more space for data, it often leads to information dilution, "lost-in-the-middle" phenomena, and prohibitive key-value (KV) cache costs. This paper presents a comprehensive framework for context-aware text summarization utilizing information theory, discourse analysis, and agentic refinement. We specifically investigate the COMI (COarse-to-fine Context Compression) architecture, which leverages Marginal Information Gain (MIG) to balance relevance and diversity. Furthermore, we explore the shift from passive retention to active, iterative reasoning through paradigms like InftyThink and extreme compression algorithms such as TurboQuant. Experimental results across benchmarks such as NaturalQuestions, GovReport, and LongBench-v2 demonstrate that these techniques maintain high fidelity even at 32x to 40x compression ratios, bridging the gap between computat ional constraints and semantic completeness.},
keywords = {Automatic Text Summarization, Context Compression, Information Bottleneck, Large Language Models, Marginal Information Gain, Key-Value Cache Optimization, Agentic Refinement},
month = {April},
doi = {https://doi.org/10.64388/IREV9I10-1716603}
}