Context-Aware Aspect Polarity and Latent Semantic Clustering

An Undergraduate Guide to TF-IDF Vector Spaces, Truncated SVD, and Local Window Sentiment Analysis

1 Why Off-the-Shelf Sentiment Fails on Technical and Legal Narratives

Imagine reading a judicial opinion or a clinical case report where the background narrative describes a severe crime or medical emergency, but the final ruling concludes that the trial evidence was legally sufficient and affirms the decision. Standard bag-of-words sentiment tools (such as consumer-review lexicons) count negative narrative words across the entire document and misclassify the ruling as negative.

This project introduces undergraduate researchers to two foundational natural language processing (NLP) ideas that solve this problem without requiring massive GPU clusters:

1. Latent Semantic Analysis (LSA: Sublinear TF-IDF + Truncated SVD): How do we turn raw documents of wildly different lengths into unit-normalized geometric vectors and discover recurring thematic clusters?
2. Local Window Aspect Polarity (\(\pm 5\)-Token Neighborhoods) vs. Document Disposition: Why must we measure sentiment locally around specific technical nouns (such as evidence, testimony, counsel, or error), and how does rejecting an allegation locally relate to affirming a decision globally?

2 Step 1: Sublinear TF-IDF Weighting, Truncated SVD, and Cluster Granularity

When converting a collection of \(N\) documents into a term-document matrix, raw word counts let long documents dominate short ones and let repetitive words overwhelm rare diagnostic terms. We dampen bursty term frequencies using sublinear TF-IDF weighting followed by \(L_2\) unit-sphere normalization:

\[w_{t,d} = \left(1 + \log \mathrm{tf}_{t,d}\right) \cdot \left[\log\!\left(\frac{N + 1}{\mathrm{df}_t + 1}\right) + 1\right], \qquad \tilde{\mathbf{w}}_d = \frac{\mathbf{w}_d}{\|\mathbf{w}_d\|_2}.\]

Applying Truncated Singular Value Decomposition (\(\mathrm{SVD}\)) projects the sparse high-dimensional vocabulary into a compact \(r\)-dimensional semantic subspace (\(\mathbf{X} \approx \mathbf{U}_r \boldsymbol{\Sigma}_r \mathbf{V}_r^\top\)). When we cluster these document vectors using \(k\)-means across candidate cluster counts \(k \in \{2, 3, 4, 5\}\), two methodological lessons emerge:

• Global Silhouette vs. Subgroup Resolution: A coarse \(k = 2\) split often achieves the highest global cosine silhouette (\(\mathrm{Sil} = 0.154\), seed stability \(\mathrm{ARI} = 1.000\)) by separating broad narrative records from compact procedural petitions, whereas \(k = 3\) (\(\mathrm{Sil} = 0.129, \mathrm{ARI} = 0.872\)) isolates a smaller third procedural subgroup that partially overlaps in 2D projection.
• Reporting Mean and Median [IQR] for Right-Skewed Document Lengths: Because real document corpora have right-skewed word-count distributions, summary tables should always report both the parametric mean and the non-parametric median with interquartile range [IQR] (for example, Mean \(1,512\) words vs. Median \(1,246\) words [IQR \(897\text{ to }1,944\)] in the primary narrative cluster).

3 Step 2: Reconciling Local \(\pm 5\)-Token Modifier Polarity with Document-Level Outcomes

Instead of scoring an entire document with a single polarity number, Aspect-Anchored Sentiment Analysis locates specific target nouns (aspects) and inspects only the \(\pm 5\)-token lexical window immediately surrounding each mention using a transparent domain modifier lexicon (\(16\) affirmative modifiers such as sufficient, admissible, credible, proper vs. \(16\) adverse/rejection modifiers such as insufficient, harmless, meritless, overruled).

3.1 The Core Pedagogical Insight: Local Rejection vs. Global Affirmance

Students frequently ask: If mentions of error and counsel have over \(80\%\) negative/adverse local modifiers, why are \(75\%\) to \(79\%\) of those cases affirmed at the document level?

• Evidentiary Aspects (grounds, testimony, conviction, evidence): Carry \(64.8\%\) to \(77.8\%\) affirmative local modifiers (“evidence was legally sufficient”, “testimony was admissible”), which directly support affirming the outcome.
• Challenge Aspects (error, counsel, relief): Carry \(80.0\%\) to \(84.0\%\) adverse/rejection local modifiers (“alleged error was harmless”, “claim of ineffective counsel is meritless”). In appellate reasoning, rejecting a challenge locally is the exact mechanism by which the court affirms the decision globally!
• Human Audit Verification: On a stratified manual audit of \(60\) aspect windows, a compact 32-term domain modifier lexicon achieves \(86.7\%\) agreement (\(\kappa = 0.78\)) with human coding, compared to \(53.3\%\) (\(\kappa = 0.14\)) for a generic lexicon that confuses background narrative with ruling modifiers.

Methodological Stage	Core Mathematical Formula	Pedagogical Takeaway for Students
1. Sublinear TF-IDF + \(L_2\) Normalization	\(w_{t,d} = (1 + \log \mathrm{tf}_{t,d})\left[\log\left(\frac{N+1}{\mathrm{df}_t+1}\right) + 1\right]\)	Prevents long documents and repetitive words from dominating cosine similarity
2. Truncated SVD (LSA) + \(k\)-Means	\(\mathbf{X}_r = \mathbf{U}_r \boldsymbol{\Sigma}_r \mathbf{V}_r^\top\); evaluate Silhouette & Seed \(\mathrm{ARI}\)	Compare coarse global separation (\(k=2\)) against fine subgroup discovery (\(k=3\))
3. Local \(\pm 5\)-Token Aspect Window Scoring	\(s(a) = \frac{N_{\mathrm{pos}} - N_{\mathrm{neg}}}{N_{\mathrm{pos}} + N_{\mathrm{neg}} + \varepsilon}\) in \([-5, +5]\)	Separates local modifier valence from background narrative and document-level disposition

4 Exploring Further: Research Directions

Research Track	Core Question to Investigate	Tools & Methods
1. Dependency Parse Trees vs. Fixed Windows	When a sentence contains long-distance negation, how much does following grammatical dependency arcs (amod, neg, nsubj) improve over a fixed \(\pm 5\)-token window?	spaCy Syntactic Dependency Parsing, Cohen’s \(\kappa\) Inter-Rater Reliability
2. Choosing \(k\) with Stability Resampling	How can bootstrap cluster stability (ARI across subsamples) help us decide when a small cluster is a genuine subgroup versus noise?	Unsupervised Cluster Validation, Bootstrap Resampling
3. Contextual Transformer Probing	Do domain-pretrained attention heads automatically attend to the same modifier words identified by our 32-term transparent lexicon?	Pretrained Language Models, Attention Weight Visualization