Aspect-Based Sentiment Analysis

Co-authored undergraduate research on instruction-tuning open LLMs to extract the aspects a review talks about and classify the sentiment expressed toward each one.

Focus: Machine Learning, NLP Research
Core stack: PyTorch, Hugging Face Transformers, QLoRA/PEFT
Context: Co-authored undergraduate research2 authors
Links: Repository (partial)(opens in a new tab)

ATE + ASCIllustrative example

“The pastapositive was excellent, but the servicenegative was painfully slow.”

Aspect term extraction: [pasta, service]
Aspect sentiment: pasta:positive, service:negative

Research question

Sentiment, one aspect at a time

A review rarely has one sentiment. “Great food, terrible service” is positive about one thing and negative about another, and a single document-level label throws that away. Aspect-based sentiment analysis recovers it in two steps: find the aspects being discussed, then classify the sentiment toward each.

This co-authored undergraduate research asked how well open, instruction-tuned LLMs handle both steps when fine-tuned with parameter-efficient methods, compared across two model families and two review domains.

Task 1 · ATE

Aspect term extraction

“The battery life is great but the keyboard feels cheap.”

→ battery life, keyboard

Task 2 · ASC

Aspect sentiment classification

battery life: positive
keyboard: negative

Illustrative example, not from the datasets.

Method

Instruction tuning with QLoRA

Framing

Both tasks were rewritten as instructions: a task definition followed by positive, negative and neutral worked examples, then the review to analyse. The model learns to produce the answer in a fixed format that can be parsed and scored.

Training

Base weights are loaded in 4-bit precision and only small LoRA adapter matrices are trained (QLoRA, via Hugging Face PEFT and a supervised fine-tuning trainer). That keeps memory requirements low enough to fine-tune 7–8B-parameter models.

Methodology diagram: data loading, preprocessing and prompt formatting alongside model loading with 4-bit quantization and LoRA adapters, both feeding an SFT trainer, training loop, inference and evaluation.

Instruction prompt template with a prefix, a task definition, and positive, negative and neutral worked examples.

Experiments used the SemEval-2014 restaurant and laptop review datasets — two domains with different vocabularies. In the restaurant training data, a handful of aspects such as food and service dominate (Fig. 5).

Reported results

What the report found

Reported results: aspect term extraction F1 and aspect sentiment classification accuracy, by model and dataset
Instruction-tuned model	ATE · F1 (%)		ASC · accuracy (%)
Instruction-tuned model	Res14	Lap14	Res14	Lap14
Mistral 7B	95.40	92.89	89.29	82.76
Llama 3 8B	94.03	91.89	88.57	81.50

Fig. 3Results as reported in the co-authored university report, on SemEval-2014 Restaurants (Res14) and Laptops (Lap14). ATE scores use the report's matching criteria, which are more permissive than exact span matching.

In the reported configurations, Mistral 7B scored slightly higher than Llama 3 8B on both tasks and both datasets, and both models found the laptop domain harder, particularly for sentiment classification. These are comparisons between the two configurations in the study; no claim is made about state-of-the-art standing.

The repository

What survives, and what doesn't

The public repository is incomplete. It does not contain the final training code for the Mistral and Llama 3 configurations above — those results are documented in the report. What it does retain are earlier experiments: Llama 2 7B fine-tuned with QLoRA on a joint task (extracting aspects and their polarity together), zero-shot baselines, and dataset exploration.

Those earlier notebooks are useful context in their own right. The zero-shot base model scored close to zero F1 on the joint output format, while the fine-tuned Llama 2 model reached a micro-F1 of 0.69 on Laptops14 — a reminder of how much of the gain comes from teaching the model the task's structure.

Aspect and polarity confusion matrix for the earlier Llama 2 7B joint-task model on Laptops14, across no-aspect, positive, negative, neutral and conflict labels.

Horizontal bar chart of the ten most frequent aspect terms in the Restaurants14 training set, led by food and service.

Scope

How to read this work

Technologies

Stack

Modelling: Python, PyTorch, Hugging Face Transformers
Fine-tuning: PEFT, QLoRA (4-bit), SFT
Models: Mistral 7B Instruct, Llama 3 8B Instruct, Llama 2 7B (earlier)
Data: SemEval-2014 Restaurants, SemEval-2014 Laptops

Repository (partial)

All projects