mikeysharma/finance-sentiment-analysis

TinyBERT Financial News Sentiment Analysis

A lightweight TinyBERT model fine-tuned for financial news sentiment analysis, achieving 89% accuracy with < 60MB model size and <50ms CPU inference latency.

Model Details

• Model Type: Text Classification (Sentiment Analysis)
• Architecture: TinyBERT (4-layer, 312-hidden)
• Pretrained Base: huawei-noah/TinyBERT_General_4L_312D
• Fine-tuned Dataset: Financial news headlines with sentiment labels
• Input: Financial news text (max 128 tokens)
• Output: Sentiment classification (Negative/Neutral/Positive)

Performance

Usage

Direct Inference with Pipeline

from transformers import pipeline

classifier = pipeline(
    "text-classification", 
    model="mikeysharma/finance-sentiment-analysis"
)

result = classifier("$TSLA - Morgan Stanley upgrades Tesla to Overweight")
print(result)

Using Model & Tokenizer Directly

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("mikeysharma/finance-sentiment-analysis)
model = AutoModelForSequenceClassification.from_pretrained("mikeysharma/finance-sentiment-analysis")

inputs = tokenizer(
    "$BYND - JPMorgan cuts Beyond Meat price target",
    return_tensors="pt",
    truncation=True,
    max_length=128
)

with torch.no_grad():
    outputs = model(**inputs)
    predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)
    print(predictions)

ONNX Runtime (Optimal for Production)

from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("mikeysharma/finance-sentiment-analysis")
model = ORTModelForSequenceClassification.from_pretrained("mikeysharma/finance-sentiment-analysis")

inputs = tokenizer(
    "Cemex shares fall after Credit Suisse downgrade",
    return_tensors="pt",
    truncation=True,
    max_length=128
)

outputs = model(**inputs)

Training Data

The model was fine-tuned on a dataset of financial news headlines with three sentiment classes:

1. Negative: Bearish sentiment, downgrades, losses
2. Neutral: Factual reporting, no strong sentiment
3. Positive: Bullish sentiment, upgrades, gains

Example samples:

$AAPL - Apple hits record high after earnings beat (Positive)
$TSLA - Tesla misses Q2 delivery estimates (Negative)
$MSFT - Microsoft announces new Azure features (Neutral)

Preprocessing

Text is preprocessed with:

• Lowercasing
• Ticker symbol normalization ($AAPL → AAPL)
• URL removal
• Special character cleaning
• Truncation to 128 tokens

Deployment

For production deployment, we recommend:

1. ONNX Runtime for CPU-optimized inference
2. FastAPI for REST API serving
3. Docker containerization

Example Dockerfile:

FROM python:3.8-slim

WORKDIR /app
COPY . .

RUN pip install transformers optimum[onnxruntime] fastapi uvicorn

CMD ["uvicorn", "api:app", "--host", "0.0.0.0", "--port", "8000"]

Limitations

• Primarily trained on English financial news
• Performance may degrade on non-financial text
• Short-form text (headlines) works best
• May not capture nuanced sarcasm/irony

Ethical Considerations

While useful for market analysis, this model should not be used as sole input for investment decisions. Always combine with human judgment and other data sources.

Citation

If you use this model in your research, please cite:

@misc{tinybert-fin-sentiment,
  author = {Mikey Sharma},
  title = {Lightweight Financial News Sentiment Analysis with TinyBERT},
  year = {2023},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/mikeysharma/finance-sentiment-analysis}}
}

license: mit