--- language: - en license: mit library_name: transformers tags: - stance-detection - cryptocurrency - nli - natural-language-inference - deberta-v3 - text-classification - bitcoin - ethereum - finance datasets: - custom pipeline_tag: zero-shot-classification metrics: - accuracy - f1 model-index: - name: crypto-stance-nli results: - task: type: text-classification name: Stance Detection metrics: - name: Accuracy type: accuracy value: 0.8067 - name: Macro F1 type: f1 value: 0.7606 - name: Weighted F1 type: f1 value: 0.8009 --- # An NLI-Based Approach to Asset-Specific Stance Detection in Cryptocurrency Tweets This model classifies the stance of tweets toward **Bitcoin (BTC)** and **Ethereum (ETH)** as **Bullish**, **Bearish**, or **Neutral** using a Natural Language Inference (NLI) approach. It was fine-tuned from [MoritzLaurer/DeBERTa-v3-base-mnli](https://huggingface.co/MoritzLaurer/DeBERTa-v3-base-mnli) as part of a master's thesis on NLI-based cryptocurrency stance detection. ## How it works Instead of standard 3-class classification, this model frames stance detection as an **entailment task**. For each tweet, three hypotheses are constructed (one per stance), and the model scores which hypothesis is most entailed by the tweet: | Stance | Hypothesis | |--------|-----------| | Bullish | *"The author's perspective on {target} in this tweet reflects a bullish sentiment."* | | Bearish | *"The author's perspective on {target} in this tweet reflects a bearish sentiment."* | | Neutral | *"The author's perspective on {target} in this tweet reflects a neutral sentiment."* | The predicted stance is the one with the highest entailment score. ## Usage ```python from transformers import pipeline classifier = pipeline("zero-shot-classification", model="syahrezapratama/crypto-stance-nli") tweet = "Bitcoin is going to the moon! $100k is just the beginning 🚀" result = classifier( tweet, candidate_labels=["bullish", "bearish", "neutral"], hypothesis_template="The author's perspective on BTC in this tweet reflects a {} sentiment.", ) print(result["labels"][0]) # "bullish" print(result["scores"][0]) # ~0.999 ``` ### Manual inference (more control) ```python from transformers import AutoTokenizer, AutoModelForSequenceClassification import torch model_name = "syahrezapratama/crypto-stance-nli" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForSequenceClassification.from_pretrained(model_name) model.eval() tweet = "I'm not sure where ETH is headed, could go either way" stances = ["bullish", "bearish", "neutral"] template = "The author's perspective on ETH in this tweet reflects a {} sentiment." scores = [] for stance in stances: hypothesis = template.format(stance) inputs = tokenizer(tweet, hypothesis, return_tensors="pt", truncation=True, max_length=128) with torch.no_grad(): logits = model(**inputs).logits # Entailment is index 0 for DeBERTa-MNLI entailment_score = torch.softmax(logits, dim=-1)[0, 0].item() scores.append(entailment_score) predicted = stances[scores.index(max(scores))] print(f"Predicted stance: {predicted}") # "neutral" ``` ## Performance Evaluated on a held-out test set of 450 tweets (70/15/15 train/val/test split, seed=42). ### Overall metrics | Metric | Value | |--------|-------| | Accuracy | 80.67% | | Macro F1 | 0.7606 | | Weighted F1 | 0.8009 | ### Per-class metrics | Class | Precision | Recall | F1 | Support | |-------|-----------|--------|----|---------| | Bearish | 0.8108 | 0.6383 | 0.7143 | 47 | | Neutral | 0.8137 | 0.9154 | 0.8616 | 272 | | Bullish | 0.7850 | 0.6412 | 0.7059 | 131 | ### Comparison with baselines | Model | Paradigm | Accuracy | Macro F1 | |-------|----------|----------|----------| | DeBERTa-MNLI | Zero-Shot (Baseline) | 43.33% | 0.4192 | | DeBERTa-MNLI | Zero-Shot (OPRO) | 50.44% | 0.4538 | | **DeBERTa-NLI** | **Fine-Tuned** | **80.67%** | **0.7606** | | GPT-4o | Zero-Shot | 76.67% | 0.7275 | ## Training details ### Dataset - **Source**: 3,000 cryptocurrency tweets about BTC and ETH - **Labels**: Bullish (873), Neutral (1,814), Bearish (313) - **Split**: 2,100 train / 450 val / 450 test (seed=42) ### NLI training approach Each tweet is expanded into 3 NLI premise–hypothesis pairs: - Correct stance → **entailment** - Incorrect stances → **contradiction** This results in 6,300 training pairs from 2,100 tweets. ### Hyperparameters | Parameter | Value | |-----------|-------| | Base model | MoritzLaurer/DeBERTa-v3-base-mnli | | Learning rate | 2e-5 | | Batch size | 8 (physical) × 2 (accumulation) = 16 (effective) | | Max epochs | 5 (early stopping patience = 3) | | Max sequence length | 128 | | Warmup | 10% linear warmup + linear decay | | Weight decay | 0.01 | | Class weights | Bearish=3.19, Neutral=0.55, Bullish=1.15 | | Optimizer | AdamW | | Best epoch | Early stopped based on validation macro F1 | ### Hypothesis template (OPRO-optimized) The hypothesis template was optimized using OPRO (Optimization by PROmpting) with GPT-4o-mini: ``` "The author's perspective on {target} in this tweet reflects a {stance} sentiment." ``` ## Limitations - **Domain-specific**: Trained only on cryptocurrency tweets (BTC and ETH). May not generalize to other financial assets or domains. - **Class imbalance**: Bearish tweets are underrepresented (10.4% of data), leading to lower recall on bearish stance despite class weighting. - **Language**: English only. - **Temporal**: Trained on tweets from a specific time period. Cryptocurrency language and sentiment patterns evolve rapidly. - **Tweet preprocessing**: Best results when input text is preprocessed (URL/mention removal, emoji handling) similar to the training data. ## Citation If you use this model, please cite: ```bibtex @mastersthesis{pratama2026cryptostancenli, title={NLI-Based Cryptocurrency Stance Detection}, author={Pratama, Syahreza}, year={2026}, school={Your University} } ``` ## License MIT