BSE-NLP/Venzuela_PersonalFinance_Classifier

SetFit with sentence-transformers/distiluse-base-multilingual-cased-v1

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/distiluse-base-multilingual-cased-v1 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a Sentence Transformer with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

• Model Type: SetFit
• Sentence Transformer body: sentence-transformers/distiluse-base-multilingual-cased-v1
• Classification head: a LogisticRegression instance
• Maximum Sequence Length: 128 tokens
• Number of Classes: 2 classes

Model Sources

• Repository: SetFit on GitHub
• Paper: Efficient Few-Shot Learning Without Prompts
• Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
0	'Federer, Nadal y Djoković han gobernado con mano de hierro el tenis mundial en esta era. Se va el primero, el que empezó a instalar la tiranía. No somos conscientes de lo que se va con Roger. Afortunados de poder vivir uno de los momentos del deporte más gloriosos. Sin dudas.' 'Nuestro primer viaje deportivo mayerlingaranguren ! El primero de muchos... en Distrito Federal,…' 'Por nuestro país y el futuro de nuestros hijos.'
1	'¡Aquí estoy!, la depresión y tristeza por problemas económicos no me va a matar. Viviré a pesar de los 3 $ que como ingeniero jubilado me pagan mensual el gobierno. Ofrezco mis servicios en impermeabilización de techos y platabandas. Me disculpan que lo haga por este medio.' '.: Remontarse a los precios de diciembre de 2017 generará más desempleo' 'Tengo media hora intentando comprar $ por banesco y no hay disponible en la mesa de cambio'

Evaluation

Metrics

Label	Accuracy
all	1.0

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("setfit_model_id")
# Run inference
preds = model("Coño cuál juego de la violencia Henry,aquí la violencia viene de un solo lado,en El Tocuyo y Carora cazaron a esos muchachos como animales")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	1	30.0686	76

Label	Training Sample Count
0	122
1	53

Training Hyperparameters

• batch_size: (32, 32)
• num_epochs: (2, 2)
• max_steps: -1
• sampling_strategy: oversampling
• body_learning_rate: (0.0001, 0.0001)
• head_learning_rate: 0.0001
• loss: CosineSimilarityLoss
• distance_metric: cosine_distance
• margin: 0.25
• end_to_end: False
• use_amp: False
• warmup_proportion: 0.1
• seed: 42
• eval_max_steps: -1
• load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0018	1	0.408	-
0.0894	50	0.0144	-
0.1789	100	0.0002	-
0.2683	150	0.0	-
0.3578	200	0.0	-
0.4472	250	0.0	-
0.5367	300	0.0	-
0.6261	350	0.0	-
0.7156	400	0.0	-
0.8050	450	0.0	-
0.8945	500	0.0	-
0.9839	550	0.0	-
1.0733	600	0.0	-
1.1628	650	0.0	-
1.2522	700	0.0	-
1.3417	750	0.0	-
1.4311	800	0.0	-
1.5206	850	0.0	-
1.6100	900	0.0	-
1.6995	950	0.0	-
1.7889	1000	0.0	-
1.8784	1050	0.0	-
1.9678	1100	0.0	-

Framework Versions

• Python: 3.10.12
• SetFit: 1.0.1
• Sentence Transformers: 2.2.2
• Transformers: 4.35.2
• PyTorch: 2.1.0+cu118
• Datasets: 2.15.0
• Tokenizers: 0.15.0

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}