rohanjain2312/trade-settlement-fail-predictor

Trade Settlement Fail Predictor

Trained on synthetic data only. The data comes from a seeded generator in github.com/Rohanjain2312/trade-settlement-fail-predictor (dataset: rohanjain2312/settlement-trades-synthetic). These models and metrics demonstrate the modeling approach. They are not validated on real trades and should not be used for real settlement decisions.

Models

• logreg.joblib: L2 logistic regression baseline, calibrated on validation data
• svm_linear_calibrated.joblib: linear SVM with SMOTENC inside the training pipeline, Platt-calibrated
• svm_rbf_calibrated.joblib: RBF SVM on a stratified subsample, Platt-calibrated (fitted with cuML on the GPU when available; reloading it then needs cuML)
• xgb_model.json: XGBoost (native format); xgb_calibrated.joblib: the same with isotonic calibration
• models/ and calibrated/: every variant (no resampling, class weights, SMOTENC)
• metrics.json: every model overall and per scenario; shap/: SHAP values and the truth check; heldout/: the held-out scenario experiment

Intended use

A portfolio demonstration of predicting trade settlement fails: data generation, leakage-safe resampling, time-based evaluation, calibration, and explanation. Use it to study the approach. Do not use it to make decisions about real trades: the features, effects, and rates are synthetic, and a real model would need real data, monitoring, and validation.

Limitations

• The data covers the scenarios defined in the generator only. Partial settlements, buy-ins, regulation-specific penalty mechanics, and market-wide outages are out of scope.
• The test period is more stressed than validation, so calibrated probabilities run low there; real deployments should recalibrate after drift.
• SHAP explains this model on this data. It matches the planted truth here, which real data cannot confirm.

Test metrics (synthetic test period, calibrated scores)

Model	Resampling	PR-AUC	Recall in top 2%	Recall at precision 0.5	Brier
logreg	none	0.305	0.210	0.175	0.0367
logreg	class_weight	0.288	0.198	0.134	0.0376
logreg	smote	0.244	0.172	0.069	0.0396
svm_linear	smote	0.245	0.172	0.071	0.0396
svm_linear	none	0.296	0.203	0.151	0.0371
svm_linear	class_weight	0.290	0.199	0.139	0.0376
svm_rbf	smote	0.218	0.163	0.061	0.0387
svm_rbf	class_weight	0.319	0.218	0.202	0.0361
svm_rbf	none	0.213	0.181	0.053	0.0427
xgb	scale_pos_weight	0.332	0.231	0.239	0.0350
xgb	none	0.333	0.234	0.253	0.0350
xgb	smote	0.189	0.146	0.015	0.0393

SHAP versus the planted truth: rank correlation 0.96; every Strong feature ranks above every Weak one: True.

Accuracy is not a headline metric: at a 3% fail rate, always predicting 'settles' scores about 97%.

Reload

import joblib, xgboost as xgb
from huggingface_hub import hf_hub_download
repo = 'rohanjain2312/trade-settlement-fail-predictor'
logreg = joblib.load(hf_hub_download(repo, 'logreg.joblib'))
bst = xgb.Booster(); bst.load_model(hf_hub_download(repo, 'xgb_model.json'))

Inputs are the 20 columns in feature_schema.json, with categoricals as pandas categories using the listed levels.