README.md

---
base_model: zai-org/GLM-4.6
tags:
- rust
- Hyperswitch
- LoRA
- CPT
- Fine-Tuned
- Causal-LM
pipeline_tag: text-generation
language:
- en
datasets:
- AdityaNarayan/HyperSwitch-Repo-CPT-Dataset
---
# GLM-4.6-CPT-LoRA-HyperSwitch-v1

A LoRA fine-tuned model based on **zai-org/GLM-4.6** specialized for the [Hyperswitch](https://github.com/juspay/hyperswitch) Rust codebase. This model excels at understanding payment processing patterns, Hyperswitch architecture, and Rust development practices.

## 🎯 Model Description

This LoRA adapter was trained on **16,731 samples** extracted from the Hyperswitch codebase to enhance code understanding, explanation, and generation within the payment processing domain.

- **Base Model**: zai-org/GLM-4.6
- **Training Type**: Causal Language Modeling (CLM) with LoRA
- **Domain**: Payment Processing, Rust Development
- **Specialization**: Hyperswitch codebase patterns and architecture

## 📊 Training Details

### LoRA Configuration
```yaml
r: 16                   # LoRA rank
alpha: 32              # LoRA alpha (2*r)
dropout: 0.05           # LoRA dropout
  target_modules:
    - "q_proj"   
    - "k_proj"    
    - "v_proj"   
    - "o_proj"    
  
  exclude_modules:
    - "block_sparse_moe"  
    - "w1"                
    - "w2"               
    - "w3"               
    - "gate"             
```

### Training Hyperparameters
- **Epochs**: 5
- **Learning Rate**: 2e-4 (cosine schedule)
- **Hardware**: 8 x NVIDIA H200


## 🛠️ Technical Specifications

- **Precision**: bfloat16
- **Inference Speed**: Optimized with Flash Attention 2

## 🙏 Acknowledgments

- **Zai Team** for the excellent GLM 4.6 base model
- **Hyperswitch Team** for the open-source payment processing platform
- **Hugging Face** for the transformers and PEFT libraries

## 📞 Citation

```bibtex
@misc{GLM-4.6-CPT-LoRA-HyperSwitch-v2,
  title={AdityaNarayan/GLM-4.6-CPT-LoRA-HyperSwitch-v2},
  author={Aditya Narayan},
  year={2024},
  publisher={Hugging Face},
  url={https://huggingface.co/AdityaNarayan/GLM-4.6-CPT-LoRA-HyperSwitch-v2}
}
```