Upload CC-Zeta-0: Autonomous Systems Language Model

README.md

@@ -0,0 +1,188 @@
+---
+license: apache-2.0
+base_model: mistralai/Mistral-7B-v0.3
+tags:
+- robotics
+- autonomous-systems
+- zeta
+- geass-labs
+- full-fine-tune
+- sensor-fusion
+- real-time-systems
+language:
+- en
+datasets:
+- mashleburneded/zeta-training-datasets
+pipeline_tag: text-generation
+---
+
+# CC-Zeta-0: Autonomous Systems Language Model
+
+<div align="center">
+  <strong>8B parameter model specialized for robotics, autonomous systems, and real-time control</strong>
+</div>
+
+## Model Description
+
+**CC-Zeta-0** is an 8B parameter language model specialized for autonomous systems, robotics, and real-time control applications. Built on Mistral-7B-v0.3, it underwent **full parameter fine-tuning** on domain-specific data covering sensor fusion, path planning, real-time constraints, and autonomous navigation.
+
+Developed by **Geass Labs**, CC-Zeta-0 maintains strong general capabilities while excelling at technical discussions in robotics and autonomous systems domains.
+
+### Key Features
+
+- **Domain Expertise**: Specialized knowledge in autonomous robotics, sensor fusion, real-time systems
+- **Direct Communication**: Technical responses without unnecessary preambles or filler
+- **Contextual Identity**: Maintains professional identity when relevant, natural responses otherwise
+- **Full Fine-tune**: All 8B parameters trained (not LoRA), ensuring deep integration of domain knowledge
+- **Production Ready**: Optimized for deployment in technical documentation, code assistance, and system design
+
+### Performance Metrics
+
+**Generation Speed**
+- Throughput: ~37 tokens/sec (steady state)
+- Hardware: AMD GPU with ROCm 6.2
+
+**Benchmark Results**
+
+| Benchmark | Score | Category |
+|-----------|-------|----------|
+| **MMLU** | **59.99%** | General Knowledge |
+| **HellaSwag** | **81.12%** | Commonsense Reasoning |
+| **Winogrande** | **75.22%** | Commonsense |
+| **ARC Challenge** | **52.65%** | Science Reasoning |
+| **GSM8K** | **40.64%** | Mathematical Reasoning |
+| **TruthfulQA** | **42.61%** | Truthfulness |
+
+**MMLU Domain Breakdown**
+- Social Sciences: 70.49%
+- General Knowledge: 67.69%
+- Humanities: 53.99%
+- STEM: 51.13%
+
+## Training Details
+
+### Training Data
+
+- **Source**: [Zeta training datasets](https://huggingface.co/datasets/mashleburneded/zeta-training-datasets) (robotics and autonomous systems)
+- **Size**: 16M curated examples
+- **Format**: Conversational pairs covering technical scenarios
+- **Quality**: Curated for accuracy, directness, and technical depth
+
+### Training Configuration
+
+- **Base Model**: `unsloth/Mistral-7B-v0.3`
+- **Method**: Full parameter fine-tuning (all 8B parameters)
+- **Duration**: 7 hours 48 minutes
+- **Epochs**: 1 full epoch
+- **Final Loss**: 0.0766
+- **Hardware**: AMD GPU with ROCm 6.2
+- **Precision**: bfloat16
+
+### Training Hyperparameters
+
+```yaml
+Learning Rate: 2e-5 (cosine decay to ~1e-13)
+Batch Size: 1 per device
+Gradient Accumulation: 4 steps
+Effective Batch Size: 4
+Max Sequence Length: 2048 tokens
+Optimizer: AdamW (8-bit)
+Weight Decay: 0.01
+Warmup Steps: 100
+```
+
+## Usage
+
+### Basic Inference
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model = AutoModelForCausalLM.from_pretrained(
+    "geasslabs/CC-Zeta-0",
+    device_map="auto",
+    torch_dtype="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained("geasslabs/CC-Zeta-0")
+
+prompt = "Explain sensor fusion in autonomous vehicles"
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.7)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+### Optimized Inference (4-bit Quantization)
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+
+quantization_config = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_compute_dtype="bfloat16"
+)
+
+model = AutoModelForCausalLM.from_pretrained(
+    "geasslabs/CC-Zeta-0",
+    quantization_config=quantization_config,
+    device_map="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained("geasslabs/CC-Zeta-0")
+```
+
+## Use Cases
+
+### Ideal Applications
+
+- **Technical Documentation**: Generate accurate robotics and autonomous systems documentation
+- **Code Assistance**: Help with ROS, sensor drivers, control algorithms
+- **System Design**: Discuss architecture for autonomous systems
+- **Education**: Explain complex concepts in robotics and real-time systems
+- **Research Support**: Assist with literature review and concept exploration
+
+### Example Prompts
+
+```
+"What are the real-time constraints for autonomous navigation?"
+"Explain Kalman filtering for sensor fusion"
+"How do you handle dynamic obstacles in path planning?"
+"Design a sensor fusion pipeline for a mobile robot"
+"Implement a PID controller for robotic arm positioning"
+```
+
+## Limitations
+
+- **Domain Focus**: Optimized for robotics/autonomous systems; may be less creative in unrelated domains
+- **Recency**: Training data cutoff means recent developments may not be reflected
+- **Verification**: Always verify technical claims and code in production environments
+- **Scale**: 8B parameters provide strong performance but may not match larger models on highly complex reasoning
+
+## Ethical Considerations
+
+- **Autonomous Systems**: Use responsibly in safety-critical applications
+- **Verification**: Always validate outputs in production robotics systems
+- **Bias**: May reflect biases present in training data
+- **Transparency**: Clearly indicate AI-generated content in documentation
+
+## Citation
+
+```bibtex
+@misc{cc-zeta-0-2026,
+  title={CC-Zeta-0: Autonomous Systems Language Model},
+  author={Geass Labs},
+  year={2026},
+  publisher={HuggingFace},
+  howpublished={\url{https://huggingface.co/geasslabs/CC-Zeta-0}}
+}
+```
+
+## License
+
+Apache 2.0
+
+## Model Card Authors
+
+**Geass Labs** - Autonomous Systems AI Research
+
+---
+
+*For questions, issues, or collaboration inquiries, please open an issue on the model repository.*

checkpoint-6500/config.json

@@ -0,0 +1,31 @@
+{
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 770,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 1000000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.2.0",
+  "unsloth_version": "2024.9",
+  "use_cache": false,
+  "vocab_size": 32768
+}

checkpoint-6500/generation_config.json

@@ -0,0 +1,8 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "max_length": 32768,
+  "pad_token_id": 770,
+  "transformers_version": "5.2.0"
+}

checkpoint-6500/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:32b0f7dc5367600218cc055bde1a5484f642ad34d4a581606a789771ca85d2f2
+size 14496081136

checkpoint-6500/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:87507e649f52205f6fdf3a19c0d566a4745f5e109292e34bc47eafeb7eef908e
+size 15524017779

checkpoint-6500/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61c19bab1174704a4a4441475683bf1270277af15d2e2c95e964789128e482c4
+size 14645

checkpoint-6500/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:46a6b482fe70a90fb7f32366a238c92b054b6f18ce00b63ba8762f245e8cfdac
+size 1465

checkpoint-6500/tokenizer_config.json

@@ -0,0 +1,17 @@
+{
+  "add_prefix_space": true,
+  "backend": "tokenizers",
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "is_local": false,
+  "legacy": false,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "</s>",
+  "padding_side": "right",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

checkpoint-6500/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f2579f1f92493888ad963448c2ba4b2d2e3c9ccbdbf0bc5743da86c84a49f91
+size 5201

checkpoint-6670/config.json

@@ -0,0 +1,31 @@
+{
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 770,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 1000000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.2.0",
+  "unsloth_version": "2024.9",
+  "use_cache": false,
+  "vocab_size": 32768
+}

checkpoint-6670/generation_config.json

@@ -0,0 +1,8 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "max_length": 32768,
+  "pad_token_id": 770,
+  "transformers_version": "5.2.0"
+}

checkpoint-6670/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c783ca6294e81310e3871e2e03b78175ea4fc801fbc87503589c8604ef906dbd
+size 14496081136

checkpoint-6670/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e9404cc9fb1d142d2bdce6da1d557955781e7e5d4d3b02749943532b73da3096
+size 15524017779

checkpoint-6670/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61c19bab1174704a4a4441475683bf1270277af15d2e2c95e964789128e482c4
+size 14645

checkpoint-6670/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5138e533ba751b47b6334d8a30f897acd77cc8655701ba54955421a238ea8c89
+size 1465

checkpoint-6670/tokenizer_config.json

@@ -0,0 +1,17 @@
+{
+  "add_prefix_space": true,
+  "backend": "tokenizers",
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "is_local": false,
+  "legacy": false,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "</s>",
+  "padding_side": "right",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

checkpoint-6670/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f2579f1f92493888ad963448c2ba4b2d2e3c9ccbdbf0bc5743da86c84a49f91
+size 5201

config.json

@@ -0,0 +1,31 @@
+{
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 770,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 1000000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.2.0",
+  "unsloth_version": "2024.9",
+  "use_cache": false,
+  "vocab_size": 32768
+}

generation_config.json

@@ -0,0 +1,8 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "max_length": 32768,
+  "pad_token_id": 770,
+  "transformers_version": "5.2.0"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c783ca6294e81310e3871e2e03b78175ea4fc801fbc87503589c8604ef906dbd
+size 14496081136

tokenizer_config.json

@@ -0,0 +1,17 @@
+{
+  "add_prefix_space": true,
+  "backend": "tokenizers",
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "is_local": false,
+  "legacy": false,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "</s>",
+  "padding_side": "right",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f2579f1f92493888ad963448c2ba4b2d2e3c9ccbdbf0bc5743da86c84a49f91
+size 5201