dronefreak/mc3-18-ucf101

MC3-18 for UCF-101 Action Recognition

Fine-tuned MC3-18 (Mixed 3D Convolutions) network, trained as part of the video pipeline in human-action-classification. MC3-18 mixes 2D and 3D convolutions across its stages, giving a lighter parameter count than a full 3D ResNet while still modeling temporal structure -- this is the best-accuracy model in this project's UCF-101 lineup. For a smaller/faster baseline, see R3D-18.

---

Performance

Metric	Value
Accuracy (Top-1)	87.05%
Precision (macro)	87.34%
Recall (macro)	86.95%
F1 Score (macro)	86.18%
Parameters	11.5M
Best epoch	170 / 200

These are the values the training script's own validation loop saved into the checkpoint at its best epoch (no further improvement occurred in the remaining 30 epochs of the run).

---

Evaluation Protocol

Metrics above come from VideoTrainer.validate() in hac.video.training.train, run on UCF-101 split 1's official test list (3,783 videos), at the checkpoint's best-performing epoch. Each clip: 16 frames sampled uniformly across the full video, resized preserving aspect ratio to roughly 128x171, center-cropped to 112x112, normalized with Kinetics-400 statistics -- a single center clip per video, no test-time augmentation or multi-crop averaging.

---

UCF-101 Model Zoo

Models from this project trained on UCF-101 split 1 with the same hac.video.training.train pipeline, for direct comparability:

Model	Accuracy	Notes
MC3-18 (this model)	87.05%	Best accuracy in this project's UCF-101 lineup; mixed 2D/3D convolutions
R3D-18	83.43%	Fast, lightweight baseline

---

Usage

Install Dependencies

Not yet published on PyPI -- install from source:

git clone https://github.com/dronefreak/human-action-classification
cd human-action-classification
pip install -e .

Load the Model from Hugging Face

import json
import torch
from huggingface_hub import hf_hub_download
from hac.video.models.classifier import Video3DCNN

config_path = hf_hub_download(repo_id="dronefreak/mc3-18-ucf101", filename="config.json")
weights_path = hf_hub_download(
    repo_id="dronefreak/mc3-18-ucf101",
    filename="mc318-ufc101-split-1.pth",
)

with open(config_path) as f:
    config = json.load(f)

model = Video3DCNN(
    num_classes=config["num_classes"],
    model_name=config["model_type"],
    pretrained=False,
)

checkpoint = torch.load(weights_path, map_location="cpu", weights_only=False)
model.load_state_dict(checkpoint["model_state_dict"])
model.eval()

Run Inference on a Video

The repo's VideoPredictor wraps frame sampling, transforms, and the forward pass end-to-end:

from hac.video.inference.predictor import VideoPredictor

predictor = VideoPredictor(model_path=weights_path, device="cpu")
result = predictor.predict_video("path/to/video.mp4", top_k=5)

print(result["top_class"], result["top_confidence"])
for pred in result["predictions"]:
    print(f"  {pred['class']}: {pred['confidence']:.3f}")

---

Training Configuration

Setting	Value	Source
Dataset	UCF-101, official split 1 (9,537 train / 3,783 test videos)	UCF-101 split files
Architecture	MC3-18 (torchvision.models.video.mc3_18)	checkpoint config
Pretrained init	Kinetics-400	checkpoint config
Optimizer	SGD (momentum=0.9, nesterov=False)	checkpoint optimizer state
Initial learning rate	0.001	checkpoint optimizer state
Weight decay	0.0005	checkpoint optimizer state
LR schedule	StepLR (step_size=20, gamma=0.1)	checkpoint scheduler state
Epochs trained	200 (best at epoch 170)	checkpoint + training history
Frames per clip	16	training script default
Spatial resolution	112x112 (aspect-preserving resize + random crop)	training script default
Batch size	not recorded in checkpoint	--
Augmentation	ColorJitter, RandomHorizontalFlip, RandomGrayscale(p=0.1), plus video-level MixUp/CutMix/FrameDrop/TemporalJitter	training script default

Rows marked "checkpoint ..." are read directly out of the optimizer/scheduler state and config dict stored inside mc318-ufc101-split-1.pth. Rows marked "training script default" reflect hac.video.training.train's CLI defaults at the time of training but weren't independently confirmed for this exact run -- no separate run-config file was saved alongside the checkpoint.

---

Use Cases

✅ Best for:

• Highest-accuracy option in this project's UCF-101 model zoo
• Action classification in short, trimmed videos similar in distribution to UCF-101
• Clip-level human activity tagging

⚠️ Consider alternatives for:

• Fastest inference / smallest footprint -- use R3D-18 instead (fewer FLOPs per clip, similar accuracy)
• Long-horizon temporal reasoning or untrimmed video detection -- not supported without adaptation

---

Known Limitations

• Fixed to UCF-101's 101 action classes -- no open-vocabulary or unseen-action support.
• Whole-clip classification only: no temporal localization, so a video containing multiple actions gets a single label.
• 16-frame uniform sampling can miss brief or rapid actions embedded in a longer clip.
• Trained only on UCF-101's largely trimmed, single-action YouTube clips; performance on untrimmed, multi-person, or surveillance-style footage is unverified.

---

Repository Contents

mc318-ufc101-split-1.pth
config.json
demo.gif
README.md

config.json doubles as the Hub's download-count query file: since this repo has no library_name integration the Hub recognizes, it falls back to counting requests against config.json (per Hugging Face's download-stats docs) -- the loading snippet above fetches it as part of normal usage, so downloads register.

---

Related Resources

• R3D-18 for UCF-101 -- faster, lighter sibling model in this project's UCF-101 model zoo
• human-action-classification -- the training/inference framework used to produce this checkpoint

---

Citation

If you use this model, please consider citing the UCF-101 dataset, the MC3 architecture, and the training framework:

@article{soomro2012ucf101,
  title={UCF101: A Dataset of 101 Human Actions Classes From Videos in the Wild},
  author={Soomro, Khurram and Zamir, Amir Roshan and Shah, Mubarak},
  journal={arXiv preprint arXiv:1212.0402},
  year={2012}
}

@inproceedings{tran2018closer,
  title={A Closer Look at Spatiotemporal Convolutions for Action Recognition},
  author={Tran, Du and Wang, Heng and Torresani, Lorenzo and Ray, Jamie and LeCun, Yann and Paluri, Manohar},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2018}
}

@misc{saksena2025mc3hac,
  author = {Saumya Saksena},
  title = {{MC3-18 for UCF-101 Action Recognition}},
  year = {2025},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/dronefreak/mc3-18-ucf101}},
  note = {Trained with the human-action-classification framework, Top-1 Accuracy: 87.05\%}
}

@software{saksena2026hac,
  author       = {Saumya Saksena},
  title        = {{Human Action Classification: Pose-based and Video-based Models}},
  year         = 2026,
  publisher    = {GitHub},
  journal      = {GitHub repository},
  howpublished = {\url{https://github.com/dronefreak/human-action-classification}}
}

---

License

Apache-2.0