Simple Self-Distillation
AI & ML interests
None defined yet.
Recent Activity
Papers
Stochastic KV Routing: Enabling Adaptive Depth-Wise Cache Sharing
Embarrassingly Simple Self-Distillation Improves Code Generation
Team members 792 private
Efficient Vision Encoding for Vision Language Models
Benchmark for the design of efficient continual learning of image-text models over years.
AIM: Autoregressive Image Models
CLaRa models
MobileCLIP2: Mobile-friendly image-text models with SOTA zero-shot capabilities trained on DFNDR-2B
A collection of AIMv2 vision encoders that supports a number of resolutions, native resolution, and a distilled checkpoint.
-
apple/aimv2-large-patch14-224
Image Feature Extraction • 0.3B • Updated • 1.25k • 62 -
apple/aimv2-huge-patch14-224
Image Feature Extraction • 0.7B • Updated • 100 • 13 -
apple/aimv2-1B-patch14-224
Image Feature Extraction • 1B • Updated • 111 • 8 -
apple/aimv2-3B-patch14-224
Image Feature Extraction • 3B • Updated • 78 • 4
MobileCLIP: Mobile-friendly image-text models with SOTA zero-shot capabilities.
DataCompDR: Improved datasets for training image-text SOTA models.
-
MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training
Paper • 2311.17049 • Published • 7 -
apple/mobileclip_s0_timm
Image Classification • Updated • 274 • 12 -
apple/mobileclip_s1_timm
Image Classification • Updated • 144 • 3 -
apple/mobileclip_s2_timm
Image Classification • Updated • 92 • 6
Depth Pro: Sharp Monocular Metric Depth in Less Than a Second
CLIP Models trained using DFN-2B/DFN-5B datasets
DCLM Models + Datasets
Simple Self-Distillation
CLaRa models
Efficient Vision Encoding for Vision Language Models
MobileCLIP2: Mobile-friendly image-text models with SOTA zero-shot capabilities trained on DFNDR-2B
A collection of AIMv2 vision encoders that supports a number of resolutions, native resolution, and a distilled checkpoint.
-
apple/aimv2-large-patch14-224
Image Feature Extraction • 0.3B • Updated • 1.25k • 62 -
apple/aimv2-huge-patch14-224
Image Feature Extraction • 0.7B • Updated • 100 • 13 -
apple/aimv2-1B-patch14-224
Image Feature Extraction • 1B • Updated • 111 • 8 -
apple/aimv2-3B-patch14-224
Image Feature Extraction • 3B • Updated • 78 • 4
MobileCLIP: Mobile-friendly image-text models with SOTA zero-shot capabilities.
DataCompDR: Improved datasets for training image-text SOTA models.
-
MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training
Paper • 2311.17049 • Published • 7 -
apple/mobileclip_s0_timm
Image Classification • Updated • 274 • 12 -
apple/mobileclip_s1_timm
Image Classification • Updated • 144 • 3 -
apple/mobileclip_s2_timm
Image Classification • Updated • 92 • 6
Benchmark for the design of efficient continual learning of image-text models over years.
Depth Pro: Sharp Monocular Metric Depth in Less Than a Second
CLIP Models trained using DFN-2B/DFN-5B datasets
AIM: Autoregressive Image Models
DCLM Models + Datasets