Upload ViT-B event graph model

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Yuchn
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,85 @@
+---
+license: mit
+tags:
+  - video-understanding
+  - event-graph
+  - vjepa
+  - slot-attention
+  - pytorch
+library_name: pytorch
+pipeline_tag: video-classification
+---
+
+# Event Graph Generation — ViT-B
+
+動画からイベントグラフ（誰が・何を・どこから・どこへ）を構造化JSONとして予測するモデル。
+
+V-JEPA 2.1 ViT-B の時空間特徴トークンを入力とし、Object Pooling (Slot Attention) でオブジェクト表現を抽出した後、DETR風 Event Decoder でイベントを予測します。
+
+## Model Details
+
+| 項目 | 値 |
+|---|---|
+| パラメータ数 | 14.9M |
+| V-JEPA backbone | `vjepa2_1_vit_base_384` |
+| V-JEPA hidden_size | 768 |
+| Object Pooling slots | 24 |
+| Event queries | 20 |
+| d_model | 256 |
+| Action classes | 13 |
+| 学習エポック (best) | 20 |
+
+## Architecture
+
+```
+Video → V-JEPA 2.1 ViT-B → spatiotemporal tokens (B, S, 768)
+  → ObjectPoolingModule (Slot Attention, K=24 slots)
+    → ObjectRepresentation (identity, trajectory, existence, categories)
+      → VJEPAEventDecoder (M=20 event queries)
+        → EventPredictions (interaction, action, agent_ptr, target_ptr, source_ptr, dest_ptr, frame)
+```
+
+## Prediction Heads
+
+| Head | Shape | Description |
+|---|---|---|
+| interaction | (M, 1) | Is this a valid event? (BCE) |
+| action | (M, 13) | Action classification |
+| agent_ptr | (M, K) | Pointer to agent object slot |
+| target_ptr | (M, K) | Pointer to target object slot |
+| source_ptr | (M, K+1) | Pointer to source location (last = "none") |
+| dest_ptr | (M, K+1) | Pointer to destination (last = "none") |
+| frame | (M, T) | Event frame in temporal window |
+
+## Usage
+
+```python
+import torch
+
+# Load model weights
+state_dict = torch.load("model.pt", map_location="cpu")
+
+# Build model using the project's factory
+from event_graph_generation.config import Config
+from event_graph_generation.models.base import build_model
+
+config = Config.from_yaml("config.yaml")
+model = build_model(config.model, vjepa_config=config.vjepa)
+model.load_state_dict(state_dict)
+model.eval()
+```
+
+## Training
+
+- **Optimizer**: adamw
+- **Learning rate**: 0.0001
+- **Scheduler**: cosine_warmup
+- **Early stopping patience**: 15
+
+## License
+
+MIT License
+
+## Links
+
+- **Repository**: [ChanYu1224/event-graph-generation](https://github.com/ChanYu1224/event-graph-generation)

config.yaml

@@ -0,0 +1,72 @@
+data:
+  batch_size: 16
+  num_workers: 4
+  pin_memory: true
+  processed_dir: data/vjepa_aligned_v21_vitb
+  splits_dir: data/vjepa_aligned_v21_vitb/splits
+evaluation:
+  eval_every_n_epochs: 1
+  metrics:
+  - event_detection_map
+  - action_accuracy
+  - pointer_accuracy
+  - frame_mae
+  - graph_f1
+model:
+  d_model: 256
+  dropout: 0.1
+  name: vjepa_pipeline
+  nhead: 8
+  num_actions: 13
+  num_context_encoder_layers: 3
+  num_event_decoder_layers: 4
+  num_event_queries: 20
+  object_pooling:
+    d_model: 256
+    dropout: 0.1
+    n_categories: 29
+    nhead: 8
+    num_iterations: 3
+    num_refinement_layers: 2
+    num_slots: 24
+training:
+  checkpoint_dir: checkpoints/vjepa_vitb
+  device: cuda
+  early_stopping_patience: 15
+  epochs: 100
+  grad_clip_norm: 1.0
+  learning_rate: 0.0001
+  loss_weights:
+    action: 1.0
+    agent_ptr: 1.0
+    dest_ptr: 0.5
+    frame: 0.5
+    interaction: 2.0
+    source_ptr: 0.5
+    target_ptr: 1.0
+  optimizer: adamw
+  save_every_n_epochs: 10
+  scheduler: cosine_warmup
+  scheduler_params:
+    T_max: 100
+    warmup_epochs: 5
+  seed: 42
+  weight_decay: 0.0001
+vjepa:
+  backend: hub
+  features_dir: data/vjepa_features_v21_vitb
+  hidden_size: 768
+  hub_model_name: vjepa2_1_vit_base_384
+  spatial_tokens: 576
+  temporal_tokens: 8
+  total_tokens: 4608
+wandb:
+  enabled: true
+  entity: null
+  notes: V-JEPA 2.1 ViT-B + Slot Attention + Event Decoder
+  project: event_graph_generation
+  tags:
+  - vjepa_pipeline
+  - slot_attention
+  - vjepa_2.1
+  - vit_b

model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c3492cc812f944f14a63164e759008ab1190242069231bd6df58b056dca41b4a
+size 59823663