Can this run on a Jetson AGX Orin 32GB
#6
by SRai22 - opened
Trying to understand if this model can be loaded for inference on a Jetson AGX Orin?
Yes, it can be. But takes about 15 seconds per image on average to infer
thank you @SRai22 for the follow up. We did some inference optimization for https://huggingface.co/nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16