bovod-sjtu nielsr HF Staff commited on
Commit
55707ba
·
1 Parent(s): ed9ee75

Improve model card: add paper/code links, license and sample usage (#1)

Browse files

- Improve model card: add paper/code links, license and sample usage (e185044424134682546c6ea82f462242035fd129)


Co-authored-by: Niels Rogge <nielsr@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +45 -4
README.md CHANGED
@@ -1,11 +1,52 @@
1
  ---
2
- license: other
3
  library_name: pytorch
 
4
  pipeline_tag: audio-to-audio
5
  ---
6
 
7
  # HoliTok
8
 
9
- This repository hosts runtime-only HoliTok checkpoints for inference. The
10
- checkpoints are loaded by the HoliTok inference package through the public
11
- aliases `HoliTok-Base` and `HoliTok-Unite`.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
 
2
  library_name: pytorch
3
+ license: apache-2.0
4
  pipeline_tag: audio-to-audio
5
  ---
6
 
7
  # HoliTok
8
 
9
+ HoliTok is a continuous holistic speech tokenization model designed for unified generation-understanding modeling. It encodes 48 kHz speech into a compact 25 Hz sequence of 128-dimensional latents. It is designed to preserve signal-level fidelity, incorporate semantic information, and maintain strong latent learnability.
10
+
11
+ - **Paper:** [HoliTok: A Continuous Holistic Tokenization with Robust Dual Capabilities of Speech Generation and Understanding](https://huggingface.co/papers/2605.29948)
12
+ - **Code:** [GitHub Repository](https://github.com/bovod-sjtu/HoliTok)
13
+
14
+ This repository hosts runtime-only HoliTok checkpoints for inference. The checkpoints are loaded by the HoliTok inference package through the public aliases `HoliTok-Base` and `HoliTok-Unite`.
15
+
16
+ ## Sample Usage
17
+
18
+ You can use the HoliTok Python API for audio tokenization, reconstruction, and semantic feature extraction.
19
+
20
+ ```python
21
+ import torch
22
+ from holitok import HoliTok, SemanticModule
23
+
24
+ model = HoliTok.from_pretrained("HoliTok-Unite", device="cuda:0")
25
+ audio = torch.randn(1, 1, 48000, device="cuda:0")
26
+
27
+ # [B, 2 * latent_dim, T], concat(mu, log_std)
28
+ posterior = model.encode_posterior(audio)
29
+
30
+ # [B, latent_dim, T]
31
+ latents = model.posterior_mean(posterior)
32
+
33
+ # [B, 1, samples]
34
+ recon = model.decode(latents)
35
+
36
+ semantic = SemanticModule.from_pretrained("HoliTok-Unite", device="cuda:0")
37
+ features = semantic(latents.transpose(1, 2)) # [B, T, 1536]
38
+ ```
39
+
40
+ ## Citation
41
+
42
+ ```bibtex
43
+ @misc{li2026holitokacoutinuousholistictokenization,
44
+ title={HoliTok:A Coutinuous Holistic Tokenization with Robust Dual Capabilities of Speech Generation and Understanding},
45
+ author={Bohan Li and Shi Lian and Hankun Wang and Yiwei Guo and Yu Xi and Zhihan Li and Da Zheng and Colin Zhang and Kai Yu},
46
+ year={2026},
47
+ eprint={2605.29948},
48
+ archivePrefix={arXiv},
49
+ primaryClass={cs.SD},
50
+ url={https://arxiv.org/abs/2605.29948},
51
+ }
52
+ ```