Update README.md
Browse files
README.md
CHANGED
|
@@ -114,12 +114,12 @@ img {
|
|
| 114 |
This model is a streaming version of Sortformer diarizer. [Sortformer](https://arxiv.org/abs/2409.06656)[1] is a novel end-to-end neural model for speaker diarization, trained with unconventional objectives compared to existing end-to-end diarization models.
|
| 115 |
|
| 116 |
<div align="center">
|
| 117 |
-
<img src="sortformer_intro.png" width="750" />
|
| 118 |
</div>
|
| 119 |
|
| 120 |
Streaming Sortformer approach employs an Arrival-Order Speaker Cache (AOSC) to store frame-level acoustic embeddings of previously observed speakers.
|
| 121 |
<div align="center">
|
| 122 |
-
<img src="streaming_sortformer_ani.gif" width="1400" />
|
| 123 |
</div>
|
| 124 |
|
| 125 |
Sortformer resolves permutation problem in diarization following the arrival-time order of the speech segments from each speaker.
|
|
@@ -129,7 +129,7 @@ Sortformer resolves permutation problem in diarization following the arrival-tim
|
|
| 129 |
Streaming sortformer employs pre-encode layer in the Fast-Conformer to generate speaker-cache. At each step, speaker cache is filtered to only retain the high-quality speaker cache vectors.
|
| 130 |
|
| 131 |
<div align="center">
|
| 132 |
-
<img src="streaming_steps.png" width="1400" />
|
| 133 |
</div>
|
| 134 |
|
| 135 |
|
|
@@ -138,7 +138,7 @@ Speech Tasks (NEST)](https://arxiv.org/abs/2408.13106)[2] which is based on [Fas
|
|
| 138 |
and two feedforward layers with 4 sigmoid outputs for each frame input at the top layer. More information can be found in the [Sortformer paper](https://arxiv.org/abs/2409.06656)[1].
|
| 139 |
|
| 140 |
<div align="center">
|
| 141 |
-
<img src="sortformer-v1-model.png" width="450" />
|
| 142 |
</div>
|
| 143 |
|
| 144 |
|
|
|
|
| 114 |
This model is a streaming version of Sortformer diarizer. [Sortformer](https://arxiv.org/abs/2409.06656)[1] is a novel end-to-end neural model for speaker diarization, trained with unconventional objectives compared to existing end-to-end diarization models.
|
| 115 |
|
| 116 |
<div align="center">
|
| 117 |
+
<img src="figures/sortformer_intro.png" width="750" />
|
| 118 |
</div>
|
| 119 |
|
| 120 |
Streaming Sortformer approach employs an Arrival-Order Speaker Cache (AOSC) to store frame-level acoustic embeddings of previously observed speakers.
|
| 121 |
<div align="center">
|
| 122 |
+
<img src="figures/streaming_sortformer_ani.gif" width="1400" />
|
| 123 |
</div>
|
| 124 |
|
| 125 |
Sortformer resolves permutation problem in diarization following the arrival-time order of the speech segments from each speaker.
|
|
|
|
| 129 |
Streaming sortformer employs pre-encode layer in the Fast-Conformer to generate speaker-cache. At each step, speaker cache is filtered to only retain the high-quality speaker cache vectors.
|
| 130 |
|
| 131 |
<div align="center">
|
| 132 |
+
<img src="figures/streaming_steps.png" width="1400" />
|
| 133 |
</div>
|
| 134 |
|
| 135 |
|
|
|
|
| 138 |
and two feedforward layers with 4 sigmoid outputs for each frame input at the top layer. More information can be found in the [Sortformer paper](https://arxiv.org/abs/2409.06656)[1].
|
| 139 |
|
| 140 |
<div align="center">
|
| 141 |
+
<img src="figures/sortformer-v1-model.png" width="450" />
|
| 142 |
</div>
|
| 143 |
|
| 144 |
|