Mask padding tokens during text pooling
#7
by fepegar - opened
Pass the tokenizer attention mask into text pooling so padding tokens cannot influence the pooled representation. Use the mask both when constructing the mean attention query and as the multi-head attention key-padding mask. Add regression tests covering padding invariance and invalid masks.
fepegar changed pull request status to open
fepegar changed pull request status to merged