nielsr
/

tapas-base

Feature Extraction

sequence-classification

Model card Files Files and versions

nielsr HF Staff commited on Dec 11, 2020

Commit

1e052ba

·

1 Parent(s): 1c397d8

Fix typo

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -55,7 +55,7 @@ then of the form:
 [CLS] Sentence [SEP] Flattened table [SEP]
 ```
-### Fine-tuning
 The model was pre-trained on 32 Cloud TPU v3 cores for 1,000,000 steps with maximum sequence length 512 and batch size of 512.
 In this setup, pre-training on MLM only takes around 3 days. Aditionally, the model has been further pre-trained on a second task (table entailment). See the original TAPAS [paper](https://www.aclweb.org/anthology/2020.acl-main.398/) and the [follow-up paper](https://www.aclweb.org/anthology/2020.findings-emnlp.27/) for more details.

 [CLS] Sentence [SEP] Flattened table [SEP]
 ```
+### Pre-training
 The model was pre-trained on 32 Cloud TPU v3 cores for 1,000,000 steps with maximum sequence length 512 and batch size of 512.
 In this setup, pre-training on MLM only takes around 3 days. Aditionally, the model has been further pre-trained on a second task (table entailment). See the original TAPAS [paper](https://www.aclweb.org/anthology/2020.acl-main.398/) and the [follow-up paper](https://www.aclweb.org/anthology/2020.findings-emnlp.27/) for more details.