Hugging Face's logo

chungimungi
/
GLInt

Sentence Similarity
sentence-transformers
Safetensors
English
modernbert
colbert
late-interaction
retrieval
pylate
multi-vector
text-embeddings-inference
Model card Files
xet
 Community
1
chungimungi commited on
Commit
4e1b7b3
·
verified ·
1 Parent(s): ba194e9

Upload GLINT-base checkpoint

Browse files
Files changed (14) hide show
  1. 1_Dense/config.json +9 -0
  2. 1_Dense/model.safetensors +3 -0
  3. 2_Dense/config.json +9 -0
  4. 2_Dense/model.safetensors +3 -0
  5. 3_Dense/config.json +9 -0
  6. 3_Dense/model.safetensors +3 -0
  7. README.md +586 -0
  8. config.json +79 -0
  9. config_sentence_transformers.json +54 -0
  10. model.safetensors +3 -0
  11. modules.json +26 -0
  12. sentence_bert_config.json +10 -0
  13. tokenizer.json +0 -0
  14. tokenizer_config.json +23 -0
1_Dense/config.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "in_features": 768,
3
+ "out_features": 1536,
4
+ "bias": false,
5
+ "activation_function": "torch.nn.modules.linear.Identity",
6
+ "module_input_name": "token_embeddings",
7
+ "module_output_name": "token_embeddings",
8
+ "use_residual": true
9
+ }
1_Dense/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c1668b1fce802e052e6b3f4db9a9f53cf3247715e6ac8e536ab05badf870a146
3
+ size 9437360
2_Dense/config.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "in_features": 1536,
3
+ "out_features": 768,
4
+ "bias": false,
5
+ "activation_function": "torch.nn.modules.linear.Identity",
6
+ "module_input_name": "token_embeddings",
7
+ "module_output_name": "token_embeddings",
8
+ "use_residual": true
9
+ }
2_Dense/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:118192b6ecefbaf5d52e703e2d60d007d19ca99428c98feabd253e478403a4cd
3
+ size 9437360
3_Dense/config.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "in_features": 768,
3
+ "out_features": 128,
4
+ "bias": false,
5
+ "activation_function": "torch.nn.modules.linear.Identity",
6
+ "module_input_name": "token_embeddings",
7
+ "module_output_name": "token_embeddings",
8
+ "use_residual": false
9
+ }
3_Dense/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:919df98251b0f54aa543d4b72d6ec0f6622ded98d423140de240dcb12f99a322
3
+ size 393304
README.md ADDED
@@ -0,0 +1,586 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - ColBERT
4
+ - PyLate
5
+ - sentence-transformers
6
+ - sentence-similarity
7
+ - feature-extraction
8
+ - generated_from_trainer
9
+ - dataset_size:1046009
10
+ - loss:MixedDistillation
11
+ pipeline_tag: sentence-similarity
12
+ library_name: PyLate
13
+ ---
14
+
15
+ # PyLate
16
+
17
+ This is a [PyLate](https://github.com/lightonai/pylate) model trained on the parquet dataset. It maps sentences & paragraphs to sequences of 128-dimensional dense vectors and can be used for semantic textual similarity using the MaxSim operator.
18
+
19
+ ## Model Details
20
+
21
+ ### Model Description
22
+ - **Model Type:** PyLate model
23
+ <!-- - **Base model:** [Unknown](https://huggingface.co/unknown) -->
24
+ - **Document Length:** 300 tokens
25
+ - **Query Length:** 32 tokens
26
+ - **Output Dimensionality:** 128 tokens
27
+ - **Similarity Function:** MaxSim
28
+ - **Training Dataset:**
29
+ - parquet
30
+ <!-- - **Language:** Unknown -->
31
+ <!-- - **License:** Unknown -->
32
+
33
+ ### Model Sources
34
+
35
+ - **Documentation:** [PyLate Documentation](https://lightonai.github.io/pylate/)
36
+ - **Repository:** [PyLate on GitHub](https://github.com/lightonai/pylate)
37
+ - **Hugging Face:** [PyLate models on Hugging Face](https://huggingface.co/models?library=PyLate)
38
+
39
+ ### Full Model Architecture
40
+
41
+ ```
42
+ ColBERT(
43
+ (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'ModernBertModel'})
44
+ (1): Dense({'in_features': 768, 'out_features': 1536, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'module_input_name': 'token_embeddings', 'module_output_name': 'token_embeddings', 'use_residual': True})
45
+ (2): Dense({'in_features': 1536, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'module_input_name': 'token_embeddings', 'module_output_name': 'token_embeddings', 'use_residual': True})
46
+ (3): Dense({'in_features': 768, 'out_features': 128, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'module_input_name': 'token_embeddings', 'module_output_name': 'token_embeddings', 'use_residual': False})
47
+ )
48
+ ```
49
+
50
+ ## Usage
51
+ First install the PyLate library:
52
+
53
+ ```bash
54
+ pip install -U pylate
55
+ ```
56
+
57
+ ### Retrieval
58
+
59
+ Use this model with PyLate to index and retrieve documents. The index uses [FastPLAID](https://github.com/lightonai/fast-plaid) for efficient similarity search.
60
+
61
+ #### Indexing documents
62
+
63
+ Load the ColBERT model and initialize the PLAID index, then encode and index your documents:
64
+
65
+ ```python
66
+ from pylate import indexes, models, retrieve
67
+
68
+ # Step 1: Load the ColBERT model
69
+ model = models.ColBERT(
70
+ model_name_or_path="pylate_model_id",
71
+ )
72
+
73
+ # Step 2: Initialize the PLAID index
74
+ index = indexes.PLAID(
75
+ index_folder="pylate-index",
76
+ index_name="index",
77
+ override=True, # This overwrites the existing index if any
78
+ )
79
+
80
+ # Step 3: Encode the documents
81
+ documents_ids = ["1", "2", "3"]
82
+ documents = ["document 1 text", "document 2 text", "document 3 text"]
83
+
84
+ documents_embeddings = model.encode(
85
+ documents,
86
+ batch_size=32,
87
+ is_query=False, # Ensure that it is set to False to indicate that these are documents, not queries
88
+ show_progress_bar=True,
89
+ )
90
+
91
+ # Step 4: Add document embeddings to the index by providing embeddings and corresponding ids
92
+ index.add_documents(
93
+ documents_ids=documents_ids,
94
+ documents_embeddings=documents_embeddings,
95
+ )
96
+ ```
97
+
98
+ Note that you do not have to recreate the index and encode the documents every time. Once you have created an index and added the documents, you can re-use the index later by loading it:
99
+
100
+ ```python
101
+ # To load an index, simply instantiate it with the correct folder/name and without overriding it
102
+ index = indexes.PLAID(
103
+ index_folder="pylate-index",
104
+ index_name="index",
105
+ )
106
+ ```
107
+
108
+ #### Retrieving top-k documents for queries
109
+
110
+ Once the documents are indexed, you can retrieve the top-k most relevant documents for a given set of queries.
111
+ To do so, initialize the ColBERT retriever with the index you want to search in, encode the queries and then retrieve the top-k documents to get the top matches ids and relevance scores:
112
+
113
+ ```python
114
+ # Step 1: Initialize the ColBERT retriever
115
+ retriever = retrieve.ColBERT(index=index)
116
+
117
+ # Step 2: Encode the queries
118
+ queries_embeddings = model.encode(
119
+ ["query for document 3", "query for document 1"],
120
+ batch_size=32,
121
+ is_query=True, # # Ensure that it is set to False to indicate that these are queries
122
+ show_progress_bar=True,
123
+ )
124
+
125
+ # Step 3: Retrieve top-k documents
126
+ scores = retriever.retrieve(
127
+ queries_embeddings=queries_embeddings,
128
+ k=10, # Retrieve the top 10 matches for each query
129
+ )
130
+ ```
131
+
132
+ ### Reranking
133
+ If you only want to use the ColBERT model to perform reranking on top of your first-stage retrieval pipeline without building an index, you can simply use rank function and pass the queries and documents to rerank:
134
+
135
+ ```python
136
+ from pylate import rank, models
137
+
138
+ queries = [
139
+ "query A",
140
+ "query B",
141
+ ]
142
+
143
+ documents = [
144
+ ["document A", "document B"],
145
+ ["document 1", "document C", "document B"],
146
+ ]
147
+
148
+ documents_ids = [
149
+ [1, 2],
150
+ [1, 3, 2],
151
+ ]
152
+
153
+ model = models.ColBERT(
154
+ model_name_or_path="pylate_model_id",
155
+ )
156
+
157
+ queries_embeddings = model.encode(
158
+ queries,
159
+ is_query=True,
160
+ )
161
+
162
+ documents_embeddings = model.encode(
163
+ documents,
164
+ is_query=False,
165
+ )
166
+
167
+ reranked_documents = rank.rerank(
168
+ documents_ids=documents_ids,
169
+ queries_embeddings=queries_embeddings,
170
+ documents_embeddings=documents_embeddings,
171
+ )
172
+ ```
173
+
174
+ <!--
175
+ ### Direct Usage (Transformers)
176
+
177
+ <details><summary>Click to see the direct usage in Transformers</summary>
178
+
179
+ </details>
180
+ -->
181
+
182
+ <!--
183
+ ### Downstream Usage (Sentence Transformers)
184
+
185
+ You can finetune this model on your own dataset.
186
+
187
+ <details><summary>Click to expand</summary>
188
+
189
+ </details>
190
+ -->
191
+
192
+ <!--
193
+ ### Out-of-Scope Use
194
+
195
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
196
+ -->
197
+
198
+ <!--
199
+ ## Bias, Risks and Limitations
200
+
201
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
202
+ -->
203
+
204
+ <!--
205
+ ### Recommendations
206
+
207
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
208
+ -->
209
+
210
+ ## Training Details
211
+
212
+ ### Training Dataset
213
+
214
+ #### parquet
215
+
216
+ * Dataset: parquet
217
+ * Size: 1,046,009 training samples
218
+ * Columns: <code>query_id</code>, <code>document_ids</code>, <code>scores</code>, <code>query</code>, and <code>documents</code>
219
+ * Approximate statistics based on the first 1000 samples:
220
+ | | query_id | document_ids | scores | query | documents |
221
+ |:---------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------|:------------------------------------|:----------------------------------------------------------------------------------|:------------------------------------|
222
+ | type | int | list | list | string | list |
223
+ | modality | | | | text | |
224
+ | details | <ul><li>0: ~0.96%</li><li>1: ~0.96%</li><li>2: ~0.96%</li><li>3: ~0.96%</li><li>4: ~0.96%</li><li>5: ~0.96%</li><li>6: ~0.96%</li><li>7: ~0.96%</li><li>8: ~0.96%</li><li>9: ~0.96%</li><li>10: ~0.96%</li><li>11: ~0.96%</li><li>12: ~0.96%</li><li>13: ~0.96%</li><li>14: ~0.96%</li><li>15: ~0.96%</li><li>16: ~0.96%</li><li>17: ~0.96%</li><li>18: ~0.96%</li><li>19: ~0.96%</li><li>20: ~0.96%</li><li>21: ~0.96%</li><li>22: ~0.96%</li><li>23: ~0.96%</li><li>24: ~0.96%</li><li>25: ~0.96%</li><li>26: ~0.96%</li><li>27: ~0.96%</li><li>28: ~0.96%</li><li>29: ~0.96%</li><li>30: ~0.96%</li><li>31: ~0.96%</li><li>32: ~0.96%</li><li>33: ~0.96%</li><li>34: ~0.96%</li><li>35: ~0.96%</li><li>36: ~0.96%</li><li>37: ~0.96%</li><li>38: ~0.96%</li><li>39: ~0.96%</li><li>40: ~0.96%</li><li>41: ~0.96%</li><li>42: ~0.96%</li><li>43: ~0.96%</li><li>44: ~0.96%</li><li>45: ~0.96%</li><li>46: ~0.96%</li><li>47: ~0.96%</li><li>48: ~0.96%</li><li>49: ~0.96%</li><li>50: ~0.96%</li><li>51: ~0.96%</li><li>52: ~0.96%</li><li>53: ~0.96%</li><li>54: ~0.96%</li><li>55: ~0.96%</li><li>56: ~0.96%</li><li>57: ~0.96%</li><li>58: ~0.96%</li><li>59: ~0.96%</li><li>60: ~0.96%</li><li>61: ~0.96%</li><li>62: ~0.96%</li><li>63: ~0.96%</li><li>64: ~0.96%</li><li>65: ~0.96%</li><li>66: ~0.96%</li><li>67: ~0.96%</li><li>68: ~0.96%</li><li>69: ~0.96%</li><li>70: ~0.96%</li><li>71: ~0.96%</li><li>72: ~0.96%</li><li>73: ~0.96%</li><li>74: ~0.96%</li><li>75: ~0.96%</li><li>76: ~0.96%</li><li>77: ~0.96%</li><li>78: ~0.96%</li><li>79: ~0.96%</li><li>80: ~0.96%</li><li>81: ~0.96%</li><li>82: ~0.96%</li><li>83: ~0.96%</li><li>84: ~0.96%</li><li>85: ~0.96%</li><li>86: ~0.96%</li><li>87: ~0.96%</li><li>88: ~0.96%</li><li>89: ~0.96%</li><li>90: ~0.96%</li><li>91: ~0.96%</li><li>92: ~0.96%</li><li>93: ~0.96%</li><li>94: ~0.96%</li><li>95: ~0.96%</li><li>96: ~0.96%</li><li>97: ~0.96%</li><li>98: ~0.96%</li><li>99: ~0.96%</li><li>100: ~0.96%</li><li>101: ~0.96%</li><li>102: ~0.96%</li><li>103: ~0.96%</li></ul> | <ul><li>size: 32 elements</li></ul> | <ul><li>size: 32 elements</li></ul> | <ul><li>min: 8 tokens</li><li>mean: 15.26 tokens</li><li>max: 25 tokens</li></ul> | <ul><li>size: 32 elements</li></ul> |
225
+ * Samples:
226
+ | query_id | document_ids | scores | query | documents |
227
+ |:---------------|:-----------------------------------------------------|:----------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
228
+ | <code>0</code> | <code>[1800, 51106, 20778, 50872, 18727, ...]</code> | <code>[0.46484375, 0.18359375, 0.1708984375, -0.038330078125, -0.10205078125, ...]</code> | <code>What is considered a business expense on a business trip?</code> | <code>['The IRS Guidance pertaining to the subject. In general the best I can say is your business expense may be deductible. But it depends on the circumstances and what it is you want to deduct. Travel Taxpayers who travel away from home on business may deduct related expenses, including the cost of reaching their destination, the cost of lodging and meals and other ordinary and necessary expenses. Taxpayers are considered “traveling away from home” if their duties require them to be away from home substantially longer than an ordinary day’s work and they need to sleep or rest to meet the demands of their work. The actual cost of meals and incidental expenses may be deducted or the taxpayer may use a standard meal allowance and reduced record keeping requirements. Regardless of the method used, meal deductions are generally limited to 50 percent as stated earlier. Only actual costs for lodging may be claimed as an expense and receipts must be kept for documentation. Expenses must be reasonable and appropriate; deductions for extravagant expenses are not allowable. More information is available in Publication 463, Travel, Entertainment, Gift, and Car Expenses. Entertainment Expenses for entertaining clients, customers or employees may be deducted if they are both ordinary and necessary and meet one of the following tests: Directly-related test: The main purpose of the entertainment activity is the conduct of business, business was actually conducted during the activity and the taxpayer had more than a general expectation of getting income or some other specific business benefit at some future time. Associated test: The entertainment was associated with the active conduct of the taxpayer’s trade or business and occurred directly before or after a substantial business discussion. Publication 463 provides more extensive explanation of these tests as well as other limitations and requirements for deducting entertainment expenses. Gifts Taxpayers may deduct some or all of the cost of gifts given in the course of their trade or business. In general, the deduction is limited to $25 for gifts given directly or indirectly to any one person during the tax year. More discussion of the rules and limitations can be found in Publication 463. If your LLC reimburses you for expenses outside of this guidance it should be treated as Income for tax purposes. Edit for Meal Expenses: Amount of standard meal allowance. The standard meal allowance is the federal M&IE rate. For travel in 2010, the rate for most small localities in the United States is $46 a day. Source IRS P463 Alternately you could reimburse at a per diem rate', '"It depends on what the ""true"" reason for the trip is. If you decide to deduct the trip as a business expense, then during an audit you will be asked why you had to go there. If there was nothing accomplished via the travel (that is, you worked from the hotel, met with no clients, visited no tradeshows, etc) then the expense is unlikely to be allowed. Yes, on a business trip you can do sightseeing if you wish (though you can\'t deduct any sightseeing specific expenses, like admission to a tourist attraction), but if you are just working while on vacation, then the trip itself is not deductible, since there was no business benefit to traveling in the first place."', "Food is almost never a valid expense. Reason for it is simple - if you were not conducting business you would have to eat too. Ad 1. I don't see why travel in that case would not be a valid expense, as the only reason for you to travel there is for business reasons. Ad 2. Unlikely as there is a duality of purpose. So while part of it may be business, you are also getting personal benefit from the visit (coffee/cakes etc) so that generally is a no. Ad 3. No, while you can claim for entertainment of employees (to sensible extends), that doesn't work when entertaining clients. Ad 4. If any part of the trip is for leisure then you cannot claim it as business expense, sorry! If there is any duality of use then it's not a business expense. And food, as always, is a no go.", "No, you cannot deduct it. There's no business substance in such a trip, it is your vacation, and as such cannot be claimed as an expense against the rental income. You may be able to deduct the coffee you buy for the meeting with the property manager while there, but there's no way you can justify a 7-10 days vacation with your whole family as an expense to maintain the rental property. Since you will only have less than 2 weeks personal use, you won't need to prorate expenses, so you have that at least.", "Well, if you were a business, and your food and rent and travel expenses were business expenses, and you paid out less money than you earned, you *would* get a refund. If you can prove that an expense is tax deductible, then that's just what it is. For businesses, a net operating loss is tax deductible.", ...]</code> |
229
+ | <code>1</code> | <code>[18927, 38239, 42978, 51106, 1800, ...]</code> | <code>[0.392578125, -0.0595703125, 0.03271484375, -0.0223388671875, 0.2021484375, ...]</code> | <code>Business Expense - Car Insurance Deductible For Accident That Occurred During a Business Trip</code> | <code>["As a general rule, you must choose between a mileage deduction or an actual expenses deduction. The idea is that the mileage deduction is supposed to cover all costs of using the car. Exceptions include parking fees and tolls, which can be deducted separately under either method. You explicitly cannot deduct insurance costs if you claim a mileage deduction. Separately, you probably won't be able to deduct the deductible for your car as a casualty loss. You first subtract $100 from the deductible and then divide it by your Adjusted Gross Income (AGI) from your tax return. If your deductible is over 10% of your AGI, you can deduct it. Note that even with a $1500 deductible, you won't be able to deduct anything if you made more than $14,000 for the year. For most people, the insurance deductible just isn't large enough relative to income to be tax deductible. Source", 'I contacted Stephen Fishman, J.D., the author of Home Business Tax Deductions, to let him know that this question was missing from his book. He was kind enough to send a reply. My original phrasing of the question: If your car is used for both business and personal use, and you deduct via the actual expense method, do trips to the mechanic, gas station, and auto parts store to service or repair the car count as business miles, personal miles, or part-business-part-personal miles? What about driving the newly-purchased car home from the dealership? And his response: Good question. I can find nothing about this in IRS publication or elsewhere. However, common sense would tell us that the cost of driving to make car repairs should be deductible. If you use your car for business, it is a business expense, just like transporting any other piece of business equipment for repairs is a business expense. This should be so whether you use the standard mileage rate or actual expense method. You should probably reduce the amount of your deduction by the percentage of personal use of the car during the year. The same goes for driving a car home from the dealer.', "You don't say what country you live in. If it's the U.S., the IRS has very specific rules for business use of a car. See, for starters at least, http://www.irs.gov/publications/p463/ch04.html. The gist of it is: If you use the car 100% for business purposes, you NEVER use it to drive to the grocery store or to your friend's house, etc, then it is a deductible business expense. If you use a car party for business use and partly for personal use, than you can deduct the portion of the expense of the car that is for business use, but not the portion that is for personal use. So basically, if you use the car 75% for business purposes and 25% for personal use, you can deduct 75% of the cost and expenses. You can calculate the business use by, (a) Keeping careful records of how much you spent on gas, oil, repairs, etc, tracking the percentage of business use versus percentage of personal use, and then multiplying the cost by the percentage business use and that is the amount you can deduct; or (b) Use the standard mileage allowance, so many cents per mile, which changes every year. Note that the fact that you paid for the car from a business account has absolutely nothing to do with it. (If it did, then everyone could create a small business, open a business account, pay all their bills from there, and all their personal expenses would magically become business expenses.) Just by the way: If you are going to try to stretch the rules on your taxes, business use of a car or personal computer or expenses for a home office are the worst place to do it. The IRS knows that cars and computers are things that can easily be used for either personal or business purposes and so they keep a special eye out on these.", '"It depends on what the ""true"" reason for the trip is. If you decide to deduct the trip as a business expense, then during an audit you will be asked why you had to go there. If there was nothing accomplished via the travel (that is, you worked from the hotel, met with no clients, visited no tradeshows, etc) then the expense is unlikely to be allowed. Yes, on a business trip you can do sightseeing if you wish (though you can\'t deduct any sightseeing specific expenses, like admission to a tourist attraction), but if you are just working while on vacation, then the trip itself is not deductible, since there was no business benefit to traveling in the first place."', 'The IRS Guidance pertaining to the subject. In general the best I can say is your business expense may be deductible. But it depends on the circumstances and what it is you want to deduct. Travel Taxpayers who travel away from home on business may deduct related expenses, including the cost of reaching their destination, the cost of lodging and meals and other ordinary and necessary expenses. Taxpayers are considered “traveling away from home” if their duties require them to be away from home substantially longer than an ordinary day’s work and they need to sleep or rest to meet the demands of their work. The actual cost of meals and incidental expenses may be deducted or the taxpayer may use a standard meal allowance and reduced record keeping requirements. Regardless of the method used, meal deductions are generally limited to 50 percent as stated earlier. Only actual costs for lodging may be claimed as an expense and receipts must be kept for documentation. Expenses must be reasonable and appropriate; deductions for extravagant expenses are not allowable. More information is available in Publication 463, Travel, Entertainment, Gift, and Car Expenses. Entertainment Expenses for entertaining clients, customers or employees may be deducted if they are both ordinary and necessary and meet one of the following tests: Directly-related test: The main purpose of the entertainment activity is the conduct of business, business was actually conducted during the activity and the taxpayer had more than a general expectation of getting income or some other specific business benefit at some future time. Associated test: The entertainment was associated with the active conduct of the taxpayer’s trade or business and occurred directly before or after a substantial business discussion. Publication 463 provides more extensive explanation of these tests as well as other limitations and requirements for deducting entertainment expenses. Gifts Taxpayers may deduct some or all of the cost of gifts given in the course of their trade or business. In general, the deduction is limited to $25 for gifts given directly or indirectly to any one person during the tax year. More discussion of the rules and limitations can be found in Publication 463. If your LLC reimburses you for expenses outside of this guidance it should be treated as Income for tax purposes. Edit for Meal Expenses: Amount of standard meal allowance. The standard meal allowance is the federal M&IE rate. For travel in 2010, the rate for most small localities in the United States is $46 a day. Source IRS P463 Alternately you could reimburse at a per diem rate', ...]</code> |
230
+ | <code>2</code> | <code>[6712, 50185, 46217, 33601, 51864, ...]</code> | <code>[0.162109375, 0.1513671875, 0.044189453125, -0.04052734375, -0.037109375, ...]</code> | <code>Starting a new online business</code> | <code>["Most US states have rules that go something like this: You will almost certainly have to pay some registration fees, as noted above. Depending on how you organize, you may or may not need to file a separate tax return for the business. (If you're sole proprietor for tax purposes, then you file on Schedule C on your personal Form 1040.) Whether or not you pay taxes depends on whether you have net income. It's possible that some losses might also be deductible. (Note that you may have to file a return even if you don't have net income - Filing and needing to pay are not the same since your return may indicate no tax due.) In addition, at the state level, you may have to pay additional fees or taxes beyond income tax depending on what you sell and how you sell it. (Sales tax, for example, might come into play as might franchise taxes.) You'll need to check your own state law for that. As always, it could be wise to get professional tax and accounting advice that's tailored to your situation and your state. This is just an outline of some things that you'll need to consider.", "Take your time. When I started my business I thought I had a killer name at first and a few weeks later I found a much better one. Brainstorm with your partners. For an internet business having a short domain name is going to matter, and you should really try to get a .com. A .us or .io are also good, but if the .com is already taken people will confuse them for you so it's good to get the .com first. Make sure you don't have any weird characters in your domain like hyphens which are harder to type on a mobile device.", 'Are you an established business owner or looking to start a new business? We, at Valis International possess expertise in providing online incorporation services. Through our online incorporation services, we let you form a company in just a few minutes. All you need is to just fill in the personal information and our service provider does the rest of the work. Through our services, we can get your business incorporated for a reasonable cost.', 'Become your own boss web business can be easily done with great fortune. The bonus are apparent to running your own kind of business particularly online, however, what you require to attain those bonus might not be so apparent. There’re various elements, however, the 3 most essential elements are mindset, proper training and education and posting yourself as a leader.', 'The fastest way to start is to go the opposite way of how startups usually work. Instead of inventing a market, find something that is getting commoditized online, and see if you can fit in that established market via either a price cut or a unique feature. If done properly, you could start earning in less than 10-20 days.', ...]</code> |
231
+ * Loss: <code>losses_mixed.MixedDistillation</code>
232
+
233
+ ### Training Hyperparameters
234
+ #### Non-Default Hyperparameters
235
+
236
+ - `num_train_epochs`: 1.0
237
+ - `learning_rate`: 3e-05
238
+ - `gradient_accumulation_steps`: 2
239
+ - `bf16`: True
240
+ - `ddp_find_unused_parameters`: False
241
+
242
+ #### All Hyperparameters
243
+ <details><summary>Click to expand</summary>
244
+
245
+ - `per_device_train_batch_size`: 8
246
+ - `num_train_epochs`: 1.0
247
+ - `max_steps`: -1
248
+ - `learning_rate`: 3e-05
249
+ - `lr_scheduler_type`: linear
250
+ - `lr_scheduler_kwargs`: None
251
+ - `warmup_steps`: 0
252
+ - `optim`: adamw_torch_fused
253
+ - `optim_args`: None
254
+ - `weight_decay`: 0.0
255
+ - `adam_beta1`: 0.9
256
+ - `adam_beta2`: 0.999
257
+ - `adam_epsilon`: 1e-08
258
+ - `optim_target_modules`: None
259
+ - `gradient_accumulation_steps`: 2
260
+ - `average_tokens_across_devices`: True
261
+ - `max_grad_norm`: 1.0
262
+ - `label_smoothing_factor`: 0.0
263
+ - `bf16`: True
264
+ - `fp16`: False
265
+ - `bf16_full_eval`: False
266
+ - `fp16_full_eval`: False
267
+ - `tf32`: None
268
+ - `gradient_checkpointing`: False
269
+ - `gradient_checkpointing_kwargs`: None
270
+ - `torch_compile`: False
271
+ - `torch_compile_backend`: None
272
+ - `torch_compile_mode`: None
273
+ - `use_liger_kernel`: False
274
+ - `liger_kernel_config`: None
275
+ - `use_cache`: False
276
+ - `neftune_noise_alpha`: None
277
+ - `torch_empty_cache_steps`: None
278
+ - `auto_find_batch_size`: False
279
+ - `log_on_each_node`: True
280
+ - `logging_nan_inf_filter`: True
281
+ - `include_num_input_tokens_seen`: no
282
+ - `log_level`: passive
283
+ - `log_level_replica`: warning
284
+ - `disable_tqdm`: False
285
+ - `project`: huggingface
286
+ - `trackio_space_id`: trackio
287
+ - `per_device_eval_batch_size`: 8
288
+ - `prediction_loss_only`: True
289
+ - `eval_on_start`: False
290
+ - `eval_do_concat_batches`: True
291
+ - `eval_use_gather_object`: False
292
+ - `eval_accumulation_steps`: None
293
+ - `include_for_metrics`: []
294
+ - `batch_eval_metrics`: False
295
+ - `save_only_model`: False
296
+ - `save_on_each_node`: False
297
+ - `enable_jit_checkpoint`: False
298
+ - `push_to_hub`: False
299
+ - `hub_private_repo`: None
300
+ - `hub_model_id`: None
301
+ - `hub_strategy`: every_save
302
+ - `hub_always_push`: False
303
+ - `hub_revision`: None
304
+ - `load_best_model_at_end`: False
305
+ - `ignore_data_skip`: False
306
+ - `restore_callback_states_from_checkpoint`: False
307
+ - `full_determinism`: False
308
+ - `seed`: 42
309
+ - `data_seed`: None
310
+ - `use_cpu`: False
311
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
312
+ - `parallelism_config`: None
313
+ - `dataloader_drop_last`: True
314
+ - `dataloader_num_workers`: 0
315
+ - `dataloader_pin_memory`: True
316
+ - `dataloader_persistent_workers`: False
317
+ - `dataloader_prefetch_factor`: None
318
+ - `remove_unused_columns`: True
319
+ - `label_names`: None
320
+ - `train_sampling_strategy`: random
321
+ - `length_column_name`: length
322
+ - `ddp_find_unused_parameters`: False
323
+ - `ddp_bucket_cap_mb`: None
324
+ - `ddp_broadcast_buffers`: False
325
+ - `ddp_backend`: None
326
+ - `ddp_timeout`: 1800
327
+ - `fsdp`: []
328
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
329
+ - `deepspeed`: None
330
+ - `debug`: []
331
+ - `skip_memory_metrics`: True
332
+ - `do_predict`: False
333
+ - `resume_from_checkpoint`: None
334
+ - `warmup_ratio`: None
335
+ - `local_rank`: -1
336
+ - `prompts`: None
337
+ - `batch_sampler`: batch_sampler
338
+ - `multi_dataset_batch_sampler`: proportional
339
+ - `router_mapping`: {}
340
+ - `learning_rate_mapping`: {}
341
+
342
+ </details>
343
+
344
+ ### Training Logs
345
+ <details><summary>Click to expand</summary>
346
+
347
+ | Epoch | Step | Training Loss |
348
+ |:------:|:----:|:-------------:|
349
+ | 0.0061 | 50 | 0.3159 |
350
+ | 0.0122 | 100 | 0.2645 |
351
+ | 0.0184 | 150 | 0.2579 |
352
+ | 0.0245 | 200 | 0.2527 |
353
+ | 0.0306 | 250 | 0.2427 |
354
+ | 0.0367 | 300 | 0.2338 |
355
+ | 0.0428 | 350 | 0.2380 |
356
+ | 0.0490 | 400 | 0.2306 |
357
+ | 0.0551 | 450 | 0.2303 |
358
+ | 0.0612 | 500 | 0.2258 |
359
+ | 0.0673 | 550 | 0.2250 |
360
+ | 0.0734 | 600 | 0.2265 |
361
+ | 0.0795 | 650 | 0.2138 |
362
+ | 0.0857 | 700 | 0.2165 |
363
+ | 0.0918 | 750 | 0.2135 |
364
+ | 0.0979 | 800 | 0.2092 |
365
+ | 0.1040 | 850 | 0.2124 |
366
+ | 0.1101 | 900 | 0.2112 |
367
+ | 0.1163 | 950 | 0.2049 |
368
+ | 0.1224 | 1000 | 0.2105 |
369
+ | 0.1285 | 1050 | 0.2057 |
370
+ | 0.1346 | 1100 | 0.2071 |
371
+ | 0.1407 | 1150 | 0.2043 |
372
+ | 0.1469 | 1200 | 0.2080 |
373
+ | 0.1530 | 1250 | 0.2035 |
374
+ | 0.1591 | 1300 | 0.2067 |
375
+ | 0.1652 | 1350 | 0.1997 |
376
+ | 0.1713 | 1400 | 0.1989 |
377
+ | 0.1774 | 1450 | 0.1952 |
378
+ | 0.1836 | 1500 | 0.1956 |
379
+ | 0.1897 | 1550 | 0.1984 |
380
+ | 0.1958 | 1600 | 0.1977 |
381
+ | 0.2019 | 1650 | 0.1943 |
382
+ | 0.2080 | 1700 | 0.1918 |
383
+ | 0.2142 | 1750 | 0.1959 |
384
+ | 0.2203 | 1800 | 0.1953 |
385
+ | 0.2264 | 1850 | 0.1935 |
386
+ | 0.2325 | 1900 | 0.1953 |
387
+ | 0.2386 | 1950 | 0.1946 |
388
+ | 0.2448 | 2000 | 0.1937 |
389
+ | 0.2509 | 2050 | 0.1895 |
390
+ | 0.2570 | 2100 | 0.1902 |
391
+ | 0.2631 | 2150 | 0.1878 |
392
+ | 0.2692 | 2200 | 0.1933 |
393
+ | 0.2753 | 2250 | 0.1890 |
394
+ | 0.2815 | 2300 | 0.1873 |
395
+ | 0.2876 | 2350 | 0.1840 |
396
+ | 0.2937 | 2400 | 0.1872 |
397
+ | 0.2998 | 2450 | 0.1889 |
398
+ | 0.3059 | 2500 | 0.1934 |
399
+ | 0.3121 | 2550 | 0.1922 |
400
+ | 0.3182 | 2600 | 0.1889 |
401
+ | 0.3243 | 2650 | 0.1897 |
402
+ | 0.3304 | 2700 | 0.1879 |
403
+ | 0.3365 | 2750 | 0.1852 |
404
+ | 0.3427 | 2800 | 0.1859 |
405
+ | 0.3488 | 2850 | 0.1838 |
406
+ | 0.3549 | 2900 | 0.1867 |
407
+ | 0.3610 | 2950 | 0.1864 |
408
+ | 0.3671 | 3000 | 0.1793 |
409
+ | 0.3732 | 3050 | 0.1863 |
410
+ | 0.3794 | 3100 | 0.1839 |
411
+ | 0.3855 | 3150 | 0.1875 |
412
+ | 0.3916 | 3200 | 0.1800 |
413
+ | 0.3977 | 3250 | 0.1818 |
414
+ | 0.4038 | 3300 | 0.1810 |
415
+ | 0.4100 | 3350 | 0.1844 |
416
+ | 0.4161 | 3400 | 0.1813 |
417
+ | 0.4222 | 3450 | 0.1846 |
418
+ | 0.4283 | 3500 | 0.1818 |
419
+ | 0.4344 | 3550 | 0.1761 |
420
+ | 0.4406 | 3600 | 0.1869 |
421
+ | 0.4467 | 3650 | 0.1840 |
422
+ | 0.4528 | 3700 | 0.1764 |
423
+ | 0.4589 | 3750 | 0.1745 |
424
+ | 0.4650 | 3800 | 0.1835 |
425
+ | 0.4711 | 3850 | 0.1777 |
426
+ | 0.4773 | 3900 | 0.1757 |
427
+ | 0.4834 | 3950 | 0.1746 |
428
+ | 0.4895 | 4000 | 0.1756 |
429
+ | 0.4956 | 4050 | 0.1753 |
430
+ | 0.5017 | 4100 | 0.1785 |
431
+ | 0.5079 | 4150 | 0.1762 |
432
+ | 0.5140 | 4200 | 0.1747 |
433
+ | 0.5201 | 4250 | 0.1787 |
434
+ | 0.5262 | 4300 | 0.1713 |
435
+ | 0.5323 | 4350 | 0.1782 |
436
+ | 0.5385 | 4400 | 0.1716 |
437
+ | 0.5446 | 4450 | 0.1685 |
438
+ | 0.5507 | 4500 | 0.1742 |
439
+ | 0.5568 | 4550 | 0.1754 |
440
+ | 0.5629 | 4600 | 0.1716 |
441
+ | 0.5691 | 4650 | 0.1771 |
442
+ | 0.5752 | 4700 | 0.1723 |
443
+ | 0.5813 | 4750 | 0.1759 |
444
+ | 0.5874 | 4800 | 0.1770 |
445
+ | 0.5935 | 4850 | 0.1773 |
446
+ | 0.5996 | 4900 | 0.1826 |
447
+ | 0.6058 | 4950 | 0.1736 |
448
+ | 0.6119 | 5000 | 0.1727 |
449
+ | 0.6180 | 5050 | 0.1719 |
450
+ | 0.6241 | 5100 | 0.1799 |
451
+ | 0.6302 | 5150 | 0.1708 |
452
+ | 0.6364 | 5200 | 0.1726 |
453
+ | 0.6425 | 5250 | 0.1687 |
454
+ | 0.6486 | 5300 | 0.1755 |
455
+ | 0.6547 | 5350 | 0.1731 |
456
+ | 0.6608 | 5400 | 0.1683 |
457
+ | 0.6670 | 5450 | 0.1696 |
458
+ | 0.6731 | 5500 | 0.1738 |
459
+ | 0.6792 | 5550 | 0.1687 |
460
+ | 0.6853 | 5600 | 0.1706 |
461
+ | 0.6914 | 5650 | 0.1685 |
462
+ | 0.6975 | 5700 | 0.1694 |
463
+ | 0.7037 | 5750 | 0.1695 |
464
+ | 0.7098 | 5800 | 0.1689 |
465
+ | 0.7159 | 5850 | 0.1619 |
466
+ | 0.7220 | 5900 | 0.1702 |
467
+ | 0.7281 | 5950 | 0.1665 |
468
+ | 0.7343 | 6000 | 0.1684 |
469
+ | 0.7404 | 6050 | 0.1648 |
470
+ | 0.7465 | 6100 | 0.1710 |
471
+ | 0.7526 | 6150 | 0.1739 |
472
+ | 0.7587 | 6200 | 0.1657 |
473
+ | 0.7649 | 6250 | 0.1737 |
474
+ | 0.7710 | 6300 | 0.1677 |
475
+ | 0.7771 | 6350 | 0.1711 |
476
+ | 0.7832 | 6400 | 0.1656 |
477
+ | 0.7893 | 6450 | 0.1619 |
478
+ | 0.7954 | 6500 | 0.1684 |
479
+ | 0.8016 | 6550 | 0.1684 |
480
+ | 0.8077 | 6600 | 0.1654 |
481
+ | 0.8138 | 6650 | 0.1673 |
482
+ | 0.8199 | 6700 | 0.1676 |
483
+ | 0.8260 | 6750 | 0.1693 |
484
+ | 0.8322 | 6800 | 0.1703 |
485
+ | 0.8383 | 6850 | 0.1658 |
486
+ | 0.8444 | 6900 | 0.1674 |
487
+ | 0.8505 | 6950 | 0.1705 |
488
+ | 0.8566 | 7000 | 0.1691 |
489
+ | 0.8628 | 7050 | 0.1661 |
490
+ | 0.8689 | 7100 | 0.1689 |
491
+ | 0.8750 | 7150 | 0.1675 |
492
+ | 0.8811 | 7200 | 0.1692 |
493
+ | 0.8872 | 7250 | 0.1687 |
494
+ | 0.8933 | 7300 | 0.1662 |
495
+ | 0.8995 | 7350 | 0.1666 |
496
+ | 0.9056 | 7400 | 0.1670 |
497
+ | 0.9117 | 7450 | 0.1654 |
498
+ | 0.9178 | 7500 | 0.1630 |
499
+ | 0.9239 | 7550 | 0.1692 |
500
+ | 0.9301 | 7600 | 0.1672 |
501
+ | 0.9362 | 7650 | 0.1681 |
502
+ | 0.9423 | 7700 | 0.1671 |
503
+ | 0.9484 | 7750 | 0.1641 |
504
+ | 0.9545 | 7800 | 0.1692 |
505
+ | 0.9607 | 7850 | 0.1686 |
506
+ | 0.9668 | 7900 | 0.1679 |
507
+ | 0.9729 | 7950 | 0.1643 |
508
+ | 0.9790 | 8000 | 0.1672 |
509
+ | 0.9851 | 8050 | 0.1682 |
510
+ | 0.9913 | 8100 | 0.1670 |
511
+ | 0.9974 | 8150 | 0.1680 |
512
+
513
+ </details>
514
+
515
+ ### Framework Versions
516
+ - Python: 3.12.13
517
+ - Sentence Transformers: 5.5.1
518
+ - PyLate: 1.6.0
519
+ - Transformers: 5.3.0
520
+ - PyTorch: 2.11.0+cu130
521
+ - Accelerate: 1.12.0
522
+ - Datasets: 4.8.5
523
+ - Tokenizers: 0.22.2
524
+
525
+
526
+ ## Citation
527
+
528
+ ### BibTeX
529
+
530
+ #### Sentence Transformers
531
+ ```bibtex
532
+ @inproceedings{reimers-2019-sentence-bert,
533
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
534
+ author = "Reimers, Nils and Gurevych, Iryna",
535
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
536
+ month = "11",
537
+ year = "2019",
538
+ publisher = "Association for Computational Linguistics",
539
+ url = "https://arxiv.org/abs/1908.10084"
540
+ }
541
+ ```
542
+
543
+ #### PyLate
544
+ ```bibtex
545
+ @inproceedings{DBLP:conf/cikm/ChaffinS25,
546
+ author = {Antoine Chaffin and
547
+ Rapha{"{e}}l Sourty},
548
+ editor = {Meeyoung Cha and
549
+ Chanyoung Park and
550
+ Noseong Park and
551
+ Carl Yang and
552
+ Senjuti Basu Roy and
553
+ Jessie Li and
554
+ Jaap Kamps and
555
+ Kijung Shin and
556
+ Bryan Hooi and
557
+ Lifang He},
558
+ title = {PyLate: Flexible Training and Retrieval for Late Interaction Models},
559
+ booktitle = {Proceedings of the 34th {ACM} International Conference on Information
560
+ and Knowledge Management, {CIKM} 2025, Seoul, Republic of Korea, November
561
+ 10-14, 2025},
562
+ pages = {6334--6339},
563
+ publisher = {{ACM}},
564
+ year = {2025},
565
+ url = {https://github.com/lightonai/pylate},
566
+ doi = {10.1145/3746252.3761608},
567
+ }
568
+ ```
569
+
570
+ <!--
571
+ ## Glossary
572
+
573
+ *Clearly define terms in order to be accessible across audiences.*
574
+ -->
575
+
576
+ <!--
577
+ ## Model Card Authors
578
+
579
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
580
+ -->
581
+
582
+ <!--
583
+ ## Model Card Contact
584
+
585
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
586
+ -->
config.json ADDED
@@ -0,0 +1,79 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "ModernBertModel"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": null,
8
+ "classifier_activation": "gelu",
9
+ "classifier_bias": false,
10
+ "classifier_dropout": 0.0,
11
+ "classifier_pooling": "mean",
12
+ "cls_token_id": 50281,
13
+ "decoder_bias": true,
14
+ "deterministic_flash_attn": false,
15
+ "dtype": "float32",
16
+ "embedding_dropout": 0.0,
17
+ "eos_token_id": null,
18
+ "global_attn_every_n_layers": 3,
19
+ "gradient_checkpointing": false,
20
+ "hidden_activation": "gelu",
21
+ "hidden_size": 768,
22
+ "initializer_cutoff_factor": 2.0,
23
+ "initializer_range": 0.02,
24
+ "intermediate_size": 1152,
25
+ "layer_norm_eps": 1e-05,
26
+ "layer_types": [
27
+ "full_attention",
28
+ "sliding_attention",
29
+ "sliding_attention",
30
+ "full_attention",
31
+ "sliding_attention",
32
+ "sliding_attention",
33
+ "full_attention",
34
+ "sliding_attention",
35
+ "sliding_attention",
36
+ "full_attention",
37
+ "sliding_attention",
38
+ "sliding_attention",
39
+ "full_attention",
40
+ "sliding_attention",
41
+ "sliding_attention",
42
+ "full_attention",
43
+ "sliding_attention",
44
+ "sliding_attention",
45
+ "full_attention",
46
+ "sliding_attention",
47
+ "sliding_attention",
48
+ "full_attention"
49
+ ],
50
+ "local_attention": 128,
51
+ "max_position_embeddings": 8192,
52
+ "mlp_bias": false,
53
+ "mlp_dropout": 0.0,
54
+ "model_type": "modernbert",
55
+ "norm_bias": false,
56
+ "norm_eps": 1e-05,
57
+ "num_attention_heads": 12,
58
+ "num_hidden_layers": 22,
59
+ "pad_token_id": 50284,
60
+ "position_embedding_type": "absolute",
61
+ "repad_logits_with_grad": false,
62
+ "rope_parameters": {
63
+ "full_attention": {
64
+ "rope_theta": 160000.0,
65
+ "rope_type": "default"
66
+ },
67
+ "sliding_attention": {
68
+ "rope_theta": 10000.0,
69
+ "rope_type": "default"
70
+ }
71
+ },
72
+ "sep_token_id": 50282,
73
+ "sparse_pred_ignore_index": -100,
74
+ "sparse_prediction": false,
75
+ "tie_word_embeddings": true,
76
+ "transformers_version": "5.3.0",
77
+ "use_cache": false,
78
+ "vocab_size": 50370
79
+ }
config_sentence_transformers.json ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "pytorch": "2.11.0+cu130",
4
+ "sentence_transformers": "5.5.1",
5
+ "transformers": "5.3.0"
6
+ },
7
+ "attend_to_expansion_tokens": false,
8
+ "default_prompt_name": null,
9
+ "do_query_expansion": false,
10
+ "document_length": 300,
11
+ "document_prefix": "[D] ",
12
+ "model_type": "SentenceTransformer",
13
+ "prompts": {
14
+ "document": "",
15
+ "query": ""
16
+ },
17
+ "query_length": 32,
18
+ "query_prefix": "[Q] ",
19
+ "similarity_fn_name": "MaxSim",
20
+ "skiplist_words": [
21
+ "!",
22
+ "\"",
23
+ "#",
24
+ "$",
25
+ "%",
26
+ "&",
27
+ "'",
28
+ "(",
29
+ ")",
30
+ "*",
31
+ "+",
32
+ ",",
33
+ "-",
34
+ ".",
35
+ "/",
36
+ ":",
37
+ ";",
38
+ "<",
39
+ "=",
40
+ ">",
41
+ "?",
42
+ "@",
43
+ "[",
44
+ "\\",
45
+ "]",
46
+ "^",
47
+ "_",
48
+ "`",
49
+ "{",
50
+ "|",
51
+ "}",
52
+ "~"
53
+ ]
54
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d8222f2f8f7873de8e79e6722cf850b10b1a82f72b77fa0a47b466b39c1e9029
3
+ size 596076280
modules.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.base.modules.transformer.Transformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_Dense",
12
+ "type": "pylate.models.Dense.Dense"
13
+ },
14
+ {
15
+ "idx": 2,
16
+ "name": "2",
17
+ "path": "2_Dense",
18
+ "type": "pylate.models.Dense.Dense"
19
+ },
20
+ {
21
+ "idx": 3,
22
+ "name": "3",
23
+ "path": "3_Dense",
24
+ "type": "pylate.models.Dense.Dense"
25
+ }
26
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "transformer_task": "feature-extraction",
3
+ "modality_config": {
4
+ "text": {
5
+ "method": "forward",
6
+ "method_output_name": "last_hidden_state"
7
+ }
8
+ },
9
+ "module_output_name": "token_embeddings"
10
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "backend": "tokenizers",
3
+ "clean_up_tokenization_spaces": true,
4
+ "cls_token": "[CLS]",
5
+ "is_local": true,
6
+ "mask_token": "[MASK]",
7
+ "max_length": 299,
8
+ "model_input_names": [
9
+ "input_ids",
10
+ "attention_mask"
11
+ ],
12
+ "model_max_length": 299,
13
+ "pad_to_multiple_of": null,
14
+ "pad_token": "[MASK]",
15
+ "pad_token_type_id": 0,
16
+ "padding_side": "right",
17
+ "sep_token": "[SEP]",
18
+ "stride": 0,
19
+ "tokenizer_class": "TokenizersBackend",
20
+ "truncation_side": "right",
21
+ "truncation_strategy": "longest_first",
22
+ "unk_token": "[UNK]"
23
+ }