Remove FIM limitation note
Browse files
README.md
CHANGED
|
@@ -101,10 +101,6 @@ next_logits = fim_forward(model, before_ids, after_ids, lang_id=LANG_ID)
|
|
| 101 |
print(tok.decode([next_logits.argmax().item()]))
|
| 102 |
```
|
| 103 |
|
| 104 |
-
> **Note**: These models were trained primarily for prefix completion. The `wne` embedding
|
| 105 |
-
> (norm ~11 vs `wpe` ~110) is small, suggesting limited FIM training. Suffix-first FIM
|
| 106 |
-
> (Method 1) leverages causal attention to incorporate suffix context without special tokens.
|
| 107 |
-
|
| 108 |
## Configuration
|
| 109 |
|
| 110 |
| Field | Value |
|
|
|
|
| 101 |
print(tok.decode([next_logits.argmax().item()]))
|
| 102 |
```
|
| 103 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 104 |
## Configuration
|
| 105 |
|
| 106 |
| Field | Value |
|