Elafnawaf/gliner2-arabic-saudi

GLiNER2 Arabic — Saudi round 2

One mDeBERTa encoder, four heads, tuned for Saudi text. This is the second round of Elafnawaf/gliner2-arabic-multitask, a GLiNER2 boundary checkpoint (287M parameters) built on fastino/gliner2.5-multi-v1. Round 1 beat the base model on its held-out split and then lost to it on realistic Saudi sentences: asked for eight types at once it returned no dates, amounts or percentages. Round 2 fixes that in the training data and is measured on a hand-written Saudi benchmark.

What changed from round 1

• Negatives only for labels a corpus annotates. WikiANN/IAHLT sentences no longer carry the 14 labels they never annotate as explicit negatives; the label subset is sampled per example and shuffled at train time.
• Real numeric mentions. Regex-labelled dates, Hijri dates, money, percentages, times, phones and IDs on SANAD news and Gulf tweets; 9k generated Saudi sentences (half in dialect) that put those types beside people, organisations and places; ShamNER spoken Levantine for clean entity boundaries.
• Type-balanced replay (person/org/place from 76 % to 50 % of mentions) and class-balanced sentiment, sarcasm and dialect with ArSarcasm v2, IADD and Gulf tweets.
• Started from a 50 % weight average of round 1 and the base model, then one epoch (129k examples, fp32).

Ask for every type you need in one request. Splitting the label list, which helped round 1, now costs 0.04 F1.

Results

Saudi benchmark (42 sentences, 20 in Saudi dialect, 179 mentions, mention-level F1, all 10 types asked at once)

model	F1	P	R	dialect	MSA
fastino/gliner2.5-multi-v1 (base)	0.783	0.790	0.777	0.728	0.829
Elafnawaf/gliner2-arabic-multitask (round 1)	0.650	0.756	0.570	0.684	0.615
soup α=0.5 of round 1 and base	0.802	0.836	0.771	0.745	0.852
this model (round 2)	0.895	0.913	0.877	0.850	0.935

type	n	base	round 1	round 2
شخص / person	12	0.593	0.640	0.833
منظمة / organization	39	0.784	0.827	0.923
مكان / place	39	0.754	0.640	0.694
تاريخ / date	35	0.794	0.458	0.928
وقت / time	9	0.600	0.000	1.000
مبلغ مالي / money	25	0.960	0.750	1.000
نسبة مئوية / percentage	9	0.947	0.500	1.000
رقم جوال / mobile number	6	0.800	0.909	1.000
رقم الهوية الوطنية / national id number	3	0.500	1.000	1.000
مطار / airport	2	1.000	1.000	1.000

Conventions of the benchmark: person without title, organisation without a generic شركة, numeric dates without يوم, weekday names with it, clock times without الساعة, amounts with their currency word. Seven of round 2's 37 remaining errors are preposition clitics (بجدة returned for جدة), the convention WikiANN and IAHLT follow.

Held-out split (data/gliner test, 1,500 sentences of WikiANN, IAHLT, Wojood and synthetic PII)

query	round 1	round 2
all 17 labels	0.694	0.586
only the labels each sentence's corpus annotates	0.669	0.667

The 17-label number falls because WikiANN and IAHLT never annotated dates, ordinals or cardinals, so a model that now finds them is charged a false positive for each one (423 of round 2's extra false positives). On the labels the corpora annotate the two rounds tie.

Other heads (round-2 dev mix, 600 examples each; the start checkpoint is the 50 % soup)

head	start	round 2
structure, field F1	0.645	0.867
relations, triple F1 (REDFM)	0.381	0.391
topic / الموضوع accuracy	0.879 / 0.981	0.966 / 1.000
sentiment / المشاعر accuracy	0.734 / 0.681	0.766 / 0.714
speech act / نوع الخطاب accuracy	0.641 / 0.657	0.688 / 0.642
dialect / اللهجة accuracy	0.155 / 0.172	0.333 / 0.356

Known limitation: the dialect task over-predicts خليجي (80 % of its predictions on the dev mix), because Gulf sources outweigh the others in this round's mix. Treat dialect as a coarse signal.

Quick start

pip install "gliner2[local]"

from gliner2.auto import AutoExtractor

# AutoExtractor reads the architecture field in config.json and picks the right
# class. Plain GLiNER2.from_pretrained is span-only and refuses this checkpoint.
model = AutoExtractor.from_pretrained("Elafnawaf/gliner2-arabic-saudi")

Normalise first — this is not optional

GLiNER's word splitter is \w+(?:[-_]\w+)*|\S, and Python's \w does not match Arabic diacritics. A diacritised word shatters into one token per character and every span offset after it is meaningless. Tatweel (ـ) fails the other way: it does match \w, so it survives glued to a real word. Strip both before inference:

import re, unicodedata

_DIACRITICS = re.compile(r"[\u0610-\u061A\u064B-\u065F\u0670\u06D6-\u06ED]")
_INVISIBLE  = re.compile(r"[\u200B-\u200F\u202A-\u202E\u2066-\u2069\uFEFF]")

def normalise(text: str) -> str:
    text = unicodedata.normalize("NFC", text)
    text = _DIACRITICS.sub("", text)       # حركات
    text = text.replace("\u0640", "")      # tatweel ـ
    text = _INVISIBLE.sub("", text)        # ZWJ / RLM / BOM ...
    return text

1 · Entities

text = normalise("أعلنت شركة أرامكو السعودية في الظهران يوم 5 مارس 2024 عن أرباح بلغت "
                 "121 مليار دولار، ويمكن التواصل عبر الرقم 0501234567.")

model.extract_entities(text, ["منظمة / organization", "مكان / place",
                              "مبلغ مالي / money", "رقم جوال / mobile number"])
# {'entities': {'منظمة / organization': ['أرامكو السعودية'],
#               'مكان / place': ['الظهران'],
#               'مبلغ مالي / money': ['121 مليار دولار'],
#               'رقم جوال / mobile number': ['0501234567']}}

# character offsets and confidence, e.g. for redaction
model.extract_entities(text, ["منظمة / organization"],
                       include_spans=True, include_confidence=True)
# {'entities': {'منظمة / organization': [
#     {'text': 'أرامكو السعودية', 'confidence': 0.866, 'start': 11, 'end': 26}]}}

The entity head was trained with bilingual label names of the form عربي / english (full list below). Use those exact strings for the best precision; plain منظمة or organization also work.

2 · Classification

review = normalise("الخدمة كانت ممتازة والتوصيل وصل قبل الموعد، شكراً لكم")

model.classify_text(review, {
    "المشاعر":    ["إيجابي", "سلبي", "محايد"],
    "نوع الخطاب": ["سؤال", "طلب", "شكوى", "تعبير", "إعلان", "أخرى"],
})
# {'المشاعر': 'إيجابي', 'نوع الخطاب': 'تعبير'}

# the same task, asked in English, with confidence
model.classify_text(review, {"sentiment": ["positive", "negative", "neutral"]},
                    include_confidence=True)
# {'sentiment': {'label': 'positive', 'confidence': 0.998}}

model.classify_text(normalise("والله الجو اليوم يجنن، بس الزحمة تقتل"),
                    {"اللهجة": ["خليجي", "مصري", "شامي", "مغاربي", "فصحى"]})
# {'اللهجة': 'فصحى'}

3 · Structured records (JSON)

news = normalise("تأسست شركة الاتصالات السعودية عام 1998 ومقرها الرياض، "
                 "ويرأسها المهندس عليان الوتيد.")

model.extract_json(news, {"شركة": ["الاسم", "المقر", "سنة التأسيس", "الرئيس"]})
# {'شركة': [{'الاسم': ['الاتصالات السعودية'], 'المقر': ['الرياض'],
#            'سنة التأسيس': ['1998'], 'الرئيس': ['عليان الوتيد']}]}

Field values are lists (a field may occur more than once) and extraction is extractive: every value should be a substring of the input. A value that is not is a hallucination — log it. Richer schemas from the gliner2 API (ChoiceField for a classification inside a record, list-valued fields, several records of the same name under an occurrence_policy) are supported and were part of training.

4 · Relations

model.extract_relations(news, ["المقر", "تاريخ التأسيس", "المهنة"])
# {'relation_extraction': {'المقر': [['شركة الاتصالات السعودية', 'الرياض']],
#                          'تاريخ التأسيس': [['شركة الاتصالات السعودية', '1998']],
#                          'المهنة': []}}

model.extract_relations(news, ["headquarters location", "inception"])
# {'relation_extraction': {'headquarters location': [['شركة الاتصالات السعودية', 'الرياض']],
#                          'inception': [['شركة الاتصالات السعودية', '1998']]}}

Everything at once

from gliner2 import Schema

schema = (Schema()
          .entities(["شخص / person", "منظمة / organization", "مكان / place"])
          .classification("المشاعر", ["إيجابي", "سلبي", "محايد"])
          .structure("شركة").field("الاسم").field("المقر")
          .relations(["المقر"]))
model.extract(news, schema)
# {'entities': {'شخص / person': ['عليان الوتيد'],
#               'منظمة / organization': ['شركة الاتصالات السعودية'],
#               'مكان / place': ['الرياض']},
#  'المشاعر': 'محايد',
#  'شركة': [{'الاسم': ['شركة الاتصالات السعودية'], 'المقر': ['الرياض']}],
#  'relation_extraction': {'المقر': [['شركة الاتصالات السعودية', 'الرياض']]}}

5 · Multi-label classification

One task, several true labels. Ask with multi_label=True through the schema builder:

from gliner2 import Schema

ticket = normalise("تواصل معنا العميل سعود الدوسري، رقم الهوية 1098765432، عبر الجوال 0551234567 "
                   "بخصوص فاتورة بقيمة 450 ريالاً صادرة بتاريخ 3 مايو 2024 من فرع الرياض.")
TYPES = ["شخص / person", "منظمة / organization", "مكان / place", "تاريخ / date",
         "مبلغ مالي / money", "رقم جوال / mobile number", "رقم الهوية الوطنية / national id number"]

model.extract(ticket, Schema().classification("أنواع الكيانات", TYPES, multi_label=True))
# {'أنواع الكيانات': ['شخص / person', 'مكان / place', 'تاريخ / date',
#                    'رقم جوال / mobile number', 'رقم الهوية الوطنية / national id number']}

On held-out data this returns the exact label set 77% of the time (per-label F1 0.94). It is a cheap first pass before running the entity head on a large corpus.

6 · Records with choice fields and list fields

A choices=[...] field is a classification inside a record: its value comes from the list, not from the text. dtype="list" lets a field hold several values. Together they give you an extracted record plus a decision in one call, which is the shape a PDPL redaction log needs.

record = (Schema().structure("سجل")
          .field("شخص / person", dtype="list")
          .field("رقم الهوية الوطنية / national id number")
          .field("نوع السجل", choices=TYPES)
          .field("يحتوي بيانات شخصية", choices=["نعم", "لا"]))

model.extract(ticket, record)
# {'سجل': [{'شخص / person': ['سعود الدوسري'],
#           'رقم الهوية الوطنية / national id number': ['1098765432'],
#           'نوع السجل': ['شخص / person', 'رقم الهوية الوطنية / national id number'],
#           'يحتوي بيانات شخصية': ['لا']}]}

Read the last line carefully. On the held-out set the record-type field is right 96% of the time and the personal-data flag 100%, but on fresh support text the flag under-fires, as above, where an ID number is present and the answer should be نعم. Do not let the model's flag be the only gate: derive it from the extracted fields (bool(record["شخص / person"] or record["رقم الهوية الوطنية / national id number"])) and keep the choice field as a second opinion.

7 · Several records of one name from one text

Records were trained with mode/occurrence_policy metadata so that one text can yield a list of records. This is the weakest shape in the current checkpoint:

meeting = normalise("حضر الاجتماع المهندس فهد العتيبي من شركة أرامكو، والدكتورة ريم الشهري من "
                    "جامعة الملك سعود، والأستاذ ماجد القحطاني من هيئة الاتصالات.")

model.extract(meeting, Schema().structure("مشارك", mode="natural", occurrence_policy="all")
                                .field("الاسم").field("الجهة"))
# {'مشارك': [{'الاسم': 'فهد العتيبي', 'الجهة': ['أرامكو']}]}      <- one of three

Held-out pair F1 is 0.68 with recall as the limit (about one record returned per two or three in gold). When you need every participant, ask the entity head, which does return them all, and pair by proximity yourself:

model.extract_entities(meeting, ["شخص / person", "منظمة / organization"])
# {'entities': {'شخص / person': ['فهد العتيبي', 'ريم الشهري'],
#               'منظمة / organization': ['أرامكو', 'جامعة الملك سعود', 'هيئة الاتصالات']}}

8 · New labels, explained rather than named

The schema is text, so a label the model never saw can carry a description. Pass a dict instead of a list, for entity types and for classification labels alike:

model.extract(ticket, Schema().entities({
    "رقم مرجعي": "رقم يعرّف عميلاً أو وثيقة، مثل رقم الهوية أو رقم الفاتورة",
    "مبلغ":      "قيمة مالية مع عملتها",
}))
# {'entities': {'رقم مرجعي': ['1098765432'], 'مبلغ': ['450 ريالاً']}}

model.extract(ticket, Schema().classification("نوع الطلب", {
    "شكوى":     "العميل غير راضٍ عن خدمة أو فاتورة",
    "استفسار":  "العميل يسأل عن معلومة",
    "طلب خدمة": "العميل يريد تفعيل أو إلغاء خدمة",
}))
# {'نوع الطلب': 'استفسار'}

Relation descriptions are accepted by the API but did not help this checkpoint on business relations (شراكة, استحواذ came back empty); the relation head knows the 32 Wikidata predicates listed below.

9 · Batches and long documents

Every extract_* call has a batch_* twin that takes a list of texts, and a *_long twin that chunks a document (default 384 tokens with 64 of overlap) and merges the spans:

model.batch_extract_entities([ticket, meeting], ["شخص / person", "منظمة / organization"],
                             batch_size=8, threshold=0.3)
# [{'entities': {'شخص / person': ['سعود الدوسري'], 'منظمة / organization': []}},
#  {'entities': {'شخص / person': ['فهد العتيبي', 'ريم الشهري'],
#                'منظمة / organization': ['أرامكو', 'جامعة الملك سعود', 'هيئة الاتصالات']}}]

model.extract_entities_long(long_document, ["شخص / person", "رقم جوال / mobile number"],
                            chunk_size=384, chunk_overlap=64, include_spans=True)

Batching sorts by length internally; on one L4 the entity head scores about 200 sentences per second at batch 8.

threshold (default 0.5) is accepted by every call; lower it for recall, raise it for precision. Per-label entity thresholds tuned on the dev set are in inference_config.json, together with every task name and label set the model was trained on, in both languages.

What it was trained to answer

Entity types (17)

تاريخ / date, ترتيب / ordinal, رقم الهوية الوطنية / national id number, رقم جوال / mobile number, شخص / person, عدد / cardinal number, عملة / currency, عنوان / address, كمية / quantity, مبلغ مالي / money, مطار / airport, مكان / place, منظمة / organization, موقع إلكتروني / website, نسبة مئوية / percentage, وحدة قياس / unit of measurement, وقت / time

Classification tasks

task (ar)	task (en)	labels
اللهجة	dialect	مصري (egypt), خليجي (gulf), شامي (levant), مغاربي (magreb), فصحى (msa)
السخرية	sarcasm	غير ساخر (non-sarcastic), ساخر (sarcastic)
المشاعر	sentiment	سلبي (negative), محايد (neutral), إيجابي (positive), سلبي (Negative), محايد (Neutral), إيجابي (Positive), مختلط (Mixed)
نوع الخطاب	speech act	تأكيد (Assertion), توصية (Recommendation), تعبير (Expression), سؤال (Question), طلب (Request), أخرى (Miscellaneous)
الموضوع	topic	تقنية (Tech), اقتصاد (Finance), سياسة (Politics), دين (Religion), طب (Medical), ثقافة (Culture), رياضة (Sports)

Relation types (32, Wikidata predicates from REDFM)

English	Arabic
author	المؤلف
capital	العاصمة
cast member	من طاقم العمل
characters	الشخصيات
child	الابن
country	الدولة
country of citizenship	بلد الجنسية
director	المخرج
follows	يتبع
founded by	أسسها
genre	النوع
headquarters location	المقر
inception	تاريخ التأسيس
instance of	نوع من
league	الدوري
located in or next to body of water	يقع على مسطح مائي
location	الموقع
manufacturer	الشركة المصنعة
member of	عضو في
mouth of the watercourse	مصب النهر
named after	سمي نسبة إلى
notable work	عمل بارز
occupation	المهنة
owned by	مملوك لـ
part of	جزء من
participant	مشارك
place of birth	مكان الميلاد
replaces	يحل محل
shares border with	يحد
sibling	شقيق
sport	الرياضة
spouse	الزوج

Training details

Start: weight average (α=0.5) of round 1 and fastino/gliner2.5-multi-v1. One epoch over 129,105 examples (entities 48k, classification 26k + ArSAS 12k + SANAD 10k, SREDFM relations 16k, structures 23k), effective batch 16, encoder LR 5e-06, task LR 0.0002, fp32 (bf16 diverges on this model), gradient checkpointing, gliner2 SamplingConfig with shuffle_entities=True, remove_entity_prob=0.2. 143 minutes on one NVIDIA L4. Every example was passed through gliner2's own collate before training; 5 were rejected.

Files

model.safetensors, config.json, encoder_config/, tokenizer files, and inference_config.json (entity labels and thresholds, task names in both languages, relation types, benchmark numbers).

Credits

Base model by Fastino AI. Data: WikiANN, IAHLT/MAFAT, Wojood sample (SinaLab), ArSarcasm v1/v2, ArSAS, SANAD, SREDFM/REDFM (Babelscape), IADD, Gulf Arabic Tweets 2018–2020, ShamNER. Trained on an OpenShift AI workbench.