PalmSat

Farm-level intelligence for date-palm orchards from free Sentinel-2 imagery alone — no field sensors, no gateways, no drones.

Code, data, and the scripts behind every number in the IEEE paper PalmSat: Satellite-Only Machine Learning for Date-Palm Farm Intelligence (paper/main.pdf).

The problem

Precision agriculture assumes instrumentation: soil probes, weather stations, gateways, UAV overflights. Date orchards are where that assumption breaks. They are dispersed across arid regions, often outside reliable cellular coverage, and fragmented into smallholder plots where the cost of installing and maintaining hardware exceeds what the parcel earns. The result is that the farms that would benefit most from monitoring are the ones least likely to get it.

Satellites remove the hardware entirely. Sentinel-2 revisits every point on Earth roughly every five days at 10 m resolution, for free. But satellite agriculture research is almost entirely single-task — map the crop, or classify it, or forecast one index — while running a farm means doing three things at once: judging how a parcel is performing, anticipating where its canopy is heading, and noticing when something is wrong.

PalmSat asks whether one satellite-derived representation can serve all three, and then measures honestly how far that gets you on ten real farms.

Results

Ten commercial farms in Al Madinah, Saudi Arabia. 1,835 cloud-filtered acquisitions, January 2022 – December 2024, 30 complete farm-years, mean NDVI 0.437. Models train on 2022–2023 and are tested on a held-out 2024, with hyper-parameters chosen by chronological cross-validation inside the training years only.

Forecasting beats the baseline by a wide, consistent margin

One-step-ahead NDVI prediction on the held-out year, after a 132-configuration search:

Model	RMSE (default)	RMSE (tuned)	R² (tuned)	95% CI on RMSE
Persistence (baseline)	0.1381	0.1381	−0.766	[0.124, 0.150]
MLP (PyTorch)	0.1150	0.1049	−0.018	[0.098, 0.112]
Ridge	0.0970	0.0955	0.156	[0.087, 0.102]
Random Forest	0.0992	0.0950	0.165	[0.086, 0.103]
Gradient Boosting	0.0964	0.0946	0.172	[0.087, 0.102]

The gain over persistence is 31.5% (95% CI 29.4–33.2%), and gradient boosting beats persistence on 10/10 farms individually (sign test p = 0.001).

The gaps between the learned models are not real: the bootstrap CI for gradient boosting's gain over ridge regression is [−0.4, +2.2]%. A twelve-feature ridge regression is as good an operational choice.

The ceiling is measurement noise, not model capacity

On near-clear scenes (<5% cloud) the same model reaches R² = 0.276; on cloudier ones R² goes negative. Decomposing the test year into a 3-visit trend plus a residual puts 58.6% of NDVI variance in the residual, which no lag-based model can recover — an indicative R² ceiling around 0.41, of which the tuned model attains about 42%.

Farms rank reproducibly; the tier labels are a convention

Unsupervised clustering of 30 farm-years yields three groups (silhouette 0.466 at k = 3; bootstrap ARI 0.87). The ordering of farms by cumulative greenness survives a change of growing-year definition (Spearman ρ = 0.96) and partially survives a change of period (ρ = 0.67 between 2022–23 and 2024).

The three-group structure does not: silhouette is flat across k = 2–4, a September–August growing year moves the optimum to k = 2, and no farm holds the same tier across all three years. Premium / Standard / Economy are descriptive names for phenological groups, not economic grades — there is no yield or revenue label anywhere in this dataset.

Anomaly triage ranks well but is not an alarm

Isolation Forest flags 92 observations (5.0%) at a chosen inspection budget. Benchmarked by planting synthetic NDVI drops of known size, it scores ROC-AUC 0.74 for 1σ events rising to 0.96 at 4σ — but at a 5% budget it recovers only about half of even 4σ events.

46.7% of its flags fall in the top decile of cloud cover, so close to half are plausibly atmospheric rather than agronomic. Cloud fraction should be displayed next to every flag.

What the indices actually measure

NDVI and SAVI correlate at r = 1.000 — both are monotone functions of the same red/NIR pair — and NDVI and NDWI at r = −0.979. A single principal component explains 99.1% of their joint variance. NDWI is not an independent irrigation signal in this dataset; separating water status from greenness needs an SWIR index or radar.

One more thing worth stating plainly: canopy greenness in these orchards peaks in the cool season (January 0.502) and troughs in high summer (July 0.395). An earlier draft of the paper described this backwards.

Scope of the result

Ten farms, one region, three years, one held-out test year, and no ground-truth yield, revenue, or agronomic labels. This is a feasibility study, not a generalizable benchmark. Per-farm greenness ranking is reproducible enough to prioritise attention (ρ = 0.67); flag rate is not a stable farm attribute (ρ = 0.10, p = 0.78) and belongs on an observation as an inspection trigger, never aggregated into a farm score.

How it works

Sentinel-2 L2A scenes are cloud-masked and reduced to a per-visit vector of NDVI, NDWI, SAVI and scene cloud percentage. One feature-engineering pass turns that series into a shared representation — autoregressive lags, rolling statistics and cyclical date encodings for the per-visit view; six phenological descriptors per farm-year for the seasonal view. Three modules then read that single representation: K-Means over farm-years, a five-model forecasting benchmark, and an Isolation Forest over per-visit vectors.

Two design decisions a reader may want to question:

Scene cloud percentage is kept as a model feature, not just a filter. In this regime optical quality is itself predictive — it ranks among the top features — because it tracks measurement error rather than vegetation.
The split is chronological, never random. Train is 2022–2023, test is 2024, and the hyper-parameter search runs inside the training years. Random splits on a panel time series leak neighbouring revisits across the boundary and inflate every score.

Repository layout

├── data/
│   ├── raw/              10 per-farm Sentinel-2 time series — the source of truth
│   ├── processed/        derived tables (observations, features, metrics, flags)
│   └── reference/        farm catalogue with coordinates + append-only update log
├── src/                  the analysis pipeline and its validation experiments
├── notebooks/            explore the corpus, reproduce the paper, extend the data
├── paper/                LaTeX source, figures, compiled PDF, result artifacts
├── pipeline/             incremental Earth Engine updater (extends the corpus)
├── docs/                 figure prompts and superseded drafts
└── third_party/          upstream snapshots, unmodified

Key files:

File	What it is
`paper/main.pdf`	The paper.
`paper/results.json`	Every headline result, machine-readable.
`paper/validation.json`	Tuning, bootstrap CIs, stability and benchmark results.
`paper/response_to_reviewers.md`	Point-by-point reviewer response, including the analyses that returned negative results.
`src/run_palm_ml.py`	The pipeline: clustering, forecasting, anomalies, figures.
`src/run_reviewer_experiments.py`	Hyper-parameter search, uncertainty, stability, injection benchmark.
`data/processed/headline_numbers.csv`	Every number quoted above, with its source artifact.

Reproducing it

git clone https://huggingface.co/FatimahEmadEldin/palmsat-paper
cd palmsat-paper
pip install -r requirements.txt

python src/run_palm_ml.py               # -> paper/results.json + paper/figs/
python src/run_reviewer_experiments.py  # -> paper/validation.json  (~3 min)
python src/export_processed_data.py     # -> data/processed/*.csv
python src/check_readme_numbers.py      # fails if this README has drifted

Or make all. Everything is seeded (SEED = 42) and deterministic; re-running from a clean clone reproduces the committed artifacts.

To rebuild the paper you also need a LaTeX distribution:

cd paper && latexmk -pdf -bibtex main.tex

Start with notebooks/01_explore_dataset.ipynb to see the corpus, or notebooks/02_reproduce_paper_results.ipynb to walk through the results and re-fit the winning model.

Extending the corpus

The orchards keep being imaged. pipeline/ holds an append-only Earth Engine updater that fetches only scenes newer than each farm's last observation, backs up before writing, and de-duplicates on acquisition timestamp, so re-running is safe. notebooks/03_update_data_to_2026.ipynb runs it in Colab. More years means a larger held-out test set and more farm-years for clustering — the two limits this study runs into hardest.

Citing

@inproceedings{emadeldin2026palmsat,
  title     = {{PalmSat}: Satellite-Only Machine Learning for Date-Palm Farm Intelligence},
  author    = {Emad Eldin, Fatimah and Alnakhli, Mohammad},
  booktitle = {Proc. MCAIT},
  year      = {2026}
}

Data derived from Copernicus Sentinel-2, accessed via Google Earth Engine. Built on and extending astral-fate/Palm-analyzer and the Farm-Performance-Score Space.

Released under the MIT License.

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support