Shipped checkpoint re-evaluated on the training split (this code)¶

MEIDNet verified · ●○○○○ generated


Dataset	Perov-5
Modalities	`structure`, `property:heat_all`, `property:dir_gap`
Model	MEIDNet early fusion + curriculum (production model) (MEIDNet 2.0.0, perovskite_abx3)
Category	Representation quality
Submitted by	Anand Babu (UCLouvain) on 2026-10-03
Configuration	`examples/perov5/meidnet.yaml`
Checkpoint	dual_autoencoder_clip_earlyfusion_propertyaware_2k.pth
Resources	laptop CPU (AMD64)

Results¶

metric	value	split	meaning	note
`cosine_matched`	0.771	train	mean cosine similarity between the structure and property latents of the same material (1 = aligned)	mean over the 11,356 training materials
`l2_matched`	0.676	train	mean L2 distance between those two latents (0 = identical)
`retrieval_top1`	0.0101	train	fraction of validation materials whose property latent is nearest to its own structure latent
`retrieval_top5`	0.0405	train	the same within the five nearest
`mae_heat_all`	0.397 eV/atom	train	mean absolute error of `heat_all` predicted from the structure alone (physical units)
`r2_heat_all`	0.486	train	coefficient of determination of that prediction; 0 = no better than the mean, 1 = perfect
`mae_dir_gap`	2.78 eV	train	mean absolute error of `dir_gap` predicted from the structure alone (physical units)
`r2_dir_gap`	-29.8	train	coefficient of determination of that prediction; 0 = no better than the mean, 1 = perfect
`n_evaluated`	11,356 materials	train	materials in the evaluation split

Validation¶

Evidence level: ●○○○○ generated.

Evidence:

checkpoints/dual_autoencoder_clip_earlyfusion_propertyaware_2k.pth

Reproduce¶

Configuration examples/perov5/meidnet.yaml, split train:

python scripts/benchmarks.py reproduce shipped-checkpoint-train-split

Verified here¶

Reproduced on 2026-10-03 with MEIDNet 2.0.0 by Anand Babu on AMD64 CPU:

metric	submitted	obtained here
`mae_heat_all`	0.397	0.397
`r2_heat_all`	0.486	0.486
`mae_dir_gap`	2.78	2.78
`r2_dir_gap`	-29.8	-29.8
`retrieval_top1`	0.0101	0.0101
`retrieval_top5`	0.0405	0.0405
`cosine_matched`	0.771	0.771
`n_evaluated`	11,356	11,356
`l2_matched`	0.676	0.676

Per-material analysis¶

Every validation material, predicted from its structure alone; the dashed line is perfect agreement.

Notes¶

The same evaluation on the data the model was trained on. The numbers equal the held-out ones within noise: the gap between the paper's figures and these is not a train / test effect.