Visual Intelligence Seminar: EoS-FM: Can an Ensemble of Specialists Models Act As a Generalist Feature Extractor?

Abstract

Foundation models are emerging in Earth Observation with the goal of generalizing across tasks while reducing the need for task-specific training. However, current approaches rely on ever-larger models and datasets, requiring substantial computational resources and raising concerns about accessibility and sustainability. We introduce EoS-FM, an Ensemble-of-Specialists framework that decomposes training into lightweight, task-specific ConvNeXtV2 specialists that can be frozen and reused. This modular design improves training efficiency, interpretability, and extensibility while naturally supporting federated training, pruning, and continuous integration of new specialists, making it well-suited for collaborative and resource-constrained environments.