Next-Generation Earth Observation: One Model for All Cases

Earth observation (EO) is experiencing rapid progress due to constantly growing data volumes and new sensors. The analysis of this data, collected by satellites, aircraft, and drones, offers valuable insights into environmental changes, agriculture, urban planning, and many other areas. A central problem is the heterogeneity of the data: Resolutions, scales, and modalities (e.g., optical, radar, or hyperspectral data) vary greatly, making the development of universally applicable models difficult. Previous approaches often require fixed input configurations and are therefore only applicable to a limited extent.

A new research contribution promises a remedy: AnySat, a multimodal model based on a "Joint Embedding Predictive Architecture" (JEPA) and adaptive spatial encoders. This approach enables the training of a single model with heterogeneous data in a self-supervised manner. By combining JEPA, which projects the different modalities into a common embedding space, and the adaptive encoders, which take into account the different resolutions, AnySat can effectively combine and process information from different data sources.

GeoPlex: A Versatile Dataset for Training AnySat

To demonstrate the capabilities of AnySat, GeoPlex was developed, a collection of five multimodal datasets with different characteristics and data from eleven different sensors. This broad spectrum of data allows AnySat to be trained on a variety of scenarios and to test its adaptability. Simultaneous training with these diverse datasets leads to a robust and generalizable model.

Versatile Applications and Convincing Results

AnySat can be used for various environmental monitoring tasks, including:

• Land cover mapping
• Tree species identification
• Crop classification
• Change detection
• Flood segmentation

After fine-tuning, AnySat achieved results in tests with GeoPlex and four other datasets that match or even exceed the current state of the art. This underscores the model's potential to revolutionize the analysis of Earth observation data.

Easy Application and Flexible Adaptation

The developers of AnySat emphasize user-friendliness. The model can be installed and downloaded with a single line of code. Selecting the desired modalities and patch size allows for the immediate generation of meaningful features. Furthermore, AnySat supports both fine-tuning and linear probing for tasks such as tile classification and semantic segmentation.

Outlook and Significance for the Future of Earth Observation

AnySat represents an important step towards a unified and flexible analysis of Earth observation data. The ability to train a single model for different resolutions, scales, and modalities simplifies the workflow and enables new applications. The promising results in various environmental monitoring tasks indicate that AnySat has the potential to significantly influence Earth observation and contribute to more precise and efficient analyses. The availability of the code and models on platforms like GitHub promotes further research and development in this area.