Eyes in the Sky: How Artificial Intelligence Learned to “Read” Earth from Obit

Today, thousands of satellites orbit Earth, generating terabytes of visual data every day. The volume is so enormous that human operators simply cannot review it all, let alone analyze it in detail. Neural networks have become indispensable for processing this imagery, using machine learning algorithms to streamline the task. AI has made it possible to analyze vast datasets in a fraction of the time, leading to the near-complete automation of many monitoring applications, including crop forecasting, weather analysis, and near-real-time object tracking.

Yet alongside the remarkable speed this technology has brought to satellite operations, a fundamental question has emerged: How much can we trust machine algorithms that are capable not only of identifying objects with remarkable accuracy, but also of hallucinating: “detecting” things that were never there in the first place?

Remote sensing and AI: from sensor data to actionable intelligence

The transformation of raw satellite data into actionable intelligence has advanced dramatically over the past five years. Until recently, the field relied heavily on labor-intensive manual interpretation of vast amounts of raw data at ground stations. Satellites continuously captured and transmitted terabytes of imagery, including countless pictures of clouds and empty stretches of ocean, which placed an enormous burden on communications bandwidth.

This bottleneck began to ease with the widespread adoption of computer vision systems in the satellite industry. The technology introduced a dynamic processing pipeline in which artificial intelligence acts as the first filter and interpreter, automatically discarding uninformative images and removing them from the queue for further analysis.

At the initial stage, AI takes over one of the most routine and computationally intensive tasks: orthorectification. This process corrects satellite imagery for geometric distortions caused by terrain and the effects of viewing angle, as well as radiometric distortions that affect image quality. Orthorectification was once a major bottleneck, requiring substantial computing resources at ground receiving stations. Thanks to modern neural network algorithms, however, the process has become largely automated while achieving exceptionally high accuracy, with image alignment reaching subpixel precision.

At the heart of modern computer vision are convolutional neural networks (CNNs). These are not merely algorithms but a specialized neural network architecture designed for deep analysis of visual data. Their operating principles closely resemble those of the human visual cortex, although they function at a far greater scale and speed. CNNs emulate the way neurons in the brain’s visual cortex respond only to stimuli within a limited region of the visual field.

Recognition takes place hierarchically. Step by step, the neural network extracts increasingly complex features from an image, beginning with simple edges and corners before progressing to intricate outlines of industrial facilities, vehicles, and other objects. Eventually, the network develops its own internal representation of what it is seeing. This entire process unfolds within the so-called hidden layers, where the model analyzes incoming imagery and classifies it based on the patterns it has learned.

A CNN’s ability to classify objects is not based on a static database. Instead, the recognition process continually evolves. In other words, CNNs are capable of learning from experience. With every new training iteration and every comparison it performs, the model refines its internal parameters, improving its ability to distinguish between different objects and steadily reducing the likelihood of classification errors.

The object recognition accuracy of modern CNN models such as YOLO and Faster R-CNN can exceed 95%, making them exceptionally powerful tools for automating monitoring tasks, including satellite image analysis. But for these algorithms to do more than simply operate aboard satellites – that is, to perform efficiently in the demanding environment of space — another crucial challenge must first be addressed.

At the cutting edge: integrating Edge AI systems onto satellites

Image recognition using CNN models requires substantial computing power and energy. That is why queries submitted to familiar AI assistants such as ChatGPT or Gemini are processed not on your own device, but in the powerful data centers operated by the companies providing those services.

For satellites, however, this approach is impractical. If every image had to be transmitted to Earth before it could be analyzed, the ability to make decisions in real time would be lost. At the same time, the satellite would waste valuable communication bandwidth by transmitting vast amounts of uninformative data.

This has led to the conclusion that, for space missions, especially those operating in deep space, it is essential to place the computing capability directly aboard the spacecraft. These ideas are embodied in the concept of Edge AI, a technology that moves data processing and decision-making from ground-based computers to the satellite itself.

A recent example of Edge AI adapted specifically for an Earth observation satellite is the European Space Agency’s PhiSat-2 (Φsat-2) mission.

The tiny 6U CubeSat, which is roughly the size of a shoebox, reached orbit on August 16, 2024. Its mission was far more ambitious than simply photographing Earth. Instead, the European spacecraft was designed to learn how to understand what it was seeing. Only after analyzing and interpreting the imagery onboard would it transmit the results to a ground station.

The first results exceeded expectations. Following a successful calibration phase, PhiSat-2 began detecting wildfires in real time, classifying ships at sea, and identifying oil spills. This approach dramatically reduced the burden on communication channels, as the onboard AI automatically filters out cloud-covered images and sends researchers only useful, analysis-ready data.

The European CubeSat mission demonstrated just how effective Edge AI can be aboard Earth observation satellites. Around 70% of optical images captured from orbit are unusable, typically because of cloud cover. Until recently, these worthless images consumed valuable communication bandwidth, accounting for as much as half of a satellite’s data transmission capacity. Today, the onboard Cloud Detection application instantly identifies cloud-covered scenes and instructs the satellite to transmit only images of clear areas.

This has dramatically reduced latency, the time between capturing an image and delivering actionable information to the end user. In the traditional workflow, where satellite imagery is processed only after reaching Earth, this cycle typically took between 30 and 90 minutes. Satellites equipped with onboard AI, by contrast, can independently identify emergencies such as wildfires and transmit their precise coordinates to emergency services within minutes. This speed has transformed satellite reconnaissance into a genuine real-time response tool, proving valuable in both civilian and military applications.

Through clouds and camouflage: radar intelligence in modern SAR systems

One of the most significant breakthroughs has been the application of AI to Earth observation satellites that rely not on optical imaging, but on Synthetic Aperture Radar (SAR). This approach quickly attracted the attention of the defense sector, where conventional optical imagery often becomes ineffective due to adverse weather or an adversary’s camouflage and concealment measures. SAR imagery, by contrast, relies on a fundamentally different principle: instead of recording visible light, it captures reflected microwave pulses and analyzes their amplitude and phase.

The resolution of a radar image normally depends on the length of the antenna. Synthetic aperture radar overcomes this limitation by exploiting the satellite’s motion along its orbit. In effect, the spacecraft creates a virtual antenna whose length corresponds to the distance it travels while collecting radar echoes. By combining observations over time, SAR systems can generate not only highly detailed maps but also three-dimensional models of the terrain. This is possible either through a constellation of SAR satellites observing the same area simultaneously or by having a single satellite image the same region multiple times from different viewing angles.

Interpreting radar imagery of this kind can be extremely challenging. To the human eye, a raw SAR image appears as a chaotic collection of bright spots and grainy interference known as speckle noise. This is precisely where AI excels, thanks to its ability to rapidly process and interpret vast amounts of highly repetitive data.

Modern neural network classifiers have even learned to distinguish the unique radar signatures of different objects. Automatic Target Recognition (ATR) technology enables these algorithms to do far more than simply detect metallic structures. They can accurately identify classes of naval vessels, distinguish different types of aircraft on airfields, and even detect mobile missile launchers that an adversary has attempted to conceal beneath radar-transparent camouflage nets or tree canopies.

The next step in the military application of AI has been Coherent Change Detection (CCD), in which algorithms analyze extremely subtle phase shifts between two consecutive satellite passes over the same location. If heavy vehicles have crossed a particular area or even microscopic ground disturbances like mine emplacement or the preparation of defensive positions have occurred, the phase pattern of the reflected radar signal changes, even though the landscape may appear completely unchanged to the naked eye. Neural network models process these enormous interferometric datasets in near real time.

High-resolution SAR imagery, including data from commercial providers, is now being actively integrated into military AI analytics platforms such as Palantir Foundry. This allows operators to receive automatically generated heat maps of enemy activity without relying on teams of human analysts.

Another breakthrough at the intersection of civilian and defense applications has been the emergence of multispectral, cross-sensor analysis services. In these systems, a neural network acts as an intelligent coordinator, seamlessly combining SAR imagery, optical satellite data, and signals intelligence (SIGINT).

In practice, this creates a fully automated intelligence pipeline. A satellite or airborne electronic surveillance platform first detects emissions from an air-defense radar or communications terminal. AI then automatically directs the nearest SAR satellite to image those coordinates through cloud cover. The resulting radar imagery is immediately overlaid with archived optical maps, allowing the system to verify the type of equipment present and assess its operational readiness.

This kind of multimodal data fusion is giving rise to a new generation of commercial services that extend far beyond military applications. Models such as the Remote-sensing Agricultural Assessments for Policy Impact and Decision-making (RAAPID) framework, developed as part of NASA Harvest, combine these diverse data streams for global crop monitoring, detecting subtle changes in soil conditions that cannot be identified through optical observations alone.

However, most modern ATR and CCD algorithms remain highly specialized, designed to perform only a limited set of tasks. The growing demand for a more universal, adaptable, and multifunctional onboard intelligence has led to the emergence of geospatial foundation models.

Foundation models: Prithvi for Earth and Surya for the Sun

The latest stage in the evolution of space-based AI is the emergence of geospatial foundation models: universal neural network platforms capable of solving a wide range of tasks without requiring extensive retraining for each new application.

One of the most prominent examples is Prithvi, a geospatial foundation model developed jointly by NASA and IBM. In 2023, Prithvi (whose name literally means “Earth” in Sanskrit) became the first AI model of its kind to be successfully deployed aboard the International Space Station. The model was installed in late March 2024 using Thales Alenia Space’s IMAGIN-e computing module. Later, in 2026, updated Prithvi hardware was also integrated into the Kanyini satellite, which operates in support of the Government of South Australia.

For many years, satellites functioned as little more than one-way data pipelines, transmitting vast streams of raw pixels to Earth, where they waited in long processing queues. Today, thanks to energy-efficient processors such as the Qualcomm Cloud AI 100, foundation models can serve as the onboard intelligence of an orbital spacecraft. This capability is already proving especially valuable for deep-space missions, where communication delays with Earth can become a critical limitation.

Prithvi was trained on the Harmonized Landsat Sentinel-2 (HLS) dataset, which contains terabytes of Earth observation data collected by NASA over the past 13 years. The model was developed under IBM’s 5+1 AI for Science initiative, bringing together machine learning specialists from IBM Research and Germany’s Jülich Supercomputing Centre (JSC).

Because the project is open source, researchers around the world can build upon the geospatial knowledge already embedded in the model rather than spending enormous resources training their own foundation models from scratch. Prithvi does not require complete retraining for every new application, enabling commercial organizations and government agencies alike to rapidly scale geospatial analytics across entire continents. This makes it possible to forecast a wide range of phenomena, including natural disasters and other emergencies.

When the onboard AI detects an anomaly resembling a natural disaster, it immediately generates an alert with precise geographic coordinates. During orbital testing, the model has already demonstrated its effectiveness in identifying flooded areas and detecting cloud cover in real time, while simply ignoring static portions of the landscape. This virtually eliminates the problem of data latency: in such an architecture, satellite constellations become a global nervous system for the planet, capable of responding to changes within seconds.

Beyond disaster monitoring, Prithvi has been trained to forecast crop yields and track land-use changes around the world. By learning from vast scientific datasets, the model has developed a deep understanding of Earth’s terrain. Its pattern recognition process resembles assembling an intricate puzzle and is based on a masked autoencoder architecture. During training, the neural network was repeatedly tasked with reconstructing deliberately hidden portions of satellite images, forcing Prithvi to develop a sophisticated understanding of context and the complex geometry of the Earth’s surface.

Because Prithvi is inherently multi-functional by design, it can be adapted to new missions almost instantly. Only minor fine-tuning is required for a satellite to switch from one application to another, for example, monitoring urban expansion, detecting wildfires, or tracking river flooding in real time. Geospatial foundation models also solve one of the biggest constraints of satellite operations: limited communication bandwidth. Rather than uploading an entirely new software package, operators need only transmit a small decoder package that adapts the existing model to a specific task.

Neural networks are increasingly being used not only to observe Earth but also to predict the behavior of other objects throughout the Solar System. In 2025, an open-source heliophysics model called Surya (the Sanskrit word for “Sun”) was developed using Prithvi as its foundation. Trained on nine years of observations from NASA’s Solar Dynamics Observatory (SDO), the model learned to forecast solar activity by analyzing subtle changes in the Sun’s photosphere. By detecting even the slightest fluctuations on the solar surface, Surya can generate predictions about solar storms and other space weather events.

This approach is fundamentally reshaping the architecture of orbital monitoring. For commercial satellite operators — particularly those managing large constellations — models like these make it possible to deploy planet-scale satellite analytics without the need for a massive expansion of ground infrastructure. But for this transition to succeed and become truly widespread, one major challenge must first be addressed: the persistent shortcomings of neural networks, chief among them the problem of AI hallucinations.

Digital hallucinations and methods of combating them

As we trace the evolution of neural networks for satellite monitoring, we arrive at one of the technology’s greatest challenges. The problem lies in the unique ability of AI algorithms to distort or even fabricate data. In the context of Earth observation, this can manifest as visual hallucinations or incorrect interpretations of perfectly ordinary imagery: in other words, a “crisis of geospatial truth.”

One telling example comes from Sweden, where an automated wildfire detection system based on data from the VIIRS satellite was found to produce false alarms in roughly 10% of cases. The recognition algorithm mistakenly interpreted sunlight reflected from large solar power plants and certain industrial facilities as potential wildfire hotspots. Although the system was not a fully generative AI model, the incident illustrates a fundamental weakness of automated satellite image analysis: algorithms can misclassify real-world objects and phenomena, generating misleading alerts.

Because neural networks build their own internal relationships between patterns, they are inherently susceptible to a particular class of errors known as hallucinations. During image interpretation, a model may effectively “imagine” details that appear plausible but do not actually exist. Such hallucinations most commonly occur when an algorithm attempts to recover detail from extremely noisy or low-quality imagery or artificially enhance its resolution. In its pursuit of a cleaner image, the AI fills missing information with patterns learned during training, even when those patterns bear no resemblance to the actual scene. This creates a dangerous trap: instead of objectively recording reality, the model substitutes familiar templates into the gaps, producing a convincing but entirely artificial representation of the world.

The technical origins of these errors often lie in the use of Generative Adversarial Networks (GANs). While GAN-based models are exceptionally effective at enhancing image quality, they have no inherent understanding of physical truth or factual correctness. During processing, an AI system might mistake cloud shadows for building foundations or “see” the outlines of submarines within the texture of ocean waves. Such distortions become especially dangerous when satellite imagery is used to support strategic decisions at the national level, where even a single convincing hallucination can have serious real-world consequences.

Algorithmic errors of this kind could trigger large-scale instability. Imagine a scenario in which an AI-powered border surveillance system, attempting to compensate for poor-quality nighttime imagery, hallucinates the deployment of ballistic missile launchers inside a demilitarized zone. During a period of heightened political tension, such automated misinformation could prompt a preventive strike or military mobilization by major powers before human analysts had time to verify the original data. In such a case, what began as a technical error could become the catalyst for uncontrolled military escalation.

For the commercial sector, these AI failures carry direct financial consequences. Commodity traders increasingly rely on satellite imagery to estimate oil inventories and forecast crop yields. An AI error of just 2–3% could artificially influence financial markets. If an algorithm misjudges the fill level of oil storage tanks because it incorrectly interprets shadows in an image, investors could suffer losses worth millions of dollars.

The consequences are equally serious for environmental monitoring and disaster response. During floods or wildfires, rapid analysis is critical, yet AI systems may generate inaccurate heat maps or incorrectly delineate flooded areas because of spectral mixing and other imaging artifacts. As a result, emergency resources could be dispatched to locations where no disaster exists, while the areas that genuinely require assistance remain overlooked. Such distortions of reality threaten not only the effectiveness of humanitarian operations but also the lives of people caught in disaster zones.

Beyond accidental errors lies an even more serious threat: adversarial attacks. In these attacks, malicious actors introduce almost imperceptible visual perturbations into an image. While these alterations are effectively invisible to the human eye, they can cause an AI system to perceive nonexistent objects or fail to detect real ones altogether. Such attacks generally require compromising the digital processing pipeline, for example, by intercepting and manipulating data during transmission. Nevertheless, the prospect of creating genuine blind spots in national security surveillance systems raises profound concerns about the reliability of AI-assisted satellite intelligence.

To address the problem of machine hallucinations, the industry is increasingly turning to concepts such as Explainable AI (XAI), a collection of techniques designed to reveal the reasoning behind every decision an AI system makes. Rather than simply producing a result, XAI systems visualize the decision-making process itself, showing analysts exactly which pixels, image features, or patterns influenced the model’s conclusion, particularly when that conclusion is incorrect. This allows human operators to quickly determine whether a detection is based on meaningful evidence or merely the product of random sensor noise.

Nevertheless, no matter how advanced a neural network for satellite image analysis becomes, the most important safeguard in modern processing architectures remains the Human-in-the-Loop (HITL) concept. According to this approach, AI serves as a powerful filter, eliminating 99% of routine background information, while the final confirmation of any critical target or anomaly is always left to a qualified human analyst. Leading satellite operators such as Maxar and Planet have already adopted multi-layered verification protocols in which automation functions as an assistant to human experts rather than a replacement for them.

Another effective way to combat AI distortions is cross-validation using independent data sources. Comparing the AI’s interpretation of an optical image with observations from a Synthetic Aperture Radar (SAR) satellite allows analysts to verify whether an object is physically present. Because radar operates on entirely different physical principles, the probability that both systems would produce the same hallucination is extremely small. This kind of multimodal verification has become a standard requirement in many of NASA’s most advanced Earth observation and monitoring programs.

The growing realization of how easily digital systems can distort reality is already forcing the industry to fundamentally rethink its approach to information integrity. We have reached a point where the central challenge is identifying a clear boundary between objective fact and a false simulation invented by a machine-learning algorithm. This, in turn, highlights the need for new ethical and security standards governing the use of AI in space, particularly in military applications. The next major step in the evolution of satellite AI will likely be the development of “hallucination-immune” systems built around transparent, traceable decision-making processes.

The satellite industry has already moved beyond the era of simply collecting raw data and is now entering the age of autonomous computing in orbit. The key unresolved challenge is the creation of robust verification architectures capable of distinguishing genuine anomalies on Earth’s surface from errors generated by the neural networks themselves, all in real time. Solving this technological problem would transform satellite observation systems into trusted sources of verified geospatial intelligence, where every AI-generated conclusion is transparent, explainable, and demonstrably reliable.