Perspective Chapter: Downscaling of Satellite Soil Moisture Estimates

Abstract

Soil moisture is a key parameter in the hydrological cycle and plays a critical role in global climate. The capacity to forecast drought and floods, manage water resources, and make field-scale decisions depends on accurate and thorough information on soil moisture. In addition to the instrument-based field observation approaches, dynamic mapping of soil moisture has been made possible by satellite remote sensing technologies. Estimates of soil moisture at a global and regional scale from optical and thermal remote sensing have been explored, and considerable advancements have been made. However, these global soil moisture products have coarse spatial resolutions and are typically unsuitable for field-level hydrological and agricultural applications. In this regard, this chapter presents a comprehensive review of the latest downscaling methods to improve the coarse-spatial and temporal resolution of soil moisture products. The main approaches discussed in the chapter include active passive fusion, optical/thermal based, topography based, and data assimilation methods. The physical background, current status, advantages and limitations associated with each downscaling approach has been thoroughly examined. Each of these optical/thermal, microwave-based methods for soil moisture estimation involves intricate derivation at different spatiotemporal scales, which can be combined using recent advances in machine learning.

1. Introduction

Soil moisture (SM) governs the interactions between the land surface and atmospheric processes and is a critical component of energy cycles at both regional and global scales [1, 2]. It largely impacts and controls climate and weather feedback by influencing the partitioning of latent and sensible heat fluxes at the land surface [3, 4]. SM also controls the distribution of precipitation into infiltration and surface runoff, which regulates plant growth [5]. Soil moisture information is, therefore, central to various applications such as drought monitoring [6, 7], irrigation scheduling [8], crop yield prediction [9, 10], weather prediction [11], flood forecasting [12, 13], and forest fires [14]. Traditionally, in situ measurements (gravimetric method, time domain reflectometry, etc.) have been used for quantification of soil moisture data, but being labor intensive, these methods are unfavorable for large spatial and temporal scales. Installation of monitoring stations to create a dense network of continuous soil moisture observations covers a large area but it is costly and needs high maintenance. Both of these approaches provide sparse point measurements and remain insufficient to justify the heterogeneous nature of soil moisture on larger scales, though they are widely used for validation purposes.

Developments in satellite remote sensing (RS) provided global coverage for frequent monitoring of surface soil moisture (SSM) that is not attainable by conventional techniques. In particular, microwave RS has been widely used to retrieve soil moisture by capturing the difference in the dielectric constant of dry versus wet soil and hence changes in the backscatter (active) or brightness temperatures (passive) [15]. For SM studies, frequencies below 6 GHz of the microwave region in the electromagnetic spectrum are found most sensitive and have been widely used in satellite missions [16]. Passive microwave RS-based radiometers are efficient in SSM retrieval, especially in the L-band (1.4 GHz) due to their better penetration ability through vegetation canopies and less sensitivity toward surface roughness [17, 18]. Based on this, various radiometer-based SM products are available with global coverage from 1) Tropical Rainfall Measuring Mission Microwave Imager (TMI), 2) Metop-A Advanced Scatterometer (ASCAT), 3) Advanced Microwave Scanning Radiometer-Earth Observing System (AMSR-E), and 4) Advanced Microwave Scanning Radiometer-2 (AMSR2) in C & X-band. European Space Agency’s (ESA) Soil Moisture and Ocean Salinity (SMOS) satellite launched in 2009 was the first L-band passive microwave satellite that offers global SM. However, the spatial resolutions of these RS-based SSM estimates are coarse (25–40 km) and not suitable for field-level implementation [19, 20].

Active microwave sensors offer high spatial resolution but are strongly sensitive to surface roughness and vegetation that affects their retrieval accuracy. For example, ASCAT generates global SSM at a 25 km resolution using backscatter measurements from transmitted linear frequency-modulated pulses in C-band [21]. NASA launched soil moisture active/passive (SMAP) mission in 2015, with a passive microwave sensor and a radar component providing SM at a resolution of ∼36 km, mapping the globe every 2–3 days. However, the radar component failed after a few months and the mission continued with its mechanism of radio frequency interference (RFI) mitigation to provide high-quality data [22]. More recently, NASA combined Sentinel-1 C-band backscatter data with SMAP L-band radiometer data to create SSM maps with 3-km and 1-km resolutions [23]. A recent publication also included the combined high-resolution ASCAT/Sentinel-1 (1 km) SSM product [24].

The direct applications of these global SM datasets are further hindered by region-wise high spatiotemporal variability in SSM data due to variations in topography, soil texture, precipitation, and land cover changes [25, 26]. Therefore, improvement in the spatial resolution of SM data is essential to bridge the gap between space-borne SSM estimates and field-based requirements (e.g., hydrology and agriculture). Downscaling approaches are studied for obtaining required pixel information on SSM utilizing the high-accuracy SM products available from dedicated satellite missions (Figure 1). The fusion of radar and radiometer data has been exploited [28, 29] and dedicated satellite missions such as SMAP were launched that combined active and passive microwave measurements at L-band [30]. Other general approaches for downscaling SM to appropriate resolutions are by using optical/thermal data [31, 32], topographical data, and soil properties [33, 34] that might be based on physical or statistical method. Recently, data assimilation techniques are also being used for downscaling SM data [35].

Figure 1.

SMAP level 3 radiometer soil moisture (m³/m³) retrievals for, 9 km and 36 km (for visualization of details at different resolutions only). Source [27].

In this chapter, we have discussed some pertinent downscaling approaches for obtaining high-resolution SSM estimates using the latest research studies. These include merging optical/thermal, active-passive microwave data, topographical information, and vegetation data with microwave SSM products using machine learning algorithms and land surface models [32, 36, 37]. The physical processes involved, their limitations, and the advantages of these methods have been systematically collated and compiled in the chapter. Unlike previous reviews, the chapter contains an updated in-depth analysis of recent advances in machine learning techniques and their comparisons for SSM estimation at improved spatiotemporal resolution. Finally, some recommendations have been proposed based on the overall assessment of the methods discussed in the chapter.

2. Downscaling approaches

2.1 Combining active and passive microwave data

The fundamental approach in integrating active and passive microwave sensors is to combine high spatial resolution obtained by active sensors and good temporal resolution from passive sensors. NASA launched SMAP with the view of merging the advantages of radar and radiometer to obtain a medium resolution for SSM products. The baseline algorithm for SMAP SSM data was based on a change detection approach [38], which showed a 2% improvement in root mean square (RMSE). The latter showed that the synergy of active-passive algorithms provided more spatial details. Das et al. [30] improved this by eliminating the need for previous satellite passes and obtaining absolute SM as compared to relative SM. Further investigations showed that disaggregation of radiometer brightness temperature (TB) using synthetic aperture radar (SAR) backscatter gave better results with certain assumptions such as 1) vegetation water content is limited to 5 kg/m², 2) surface roughness is invariable for short durations, and 3) near-linear relationship between backscatter and brightness temperatures. The baseline algorithm of SMAP was tested for different hydro-climatic regions and it was concluded that the algorithm is well suited for heterogeneous areas with the error of 2.4 K when TB is downscaled to 9-km resolution [39].

However, the failure of SMAPs radar sensor led to the use of downscaling algorithms for improved spatial resolution of SM products. Recent studies have integrated the passive radiometer observations from SMAP or SMOS with Sentinel-1 C-band SAR data to obtain SM estimates at a regional/local scale. Due to its similar orbit configuration with SMAP and availability of both co- and cross-polarizations, it showed the potential to give promising results in a few studies [40, 41]. He et al. [42] attempted to test Sentinel-1 and SMAP data with different downscaling algorithms in the southeastern region of Australia and found that the SM-based downscaling algorithm had lower RMSE (0.056 cm³/cm³) at 9-km resolution. However, further lowering the resolution at 1 km increased the RMSE to 0.092 cm³/cm³. The three different algorithms that are commonly used for active-passive downscaling include 1) brightness temperature-based downscaling algorithm, 2) soil moisture-based downscaling algorithm, and 3) change detection.

2.1.1 Brightness temperature-based downscaling algorithm

In this approach, the TB provided by the passive radiometers with the coarse resolution is disaggregated to high resolution using radar backscatter. Consequently, the downscaled brightness temperature is converted to SM with finer resolution (∼ 3–9 km) using a near-linear relationship between TB and backscatter [43, 44]. The method can be described as

$${\mathit{TB}}_p\left(C\right)={\alpha }_1\left(C\right)+{\beta }_1\left(C\right)\ast \sigma 0_{\mathit{pp}}\left(C\right)$$E1

$${\mathit{TB}}_p\left(F_j\right)={\alpha }_1\left(F_j\right)+{\beta }_1\left(F_j\right)\ast \sigma 0_{\mathit{pp}}\left(F_j\right)$$E2

where C represents coarse scale, F represents fine-scale resolution, ${\alpha }_1$ and ${\beta }_1$ are the intercept and slope, respectively, p is the polarization (h- or v-polarization) and pp. represents co-polarization of radar observations σ0 (hh or vv), ${\mathit{TB}}_p\left(F_j\right)$ is the brightness temperature value of pixel j of resolution F, and $\sigma 0_{\mathit{pp}}\left(F_j\right)$ is the corresponding radar backscatter value of pixel j. The radar backscatter value $\sigma 0_{\mathit{pp}}\left(C\right)$ at coarse scale can be obtained by aggregating the pixel of high-resolution radar data to that of resolution F.

When compared to radar backscatter inversions, the inversion of brightness temperature to estimate SSM is more efficient. However, this approach has some inconsistencies such as added instrument errors from radiometer and radar and water body-based uncertainties in radiometer data [45].

2.1.2 Soil moisture-based downscaling algorithm

Das et al. [30] tested an approach with SMAP mission data in which estimated SM data at a coarser resolution are integrated with active microwave data to produce a high-resolution optimal SM (9 km). Almost linear relationship between radar backscatter and volumetric SM is used in this method instead of TB assuming that vegetation and surface roughness has a constant impact. The method can be applied as:

$${\mathit{ϴF}}_j=ϴ\left(C\right)+{\beta }_2\left(C\right)\ast \left\{\left[{\sigma }_{\mathit{pp}}^0\left(F_j\right)-{\sigma }_{\mathit{pp}}^0\left(C\right)\right]\ast \Gamma \ast \left[{\sigma }_{\mathit{pq}}^0\left(C\right)-{\sigma }_{\mathit{pq}}^0{F}_j\right]\right\}$$E3

where ${\mathit{ϴF}}_j$ is the downscaled SM of pixel j of fine-resolution F, θ(C) is the SM derived from the radiometer at a coarser resolution C, ${\beta }_2\left(C\right)$ (cm³/cm³/dB) is obtained through linear regression of the time series of θ(C) and ${\sigma }_{\mathit{pp}}^0\left(C\right)$, and $\Gamma$ is a sensitivity parameter for each particular grid cell C and season, which can be estimated using high-resolution ${\sigma }_{\mathit{pp}}^0\left(F\right)$ and ${\sigma }_{\mathit{pq}}^0\left(F\right)$ (measurements)

$$\Gamma ={\left[\delta {\sigma }_{\mathit{pp}}^0\left(F_j\right)/\delta {\sigma }_{\mathit{pq}}^0\left(F_j\right)\right]}_C$$E4

Here, pp and pq represent co and cross-polarizations, respectively. The fundamental benefit of this method is that it does not require high-resolution ancillary data, which is typically difficult to get and is needed for SM inversions, such as land surface temperature (LST) and normalized difference vegetation index (NDVI).

2.1.3 Change detection method

The approach suggested by Piles et al. [38] is based on the nearly linear relationship between the temporal changes in SM and radar backscatter within the radiometer footprint. The method improved the spatial resolution of radiometer-based SM products to 10 km. It quantifies the changes in the preceding radiometer-derived SM as a function of the moisture change observed in the higher-resolution radar backscatter and is illustrated as:

$$ϴ\left(F_j,t\right)=ϴ\left(C,t-t_R\right)+{\beta }_3\left(C\right)\ast [{\sigma }_{\mathit{pp}}^0\left(F_j,t\right)-{\sigma }_{\mathit{pp}}^0\left(F_j,t-t_R\right)$$E5

where $ϴ\left(F_j,t\right)$ is the soil moisture of a pixel j of resolution F acquired at time t, $ϴ\left(C,t-t_R\right)$ is the SM of coarse-resolution C acquired at time $t-t_R$, $t_R$ is the revisit time of the observations, and ${\beta }_3\left(C\right)$ (cm³/cm³/dB) represents the sensitivity of volumetric SM $(ϴ$) to radar backscatter ${\sigma }_0$, obtained through the time series of $ϴ$ (C) and ${\sigma }_{\mathit{pp}}^0$ (C).

The main advantage of this method is that it maintains the accuracy of the radiometer, meanwhile capturing more structural information provided by the radar at the radiometer scale [38].

2.1.4 Active- and passive-derived SM fusion

This technique uses radar SM residuals to disaggregate the radiometer-based SM product using the following equation [46]:

$${ϴ}_{{ϴ}_{\mathit{Tb},}{ϴ}_{\sigma }}\left(M_i\right)={ϴ}_{\mathit{Tb}}\left(C\right)+{\beta }_C\left({ϴ}_{\sigma }\left(M_i\right)-\stackrel{-}{{ϴ}_{\sigma }\left(M_{1:16}\right)}\right)$$E6

where ${ϴ}_{{ϴ}_{\mathit{Tb},}{ϴ}_{\sigma }}\left(M_i\right)$ is the fusion result of radar and radiometer-based SM products, ${ϴ}_{\mathit{Tb}}$ is the radiometer-based SM product; ${ϴ}_{\sigma }$ is the radar soil moisture product; ${\beta }_C$ is the statistical correlation (slope) parameters; and i is the M scale pixel index ranging from 1 to 16 within one C scale pixel.

2.2 Optical−/thermal-based downscaling

Many researchers have focused on employing optical approaches for downscaling since variations in LST and vegetation at the surface are well associated with changes in SSM. Optical−/thermal-based methods provide high-resolution information that can be merged with radiometer-based products [47, 48, 49]. The universal triangle feature space concept developed by Toby N Carlson et al. [50] relates surface temperature to vegetation indices to provide information on dry-wet edge and evaporation-transpiration through the vertex of the triangle. X Zhan et al. [51] and Chauhan et al. [52] further proposed an empirical polynomial fitting downscaling method based on this approach. According to this, vegetation indices and LST, which are all generated from optical and thermal data, are polynomial functions of high-resolution SM, expressed as:

$$\mathit{SM}=\sum _{i=0}^{i=n}\sum _{j=0}^{j=n}{a}_{\mathit{ij}}{\mathit{\text{NDVI}}}^{\ast \left(i\right)}{T}^{\ast j}$$E7

where T is the normalized surface temperature; $a_{\mathit{ij}}$ is the regression coefficient; and the superscripts i and j imply the degree of the polynomials selected for the regression. This polynomial fitting-based approach has been explored widely to downscale global SM datasets from SMOS and AMSR-E using moderate-resolution imaging spectroradiometer (MODIS) data [53, 54].

With the availability of high-resolution optical/thermal-based information on vegetation cover, surface temperature, and different surface parameters, a body of research has tried to downscale coarse-resolution microwave SM products using this approach (Figure 2). The central idea is to find a downscaling factor from high-resolution optical/thermal data and use it to improve the spatial variability of the coarse-resolution microwave SM products. The optical−/thermal-based methods have been widely used for downscaling; however, their inherent limitations under cloudy weather limit their applications to different agro-climatic zones.

Figure 2.

Schematic general steps followed for spatial downscaling of coarse-resolution SM products obtained using optical/thermal based approach. Source [55].

Disaggregation based on physical and theoretical scale change (DISPATCH) is another method suggested by Merlin et al. [56, 57] that disaggregates coarse SM based on evaporation and universal trapezoid. This method was developed over years using soil evaporative efficiency (SEE), defined as the ratio of actual versus potential evaporation, and evaporative fraction as SM proxies due to their consistency during daytime and being directly proportional to SSM changes. Malbéteau et al. [58], in a study, showed that the DISPATCH algorithm performed best for semi-arid regions, increasing the correlation coefficient to 0.63 from 0.37. When used with the land surface technique, DISPATCH can also offer SM estimations even in cloudy conditions with high accuracy. Djamai et al. [59] used such combination with DISPATCH and CLASS (Canadian Land Surface Scheme) model simulations and produced a continuous time series of SM at 1-km resolution with 0.07m³/m³ RMSE. More recently, MODIS-derived LST and NDVI data were interpolated and geographically weighted regression (GWR) method was implemented to downscale AMSR-2 products at 1-km spatial resolution [60]. The method can potentially generate better and high spatial resolutions if observations with greater accuracy are used, even on cloudy days.

2.3 Topography-based downscaling algorithms

There is a high spatial and temporal variability in SSM information with changes in topography and soil properties [61]. Considering this, topographic information has been accounted often for downscaling moisture estimates [62, 63]. SM exhibits at least two pattern types related to topography. The first form of pattern has regions that are wetter in the valley bottoms and is structured similar to the catchment drainage pattern, whereas the second pattern has an arrangement that is tied to the orientations of the hillslopes, with wetter places appearing on the slopes that are more shadowed from the sun.

Temimi et al. [62] proposed a topography-based wetness index developed using MODIS Leaf Area Index (LAI) and topographic attributes derived from the SRTM digital elevation model (DEM) to effectively downscale AMSR-E-based SM. This topographic wetness index distinguishes inundated areas with responses from wet soils in the passive microwave signal (Figure 3).

Figure 3.

Flowchart of the topography-based soil moisture downscaling algorithm. BWI is basin wetness index and TWI is terrain-based wetness index. Image source [62].

A statistical method such as empirical orthogonal function (EOF) with topographic attributes was also proposed for downscaling of coarse-resolution SM products [64, 65]. The method analyzes large multidimensional datasets and is frequently used in meteorology. The approach enables an analysis of both the spatial and temporal anomalies of SM using a set of EOFs, which are invariant in time, and a set of time series called expansion coefficients ECs, which are invariant in space. The average SM for a given observation time is subtracted from all data collected at that time to derive spatial anomalies given as:

$$z_i\left(t\right)=s_i\left(t\right)-\frac{1}{m}\sum _{j=1}^m{s}_j\left(t\right)$$E8

where $z_i\left(t\right)$ is the spatial anomaly; $s_i\left(t\right)$ is the SM observation at location i and time t; j is an index of observation locations; and m is the number of observation locations. On the other hand, temporal anomaly$z_i^{,}\left(t\right)$ at location i and time t can be computed as:

$$z_i^{,}\left(t\right)=s_i\left(t\right)-\frac{1}{n}\sum _{t=1}^n{s}_i\left(\tau \right)$$E9

s is an index of sample dates. This approach depends on site-specific parameters and hence effective only at catchments it is calibrated for. Additionally, observations must be available at the same locations during each sample period.

Alternatively, equilibrium moisture from topography (EMT) model was developed with the advantage of using fewer variables as compared to EOF [66]. Conceptual descriptions of the vadose zone processes [67] are used in this model to determine the interactions with the topographic indices. It performs better than EOF albeit availability of sparse SM observations for calibration [33]. This was further improved by Ranney et al. [34] with the inclusion of high-resolution soil and vegetation properties with topography and named as equilibrium moisture from topography, vegetation, and soil (EMT + VS). The major improvements in the model included enhanced representation of the primary roles of vegetation, transpiration, and soil evaporation. The structure of the model has also been altered, to support fine-scale variation in soil and vegetation attributes.

It was anticipated that spatial patterns of SSM are also influenced by potential evapotranspiration (PET) and precipitation variations. Hence, these factors were explored for their influence on SM using EMT + VS and it was concluded that they improve the downscaling of PET better than precipitation-based downscaling (Figure 4) [68].

Figure 4.

Comparisons of observed and SM obtained using from topography, vegetation, and soil (EMT + VS) model when: (a) the preexisting EMT + VS model is used, (b) precipitation downscaling is included, (c) PET downscaling is included. The Nash-Sutcliffe coefficient of efficiency (NSCEs) shown are the space–time values [68].

2.4 Data assimilation approaches

In such methods, coarse SSM observations or in situ observations are used with fine-scale models that produce a downscaled product [35, 69, 70]. Integration of data assimilation techniques with process models can expedite the reallocation of SM by compiling timely updates on surface SM information [35]. In general, two types of approaches are mainly followed under assimilation methods: 1) Remotely sensed data are initially reduced to the resolution of the land surface model, before application of further downscaling, and 2) dynamic downscaling in which coarse observations are assimilated with a model to update fine-scale model states. Hence, data assimilation works in both ways in which downscale products can be produced as well as in turn such products increase the efficiency of land surface models [60].

Assimilation techniques such as ensemble Kalman filters are being increasingly used in forecasting SM information at a higher spatial resolution [69, 71, 72]. The technique was first shown to be flexible and robust for SM downscaling even at moderate ensemble sizes by merging L-band (1.4 GHz) microwave radio-brightness observations with land surface model estimates [73]. Sahoo et al. [35] generated SM product at 1 km by assimilating the AMSR-E SM data (25 km) into NOAH land surface model using a three-dimensional Ensemble Kalman filter (3-D EnKF) and a one-dimensional EnKF (1-D EnKF). The equation for a single ensemble member “j” at a single fine-scale location “k” at one-time step can be given as:

$${\widehat{x}}_j^{k+}={\widehat{x}}_j^{k-}+K^k\left[y_j-{\widehat{y}}_j^{-}\right]$$E10

where the Kalman gain is defined as

$$k^k=\mathit{Cov}\left[{\widehat{x}}^{k-},{\widehat{y}}^{-}\right]{\left[\mathit{Cov}\left[{\widehat{y}}^{-},{\widehat{y}}^{-}\right]+R\right]}^{-1}$$E11

In Eq. (10), ${\widehat{x}}_j^{k-}$ is the a priori state estimate (forecast), whereas ${\widehat{x}}_j^{k+}$ is the a posteriori state estimate (analysis). The term ${\widehat{x}}_j^{k-}$ is a part of the total state vector ${\widehat{x}}_j^{-}$ and contains the state variables at a single grid cell k. The term ${\widehat{x}}^{k-}$ in Eq. (11) refers to the ensemble of forecasts ${\widehat{x}}_j^{k-}$ and R denotes the observation error covariance. In the given study, the state vector x contains eight state variables (four SM variables and four soil temperature variables for the four soil layers). The model state variables are updated using Eq. (10) at the time step when the observations $y_j$ are available, after contrasting them to the observation prediction ${\widehat{y}}_j^{-}$. The latter is calculated as the spatial mean of all the 1 km fine-scale SSM values within any single 25 km coarse-grid cell. First term in Eq. (11) includes the error correlation between the ensemble of forecasted SM over different coarse-scale areas (${\widehat{y}}^{-}$; size depends on the localization radius) and the ensemble of the eight fine-scale state variables at a single fine-scale model grid cell ${\widehat{x}}^{k-}$. This correlation information is the basis for updating the eight state variables in response to coarse-surface layer SM observations $y_j$. The forecasts were improved by adjusting a bias that was computed by estimating the spatial and temporal mean difference between the model forecasts and in situ observations. Both 1-D and 3-D EnKF methods enhance SM predictions with 3-D EnKF being better at handling spatial coherence [35].

The inclusion of satellite-based SSM estimates improves the performance of process models. Rouf et al. [69] in a study explored the efficiency of different SMAP satellite products in data assimilation (DA) using NOAH-MP land surface model and an Ensemble Kalman filter for SM downscaling. They tested six different frameworks in which two sets of atmospheric forcing North America Land Data Assimilation System (NLDAS-2) at 12.5 km and 500 m (downscaled) were assimilated with two sets of SMAP SM products (SMAP-36 km and SMAP-9 km). The study demonstrated that the RMSE of the open loop (NLDAS-OL) simulation (without SMAP SM data) was largely reduced when the downscaled atmospheric variables were assimilated with SMAP SM data (Figure 5).

Figure 5.

Maps of RMSE between surface soil moisture standard normal deviates observed at each Mesonet station and simulated in each of the following experiments: (a) NLDAS-OL, (b) NLDAS-DA-36 km, (c) NLDAS-DA-9 km, (d) downscaled-OL, (e) downscaled-DA-36 km, (f) downscaled-DA-9 km. Image source [69].

Recently, Bayes’ approach has also been applied as an assimilation technique for SM downscaling. Vergopolan et al. [70] used this approach to merge the brightness temperature from a radiative transfer model (HydroBlocks-RTM) and SMAP L3 product to retrieve updated SM information at 30-m resolution. Bayes provides a Gaussian-based merging and assimilation technique that can disintegrate the nonlinear relationship between SSM and TB from the assimilation method [70]. The three-step process included 1) coupling process models-HydroBlocks and the Tau-Omega RTM to predict fine-scale TB (30 m); 2) merging these TB estimates with the 36 km coarse-scale SMAP TB observations using the Bayes rule; and 3) inversion of RTM to retrieve the downscaled SSM at 30 m (Figure 6).

Figure 6.

Overall flowchart of the merging framework. Image source [70].

The assimilation methods offer a simulation of land surface processes and interaction of SM with elevation, aspect, soil properties, vegetation, subsurface water dynamics, and climate at fine scales. This results in an enhanced representation of the water and energy balances in these methods [74]. These physical interactions are generally ignored in machine learning and statistical downscaling approaches [75]. However, the merging approach has certain limitations such as the process being sensitive to biases in the model and satellite estimates. One of the main sources of uncertainty is known to be meteorological inputs, particularly precipitation [76]. Similarly, previous SMAP- and SMOS-based studies have also identified the impact of the forecast bias between the model and satellite observation on the merged SSM [77, 78].

3. Recent developments

Advancements in machine learning (ML) have increased their potential in downscaling SSM, which is generally a complicated process due to the nonlinearity and is treated using surface information. The potential of some state-of-art ML algorithms-artificial neural network (ANN) [79, 80, 81, 82], support vector machine (SVM) [80, 81, 82], Bayesian (BAYE) [70, 83, 84], classification and regression trees (CART) [82, 85], K nearest neighbor (KNN) [85], and random forest (RF) [80, 82, 86] in SM downscaling has been demonstrated in recent research works.

In particular, RF-based models have been used extensively in developing the complex relationships between different land surface variables and SSM [87, 88, 89]. Zhao et al. [90] studied the random forest approach to capture the nonlinear relationship between SSM and surface variables at the Iberian Peninsula that was earlier assumed as linear in most of the studies. The dataset included SMAP L3 data, ancillary data acquired from MODIS- LST, NDVI, LAI, Enhanced Vegetation Index (EVI), NDWI (Normalized Difference Water Index), surface albedo, and SRTM DEM at 30 arcsec for elevation, slope, and aspect information. The study tested four different combinations of Terra and Aqua MODIS LST products and SMAP ascending and descending overpasses, having similar statistical analysis. Spatiotemporally continuous SM (1 km) is generated using RF models to downscale ESA-CCI and China Meteorological Administration Land Data Assimilation System (CLDAS) SSM products [88]. Similarly, Abbaszadeh et al. [55] downscaled SMAP L3 SSM to 1 km across the contiguous United States using high-resolution land surface variables (i.e., NDVI, LST, precipitation, elevation, and soil texture) that substantially represent spatial variability in SSM. Bai et al. [91] performed a similar study in addition to testing the importance of input variables and using Sentinel-1 data through RF models. The study made a comparative assessment of five different RF models considering the combination of SAR backscatter in VV polarization and topographical information.

However, there is no universal method to downscale SM with the highest accuracy as the performance of each method differs with environmental and land cover conditions. But despite the significance of ML-based SM downscaling, there are still only a few inter-comparison studies available. One such attempt was undertaken by Sabaghy et al. [92], who conducted a thorough comparative examination of downscaling methods based on radar, optical, radiometer, and oversampling over the Yanco region in Australia. The study used data from SMAPEx-4 and -5 airborne experimental campaigns for Cal/Val of SMAP-derived SSM products and in situ observation from a network of rainfall monitoring and SM measuring stations. The radar-based technique comprised of SMAP baseline active/passive method and multi-objective evolutionary algorithm (MOEA). Both of these techniques downscaled 36 km of passive microwave data to 9 km. For optical-based downscaling, DISPATCH and vegetation temperature condition index (VTCI) have been used to produce 1-km spatial resolution of SSM from SMOS level 3 products. The oversampling-based Backus–Gilbert interpolation was applied to enhance spatial resolution with better accuracy to produce a precise optimal observation value of TB similar to that of radiometer measurements. The accuracy of the method depends upon the sampling density. Though optical-based VTCI method showed the best temporal agreement with monitoring networks as compared to other algorithms, it suffered from the lack of data due to cloudy conditions. Overall, oversampling-based downscaled SSM products were able to capture the spatial heterogeneity and temporal pattern and provided the best accuracy. This study also concluded that validation of the SSM estimates should not be made solely based on in situ measurements as they cannot cover spatial variability being sparsely distributed.

Another study carried out by Lie et al. [85] implemented six machine learning algorithms: ANN, BAYE, CART, KNN, RF, and SVM to establish the spatial downscaling models with reliable continuous in situ SSM observations over four case study areas. These areas represented different climatic zones: 1) Oklahoma Mesonet (OKM) in North America, 2) Naqu network (NAN) in the Tibetan Plateau, 3) REMEDHUS (REM) network in northeast Spain, and 4) OZNNET (OZN) in southeast Australia. The study used LST, NDVI, albedo, DEM, and geographic coordinates as the explanatory variables. The results concluded that 1) RF performed best with high correlation coefficient and a low regression error, 2) although BAYE and KNN both showed promising skills for SM downscaling, the robustness of their algorithms still needed improvement, and 3) the ANN, CART, and SVM techniques all produced a series of anomalous values throughout the process indicating their limitations in SM downscaling.

Deep learning (DL) has been a recent advancement in terms of downscaling algorithms and has proved its potential over traditional methods having shallow architectures. It is capable of handling deeper and higher-level features for larger datasets as compared to RF and Gaussian Process Regression (GPR) methods. Deep neural networks are a step forward in ML research area with increased complexity and a greater number of hidden layers. Though some studies [93, 94] have employed deep learning to estimate SM, still its application is limited in the field of downscaling SM. Deep learning models were compared with RF and back propagation neural network (BPNN), and were applied at Tibetan Plateau with complex terrain and downscaled SMAP SM product from 36-km to 1-km resolution [95]. The novel models are deep belief network (DBN), neighborhood constrained-based DBN (NC-DBN), and residual network (ResNet). DBN has a good ability of nonlinear fitting between SM and auxiliary data such as NDVI, LST, albedo. But it is incapable of addressing the issues of effect from neighborhood surface data for each pixel in SM products. NC-DBN was further introduced to incorporate these effects, and further, ResNet provided the information about spatial integrity and enhanced the information. They found that ResNet model showed better performance in terms of accuracy and correlation. Another DL approach (wide and deep learning) combining deep neural networks and generalized linear model [96] was applied over Continental United States (CONUS) and validated with soil moisture networks dataset at 211 sites. It concluded that downscaled SM product was detailed in terms of spatial pattern and increased accuracy was achieved using climatic and land cover patterns in the downscaling scheme.

4. High-resolution SSM mapping through downscaling of LANDSAT 8 LST: a case study at IARI, New Delhi

In this study, Landsat 8-based LST data were downscaled using satellite covariates from Sentinel-1 and 2A and RF models to obtain SSM maps at high resolution (10 m) in a semi-arid agricultural farm (Figure 7). The covariates included SAR-based backscattering coefficients of VH and VV bands from Sentinel-1 and spectral bands and twenty spectral indices obtained from Sentinel-2. To downscale LST, the input covariates (fine resolution) were first aggregated to the coarse resolution of LST data (e.g., match the spatial resolution equivalent to the ∼100 m LST of Landsat-8/9 OLI/TIRS). The statistical relationship between the LST and predictor variables was developed using RF algorithm:where S2 is a covariate derived from Sentinel-2A and ${\sigma }_{\mathit{VV}}$ and ${\sigma }_{\mathit{VH}}$ are backscattering from Sentinel-1 SAR data, and f is a nonlinear function fitted by the RF. This regression model was successively applied with a finer scale to calculate LST at a high resolution.

Figure 7.

(a) SSM map derived using the downloaded LST for IARI, New Delhi, and b) scatter diagram of observed LST versus predicted aggregated LST derived using RF model over study area.

In addition, the residual correction [97] was also applied to interpret the total variation in the LST distribution by the regression model. The correction approach involves the following: 1) the aggregation of high-resolution LST predictions (∼10 m) to the original LST scale (∼100 m), 2) the LST residual ($∆{\mathit{LST}}_{100m}$) calculated between the original LST (${\mathit{LST}}_{100m}$) and the new coarse-predicted LST (${\mathit{\text{PLST}}}_{100m}$), and 3) resampling and addition of these residuals to the fine resolution predictions (${\mathit{\text{PLST}}}_{10m}$), which yields the final downscaled LST (${\mathit{\text{PLST}}}_{10m}$) given as:

$${\mathit{LST}}_{100m}=f\left(S_2,{\sigma }_{\mathit{VV}},{\sigma }_{\mathit{VH}}\right)$$E12

$$∆{\mathit{LST}}_{100m}={\mathit{LST}}_{100m}-{\mathit{\text{PLST}}}_{100m}$$E13

$${\mathit{\text{PLST}}}_{10m}={\mathit{\text{PLST}}}_{10m}+∆{\mathit{LST}}_{100m}$$E14

The residual correction confirms that the re-aggregated downscaled LST match with the original LST and also corrects for a prediction bias that might result from omitted variables.

5. Challenges

The validation of these downloaded SM products remains an existing hurdle. Current practices use direct comparisons with in situ point observations, which are not ideal for the assessment of the RS-based coarse-resolution downloaded SM products. The accuracy of the remotely sensed SM could be compromised by spatial sampling errors caused by this scale mismatch. Famiglietti et al. [98] generalized the regional variability of SSM within spatial scales ranging from 2.5 m to 50 km based on more than 3600 in situ observations and found that the mean SM variability increased from 0.036 cm³/cm³ at 2.5 m to 0.071 cm³/cm³ at 50 km. Therefore, the average SM value within 1 km² cannot unquestionably demonstrate better agreement with point measurements than that of 5 km² or even broader extent due to the significant geographical variability of SSM.

Inaccuracies in the topography, land use, soil characteristics, and meteorological input data, as well as inadequacies in the physical process parameterizations in the land surface models, can also affect SSM retrieval [76]. Inadequate phenology as well as incorrect land cover classification can also have an impact on the estimations of SM, especially in dry conditions [99]. Vegetation and land cover characteristics play an important role, including uncertainties derived from land cover class, vegetation index, albedo, vegetation optical depth, and surface roughness. In terms of model representativeness, a significant source of uncertainties is the presence of human activities, such as irrigation, and reservoir operation that can significantly affect SM dynamics, especially at fine scales [100, 101]. Using ensemble model simulations and ensemble Kalman filtering to take into account the distribution of potential SM states is a viable substitute to lower the LSM uncertainties. However, doing so requires several LSM-RTM simulations and is therefore computationally expensive.

Another important source of uncertainties in SM retrieval comes from remotely sensed products. The coarse-level global microwave RS-based SM products might have uncertainties due to biased measurements of vegetation and soil roughness parameters that would add errors to the retrieved SM. The accuracy of SM products therefore can be increased by improvements in retrieval algorithms and the quality of the input parameters [102]. Quantification of the retrieval errors using different methods such as the error propagation model [103, 104] or triple collocation method [105] can provide insight into the development of the retrievals. Despite the potential of the above-discussed approaches in SM downscaling, the methods have limitations in their applicability over different surface and climatic conditions. Since optical−/thermal-based downscaling methods depend strongly on atmospheric conditions, this approach is more suitable for arid and semiarid regions. The sensitivity of microwave data with vegetation cover can also hinder the application of active/passive fusion-based methods in different land uses.

6. Conclusion

SM is an important but challenging variable to predict as it varies highly in space and time due to the spatial heterogeneity of the landscape in terms of topography, soil properties, land cover, and variations in microclimates. In this regard, several microwave RS-based global SM products have been developed but at a coarser resolution that is not suitable for field-scale hydrology applications. This chapter provides insight into the most pertinent downscaling approaches with a comprehensive evaluation of their advantages and limitations. Several statistically and physically based techniques to downscale SM have been proposed; however, their applications vary with data availability, climatic conditions, land use, etc. While statistical and ML-based methods are relatively easy to develop, they cannot be used globally without calibration for a specific climatic zone. On the other hand, model-based downscaling techniques rely on hydrological models or radiative transfer models that can be computationally intensive when applied at fine scales over continental domains. The chapter also covers different uncertainties related to input RS data and the downscaling method used. Further understanding of it can be obtained with more studies on the inter-comparison of different downscaling methods under temporal and spatial heterogeneities and patterns of soil moisture.

Acknowledgments

The authors acknowledge the support through the National Fellow project of the Indian Council of Agricultural Research (ICAR), New Delhi, for facilitating the research work on this chapter. The authors acknowledge the fellowship received from ICAR to undertake this research work as part of the ICAR Fellowship. Support from the Head and Fellow Scientists in the Division of Agricultural Physics, IARI, is duly acknowledged.

Conflict of interest

The authors declare no conflict of interest.