Introductory Chapter: Time Series Analysis

1. Introduction

Time series, defined as sequentially observed data points over time [1], find applications across diverse domains such as economics and engineering. The statistical analysis of time series is crucial, and Chatfield’s taxonomy identifies six main categories: Economic and Financial Time Series, Physical Time Series, Marketing Time Series, Process Control Data, Binary Processes, and Point Processes.

To effectively categorize time series, consideration of features like seasonality, trend, and outliers is essential [1]. Seasonality reflects recurring patterns over time intervals, while trend represents a systematic linear or nonlinear component. Outliers are observations distant from others, often indicating anomalies. The categorization and analysis of time series are pivotal for drawing meaningful inferences from the diverse structures encountered in engineering, science, sociology, and economics [2].

The objectives of time series analysis encompass description, explanation, prediction, and control. Description involves plotting observations over time to reveal patterns, while explanation explores relationships between variables. Prediction focuses on forecasting future values, and control utilizes time series to enhance control over physical or economic systems.

Possible applications span from land use-cover [3, 4] and agriculture changes [5, 6], tourism [7, 8], socioeconomic vulnerability [9], epidemiology [10], and health [11]. This chapter delves into advanced approaches for time series analysis.

2. Time series analysis

Time Series Analysis (TSA) involves constructing predictive models that generate a target variable or label based on sequential observations across a defined period, that is, data that is time-dependent. The analysis of time series involves studying the relationships between variables that change over time [12].

There are two types of time series: deterministic and stochastic. Deterministic time series can be predicted with certainty based on previous experience, while stochastic time series have random fluctuations [13]. Time series analysis is widely used in various fields such as economics, finance, and health research [12, 14]. It helps in identifying patterns, forecasting future values, and understanding the underlying dynamics of the data [14, 15, 16]. As such, it has applications in various domains and is important for making inferences about the future based on past observations [12, 13, 15, 17, 18].

To perform time series analysis, it is important to convert nonstationary data into stationary data using techniques like differencing [13]. Time series data, common in various fields, present unique challenges due to random noise and interdependencies between measurements at different time points. Autocorrelations and partial autocorrelations of the series can indicate the degree of correlation between each point and earlier values in the series [18]. Autocorrelation measures the correlation between a data point and its lagged values, while partial autocorrelation measures the correlation between a data point and its lagged values after removing the effects of intermediate lags. Recent developments have led to prominent approaches (Figure 1).

Figure 1.

Different approaches in time series analysis.

2.1 Forecasting

Forecasting entails predicting unseen values within an observed time series, crucial in domains like economics and production planning. Despite a plethora of forecasting methods, challenges persist in achieving satisfactory generalization capabilities [19]. There is still the need to choose the most suitable methodology assuming a set of preconditions. Being a form of extrapolation, forecasting stands all the risks of it. The forecasting horizon introduces risks of error escalation, requiring careful model adaptation based on incoming information.

2.2 Anomaly detection

Anomaly Detection, synonymous with outlier or novelty detection, identifies abnormal data within a dataset [20]. Anomalies signify rare events, prompting critical actions in diverse domains such as network security and healthcare [21]. Anomalies can be categorized as point anomalies (deviations from normal patterns), contextual anomalies (anomalies in specific environments), and collective anomalies (erratic behavior in a group of similar data instances) [22].

2.3 Case-based reasoning

Case-based Reasoning (CBR) replicates human problem-solving by drawing comparisons between previously solved cases and applying similar solutions to new cases. It relies on specific knowledge and maintains a case base for reference, contributing to problem-solving and learning [23].

2.4 Bayesian optimization

Bayesian Optimization addresses global optimization problems by iteratively developing a statistical model of the unknown objective function [24]. It balances exploitation and exploration using an acquisition function, with Expected Improvement (EI) commonly employed [25]. This approach proves beneficial when dealing with black-box functions and limited samples [26].

2.5 Competitor analysis

Competitor Analysis in time series analysis considers frameworks aiding in forecasting and anomaly detection, ranging from to meteorological readings to Internet of Things (IoT) data [27]. Understanding competitors in these areas involves examining tools, methodologies, and their applicability to specific domains. Naturally, to have a superior understand of trending, anomaly occurrences, and forecasting, one has to perform analysis on these data structures. This process can be smoothed with the help of frameworks that provide the researcher with tools to perform forecasting and anomaly detection. In addition, comparative summaries provide insights into the strengths and weaknesses of each competitor.

3. Advancements in time series analysis

The latest advancements in TSA have focused on addressing the challenges posed by the increasing amount of data and the need for more efficient algorithms for accurate forecasting. These advancements offer more efficient ways of analyzing and predicting data compared to traditional approaches [28]. These advancements have been driven by the demand for accurate forecasting and decision-making in various domains such as finance, healthcare, and environmental protection [29, 30].

One can emphasize that new technologies and smarter algorithms are being developed to analyze large time series collections [28]. Likewise, the development of Python packages like tsfresh has accelerated the process of feature extraction from time series data [31]. There are also some emerging applications in various scenarios, such as subject theme evolution, academic influence evaluation, network sentiment analysis, and technology trend analysis [32].

New methods in time series analysis offer more efficient and accurate ways of analyzing and predicting data compared to traditional approaches [27]. The use of advanced techniques, such as nonlinear time series analysis, has improved the ability to model and predict complex systems [33].

However, implementing new methods can be challenging, and the impact on forecasting accuracy and model performance depends on the specific techniques used. For example, the systematic framework proposed in TsP-SA enables qualitative comparison and assessment of different time series prediction techniques, leading to improved forecasting accuracy [34].

4. Persisting issues in TSA

There are some common challenges and limitations in time series analysis. These include dealing with nonstationarity, missing data, outliers, and the curse of dimensionality [13, 14]. Nonstationarity refers to the situation where the statistical properties of a time series change over time, making it difficult to model and forecast accurately [13]. The selection and evaluation of the most suitable method for a specific task can be challenging due to the lack of comprehensive categorization and comparison of techniques [34].

In terms of advancements and trends in time series analysis, there is ongoing research in developing more efficient algorithms and techniques to handle large-scale time series data [28]. Additionally, there is a focus on integrating time series analysis with other data mining and machine learning (ML) techniques to improve prediction accuracy and discover hidden patterns [35].

Time series analysis is a valuable tool for exploring, analyzing, and forecasting data indexed over time. It involves concepts such as autocorrelation, ARIMA (Autoregressive Integrated Moving Average) models, and stochastic volatility models. ARIMA models are commonly used in time series analysis. They combine autoregressive (AR), moving average (MA), and differencing components to model the trend, seasonality, and noise in a time series [14]. In the other hand, stochastic volatility models aim to model the change over time in the variability or volatility of a time series. These models are particularly useful in finance and economics to capture the volatility clustering phenomenon observed in financial markets [14].

TSA finds applications in finance and economics, helping in predicting future values and analyzing the effects of economic policies in market forecasting, risk management, and customer requirement analysis [36]. However, it also faces challenges such as nonstationarity and missing data. Ongoing advancements include the development of more efficient algorithms and the integration of time series analysis with other techniques for improved prediction and pattern discovery.

5. Conclusion

The state-of-the-art in time series analysis shows the prevalence of ML approaches, demonstrating excellent results in forecasting and anomaly detection. While classical models maintain relevance, Bayesian Optimization enhances the quality of results, as evidenced in specific scenarios. The discussion underscores the continued importance of classical models and the evolving role of ML in time series analysis.

The demand for time series forecasting spans various challenging domains and data analytics issues. This process involves utilizing models to interpret past sequences of values at evenly spaced intervals to predict subsequent values along the same time axis. Initially, methods like Holt-Winters [37] and later Box and Jenkins [38] laid the foundation. Subsequently, statistical models such as ARIMA, X11ARIMA, X12ARIMA [39, 40], Seasonal Autoregressive Integrated Moving Average (SARIMA) [38], Seasonal Autoregressive Integrated Moving Average with Exogenous factors (SARIMAX), and Support Vector Machine (SVM) [41] were introduced to this domain.

However, a limitation arises with these models in discerning exogenous input features [42]. Additionally, Autoregressive Moving Average (ARMA) models’ linear basis poses challenges in learning and predicting the nonlinear dynamics of time series [43]. Nonlinear machine learning approaches such as kernel methods [44], Gaussian processes [45], and ensemble methods [46] have shown significant improvements but still lack the ability to assimilate true nonlinear relationships.

Neural networks, on the other hand, offer promising solutions. Multi-Layer Perceptron (MLP) was among the first neural networks used for time series forecasting, followed by recurrent networks [47]. Long Short Term Memory (LSTM) networks have significantly improved accuracy by incorporating information from long-term dependencies [48]. Fuzzy approaches and fuzzy neural networks have also been explored [49].

In recent years, deep learning approaches, particularly in domains like natural language processing, have inspired algorithms applicable to time series forecasting [50]. Attention-based architectures [51], which excel in sequence prediction tasks [52, 53], hold promise in this domain. Most recent deep learning architectures have been Vanilla Long Short Term Memory (V-LSTM) [54], Gated Recurrent Unit (GRU) [55], Bidirectional LSTM (BD-LSTM) [56], Autoencoder LSTM (AELSTM) [57], Convolutional Neural Networks combined with LSTM (CNN-LSTM) [58], Attention Mechanism Network [59], and the Transformer network [60].

Acknowledgments

We acknowledge GEOMODLAB (Laboratory for Remote Sensing, Geographical Analysis, and Modeling) of the Center of Geographical Studies/IGOT for providing the required equipment and software. This research was supported by the Portuguese Foundation for Science and Technology (FCT) under grant number 2022.09372.PTDC and grant number 2022.05015.PTDC and Centre for Geographical Studies—University of Lisbon and FCT under grant numbers UIDB/00295/2020 and UIDP/00295/2020.