Big Data Analytics for Yield Prediction in Precision Agriculture

Abstract

India’s agriculture industry is crucial to its economic growth and one of its most fundamental endeavors. Regarding a country’s economic prosperity, agriculture is among the most significant factors contributing to the happiness and well-being of its citizens. To improve agricultural output, “smart agriculture,” or the use of technology, strives for more accurate disease control, irrigation, and yield prediction. Precision agriculture is applying big data analytics and the Internet of Things to the farming industry. Agricultural production will increase dramatically as a result of this. The Internet of Things (IoT) and massive amounts of data are used in precision agriculture to improve crop quality and yields. In this research, we use the grape plants and their associated factors (temperature, humidity, rainfall, pH, sun irradiance, etc.) from the Smart Agriculture dataset to develop an N-stage CNN. In this work, we use machine learning approaches for irrigation scheduling and the DoubleGAN methodology for disease diagnosis in plants. This effort aims to create an N-stage CNN model that will significantly boost agricultural output by enhancing grape plant yield. The yield prediction is quite accurate since we considered practically all necessary characteristics and photos for plant development, including irrigation schedule and disease detection.

1. Introduction

Food production must grow by 70 percent in the year 2050 in order, according to Beecham’s paper entitled “Towards intelligent farming: agriculture that embraces IoT vision,” to meet our anticipated world population of 9.6 billion persons. Commercial markets are coming up nowadays; there is also a need for industrialization, and construction of residential buildings is taking place in agricultural fields, and these can be directly blamed on the increase in population, which in turn requires a rise in food production.

In simple words, IoT can be explained as an infrastructure of interconnected objects, people, or systems to exchange and connect information with other systems and devices over the online network. It is a system of interrelated digital machines, computing and mechanical devices, objects, and people. Implementing this interaction between a human and a human, as well as between a computer and a human, comes into action. In today’s world, seamless communication is possible through processes, things, and people. This is because of the evolution of the enormously growing technology Internet of Things. These pre-built Software-as-a-Service (SaaS) applications have made lives more accessible in the modern age. Various kinds of applications, data, and sensors are being connected here. Some notable applications are self-driving cars, smart microwaves, fitness devices, connected footballs, voice controllers, smart locks, and many others.

When there are massive datasets, analytical techniques are implemented, called Big Data Analytics. The advancement of Big Data incorporates various primer strides for its establishment. “Big Data” is a relative term depending on who discusses it and for what purpose. This includes semi-structured, unstructured, and structured data from various sources. This might also include massive amounts of datasets ranging from terabytes to zettabytes. Structured data sets are collections of data that can be utilized in their original form and from which results may be obtained. Unstructured data sets are the opposite of structured ones, without proper formatting and alignment. Relational data, such as employee salary records, comes under structured data, and human texts, Google search result outputs, etc., come under unstructured data. When there is a combination of both structured and unstructured data, it is called a semi-structured dataset. These data sets lack defining elements for sorting and processing, yet, they might have a proper structure. RFID and XML data are generally semi-structured data.

The quality of crops is essential for increased production, which also requires yield prediction. Observing the plant and disease detection is also necessary for determining the plant growth and, thereby, the total yield. Almost all the properties needed for the development of plants will be considered, and diseases also will be detected as a supporting factors. Previous works have concentrated only on two or three growth parameters, and some others have only devised methods for disease detection. In this paper, we have considered growth parameters as well as the detection of disease. Nowadays, farming is not passionately taken forward by farmers due to various issues such as insect attack diseases, sudden changes in temperature, climate, irregular rainfall, and many others. Technology advancements are required to make farming more friendly, profitable, and intelligent. There are great opportunities for discovery and innovation in precision agriculture. We have surveyed and researched extensively and used the most efficient methods to provide maximum yield.

In this work, we have immensely used the Internet of Things and Big Data Analytics methods to increase crop production. The application of these technologies in an agricultural field is called Precision Agriculture System. Big data and IoT are the two significant on which our model relies. These will then be combined by Precision Agricultural System using n-stage CNN, and we will devise a smart farming method, which will provide maximum yield.

CNN is a deep learning technique that is used to solve complicated issues. CNN is the most common approach utilized. It surpasses standard methods of machine learning [1, 2]. An automated CNN design approach can be formed by employing evolutionary algorithms to construct photo-classification jobs successfully. The most significant advantage of this method remains its “automated” feature that users do not need CNN domain expertise yet have an excellent CNN architecture for the pictures [3]. To make the training of networks that are far deeper than those used earlier, the DoubleGAN technique using VGG16 and the Residual learning framework was utilized [4, 5].

In contemporary farming management, water supply distribution plays an essential role. Irrigation controls based on evapotranspiration can ideally supply irrigation in line with the plant’s water needs. A predictive irrigation schedule was employed in nurseries with various crops and high-frequency water requirements under limited resources. The time-series analysis is vital to the water balance equation using historical data to calculate evapotranspiration. A hierarchical research algorithm was proposed to predict irrigation schedules to minimize crop stress periods and optimize resources. This algorithm includes the dispatch of priority regulations and the accounting for crop characteristics, available water, and hydraulic system constraints [6]. Irrigation plans can consist of real-time input on soil humidity and climate sensors. However, models of the dynamism of soil humidity are necessary for robust closed-loop decisions to anticipate crop water requirements while responding to outside perturbations and troubles [7].

One of the most critical tasks in agriculture is the identification and categorization of plant diseases by using appropriate imaging and machine learning technology. Through the proper detection of illnesses, we can manage them. This increases crop production automatically.

The methods used to diagnose plant illnesses from the year 2015 to the year 2017 include Support vector machines [8], Delta color difference algorithms, Histograms and text characteristics [9], Sparse Representation [10], and Convolution neural networks [11].

The neural networks significantly affected plant disease recognition between 2018 and 2019. During those eras GoogleNet [12], radial-based functions [13], VGG16 (Visual Geometrical Group) [14], MCNN (Multilayer CNN), ResNet, etc. were the most often utilized neural-network techniques [15, 16, 17]. The current detection of plant disease includes generative adversarial Networks, Extended ROI (Region of Interest), LSTM, and Inception V3 [18, 19, 20]. LeafGAN, DATGAN, and DoubleGAN are the most utilized neural network techniques using GAN [21, 22, 23].

Machine learning is a way to gather and publish enormous volumes of data. The agronomic concepts of crop modeling were integrated with machine learning to develop a machine-learning basis for a broad crop output projection. The primary line is an accuracy, modularity, and reusability procedure. For correction, we can concentrate on creating and using machine learning without information leakage, explainable predictors, or characteristics in connection to crop growth and development [24]. ML with system vision is monitored to categorize several crop images to observe the crop quality and the output valuation. This technique may be combined into improved livestock production by predicting reproductive patterns, diagnosing eating problems, animal behavior using ML models, etc. [25, 26].

The primary need in agriculture is to develop an effective model using machine learning algorithms and neural networks for the maximum yield. And the model should fulfill the following needs helping farmers to produce a total yield, early detection of diseases before it becomes fatal, and finding appropriate irrigation duration.

The stated objective of this article is as follows:

• We are developing a methodology that uses Convolutional Neural Networks to forecast the optimal production yield. The variables considered for prediction include temperature, humidity, rainfall, and sun irradiation.

• Furthermore, we are now developing a technique for illness diagnosis using the DoubleGAN approach. This approach enables more efficient and exact yield prediction than other methods relying only on certain metrics or illness detection.

2. Dataset and study area description

Machine learning models require an enormous volume of data to be processed efficiently. Adaptable information simplifies the effort to identify regularities by eliminating unnecessary features for learning. Developing a deep-reinforcement learning model for the agricultural framework is exceedingly laborious as it is a highly linear dynamic behavior. This section details the data set utilized for irrigation and agrarian yield prediction. The study examines the predictive yield of grape plant land in southern India for the district of Salem. The dataset contains particular climate, soil, and soil characteristics and the number of fertilizers consumed by the grape plants in the research field, according to the typical meteorological and soil factors.

The data collected for the irrigation phase included temperature, humidity, precipitation, and pH of grape crop type. The grape plant leaves picture data sets was obtained for plant disease identification and agricultural yield forecast. Data are collected for 15 years. Knowledge has been used about regular climate variables, such as temperature, precipitation, crop evapotranspiration reference, potential evapotranspiration, humidity, and character distinct characteristics, such as ground frost frequency and diurnal and wind speeds. The Indian Meteorological Department supplies climate data and grape plant images through their metadata tool webpage. The soil characteristics include the highest soil density and PH and the number of soil macronutrients (nitrogen, phosphorus, and potassium). The following section illustrates how the technique proposed for these datasets is utilized.

3. Methods and materials

The proposed work is divided into the Irrigation phase, plant disease detection phase, and crop yield phase. The irrigation phase output is compared using the machine learning methods naïve Bayes algorithm, Random forest tree algorithm, and decision tree algorithm. The plant disease is identified using the DoubleGAN method, and the final crop yield is predicted using the aggregation of the irrigation phase, plant disease detection, and crop yield prediction (Figure 1).

Figure 1.

System architecture.

The above figure shows that the sensor data which n-stage CNN segmented. It had two phases, the first phase was the irrigation phase, in which we had the input taken to a convoluted layer, and then it was shortened in size by pooling. And this process continued until the system reached got fully connected layer. From this, two outputs are formed and sent to the second phase, the disease detection phase. Again, the same process happened in the second phase, and we got two results: the healthy plant and yield prediction, which is our final output. The algorithm of the proposed work is given below.

3.1 Irrigation

The irrigation phase is the first phase in our convolutional neural network. In this phase, datasets had collected such as temperature, humidity, rainfall, pH, and nature of crops. The n-stage CNN segments these datasets. At the end of this phase, two outputs formed: irrigation start time and duration. In this, the two outputs, the irrigation start time and duration, are static, and the irrigation details and plant images are sent to the second phase, plant disease detection. So this is the overall methodology of our first phase. To implement the irrigation phase, three algorithms were used with grape plants. They are the decision tree, random forest, and naive Bayes algorithms, and these algorithm results are compared.

3.2 Disease detection

First of all, the image datasets are collected. Now let’s see the steps involved in the CNN implementation for the disease detection phase. The output, the irrigation details from the first phase, and plant leaf images were taken as the input for this phase. First, the input was information to the convolutional layer and then moved to the pooling layer, which reduced its size to half. Then the same process continued until it reached the fully connected layer. From this phase, two outputs formed: the healthy plant and yield prediction, which is our final output. The DoubleGAN method is used for image processing as an augmentation tool [27].

Plant leaves can be used to identify infection in plants. However, there are generally uneven depictions of uneasy leaves taken from different plants. Diseases with such an imbalanced dataset are challenging to detect. We employed DoubleGAN (a double generative adversarial Network) to equilibrate such data sets. We suggested utilizing DoubleGAN to create high-resolution pictures with fewer samples of sick leaves (Figure 2).

Figure 2.

(a): DoubleGAN first stage generation grape leaf image; (b): DoubleGAN second stage generation grape leaf image.

The two steps of Double GAN are split. As ingredients in stage 1, we used healthy leaves and unhealthy leaves. Firstly, the pre-trained model was supplied with the WGAN using healthy photos (Wasserstein generative opposing network). The model then used sick leaves to create sick leaves of 64*64 pixels. In the second stage, 256 × 256-pixel frames were obtained to extend unbalanced data settings utilizing an adversarial network in super-resolution (SRGAN). Finally, the DCGAN images were compared (Deep convolution generative adversarial network) (Figure 3) [24].

Figure 3.

DoubleGAN architecture.

3.3 Crop yield prediction

The forecast of crop production and how yield may be enhanced is essential knowledge for every farmer. The pH value, soil type, and quality are critical in providing crop yields: temperature, precipitation, moisture, sunlight hours, fertilizers, and harvest schedules [28]. Manual farming may be viewed scientifically as a feedback control system where remedial measures can be taken when a reverse effect is seen in a crop. Crop production depends heavily on the effectiveness of the best use of these resources. In the first step, if any abnormality is not recognized, the agricultural production might be damaged without precedent [29].

For crop monitoring and estimating/prediction of yield using remotely sensed data, machine learning (ML) approaches are used. The difficulties with harvest forecast methods using time series of distant information might lead to discrepancies between the fields of the standard index of vegetation changes generated from satellite (NDVI). This study employed the advanced approaches used for ML, such as boosting regression tree (BRT), random forest regression (RFR), and support for the reversal of vectors and Gaussian GPR. The normalized vegetation differential index (NDVI) measurements were averaged and integrated each year for each field to generate a double-dimensional data set. The ML methods have been utilized 100 times, and their assessment measures have been used to assess performance and study stability. Finally, every ML method has averaged its findings to produce returns. Comparisons between these approaches reveal that the average R-value of BRT throughout all years is over 0.87 [30].

The NDVI is a simple graphical indicator for the analysis of distant sensing measures, frequently from a space platform, which assesses whether the target is seen containing living green vegetation. This is a standardized difference vegetation index. The vital forces of the two algorithms are combined via boosted regression trees. It’s a regression tree and growing. The regression tree relates a reaction via recurrent binary splits to its predictors. The boost is an adaptive way to combine multiple accessible models to enhance prediction performance.

3.4 Algorithm: precise agriculture system

1. Input : H,R,T,C,pH,SI, Images

   Where H denotes Humidity,

   R denotes Rainfall,

   T denotes Temperature,

   C denotes Nature of Crop,

   pH denotes Power of Hydrogen,

   SI denotes Solar Irradiance.

2. Output: IS, ID, PS, YT

   Where IS denotes Irrigation

   Start Time,

   ID denotes Irrigation

   Duration,

   PS denotes Plant Healthy

   Status,

   YT denotes Yield Time

3. Apply Decision tree, Random Forest and Naïve Method on Inputs H,R,T,C,pH and SI

4. Choose the method which produce the utput i.e IS and ID with better accuracy

5. Improve the input images resolution by applying the DoubleGAN method

6. Identify the Plant status i.e plant is defected or healthy

7. Apply Boosted Regression Tree method and predict the yield time

8. Fine-tune the irrigation scheduling, plant disease detection and crop yield prediction output by considering all the applications output.

The precision agriculture system algorithm uses humidity, rainfall, temperature, crop, pH, solar irradiance, and plant images as inputs. The algorithm’s output includes the irrigation schedule, plant healthy status, and yield prediction. The precision agriculture system algorithm comprises the irrigation, plant disease, and yield prediction phases. The results and findings of this algorithm are explained in the following section.

4. Experimental results and discussion

The proposed method is evaluated on grape leaves with humidity, temperature, rainfall, crop, pH, and soil irradiance to identify the irrigation schedule. Then the image dataset is used to detect plant status and crop yield. The evaluation of the proposed approach is planned through classification methods, including the Decision tree method, random forest method, and naïve Bayes methods. The plant disease detection is performed using DoubleGAN and is evaluated using ResNet and VGG algorithms. Finally, the crop yield prediction is made using Boosted regression tree method.

4.1 Irrigation result

The correct irrigation schedule is evaluated using the Decision tree, random forest, and naïve Bayes methods with the parameters Mean Square Error (MSE), Mean Absolute Error (MAE), R-Squared Error, Root Mean Square Error, and accuracy. But it still requires the expert’s validation to predict the exact irrigation scheduling. We can also consider other criteria like plant disease detection and crop yield time to get precise predictions about irrigation scheduling. The following section and Table 1 show the different machine-learning algorithm performance results for grape leaves irrigation scheduling.

Algorithm	MSE	MAE	R-Squared	RMSE	Accuracy (%)
Decision Tree	7.63	2.16	−0.87	2.77	70
Random Forest	7.51	2.14	−0.82	2.74	75
Naive Bayes	10.82	2.58	−1.62	3.29	65

Table 1.

Comparison of ML models on irrigation.

The above graphs represent the Random Forest, Decision Tree, and Naive Bayes metric values. As we can see, among the three algorithms, Random Forest had the highest accuracy, and thus, it proves that this algorithm is the best implementation method for the irrigation phase.

The average or mean of the squares of the expected and predictable target values in a dataset is used to calculate the Mean Square Error.

$${\text{MSE=1/n}}^{\text{*}}\Sigma \text{}\left(\text{e\_i–p\_i}\right)\text{^2}$$E1

Where e_i is the dataset’s i’s expected value and p_i is the dataset’s i’s predicted value.

The mean absolute error is a popular statistic because the error score units, such as RMSE, correspond to the team of the projected target value. The MAE may be calculated using the formula shown below.

$${\text{MAE=1/n}}^{\text{*}}\Sigma \text{abs}\left(\text{e\_i–p\_i}\right)$$E2

Where e_i is the dataset’s i’s expected value and p_i is the dataset’s i’s predicted value, and abs() is the absolute function.

The R-squared (R2) statistic shows the variation in a dependent variable that can be explained by an independent variable or variable in a regression model.

The formula below can be used to calculate the RMSE.

$$\text{RMSE=sqrt }\left({\text{1/n}}^{\text{*}}\Sigma \left(\text{e\_i–p\_i}\right)\text{^2}\right)$$E3

Where e_i is the dataset’s i’s expected value and p_i is the dataset’s i’s predicted value.

Accuracy is defined as the percentage of correct test data predictions. The amount of right predictions may be readily estimated by dividing them by the total.

Accuracy can be defined as the proportion of adequately categorized cases.

$$\text{Accuracy=x/y}$$E4

X is the sum of true positive and true negative, and y is the sum of True Negative, False Negative, False Positive, and True Positive (Figure 4).

Figure 4.

(a): MSE Metrics; (b): MAE Metrics; (c) R-Squared metrics; (d): RMSE metrics; (e) Accuracy Metrics.

4.2 Plant disease detection results

The DoubleGAN is used to expand the low-resolution images of grape leaves into high-resolution images. And then, the accuracy is evaluated with the VGG16 and resNet50. The results of the various algorithm are shown in Table 2.

Plants		Tomato	Potato	Corn	Accuracy average
Original DataSet	VGG16	96.45	97.65	97.48	97.19
	ResNet50	96.65	97.83	97.61	97.36
Flipping and Translation Expansion	VGG16	98.45	98.15	97.85	98.15
	ResNet50	98.75	98.35	97.85	98.32
DoubleGAN Expansion	VGG16	97.83	99.25	99.8	98.96
	ResNet50	98.85	99.45	99.88	99.39

Table 2.

Plant disease detection model accuracy.

The results demonstrate that the classifications are better after DoubleGAN expands the data set. VGG16 and ResNet50 are classified very closely in the data sets, so DoubleGAN images are similar to those produced initially. Based on random noise, the DoubleGAN may create multiple images, overcoming the difficulty of insufficient variety generated by the extension of the original data set by spinning and transformation expansion. These plant disease detections also influence irrigation and crop yield time predictions (Figure 5).

Figure 5.

Plant disease detection model accuracy.

4.3 Crop yield prediction results

In this work, complex ML technologies were employed, such as Boosted Regression Tree (BRT), random forest regression (RFR), support for vector regression (SVR), and Gaussian GPR with grape plants. And these ML techniques are compared with the error parameters MAE, RMSE, and R. The following section and Figure 6 shows that the Boosted regression tree method has lower error rates than RFR, SVR, and GPR. Finally, based on all these application’s outputs, such as irrigation scheduling, plant disease detection, and yield time prediction, the final result can be fine-tuned, i.e., the production of irrigation scheduling can be predicted effectively and efficiently based on plant disease detection output and crop yield time prediction (Table 3).

Figure 6.

Yield prediction methods error rate.

ML Model	MAE	RMSE	R
BRT	2.32	4.96	1.21
RFR	1.91	3.63	0.71
SVR	6.02	8.21	0.43
GPR	2.85	5.48	0.75

Table 3.

Yield prediction methods error rate.

The suggested methodology will be assessed using classification techniques, such as the Decision tree method, random forest method, and naive Bayes methods. Among these three machine learning models, the random forest yielded superior outcomes. The identification of plant diseases is conducted via the DoubleGAN technique, while the assessment of its performance is carried out using the ResNet and VGG algorithms. Using the DoubleGAN picture dataset, in conjunction with ResNet and VGG models, yields higher accuracy rates than both the original dataset images and conventional enhanced images. The crop yield forecast error rate is calculated using several machine learning algorithms, including BRT, RFR, SVR, and GPR. The support vector regression has a much lower error rate than other approaches.

5. Conclusion

The forecast of crop output has great importance in enhancing the pace of production. The dwindling population of seasoned farmers underscores the need to comprehend optimal crop production methods in regions with limited resources, particularly for novice agricultural practitioners. Despite the existence of other studies conducted in this particular domain, the findings of this research indicate that the use of maximum rendering techniques may provide advantageous outcomes due to a multitude of aspects, including the optimization of watering schedules and the facilitation of plant disease diagnosis. A viable methodology for precision agriculture has been devised using N stage Convolutional Neural Networks, which have been integrated with machine-learning techniques. The experimental findings demonstrate that this method provides a pragmatic and accurate outcome by considering irrigation scheduling, disease detection, and crop production prediction. This study has the potential for future expansion by using more GAN approaches that generate a larger quantity of pictures. By doing so, the accuracy of disease identification and crop production prediction may be significantly enhanced.

Acknowledgments

Dr. T. Karthick, Department of Data Science and Business Systems, SRM Institute of Science and Technology, Chennai, India, supported and co-authored this study. So I thank Dr. T. Karthick for his excellent technical support.

Conflict of interest

This paper Titled “Precision Agriculture System using Big Data Analytics in Predicting Yield by Controlling Plant Disease and Irrigation Scheduling,” by the authors Mr. N. Vasudevan and Dr. T. Karthick declares that the publishing of this paper does not involve any conflicts of interest. The authors certified that the article was plagiarism-free.