Document Type : Research Paper
Abstract
In this research, a simple experiment in the field of agriculture was studied, in terms of the effect of out-of-control noise as a result of several reasons, including the effect of environmental conditions on the observations of agricultural experiments, through the use of Discrete Wavelet transformation, specifically (The Coiflets transform of wavelength 1 to 2 and the Daubechies transform of wavelength 2 To 3) based on two levels of transform (J-4) and (J-5), and applying the hard threshold rules, soft and non-negative, and comparing the wavelet transformation methods using real data for an experiment with a size of 26 observations. The application was carried out through a program in the language of MATLAB. The researcher concluded that using the wavelet transform with the Suggested threshold reduced the noise of observations through the comparison criteria.
Main Subjects
Highlights
1- When applying The Discrete wavelet transform at the level (j-4), it led to a decline in the value of MSe and CV for all cases, but at the level (j-5), there was an increase and decrease in the values of the Criteria MSe and CV, in this experiment.
2- Obtaining the best results when applying the hard threshold rule with the Universal and suggested threshold according to standards.
3-When processing the observations noise (wheat crop 26) and the level (j-4), the suggested threshold gave better results than the Universal Threshold based on the criteria MSe, CV, MSe(w), SNR. and The best results were obtained for the first-order Coiflet wavelet transformation filter with the hard threshold rule, the second-order Daubechies wavelet transformation filter with the non-negative threshold rule, and second-order Coiflet wavelet transformation filter with the non-negative threshold rule, in this experiment.
4-When processing the observations noise (wheat crop 26) and the level (j-5), the suggested threshold gave better results than the Universal Threshold based on the criteria MSe, CV, MSe(w), SNR. and The best results were obtained for the first-order Coiflet wavelet transformation filter with the hard threshold rule, the second-order Daubechies wavelet transformation filter with the hard threshold rule, and the second-order Coiflet wavelet transformation filter with the hard threshold rule, in this experiment.
Full Text
1. Introduction
The Discrete Wavelet transformation is one of the very important topics used in several application fields, especially in data noise processing. In general, noise represents an unexplained variance within the data (Reid & Reading,2010); that is, it cannot be eliminated. But it can be reduced in several ways according to the study or its field of application. In agricultural experiments, the experiment is divided into blocks. It replicates according to the different experimental units, which directly or indirectly affect the observation value, resulting from the mathematical model according to the applied design. The method or procedure cannot be controlled in most experiments due to out-of-control conditions during the application of the experiment.
The wavelet transform has been discussed in addressing pollution and heterogeneity (Ali & Mawlood,2010) for the complete randomized design using wavelet filters and some types of threshold rules. In addition, a threshold was Suggested to reduce the observations noise for a factorial experiment by (Taher&Sabah,2022) compared with the Universal Threshold. In this research, the application of different levels of analysis, through the use of the Daubechies transform of wavelengths from 2 to 3 and the transformation of Coiflets of wavelengths from 1 to 2 with hard, soft, and non-negative threshold rules, and comparison of the results.
2. Discrete Wavelet transform (DWT)
The Discrete wave transform is one of the most Transfers used in the wavelet due to its multiple applications in various practical fields and its theoretical uses in various sciences. The researcher will give a comprehensive idea of this transformation and focus on its use in designing experiments Through the application of a simple experiment.
The work of the discrete wavelet transform depends on the Mallat pyramidal algorithm, which is an efficient algorithm proposed by the researcher Mallat (1989) (Nason, 2008) to calculate the wavelet coefficients for a set of data containing noise
The principle of The work of this algorithm is to create filters for smoothing and heterogeneity from the wavelet coefficients, and these filters are used frequently to obtain data for all scales, meaning that the wavelet transform splits the data into two components, the first component is called detail, which includes high frequencies and can be calculated from the mother wavelet by the following formula ( In & Kim, 2013)
Whereas
Represents a high-pass filter (In & Kim, 2013)
he Second component is called Approximate, which includes low frequencies (noise) or (anomalous) values according to the nature of the study and its application, and it can be calculated from the father wavelet by the following formula: (In, F., & Kim, S 2013)
Whereas Represents a low-pass filter, (In & Kim, 2013).and it is related to through
The following figure shows the division of data into two components
|
Low pass filter
|
|
Approximation coefficients
|
|
X[n] |
|
High pass filter
|
|
Detail coefficients
|
|
Data filter coefficients
|
Figure (1): High-pass filter and low-pass filter for x vector
In general, the discrete wavelet transform is used with data that contain discrete variables and have discrete outputs.
3. The Transfer levels Multi Resolution Analysis
We will give a brief overview of the key concepts of multiscale analysis before attempting formal definitions of wavelets and the wavelet transform and How we extract multiscale "information" from the vector y . We identify the "detail" in the sequence at various scales and places as the essential information.
Transform levels are determined from the design observation, and through the application side, we will have a simple experiment containing sixteen treatments and four blocks represented by the following vector.
Through the observations vector, the levels of analysis for this experiment will be (Nason, 2008).
The next stage is how to extract information from vector , where the information extracted from vector It is called (detail), which can be obtained from different locations and levels, and in general, the word "detail" means "degree of difference" or "variance" in the observations of the vector. This information is calculated based on the following two equations (Nason,2008).
The next step is calculating the elaboration and measurement coefficients for the other levels (Nason,2008).
This process is called the multiscale transform algorithm, Through the simple experiment applied, we will have the following levels { (j) , (j-1) , (j-2) , (j-3) , (j-4) , (j-5)}.
4. Stages of Discrete Wavelet Transform Using Orthogonality
4.1. Representing the observations of a randomized complete block design (RCBD) with a vector That contains all the observations of the experiment, which the following mathematical model represents
Whereas
observation value (j) from treatment (i), The general arithmetic mean, The effect of the i-treatment for this observation, The amount of random error , number of treatments number of blocks
The vector x contains all the observations of the experiment. One of the critical conditions in the wavelet transformation is The size of the observations fulfills the following condition.
4.2.Applying Coiflets transform and the wavelet Daubechies on observation the Experiment, we will have wavelet coefficients that can be represented Using an orthogonal matrix , and multiplying it by the observations vector, the Wavelet Coiflets and Daubechies coefficients vector of the following form is Obtained.
whereas
The wavelet coefficients vector
Orthogonal matrix
Vector observation
From the formula (9), we get a vector coefficient of a discrete wavelet which can be represented in the following form.
|
Approximation coefficients |
Whereas, Represents the first component of the transform, which is the detail coefficients computed from the rate of the difference of the data at each measurement and is symbolized by CD. As for Represents the second component of the transform, which is the approximation coefficients and represents the rate of The measurement is symbolized by CA.
4.3. We note that the formula (6) can be obtained through which the values of the original Data Depending on the orthogonality condition of the Discrete wavelet coefficients.
To clarify what was mentioned above, we take the following figure, which shows the discrete wavelet transformation coefficients for the data for four levels .
|
|
|
DWT1 |
|
Approximation |
|
X |
|
CAA |
|
CD |
|
CAD |
|
+ |
|
X |
|
CA |
|
CD |
|
+ |
|
Level 1 |
|
Level 2 |
|
DWT2 |
|
X |
|
CD |
|
|
|
CAAA |
|
+ |
|
|
|
CAAD |
|
|
|
CAD |
|
Level 3 |
|
DWT3 |
|
X |
|
|
|
CAAAA |
|
CD |
|
|
|
CAAAD |
|
|
|
CAAD |
|
|
|
CAD |
|
+ |
|
Level 4 |
|
DWT4 |
|
Details |
|
|
Figure(2): The wavelet transform levels for size N = 32
Several types of Wavelets exist through the above offer About Discrete Wavelet transformation and orthogonality. In this research, we will address: Daubechies Wavelet, Coiflets Wavelet
4.4. Threshold rules (hard, soft, non-negative) are applied with threshold limit Universal Threshold and suggested threshold.
Equation (7) represents the Universal Threshold (UT) value (Gençay et al.,2001) (Zhang et al., 2021), while equation (8) represents the Suggested Threshold (ST) (Taher & Ridha, 2022).
5. Daubechies Wavelet
The name came in relation to the researcher Ingrid Daubechies (Tammireddy & Tammu, 2014). It has made a boom in the wavelet theory, as it is generated from a group of intermittent wavelets. The most crucial characteristic of this family of wavelets is their smoothness. It is abbreviated as follows.
where's
Db: Abbreviation for the name of the researcher Daubechies
N: Wavelet rank
considered wavelet haar of one the members of this family of wavelets and is symbolized by the symbol db1 or The wavelet is called Daubechies the first order Because built from the function of the father (father wave), the function of the mother (mother wave), as follows (Tammireddy & Tammu, 2014).
father wavelet Daubechies
Where Daubechies scaling function coefficients
Mother wavelet Daubechies
Where Daubechies wavelet function coefficients
Figure Next represents a Daubechies Wavelet of several lengths
Figure(3): the wave function and The scale function of a wave Daubechies
6. Coiflets Wavelet
They are discrete wavelets designed by Ingrid Daubechies, at the request of Ronald Coifman (Tammireddy & Tammu, 2014).
where's
: Abbreviation for the name of the researcher Ronald Coifman
: Wavelet rank
This family of wavelets is characterized by the presence of a relationship that relates the length of the filter with its rank
It is also built from the father function (father wavelet) and mother function (mother wavelet), in the following formula (Tammireddy & Tammu, 2014)
father wavelet Coiflets
Where Coiflets scaling function coefficients
Mother wavelet Coiflets
Where Coiflets wavelet function coefficients
,
Figure(4): the wave function and The scale function of a wave Coiflets
This family of wavelets is considered orthogonal and is close to symmetry, as it connects the mother and father function through the high-pass filter and the low-pass filter by obtaining the vanishing torque, unlike each filter separately.
7. Threshold Rules
Three types of threshold rules will be applied in this research, namely:
7.1. Hard Threshold Rule
One of the types of threshold rules, it is applied in discrete wavelet transform and takes the following form (Dehda & Melkemi, 2017),( Zaeni et al.,2018).
Through the formula (9) where the coefficients whose values are greater and equal to the threshold do not change, and the coefficients whose values are less than the threshold are replaced by the value zero
7.2. Soft Threshold Rule
It is another type of threshold rule and can be written as (Bruce & Gao , 1995).
And It is considered one of the standard techniques for processing observational noise (Tang et al., 2013) (Han & Xu, 2016).
Through the above formulas (10) and (11), we note that if the coefficients of the wavelet are less than the threshold value, it goes to zero, but in the case of being greater than the threshold value, it preserves its value. Using a Shrinking wavelet based on a soft-threshold rule tends to bias because all large coefficients Shrink towards zero.
7.3 Non-Negative Garrote Threshold Rule
This threshold is characterized by the small samples. It is less sensitive to observation than the hard threshold, especially in small fluctuations, and it is less biased than the soft threshold and can be written as follows (Gao,1997).
It is also the base of wavelet Shrink introduced by Gao
8. Evaluation Criteria
For the purpose of comparing the results between the Coiflets Wavelet Transformation and the Daubechies Wavelet Transformation, several criteria were applied,
8.1.The mean square error for design is defined by the following formula(Montgomery, 2020):
1-
Where total sum of squares , treatments Sum of squares , blocks Sum of squares
8.2. The coefficient of variation(cv) is defined by the following formula: (Montgomery, 2020):
8.3. The mean of squares for original and transform observation is defined by the following
Formula (He et al.,2014):
, observation value of experiment , Experiment data after transformation
8.4. The Signal-to-noise ratio (SNR) is defined by the following formula(He et al.,2014):
Where’s: ,
and The following diagram shows the steps of the wavelet transform, with the comparison criteria
|
start |
|
Input Data |
|
Coiflets Wavelet |
|
Daubchies Wavelet |
|
Calculation of the Universal Threshold value |
|
Calculation of the Suggested Threshold value |
|
Evaluation Criteria's MSe ,CV. MSe(w),SNR
|
|
Best Vector |
|
|
|
|
|
|
|
Application Hard rule with Universal |
|
Application Hard rule with Suggested |
|
Application Non N rule with Universal |
|
Application Non N rule with Suggested |
|
Application Soft rule with Universal |
|
Application Soft rule with Suggested |
* The Scheme was prepared by the researcher
Figure (5): Stages of The Wavelet Transform in this research
9. The practical side
The field experiment on wheat cultivation was conducted in one of the stations of the National Program for the Development of Wheat Cultivation in Iraq, and sixteen wheat varieties were included.(al sds (12) A1, al sahel (1)A2 , al sds (1)A3 , Egypt (1)A4 .Egypt (2) A5, Sakha (93)A6, al Geza (11)A7, al Geza (168)A8, Apaa (99)A9, Italia(1)A10, Italia (2)A11, Caronia A12, gold kernels A13, aom al rabee A14, Smitto A15, Waha A16).
According to the complete random blocks design (CRBD) of four blocks, each block contained 16 experimental units. Then the characteristics of the field yield were taken, which are (number of branches, plant height cm, dry weight of g, number of spikes / m2, number of seeds/spike, weight of 1000 grains of/ g, Grain yield / m²), where the trait was studied: grain yield / m2.
By applying equations (10-19) on experiment observations, The results are shown in table (1), Which represents a summary of wavelet transformation when the level of analysis is (J-4)
Table (1): The best results of wavelet transformation when the
transformation level is (j-4) for The first experiment
|
wavelets |
Data |
MSe |
CV |
MSe(w) |
SNR |
Test Normal data |
Test Homogeneity of variance |
|
|
|
X |
16769 |
0.30 |
|
|
normal |
Pv>0.05 |
|
|
DbN
|
XD2h3st |
16439 |
0.30 |
4599.2 |
10.21 |
P.v>0.15 |
P.v=0.93 |
|
|
XD2h4st |
13560 |
0.27 |
7310.5 |
8.19 |
P.v=0.14 |
P.v=0.89 |
||
|
XD2s2st |
10373 |
0.27 |
7595.1 |
8.03 |
P.v=0.09 |
P.v=0.91 |
||
|
XD2s3st |
8634 |
0.27 |
15902 |
4.82 |
P.v>0.15 |
P.v=0.91 |
||
|
XD2n2st |
13908 |
0.28 |
2767.4 |
12.41 |
P.v>0.15 |
P.v=0.92 |
||
|
XD2n3st |
11832 |
0.27 |
7496.4 |
8.08 |
P.v>0.15 |
P.v=0.91 |
||
|
XD3h3st |
16313 |
0.30 |
5589.8 |
9.36 |
P.v=0.09 |
P.v=0.82 |
||
|
XD3s2st |
10953 |
0.28 |
11498.9 |
6.23 |
P.v=0.02 |
P.v=0.17 |
||
|
XD3n2st |
14023 |
0.28 |
4695.9 |
10.12 |
P.v=0.13 |
P.v=0.96 |
||
|
XD3n3st |
12544 |
0.28 |
10820.9 |
6.49 |
P.v=0.02 |
P.v=0.97 |
||
|
Coif N |
XC1h2st |
16207 |
0.29 |
2099.5 |
13.61 |
P.v>0.15 |
P.v=0.94 |
|
|
XC1h3st |
16762 |
0.30 |
5108.8 |
9.75 |
P.v>0.15 |
P.v=0.99 |
||
|
XC1s2st |
9374 |
0.26 |
10088.8 |
6.79 |
P.v>0.15 |
P.v=0.96 |
||
|
XC1n2st |
12623 |
0.27 |
3812.5 |
11.02 |
P.v>0.15 |
P.v=0.97 |
||
|
XC1n3st |
9621 |
0.24 |
10004.1 |
6.83 |
P.v=0.08 |
P.v=0.96 |
||
|
XC2h3st |
15856 |
0.29 |
3893.4 |
10.93 |
P.v>0.15 |
P.v=0.98 |
||
|
XC2s2st |
9109 |
0.26 |
9520.9 |
7.05 |
P.v=0.1 |
P.v=0.95 |
||
|
XC2n2st |
12508 |
0.26 |
3457.2 |
11.45 |
P.v>0.15 |
P.v=0.95 |
||
|
XC2n3st |
9154 |
0.23 |
9257.2 |
7.17 |
P.v=0.02 |
P.v=0.98 |
||
By applying equations (10-19) on experiment observations, The results are shown in table (2), Which represents a summary of wavelet transformation when the level of analysis is (J-5)
Table (2): The best results of wavelet transformation when the
transformation level is (j-5) for The first experiment
|
wavelets |
Data |
MSe |
CV |
MSe(w) |
SNR |
Test Normal data |
Test Homogeneity of variance |
|
|
X |
16769 |
0.30 |
|
|
normal |
Pv>0.05 |
|
DbN
|
XD2h2th |
17011 |
0.30 |
1587.1 |
14.83 |
P.v>0.15 |
P.v=0.89 |
|
XD2h3th |
17315 |
0.31 |
5925.5 |
9.11 |
P.v>0.15 |
P.v=0.98 |
|
|
XD2s2th |
10950 |
0.31 |
13444.8 |
5.55 |
P.v>0.15 |
P.v=0.96 |
|
|
XD2n2th |
14574 |
0.30 |
4127.1 |
10.68 |
P.v>0.15 |
P.v=0.95 |
|
|
XD2n3th |
13487 |
0.32 |
10955.3 |
6.44 |
P.v>0.15 |
P.v=0.95 |
|
|
XD3h2th |
18447 |
0.32 |
1448.4 |
15.22 |
P.v>0.15 |
P.v=0.92 |
|
|
XD3h3th |
14503 |
0.28 |
4554.9 |
10.25 |
P.v>0.15 |
P.v=0.98 |
|
|
XD3s2th |
12318 |
0.33 |
12287.2 |
5.94 |
P.v>0.15 |
P.v=0.95 |
|
|
XD3n2th |
15620 |
0.31 |
3806.6 |
11.03 |
P.v>0.15 |
P.v=0.96 |
|
|
XD3n3th |
15619 |
0.34 |
9804.4 |
6.92 |
P.v>0.15 |
P.v=0.97 |
|
|
Coif N |
XC1h2th |
16246 |
0.29 |
1522.9 |
15.01 |
P.v>0.15 |
P.v=0.85 |
|
XC1s2th |
13728 |
0.35 |
13167.1 |
5.64 |
P.v>0.15 |
P.v=0.88 |
|
|
XC1n2th |
16864 |
0.32 |
4029.3 |
10.78 |
P.v>0.15 |
P.v=0.92 |
|
|
XC1n3th |
16400 |
0.35 |
11277.3 |
6.31 |
P.v>0.15 |
P.v=0.91 |
|
|
XC2h2th |
15950 |
0.29 |
1486.1 |
15.11 |
P.v>0.15 |
P.v=0.86 |
|
|
XC2h3th |
18164 |
0.32 |
6210.8 |
8.90 |
P.v>0.15 |
P.v=0.95 |
|
|
XC2s2th |
12904 |
0.34 |
13800.4 |
5.43 |
P.v>0.15 |
P.v=0.86 |
|
|
XC2n2th |
16326 |
0.32 |
4230 |
10.57 |
P.v>0.15 |
P.v=0.89 |
|
|
XC2n3th |
15105 |
0.34 |
11486.1 |
6.23 |
P.v>0.15 |
P.v=0.91 |
9.1 Results
Table (1) represents a summary of the wavelet transform results for all cases when the transform level is (J-4), where led to decline in the MSe value of the used design and a significant improvement in the CV criterion. In addition to obtaining low values for the standard .with an increase in the SNR criterion value, Especially for ( XD2h3st , XD2n2st , XD3n2st , XC1h2st , XC2h3st , XC2n2st )
Table(2) represents a summary of the wavelet transform results for all cases when the transform level is (J-5), where led to decline in the MSe value of the used design and a significant improvement in the CV criterion. In addition to obtaining low values for the standard .with an increase in the SNR criterion value, Especially for( XD2n2th, XD3h3th , XC1h2th, XC2h2th).
Where XD2h3st : Represents the second-order Daubechies wavelet transformation filter with the hard rule and suggested threshold, And so on for the rest of the vectors
10. Conclusion
1- When applying The Discrete wavelet transform at the level (j-4), it led to a decline in the value of MSe and CV for all cases, but at the level (j-5), there was an increase and decrease in the values of the Criteria MSe and CV, in this experiment.
2- Obtaining the best results when applying the hard threshold rule with the Universal and suggested threshold according to standards.
3-When processing the observations noise (wheat crop 26) and the level (j-4), the suggested threshold gave better results than the Universal Threshold based on the criteria MSe, CV, MSe(w), SNR. and The best results were obtained for the first-order Coiflet wavelet transformation filter with the hard threshold rule, the second-order Daubechies wavelet transformation filter with the non-negative threshold rule, and second-order Coiflet wavelet transformation filter with the non-negative threshold rule, in this experiment.
4-When processing the observations noise (wheat crop 26) and the level (j-5), the suggested threshold gave better results than the Universal Threshold based on the criteria MSe, CV, MSe(w), SNR. and The best results were obtained for the first-order Coiflet wavelet transformation filter with the hard threshold rule, the second-order Daubechies wavelet transformation filter with the hard threshold rule, and the second-order Coiflet wavelet transformation filter with the hard threshold rule, in this experiment.
)
