The spectral form of the Dai-Yuan conjugate gradient algorithm

Conjugate gradient (CG) methods comprise a class of unconstrained optimization algorithms which characterized by low memory requirements and strong local and global convergence properties. Most of CG methods don't always generate a descent search directions, so the descent or sufficient descent condition is usually assumed in the analysis and implementations. By assuming a descent and pure conjugacy conditions a new version of spectral Dai-Yuan (DY) non-linear conjugate gradient method introduced in this article. Descent property for the suggested method is proved and numerical tests and comparisons with other methods for large-scale unconstrained problems are given. صخهمنا قئازط تايجتمنا ةقفازتمنا (CG) ت مكش فنص نم تاْمسراٌخ وْهثملأأ زْغ ةذْقمنا شْمتتً هذى قئازطنا اينأب لا ااتحت َاانا شاخ تافٌفااصم نذااكً ااين ةْااصاخ رااقتنا ِااهحمنا قااهطمناً . اهغا قاائازط (CG) ذاانٌتلأ تايجتم ثحب ةّراذحنا امئاد نذن ةْصاخ راذحنلاا داع ة ضزفّ ذنع ت مْهح مْثمتً هذى تاْمسراٌخنا . ضزفب ِتْصاخ راذحنلاا قفازتناً خنا نا ص انحزتقا ةغْص ةْفْط ةذّذج ةْمسراٌخن ُاد اٌّ تايجتمهن ةقفازتمنا زْغ ةْطخنا , مت اىزب ةْصاخ راذحنلاا ةْمسراٌخهن ةحزتقمنا نذكً مت ايتنراقم اْهمع عم تااْمسراٌخ ٍ زاخا ِ اف سفن لاجمنا .


1-Intrduction
The non-linear Conjugate Gradient (CG) method is a very useful technique for solving large scale unconstrained minimization problems and has wide applications in many fields [9].This method is an iterative process which requires at each iteration the current gradient and previous  a is obtained by a line search, and the directions k d are generated by the rule: , here k b is the CG update parameter.Different CG methods corresponding to different choice for the parameter k b see [1, 4 and 10].The first CG algorithm for non-convex problems was proposed by Fletcher and Revees (FR) in 1964 [11], which is defined as  Although all the above formulas are equivalent for convex quadratic functions, but they have different performance for non-quadratic functions, the performance of a non-linear CG algorithm strongly depends on coefficient k b .Dai and Yuan (DY) in [6] proposed a non- linear CG method (2) and (3) with k b defined as: .  9) is called the curvature condition and it's role is to force k a to be sufficiently far a way from zero [12].Which could happen if only condition (8) were to be used.Conditions (8) and ( 9 8) and ( 9) we can impose on k a the Strong Wolfe Conditions (STWC): + cannot be arbitrarily large [12].The (STWC) with the sufficient descent property ) Widely used in the convergence analysis for the CG methods.Theorem (1): Assume that f is continuously differentiable and that is bounded below along the line . Suppose also then there exists nonempty intervals of step lengths satisfying the (SDWC) and (STWC) conditions.For proof see [12].
The Fletcher-Revees (FR) and Dai-Yuan (DY) methods have common numerator . One theoretical difference between these methods and other choices for the update parameter k b i s t h a t t h e g l o b a l convergence theorems only require the Lipschitz assumption not the bounded ness assumption [9].
The global convergence for the methods with in the numerator of k b established with exact and inexact line searches for general functions [2, 7, and 13].Despite the strong convergence theory that has been developed for methods with in the numerator of k b , these methods are all susceptible to jamming, that is they begin to take small steps without making significant progress to the minimum [9].On the other hand the convergence of the methods with the performance of these methods is better than the performance of the methods with in the num erator of k b see [9], but they have weaker convergence theorems.This paper is organized as follows: in section 2 new spectral form for DY non-linear conjugate gradient algorithm is suggested.In section 3 we will show that our algorithm satisfies sufficient descent condition for every iteration.Section 4 presents numerical experiments and comparisons.

New spectral form for Dai and Yuan CG method
An attractive feature of the CG method is that the following (pure conjugacy condition ) is convex quadratic and line search is exact [8].In this section we use the relation ( 7) and ( 14) to derive new spectral DY conjugate gradient method.Consider the search direction of the form where k g is parameter.Assume that the search direction in (15) satisfies the relation (7) i .e With simple algebra we get Therefore the new spectral DY search direction is Step Step(4): Direction computation: compute 1 . If Powell restart is satisfied then

Descent property of the SPDY algorithm
An important feature for any minimization algorithm is the descent (7) or the sufficient descent (13) property.In this section we proof that our suggested new algorithm (SPDY) generates a sufficient descent directions for each iteration k.

Theorem (1):
Suppose that the step-size k a satisfies the standard Wolfe conditions (SDWC), consider the search directions k d generated from (21) where The proof is by induction.If k=1 then 0 then the sufficient descent holds with c=1, know let to proof for k+1, multiply (21)by T k g 1 Note that from second standard Wolfe condition (9)we have 20), (2.9) and (23) we get

Numerical results and comparisons
In this section we present the computation performance of a FORTRAN implementation of the SPDY, DY and FR algorithms on a set of unconstrained optimization test problems.We selected ( 15 ) largescale unconstrained optimization test problems in extended or generalized form from [5].For each function we have considered n=100, 1000 (where n is the number of variables ).All algorithms implement the standard Wolfe line search conditions with 0001 ., where
is the maximum absolute component of a vector.
The comparison of algorithms are given in the following context.We say that, in the particular problem i the performance of Algorithm(Alg1) The Fourth Scientific Conference of the College of Computer Science & Mathematics ] 220 [ was better than the performance of Alg2 if the number of iterations (iter) or the number of function-gradient evolutions (fg) or the number of restart (irs) of Alg1 was less than the number of (iter) or (fg) or the (irs) corresponding to Alg2, respectively.Table (1) and table (2) shows the details of numerical results for the Fletcher-Revees (FR), Dai-Yuan (DY) and our algorithm (SPDY).

Prof. \
College of Computers Sciences and Math.\University of Mosul
) are called Standard Wolfe Conditions (SDWC).Notice that if equation (8) satisfied then always there exists 0 and (9) will be satisfied according to the theorem (1) given later.If we wish to find a point k a , which is closer to a solution of the one dimensional problem ) .......Than a point satisfying (
direction, which is characterized by low memory requirements and strong local and global convergence properties[3 and 12].