A Modified Conjugate Gradient Method with Global Convergence Property for Unconstrained Optimization

In this paper, a modified formula for DL (Dai-Liao) is proposed for conjugate gradient method of solving unconstrained optimization problem. The new method has sufficient descent and global convergence properties. Numerical results show that this new method is very efficient compared with other similar methods in the same filed.


1-Introduction
The conjugate gradient method presents a major contribution to the panoply of methods for solving large-scale unconstrained optimization problems.They are characterized by low memory requirements and have strong local and global convergence properties.For general unconstrained optimization problems.
Where f: R n R is a continuously differentiable function, bounded from below, starting.From an initial guess, a nonlinear conjugate gradient algorithm generates a sequence of points ( k x ), according to the following recurrence formula: Where k is the step length, usually obtained by the Wolfe line search: where 1 0 ,which known as weak Wolfe condition (W.W.C.) and for strong Wolfe condition (S.W.C.) is defined by: See (Nocedal and Wright, 1999).
Dai and Yuan (Dai and Yuan, 1996) showed that the conjugate gradient method are globally convergent when they generalized, the absolute value in ( 6) is replaced by pair of inequalities. where The special case where k is a scalar and , since 1952, there have been many formulas for the scalar, for example: The method ( 2) and ( 8) is called the linear conjugate methods, within the framework of linear conjugate gradient methods, the conjugate condition is defined by: , where is symmetric positive definite matrix.
On the other hand, the method ( 2) and ( 8) is called the nonlinear conjugate gradient method for several unconstrained optimization problem.The conjugate condition is replaced by: 0 y d k T 1 k (13) holds for strictly convex quadratic objective function.The extension of the conjugacy condition was studied by Perry (Perry,1978), he tried to accelerate the conjugate gradient method by incorporating the secondorder information into it.Specifically, he used the quasi-Newton (QN) method the search direction d k can be calceolate in the form:  , by ( 14) and (15) we have (16) Eq ( 16) is called Perry's condition, which implies (13) hold if line search is exact, since in this case .

New formula for Beta and Algorithm
An idea is multiplying of the updating H and the favorable in some asses especially when the number variable are large (Scales, 1985).
In this paper we use the scalar by Al-Assady (Al-Assady,1997) which defined by: Now to drive the new methods using (8) Where k is defined in (18), i.e.: Observing that this new formula contains not only gradient value information, but also function value information at the present and where t is constant and [0<t<1] in this paper we replace this parameter by the scalar , which can be viewed as adaptive of Dai-Liao computational schemes, corresponding to t.
The following assumptions are often used in the studies of the conjugate gradient methods.

Assumption (1)
i) The level set is bounded, and f(x) is bounded below in .
ii) In some neighborhood N of , f(x) is continuously differentiable and its gradient is Lipchitz continuous namely, there exists a constant such that: (24) It follows directly from Assumption (1) that there exists two positive constants D and such that (25)

Convergence Analysis of the New Method:
Since the conjugate gradient methods belong to the descent methods for solving unconstrained optimization problems, the new k should be chosen such that 0 if the line search is used.
Furthermore, due to the sufficient descant condition

Theorem:
Suppose that Assumption (1) holds and k satisfies the strong Wolfe condition ( 5) and ( 6),then the search direction (8) where

Global Convergence Property for Convex Functions
If f is a uniformly convex function, there exists a constant >0 such that: We can rewrite (30) in the following manner:

Global Convergence for General Nonlinear Functions
For general nonlinear functions, we need to prove that the gradient of the new method cannot be bounded away from zero, we establish a bounded for the change ( ) in the normalized direction w k = d k / , (Nocedal and Gillbert, 1992) Also, we assume that there exists a positive constant > 0 such (33) where to inverse Hessian, with quasi-Newton condition which is defined by: scaling k before the update taking place.i.e. for every k 1 the scalar Newton direction, is defined by: (17) Where is an approximation to inverse Hessian, and is scalar, this scalar is added to make the sequence and efficiency as problem dimension increase.The poor-scaling is an imbalance between the values of the function and change in x. the function value may be change very little even though x is changing by good scaling factor for previous step.If the function is quadratic and the line search is exact the new formula is equal to HS k .However, we consider general nonlinear function and inexact line search.:If we compare the new version new k with Dai and Liao (Dai-Liao, 2001) computational scheme:

DL
Sataiwa,2005) Dai et al (Dai et al, 1999) proved that for any conjugate gradient method with strong Wolfe condition the followings results holds.

Then 3 . 3
Suppose that Assumption (1) hold and consider any CG-methods (2), where d is a descent direction and is obtained by the strong Wolfe condition if Theorem: Suppose that Assumption (1) hold and that f is a uniformly convex function.the new algorithm of the form (2) (8) and (22) where d k satisfies the descent condition and k is obtained by the strong Wolfe c o n d i t i o n ( 5 ) a n d ( 6 ) s a t i s f i e s t h e g l o b a l c o n v e r g e n c e ( i .e .