Global convergence properties of a Dai-Liao-type CGM for unconstrained optimization

A popular optimization technique called the conjugate gradient method (CGM) is renowned for its effectiveness in addressing problems involving unconstrained optimization. Several conjugate gradient (CG) techniques have been proven to possess global convergence properties both theoretically and numerically. The Dai-Liao-type CGM is a variant that incorporates certain modifications to enhance its convergence properties. This paper examines the global convergence properties of a Dai-Liao-type CGM for unconstrained optimization problems. Theoretically, this study investigates the conditions under which the method ensures convergence to the global minimum of the objective function, focusing on the algorithm’s descent directions, the necessary reduction in objective function values, and termination criteria. A numerical experiment is conducted on a set of unconstrained optimization problems to validate the theoretical results obtained in this work. The numerical findings of this study demonstrate the robustness and reliability of the Dai-Liao-type CGM, showing its ability to find the global optimal solution in a wide range of unconstrained optimization problems.


INTRODUCTION
Optimization entails choosing the best out of numerous possibilities [1].Optimization theory is widely applied to optimal control problems [2,3].In this paper, we consider a nonlinear, unconstrained minimization problem: where the objective function f : R n → R is smooth and its gradient is represented by ∇f (x) = g(x).To solve Eq. ( 1), the conjugate gradient method (CGM) is usually employed.The conjugate gradient (CG) method is developed iteratively by: where φ n is the steplength, and d n is the direction of search.At the first iteration, the direction of search is the steepest descent one, that is, d 0 = −g 0 .Consequently, d n is defined as: where β n is a scalar known as the CG update parameter.Some of the most frequently used update parameters are: namely, Fletcher-Reeves (FR) method [4], Hestenes-Stiefel (HS) method [5], Dai-Yuan (DY) method [6], and Polak-Ribiere-Polyak (PRP) method [7,8], respectively.
In the above classical CG formulae, the difference between two gradients is denoted by y n−1 = g n − g n−1 , ∥.∥ denotes norm, T denotes transpose, ∥g n ∥ 2 = g T n g n , and ∥g n−1 ∥ 2 = g T n−1 g n−1 .The convergence characteristics of the aforementioned CG methods have been investigated in Refs.[9][10][11][12], where the FR and DY methods have been shown to possess strong convergence properties with modest performances, while the PRP and HS methods were proven to have better computational performances, which may not be generally convergent.
Despite the DY method's strong convergence property, its efficacy can be affected by the line search technique and parameters used.The convergence may be affected by the precision of the line search in identifying the step size, and for non-convex functions, its convergence may be slower, and it could become trapped in local minima or saddle points [13,14].To overcome the drawbacks of the DY technique, Jiang & Jian in Ref. [15] presented a new hybrid CGM, called an improved Dai-Yuan method, where the update parameter is given by , which they proved to be convergent under the strong Wolfe line search.
Many researchers have developed new methods in response to the need to produce some CG methods with good convergence properties that also perform well computationally.The Dai-Liao type CGM has drawn attention within the CGM family due to improvements targeted at improving its convergence features.Dai & Liao (DL) in Ref. [16] proposed a novel CG approach that can be deemed an improvement over the HS method by incorporating the conjugacy condition: with the resulting CG parameter given by where t ≥ 0. Nevertheless, as the DL-CG method's performance is dependent on the parameter t, the optimal value of t in Eq. ( 9) is still being taken into consideration [17,18].Researchers have dedicated significant efforts to enhance the effectiveness of the DL method.For example, the authors in Ref. [17] suggested the following selections for the parameter t : Salihu et al. [19] employed the optimal choice of the parameter t to modify CG methods using the classical HS and FR methods, while Lofti & Hosseini [20] presented a new value t based on a modified BFGS method.
The fulfillment of the sufficient descent criterion: is very crucial to the global convergence of CG methods.Akinduko [21] proposed a new hybrid CGM of a Dai-Liao type, which was proved to fulfill the condition (10) and given by: Furthermore, Onuoha [22] proposed a new hybrid CG method with sufficient descent property by combining the DY and DL methods; the resulting update parameter is given by: Recently, global convergence results for CG methods under various inexact line searches have been established in Refs.[23][24][25].To demonstrate the convergence of CG techniques, the steplength φ n typically needs to meet the strong Wolfe (SW) conditions put forth by Wolfe [26] and provided by: and where Yousif et al. [27] introduced a criterion that ensures the establishment of the descent search direction property and the global convergence of CG techniques under SW line search.Many researchers (see e.g., Refs [28][29][30][31][32]) have demonstrated that several numerical techniques for unconstrained optimization converge under the SW condition.
This work aims to examine the global convergence features of the CGM, following the proposal of Onuoha [22].Specifically, it focuses on how much the Dai-Liao-type CGM attains the global minimum of the objective function.The continuous quest for optimization techniques exhibiting robust theoretical convergence characteristics and practical efficacy in practical applications serves as the driving force behind this research.

GLOBAL CONVERGENCE ANALYSIS OF HDYDL-CGM
In this section, the global convergence results for the hybrid method proposed in Ref. [22] are provided.The method was implemented based on the following algorithm: ALGORITHM 2.1: HDYDL METHOD Step 1: Input x 0 , set n = 0, d 0 = −g 0 .
Step 7: Make n := n + 1, and go back to Step 2.
The following lemma is useful for the establishment of the global convergence of the HDYDL method: Lemma 2.1 ([22]).The HDYDL method satisfies the sufficient descent condition (10) where, Definition 2.1.A CGM is said to be globally convergent if, starting from any given initial iterate x 0 , it meets the condition: where g n is the gradient of the objective function f at the point x n .
To determine the global convergence of the CG method, the following assumptions are made: Assumption 2.1 implies that there is a constant B such that: Assumption 2.2.In some neighborhood Z of ω, f is Lipschitz continuously differentiable, that is, there exists a positive constant M such that: Lemma 2.2.Suppose that Assumption 2.1 is satisfied and consider any method of the forms ( 2) and ( 3), where d n is a descent search direction and φ n satisfies the SW conditions ( 13) and ( 14), then Proof.The Lipschitz condition (17) implies that: By Eq. ( 2) Therefore, Thus, Eq. ( 14) implies that: n from both sides of the inequalities above we have: which also implies that: The above inequality, combined with Eq. ( 20), implies that Eq. ( 13) implies that where Summing the above from n = 0 to n = m, we have  (2) and (3), where φ n is obtained with the SW conditions ( 13) and (14). then Proof.Suppose Eq. ( 22) is not true, then there exists a constant u > 0 such that: Using Eq. ( 10) with k given by Eq. ( 15), Eqs. ( 21) and ( 23), we have which contradicts Lemma 2.2.Thus, lim n→∞ inf ∥g n ∥ = 0.

NUMERICAL RESULTS AND DISCUSSION
In this section, a report on the performance of the HDYDL method in comparison with the DY and DL methods is presented.
To further validate the method's efficiency, it is compared to the NM method, a recently proposed Dai-Liao-type CGM given by Akinduko [21].A total of 20 unconstrained optimization problems taken from Bongartz et al. [34] and Andrei [35] were solved.For each solved test problem (TP), four initial points and dimensions (Dim) ranging from 5000 to 10000 were considered, totaling 158 computations.The computations were carried out on a computer with the following specifications: 4 GB of RAM, 2.2 GHz processor speed, and the Windows 10 operating system.The basis of comparison includes the computational time (CPU) and the number of iterations.The iterations were terminated when ∥g n ∥ ≤ 10 −6 or the number of iterations went beyond 2000.The notation "F" is used to denote a failed iteration.For the step size computation, the SW search technique was used.
Table 1 presents the solved test problems and their initial points, while Tables 2-5 give the details of the numerical results.In the presentation of results in Tables 2-5, the form (a − b) is used in the column labeled TP, where TP represents a particular solved test problem, a denotes the serial number of the TP as it appears in Table 1, and b denotes the initial point accompanying a as it appears in Table 1.Figures 1 and 2 depict the CPU and the iteration profiles, respectively.This is based on the method of Dolan & More [36].The y-axes of the Figures show the percentage of successfully solved problems, while the top curve represents the fastest method.Based on this fact, the percentages for the compared methods are recorded: 100% for HDYDL, 88.6% for DY, 70.3% for DL, and 74.7% for NM.These results demonstrate that the HDYDL method is the most successful of the four under consideration, solving every test problem regardless of the starting point.The DY and NM methods follow this, while the DL method lags in performance.

CONCLUSION
This work investigates the global convergence properties of the Dai-Liao-type CGM for unconstrained optimization problems.
It looked into the convergence requirements, such as termination criteria, descent directions, and reduction of the objective function.This work adds to the increasing body of knowledge on optimization techniques by concentrating on the Dai-Liao-type CGM.It provides practitioners with useful information to help them select efficient strategies for solving unconstrained optimization problems.The numerical experiments serve as practical validation, bridging the gap between theory and real-world applicability.Comparing the HDYDL approach to the DY, DL, and NM methods, the results show how resilient and reliable the HDYDL method is in locating global optimal solutions.

Figure 1 .
Figure 1.Iteration Profile of the HDYDL Method in Comparison to the DY, DL, and NM Methods

Figure 2 .
Figure 2. CPU Profile of the HDYDL Method in Comparison to the DY, DL, and NM Methods