Restart particle swarm optimization with velocity modulation: a scalability test

Soft Computing manuscript No. (will be inserted by the editor) Restart Particle Swarm Optimization with Velocity Modulation: A Scalability Test Jos´ e Garc´ ıa-Nieto ·Enrique Alba Received: date / Accepted: date Abstract Large scale continuous optimization problems have more and more relevance in current benchmarks since they are more representative of real-world problems (bioinformatics, data mining, etc.). Unfortunately, the performance of most available optimization algorithms deteriorates rapidly as the dimensionality of the search space increases. In particular, Particle Swarm Optimization is a very simple and effective method for continuous optimization. Nevertheless, this algorithm usually suffers from an unsuccessful performance on large dimension problems. In this work, we incorporate two new mechanisms to the Particle Swarm Optimization with the aim of enhancing its scalability. First, avelocity modulation method is applied in the movement of particles in order to guide them within the region of interest. Second, a restarting mechanism avoids the early convergence and redirects the particles to promising areas in the search space. Experiments are carried out in the scope of this Special Issue to test scalability. The results obtained show that our proposal is scalable in all functions of the benchmark used, as well as numerically very competitive with regards to other compared optimizers. Keywords Continuous Optimization ·Scalability ·Particle Swarm Optimization ·Large Scale Benchmarking Authors acknowledge funds from the Spanish Ministry of Sciences and Innovation (MICINN) and FEDER under contract TIN2008-06491C04-01 (M* project http://mstar.lcc.uma.es) and CICE, Junta Andalucia, under contract P07-TIC-03044 (DIRICOM project http://diricom.lcc.uma.es). Jos´ e Garc´ ıa-Nieto is supported by grant BES-2009-018767 from the MICINN J. Garc´ ıa-Nieto and E.Alba University of M´ alaga, Dept. Lenguajes y Ciencias de la Computaci´ on, Campus de Teatinos, E.T.S.I. Inform´ atica, M´ alaga - 29071, Spain Tel.: +34 952 13 33 03 E-mail: {jnieto,eat}@lcc.uma.es 1 Introduction In the evaluation of the search capabilities of a given optimization algorithm the usual approach is to choose a benchmark of known problems, to perform a fixed number of function evaluations, and to compare the results against the ones of other algorithms in the state of art. However, while some real industry problems can have hundreds and thousands of variables, current benchmarks are normally adopted with less than a hundred decision variables (see CEC’05 [25], BBOB’09 [10] and BBOB’10 [11] testbeds). Large scale continuous optimization problems have attracted more and more interest (CEC’08 [28], ISDA’09 [15], and CEC’10 [27]) since they introduce a high complexity to the optimization process. Issues like the exponential increment of the solution space, as well as the change that some problems suffer from their own characteristics with the scale, can deteriorate quickly the performance of our optimization algorithms [22]. This way, we can study certain mechanisms that show the best performance in short scale optimization problems, which is the case of the covariance matrix in GCMA-ES [2], but with an unsuitable behavior for high dimensional functions (more than 100 variables). A different performance can be observed in simple algorithms like MTS [30], which combines several local search strategies using a small population. MTS was the best in the special session of large scale optimization of CEC’08 [28], where functions with a thousand of variables were tackled. All this motivates us to deeply analyze the scalable capacities of optimization algorithms. In particular, Particle Swarm Optimization (PSO) [17] is a very simple and effective method for continuous optimization. Nevertheless, this algorithm is characterized by an early convergence behavior, mainly produced by the overinfluenced best solution and its relative facility to fall in local optima [19,31]. For this rea- 2 son, PSO usually suffers from an unsuccessful performance on large dimension problems. In this work, we have incorporated two mechanisms to the Particle Swarm Optimization with the aim of enhancing its scalability. First, a velocity modulation method is applied in the movement of particles in order to guide them within the feasible region. Second, a restarting mechanism avoids the early convergence and redirects the particles to promising areas in the search space. To evaluate the scalability of the resulting approach, we have followed the experimental framework proposed in this Special Issue on Scalability of Evolutionary Algorithms and other Metaheuristics for Large Scale Continuous Optimization Problems (in URL http://sci2s.ugr.es/eamhco/CFP.php). We also studied the influence of both velocity modulation and restarting mechanisms to show real insights of the improvement of our proposal, called Restart PSO with Velocity Modulation (RPSO-vm), regarding the basic PSO. The results obtained confirm us that RPSO-vm is scalable in all functions of the benchmark used, as well as highly competitive in comparison with PSO and other well-known efficient optimizers. The remaining of this paper is organized as follows. Next section presents basic preliminary concepts. In Section 3, the RPSO-vm algorithm is introduced. Section 4 describes the experimentation procedure with the benchmark of functions and the parameter settings. In Section 5, experimental results are reported with comparisons, analyses, and discussions. Finally, concluding remarks are given in Section 6. 2 Preliminaries Particle Swarm Optimization (PSO) [5,17] has been successfully used in many problems of real parameter optimization [4,8,16,19,20] since it is a well adapted algorithm for continuous solution encoding. Basically, a continuous optimization problem consists in this: find x∗such that ∀xf(x∗)≤f(x)(minimization). Here, f(.)is a function in a real space domain that models an optimization problem, x={x1,x2,...,xDIM}is a solution for such problem, and DIM is the number of variables with xi∈[xlow,xupp](1 ≤i≤DIM). Finally, xlow,xupp ∈R correspond to lower (low) and upper (upp) limits of the variable domain, respectively. In PSO, each potential solution to the problem is given by a particle position and the population of particles is called swarm. In this algorithm, each particle position xiis updated each generation tby means of the Equation 1. xi(t+1)←xi+vi(t+1)(1) where factor vi(t+1)is the velocity of the particle and is given by vi(t+1)←vi(t) + ϕ 1·UN(0,1)·(pi(t)−xi(t)) + ϕ 2·UN(0,1)·(bi(t)−xi(t)) (2) In this formula, pi(t)is the personal best solution that the particle ihas stored so far, bi(t)is the global best particle (leader) that the entire swarm has ever generated. Finally, ϕ 1and ϕ 2are specific parameters which control the relative effect of the personal and global best particles, and UN(0,1) is a uniform random value in [0,1]which is sampled anew for each component of the velocity vector. Velocity constriction is one of the main mechanism used for controlling the movement of particles through the search space and for balancing the exploration-exploitation tradeoff of the algorithm. Therefore, an efficient movement strategy of particles could help the PSO to find an optimum even in large scale problems. We can find several velocity constriction mechanisms in the literature. Three of the most popular are the following ones: •VMAX factor. The simplest method for regulating the velocity lies in the maximum (and minimum) velocity delimitation. This mechanism uses a given value Vmax for adjusting the maximum velocity each particle undergoes each generation step. According to this method, if the new velocity exceeds Vmax then this value is aggregated to the new position calculation (Equation 1) instead of the corresponding new velocity. •Inertia weight ( ω ) [24,26] is one of the most used methods in PSO for controlling the velocity of particles in their movement. This parameter controls the trade-off between global and local search. Then, a high inertia value provides the algorithm with exploration capability and a low inertia promotes the exploitation. The inertia weight linearly changes during the optimization process (of the algorithm) by using the following equation: ω ← ω max −( ω max − ω min)·#gcurrent #gtotal (3) This way, at the beginning of the process a high inertia ( ω max) value is introduced (for exploration) which decreases until reaching the lowest value ( ω min). The inertia value is incorporated in the velocity calculation as follows: vi(t+1)← ω ·vi(t)+ ϕ 1·UN(0,1)·(pi(t)−xi(t)) + ϕ 2·UN(0,1)·(bi(t)−xi(t)) (4) •A third velocity constriction method was introduced in [3]. In that work, the author indicates that the use of a constriction factor ( χ ) may be necessary to ensure convergence of the particle swarm algorithm. A detailed discussion of the constriction factor is beyond the scope of this work in [3], but a simplified method of incorporating it appears in Equation 5, where χ is a function of ϕ 1and ϕ 2as reflected in Equation 6. vi(t+1)← χ [vi(t)+ ϕ 1·UN(0,1)·(pi(t)−xi(t)) + ϕ 2·UN(0,1)·(bi(t)−xi(t))] (5) χ =2 ¯ ¯ ¯2− ϕ −p ϕ 2−4 ϕ ¯ ¯ ¯ , ϕ = ϕ 1+ ϕ 2, ϕ >4 (6) 3 3 The Algorithm Our proposal, RPSO-vm, consists in running a PSO algorithm in which we have incorporated two main ideas: velocity modulation and restarting mechanisms. Using the velocity modulation, the algorithm controls that the overall movement calculated in each evolution step and for each particle position does not exceed the limits (xlow,xupp) of the problem domain. First, after calculating the new velocity value (vj aux) RPSO-vm performs a modulation procedure as showed in Algorithm 1. The velocity vector magnitude (b vi(t)) is then bounded, which limits the given particle to move far from the interest area. These steps are calculated in Algorithm 2 in Lines 7 and 8. Second, once obtained the new velocity vj i(t+1), the overall movement is calculated, also controlling that the new particle position (xj aux) does not exceed the problem limits. If this happens, the new position is recalculated by subtracting the new velocity to the old particle position (Lines 10 to 14 in Algorithm 2). Algorithm 1 Pseudocode of velmod procedure 1: if xj low >vj aux then 2: vj i(t+1)←xj low 3: else if vj aux ≥xj upp then 4: vj i(t+1)←xj upp 5: end if 6: Output: vj i(t+1)/*constricted velocity*/ A second phase of RPSO-vm concerns the restarting strategy. Similar to other known algorithms like CHC [6] and G-CMA-ES [2], our proposal is stopped whenever one stopping criterion described below is met, and a restart is launched. The decision on when to restart the algorithm is made according to two independent criteria: 1. Stop if the standard deviation of the fitness values of particles in the entire swarm is smaller than 1e−3. In this case, the particles are restarted by randomly initializing their positions with a probability of 1/DIM (Lines 18 to 26 in Algorithm 2). 2. Stop if the overall change in the objective function value is below 1e−8 for 10 ·DIM/size(S)generations. In this case, the particles are restarted by calculating their derivatives to the global best position band dividing them into two (Lines 27 to 33 in Algorithm 2). This way, we force the particles to go to the best but avoiding the global convergence. Applying the first restarting criteria, our algorithm tries to mitigate the early stagnation that basic PSO usually suffers, specially in multimodal functions. In spite of working with high inertia and/or high social influences ( ϕ 1and ϕ 2), which moves the particles to distant positions, the PSO tends to be easily trapped in unproductive regions. This drawback is specially sensitive in functions with multiple local optima such as Rastriging and its hybrids. The second restarting criteria is based on the existence of plateaus and quite regular regions in functions like Rosenbrock, Schwefel, and their hybrids, that makes the PSO to spent a number of function evaluations (with time and computing resources) without an effective improvement. In this case, particles tends to spread them in the search space avoiding the influence of the best particle. Therefore, after certain number of function evaluations without improvement, the particles are moved to their derivatives with regard to the best position. Algorithm 2 Pseudocode of RPSO-vm 1: t←0 2: initialize(S(t)) /* Swarm S(0)*/ 3: while not stop condition is met (MAXIMUM(t))do 4: /****************** Particle Swarm ******************/ 5: for each particle position xi(t)of the swarm S(t)do 6: for each variable jof the particle position xi(t)do 7: vj aux ← ω ·vj i(t) + ϕ 1·UN(0,1)·(pj i(t)−xj i(t)) + ϕ 2·UN(0,1)·(bj(t)−xj i(t)) 8: vj i(t+1)←velmod(vj aux) 9: xj aux ←xj i(t) +vj i(t+1) 10: if xj low <xj aux ≤xj upp then 11: xj i(t+1)←xj aux 12: else 13: xj i(t+1)←xj i(t)−vj i(t+1) 14: end if 15: end for 16: end for 17: /******************** Restarting ********************/ 18: if std(S)<1e−3then 19: for each particle position xi(t)of S(t)(with xi(t)6=b(t))do 20: for each variable jof the particle position xi(t)do 21: if rj(t)<1/DIM (with rj(t)∈[0,1])then 22: xj i(t+1)←xj low +UN(0,1)·(xj upp −xj low) 23: end if 24: end for 25: end for 26: end if 27: if change(f it(b)) <1e−8 for (10 ·DIM)/size(S)steps then 28: for each particle position xi(t)of S(t)(with xi(t)6=b(t))do 29: for each variable jof the particle position xi(t)do 30: xj i(t+1)←(bj(t)−xj i(t))/2 31: end for 32: end for 33: end if 34: t←t+1 35: end while 36: Output: b/*The best solution found*/ Algorithm 2 shows the complete pseudocode of the RPSOvm algorithm developed for this work. First, an initialization process of all particles in the swarm Sis carried out. After this, each evolution step the particle’s positions are updated 4 following the velocity variation model of the equations previously explained (Lines 5 to 16). If stopping criterions are reached, the algorithm restarts modifying the particles, excepting the best one (Lines 18 to 33). Finally, the algorithm returns the best solution found during the whole process. 4 Experimental Setup In this section, we present the experimental methodology and statistical procedure followed to evaluate and to compare our proposal. This experimentation has been defined in the scope of the Special Issue on Scalability of Evolutionary Algorithms and other Metaheuristics for Large Scale Continuous Optimization Problems (SOCO’10), available in URL http://sci2s.ugr.es/eamhco/CFP.php. We have implemented our RPSO-vm in C++ using the MALLBA library [1], a framework of metaheuristics. The benchmark of functions was tackled including the C-code provided in this Special Issue to our implementation of RPSOvm. A complete package of this software is available in the new version release of MALLBA1. Following the specifications of the SOCO’10 experimental procedure, we have performed 25 independent runs of RPSO-vm for each test function and dimension. The study has been made with dimension D=50, 100, 200, 500, and 1000 continuous variables. The measures provided are the Average, the Maximum, the Minimum, and the Median of error of the best individuals found in the 25 runs. For a solution x, the error measure is defined as: f(x)−f∗, where f∗is the optimum fitness of the function. The maximum number of fitness evaluations has been stated to 5000 ·D, which constitutes the stop condition of each run. To analyze the results we have used non-parametric [23] tests. These tests use the mean ranking of each algorithm. We have applied them since several times the functions might not follow the conditions of normality and homoskedasticity to apply parametric tests with security [7]. In particular, we have considered the application of the Iman and Davenport’s test, and Holm’s test as post-hoc procedure. The former is used to know beforehand if there are statistically relevant differences in compared algorithms. In that case, a post-hoc procedure, the Holms test, is then employed to know which algorithms are statistically worse than the reference algorithm with the best ranking. 4.1 Benchmark Functions The test suite elaborated for this Special Issue is composed by 19 functions with different properties [14]: unimodal, 1MALLBA Library http://neo.lcc.uma.es/mallba/easy-mallba/html/mallba.html. Directory Mallba/rep/PSO/soco2010 multimodal, separable, non-separable, shifted, and hybrid composed. Functions f1 to f6 were defined for CEC’08 [28] and functions f7 to f11 were defined for ISDA’09 [15] (and shifted for SOCO’10), where the previous ones were also used. Finally, functions f12 to f19 have been created specifically for this Special Issue. Table 1 shows their names, bounds, and optimum values. We can describe several properties of the functions that we consider interesting. –Functions f1 and f2 are shifted unimodal, functions f3 to f6 are shifted multimodal and functions f7 to f11 are shifted unimodal. –Functions f2, f3, f5, f9, and f10 are non-separable. That is specially interesting to analyze if our proposal obtains good results in non-separable functions since we can observe its capacity of managing correlated variables, a typical property in real world problems. –Functions f12 to f19 are hybrid composition functions. They have been generated by composing (⊕) two functions, one or both of them non-separable. For these compositions, functions f7 to f11 have been used in their non-shifted versions (NS). A composition uses a splitting mechanism to graduate the proportion (in parentheses in Table 1) of non-separable variables in the complete search space. Table 1 SOCO’10 test suite of functions Number Name Intervals f∗ f1 Shifted Sphere [-100, 100] -450 f2 Shifted Schwefel 2.21 [-100, 100] -450 f3 Shifted Rosenbrock [-100, 100] 390 f4 Shifted Rastrigin [-5, 5] -330 f5 Shifted Griewank [-600, 600] -180 f6 Shifted Ackley [-32, 32] -140 f7 Shifted Schwefel 2.22 [-10, 10] 0 f8 Shifted Schwefel 1.2 [-65.536, 65.536] 0 f9 Shifted Extended f10 [-100, 100] 0 f10 Shifted Bohachevsky [-15, 15] 0 f11 Shifted Schaffer [-100, 100] 0 f12 Hybrid NS f9⊕f1 (0.25) [-100, 100] 0 f13 Hybrid NS f9⊕f3 (0.25) [-100, 100] 0 f14 Hybrid NS f9⊕f4 (0.25) [-5, 5] 0 f15 Hybrid NS f10⊕NS f7 (0.25) [-10, 10] 0 f16 Hybrid NS f9⊕f1 (0.50) [-100, 100] 0 f17 Hybrid NS f9⊕f3 (0.75) [-100, 100] 0 f18 Hybrid NS f9⊕f4 (0.75) [-5, 5] 0 f19 Hybrid NS f10⊕NS f7 (0.75) [-10, 10] 0 Table 2 Parameter setting used in RPSO-vm Description Parameter Value Swarm size size(S)10 Inertia weight ω 0.0←0.1 Individual coefficient ϕ 11.5 Social coefficient ϕ 21.5 5 Table 3 Maximum, Minimum, Median, and Mean Errors obtained by RePSOVM for all dimensions Dimension Value f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 50 Maximum 2.84E-14 1.58E-02 1.86E+04 2.84E-14 3.20E-01 1.78E-11 1.78E-14 2.13E+03 6.77E+00 0.00E+00 Median 2.68E-14 6.60E-03 1.90E+01 2.66E-14 7.36E-02 2.66E-13 0.00E+00 1.04E+03 3.12E-06 0.00E+00 Minimum 1.87E-14 4.36E-03 5.21E-02 0.00E+00 0.00E+00 1.24E-13 0.00E+00 4.37E+02 0.00E+00 0.00E+00 Mean 2.62E-14 7.54E-03 1.75E+03 2.30E-14 9.53E-02 1.49E-12 1.14E-15 1.06E+03 2.94E-01 0.00E+00 100 Maximum 2.84E-14 2.92E-01 9.65E+03 2.84E-14 3.18E-01 9.52E-11 0.00E+00 1.89E+04 2.80E+00 0.00E+00 Median 2.80E-14 2.01E-01 1.19E+02 2.77E-14 8.80E-02 6.32E-13 0.00E+00 1.05E+04 2.58E-05 0.00E+00 Minimum 1.57E-14 1.17E-01 3.57E-01 1.17E-14 1.31E-14 2.42E-13 0.00E+00 6.82E+03 5.93E-07 0.00E+00 Mean 2.66E-14 1.98E-01 1.42E+03 2.61E-14 1.07E-01 4.76E-12 0.00E+00 1.09E+04 1.85E-01 0.00E+00 200 Maximum 2.84E-14 2.49E+00 6.60E+03 2.84E-14 7.74E-01 8.20E-12 0.00E+00 7.06E+04 2.51E+01 0.00E+00 Median 2.75E-14 2.01E+00 6.80E+01 2.74E-14 1.18E-01 1.89E-12 0.00E+00 5.05E+04 1.48E-03 0.00E+00 Minimum 0.00E+00 1.65E+00 8.75E-02 0.00E+00 1.35E-14 6.96E-13 0.00E+00 3.81E+04 1.35E-06 0.00E+00 Mean 2.53E-14 2.00E+00 1.03E+03 2.43E-14 1.73E-01 2.95E-12 0.00E+00 5.23E+04 1.67E+00 0.00E+00 500 Maximum 2.84E-14 1.81E+01 1.47E+04 2.84E-14 6.01E-01 7.01E-12 1.13E-14 3.62E+05 2.28E+01 0.00E+00 Median 2.82E-14 1.71E+01 1.38E+02 2.77E-14 2.52E-01 2.59E-12 0.00E+00 3.00E+05 1.81E+00 0.00E+00 Minimum 1.73E-14 1.49E+01 2.70E-02 0.00E+00 1.42E-14 1.21E-12 0.00E+00 2.47E+05 1.90E-03 0.00E+00 Mean 2.65E-14 1.67E+01 1.13E+03 2.44E-14 2.54E-01 3.14E-12 0.00E+00 3.00E+05 4.85E+00 0.00E+00 1000 Maximum 2.84E-14 4.69E+01 2.07E+03 3.03E-13 8.68E-01 2.41E-11 1.55E-14 1.14E+06 3.17E+01 0.00E+00 Median 2.82E-14 4.29E+01 1.37E+02 2.78E-14 1.12E-01 3.75E-12 0.00E+00 9.21E+05 9.84E+00 0.00E+00 Minimum 2.07E-14 3.99E+01 1.85E+01 2.27E-14 1.22E-14 2.60E-12 0.00E+00 7.39E+05 6.45E-02 0.00E+00 Mean 2.72E-14 4.29E+01 3.21E+02 4.81E-14 2.14E-01 4.92E-12 0.00E+00 9.35E+05 1.17E+01 0.00E+00 Dimension Value f11 f12 f13 f14 f15 f16 f17 f18 f19 50 Maximum 3.07E-01 2.14E+00 1.49E+04 1.04E+00 0.00E+00 1.05E-09 9.08E+03 1.26E+00 0.00E+00 Median 3.84E-06 0.00E+00 1.43E+01 1.72E-14 0.00E+00 1.68E-11 5.06E+01 3.18E-09 0.00E+00 Minimum 0.00E+00 0.00E+00 2.67E-02 0.00E+00 0.00E+00 2.20E-12 9.41E-01 4.60E-10 0.00E+00 Mean 1.68E-02 8.58E-02 6.57E+02 6.81E-02 0.00E+00 7.88E-11 8.73E+02 5.05E-02 0.00E+00 100 Maximum 3.83E+00 2.30E-13 2.56E+04 1.20E+00 0.00E+00 1.11E-06 3.56E+04 1.61E+00 0.00E+00 Median 1.84E-05 0.00E+00 1.06E+02 4.38E-14 0.00E+00 4.64E-10 1.51E+02 5.04E-08 0.00E+00 Minimum 2.58E-08 0.00E+00 5.36E-02 0.00E+00 0.00E+00 5.17E-12 1.71E-01 1.46E-09 0.00E+00 Mean 4.61E-01 1.60E-14 2.25E+03 1.27E-01 0.00E+00 4.87E-08 1.76E+03 1.36E-01 0.00E+00 200 Maximum 3.73E+00 1.16E+00 1.23E+05 9.95E-01 0.00E+00 1.29E+01 2.40E+05 3.73E+00 0.00E+00 Median 5.47E-02 2.79E-14 1.48E+02 3.23E-12 0.00E+00 1.02E-09 2.77E+02 2.60E-08 0.00E+00 Minimum 4.37E-05 0.00E+00 7.56E-01 1.50E-14 0.00E+00 5.35E-11 1.31E-01 3.33E-09 0.00E+00 Mean 5.66E-01 4.64E-02 1.17E+04 7.96E-02 0.00E+00 5.40E-01 2.08E+04 1.50E-01 0.00E+00 500 Maximum 1.56E+01 2.98E-07 1.79E+04 1.36E+01 0.00E+00 3.04E+01 8.31E+03 1.41E+01 0.00E+00 Median 3.95E+00 2.75E-13 7.20E+01 2.50E-11 0.00E+00 3.46E-08 2.16E+01 7.80E-01 0.00E+00 Minimum 1.70E-03 0.00E+00 2.74E-01 2.05E-13 0.00E+00 6.84E-10 7.85E-01 1.44E-08 0.00E+00 Mean 4.88E+00 1.32E-08 1.33E+03 1.29E+00 0.00E+00 2.12E+00 5.72E+02 2.47E+00 0.00E+00 1000 Maximum 3.67E+01 1.16E-08 2.78E+04 2.60E+00 0.00E+00 7.53E+00 3.28E+04 6.52E+00 0.00E+00 Median 9.61E+00 2.02E-12 2.27E+02 4.07E-08 0.00E+00 2.19E-06 4.02E+01 1.33E+00 0.00E+00 Minimum 7.67E-02 2.98E-14 4.16E+01 5.39E-13 0.00E+00 6.27E-09 1.79E+00 2.93E-07 0.00E+00 Mean 1.10E+01 1.00E-09 1.93E+03 5.27E-01 0.00E+00 9.50E-01 2.82E+03 1.80E+00 0.00E+00 4.2 Parameter Settings Table 2 shows the parameter settings used to configure our proposal, RPSO-vm. These parameters were tuned in the context of the ISDA’09 special session of real parameter optimization [15] reaching results statistically similar to the best participant algorithm in that special session. These values of parameters were kept the same for all the experiments. The inertia weight changes linearly by following Equation 3. 5 Analysis of Results In this section, the results are presented and several analyses are made as follows: first, we carry out a brief analysis of the performance of our proposal in terms of the improvement obtained by both, velocity modulation and restarting mechanisms. In this sense, an additional comparison is made concerning the neighborhood topology of RPSO-vm in terms of global best versus local best guidance of particles. Second, the scalability analysis is tacked in comparison with provided results of other algorithms (DE, CHC, and G-CMAES) for all dimensions. Finally, we present the computational effort required in terms of average running time. 5.1 RPSO-vm Performance Results As specified in benchmarking requirements of this Special Issue, we show in Table 3 the Average, the Maximum, the Minimum, and the Median of the best error values found in 25 independent runs of our RPSO-vm, for each function and for each dimension. In this table, we have marked in bold face the average error values since they will be used in advance for comparisons (as recommended in this testbed). Nevertheless, we can notice that median values are frequently better than average values, especially in shifted Extended f10 (f9) and several hybrid functions (f14, f16 and f17) where the distribution of results are scattered. A first analysis consists of studying the improvement obtained by RPSO-vm with regards to basic PSO algorithm. Table 4 shows the mean errors (in 25 runs) obtained by RPSOvm in comparison with the ones of PSO only with restarting (RPSO), PSO only with velocity modulation (PSO-vm), and the basic PSO. Additionally, we have included to this comparison the standard version of PSO (SPSO 2007) consisting on the lbest PSO. This version uses a variable random topology for selecting the best neighbor (b) for each particle [9]. The resulting algorithm (lb)RPSO-vm incorporates 6 both, modulation velocity and restart mechanisms in order to obtain an as fair as possible comparison. For this specific analysis, we have only focused on 1000 variables dimension since it allows the most interesting analysis. As we can see in Table 4, RPSO-vm obtains the higher number of best error values (15 out of 19 in bold), and followed by RPSO. The other versions are clearly worse than the formers. In addition, after applying Iman-Davenport test to see whether there are significant differences between them we obtained a test value of 166.07 with a critical value of 3.77 (with α =0.05), which proves that there is an evident improvement of RPSO-vm over PSO. Table 4 Mean Errors obtained by RPSO-vm, RPSO, PSO-vm, and PSO for dimension 1000 F/D RPSO-vm RPSO PSO-vm PSO (lb)RPSO-vm f1 2.72E-14 2.69E-14 5.61E+06 5.55E+06 6.31E+06 f2 4.29E+01 4.38E+01 1.72E+02 1.72E+02 1.89E+02 f3 3.21E+02 1.26E+04 5.43E+12 5.54E+12 7.65E+12 f4 4.81E-14 3.98E-02 2.32E+04 2.32E+04 2.56E+04 f5 2.14E-01 1.77E-01 4.98E+04 5.05E+04 5.65E+04 f6 4.92E-12 5.13E-12 2.14E+01 2.14E+01 2.15E+01 f7 0.00E+00 2.00E-14 4.93E+03 4.98E+03 3.34E+27 f8 9.35E+05 9.38E+05 2.03E+07 2.53E+07 8.33E+07 f9 1.17E+01 1.34E+01 1.20E+04 1.20E+04 1.28E+04 f10 0.00E+00 0.00E+00 2.16E+05 2.16E+05 2.57E+05 f11 1.10E+01 1.33E+01 1.20E+04 1.20E+04 1.28E+04 f12 1.00E-09 1.15E-01 4.20E+06 4.18E+06 4.73E+06 f13 1.93E+03 7.38E+02 4.13E+12 4.01E+12 5.86E+12 f14 5.27E-01 6.28E-01 1.76E+04 1.78E+04 1.96E+04 f15 0.00E+00 0.00E+00 3.81E+04 3.67E+04 5.37E+18 f16 9.50E-01 7.39E-01 2.67E+06 2.70E+06 3.14E+06 f17 2.82E+03 1.42E+04 9.53E+11 9.33E+11 1.64E+12 f18 1.80E+00 3.35E+00 7.20E+03 7.20E+03 8.29E+03 f19 0.00E+00 0.00E+00 1.14E+05 1.11E+05 6.59E+12 More precisely, Table 5 contains the results of a multicomparison Holm test where we can see that RPSO-vm is statistically better than all PSO versions, excepting RPSO. In this case, RPSO-vm obtained a better ranking than RPSO but without significant differences. Therefore, the main consequence is that velocity modulation (PSO-vm) can improve the performance of basic PSO, although it is in the case of PSO with restarting method (RPSO) where a significant improvement is obtained. In the case of (lb)RPSO-vm, we suspect that the fact of using the same parameter setting specifically fine-tuned for RPSO-vm (global best) could lead this version of PSO to perform inadequately in our experiments. Table 5 Comparison of RPSO-vm versus (lb)RPSO-vm, RPSO, PSOvm, and PSO according to Holm’s multicompare test ( α =0.05) ialgorithm z p-value α /iSig.dif? 4 (lb)RPSO-vm 7.02 2.09E-12 0.012 Yes 3 PSO 4.10 4.06E-05 0.016 Yes 2 PSO-vm 4.10 4.06E-05 0.025 Yes 1 RPSO 0.41 6.81E-01 0.050 No These preliminary results lead us to definitively use both, the velocity modulation and the restarting method to design our proposed PSO for large scale optimization (RPSO-vm). 5.2 Scalability Analysis This section is focused on analyzing the capability of our RPSO-vm to scale with the dimension of the search space of each function. As proposed in this Special Issue, the scalability study is made in comparison with other well-known algorithms in the state of the art. These algorithms are a version of DE (DE/1/exp) [21], CHC [6], and G-CMA-ES [2]. The descriptions and the parameter settings of these algorithms can be found in [13]. Therefore, we first analyze the results of RPSO-vm dimension by dimension, and secondly, we made a brief study from a general point of view of the scalability behavior of RPSO-vm regarding several selected functions (f2, f9, f14, and f19) of the SOCO’10 benchmark. An initial study with all these results consists of applying an Iman-Davenport test to see if there exist significant differences between them for all considered dimensions. Table 6 shows the results of this test, where we can effectively notice that there are statistical differences in compared results. In fact, for almost all cases the test values (I.D. value) increase with the dimension, which means that there are higher differences between compared algorithms in large scales (500 and 1000) than in small dimensions (50, 100 and 200). Hence, we can known beforehand that there is an algorithm with poor scalability behavior, at least. Table 6 Results of the Iman-Davenport’s (I.D.) test of RPSO-vm and all compared algorithms for each dimension ( α =0.05) Dimension I.D. value Critical value Sig. differences? 50 13.80 2.53 Yes 100 13.43 2.53 Yes 200 12.76 2.53 Yes 500 14.37 2.53 Yes 1000 30.04 2.84 Yes Following this general point of view, Table 7 shows the results of applying a multicomparison Holm test to all mean fitness values obtained by each algorithm for each dimension. We must notice that G-CMA-ES could not obtained any result for dimension 1000 (Table 12), hence it has not been considered for comparisons regarding the largest scale. The main observation we can draw from Table 7 is that there are two algorithms: DE, and RPSO-vm that clearly show a better average distribution than the remaining ones. In addition, these ranks are kept for all dimensions. Concretely, DE reached the best rank and for this reason it has been considered as the reference algorithm for this statistical test. Nevertheless, our RPSO-vm is the only algorithm that does not shown significant statistical differences (Sig.dif) with regard to DE (the reference), and resulting the lower difference precisely in the largest dimension (1000 variables). This is an important indicator that confirms us the successful performance of our proposal in terms of scalability. The following Tables 8, 9, 10, 11 and 12 contain the results of all compared algorithms for dimensions, 50, 100, 200, 500, and 1000, respectively. The last column in these 7 Table 7 Comparison of DE versus RPSO-vm, CHC, and G-CMA-ES according to Holm’s multicompare test ( α =0.05) DIM ialgorithm z p −value α /iSig.dif? 3CHC 4.77 1.79E-06 0.016 Yes 50 2 G-CMA-ES 2.95 3.14E-03 0.025 Yes 1RPSO-vm 1.57 1.16E-01 0.050 No 3CHC 4.71 2.45E-06 0.016 Yes 100 2 G-CMA-ES 2.89 3.85E-03 0.025 Yes 1RPSO-vm 1.44 1.48E-01 0.050 No 3CHC 4.649 3.33E-06 0.016 Yes 200 2 G-CMA-ES 2.76 5.70E-03 0.025 Yes 1RPSO-vm 1.38 1.66E-01 0.050 No 3CHC 4.52 6.07E-06 0.016 Yes 500 2 G-CMA-ES 3.70 2.09E-04 0.025 Yes 1RPSO-vm 1.57 1.16E-01 0.050 No 1000 2CHC 4.62 3.77E-06 0.025 Yes 1RPSO-vm 0.97 3.30E-01 0.050 No Table 8 Mean Errors obtained by DE, CHC, G-CMA-ES, and RPSOvm for dimension 50 f/Alg. DE CHC G-CMA-ES RPSO-vm f1 0.00E+00 1.67E-11 0.00E+00 2.62E-14 f2 3.60E-01 6.19E+01 2.75E-11 7.54E-03 f3 2.89E+01 1.25E+06 7.97E-01 1.75E+03 f4 3.98E-02 7.43E+01 1.05E+02 2.30E-14 f5 0.00E+00 1.67E-03 2.96E-04 9.53E-02 f6 1.43E-13 6.15E-07 2.09E+01 1.49E-12 f7 0.00E+00 2.66E-09 1.01E-10 0.00E+00 f8 3.44E+00 2.24E+02 0.00E+00 1.06E+03 f9 2.73E+02 3.10E+02 1.66E+01 2.94E-01 f10 0.00E+00 7.30E+00 6.81E+00 0.00E+00 f11 6.23E-05 2.16E+00 3.01E+01 1.68E-02 f12 5.35E-13 9.57E-01 1.88E+02 8.58E-02 f13 2.45E+01 2.08E+06 1.97E+02 6.57E+02 f14 4.16E-08 6.17E+01 1.09E+02 6.81E-02 f15 0.00E+00 3.98E-01 9.79E-04 0.00E+00 f16 1.56E-09 2.95E-09 4.27E+02 7.88E-11 f17 7.98E-01 2.26E+04 6.89E+02 8.73E+02 f18 1.22E-04 1.58E+01 1.31E+02 5.05E-02 f19 0.00E+00 3.59E+02 4.76E+00 0.00E+00 Table 9 Mean Errors obtained by DE, CHC, G-CMA-ES, and RPSOvm for dimension 100 f/Alg. DE CHC G-CMA-ES RPSO-vm f1 0.00E+00 3.56E-11 0.00E+00 2.66E-14 f2 4.45E+00 8.58E+01 1.51E-10 1.98E-01 f3 8.01E+01 4.19E+06 3.88E+00 1.42E+03 f4 7.96E-02 2.19E+02 2.50E+02 2.61E-14 f5 0.00E+00 3.83E-03 1.58E-03 1.07E-01 f6 3.10E-13 4.10E-07 2.12E+01 4.76E-12 f7 0.00E+00 1.40E-02 4.22E-04 0.00E+00 f8 3.69E+02 1.69E+03 0.00E+00 1.09E+04 f9 5.06E+02 5.86E+02 1.02E+02 1.85E-01 f10 0.00E+00 3.30E+01 1.66E+01 0.00E+00 f11 1.28E-04 7.32E+01 1.64E+02 4.61E-01 f12 5.99E-11 1.03E+01 4.17E+02 1.60E-14 f13 6.17E+01 2.70E+06 4.21E+02 2.25E+03 f14 4.79E-02 1.66E+02 2.55E+02 1.27E-01 f15 0.00E+00 8.13E+00 6.30E-01 0.00E+00 f16 3.58E-09 2.23E+01 8.59E+02 4.87E-08 f17 1.23E+01 1.47E+05 1.51E+03 1.76E+03 f18 2.98E-04 7.00E+01 3.07E+02 1.36E-01 f19 0.00E+00 5.45E+02 2.02E+01 0.00E+00 tables shows the results of RPSO-vm, indicating in bold face such values for which the mean error is the best found. A detailed study leads us to analyze the results dimension by dimension in the following. Table 10 Mean Errors obtained by DE, CHC, G-CMA-ES, and RPSOvm for dimension 200 f/Alg. DE CHC G-CMA-ES RPSO-vm f1 0.00E+00 8.34E-01 0.00E+00 2.53E-14 f2 1.92E+01 1.03E+02 1.16E-09 2.00E+00 f3 1.78E+02 2.01E+07 8.91E+01 1.03E+03 f4 1.27E-01 5.40E+02 6.48E+02 2.43E-14 f5 0.00E+00 8.76E-03 0.00E+00 1.73E-01 f6 6.54E-13 1.23E+00 2.14E+01 2.95E-12 f7 0.00E+00 2.59E-01 1.17E-01 0.00E+00 f8 5.53E+03 9.38E+03 0.00E+00 5.23E+04 f9 1.01E+03 1.19E+03 3.75E+02 1.67E+00 f10 0.00E+00 7.13E+01 4.43E+01 0.00E+00 f11 2.62E-04 3.85E+02 8.03E+02 5.66E-01 f12 9.76E-10 7.44E+01 9.06E+02 4.64E-02 f13 1.36E+02 5.75E+06 9.43E+02 1.17E+04 f14 1.38E-01 4.29E+02 6.09E+02 7.96E-02 f15 0.00E+00 2.14E+01 1.75E+00 0.00E+00 f16 7.46E-09 1.60E+02 1.92E+03 5.40E-01 f17 3.70E+01 1.75E+05 3.36E+03 2.08E+04 f18 4.73E-04 2.12E+02 6.89E+02 1.50E-01 f19 0.00E+00 2.06E+03 7.52E+02 0.00E+00 Table 11 Mean Errors obtained by DE, CHC, G-CMA-ES, and RPSOvm for dimension 500 f/Alg. DE CHC G-CMA-ES RPSO-vm f1 0.00E+00 2.84E-12 0.00E+00 2.65E-14 f2 5.35E+01 1.29E+02 3.48E-04 1.67E+01 f3 4.76E+02 1.14E+06 3.58E+02 1.13E+03 f4 3.20E-01 1.91E+03 2.10E+03 2.44E-14 f5 0.00E+00 6.98E-03 2.96E-04 2.54E-01 f6 1.65E-12 5.16E+00 2.15E+01 3.14E-12 f7 0.00E+00 1.27E-01 7.21E+153 0.00E+00 f8 6.09E+04 7.22E+04 2.36E-06 3.00E+05 f9 2.52E+03 3.00E+03 1.74E+03 4.85E+00 f10 0.00E+00 1.86E+02 1.27E+02 0.00E+00 f11 6.76E-04 1.81E+03 4.16E+03 4.88E+00 f12 7.07E-09 4.48E+02 2.58E+03 1.32E-08 f13 3.59E+02 3.22E+07 2.87E+03 1.33E+03 f14 1.35E-01 1.46E+03 1.95E+03 1.29E+00 f15 0.00E+00 6.01E+01 2.82E+262 0.00E+00 f16 2.04E-08 9.55E+02 5.45E+03 2.12E+00 f17 1.11E+02 8.40E+05 9.59E+03 5.72E+02 f18 1.22E-03 7.32E+02 2.05E+03 2.47E+00 f19 0.00E+00 1.76E+03 2.44E+06 0.00E+00 Table 12 Mean Errors obtained by DE, CHC, and RPSO-vm for dimension 1000 f/Alg. DE CHC RPSO-vm f1 0.00E+00 1.36E-11 2.72E-14 f2 8.46E+01 1.44E+02 4.29E+01 f3 9.69E+02 8.75E+03 3.21E+02 f4 1.44E+00 4.76E+03 4.81E-14 f5 0.00E+00 7.02E-03 2.14E-01 f6 3.29E-12 1.38E+01 4.92E-12 f7 0.00E+00 3.52E-01 0.00E+00 f8 2.46E+05 3.11E+05 9.35E+05 f9 5.13E+03 6.11E+03 1.17E+01 f10 0.00E+00 3.83E+02 0.00E+00 f11 1.35E-03 4.82E+03 1.10E+01 f12 1.68E-08 1.05E+03 1.00E-09 f13 7.30E+02 6.66E+07 1.93E+03 f14 6.90E-01 3.62E+03 5.27E-01 f15 0.00E+00 8.37E+01 0.00E+00 f16 4.18E-08 2.32E+03 9.50E-01 f17 2.36E+02 2.04E+07 2.82E+03 f18 2.37E-03 1.72E+03 1.80E+00 f19 0.00E+00 4.20E+03 0.00E+00 5.2.1 Dimension 50 Table 8 shows mean errors (of 25 runs) obtained by DE, CHC, G-CMA-ES, and RPSO-vm for dimension 50. In this 8 case, DE obtained the best results in 13 functions, 7 of them in hybrid composition functions. RPSO-vm obtained the best results in 7 functions, and G-CMA-ES obtained the best results in 3 functions. The Holm’s test with α =0.05 (Table 7) showed that DE, the algorithm with best ranking is statistically better than all algorithms, excepting RPSO-vm with p-value=0.05. 5.2.2 Dimension 100 In this case, in spite of reaching DE the best mean error in more functions than RPSO-vm and G-CMA-ES (see Table 9), both Friedman’s and Holm’s test showed similar results to dimension 50. That is, RPSO-vm and DE are statistically similar to themselves, but better than CHC, and GCMA-ES. 5.2.3 Dimension 200 As shown in Table 10, the results are quite similar to the previous ones of dimension 100. In fact, the same algorithms (RPSO-vm, DE, and G-CMA-ES) obtained the best mean errors practically in the same functions. Holm’s test also obtained that DE is statistically similar to RPSO-vm (pvalue=0.05), and better than the rest of algorithms. 5.2.4 Dimension 500 Table 11 contains the mean errors of all algorithms in dimension 500. We can see the set functions for which RPSOvm always obtained the best mean fitness: f4, f7, f9, f10, f15, and f19. In particular Shifted Schwefel 2.22 (f7) and its hybrids (f15 and f19) are optimized for all dimensions. These functions are unimodal separable (f7) and unimodal non-separable (f9, f10, f15, and 19) which could led us to think that RPSO-vm only has successful performance with unimodal functions, but we can easily check that our proposal obtained the best results for f4 (multimodal), and for all dimensions. In addition, RPSO-vm obtained the second best mean fitness for the remaining of hybrid functions (dimension 500). Statistically, Holm’s test confirms our initial hypothesis since it showed (Table 7) that the results of RPSO-vm are not significantly different to the ones of DE (p-value=0.05), and they are statistically better than the results of the rest of algorithms. 5.2.5 Dimension 1000 For the largest scale, Table 12 shows the mean errors where RPSO-vm obtained the best results in 10 out of 19 functions. As aforementioned, G-CMA-ES did not obtained any value for dimension 1000. Regarding dimension 500, the set of functions for which RPSO-vm obtained the best mean 1,00E-11 1,00E-09 1,00E-07 1,00E-05 1,00E-03 1,00E-01 1,00E+01 1,00E+03 50 100 500 1000 f2 DE CHC CMA-ES RPSO-vm 200 1,00E-01 1,00E+00 1,00E+01 1,00E+02 1,00E+03 1,00E+04 50 100 500 1000 f9 DE CHC CMA-ES RPSO-vm 200 1,00E-08 1,00E-06 1,00E-04 1,00E-02 1,00E+00 1,00E+02 1,00E+04 50 100 500 1000 f14 DE CHC CMA-ES RPSO-vm 200 1,00E+00 1,00E+01 1,00E+02 1,00E+03 1,00E+04 1,00E+05 1,00E+06 1,00E+07 50 100 500 1000 f19 DE CHC CMA-ES RPSO-vm 200 Fig. 1 Scalable results of DE, CHC, G-CMA-ES, and RPSO-vm for functions f2, f9, f14, and f19. Y axis shows the results in logarithmic scale. X axis shows the problem dimensions fitness has been increased with f2, f3, f12, and f14, having these functions different properties of modality and separability. As happened in all dimensions, in spite of having DE the best average ranking, the Holm’s test (Table 7) showed RPSO-vm is statistically similar to DE. In comparison with CHC, our proposal is statistically the best algorithm. From a graphical point of view, Fig. 1 illustrates the tendency of results of the compared algorithms and RPSOvm for functions f2, f9, f14, and f19 through the different dimensions. We have chosen these functions since they showed a representative behavior in terms of scalability. 9 Thus, we can observe in Fig. 1 that the performance of all algorithms deteriorates with the increment of the dimension. Nevertheless, this degradation is slight in almost all cases, and even nonexistent in others, as happened in functions f2 and f19 for algorithms RPSO-vm and DE. A different and anomalous behavior is observed in G-CMA-ES for functions f2 and f19, where it diminishes quickly. We suspect that the use of covariance matrix mechanism of GCMA-ES is unsuitable for large dimensions due to the great amount of resources it requires [12,18]. 5.3 Computational Effort Finally, we present in this section some remarks about the computational effort. To execute these experiments, we have used the computers of the laboratories of the Departament of Computer Science of the University of M´ alaga (Spain). Most of them are equipped with modern dual core processors, 1GB RAM, and Linux so, having into account that there are more than 180 computers, that means that up to 360 cores have been available. To run all the programs, we have used the Condor [29] middleware that acts as a distributed task scheduler (each task dealing with one independent run of RPSO-vm). In Table 13, we present the average running time (seconds) in which RPSO-vm has found the best mean error for all functions and for all dimensions. As expected, the running time increases with the number of variables, specially in non-separable functions. Specifically, f1 (Shifted Sphere) required the lowest time to be optimized for all dimensions, and f17 (Hybrid NS f9⊕f3) toke the longest time. In general, hybrid composition functions required more time to reach their best value than simple functions. Table 13 Average running time (ART), in seconds, of the 25 runs of RPSO-vm for all functions and for all dimensions D ART/D 50 100 200 500 1000 f1 7.91E-01 3.33E+00 1.36E+01 8.42E+01 3.60E+02 f2 2.72E+00 9.67E+00 4.10E+01 2.55E+02 9.37E+02 f3 6.30E+00 2.66E+01 9.85E+01 5.71E+02 2.50E+03 f4 2.82E+00 1.26E+01 5.50E+01 3.81E+02 1.52E+03 f5 2.19E+00 8.06E+00 3.82E+01 2.24E+02 8.60E+02 f6 4.82E+00 1.65E+01 7.43E+01 4.14E+02 1.76E+03 f7 2.42E+00 8.62E+00 3.58E+01 2.11E+02 8.72E+02 f8 2.27E+00 9.49E+00 3.41E+01 2.24E+02 8.02E+02 f9 9.05E+00 3.28E+01 1.32E+02 1.11E+03 3.19E+03 f10 4.14E+00 1.79E+01 6.40E+01 4.12E+02 1.74E+03 f11 9.23E+00 3.61E+01 1.43E+02 9.64E+02 3.92E+03 f12 4.17E+00 1.59E+01 6.63E+01 4.04E+02 1.33E+03 f13 7.44E+00 2.76E+01 9.97E+01 7.62E+02 3.03E+03 f14 5.91E+00 2.50E+01 9.30E+01 5.34E+02 2.20E+03 f15 2.78E+00 1.16E+01 4.43E+01 2.67E+02 8.90E+02 f16* 5.49E+00 2.20E+01 9.45E+01 6.09E+02 2.25E+03 f17* 8.48E+00 3.13E+01 1.41E+02 7.00E+02 3.24E+03 f18* 7.18E+00 3.05E+01 1.23E+02 7.47E+02 2.93E+03 f19* 4.01E+00 1.38E+01 6.15E+01 3.61E+02 1.43E+03 In this sense, an interesting observation consists in comparing the increment of both, the processing time and the optimum mean error found, through the different scales of the search space. This way, we can obtain insights about the computational effort required with regards to the quality of solutions obtained. Fig. 2 shows a representative case observed in function f2, where the increment of the processing time as well as the mean error is practically linear. If we take into account that the search space grows exponentially with the dimension [xlow,xupp]DIM in all functions, we can claim that our proposal scales successfully. Concerning the quality of solutions, the deterioration that the mean error suffers is higher in comparison with the processing time. Specifically, from dimension 200 to 500, the mean error increases in 2 orders of magnitude while the time required takes less than 1 order of magnitude. Curiously, the difference in the mean error between 500 and 1000 dimensions is not bigger than one order of magnitude which leads us to suspect that our proposal performs relatively better in larger dimensions than in smaller ones. 1,00E-04 1,00E-03 1,00E-02 1,00E-01 1,00E+00 1,00E+01 1,00E+02 50 100 500 1000 f2 TIME(seconds) MEANERROR Fig. 2 Differences in times versus mean error magnitudes of f2 for each dimension. Y axis contains values in logarithmic scale and X axis contains dimensions 6 Conclusions In this work, we have incorporated both velocity modulation and restarting mechanisms to the Particle Swarm Optimization with the aim of enhancing its scalability. Our hypothesis is that these two new mechanisms can help the PSO to avoid the early convergence and redirects the particles to promising areas in the search space. The experimentation phase has been carried out in the scope of this Special Issue to test the ability of being scalable. The results obtained show that our proposal is scalable in all functions of the benchmark used, as well as highly competitive with regard to other compared optimizers. In concrete, we can remark the following: –the new proposal, called Restarting PSO with Velocity Modulation (RPSO-vm), outperforms the basic PSO, as well as PSO with each new mechanism separately, for all dimensions. Additionally, the RPSO-vm algorithm with global best neighborhood topology outperforms (lb)RPSOvm: they two are the same algorithm but one has a global best (gbest) topology while the other has a variable neighborhood (lbest) topology.