NuMVC: An Efficient Local Search Algorithm for Minimum Vertex Cover

The Minimum Vertex Cover (MVC) problem is a prominent NP-hard combinatorial optimization problem of great importance in both theory and application. Local search has proved successful for this problem. However, there are two main drawbacks in state-of-the-art MVC local search algorithms. First, they select a pair of vertices to exchange simultaneously, which is time-consuming. Secondly, although using edge weighting techniques to diversify the search, these algorithms lack mechanisms for decreasing the weights. To address these issues, we propose two new strategies: two-stage exchange and edge weighting with forgetting. The two-stage exchange strategy selects two vertices to exchange separately and performs the exchange in two stages. The strategy of edge weighting with forgetting not only increases weights of uncovered edges, but also decreases some weights for each edge periodically. These two strategies are used in designing a new MVC local search algorithm, which is referred to as NuMVC. We conduct extensive experimental studies on the standard benchmarks, namely DIMACS and BHOSLIB. The experiment comparing NuMVC with state-of-the-art heuristic algorithms show that NuMVC is at least competitive with the nearest competitor namely PLS on the DIMACS benchmark, and clearly dominates all competitors on the BHOSLIB benchmark. Also, experimental results indicate that NuMVC finds an optimal solution much faster than the current best exact algorithm for Maximum Clique on random instances as well as some structured ones. Moreover, we study the effectiveness of the two strategies and the run-time behaviour through experimental analysis.


Introduction
The Minimum Vertex Cover (MVC) problem consists of, given an undirected graph G = (V, E), finding the minimum sized vertex cover, where a vertex cover is a subset S ⊆ V such that every edge in G has at least one endpoint in S. MVC is an important combinatorial optimization problem with many real-world applications, such as network security, scheduling, VLSI design and industrial machine assignment.It is equivalent to two other well-known combinatorial optimization problems: the Maximum Independent Set (MIS) problem and the Maximum Clique (MC) problem, which have a wide range of applications in areas such as information retrieval, experimental design, signal transmission, computer vision, and also bioinformatics problems such as aligning DNA and protein sequences (Johnson & Trick, 1996).Indeed, these three problems can be seen as three different forms of the same problem, from the viewpoint of practical algorithms.Algorithms for MVC can be directly used to solve the MIS and MC problems.Due to their great importance in theory and applications, these three problems have been widely investigated for the last several decades (Carraghan & Pardalos, 1990;Evans, 1998;Pullan & Hoos, 2006;Richter, Helmert, & Gretton, 2007;Cai, Su, & Chen, 2010;Li & Quan, 2010b;Cai, Su, & Sattar, 2011).
Theoretical analyses indicate that these three problems MVC, MIS, and MC are computationally hard.They are all NP-hard and the associated decision problems are NP-complete (Garey & Johnson, 1979).Moreover, they are hard to solve approximately.It is NP-hard to approximate MVC within any factor smaller than 1.3606 (Dinur & Safra, 2005), although one can achieve an approximation ratio of 2 − o(1) (Halperin, 2002;Karakostas, 2005).Besides the inapproximability of MVC, Håstad shows that both MIS and MC are not approximable within |V |1−ǫ for any ǫ > 0, unless NP=ZPP 1 (Håstad, 1999(Håstad, , 2001)).Recently, this conclusion has been enhanced that MC is not approximable within |V | 1−ǫ for any ǫ > 0 unless NP=P (Zuckerman, 2006), derived from a derandomization of Håstad's result.Moreover, the currently best polynomial-time approximation algorithm for MC is only guaranteed to find a clique within a factor of O(n(loglogn) 2 /(logn) 3 ) of optimum (Feige, 2004).
The algorithms to solve MVC (MIS, MC) fall into two types: exact algorithms and heuristic algorithms.Exact methods which mainly include branch-and-bound algorithms (Carraghan & Pardalos, 1990;Fahle, 2002;Östergård, 2002;Régin, 2003;Tomita & Kameda, 2009;Li & Quan, 2010b, 2010a), guarantee the optimality of the solutions they find, but may fail to give a solution within reasonable time for large instances.Heuristic algorithms, which mainly include local search algorithms, cannot guarantee the optimality of their solutions, but they can find optimal or satisfactory near-optimal solutions for large and hard instances within reasonable time.Therefore, it is appealing to use local search algorithms to solve large and hard MVC (MC, MIS) instances.
This work is devoted to a more efficient local search algorithm for MVC.Typically, local search algorithms for MVC solve the problem by iteratively solving the k-vertex cover problem.To solve the k-vertex cover problem, they maintain a current candidate solution of size k, and exchange two vertices iteratively until it becomes a vertex cover.However, we observe two drawbacks in state-of-the-art MVC local search algorithms.First, they select a pair of vertices for exchanging simultaneously according to some heuristic (Richter et al., 2007;Cai et al., 2010Cai et al., , 2011)), which is rather time-consuming, as will be explained in Section 3. The second drawback is about the edge weighting techniques.The basic concept of edge weighting is to increase weights of uncovered edges to diversify the search.Previous MVC local search algorithms utilize different edge weighting schemes.For example, COVER (Richter et al., 2007) increases weights of uncovered edges at each step, while EWLS (Cai et al., 2010) and EWCC (Cai et al., 2011) increase weights of uncovered edges only when reaching local optima.However, all these algorithms do not have a mechanism to decrease the weights.We believe this is deficient because the weighting decisions made too long ago may mislead the search.
To address these two issues in MVC local search algorithms, this paper proposes two new strategies, namely two-stage exchange and edge weighting with forgetting.The two-stage exchange strategy decomposes the exchanging procedure into two stages, i.e., the removing stage and the adding stage, and performs them separately.It first selects a vertex and removes it from the current candidate solution, and then selects a vertex in a random uncovered edge and adds it.The twostage exchange strategy yields an efficient two-pass move operator for MVC local search, in which the first pass is a linear-time search for the vertex-to-remove, while the second pass is a lineartime search for the vertex-to-add.This is in contrast to the standard quadratic, all-at-once move operator.Moreover, the two-stage exchange strategy renders the algorithm more flexible in that we can employ different heuristics in different stages.Indeed, the NuMVC algorithm utilizes a highly greedy heuristic for the removing stage, while for the adding stage, it makes good use of a diversifying heuristic within a framework similar to focused random walk (Papadimitriou, 1991).
The second strategy we propose is edge weighting with forgetting.It increases weights of uncovered edges by one at each step.Moreover, when the averaged edge weight achieves a threshold, it reduces weights of all edges by multiplying a constant factor ρ (0 < ρ < 1) to forget the earlier weighting decisions.To the best of our knowledge, this is the first time a forgetting mechanism is introduced into local search algorithms for MVC.
The two strategies are combined to design a new local search algorithm called NuMVC.We carry out a detailed experimental study to investigate the performance of NuMVC, and compare it with PLS (Pullan, 2006), COVER (Richter et al., 2007) and EWCC (Cai et al., 2011), which are leading heuristic algorithms for MVC (MC, MIS).Experimental results show that NuMVC competes well with other solvers on the DIMACS benchmark, and shows a dramatic improvement over existing results on the whole BHOSLIB benchmark.These parts of work have been published in an early version of this paper (Cai, Su, & Sattar, 2012).
In this paper, we additionally carry out more experimental analyses and provides further insights about the two strategies in NuMVC.We compare NuMVC with the exact algorithm MaxCLQdyn+EFL+SCR (Li & Quan, 2010a), which is the best exact Maximum Clique algorithm we found in the literature.Experimental results indicate that NuMVC finds an optimal solution much faster than the exact algorithm on random instances as well as some structured ones.More importantly, we conduct experimental investigations to study the run-time behaviour of NuMVC and the effectiveness of the two new strategies in NuMVC.
The remainder of this paper is organized as follows.In the next section, we introduce some definitions and notations used in this paper.We then present the two strategies: two-stage exchange and edge weighting with forgetting.In Section 5, we describe the NuMVC algorithm.Section 6 presents the experimental study of NuMVC and comparative results to other algorithms, including heuristic and exact algorithms.This is followed by more detailed investigations about the run-time behaviour of NuMVC and the effectiveness of the two new strategies in Section 7. Finally, we conclude the paper by summarizing the main contributions and some future directions.

Preliminaries
An undirected graph G = (V, E) consists of a vertex set V and an edge set E ⊆ V × V , where each edge is a 2-element subset of V .For an edge e = {u, v}, we say that vertices u and v are the endpoints of edge e.Two vertices are neighbors if and only if they both belong to some common edge.We denote N (v) = {u ∈ V |{u, v} ∈ E}, the set of neighbors of a vertex v.
For an undirected graph G = (V, E), an independent set is a subset of V with pairwise nonadjacent elements and a clique is a subset of V with pairwise adjacent elements.The maximum independent set and maximum clique problems are to find the maximum sized independent set and clique in a graph, respectively.
We note that these three problems MVC, MIS and MC can be seen as three different forms of the same problem, from the viewpoint of experimental algorithms.A vertex set S is an independent set of G if and only if V \S is a vertex cover of G; a vertex set K is a clique of G if and only if V \K is a vertex cover of the complementary graph G.To find the maximum independent set of a graph G, one can find the minimum vertex cover C min for G and return V \C min .Similarly, to find the maximum clique of a graph G, one can find the minimum vertex cover C ′ min for the complementary graph G, and return V \C ′ min .Given an undirected graph G = (V, E), a candidate solution for MVC is a subset of vertices.An edge e ∈ E is covered by a candidate solution X if at least one endpoint of e belongs to X.During the search procedure, NuMVC always maintains a current candidate solution.For convenience, in the rest of this paper, we use C to denote the current candidate solution.The state of a vertex v is denoted by ∈ C. The step to a neighboring candidate solution consists of exchanging two vertices: a vertex u ∈ C is removed from C, and a vertex v / ∈ C is put into C.The age of a vertex is the number of steps since its state was last changed.
As with most state-of-the-art MVC local search algorithms, NuMVC utilizes an edge weighting scheme.For edge weighting local search, we follow the definitions and notations in EWCC (Cai et al., 2011).An edge weighted undirected graph is an undirected graph G = (V, E) combined with a weighting function w so that each edge e ∈ E is associated with a non-negative integer number w(e) as its weight.We use w to denote the mean value of all edge weights.
Let w be a weighting function for G.For a candidate solution X, we set the cost of X as cost(G, X) = e∈E and e is not covered by X w(e) which indicates the total weight of edges uncovered by X.We take cost(G, X) as the evaluation f unction, and NuMVC prefers candidate solutions with lower costs.For a vertex v ∈ V , where C ′ = C\{v} if v ∈ C, and C ′ = C ∪ {v} otherwise, measuring the benefit of changing the state of vertex v. Obviously, for a vertex v ∈ C, we have dscore(v) ≤ 0, and the greater dscore indicates the less loss of covered edges by removing it out of C. For a vertex v / ∈ C, we have dscore(v) ≥ 0, and the higher dscore indicates the greater increment of covered edges by adding it into C.

Two-Stage Exchange
In this section, we introduce the two-stage exchange strategy, which is adopted by the NuMVC algorithm to exchange a pair of vertices.
As with most state-of-the-art MVC local search algorithms, NuMVC is an iterated k-vertex cover algorithm.When finding a k-vertex cover, NuMVC removes one vertex from the current candidate solution C and goes on to search for a (k − 1)-vertex cover.In this sense, the core of NuMVC is a k-vertex cover algorithm -given a positive integer number k, searching for a k-sized vertex cover.To find a k-vertex cover, NuMVC begins with a candidate solution C of size k, and exchanges two vertices iteratively until C becomes a vertex cover.
Most local search algorithms for MVC select a pair of vertices to exchange simultaneously according to a certain heuristic.For example, COVER selects a pair of vertices that maximize gain(u, v) (Richter et al., 2007), while EWLS (Cai et al., 2010) and EWCC (Cai et al., 2011) select a random pair of vertices with score(u, v) > 0. This strategy of selecting two vertices to exchange simultaneously leads to a quadratic neighborhood for candidate solutions.Moreover, the evaluation of a pair of vertices not only depends on the evaluations (such as dscore) of the two vertices, but also involves the relationship between the two vertices, like "do they belong to a same edge".Therefore, it is rather time-consuming to evaluate all candidate pairs of vertices.
In contrast to earlier MVC local search algorithms, NuMVC selects the two vertices for exchanging separately and exchanges the two selected vertices in two stages.In each iteration, NuMVC first selects a vertex u ∈ C with the highest dscore and removes it.After that, NuMVC selects a uniformly random uncovered edge e, and chooses one endpoint v of e with the higher dscore under some restrictions and adds it into C.Note that this two-stage exchange strategy resembles in some respect the min-conflicts hill-climbing heuristic for CSP (Minton, Johnston, Philips, & Laird, 1992), which shows surprisingly good performance for the N-queens problem.
Selecting the two vertices for exchanging separately may in some cases miss some greedier vertex pairs which consist of two neighboring vertices.However, as is usual in local search algorithms, there is a trade-off between the accuracy of heuristics and the complexity per step.Let R and A denote the set of candidate vertices for removing and adding separately.The time complexity per step for selecting the exchanging vertex pair simultaneously is |R| • |A|; while the complexity per step for selecting the two vertices separately, as in NuMVC, is only |R| + |A|.It is worthy to note that, as heuristics in a local search algorithm are often based on intuition and experience rather than on theoretically or empirically derived principles and insights, we cannot say for certain that being less greedy is not a good thing (Hoos & Stützle, 2004).On the other hand, a lower time complexity is always desirable.

Edge Weighting with Forgetting
In this section, we present a new edge weighting technique called edge weighting with forgetting, which plays an important role in NuMVC.
The proposed strategy of edge weighting with forgetting works as follows.Each edge is associated with a positive integer number as its weight, and each edge weight is initialized as one.Then in each iteration, edge weights of the uncovered edges are increased by one.Moreover, when the average weight achieves a threshold, all edge weights are reduced to forget the earlier weighting decisions using the formula w(e) := ⌊ρ • w(e)⌋, where ρ is a constant factor between 0 and 1.
Note that edge weighting techniques in MVC local search, including the one in this work, fall in the more general penalty idea for optimization problems, which dates back to Morris 's breakout method (Morris, 1993) and has been widely used in local search algorithms for constraint optimization problems such as SAT (Yugami, Ohta, & Hara, 1994;Wu & Wah, 2000;Schuurmans, Southey, & Holte, 2001;Hutter, Tompkins, & Hoos, 2002).Our results therefore provide further evidence for the effectiveness and general applicability of this algorithmic technique.
Edge weighting techniques have been successfully used to improve MVC local search algorithms.For example, COVER (Richter et al., 2007) updates edge weights at each step, while EWLS (Cai et al., 2010) and EWCC (Cai et al., 2011) update edge weights only when reaching local optima.However, all previous edge weighting techniques do not have a mechanism to decrease the weights, which limits their effectiveness.The strategy of edge weighting with forgetting in this work introduces a forgetting mechanism to reduce edge weights periodically, which contributes considerably to the NuMVC algorithm.
The intuition behind the forgetting mechanism is that the weighting decisions made too long ago are no longer helpful and may mislead the search, and hence should be considered less important than the recent ones.For example, consider two edges e 1 and e 2 with w(e 1 ) = 1000 and w(e 2 ) = 100 at some step.We use ∆w(e) to denote the increase of w(e).According to the evaluation function, in the next period of time, the algorithm is likely to cover e 1 more frequently than e 2 , and we may assume that during this period ∆w(e 1 ) = 50 and ∆w(e 2 ) = 500, which makes w(e 1 ) = 1000 + 50 = 1050 and w(e 2 ) = 100 + 500 = 600.Without a forgetting mechanism, the algorithm would still prefer e 1 to e 2 to be covered in the future search.This is not reasonable, as during this period e 2 is covered in much fewer steps than e 1 is.Thus, e 2 should take priority to be covered for the sake of diversification.Now let us consider the case with the forgetting mechanism (assuming ρ = 0.3 which is the setting in our experiments).Suppose w(e 1 ) = 1000 and w(e 2 ) = 100 when the algorithm performs the forgetting.The forgetting mechanism reduces the edge weights as w(e 1 ) = 1000×0.3= 300 and w(e 2 ) = 100×0.3= 30.After a period of time, with ∆w(e 1 ) = 50 and ∆w(e 2 ) = 500, we have w(e 1 ) = 300 + 50 = 350 and w(e 2 ) = 30 + 500 = 530.In this case, the algorithm prefers to cover e 2 rather than cover e 1 in the future search, as we expect.
Although being inspired by smoothing techniques in clause weighting local search algorithms for SAT, the forgetting mechanism in NuMVC differs from those smoothing techniques in SAT local search algorithms.According to the way that clause weights are smoothed, there are three main smoothing techniques in clause weighting local search algorithms for SAT to the best of our knowledge: the first is to pull all clause weights to their mean value using the formula (Schuurmans et al., 2001), SAPS (Hutter et al., 2002) and Swcca (Cai & Su, 2012); the second is to subtract one from all clause weights which are greater than one, as in DLM (Wu & Wah, 2000) and PAWS (Thornton, Pham, Bain, & Jr., 2004); and the last is employed in DDWF (Ishtaiwi, Thornton, Sattar, & Pham, 2005), which transfers weights from neighbouring satisfied clauses to unsatisfied ones.It is obvious that the forgetting mechanism in NuMVC is different from all these smoothing techniques.
Recently, a forgetting mechanism was proposed for vertex weighting technique in the significant MC local search algorithm DLS-MC (Pullan & Hoos, 2006), which is an important sub-algorithm in PLS (Pullan, 2006) and CLS (Pullan, Mascia, & Brunato, 2011).The DLS-MC algorithm employs a vertex weighting scheme which increases the weights of vertices (by one) not in the current clique when reaching a local optimum, and periodically decreases weights (by one) for all vertices that currently have a penalty.Specifically, it utilizes a parameter pd (penalty delay) to specify the number of penalty increase iterations that must occur before the algorithm performs a forgetting operation.However, Pullan and Hoos also observed that DLS-MC is very sensitive to the pd parameter, and the optimal value of pd varies considerably among different instances.Indeed, the performance of DLS-MC in is given by optimizing the pd parameter.In contrast, the forgetting mechanism in NuMVC is much less sensitive to its parameters (as will be shown in Section 7.4), and thus is more robust.
We also notice that the formula used in the forgetting mechanism in NuMVC has been adopted in long-term frequency-based learning mechanisms for tabu search (Taillard, 1994).However, in Taillar's algorithm, the parameter ρ (using the term in this work) is always greater than one, and the formula is used for penalizing a move rather than forgetting the penalties.

The NuMVC Algorithm
In this section, we present the NuMVC algorithm, which utilizes the strategies of two-stage exchange and edge weighting with forgetting.For better understanding the algorithm, we first describe a strategy called configuration checking (CC), which is used in NuMVC.The CC strategy (Cai et al., 2011) was proposed for handling the cycling problem in local search, i.e., revisiting a candidate solution that has been visited recently (Michiels, Aarts, & Korst, 2007).This strategy has been successfully applied in local search algorithms for MVC (Cai et al., 2011) as well as SAT (Cai & Su, 2011, 2012).
The CC strategy in NuMVC works as follows: For a vertex v / ∈ C, if all its neighboring vertices never change their states since the last time v was removed from C, then v should not be added back to C. The CC strategy can be seen as a prohibition mechanism, which shares the same spirit but differs from the well-known prohibition mechanism called tabu (Glover, 1989).
An implementation of the CC strategy is to maintain a Boolean array confChange for vertices.During the search procedure, those vertices which have a confChange value of 0 are forbidden to add into C.The confChange array is initialized as an all-1 array.After that, when a vertex v is removed from C, confChange(v) is reset to 0, and when a vertex v changes its state, for each We outline the NuMVC algorithm in Algorithm 1, as described below.In the beginning, all edge weights are initialized as 1, and dscores of vertices are computed accordingly; confChange(v) is initialized as 1 for each vertex v; then the current candidate solution C is constructed by iteratively adding the vertex with the highest dscore (ties are broken randomly), until it becomes a vertex cover.Finally, the best solution C * is initialized as C.
After the initialization, the loop (lines 7-18) is executed until a given cutoff time is reached.During the search procedure, once there is no uncovered edge, which means C is a vertex cover, NuMVC updates the best solution C * as C (line 9).Then it removes one vertex with the highest dscore from C (line 10), breaking ties randomly, so that it can go on to search for a vertex cover of size |C| = |C * | − 1.We note that, in C, the vertex with the highest dscore has the minimum absolute value of dscore since all these dscores are negative.
In each iteration of the loop, NuMVC swaps two vertices according to the strategy of two-stage exchange (lines 12-16).Specifically, it first selects a vertex u ∈ C with the highest dscore to remove, breaking ties in favor of the oldest one.After removing u, NuMVC chooses an uncovered edge e uniformly at random, and selects one of e's endpoints to add into C as follows: If there is only one endpoint whose confChange is 1, then that vertex is selected; if the confChange values of both endpoints are 1, then NuMVC selects the vertex with the higher dscore, breaking ties in favor of the older one.The exchange is finished by adding the selected vertex into C. Along with exchanging the two selected vertices, the confChange array is updated accordingly.
At the end of each iteration, NuMVC updates the edge weights (lines 17-18).First, weights of all uncovered edges are increased by one.Moreover, NuMVC utilizes the forgetting mechanism to decrease the weights periodically.In detail, if the averaged weight of all edges achieves a threshold γ, then all edge weights are multiplied by a constant factor ρ (0 < ρ < 1) and rounded down to an integer as edge weights are defined as integers in NuMVC.The forgetting mechanism forgets the earlier weighting decision to some extent, as these past effects are generally no longer helpful and may mislead the search.
We conclude this section by the following observation, which guarantees the executability of line 15.
Proposition 1.For an uncovered edge e, there is at least one endpoint v of edge e such that confChange(v) = 1.
Proof: Let us consider an arbitrary uncovered edge e = {v 1 , v 2 }.The proof includes two cases.(a) There is at least one of v 1 and v 2 which never changes its state after initialization.Without loss of generality, we assume v 1 is such a vertex.In the initialization, confChange(v 1 ) is set to 1.After that, only removing v 1 from C (which corresponds to v's state s v changing to 0 from 1) can make confChange(v 1 ) be 0, but v 1 never changes its state after initialization, so we have confChange(v 1 )= 1.(b) Both v 1 and v 2 change their states after initialization.As e is uncovered, we have v 1 / ∈ C and v 2 / ∈ C. Without loss of generality, we assume the last removing of v 1 happens before the last removing of v 2 .The last time v 1 is removed, v 2 ∈ C holds.Afterwards, v 2 is removed, which means v 2 changes its state, so confChange(v 1 ) is set to 1 as v 1 ∈ N (v 2 ).

Empirical Results
In this section, we present a detailed experimental study to evaluate the performance of NuMVC on standard benchmarks in the literature, i.e., the DIMACS and BHOSLIB benchmarks.We first introduce the DIMACS and BHOSLIB benchmarks, and describe some preliminaries about the experiments.Then, we divide the experiments into three parts.The purpose of the first part is to demonstrate the performance of NuMVC in detail.The second is to compare NuMVC with state-ofthe-art heuristic algorithms.Finally, the last part is to compare NuMVC with state-of-the-art exact algorithms.

The Benchmarks
Having a good set of benchmarks is fundamental to demonstrate the effectiveness of new solvers.We use the two standard benchmarks in MVC (MIS, MC) research, the DIMACS benchmark and the BHOSLIB benchmark.The DIMACS benchmark includes instances from industry and those generated by various models, while the BHOSLIB instances are random ones of high difficulty.

DIMACS BENCHMARK
The DIMACS benchmark is taken from the Second DIMACS Implementation Challenge for the Maximum Clique problem (1992-1993)2 .Thirty seven graphs were selected by the organizers for a summary to indicate the effectiveness of algorithms, comprising the Second DIMACS Challenge Test Problems.These instances were generated from real world problems such as coding theory, fault diagnosis, Keller's conjecture and the Steiner Triple Problem, etc, and random graphs in various models, such as the brock and p hat families.These instances range in size from less than 50 vertices and 1,000 edges to greater than 4,000 vertices and 5,000,000 edges.Although being proposed two decades ago, the DIMACS benchmark remains the most popular benchmark and has been widely used for evaluating heuristic algorithms for MVC (Richter et al., 2007;Pullan, 2009;Cai et al., 2011;Gajurel & Bielefeld, 2012), MIS (Andrade et al., 2008;Pullan, 2009) and MC algorithms (Pullan, 2006;Katayama, Sadamatsu, & Narihisa, 2007;Grosso, Locatelli, & Pullan, 2008;Pullan et al., 2011;Wu, Hao, & Glover, 2012).In particular, the DIMACS benchmark has been used for evaluating COVER and EWCC.It is convenient for us to use this benchmark also to conduct experiments comparing NuMVC with COVER and EWCC.Note that as the DIMACS graphs were originally designed for the Maximum Clique problem, MVC algorithms are tested on their complementary graphs.

BHOSLIB BENCHMARK
The BHOSLIB 3 (Benchmarks with Hidden Optimum Solutions) instances were generated randomly in the phase transition area according to the model RB (Xu, Boussemart, Hemery, & Lecoutre, 2005).Generally, those phase-transition instances generated by model RB have been proved to be hard both theoretically (Xu & Li, 2006) and practically (Xu & Li, 2000;Xu, Boussemart, Hemery, & Lecoutre, 2007).The SAT version of the BHOSLIB benchmark is extensively used in the SAT competitions 4 .Nevertheless, SAT solvers are much weaker than MVC solvers on these problems, which remains justifiable when referring to the results of SAT Competition 2011 on this benchmark.The BHOSLIB benchmark is famous for its hardness and influential enough to be strongly recommended by the MVC (MC, MIS) community (Grosso et al., 2008;Cai et al., 2011).It has been widely used in the recent literature as a reference point for new local search solvers to MVC, MC and MIS 5 .Besides these 40 instances, there is a large instance frb100-40 with 4,000 vertices and 572,774 edges, which is designed for challenging MVC (MC, MIS) algorithms.
The BHOSLIB benchmark was designed for MC, MVC and MIS, and all the graphs in this benchmark are expressed in two formats, i.e., the clq format and the mis format.For a BHOSLIB instance, the graph in clq format and the one in mis format are complementary to each other.MC algorithms are tested on the graphs in clq format, while MVC and MIS algorithms are tested on those in mis format.

Experiment Preliminaries
Before we discuss the experimental results, let us introduce some preliminary information about our experiments.
NuMVC is implemented in C++.The codes of both NuMVC and EWCC are publicly available on the first author's homepage 6 .The codes of COVER are downloaded online 7 , and those of PLS are kindly provided by its authors.All the four solvers are compiled by g++ with the '-O2' option.All experiments are carried out on a machine with a 3 GHz Intel Core 2 Duo CPU E8400 and 4GB RAM under Linux.To execute the DIMACS machine benchmarks 8 , this machine requires 0.19 CPU seconds for r300.5, 1.12 CPU seconds for r400.5 and 4.24 CPU seconds for r500.5.
For NuMVC, we set γ = 0.5|V | and ρ = 0.3 for all runs, except for the challenging instance frb100-40, where γ = 5000 and ρ = 0.3.Note that there are also parameters in other state-ofthe-art MVC (MC, MIS) algorithms, such as DLS-MC (Pullan & Hoos, 2006) and EWLS (Cai et al., 2010).Moreover, the parameters in DLS-MC and EWLS vary considerably on different instances.For each instance, each algorithm is performed 100 independent runs with different random seeds, where each run is terminated upon reaching a given cutoff time.The cutoff time is set to 2000 seconds for all instances except for the challenging instance frb100-40, for which the cutoff time is set to 4000 seconds due to its significant hardness.
For NuMVC, we report the following information for each instance: • The optimal (or minimum known) vertex cover size (V C * ).• The number of successful runs ("suc").A run is said successful if a solution of size V C * is found.
• The "VC size" which shows the min (average, max) vertex cover size found by NuMVC in 100 runs.
• The averaged run time over all 100 runs ("time").The run time of a successful run is the time to find the V C * solution, and that of a failed run is considered to be the cutoff time.For instances where NuMVC does not achieve a 100% success rate, we also report the averaged run time over only successful runs ("suc time").The run time is measured in CPU seconds.
• The inter-quartile range (IQR) of the run time for 100 runs.The IQR is the difference between the 75th percentile and the 25th percentile of a sample.IQR is one of the most famous robust measures in data analysis (Hoaglin, Mosteller, & Tukey, 2000), and has been recommended as a measurement of closeness of the sampling distribution by the community of experimental algorithms (Bartz-Beielstein, Chiarandini, Paquete, & Preuss, 2010).
• The number of steps averaged over all 100 runs ("steps").The steps of a successful run is those needed to find the V C * solution, while the steps of a failed run are those executed before the running is cut off.For instances where NuMVC does not achieve a 100% success rate, we also report the averaged steps over only successful runs ("suc steps").
If there are no successful runs for an instance, the "time" and "steps" columns are marked with "n/a".When the success rate of a solver on an instance is less than 75%, the 75th percentile of the run time sample is just the cutoff time and does not represent the real 75th percentile.In this case, we do not report the IQR, and instead we mark with "n/a" on the corresponding column.Actually, if the success rate of a solver on a certain instance is less than 75%, the solver should be considered not robust on that instance given the cutoff time.

Performance of NuMVC
In this section, we report a detailed performance of NuMVC on the two benchmarks.

PERFORMANCE OF NUMVC ON DIMACS BENCHMARK
The performance results of NuMVC on the DIMACS benchmark are displayed in Table 1.NuMVC finds optimal (or best known) solutions for 35 out of 37 DIMACS instances.Note that the 2 failed instances are both brock graphs.Furthermore, among the 35 successful instances, NuMVC does so consistently (i.e., in all 100 runs) for 32 instances, 24 of which are solved within 1 second.Overall, the NuMVC algorithm exhibits excellent performance on the DIMACS benchmark except for the brock graphs.Remark that the brock graphs are artificially designed to defeat greedy heuristics by explicitly incorporating low-degree vertices into the optimal vertex cover.Indeed, most algorithms preferring higher-degree vertices such as GRASP, RLS, k-opt, COVER and EWCC also failed in these graphs.

PERFORMANCE OF NUMVC ON BHOSLIB BENCHMARK
In Table 2, we illustrate the performance of NuMVC on the BHOSLIB benchmark.NuMVC successfully solves all BHOSLIB instances in terms of finding an optimal solution, and the size of the worst solution it finds never exceeds V C * + 1. NuMVC finds optimal solutions with 100% success rate for 33 out of these 40 instances, and the averaged success rate over the remaining 7 instances is 82.57%.These results are dramatically better than existing results in the literature on this benchmark.Also, NuMVC finds a sub-optimal solution of size V C * + 1 for all BSHOSLIB instances very quickly, always in less than 30 seconds.This indicates NuMVC can be used to approximate the MVC problem efficiently even under very limited time.
Besides the 40 BHOSLIB instances in Table 2, there is a challenging instance frb100-40, which has a hidden minimum vertex cover of size 3900.The designer of the BHOSLIB benchmark conjectured that this instance will not be solved on a PC in less than a day within the next two decades9 .The latest record for this challenging instance is a 3902-sized vertex cover found by EWLS, and also EWCC.
We run NuMVC 100 independent trials within 4000 seconds on frb100-40, with γ = 5000 and ρ = 0.3 (this parameter setting yields the best performance among all combinations from γ = 2000, 3000, ..., 6000 and ρ = 0.1, 0.2, ..., 0.5).Among these 100 runs, 4 runs find a 3902-sized solution with the averaged time of 2955 seconds, and 93 runs find a 3903-sized solution (including 3902-sized) with the averaged time of 1473 seconds.Also, it is interesting to note that NuMVC can locate a rather good approximate solution for this hard instance very quickly: the size of vertex covers that NuMVC finds within 100 seconds is between 3903 and 3905.
Generally, finding a (k+1)-vertex cover is much easier than a k-vertex cover.Hence, for NuMVC, as well as most other MVC local search algorithms which also solve the MVC problem by solving the k-vertex cover problem iteratively, the majority of running time is used in finding the best vertex cover C * (of the run), and in trying, without success, to find a vertex cover of size (|C * | − 1).

Comparison with Other Heuristic Algorithms
In the recent literature there are five leading heuristic algorithms for MVC (MC, MIS), including three MVC algorithms COVER (Richter et al., 2007), EWLS (Cai et al., 2010) and EWCC (Cai et al., 2011), and two MC algorithms DLS-MC (Pullan & Hoos, 2006) and PLS (Pullan, 2006).Note that EWCC and PLS are the improved versions of EWLS and DLS-MC respectively, and show better performance over their original versions on DIMACS and BHOSLIB benchmarks.Therefore, we compare NuMVC only with PLS, COVER and EWCC.
When comparing NuMVC with other heuristic algorithms, we report V C * , "suc", "time" as well as IQR.The averaged run time over only successful runs ("suc time") cannot indicate comparative performance of algorithms correctly unless the evaluated algorithms have close success rates, and can be calculated by "time" * 100−cutof f * (100−"suc") "suc" , so we do not report these statistics.The results in bold indicate the best performance for an instance.

COMPARATIVE RESULTS ON DIMACS BENCHMARK
The comparative results on the DIMACS benchmark are shown in Table 3.Most DIMACS instances are so easy that they can be solved by all solvers with 100% success rate within 2 seconds, and thus are not reported in the table.Actually, the fact that the DIMACS benchmark has been reduced to 11 useful instances really emphasizes the need to make a new benchmark.Table 2: NuMVC performance results, averaged over 100 independent runs, for the BHOSLIB benchmark instances.All these BHOSLIB instances have a hidden optimal vertex cover, whose size is shown in the VC * column.
As indicated in Table 3, NuMVC outperforms COVER and EWCC on all instances, and is competitive with and complementary to PLS.For the eight hard instances on which at least one solver fails to achieve a 100% success rate, PLS dominates on the brock graphs while NuMVC dominates on the others, including the two putatively hardest instances C2000.9 and MANN a81 (Richter et al., 2007;Grosso et al., 2008;Cai et al., 2011), as well as keller6 and MANN a45.For C2000.9, only NuMVC finds a 1920-sized solution, and it also finds a 1921-sized solution in 70 runs, while this number is 31, 6 and 32 for PLS, COVER, and EWCC respectively.Note that PLS performs well on the brock family because it comprises three sub-algorithms, one of which favors the lower degree vertices.Table 3 indicates that C2000.9 and MANN a81 remain very difficult for modern algorithms, as none of the algorithms can solve them with a good success rate in reasonable time.On the other hand, other instances can be solved quickly (in less than 100 seconds) by at least one algorithm, PLS or NuMVC, with a low IQR value (always less than 100), which indicates quite stable performance.

COMPARATIVE RESULTS ON BHOSLIB BENCHMARK
In Table 4, we present comparative results on the BHOSLIB benchmark.For concentrating on the considerable gaps in comparisons, we do not report the results on the two groups of small instances (frb30 and frb35), which can be solved within several seconds by all solvers.
The results in Table 4 illustrate that NuMVC significantly outperforms the other algorithms on all BHOSLIB instances, in terms of both success rate and averaged run time, which are also demonstrated in Figure 1.We take a further look at the comparison between NuMVC and EWCC, as EWCC performs obviously better than PLS and COVER on this benchmark.NuMVC solves 33 instances out of 40 with 100% success rate, 4 more instances than EWCC does.For those instances solved by both algorithms with 100% success rate, the overall averaged run time is 25 seconds for NuMVC and 74 seconds for EWCC.For other instances, the averaged success rate is 90% for NuMVC, compared to 50% for EWCC.
The excellent performance of NuMVC is further underlined by the large gaps between NuMVC and the other solvers on the hard instances.For example, on the instances where all solvers fail to find an optimal solution with 100% success rate, NuMVC achieves an overall averaged success rate of 82.57%, dramatically better than those of PLS, COVER and EWCC, which are 0.85%, 17.43% and 35.71% respectively.Obviously, the experimental results show that NuMVC delivers  the best performance for this hard random benchmark, vastly improving the existing performance results.We also observe that, NuMVC always has the minimum IQR value for all instances, which indicates that apart from its efficiency, the robustness of NuMVC is also better than other solvers.We also compare NuMVC with COVER and EWCC on the challenging instance frb100-40.Given the failure of PLS on large BHOSLIB instances, we do not run PLS on this instance.The comparative results on frb100-40 are shown in Table 5, which indicates that NuMVC significantly outperforms COVER and EWCC on this challenging instance.
Finally, we would like to remark that the performance of NuMVC on the BHOSLIB benchmark is better than a four-core version of CLS (Pullan et al., 2011), even if we do not divide the run time of NuMVC by 4 (the number of cores utilized by CLS).If we consider the machine speed ratio and divide the run time of NuMVC by 4, then NuMVC would be dramatically better than CLS on the BHOSLIB benchmark.Table 5: Comparative results on the frb100-40 challenging instance.Each solver is executed 100 times on this instance with a timeout of 4000 seconds.

Comparison with Exact Algorithms
In this section, we compare NuMVC with a state-of-the-art exact Maximum Clique algorithm.Generally, exact algorithms and heuristic algorithms are somewhat complementary in their applications.Usually, exact algorithms find solutions for structured instances faster while heuristic algorithms are faster on random ones.
We compare NuMVC with MaxCLQdyn+EFL+SCR on the DIMACS benchmark instances.The results of MaxCLQdyn+EFL+SCR are taken from the previous work (Li & Quan, 2010a).MaxCLQdyn+EFL+SCR is not evaluated on the BHOSLIB benchmark which is much harder and requires more effective technologies for exact algorithms (Li & Quan, 2010a).
The run time results of MaxCLQdyn+EFL+SCR are obtained on a 3.33 GHz Intel Core 2 Duo CPU with linux and 4 Gb memory, which required 0.172 seconds for r300.5, 1.016 seconds for r400.5 and 3.872 seconds for r500.5 to execute the DIMACS machine benchmarks (Li & Quan, 2010a).The corresponding run time for our machine is 0.19, 1.12 and 4.24 seconds.So, we multiply the reported run time of by 1.098 (=(4.24/3.872+1.12/1.016)/2=1.098,the average of the two largest ratios).This normalization is based on the methodology established in the Second DIMACS Implementation Challenge for Cliques, Coloring, and Satisfiability, and is widely used for comparing different MaxClique algorithms (Pullan & Hoos, 2006;Pullan, 2006;Li & Quan, 2010b, 2010a).In Table 6, we present the performance of NuMVC and MaxCLQdyn+EFL+SCR on the DIMACS instances.The results indicate that NuMVC finds an optimal solution much faster than MaxCLQdyn+EFL+SCR on random instances such as the p hat and sanr instances.We believe that similar results would hold for other hard random benchmarks like BHOSLIB ones, as MaxCLQdyn+EFL+SCR is not evaluated on these instances due to their high hardness (Li & Quan, 2010a), while NuMVC performs very well on them.
For structured instances, we note that MaxCLQdyn+EFL+SCR is mainly evaluated on the brock instances where NuMVC performs worst, but not on the open DIMACS instances such as MANN a81, johnson32-2-4 and keller6, which remain very difficult to solve by exact algorithms (Li & Quan, 2010a).Although MaxCLQdyn+EFL+SCR overall performs better, NuMVC also finds an optimal solution significantly faster than MaxCLQdyn+EFL+SCR on some structured instances, such as the two brock instances and keller5.
Finally, we would like to note that although heuristic solvers can find optimal solutions fast, they are unable to prove the optimality of the solutions they find.On the other hand, the run time of an exact algorithm is spent not only on finding an optimal solution but also on proving its optimality.In this sense, heuristic and exact algorithms cannot be compared in a fair way.Nevertheless, our experiments suggest that heuristic approaches are appealing for solving large instances in reasonable short time.

Discussions
In this section, we first explore the run-time distribution of NuMVC on some representative instances, and then investigate the effectiveness of the two-stage exchange strategy and the forgetting mechanism in NuMVC.Finally, we analyze the performance of NuMVC with different settings to its two parameters for the forgetting mechanism, which shows that NuMVC is not sensitive to the parameters.

Run-time Distributions of NuMVC
In this subsection, we conduct an empirical study to gain deeper insights of the run-time behavior of NuMVC.More specifically, we study the run-time distribution of NuMVC on several representative instances.For the purpose of comparison, we also report the run-time distribution of EWCC, which is the best competing MVC local search solver.
Consider a randomized algorithm solving a given optimization problem instance, and halting as soon as an optimal solution is found.The run time of the algorithm can be viewed as a random variable, which is fully described by its distribution, commonly referred to as the run-time distribution (RTD) in the literature about algorithm performance modeling (Hoos & Stützle, 2004;Bartz-Beielstein et al., 2010).The methodology of studying the run-time behavior of algorithms based on RTDs has been widely used in empirical analysis of heuristic algorithms (Hoos & Stützle, 1999;Finkelstein, Markovitch, & Rivlin, 2003;Watson, Whitley, & Howe, 2005;Pullan & Hoos, 2006).We also follow the same methodology in our study here.
For studying typical run-time behaviour, we choose instances where NuMVC reaches an optimal solution in all 100 runs, and are of appropriate difficulty.For the DIMACS benchmark, we select brock400 4 and MANN a45, both of which are of reasonable size and hardness.Also, these two instances represent two typical instance classes for NuMVC, as NuMVC has poor performance on the brock instances, while it dominates other heuristic algorithms on the MANN instances.For BHOSLIB benchmark, frb56-25-5 and frb59-26-5 are selected.These are appropriate instances for studying the run-time behavior of NuMVC, since they are neither too easy that can be solved in a short time nor too difficult to reach a 100% success rate.
The empirical RTD graphs of NuMVC and EWCC are shown in Figure 2 (the RTD for each instance is based on 100 independent runs that all reach a respective optimal solution).According to the graphs, NuMVC shows a large variability in run time.Further investigation indicates that these RTDs are quite well approximated by exponential distributions, labeled ed[m](x) = 1 − 2 −x/m , where m is the median of the distribution.To test the goodness of the approximations, we use a  Kolmogorov-Smirnov test, which fails to reject the null hypothesis that the sampled run time stems from the exponential distributions shown in the figures at a standard confidence level of α = 0.05 with p-values between 0.19 and 0.88.For EWCC, the Kolmogorov-Smirnov test shows its RTDs on MANN a45 and the two BHOSLIB instances are also exponential distributions, while its RTD on brock400 4 is not from an exponential distribution.
The observation of exponential RTDs of NuMVC is consistent with similar results for other high performance SLS algorithms, e.g., for MaxClique (Pullan & Hoos, 2006), for SAT (Hoos & Stützle, 1999), for MAXSAT (Smyth, Hoos, & Stützle, 2003), and for scheduling problems (Watson et al., 2005).By the arguments (Hoos & Stützle, 1999;Hoos & Stützle, 2004) made for stochastic local search algorithms characterized by an exponential RTD, we conclude that, for NuMVC, the probability of finding an optimal solution within a fixed amount of time (or steps) does not depend on the run time in the past.Consequently, it is very robust w.r.t. the cutoff time and thus, the restart time.Therefore, performing multiple independent runs of NuMVC in parallel will result in closeto-optimal parallelization speedup.Similar observations were made for most of the other DIMACS instances and BHOSLIB instances.
Of practical interest is also the RTD analysis for NuMVC on difficult instances for which all algorithms in our experiments fail to achieve a high success rate (i.e., 40%).The RTDs in these cases would show where the algorithm stagnates and suggest an a-posteriori restart time for the algorithm.For this purpose, we select MANN a81 and frb59-26-2 for analysis.The RTDs of NuMVC on these two instances are illustrated in Figure 3. Interestingly, from these RTDs we do not observe any obvious stagnation, which again confirms that NuMVC is robust w.r.t. the cutoff time and thus the restart time.Therefore, by increasing the cutoff time, we can expect a higher success rate of the algorithm on these difficult instances.for which NuMVC finds an optimal (or best known) solution in less than half runs.

Effectiveness of Two-Stage Exchange
To study the effectiveness of the two-stage exchange strategy, we compare NuMVC with its alternative algorithm NuMVC 0 which selects two vertices for exchanging simultaneously.In each step, NuMVC 0 first chooses an uncovered edge e uniformly at random, and then evaluates each pair of vertices u and v where u is in the current candidate solution and v is one endpoint of e such that conf Change(v) = 1.For evaluating the benefit (i.e., the decrement of the cost function) of exchanging a vertex pair u and v, NuMVC 0 first checks whether they are neighbors.If u and v are neighbors, the benefit is dscore(u) + dscore(v) + w(e{u, v}); otherwise, the benefit is dscore(u) + dscore(v).NuMVC 0 selects the vertex pair with the greatest benefit to exchange.
In the NuMVC (and also NuMVC 0 ) algorithm, there are only two candidate vertices to add to the current candidate solution C (i.e., the endpoints of the selected uncovered edge).Hence, in the worst case, NuMVC performs 2 + |C| evaluations, while NuMVC 0 has to evaluate 2 × |C| pairs of vertices.Moreover, NuMVC only needs to check the dscore of a vertex in each (vertex) evaluation, while NuMVC 0 performs a vertex-pair evaluation which involves a pair of vertices and their relationship, and thus is more time-consuming.Based on the above analysis, we conjecture that the complexity per step of NuMVC is at least 2 times lower than that of NuMVC 0 .Also, as we have mentioned in Section 3, the two-stage exchange strategy is less greedy than the one selecting two vertices for exchanging simultaneously, as NuMVC 0 does.
The investigation is carried out on 4 DIMACS instances from different families as well as 12 BHOSLIB instances.For the DIMACS benchmark, we select brock400 2, C4000.5, MANN a45, and p hat 1500-1.These instances have different characteristics, as described below (Pullan et al., 2011).Note that the following conclusions on DIMACS instances are for the complementary DIMACS graphs.
• The DIMACS brock instances have minimum vertex covers that consist of medium to lower degree vertices, and are designed to defeat greedy heuristics.
• The DIMACS C and p hat 1500-1 instances have minimum vertex covers that consist of higher degree vertices and can be effectively solved by greedy heuristics.
• The DIMACS MANN instances have a large proportion of plateaus in the instance searchspace, and thus greedy heuristics are unsuitable to solve them.
• The BHOSLIB instances have minimum vertex covers consisting of vertices whose distribution of vertex degree closely matches that for the complete graph.These are difficult instances for both greedy and diversification heuristics.The comparative results of NuMVC and NuMVC 0 are presented in Table 7.The results show that NuMVC significantly outperforms NuMVC 0 in terms of averaged run time, primarily due to its much lower complexity per step.In each second, NuMVC performs 3-4 times more steps than NuMVC 0 , which supports our conjecture that the complexity per step of NuMVC is more than 2 times lower than that of NuMVC 0 .Now we turn our attention to comparing NuMVC and NuMVC 0 in terms of step performance, which is independent from the complexity per step.For brock and MANN graphs which are difficult for greedy heuristics, NuMVC has a significantly better step performance than NuMVC 0 .On the other hand, for greedy-friendly graphs such as C4000.5 and p hat 1500-1, NuMVC needs more steps to converge to an optimal solution than NuMVC 0 does.These observations support our argument that the two-stage exchange strategy is less greedy than the one that selects two vertices for exchanging simultaneously, as NuMVC 0 does.
We also observe that the step performance of NuMVC 0 is better than that of NuMVC on BHOSLIB instances.For instance, on those BHOSLIB instances where both algorithms have a 100% success rate, NuMVC needs about 1.2 times more steps than NuMVC 0 to find an optimal solution.This is what we do not expect and cannot yet explain.Nevertheless, as NuMVC makes rather rapid modifications to a solution, a little degrade in step performance does not hurt.To further demonstrate the low complexity per step of NuMVC, we compare the number of search steps per second between NuMVC and other state-of-the-art heuristic solvers on representative instances.As indicated in Table 8, NuMVC executes many more steps in each second than the other two MVC local search solvers COVER and EWCC do.For the instances in Table 8, each second NuMVC executes 4-6 times more steps than COVER, and 3-4 times more steps than EWCC.This indicates that the two-stage exchange strategy can significantly accelerate MVC local search algorithms.Although PLS performs more steps per second than NuMVC, it is an MC local search algorithm whose search scheme is essentially different from those of MVC local search algorithms.

Effectiveness of the Forgetting Mechanism
To study the effectiveness of the forgetting mechanism in NuMVC, we compare NuMVC with its two alternative algorithms NuMVC 1 and NuMVC 2 , which are obtained from NuMVC by modifying the edge weighting scheme as below.
• NuMVC 1 works in the same way as NuMVC, except for not using the forgetting mechanism, that is, deleting line 18 from Algorithm 1.
• NuMVC 2 adopts the forgetting mechanism used in DLS-MC (Pullan & Hoos, 2006) for the weighting scheme.More specifically, NuMVC 2 increases all weights of uncovered edges by one at the end of each step, and performs a forgetting operation every pd steps by decreasing weights by one for all edges whose weights are greater than one.Note that pd is an instancedependent parameter.
The experiments were carried out with some representative instances from both benchmarks.For the DIMACS benchmark, we select brock400 2, C4000.5, keller6, and MANN a45, which are from different classes and of appropriate difficulty.For the BHOSLIB benchmark, we select three instances for each of the three largest-sized instance groups respectively.Each algorithm is performed 100 times on each instance.

Graph
An apparent observation from Table 9 is that the two algorithms with a forgetting mechanisms (i.e., NuMVC and NuMVC 2 ) outperform NuMVC 1 on almost all instances.Particularly, due to the missing of a forgetting mechanism, NuMVC 1 performs significantly worse than the other two algorithms on brock and MANN graphs.On the other hand, Table 9 demonstrates that NuMVC and NuMVC 2 exhibit competitive performance on the BHOSLIB benchmark, and dominate on different types of DIMACS instances.More specifically, NuMVC outperforms NuMVC 2 on C4000.5, keller6 and MANN a45, but performs significantly worse than NuMVC 2 on brock400 2. In order to find out the genuine performance of NuMVC 2 on brock instances, we test NuMVC 2 on the larger brock800 2 and brock800 4 instances.The results show that these two large brock instances are substantially more difficult than the two brock400 instances, and NuMVC 2 also fails to solve neither of them.
Although NuMVC 2 shows competitive performance with NuMVC, its performance is given by optimizing the pd parameter for each instance.Moreover, as with DLS-MC (Pullan & Hoos, 2006), NuMVC 2 is considerably sensitive to the pd parameter.For example, our experiments show that on the frb53-24 instances, NuMVC 2 performs quite well with pd = 10000, but it fails to find an optimal solution when pd is set to be a value less than 7000.Comparatively, NuMVC with the same parameter setting performs quite well on all types of instances but the brock family.Actually, we will show in the next section that NuMVC is not sensitive to its parameters.
It is also interesting to compare NuMVC with its alternatives which replace the forgetting mechanism with the smoothing techniques similar to those in local search for SAT.Indeed, earlier versions of NuMVC did use the smoothing techniques similar to those in SAT local search, and they did not have good performance compared with NuMVC.It would be interesting to find out the reasons for the success of the forgetting mechanism and the failure of those smoothing techniques in MVC edge weighting local search algorithms such as NuMVC.The NuMVC algorithm has two parameters γ and ρ, which specify the forgetting mechanism.Specifically, when the averaged weight of all edges achieves a threshold γ, all edge weights are multiplied by a constant factor ρ (0 < ρ < 1).In this subsection, we investigate how NuMVC performs with different settings to these two parameters.The investigation is carried out on both DIMACS and BHOSLIB benchmarks.For the DIMACS benchmark, we select the four instances used in the preceding subsection for the same reasons.For the BHOSLIB benchmark, we select frb53-24-1, frb53-24-2, frb56-25-1 and frb56-25-2, which are of different sizes and appropriate hardness.

Parameters for the Forgetting Mechanism
Table 10 presents the performance of NuMVC with various parameter combinations of γ and ρ on the representative instances.As we can see from Table 10, the parameter combination (0.5|V |, 0.3) yields relatively good performance for all instances, and exhibits a better robustness over the instances than other parameter combinations do.
On the other hand, we observe that NuMVC with various parameter combinations performs comparably on these tested instances.For example, for all parameter settings, NuMVC achieves a success rate of 100% for keller6, MANN a45, C4000.5 as well as frb53-24-1, and the averaged run time difference on these instances is not so significant.For other instances, the difference of success rate never exceeds 25% between any two parameter settings.This observation indicates that NuMVC seems not sensitive to the two parameters.Actually, as we have mentioned before, NuMVC exhibits very good performance for both DIMACS and BHOSLIB benchmarks with a fixed parameter setting.This is an advantage compared to other forgetting mechanisms such as the one used in DLS-MC (Pullan & Hoos, 2006), which is sensitive to its parameter.For algorithms that are sensitive to their parameters, considerable parameter tuning is required in order to get a good performance for a certain instance, which usually costs much more time than solving the instance.

Conclusions and Future Work
In this paper, we presented two new local search strategies for the minimum vertex cover (MVC) problem, namely two-stage exchange and edge weighting with forgetting.The two-stage exchange strategy yields an efficient two-pass move operator for MVC local search algorithms, which significantly reduces the time complexity per step.The forgetting mechanism enhances the edge weighting scheme by decreasing weights when the averaged weight reaches a threshold, to periodically forget earlier weighting decisions.Based on these two strategies, we designed a slight, yet effective MVC local search algorithm called NuMVC.The NuMVC algorithm was evaluated against the best known heuristic algorithms for MVC (MC, MIS) on standard benchmarks, i.e., the DIMACS and BHOSLIB benchmarks.The experimental results show that NuMVC is largely competitive on the DIMACS benchmark and dramatically outperforms other state-of-the-art heuristic algorithms on all BHOSLIB instances.Furthermore, we showed that NuMVC is characterized by exponential RTDs, which means it is robust w.r.t. the cutoff parameters and the restart time, and hence has close-to-optimal parallelization speedup.We also performed further investigations to provide further insights into the two new strategies and their effectiveness.Finally, we conducted an experiment to study the performance of NuMVC with different parameter settings, and the results indicate that NuMVC is not sensitive to its parameters.
The two-stage exchange strategy not only has a lower time complexity per step, but also has the flexibility to allow us to employ specific heuristics in different stages.An interesting research direction is thus to apply this idea to other combinatorial problems whose essential tasks are also to seek for an optimal subset with some fixed cardinality.
the cutoff time Output: vertex cover of G begin 2 initialize edge weights and dscores of vertices; 3 initialize the confChange array as an all-1 array; 4 construct C greedily until it is a vertex cover; 5 C * := C; 6 while elapsed time < cutoff do 7 if there is no uncovered edge then 8 C * := C; 9 remove a vertex with the highest dscore from C; 10 continue; 11 choose a vertex u ∈ C with the highest dscore, breaking ties in favor of the oldest 12 one; C := C\{u}, confChange(u) := 0 and confChange(z) := 1 for each z ∈ N (u); 13 choose an uncovered edge e randomly; 14 choose a vertex v ∈ e such that confChange(v) = 1 with higher dscore, breaking ties 15 in favor of the older one; C := C ∪ {v}, confChange(z) := 1 for each z ∈ N (v); 16 w(e) := w(e) + 1 for each uncovered edge e; 17 if w ≥ γ then w(e) := ⌊ρ • w(e)⌋ for each edge e;

Figure 1 :
Figure 1: Comparison of NuMVC and other local search algorithms on the BHOSLIB benchmark in terms of success rate (left) and averaged run time (right) avg suc time suc avg suc time suc avg suc time 3902 NuMVC and EWCC on frb59−26−5 RTD for NuMVC ed[45] RTD for EWCC ed[116]

Figure 2 :
Figure 2: Run-time distributions (RTDs) of NuMVC and EWCC applied to two DIMACS instances (top) and two BHOSLIB instances (bottom); these empirical RTDs are well approximated by exponential distributions, labeled ed[m](x) = 1 − 2 −x/m in the plots.

Figure 3 :
Figure 3: Run-time distributions (RTDs) of NuMVC on MANN a81 and frb59-26-2 instances,for which NuMVC finds an optimal (or best known) solution in less than half runs.

Table 1 :
NuMVC performance results, averaged over 100 independent runs, for the DIMACS benchmark instances.The VC * column marked with an asterisk means that the minimum known vertex cover size has been proved optimal.

Table 3 :
Comparison of NuMVC with other state-of-the-art heuristic algorithms on the DIMACS benchmark.The VC * column marked with an asterisk means that the minimum known vertex cover size has been proved optimal.

Table 4 :
Comparison of NuMVC with other state-of-the-art local search algorithms on the BHOSLIB benchmark.All these BHOSLIB instances have a hidden optimal vertex cover, whose size is shown in the VC * column.

Table 6 :
Comparison of NuMVC with the state-of-the-art exact MaxClique algorithm MaxCLQfor the DIMACS benchmark.

Table 7 :
Comparative performance of NuMVC and NuMVC 0 which selects two vertices for exchanging simultaneously.The results are based on 100 independent runs for each solver on each instance.

Table 8 :
Complexity per step on selected instances

Table 9 :
Comparative performance of NuMVC and its two alternatives NuMVC 1 and NuMVC 2 .

Table 10 :
Comparative performance of NuMVC with various parameter combinations (γ, ρ) for the forgetting mechanism.For each instance, NuMVC is performed 20 times with each parameter combination, except for the one adopted in this work (0.5|V |, 0.3), where the results are based on 100 runs.For keller6, NuMVC performs almost the same with various parameters, having the same success rate (100%) and tiny difference of averaged run time (less than 1 second), and thus the results are not reported in the table.