An Anytime Algorithm for Optimal Coalition Structure Generation

Coalition formation is a fundamental type of interaction that involves the creation of coherent groupings of distinct, autonomous, agents in order to efficiently achieve their individual or collective goals. Forming effective coalitions is a major research challenge in the field of multi-agent systems. Central to this endeavour is the problem of determining which of the many possible coalitions to form in order to achieve some goal. This usually requires calculating a value for every possible coalition, known as the coalition value, which indicates how beneficial that coalition would be if it was formed. Once these values are calculated, the agents usually need to find a combination of coalitions, in which every agent belongs to exactly one coalition, and by which the overall outcome of the system is maximized. However, this coalition structure generation problem is extremely challenging due to the number of possible solutions that need to be examined, which grows exponentially with the number of agents involved. To date, therefore, many algorithms have been proposed to solve this problem using different techniques ranging from dynamic programming, to integer programming, to stochastic search all of which suffer from major limitations relating to execution time, solution quality, and memory requirements. With this in mind, we develop an anytime algorithm to solve the coalition structure generation problem. Specifically, the algorithm uses a novel representation of the search space, which partitions the space of possible solutions into sub-spaces such that it is possible to compute upper and lower bounds on the values of the best coalition structures in them. These bounds are then used to identify the sub-spaces that have no potential of containing the optimal solution so that they can be pruned. The algorithm, then, searches through the remaining sub-spaces very efficiently using a branch-and-bound technique to avoid examining all the solutions within the searched subspace(s). In this setting, we prove that our algorithm enumerates all coalition structures efficiently by avoiding redundant and invalid solutions automatically. Moreover, in order to effectively test our algorithm we develop a new type of input distribution which allows us to generate more reliable benchmarks compared to the input distributions previously used in the field. Given this new distribution, we show that for 27 agents our algorithm is able to find solutions that are optimal in 0.175% of the time required by the fastest available algorithm in the literature. The algorithm is anytime, and if interrupted before it would have normally terminated, it can still provide a solution that is guaranteed to be within a bound from the optimal one. Moreover, the guarantees we provide on the quality of the solution are significantly better than those provided by the previous state of the art algorithms designed for this purpose. For example, for the worst case distribution given 25 agents, our algorithm is able to find a 90% efficient solution in around 10% of time it takes to find the optimal solution.


Introduction
Multi-agent systems are considered an important and rapidly expanding area of research in artificial intelligence.This is due to its natural fit to many real-world scenarios, and its wide variety of applications (Jennings, 2001).Now, typically, the agents in a multi-agent system need to be organized such that the roles, relationships, and authority structures that govern their behaviour are clearly defined (Horling & Lesser, 2005).Different organizational paradigms include hierarchies, teams, federations, and many others, each with its own strengths and weaknesses, making it more suitable for some problems, and less suitable for others.Among the organizational paradigms that are becoming increasingly important is that of coalitions.Coalitions can be distinguished from other organizations by being goal-directed and short-lived; i.e. the coalitions are formed with a purpose in mind, and are dissolved when that purpose no longer exists, or when they cease to suit their designed purpose, or when the profitability is lost as agents depart (Horling & Lesser, 2005).Another defining feature is that within each coalition, the agents coordinate their activities in order to achieve the coalition's goal(s), but no coordination takes place among agents belonging to different coalitions (except if the coalitions' goals interact).Moreover, the organizational structure within each coalition is usually flat (although there could be a coalition leader acting as a representative for the group as a whole).
The area of coalition formation has received considerable attention in recent research, and has proved to be useful in a number of real-world scenarios and multi-agent systems.For example, in e-commerce, buyers can form coalitions to purchase a product in bulk and take advantage of price discounts (Tsvetovat, Sycara, Chen, & Ying, 2000).In e-business, groups of agents can be formed in order to satisfy particular market niches (Norman, Preece, Chalmers, Jennings, Luck, Dang, Nguyen, V. Deora, Gray, & Fiddian, 2004).In distributed sensor networks, coalitions of sensors can work together to track targets of interest (Dang, Dash, Rogers, & Jennings, 2006).In distributed vehicle routing, coalitions of delivery companies can be formed to reduce the transportation costs by sharing deliveries (Sandholm & Lesser, 1997).Coalition formation can also be used for information gathering, where several information servers form coalitions to answer queries (Klusch & Shehory, 1996).
Generally speaking, the coalition formation process can be viewed as being composed of the three main activities that are outlined below (Sandholm, Larson, Andersson, Shehory, & Tohme, 1999): 1. Coalition Value Calculation: In this context, a number of coalition formation algorithms have been developed to determine which of the potential coalitions should actually be formed.
To do so, they typically calculate a value for each coalition, known as the coalition value, which provides an indication of the expected outcome that could be derived if that coalition was formed.Then, having computed all the coalition values, the decision about the optimal coalition(s) to form can be taken.The way this value is calculated depends on the problem under investigation.
In an electronic marketplace, for example, the value of a coalition of buyers can be calculated as the difference between the sum of the reservation costs of the coalition members and the minimum cost needed to satisfy the requests of all the members (Li & Sycara, 2002).In information gathering, the coalition value can be designed to represent a measure of how closely the information agents' domains are related (Klusch & Shehory, 1996).In cases where the agents' rationality is bounded due to computational complexity, the value of a coalition may represent the best outcome it can achieve given limited computational resources for solving the problem (Sandholm & Lesser, 1997).
It is usually assumed that every coalition performs equally well, given any coalition structure containing it (i.e. the value of a coalition does not depend on the actions of non-members).Such settings are known as characteristic function games (CFGs), where the value of a coalition is given by a characteristic function.Many, but clearly not all, real-world multi-agent problems happen to be CFGs (Sandholm et al., 1999).
Note that an optimal solution to the CSG problem is one that maximizes the social welfare.Now, unlike a cooperative environment where the agents are mainly concerned with maximizing the social welfare, the agents in a selfish environment are only concerned with maximizing their own utility.This, however, does not mean that a CSG algorithm cannot be applied in selfish multi-agent systems.This is because the designer of such systems is usually concerned with raising the overall efficiency of the system and, in many cases, this corresponds to maximizing the social welfare.To this end, the designer needs to design an enforcement mechanism that motivates the agents to join the optimal coalition structure and, in order to do so, he first needs to know what that structure is.Moreover, knowing the value of the optimal coalition structure, or knowing a value that is within a bound from that optimal, allows the designer to evaluate the relative effectiveness of the coalition structure currently formed in the system.
3. Pay-off Distribution: Having determined which coalitions should be formed, it is important to determine the rewards that each agent should get in order for the coalitions to be stable.
Here, stability refers to the state where the agents have no incentive to deviate from the coalitions to which they belong (or little incentive in weaker types of stability).This is desirable because it ensures that the agents will devote their resources to their chosen coalition rather than negotiating with, and moving to, other coalitions.This ensures that the coalitions can last long enough to actually achieve their goals.The analysis of such incentives has long been studied within the realm of cooperative game theory.In this context, many solutions have been proposed based on different stability concepts.These include the Core, the Shapley value, and the Kernel (more details can be found in the paper by Osborne & Rubinstein, 1994).Moreover, schemes have been developed to transfer non-stable pay-off distributions to stable ones while keeping the coalition structure unchanged (Kahan & Rapoport, 1984, provide a comprehensive review on stability concepts and transfer schemes in game theory).Note, however, that the agents in a cooperative environment have no incentive to dissolve a coalition that improves the performance of the system as a whole.Therefore, pay-off distribution is less important, and the main concern is generating a coalition structure so as to maximize the social welfare.
One of the most challenging of all of these activities is that of coalition structure generation, and this is due to the number of possible solutions which grows exponentially (in O(n n ) and ω(n n 2 ) with the number of agents involved (n)).More specifically, it has been proved that finding an optimal coalition structure is NP-complete (Sandholm et al., 1999).To combat this complexity, a number of algorithms have been developed in the past few years, using different search techniques (e.g.dynamic programming, integer programming, and stochastic search).These algorithms, however, suffer from major limitations that make them either inefficient or inapplicable, particularly given larger numbers of agents (see Section 2 for more details).
This motivates our aim to develop an efficient algorithm for searching the space of possible coalition structures.In more detail, given a CFG setting, we wish to develop an algorithm that satisfies the following properties: 1. Optimality: When run to completion, the algorithm must always be able return a solution that maximizes the social welfare.
2. Ability to prune: the algorithm must be able to identify the sub-spaces that have no potential of containing an optimal solution so that they can be pruned from the search space.This property is critical given the exponential nature of the problem (e.g.given 20 agents, the number of possible coalition structures is 51,724,158,235,372).
3. Discrimination: the algorithm must be able to verify, during the search, that it has found an optimal solution, instead of proceeding with the search in the hope that a better solution can be found.

4.
Anytime: the algorithm should be able to quickly return an initial solution, and then improve on the quality of this solution as it searches more of the space, until it finds an optimal one.This is particularly important since the agents might not always have sufficient time to run the algorithm to completion, especially given the exponential size of the search space.Moreover, being anytime makes the algorithm more robust against failure; if the execution is stopped before the algorithm would have normally terminated, then it would still provide the agents with a solution that is better than the initial solution, or any other intermediate one.
5. Worst Case Guarantees: the algorithm should be able to provide worst-case guarantees on the quality of its solution.Otherwise, the generated solution could always be arbitrarily worse than the optimal one.Such guarantees are important when trading off between the solution quality and the search time.For example, if the quality of the current solution is known to be no worse than, say, 95% of the optimal one, and if there is still a significant portion of the space left to be searched, then the agents might decide that it is not worthwhile to carry on with the search.Obviously, the better the guarantees, the more likely it is that the agents will decide to stop searching for a better solution.
Against the research aims outlined above, this paper makes the following contributions to the state of the art in coalition structure generation: 1. We provide a new representation of the space of possible coalition structures.This representation partitions the space into much smaller, disjoint sub-spaces that can be explored independently to find an optimal solution.As opposed to the other widely-used representation (Sandholm et al., 1999;Dang & Jennings, 2004), by which the coalition structures are categorized based on the number of coalitions they contain, our representation categorizes the coalition structures into sub-spaces based on the sizes of the coalitions they contain.A key advantage of this representation is that, immediately after scanning the input to the algorithm (i.e. the coalition values), we can compute the average value of the coalition structures within each sub-space.Moreover, by scanning the input, we can also compute an upper and a lower bound on the value of the best coalition structure that could be found in each of these subspaces.Then, by comparing these bounds, it is possible to identify the sub-spaces that have no potential of containing an optimal solution so that they can be pruned.A second major advantage of this representation is that it allows the agents to analyse the trade-off between the size of (i.e. the number of coalition structures within) a sub-space and the improvement it may bring to the current solution by virtue of its bounds.Hence, rather than constraining the solution to fixed sizes, as Shehory and Kraus (1998) do, agents using our representation can make a more informed decision about the sizes of coalitions to choose (since each of the sub-spaces are defined by the sizes of coalitions within the coalition structures).
2. We develop a novel, anytime, integer-partition based algorithm (called IP) for coalitions structure generation which uses the representation discussed above, and provides very high guarantees on the quality of its solutions very quickly.Moreover, IP is guaranteed to return an optimal solution when run to completion.
3. We prove that our algorithm is able to enumerate coalition structures efficiently by avoiding redundant and invalid solutions.Our enumeration technique also allows us to apply branchand-bound to reduce the amount of search needed.
4. While many CSG algorithms in the literature have been evaluated using the input distributions that were defined by Larson and Sandholm (2000), we prove that these distributions are biased as far as the CSG problem is concerned.Moreover, we propose a new distribution and prove that it tackles this problem, making it much more suitable for evaluating CSG algorithms in general.

5.
When evaluating the time required to return an optimal solution, we compare IP with the fastest algorithm guaranteed to return an optimal solution (i.e. the Improved Dynamic Programming (IDP) algorithm by Rahwan & Jennings, 2008b).This comparison shows that IP is significantly faster.In more detail, IP is empirically shown to find an optimal solution in 0.175% of the time taken by IDP given 27 agents.
6.We benchmark IP against previous anytime algorithms ( Sandholm et al., 1999;Dang & Jennings, 2004), and show that it provides significantly better guarantees on the quality of the solutions it generates over time.In more detail, we empirically show that, for various numbers of agents, the quality of its initial solution (i.e. the solution found after scanning the input) is usually guaranteed to be at least 40% of the optimal, as opposed to 2 n (which means for example, 10% for 20 agents and 8% for 25 agents) for both Sandholm et al.'s algorithm and Dang and Jennings's algorithm.For the standard distributions with which we evaluate our algorithm, we also find that it usually terminates by searching only minute portions of the search space and generates near-optimal solutions (i.e.> 90% of the optimal) by searching even smaller portions of the search space (i.e. on average around 0.0000002% of the search space).This is a tremendous improvement over the aforementioned algorithms which could guarantee solutions higher than 50% of the optimal only after searching the whole space.
Note that this is a significantly revised and extended version of previous papers (Rahwan, Ramchurn, Dang, & Jennings, 2007a;Rahwan, Ramchurn, Giovannucci, Dang, & Jennings, 2007b).Specifically, we provide in this paper a more comprehensive review of the available algorithms in the CSG literature.We also provide a detailed analysis of our IP algorithm, describe the pseudo code of all the functions used in IP, and prove the correctness of the function that searches the different sub-spaces.A mathematical proof is also provided regarding the way the size of a sub-space is computed.Moreover, we question the validity of the standard value distributions that are used in the literature, and propose a new value distribution (called NDCS) that is more suitable for evaluating CSG algorithms.Finally, we benchmark our algorithm against the improved dynamic programming algorithm (IDP) by Rahwan and Jennings (2008b) (instead of the standard DP algorithm).
The remainder of the paper is organized as follows.In Section 2, we describe the algorithms that are currently available for solving the coalition structure generation problem, and discuss their relative advantages and limitations.In Section 3, we present our novel representation of the search space and, in Section 4, we present our integer-partition based algorithm (IP), showing how it identifies the sub-spaces that can be pruned, and how it searches through the remaining ones without going through invalid or redundant coalition structures, using a branch-and-bound technique.Section 5 provides an empirical evaluation of the algorithm, and benchmarks it against the current state of the art in the CSG literature.Section 6 concludes the paper and outlines future work.We also provide, in the appendices, a summary of the main notations employed, as well as detailed proofs of the theorems provided in the paper.

Related Work
Previous algorithms that have been designed for the coalition structure generation problem can be classified into two main categories: • Exact algorithms1 -using heuristics, integer programming, or dynamic programming.
• Non-exact algorithms -using genetic algorithms, or limiting the search space in some way.
Next, we discuss both the advantages and the limitations of the algorithms that fall within each of these classes.Throughout the paper, we denote by n the number of agents, and by A = {a 1 , a 2 , • • • , a n } the set of agents.Moreover, we define an order over the agents in A as follows: ∀a i , a j ∈ A, a i < a j iff i < j, and a i = a j iff i = j.In other words, we have: Finally, we denote by v(C) the value of coalition C, and V (CS) the value of coalition structure CS.

Exact Algorithms for Coalition Structure Generation
There are very few exact algorithms for coalition structure generation.Those that have been developed can be distinguished based on whether they use dynamic programming or heuristics.In what follows, we outline their features and discuss how they relate to our ultimate goal of developing an efficient, anytime, optimal coalition structure generation algorithm.

DYNAMIC PROGRAMMING
Here we consider computationally efficient algorithms designed to return an optimal solution.Note that the emphasis, here, is on providing a guarantee on the performance of the algorithm in worstcase scenarios.In this context, Yeh (1986) developed a dynamic programming algorithm to solve the complete set partitioning problem.A very similar algorithm was later developed by Rothkopf, Pekec, and Harstad (1995) to solve the winner determination problem in combinatorial auctions.These algorithms can be directly applied to find optimal coalition structures, since the problems they were originally designed to solve are very similar to the CSG problem.2Also note that both of these algorithms use basically the same technique and, therefore, have the same computational complexity.Thus, throughout this paper, we do not distinguish between them, and refer to both as the dynamic programming (DP) algorithm.The biggest advantage of this algorithm is that it runs in O(3 n ) time (Rothkopf et al., 1995).This is significantly less than exhaustive enumeration of all coalition structures (which is O(n n )).In fact, DP is polynomial in the size of the input.This is because the input includes 2 n − 1 values, and the following holds: Therefore, the computational complexity of the algorithm is O(y log 2 3 ), where y is the number of values in the input.While, on the one hand, no other algorithm in the literature is guaranteed to find an optimal coalition structure in polynomial time (in the size of the input), on the other hand, the main limitation of DP is that it does not generate solutions anytime, and has a large memory requirement.Specifically, it requires maintaining three tables in memory containing 2 n entries each.
More recently, Rahwan and Jennings (2008b) developed an Improved Dynamic Programming algorithm (called IDP) that performs fewer operations and requires less memory than DP (e.g.given 25 agents, it performs only 38.7% of the operations, and requires 66.6% of the memory in the worst case, and 33.3% in the best).However, IDP does not return solutions anytime.As mentioned earlier, this is undesirable, especially given large numbers of agents, because the time required to return the optimal solution might be longer than the time available to the agents.Sandholm et al. (1999) were the first to introduce an anytime algorithm for coalition structure generation that establishes bounds on the quality of the solution found so far.They view the coalition structure generation process as a search in what they call the coalition structure graph (see Figure 1).In this undirected graph, every node represents a possible coalition structure.The nodes are categorized into n levels, noted as LV 1 , • • • , LV n where level LV i contains the coalition structures that contain i coalitions.The arcs represent mergers of two coalitions when followed upwards, and splits of a coalition into two coalitions when followed downwards.

ANYTIME ALGORITHMS WITH WORST CASE GUARANTEES
Figure 1: The coalition structure graph for 4 agents.Sandholm et al. (1999) have proved that, in order to establish a bound on the quality of a coalition structure, it is sufficient to search through the first two levels of the coalition structure graph.In this case, the bound β would be equal to n, and the number of searched coalition structures would be 2 n−1 .They have also proved that this bound is tight; meaning that no better bound exists for this search.Moreover, they have proved that no other search algorithm (other than the one that searches the first two levels) can establish any bound while searching only 2 n−1 coalition structures or fewer.This is because, in order to establish a bound, one needs to go through a subset of coalition structures in which every coalition appears at least once.3This implies that the smallest subset of coalition structures to be searched before a bound can be established is the one in which every coalition appears exactly once, and the only subset in which this occurs is the one containing all the coalition structures that belong to the first two levels of the graph.
If the first two levels have been searched, and additional time remains, then it would be desirable to lower the bound with further search.Sandholm et al. (1999) have developed an algorithm for this purpose.Basically, the algorithm searches the remaining levels one by one, starting from the bottom level, and moving upwards in the graph.Moreover, Sandholm et al. have also proved that the bound β is improved whenever the algorithm finishes searching a particular level.What is interesting here is that, by searching the bottom level (which only contains one coalition structure) the bound drops in half (i.e.β = n 2 ).Then, roughly speaking, the divisor in the bound increases by one every time two more levels are searched, but seeing only one more level helps very little (Sandholm et al., 1999). 4 This algorithm has the advantage of being anytime, and being able to provide worst case guarantees on the quality of the solution found so far.However, the algorithm has two major limitations: • The algorithm needs to search through the entire search space in order for the bound to become 1.In other words, to return a solution that is guaranteed to be optimal, the algorithm simply performs a brute-force search.As discussed in Section 1, this is intractable even for small numbers of agents.
• The bounds provided by the algorithm might be too large for practical use.For example, given n = 24, and given that the algorithm has finished searching levels LV 1 , LV 2 , and LV 24 (which contain 8,388,609 coalition structures) the bound would be β = n/2 = 12.This means that, in the worst case, the optimal solution can be 12 times better than the current solution.In other words, the value of the current solution is only guaranteed to be no worse than 8.33% of the value of the optimal solution.After that, in order to reduce the bound to β = n/4, four more levels need to be searched, namely LV 23 , LV 22 , LV 21 , and LV 20 .In other words, after searching an additional 119,461,563 coalition structures, the value of the solution is only guaranteed to be no worse than 16.66% of the optimal value.Similarly, to reduce the bound to β = n/6, the algorithm needs to search an additional 22,384,498,067,085 coalition structures only to guarantee that the value of the solution is no worse than 25% of the optimal value.Moreover, the guarantee does not go beyond 50% until the entire space has been searched.
Given the limitations of Sandholm et al.'s (1999) algorithm, Dang and Jennings (2004) developed an anytime algorithm that can also establish a bound on the quality of the solution found so far, but that uses a different search method.In more detail, the algorithm starts by searching the top two levels, as well as the bottom one (as Sandholm et al.'s algorithm does).After that, however, instead of searching through the remaining levels one by one (as Sandholm et al. do), the algorithm searches through specific subsets of the remaining levels.Figure 2 compares the performance of both algorithms, and, by looking at this figure, we can see that neither of the two algorithms significantly outperforms the other.Note, however, that both algorithms were not meant for the case where the entire space will eventually be searched.This is because if we had enough time to perform this search, then we would have used the dynamic programming algorithm, which performs this search much quicker.Instead, these algorithms were mainly developed for the cases where the space is too large to be fully searched, even when the dynamic programming algorithm is being used.
Having discussed two algorithms that use similar techniques (i.e. by Sandholm et al., 1999 andDang &Jennings, 2004), we now discuss a different approach that can also provide solutions anytime, and can establish worst-case guarantees on the quality of its solution.This involves the use of standard problem solving techniques that rely on general purpose solvers.In more detail, the coalition structure generation problem can be formulated as a binary integer programming problem (or 4. To be more precise, depending on the number of agents and the level searched, the bound will either be ˚n m ˇor ¨n m where m = 2, 3, • • • , n.However, to ease the discussion and without loss of generality, we will assume throughout the paper that the bound is simply n m . Figure 2: Given 24 agents, the figure shows, on a log scale, a comparison between the bound provided by Sandholm et al. (1999) and that provided by Dang and Jennings (2004), given different numbers of searched coalition structures.
a 0-1 integer programming problem), since any variable representing a possible coalition can either take a value of 1 (indicating that it belongs to the formed coalition structure) or 0 (indicating that it doesn't).Specifically, given n agents, the integer model for the CSG problem can be formulated as follows: Maximize where Z is an n × 2 n matrix of zeros and ones, X is a vector containing 2 n binary variables, and e T is the vector of n ones.In more detail, every line in Z represents an agent, and every column represents a possible coalition.As for X, having an element x i = 1 corresponds to coalition C i being selected in the coalition structure.The first constraint ensures that the selected coalitions are both disjoint and exhaustive.
Such an integer programming problem is typically solved by applying linear relaxation coupled with branch-and-bound (Hillier & Lieberman, 2005).However, the main disadvantage of this approach is the huge memory requirement, which make it only applicable for small numbers of agents (see Section 5 for more details).

Non-Exact Algorithms for Coalition Structure Generation
These algorithms do not provide any guarantees on finding an optimal solution, nor do they provide worst-case guarantees on the quality of their solutions.Instead, they simply return "good" solutions.However, it is the fact that they can return a solution very quickly, compared to other algorithms, that often makes this class of algorithms more applicable, particularly given larger numbers of agents.
Generally speaking, as long as there is some regularity in the search space (i.e., the evaluation function is not arbitrary), genetic algorithms have the potential to detect that regularity and hence find the coalition structures that perform relatively effectively.To this end, Sen and Dutta (2000) developed a genetic algorithm for coalition structure generation.The algorithm starts with an initial set of candidate solutions (i.e. a set of coalition structures) called a population, which then gradually evolves towards better solutions.This is done in three main steps: evaluation, selection, and re-combination.In more detail, the algorithm evaluates every member of the current population, selects members based on their evaluation, and constructs new members from the selected ones by exchanging and modifying their contents.More details on the implementation can be found in the paper by Sen and Dutta (2000).The main advantage of this algorithm is that it can return solutions anytime, and that it scales up well with the increase in the number of agents.However, the main limitation is that the solutions it provides are not guaranteed to be optimal, or even guaranteed to be within a finite bound from the optimal.Moreover, even if the algorithm happens to find an optimal solution, it is not possible to verify this fact.
Another algorithm that belongs to this class of algorithms is the one developed by Shehory and Kraus (1998).This algorithm is greedy and operates in a decentralized manner.The heuristics they propose (in order to reduce the complexity of finding an optimal coalition structure) involve adding constraints on the size of the coalitions that are allowed to be formed.Specifically, only the coalitions up to a size q < n are taken into consideration.The main advantage of this algorithm is that it can take into consideration overlapping coalitions.5Moreover, Shehory and Kraus prove that the solution they provide is guaranteed to be within a bound from the optimal solution.However, by optimal, they mean the best possible combination of all permitted coalitions.On the other hand, the algorithm provides no guarantees on the quality of its solutions compared to the actual optimal that could be found if all coalitions were taken into consideration.
To summarize, as discussed earlier, the main limitation of these algorithms is that they provide no guarantees on the solutions they generate while they search or when they terminate.However, these algorithms scale up well with the increase in the number of agents, making them particularly suitable for the cases where the number of agents is so large that no algorithm with exponential complexity can be executed in time.
After discussing the different approaches to the coalition structure generation problem, we can see that each of these approaches suffers from major limitations, making it either inefficient or inapplicable.This motivates our aim to develop more efficient CSG algorithms that can be applied to a wider range of problems, while taking into consideration the objectives outlined in Section 1.With this in mind, we first present in Section 3 a novel representation of the search space, and then present in Section 4 a novel algorithm that belongs to the first class of the aforementioned classification.As we will show, this algorithm avoids all the limitations that exist in state-of-the-art algorithms belonging to this class, and meets all of the design objectives placed in Section 1 on CSG algorithms.

Search Space Representation
In this section, we describe our novel representation of the search space (i.e. the space of possible coalition structures).Recall that the space representation employed by most existing anytime algorithms is an undirected graph (see Figure 1 for an example), where the vertices represent coalition structures (Sandholm et al., 1999;Dang & Jennings, 2004).This representation, however, forces all possible solutions to be explored in order to guarantee that the optimal one has been found.Given this, we believe an ideal representation of the search space should allow the computation of solutions anytime, while establishing bounds on their quality, and should allow the pruning of the space to speed up the search.With this objective in mind, in this section we describe just such a representation.In particular, it supports an efficient search for the following reasons.First, it partitions the space into smaller, independent, sub-spaces for which we can identify upper and lower bounds, and thus, compute a bound on the solutions found during the search.Second, we can prune most of these sub-spaces since we can identify the ones that cannot contain a solution better than the best one found so far.Third, since the representation pre-determines the size of coalitions present in each sub-space, agents can balance their preference for certain coalition sizes against the cost of computing the solution for these sub-spaces.Next, we formally define our representation of the search space, describe its algebraic properties, and describe how to compute worst case bounds on the quality of the solution that our representation allows us to generate.

Partitioning the Search Space
We partition the search space P by defining sub-spaces that contain coalition structures that are similar according to some criterion.The particular criterion we specify here is based on the integer partitions of the number of agents. 6Recall that an integer partition of n is a multiset of positive integers that add up to exactly n (Andrews & Eriksson, 2004).For example, given n = 4, the five distinct partitions are: and [1, 1, 1, 1]. 7 It can easily be shown that the different ways to partition a set of n elements can be directly mapped to the integer partitions of n, where the parts of the integer partition correspond to the cardinalities of the subsets (i.e. the sizes of the coalitions) within the set partition (i.e.coalition structure).For instance, the coalition structures {{a 1 , a 2 }, {a 3 }, {a 4 }} and {{a 4 , a 1 }, {a 2 }, {a 3 }} can be mapped to the integer partition [2, 1, 1] since they each contain one coalition of size 2, and two coalitions of size 1.We define the aforementioned mapping by the function F : P → G, where G is the set of integer partitions of n.Thus, F defines an equivalence relation ∼ on P such that CS ∼ CS iff F (CS) = F (CS ) (i.e. the sizes of the coalitions in CS are the same as those in CS ).Given this, the pre-image 8 of an integer partition G, noted as P G = F −1 [{G}], contains all the coalition structures that correspond 6.Other criteria could be developed to further partition the space into smaller sub-spaces, but the one we develop here allows us to choose coalition structures with certain properties as we show later.
7. For presentation clarity, square brackets are used throughout the paper (instead of the curly ones) to distinguish between multisets and sets.
8. Recall that the pre-image or inverse image of G ⊆ G under F : P → G is the subset of P defined by to the same integer partition G. Every such pre-image represents a sub-space in our representation.This implies that the number of sub-spaces in our representation is the same as the number of possible integer partitions, which grows exponentially with n.This number, however, remains insignificant compared to the number of possible coalitions and coalition structures (e.g., given 24 agents, the number of possible integer partitions is only 1575, while the number of possible coalitions is 16777215, and the number of possible coalition structures is nearly 4.4 × 10 17 ).
We categorize the sub-spaces into levels based on the number of parts within the integer partitions.Specifically, level P i = {P G : |G| = i} contains all the sub-spaces that correspond to an integer partition with i parts (see Figure 3 for an example of 4 agents). 9In what follows, we show how to compute bounds for the sub-spaces (P G : G ∈ G) in our representation.

Computing Bounds for Sub-spaces
For each sub-space P G , it is possible to compute an upper and a lower bound on the value of the best10 coalition structure that could be found in it.To this end, let L s be the list of coalitions of size s, and let max s , min s , and avg s , be the maximum, minimum, and average value of the coalitions in L s respectively.Moreover, given an integer partition G, let T G be the Cartesian product of the lists s) , where G(s) is the multiplicity of s in G.For example, given G = [5, 4, 4, 4, 1, 1], we have T G = (L 5 ) 1 × (L 4 ) 3 × (L 1 ) 2 .Note that T G contains many combinations of coalitions that are considered invalid coalition structures.This is because some of the coalitions within these combinations may overlap.For example, T [2,1,1] contains the following combination, {{a 1 , a 2 }, {a 1 },{a 3 }}, which is not a valid coalition structure because agent a 1 appears in two coalitions.Now, if we consider the value of each element (i.e.combination of coalitions) in T G to be the sum of the values of all the coalitions in that element, then the maximum value that an element in T G can take, denoted M AX G , is computed as follows: M AX G = s∈G max s × G(s).Based on this, it is easy to demonstrate that M AX G is an upper bound on the value of the best coalition structure in P G (since P G is a subset of T G ).
Similarly, the minimum value that an element in Although this could intuitively be considered a lower bound on the value of the best coalition structure (i.e.solution) in P G , we show that it is actually possible to compute a higher (i.e.better) lower bound than M IN G .
In more detail, let AV G G be the average value of all the coalition structures in P G .Then, AV G G would be a lower bound on the value of the best coalition structure in P G (since an average is always greater than, or equal to, a minimum).The key point to note, here, is that we can compute AV G G without having to go through any of the coalition structures in P G .Instead, we can compute it by simply summing the averages of the coalition lists (see Theorem 1), and these averages can be computed immediately after scanning the input, which is significantly smaller than the space of possible coalition structures.
be an integer partition, and let AV G G be the average of the values of all the coalition structures in P G .Also, let avg g i be the average of the values of all the coalitions in L g i .Then, the following holds: Having described our novel representation of the search space, we present (in the following section) an anytime algorithm that uses this representation to search through the possible coalitions structures to eventually find an optimal one.

Solving the Coalition Structure Generation Problem
Assuming that the value of every coalition C is given by a characteristic function v(C) ∈ R, and that the value of every coalition structure is given by the function V (CS) = C∈CS v(C), our goal is to search through the set of possible coalition structures, noted as P, in order to find an optimal coalition structure which is computed as: given v(C) for all C ∈ 2 A \{∅}.Note that, in this section, the terms "coalition structure" and "solution" will be used interchangeably.Basically, our novel anytime Integer-Partition based algorithm (which we call IP) consists of the following two main steps: 1. Scanning the input in order to compute the bounds (i.e.M AX G and AV G G ) for every subspace P G -while doing so, we can (at a very small cost): (a) find the best coalition structures within particular sub-spaces.
(b) prune other sub-spaces based on their upper-bounds.
(c) establish a worst-case bound on the quality of the best solution found so far.
2. Searching within the remaining sub-spaces -the techniques we use allow us to: (a) avoid making unnecessary comparisons between coalitions to generate valid coalition structures (i.e.those that contain disjoint coalitions).
(b) avoid computing the same coalition structure more than once.
(c) apply branch-and-bound to further reduce the amount of search to be done.
The following sub-sections describe each of the aforementioned steps in more detail.To this end, we will use CS to denote the best coalition structure found so far, and G ⊆ G to denote the integer partitions that represent the sub-spaces that have not been searched.

Scanning the Input
The input to the coalition structure generation problem is the value associated to each coalition, i.e. v(C) for all C ∈ 2 A \{∅}.One way of representing this input is to use a table containing every coalition along with its value.Another way is to agree on an ordering of the coalitions, and to use a list containing only the values of these ordered coalitions (i.e. the first value in the list corresponds to the first coalition, the second value corresponds to the second coalition, and so on).
We use the latter representation since it does not require maintaining the coalitions themselves in memory.In more detail, we assume that the input is given as follows: v(L s ) ∀s ∈ {1, 2, . . ., n}, where v(L s ) is a list containing the values of all the coalitions of size s.Moreover, we assume that the coalitions in L s are ordered lexicographically.For example, coalition {a 1 , a 2 , a 4 } has its elements ordered according to their indices, and the coalition itself is found above {a 1 , a 2 , a 3 } and below {a 1 , a 3 , a 4 } in the list L 3 (this is depicted in Figure 4).This ordering can easily be generated using the techniques that are used by Rahwan and Jennings (2007).Next, we describe the individual steps of the algorithm that depicts the scanning process (see Algorithm 1).
At first, we scan the value of the one coalition of size n (i.e. the grand coalition).This would be the value of the only coalition structure in P [n] (which is the only sub-space in P 1 ).After that, we scan the values of all the coalitions of size 1 (i.e.singleton coalitions), and by summing these values, we get the value of the only coalition structure in P [1,1,...,1] (which is the only sub-space in P n ).At this point (step 1), it is possible to compute the best coalition structure found so far (i.e.CS ).
Having searched through levels P 1 and P n , we now show how to search through level P 2 at a very low cost while scanning the input.To this end, let G 2 = {G ∈ G : |G| = 2} be the set of integer partitions that contain two parts each.Then, as a result of the assumed ordering of the input, any two complementary coalitions C and C in a coalition structure CS = {C, C} are always diametrically positioned in the coalition lists L |C| and L | b C| , and that happens even if |C| = |C |.For example, given 6 agents, the coalitions {a 1 } and {a 2 , a 3 , a 4 , a 5 , a 6 } are diametrically positioned in x end for 25:   Based on this, for every integer partition G = [g 1 , g 2 ] ∈ G 2 , we compute the values of all the coalition structures in P G by simply summing the values of the coalitions as we scan the lists v(L g 1 ) and v(L g 2 ), starting at different extremities for each list.Once these lists have been scanned (steps 10 to 24), it is possible to obtain the two values of which the sum is maximized.Moreover, it is possible to obtain the indices in the lists at which these values are located (see how x max and x max are computed in steps 15 and 25 respectively).Then, by obtaining these indices, we know where in L g 1 and L g 2 to find the two coalitions that belong to the best coalition structure in P [g 1 ,g 2 ] (this comes from the fact that the position of any value in v(L s ) : s ∈ {1, ..., n} is exactly the position of the corresponding coalition in L s ).
Note, however, that the input includes only v(L g 1 ) and v(L g 2 ) (i.e. it does not include L g 1 and L g 2 ).For this reason, an algorithm is required that can return a coalition C given its position in the ordered list L |C| .Rahwan and Jennings (2007) have developed a polynomial-time algorithm that does exactly that.Therefore, we use it to find the required coalitions and compose the best coalition structure in P {g 1 ,g 2 } (see steps 26 and 27). 11hile scanning v(L g 1 ) and v(L g 2 ), we also compute max g 1 and max g 2 (steps 17 to 22), as well as avg g 1 and avg g 2 (step 29).Note that, in Algorithm 1, we scan v(L s ) and v(L n−s ) for all s ∈ {1, . . ., n 2 } and this implies that max s and avg s are computed for all s ∈ {1, . . ., n}.Also note that this whole process is linear in the size of the input (i.e.O(y) where y = 2 n − 1 is the size of the input).
Having computed max s and avg s for every size s, we can now compute upper and lower bounds for every sub-space (as in steps 32 to 34).By using these bounds, it is possible to compute an upper bound U B * and a lower bound LB * on the value of the optimal coalition structure (see steps 36 and 37).Hence, every sub-space P G that has an upper bound M AX G < LB * can be pruned straight away.The prune function (used in step 38) is implemented as in Algorithm 2.
end if 5: end for 6: return G Another advantage of our scanning procedure is that it allows us to compute a worst-case bound β on the value of CS as follows: β = min( n 2 , U B * V (CS ) ) (see step 39).This comes from the fact Sandholm et al. (1999) have proved that the value of the best coalition structure in levels LV 1 , LV 2 and LV n (corresponding to P 1 , P 2 , and P n respectively) is within a bound n 2 from the optimal.So far, by only scanning the input, we have calculated max s and avg s for all s ∈ {1, . . ., n}, we have searched levels P 1 , P 2 , P n , we have calculated M AX G and AV G G for all the sub-spaces within the remaining levels (i.e.P 3 , ..., P n−1 ), we have pruned some of these sub-spaces, and we have established a worst-case bound β on the quality of the best solution found so far.Moreover, it is possible to specify a bound β * ≥ 1 within which any solution is acceptable.In more detail, if the best solution found so far fits within the specified bound (i.e. if β ≤ β * ) then no further search is required.Otherwise, the sub-spaces that have not been pruned (if there are any) must be searched.Next, we specify how this search is done.

Selecting and Searching a Sub-Space
Given the set of sub-spaces left after scanning the input, we select a sub-space to be searched, and we find the best coalition structure in it.After that, we prune all the remaining sub-spaces that have an upper bound lower than the best value found so far.This process of selecting, searching, and pruning, is repeated until either of the following termination conditions is reached: • The best coalition structure found so far fits within the specified bound β * .
• All the remaining sub-spaces have either been searched or pruned.
This can be seen in Algorithm 3. Basically, the algorithm works as follows.A sub-space P G is selected to be searched (step 2). 12Once P G has been searched (step 3), it is removed from the set of remaining sub-spaces (step 4).After that, we check whether CS has been modified during the search (step 5), and, if that is the case, then every sub-space with an upper bound lower than V (CS ) is pruned (step 6). 13U B * and β are then updated in steps 8 and 9 respectively, and if the current 12.In step 2, we actually select an integer partition, but this implies that the corresponding sub-section is to be searched.
13. Checking whether CS belongs to P G can easily be done by checking whether the sizes of the coalitions in CS match the parts in G .
best solution fits within the specified bound β * then it is returned (steps 10 and 11).Otherwise, this whole process is repeated given the remaining sub-spaces (if there are any).In what follows, we further elaborate on the sub-space selection strategy and the sub-space search algorithm since these are the key parts of this algorithm.
Select G {select the integer partition that represents the next sub-space to be searched.}

4:
G ← G \ G {remove P G from the list of sub-spaces that are yet to be searched.}5: if CS ∈ P G {If CS has been modified while searching P G .} then 6: {prune the sub-spaces that have upper bounds lower than V (CS ).} 7: ] {update the upper bound on the value of the optimal coalition structure(s).}9: , β] {update the worst-case bound on V (CS ).} 10: if β ≤ β * {if CS is within the specified bound from the optimal.}then 11: return CS 12: end if 13: end while 14: return CS

SELECTING A SUB-SPACE
It can easily be seen that, unless we search the sub-spaces that have an upper bound greater than V (CS ), we cannot verify that CS is an optimal solution.This implies that β remains greater than 1 until the following sub-spaces are searched: {P G : M AX G ≥ V (CS * )}.This can be done by selecting the next sub-space to be searched using the following selection rule: As a result of this selection strategy, all sub-spaces with an upper bound lower than V (CS * ) will not be searched and these can constitute a significant portion of the search space (see Section 5.3 for more details).Another result is that it will always be beneficial to search a sub-space, even if that sub-space does not contain a better solution than the one found so far.This is because the above selection strategy ensures that U B * is reduced whenever a sub-space is searched, and this improves the worst-case guarantee β on the quality of the current best solution.
Note that this selection rule is mainly for the cases where an optimal solution is sought.In case we are after a near-optimal solution where a bound β * > 1 is specified (e.g., β * = 1.05 means that the solution sought needs to have a value that is at least 95% of the optimal one), then other selection rules may be used.For example, one could select to search the smallest sub-space that could, potentially, give a value greater than or equal to U B * β * (hoping to find an acceptable solution in the least amount of search).This can be expressed as: where |P G | is the size of (i.e. the number of coalition structures in) P G .More specifically, |P G | is computed as follows: Theorem 2. Let G = [g 1 , . . ., g |G| ] be an integer partition, and let |P G | be the number of coalition structures in P G .Moreover, let C n s be the binomial coefficient, 14 and let E(G) be the underlying set of G. 15 Then, the following holds: The key point to note is that, given our representation, we can specify β * in cases where computing the optimal solution would be too costly and, given this, we can modify the selection rule accordingly to speed up the search.
Another advantage of being able to control the sub-spaces to be searched is that the agents can choose what types of coalition structures to build according to their computational resources or private preferences.For example, it has been argued that the computation time could be reduced if we limit the size of the coalitions that can be formed (Shehory & Kraus, 1998).However, this is a very costly, self-imposed constraint since it possibly means neglecting a number of highly efficient solutions.Instead, by using IP, it is possible to determine, ex-ante (i.e.before performing the search), which sub-spaces are most promising according to their upper and lower bounds.Therefore the computation time can be focused on these sub-spaces and the gains can be traded-off against the computation time.
In some other cases, agents may need to form q coalitions (Shehory & Kraus, 1995).For example, they may need to perform q tasks and therefore need to divide up into q teams to perform these tasks separately.Moreover, they may wish to have coalitions with a maximum size of z as they may have certain constraints on the amount of resources available to each coalition.By using our representation, such preferences can be naturally expressed and the search can be directed to fit these preferences transparently.Formally, our search space can easily be redefined as follows: In all the above cases where agents can express preferences for coalition structures of certain sizes, they can now, a priori, balance such preferences with the quality of the solutions that can be 14.Recall that the binomial coefficient represents the number of possible combinations of size s taken from n elements, and is computed as follows: , where n! is the factorial of n. 15.Recall that the underlying set E(G) of a multiset G is a subset of G in which each element in G appears only once in E(G).For example, {1, 2} is the underlying set of [1, 1, 2].
obtained.This is because we are able to determine the worst-case bound from the optimal that the search of a given sub-space will generate (i.e.U B * AV G G ).We next describe how we search through the chosen sub-space.

SEARCHING A SUB-SPACE
Given an integer partition G = [g 1 , g 2 , • • • , g |G| ] ∈ G, we need to cycle through the coalition structures that belong to P G in order to find the best one.Here, without loss of generality, we assume that g 1 ≤ g 2 ≤ • • • , ≤ g |G| .Perhaps the most obvious way of performing this cyclation process is shown in Figure 5. Here, a variable |G| is used to cycle though the coalition structures in P G as follows.First, C 1 is assigned to one of the coalitions in L g 1 .After that, C 2 is used to cycle through L g 2 until a coalition that does not overlap with C 1 is found.After that, C 3 is used to cycle through L g 3 until a coalition that does not overlap with {C 1 , C 2 } is found.This is repeated until every C k ∈ − → CS is assigned to a coalition in L g k .In this case, − → CS would be a valid coalition structure belonging to P G .The value of this coalition structure is then calculated and compared with the maximum value found so far.After that, the coalitions in − → CS are updated so as to set − → CS to another coalition structure in P G .Here, a coalition C k is only updated once we have examined all the possible instances of C k+1 , . . ., , C |G| that do not overlap with {C 1 , . . ., C k }.For example, in Figure 5, we only update C 2 (step 5 in the figure) once we have examined all the possible instances of C 3 that do not overlap with {C 1 , C 2 } (steps 2, 3, 4 in the figure).This ensures that − → CS is assigned to different coalition structures, and that, eventually, every possible coalition structure in P G is examined.
Next, we show how this process can be done without storing any of the lists L g 1 , L g 2 , • • • , L g |G| in memory.To this end, let LC n g k : 1 ≤ g k ≤ n be the list of combinations of size g k that are taken from the set {1, 2, • • • , n}, where the combinations are ordered lexicographically in the list.Given this, both LC n g k and L g k contain the subsets of size g k that are taken from a set of size n.The only difference is that LC n g k is a list of combinations of numbers while L g k is a list of coalitions of agents.Now, Rahwan and Jennings (2007) have shown how to cycle through the combinations in LC n g k without storing the entire list in memory.Instead, only one combination is stored at a time.This is based on the assumed ordering which implies that the last combination in LC n g k is always: This ordering also implies that, given any combination located at index x in the list, where 1 < x ≤ LC n g k , it is possible to compute the combination located at index x − 1 (for more details, see the paper by Rahwan & Jennings, 2007).Hence, in order to go through the coalitions in L g k , we use a variable M k to cycle 16 through the combinations in LC n g k and, for every instance of M k , we extract the corresponding coalition C k ∈ L g k using the following operation: For example, given that M k = {2, 4, 5}, the corresponding coalition would be {a 2 , a 4 , a 5 }.Since there is a direct mapping (as defined by equation 2) from every combination in LC n g k to a coalition in L g k , then, by having M k cycle through every combination in LC n g k , we cycle through all the coalitions in L g k .
16.This can be done by initializing M k to the last combination in LC n g k (i.e. to {1, 2, • • • , g k }), and then iteratively shifting M k up in the list as in the paper by Rahwan and Jennings (2007), until every combination in LC n g k is examined.
Figure 5: A naïve cyclation process for cycling through the coalition structures in a sub-space.
Intuitively, this naïve cyclation process, which we call NCP, can be viewed as being efficient.After all, what we need is to find the coalition structure in P G that has the maximum value, and NCP guarantees to find such a coalition structure.However, it suffers from the following major limitations: 1. NCP works by searching through the ordered sets in T G -the Cartesian product of the lists L s : s ∈ G -in order to find those that belong to P G (i.e.those that contain disjoint coalitions).This is a major limitation since the space of coalition structures is already exponentially large, and it would be counter-intuitive to search for it in an even bigger space.For example, given 28 agents, the number of coalition structures in P [1,2,3,4,5,6,7] is only 7.8 × 10 −9 % of the number of ordered sets in T [1,2,3,4,5,6,7] .Note that the difference in size between the two spaces grows exponentially with the number of agents involved.
2. Although NCP does not generate the same ordered set twice, it generates multiple ordered sets containing the same coalitions, but ordered differently.For example, given NCP generates the following ordered sets, {a 1 , a 2 }, {a 3 , a 4 }, {a 5 , a 6 , a 7 } and {a 3 , a 4 }, {a 1 , a 2 }, {a 5 , a 6 , a 7 } , which correspond to the same coalition structure.Note that we need to find the best coalition structure and, in order to do so, it is sufficient to examine the value of every coalition structure once.In other words, any operation that results in the same coalition structure being generated more than once is considered redundant.
What would be desirable, then, is to find a way to cycle through the lists L g 1 , . . ., L g |G| such that only valid combinations are generated.In other words, it would be desirable if C k only cycles through the valid coalitions in L g k , rather than going through every coalition in L g k and verifying whether it overlaps with {C 1 , . . ., C k−1 }.Moreover, in order to avoid performing any redundant operations, it would be desirable if the cyclation process is guaranteed not to go through the same coalition structure more than once.Algorithm 4 describes a novel cyclation process that meets these requirements.
The basic idea is to use the searchList function to cycle through the coalitions in L g 1 .For each of these coalitions, searchList is called recursively17 to cycle through the coalitions in L g 2 that do not overlap with the first coalition (i.e. the one taken from L g 1 ).Similarly, while cycling through L g 2 , searchList is called recursively to cycle through the coalitions in L g 3 that do not overlap with the first two coalitions, and so on.This is repeated until searchList is called to cycle through the coalitions in L g |G| , in which case we have a valid coalition structure (denoted − → CS in Algorithm 4) that belongs to P G .Then, if − → CS has a value that is greater than V (CS ) then CS is updated accordingly.The remainder of this section describes how Algorithm 4 avoids generating invalid or redundant coalition structures without making any comparison between coalitions.It also describes how the algorithm applies a branch-and-bound technique to speed up the search.
Avoiding invalid coalition structures: Given G = [g 1 , . . ., g |G| ], we define the following ordered sets of agents: Moreover, we assume that the agents in {stop if the required solution has been found or if the current best is equal to the upper bound of this sub-space.}then 12: return CS 13: end if 14: end for 15: return CS ascendingly based on their indices in A (e.g. if A k contains agents a 5 , a 7 , and a 2 , then the order would be A k = a 2 , a 5 , a 7 ).In other words, we assume that: respectively, we show how to cycle through the coalitions in L g k that do not overlap with any of the aforementioned ones, and that is without storing L g k in memory.In more detail, this can be done using the following modifications over NCP: • Instead of using M k to cycle through the combinations in LC n g k (as in NCP), we use it to cycle through the combinations in LC • For any given instance of M k , we extract the corresponding coalition C k ∈ L g k using the following operation: For example, given M k = {1, 3, 5}, the corresponding coalition does not contain agents a 1 , a 3 , and a 5 (as in NCP).Instead, it contains the 1 st , the 3 rd , and the 5 th element of A k .
These differences ensure that M k cycles through all the possible coalitions of size g k that are taken from A k (instead of those taken from A).Based on this, if we set we ensure that every instance of C k does not overlap with any of the coalitions Figure 6 shows an example given A = A 1 = {a 1 , a 2 , a 3 , a 4 , a 5 , a 6 , a 7 } and G = [2, 2, 3].As can be seen, having M 1 = {1, 6} implies that C 1 contains the 1 st and 6 th agents in A 1 (i.e. it implies that C 1 = {a 1 , a 6 }).By knowing the agents that belong to C 1 , we can then assign A 2 to those that do not belong to C 1 , i.e.A 2 = {a 2 , a 3 , a 4 , a 5 , a 7 } (see how the agents in A 2 are ordered based on their indices in A).As mentioned earlier, M 2 would then cycle through all the possible coalitions of size 2 out of A 2 , and none of these coalitions would overlap with C 1 .Similarly, having M 2 = {3, 5} implies that C 2 contains the 3 rd and 5 th elements of A 2 (i.e. it implies that C 2 = {a 4 , a 7 }), and by knowing the agents that belong to C 2 , we can then assign A 3 to those that do not belong to C 1 or C 2 (i.e.A 3 = {a 2 , a 3 , a 5 }), and so on.The modified cyclation process (MCP), which we describe above, generates all the coalition structures in P G (see Theorem 3), and that is without performing any comparison between the coalitions.
Theorem 3. Given an integer partition G ∈ G, every coalition structure in P G is generated by MCP.
Proof.See Appendix D.
Note, however, that MCP suffers from the same limitation of NCP in that it could generate the same coalition structure more than once (e.g.given G = [2, 2, 3], both {a 1 , a 2 }, {a 3 , a 4 }, {a 5 , a 6 , a 7 } and {a 3 , a 4 }, {a 1 , a 2 }, {a 5 , a 6 , a 7 } are generated by MCP).Next, we show how this can be avoided.
Avoiding redundant coalition structures: We note that, by using MCP, the same coalition structure can only be generated twice if there are repeated parts in the integer partition G (e.g.G = [1, 2, 2, 3] or G = [1,4,4,4,6]).This is because MCP generates ordered sets C 1 , • • • , C |G| containing disjoint coalitions of which the sizes match the parts in Based on this, if an ordered set − → CS is generated by MCP, then, any other ordered set − → CS that contains the same coalitions but with a different order (compared to − → CS) will also be generated by MCP as long as the sizes of the coalitions match the parts in G. This, of course, can only happen if we have g k = g j : k = j.Based on this, MCP only needs to be modified for the cases where there are repeated parts in G. 19 This modification is done as follows: , and α = 1 otherwise (see step 4 of Algorithm 4).This is illustrated in Figure 6 using the connected boxes.In more detail, M 1 only cycles through the combinations in LC 7 2 that are contained in boxes (e.g. it does not cycle through combinations {5, 6}, {5, 7}, and {6, 7}).Moreover, M 2 only cycles through the combinations in LC 5 2 that are contained in boxes connected to the one in which M 1 is currently cycling.This modification ensures that M k+1,1 ≥ M k,1 when g k+1 = g k .For example, while M 1 is cycling through the box in LC 7 2 containing the combinations in which the smallest element is 3, we have M 1,1 = 3.In this case, M 2 only cycles through the boxes in LC 5 2 containing the combinations in which the smallest element is 3 or 4 (see how these boxes are connected in Figure 6), and this ensures that M 2,1 ≥ M 1,1 .
The final cyclation process (FCP), which we describe above, generates every coalition structure in P G exactly once.
Theorem 4. Given an integer partition G ∈ G, every coalition structure in P G is generated exactly once by FCP.
Proof.See Appendix E. Note, however, that given the exponential size of P G , it would be more desirable if we can avoid generating any coalition structure with no potential of having a value greater than the maximum one found so far.Next, we show how this can be done using a branch-and-bound technique.
Applying Branch-and-Bound: As mentioned earlier, when cycling through the coalition structures in P G , we only update C k once we have examined all the possible instances of {C k+1 , . . ., C |G| } that do not overlap with {C 1 , . . ., C k }.In other words, we only update C k once we have examined all the possible coalition structures that start with {C 1 , . . ., C k }.However, if we knew that none of these coalition structures could have a value greater than the maximum value found so far, then we could update C k straight away (i.e.without having to go through any of the possible instances of {C k+1 , . . ., C |G| }).In order to do so, we calculate an upper bound on the values of the coalitions that can be added to {C 1 , . . ., C k }.Specifically, having computed max s for every possible coalition of size s ∈ {1, 2, . . ., n}, we can then calculate such an upper bound, denoted M AX [g k+1 ,...,g |G| ] , as follows: ..,g |G| ] represents an upper bound on the value of the coalition structure that could be obtained with a coalition structure starting with {C 1 , • • • , C k } and ending with coalition sizes [g k+1 , . . ., g |G| ].
Hence, having V (CS ) ≥ V ({C 1 , ..., C k }) + M AX [g k+1 ,...,g |G| ] implies that none of the coalition structures that start with {C 1 , ..., C k } and end with coalitions of sizes: g k+1 , ..., g |G| has a value greater than V (CS ) (this is checked in step 8 of Algorithm 4).On the other hand, having V (CS ) < V ({C 1 , . . ., C k }) + M AX [g k+1 ,...,g |G| ] implies that there could be a coalition structure that starts with {C 1 , • • • , C k } and is better than the current best coalition.However, this still does not necessarily imply that all of these coalition structures need to be examined.This is because, when the algorithm moves to the next list, it may find that there are certain coalition structures that are not better than the current best.Formally, for every coalition C j : k < j < |G|, we can still have: 7 illustrates how this branch-and-bound technique is applied while searching a sub-space.To remedy this, we propose a new input distribution that is tailored specifically to the CSG problem.This distribution, which we define as NDCS (Normally Distributed Coalition Structures), is constructed by generating coalition values in the following way: , where µ = |C| and σ = |C|.
In this case, it turns out that the value of every possible coalition structure is independently drawn from the same normal distribution which leads us to the following theorem: Theorem 6. Iff we have: ∀C ⊆ A, v(C) ∼ N (µ, σ 2 ), where µ = |C| and σ = |C|, then the following holds: Since the NDCS distribution ensures that every coalition structure value is drawn from the same distribution, it ensures that the search space is not biased.Thus, the efficiency of search algorithms in finding the optimal coalition structure is more strongly tested than in the other cases.
Using the above input distributions, we benchmark our algorithm against the other state-of-theart algorithm, namely IDP (see Section 2).Note that we do not experiment with the other anytime algorithms since they need to search the whole space to find the optimal value and this is generally not feasible within reasonable time, even for small numbers of agents.Also, it was shown by Rahwan et al. (2007b) that industrial strength software such as CPLEX cannot handle inputs of more than 18 agents since it runs out of memory and therefore we do not run experiments with it here.On all our graphs we plot the 95% confidence interval at every point (given 800 runs for 15 to 20 agents and 100 runs for 21 to 25 agents). 21

Experiment 1: Optimality
In this experiment, we compare the algorithms' performances given different numbers of agents (from 15 to 27).The time to find the optimal coalition structure is measured in terms of clock time (in milliseconds) on an Intel 2.6GHz Quad Core PC with 3Gigabytes of RAM.The algorithms are coded using JAVA 1.6.The running times are plotted on a log scale in Figure 8. 22 We note as IP-X the application of IP to distribution X, where X can be NDCS, Normal, or Uniform (as described above).As can be seen, IP finds the optimal coalition structure significantly faster than IDP for all distributions.In the best case (Uniform for 27 agents) IP is 570 times better than IDP (i.e. it takes 0.175% of the time taken by IDP) and in the worst case (NDCS for 16 agents) it is 1.7 times faster than IDP.It can also be seen that the performance of IP is the slowest given the NDCS distribution (compared to IP-Normal and IP-Uniform).To determine the cause for this, we first discuss the two main problems that can affect the performance of IP: 21.By plotting the 95% confidence interval, we aim to check statistical significance of the difference between the means taken at each point across different series.Thus, if two points from two different series have overlapping confidence intervals, it is equivalent to saying that the null hypothesis is validated (i.e. the means are not significantly different) for a t-test with α = 0.05.If the confidence intervals do not overlap, then the means are significantly different.
22. The running time for IDP is deterministic since it runs in O(3 n ).Hence, we recorded its running time for up to 25 agents and extrapolated the results to 27 agents.1. Pruning sub-spaces: the higher the upper bounds of sub-spaces and the lower the value of the optimal coalition structure, the harder it is to prune sub-spaces.This can be deduced from the pruning function we use in Algorithm 2.Moreover, the bigger the sub-spaces with higher upper bounds, the longer the algorithm will take to find the optimal solution.This is because the algorithm always has to search the sub-space with the highest upper bound to check that the solution it has found is optimal.
2. Branch-and-bound: the higher the upper bounds of sub-spaces and the lower the optimal coalition structure value, the harder it is to prune with branch-and-bound.This can be deduced from the pruning applied in step 8 of Algorithm 4. This is because, when applying branchand-bound within a sub-space P {g 1 ,g 2 ,...,gn} , the current best solution CS is compared against the sum of coalition values and the maximum value of coalitions of the remaining coalition sizes as follows: max g then move to next coalition structure Now, if the best solution is very low compared to the upper bound, that is: then, branch-and-bound has to be applied deeper (i.e.increasing variable k in the condition above) in the sub-space in order to make sure that the coalition structure being evaluated is not optimal.Hence, in the worst case it would have to search the whole sub-space (i.e.apply step 9 in Algorithm 4 on increasing values of k up to n).
In order to see how these different issues affect the performance of our algorithm with respect to different distributions, we recorded the value of the optimal coalition structure and the upper bounds of all the sub-spaces (given 21 agents) and averaged them over 20 runs. 23We also exactly recorded the size of each sub-space (i.e. in the number of coalition structures per sub-space).The results are plotted in Figure 9.We note the following for each distribution: • NDCS: The biggest sub-spaces are the ones with the highest upper bounds.Hence, it is much harder to prune large portions of the search space.Moreover, the average optimal coalition structure value is relatively low compared to the upper bounds of the bigger subspaces.Hence, applying branch-and-bound in this distribution is very hard.
• Normal: The smaller sub-spaces are the ones with the highest upper bounds.Hence, pruning large portions of the space can easily be done by searching smaller sub-spaces in which good solutions are.Moreover, the value of the optimal coalition structure tends to be higher than the upper bounds of most large sub-spaces, and relatively close to the highest upper bounds.Hence, it is easier for branch-and-bound to prune large portions of the sub-spaces.
• Uniform: The upper bounds of most sub-spaces are relatively high compared to those of other distributions (i.e. they are close to the highest upper bound).In fact, the upper bounds are actually nearly equal to the average optimal solution and this allows the algorithm to prune most of the sub-spaces as soon as it has found an optimal solution, and this happens almost immediately after scanning the input.
Finally, note that Figure 9 shows the portion of the space that will be avoided given the selection strategy described earlier in Section 4.2.In more detail, recall that this strategy is guaranteed to avoid searching the sub-spaces that have an upper bound lower that V (CS * ).As can be seen from the figure, many of the sub-spaces (in the case of NDCS and Uniform distributions) have an upper bound lower than V (CS * ), although most of these sub-spaces are relatively small.Moreover, in the case of the Normal distribution, almost all the sub-spaces have an upper bound lower than V (CS * ), most of which are among the largest ones!Having studied the performance of IP in terms of completion time, we next focus on studying its ability to generate solutions anytime.

Experiment 2: Anytime Quality
In this experiment, we further evaluate the anytime property of our algorithm, and that is by recording the value of the solutions that were generated before returning the guaranteed optimal one.In particular, we recorded two indicative measures of the quality of the solutions.First, we computed the ratio between the value of the current best solution and the optimal solution (obtained at the end of the run).This ratio is noted as r opt = V (CS ) V (CS * ) .This measure shows how effective the algorithm is at zooming on good solutions.Second, we recorded the ratio r bound between the value of the current best solution and the upper bound on the optimal value (i.e.r bound = V (CS ) U B * ).This measure is the theoretical guarantee that the algorithm places on the quality of the solution (see Section 4.1).Ideally, the algorithm should be able to minimise the difference between r opt and r bound in minimal time.
The results are plotted in Figure 10 for the distributions: NDCS, Normal, and Uniform. 24We discuss the results for each of the distributions in turn.
• NDCS: As can be seen, the algorithm very high quality guarantees (i.e.r bound > 90%) in less than half of the time required to find the optimal solution.It also produces a very high quality solutions (i.e.r opt > 90%) within less than 10% of the time required to terminate.
Figure 10: Quality (r opt ) and bound (r bound ) for the generated solution.In all cases, the x-axis represents the time (in milliseconds) and the y-axis represents the ratio of the solution to the optimal.
• Normal: In this case, our algorithm is able to come up with guaranteed high quality solutions much faster than for the NDCS distribution.Moreover, in this case, very high quality solutions (i.e.r opt > 90%) can be guaranteed (i.e.r bound > 90%) in less than 10% of the time to find the optimal value.This results from the fact that the upper bounds are not as far from the optimal value as in the NDCS case.
• Uniform: As expected from earlier results presented in Section 5.2, the algorithm generates very high quality solutions (i.e.r opt ≈ 100%) faster than for the other distributions (shortly after scanning the input).Moreover, the solutions can be guaranteed to be near-optimal (i.e.r bound > 99%) within 15% of the time to find the optimal.
Next, we compare the worst-case guarantees that are provided by IP with those provided by Sandholm et al.'s (1999) and Dang and Jennings's (2004) algorithms (see Figure 11).As can be seen, our algorithm significantly outperforms both Dang and Jennings's and Sandholm et al.'s for all distributions.In particular, after scanning the input, IP is able to guarantee that its solution is nearly 40% (in the worst case) of the optimal compared to below 10% for the other algorithms.Moreover, our guarantee usually reaches 100% after searching minute portions of the search space (on average around 0.0000019% for the hardest distribution), while the guarantees provided by other algorithms do not go beyond 50% until the whole space has been searched.Also note that we generate very high quality solutions (i.e.> 90%) by searching even smaller portions of the of the search space (on average around 0.0000002% for the hardest distribution).Thus, in actual computational time, for 25 agents for example, we are able to return a solution that is guaranteed to be higher than 90% of the optimal in around 250 seconds in the worst case and 300 milliseconds in the best case.

Conclusions and Future Work
Coalition formation, the process by which a group of software agents come together and agree to coordinate and cooperate in the performance of a set of tasks, is an important form of interaction in multi-agent systems.Such coalitions can improve the performance of the individual agents and/or the system as a whole, especially when tasks cannot be performed by a single agent, or when a group of agents performs the tasks more efficiently.One of the most challenging problems that arise in the coalition formation process is that of coalition structure generation, which involves partitioning the set of agents into exhaustive and disjoint coalitions such that the social welfare is maximized.
In this paper, we have developed and evaluated an anytime integer-partition based algorithm (called IP) that finds optimal solutions much faster than any previous algorithm designed for this purpose.The strength of our approach is founded upon two main components: • We use a novel representation of the search space which partitions it into smaller, disjoint sub-spaces that can be explored independently to find optimal solutions.This representation, which is based on the integer partitions of the number of agents involved, allows the agents to balance the trade-offs between their preferences for certain coalition sizes against the computation required to find the solution.Moreover, such trade-offs can be made in an informed manner since we can compute bounds on sub-spaces of the search space.These bounds allow us to prune the search space and guarantee the quality of the solution found during the search.They may also, depending on the distribution of the input values, allow us to obtain the optimal solution almost immediately after scanning the input.• We devise a technique that allows us to cycle through the coalition structures within a given sub-space.Unlike a naïve cyclation technique that generates combinations of coalitions, and verifies whether each of these combinations is a valid coalition structure, our cyclation technique only generates valid ones (thus, avoiding the search through the space of possible combinations of coalitions, which is exponentially larger than the space of coalition structures).
In addition, the cyclation technique does not perform any redundant operations since it avoids generating the same coalition structure more than once.Finally, by applying a branch-andbound technique, we are able to identify the coalition structures that cannot improve on the quality of the solution found so far, and thus, avoid generating them.
Altogether, these components allow us to make significant performance gains over other existing approaches.In more detail, the experiments show that IP avoids searching most of the search space, and therefore, requires significantly less time, compared to the other algorithms, in order to return an optimal solution.For example, IP outperforms IDP by orders of magnitude (0.175% of the time taken by IDP for 27 agents in the best case).Moreover, if IP is interrupted before an optimal value is found, it can still return solutions that are very close to the optimal (usually above 95% of the optimal), with very high worst-case guarantees on them (usually above 90%).These solutions are always better (above 40% of the optimal right after scanning the input) than those returned by Sandholm et al.'s (1999) and Dang and Jennings's (2004) algorithms (i.e. less than 10% of the optimal).These algorithms also have to search a large portion of the search space before being able to get better guarantees while our algorithm is able to prune and find near-optimal solutions relatively quickly (above 90% of the optimal within 10% of the time to find the optimal solution for 25 agents).
A number of important extensions to IP could be envisaged.For example, we have recently combined the IDP algorithm with IP (IP-IDP) (Rahwan & Jennings, 2008a) and will explore other approaches including linear programming techniques to improve the bounds used in IP.However, these extensions have to deal with an exponential input (i.e. 2 n memory locations at least for n agents) as we do in IP.Therefore, it is important to develop techniques that will extend our approach in order to minimise cycling through all coalition values as the number of agents increases.This will require adapting our cyclation technique and the bound computation.Hence, in future work, we will need to devise representations for sub-spaces that allow us to cycle more intelligently over larger inputs and develop new techniques to compute bounds to be used by our branch-andbound algorithm.In trying to adapt our approach to other problems, we also aim to determine the degree to which IP can be used to solve other common incomplete set partitioning problems which occur in combinatorial auctions (Rothkopf et al., 1995) or crew scheduling (Hoffman & Padberg, 1993).Finally, we aim to see whether the patterns that we exploit in our algorithm also arise in other combinatorial optimisation problems that have been studied in the area of combinatorics (e.g., Kreher & Stinson, 1998;Papadimitriou & Steiglitz, 1998).

Acknowledgments
The research in this paper was undertaken as part of the ALADDIN (Autonomous Learning Agents for Decentralised Data and Information Systems) project and is jointly funded by a BAE Systems and EPSRC (Engineering and Physical Research Council) strategic partnership (EP/C548051/1).Andrea Giovannucci was funded by the Juan de la Cierva programme (JCI-2008-03006) and the EU funded Synthetic Forager project (ICT-217148-SF).We also wish to thank Professor Tuomas Sandholm for his comments, as well as the anonymous reviewers for their valuable comments on previous versions of the paper.We are also very grateful to Dr. Viet Dung Dang for his contributions to earlier versions of the paper.Finally, we wish to thank to Dr. W. T. Luke Teacy for his help with some of the proofs and the anonymous reviewers for their very constructive comments.increasing order.That is: Now, the way MCP works is by generating ordered sets of coalitions such that, for every ordered set, the first coalition belongs to L g 1 and the second belongs to L g 2 and so on.Moreover, the way these ordered sets are generated ensures that the coalitions in each of the ordered sets do not overlap.In other words, MCP generates a subset of T G , denoted TG , which is defined as follows: 26 = { {a 1 }, {a 2 }, {a 3 , a 4 } , {a 2 }, {a 1 }, {a 3 , a 4 } }.Next, given any coalition structure CS ∈ P G , we will prove that | T CS G | ≥ 1.To this end, let C 1 , C 2 , . . ., C |G| be an ordering on the coalitions that belong to CS.Then, from ( 7) and ( 8), we can see that: Based on this, if we denote by c k the smallest28 agent in C k , then it is sufficient to prove that FCP only generates the one permutation that satisfies: c x < c x+1 < ... < c x+y .Note that the agents in A k are ordered such that A k,1 < A k,2 < • • • < A k,|A k | .Based on this, if c k = A k,i , then there are i − 1 agents in A k that are smaller than c k , and since A k+1 = A k \C k , then there are i − 1 agents in A k+1 that are smaller than c k .Therefore, to ensure that c k < c k+1 , it is sufficient to generate C k+1 such that it does not contain the first (i.e.smallest) i − 1 agents of A k+1 .For example, given A k = a 1 , a 4 , a 5 , a 7 , a 8 , a 9 and M k = {3, 5}, we would have C k = {a 5 , a 8 } and c k = A k,3 .This implies that A k+1 contains two agents that are smaller than c k (namely, agents a 1 and a 4 ).Therefore, to ensure that c k < c k+1 , it is sufficient to generate C k+1 such that it does not contain the first (i.e.smallest) two agents in A k+1 .This can be done by ensuring that M k+1 does not contain elements 1 or 2. In other words, it can be done by ensuring that M k+1,1 ≥ M k,1 , which is a direct result of the way FCP is modified.
By proving that | T CS G | = 1 for all CS ∈ P G , we prove that FCP generates every coalition structure in P G exactly once.

Figure 3 :
Figure 3: An example of our representation of the search space given 4 agents.

Figure 4 :
Figure 4: An example of the assumed ordering of the coalition lists.

Figure 7 :
Figure 7: Applying branch-and-bound while searching through the coalition structures in a subspace.

Figure 8 :
Figure 8: Time to find the optimal solution for IDP, IP applied to NDCS, Normal, and Uniform distributions.

Figure 9 :
Figure 9: Top: upper bounds and optimal coalition structure value, bottom: size of sub-spaces.Note that the values in the bottom graph are plotted on a log scale.Points with the same abscissa on the two graphs correspond to the same sub-space.The arrows show the direction of the search for each distribution.

Figure 11 :
Figure 11: Worst case bounds generated by IP using the Normal and NDCS distributions compared to Sandholm et al.'s (1999) and Dang and Jennings's (2004) algorithms for 25 agents.The results for the Uniform distribution are trivial since IP on average finds the optimal almost immediately after scanning the input.Note that error bars have been omitted from the IP results for reasons of clarity.

C
1 , C 2 , . . ., C |G| ∈ T CS G iff |C 1 | ≤ |C 2 | ≤ . . .≤ C |G|(9)Now since there is at least one way of ordering the coalitions in CS such that |C 1 | ≤ ... ≤ C |G| , then there is at least one ordered set in T CS G .In other words, | T CS G | ≥ 1.This, in turn, implies that every coalition structure in P G is generated by MCP.that:if G(g i ) = 1 ∀i ∈ {1, . . ., |G|} then T G = TG and ∀CS ∈ P G , T CS G = T CS G (10) else T G ⊆ TG and ∀CS ∈ P G , T CS G ⊆ T CS G (11)Now, given a coalition structure CS ∈ P G , let C 1 , C 2 , . . ., C |G| be defined as in Appendix D (i.e. it is an ordering on the coalitions that belong to CS).Then, from (9), we find that the number of ordered sets in T CS G is equal to the number of possible ways of ordering the coalitions in CS such that:|C 1 | ≤ |C 2 | ≤ ... ≤ C |G| .Based on this, as well as (10) and (11), we distinguish between two cases:• If G(g i ) = 1 ∀i ∈ {1, ...,|G|}, then there would only be one possible way of ordering the coalitions in CS such that |C 1 | ≤ ... ≤ C |G| (because every coalition in CS has a unique size).This implies that | T CS G | = 1, and from (10), we find that | T CS G | = 1.• If ∃i ∈ {1, ..., |G|} : G(g i ) > 1, then there would be multiple ways of ordering the coalitions in CS such that |C 1 | ≤ ... ≤ C |G| , which implies that | T CS G | > 1.However, from (11), we know that T CS G is a subset of T CS G .Then, by proving that T CS G contains exactly one of the ordered sets in T CS G , we prove that | T CS G | = 1.To be more precise, in case we have: |C x | = |C x+1 | = ... = |C x+y |, then every possible permutation of those coalitions will be generated by MCP, and we need to prove that only one of them will be generated by FCP.
•• ,gn} max s {branch only if there is potential of finding a coalition structure better than CS .}then