BnB-ADOPT: An Asynchronous Branch-and-Bound DCOP Algorithm

Distributed constraint optimization (DCOP) problems are a popular way of formulating and solving agent-coordination problems. A DCOP problem is a problem where several agents coordinate their values such that the sum of the resulting constraint costs is minimal. It is often desirable to solve DCOP problems with memory-bounded and asynchronous algorithms. We introduce Branch-and-Bound ADOPT (BnB-ADOPT), a memory-bounded asynchronous DCOP search algorithm that uses the message-passing and communication framework of ADOPT (Modi, Shen, Tambe, and Yokoo, 2005), a well known memory-bounded asynchronous DCOP search algorithm, but changes the search strategy of ADOPT from best-first search to depth-first branch-and-bound search. Our experimental results show that BnB-ADOPT finds cost-minimal solutions up to one order of magnitude faster than ADOPT for a variety of large DCOP problems and is as fast as NCBB, a memory-bounded synchronous DCOP search algorithm, for most of these DCOP problems. Additionally, it is often desirable to find bounded-error solutions for DCOP problems within a reasonable amount of time since finding cost-minimal solutions is NP-hard. The existing bounded-error approximation mechanism allows users only to specify an absolute error bound on the solution cost but a relative error bound is often more intuitive. Thus, we present two new bounded-error approximation mechanisms that allow for relative error bounds and implement them on top of BnB-ADOPT.


Introduction
A distributed constraint optimization (DCOP) problem consists of agents, each responsible for taking on (= assigning itself) a value from its finite domain of values.The agents coordinate their values, which are subject to constraints.Two agents are constrained if they share a constraint.Each constraint has an associated constraint cost, which depends on the values of the constrained agents.A (complete) solution is an assignment of values to all agents, and a partial solution is an assignment of values to a subset of agents.The solution cost of a (partial or complete) solution is the sum of the constraint costs of all constraints resulting from the given assignment of values to agents.Solving a DCOP problem optimally means finding a solution with minimal solution cost and is NP-hard (Modi et al., 2005).
Formulating agent-coordination problems as constraint optimization (COP) problems, a specific type of weighted constraint satisfaction problems (Schiex, Fargier, & Verfaillie, 1995;Bistarelli,  Montanari, Rossi, Schiex, Verfaillie, & Fargier, 1999), is more general than formulating them as the more common constraint satisfaction problems (Dechter, 2003).Constraint satisfaction problems have constraints that are either satisfied or unsatisfied.Solving a constraint satisfaction problem means finding a solution such that all constraints are satisfied.An example application is the scheduling of jobs in a job-shop, where constraints express that some jobs can only be performed by certain machines and some jobs can only be performed after some other jobs.There could potentially be multiple solutions that satisfy all constraints.However, some solutions might be more desirable than others.For example, one might prefer the solution with the shortest completion time.
It is common to visualize a DCOP problem as a constraint graph where the vertices are the agents and the edges are the constraints.Most DCOP algorithms operate on a pseudo-tree, which is a spanning tree of the (completely connected) constraint graph with the property that edges in the constraint graph connect a vertex with one of its ancestor or descendant vertices in the constraint tree (Freuder & Quinn, 1985;Bayardo & Miranker, 1995).An edge of the constraint graph that is not part of the pseudo-tree is a backedge.An agent c is a pseudo-child agent of agent p if agent c is a descendant agent of agent p in the pseudo-tree and they are constrained via a backedge.Similarly, agent p is the pseudo-parent agent of agent c.Sibling subtrees represent independent DCOP subproblems (since no two agents in different sibling subtrees share a constraint).Figure 1(a) shows the constraint graph of an example DCOP problem with four agents that can each take on value 0 or value 1, Figure 1(b) shows one possible pseudo-tree where the assignments of values to agents a 3 and a 4 are independent DCOP subproblems (the dotted line is a backedge), and Figure 1(c) shows the constraint costs.For our example DCOP problem, a cost-minimal solution results if all agents take on value 1.The minimal solution cost is 12.

DCOP Algorithms
We now provide a taxonomy of DCOP algorithms.Figure 2 shows the taxonomy.DCOP algorithms are divided into two groups: complete and incomplete DCOP algorithms.Complete DCOP algorithms find cost-minimal solutions while incomplete DCOP algorithms are often faster but typically find suboptimal solutions.

Complete DCOP Algorithms
Complete DCOP algorithms are generally divided into two groups, namely partially centralized and fully decentralized DCOP algorithms.

Partially Centralized DCOP Algorithms
Partially centralized DCOP algorithms allow some agents to transfer their constraint information (= information regarding the constraints that they are involved in) to a central agent for processing.OptAPO (Mailler & Lesser, 2004) is an example of a partially centralized DCOP algorithm that uses cooperative mediation, where certain agents act as mediators to solve overlapping DCOP subproblems centrally.

Fully Decentralized DCOP Algorithms
Fully decentralized DCOP algorithms do not have central agents that collect constraint information of other agents that are not constrained with them.Rather, every agent has access to only its own constraint information.Fully decentralized DCOP algorithms are generally divided into two groups, namely DCOP inference and search algorithms.
• DCOP inference algorithms: DCOP inference algorithms typically use dynamic programming to propagate aggregated constraint costs from one agent to another agent and thus reduce the DCOP problem size by one agent at each step.They repeat this procedure until the DCOP problem size is reduced to only one agent and the solution space (= space of all possible partial solutions) thus cannot be reduced anymore.The sole remaining agent has then sufficient knowledge to find a cost-minimal solution.DPOP (Petcu & Faltings, 2005b) is an example of a DCOP inference algorithm.The number of messages sent between agents is only linear in the number of agents.However, its memory requirements are exponential in the induced width of the DCOP problem.The induced width depends on the number of backedges in the pseudo-tree.It can be as large as the number of agents minus one if the constraint graph is fully connected and every agent is thus constrained with every other agent.
• DCOP search algorithms: DCOP search algorithms use search strategies to search through the solution space to find a cost-minimal solution.ADOPT (Modi et al., 2005) uses best-first search, and SBB (Hirayama & Yokoo, 1997), NCBB (Chechetka & Sycara, 2006), AFB (Gershman, Meisels, & Zivan, 2009) and our new DCOP search algorithm, BnB-ADOPT, use depth-first branch-and-bound search.Their memory requirements are only polynomial in the number of agents.However, the number of messages sent between agents can be exponential in the number of agents.
Therefore, both groups of fully decentralized DCOP algorithms are desirable under different conditions as there is a tradeoff between space (memory requirements) and time (number of messages sent).

Motivation
We now describe the motivation behind our work.

BnB-ADOPT
We study DCOP search algorithms because they can be memory-bounded.This property is important for applications, such as sensor networks, where every agent/sensor has only a fixed amount of memory available.As a result, several DCOP search algorithms, such as SBB, ADOPT, NCBB and AFB, were developed with this limitation in mind.As described earlier, their memory requirements are polynomial in the number of agents.Table 1 shows the properties of these DCOP search algorithms as well as the properties of our new DCOP search algorithm, BnB-ADOPT.We now describe each property in more detail and justify the properties of BnB-ADOPT.
• Search strategy: ADOPT uses best-first search to search the solution space, while SBB, NCBB and AFB use depth-first branch-and-bound (DFBnB) search.Best-first search repeatedly searches for the next best partial solution until it finds a cost-minimal solution.The next best partial solution is the cost-minimal partial solution among all partial solutions that have not yet been found.Depth-first branch-and-bound search starts by finding a complete (but often suboptimal) solution and stores its solution cost as the upper bound.It then continues to search for a solution whose solution cost is less than the upper bound.It stores the solution cost of this solution as the upper bound, and the search proceeds until it can no longer find a solution whose solution cost is less than the upper bound.
For centralized search, it is known that search problems with depth-bounded search trees can often be solved faster with depth-first branch-and-bound search than with memory-bounded best-first search because memory-bounded best-first search algorithms, such as RBFS (Korf, 1993), need to repeatedly reconstruct partial solutions that they purged from memory.Depthfirst branch-and-bound search algorithms are memory-bounded but do not suffer from this problem (Zhang & Korf, 1995).Since DCOP problems are search problems with depthbounded search trees, we hypothesize that depth-first branch-and-bound search might be faster than best-first search.Therefore, we decided that BnB-ADOPT should use depth-first branchand-bound search.
• Agent operation: Agents of SBB and NCBB operate sequentially.Only agents with tokens are active while the other agents remain idle.Once the token-holding agents are done, they pass their tokens on and then remain idle.On the other hand, agents of ADOPT and AFB operate concurrently (= at all times).Agents that operate concurrently might be able to solve DCOP problems faster than agents that operate sequentially since the former agents can perform potentially useful computation instead of having to wait for other agents.Therefore, we decided that all agents of BnB-ADOPT should operate concurrently.Agents of SBB and NCBB also operate synchronously.Communication between agents is often in form of messages.Synchronous agents operate in cycles (Modi et al., 2005).A cycle is the time required for an agent to process all incoming messages in its queue and send all outgoing messages, which are then processed by the receiving agents in the next cycle (see Section 6.1 for more details).Therefore, all agents wait until the last agent is done sending its messages before they start a new cycle.On the other hand, asynchronous agents, such as agents of ADOPT and AFB, are able to operate independently of each other, which often increases robustness (Silaghi, Landwehr, & Larrosa, 2004).For example, all synchronous agents are affected if a single communication link suffers from congestion while only a small number of asynchronous agents are affected.We therefore decided that agents of BnB-ADOPT should operate asynchronously.
• Communication: DCOP search algorithms such as SBB, ADOPT and NCBB restrict communication to agents that share constraints.This restriction is motivated by applications such as sensor networks where communication is restricted to neighboring agents/sensors due to their limited communication radius.Neighboring sensors share constraints since they need to coordinate to sense the areas near them.DCOP search algorithms such as AFB do not have this restriction and allow agents to broadcast messages to all other agents.We decided that agents of BnB-ADOPT should obey the restrictions of applications such as sensor networks and thus communicate only with neighboring agents.
• Agent ordering: All DCOP search algorithms mentioned above start with a pre-processing step that arranges the agents into a pseudo-tree.DCOP search algorithms such as SBB and AFB arrange the agents into a chain, while ADOPT and NCBB arrange the agents into a tree.A tree ordering can capture independent DCOP subproblems (represented as sibling subtrees) while a chain ordering can not.DCOP search algorithms that operate on trees can thus operate on independent DCOP subproblems independently, while DCOP search algorithms that operate on chains can not.Therefore, we decided that BnB-ADOPT should arrange agents into a tree.
ADOPT has all preferred properties mentioned above except that it uses best-first search.We therefore introduce BnB-ADOPT, a memory-bounded asynchronous DCOP search algorithm that uses the message passing and communication framework of ADOPT but changes the search strategy of ADOPT from best-first search to depth-first branch-and-bound search.

Bounded-Error Approximations
Solving DCOP problems optimally is NP-hard, which makes it advantageous to allow users to trade off solution cost for a smaller runtime.It is also desirable to have the error of the resulting solution cost be bounded to provide guarantees on the solution cost.ADOPT is, to the best of our knowledge, the only DCOP search algorithm with this property.Its Absolute Error Mechanism allows its users to specify an absolute error bound on the solution cost, for example, that the solution cost should be at most 10 larger than the minimal solution cost.However, it is often much more desirable to specify a relative error bound on the solution cost, for example, that the solution cost should be at most 10 percent larger than the minimal solution cost or, equivalently, 1.1 times larger than the minimal solution cost.This cannot be done with the Absolute Error Mechanism without knowing the minimal solution cost a priori.Thus, we propose two approximation mechanisms that allow users to specify a relative error bound on the solution cost, namely the Relative Error Mechanism and the Weighted Heuristics Mechanism, and implement them on top of BnB-ADOPT.These approximation mechanisms allow BnB-ADOPT to find solutions with bounded errors faster than cost-minimal solutions.

Experimental Results
We experimentally compare ADOPT, BnB-ADOPT and NCBB on three different DCOP problem types, namely graph coloring problems, sensor network problems and meeting scheduling problems.Our results show that BnB-ADOPT is up to one order of magnitude faster (measured in the number of non-concurrent constraint checks and the number of cycles) than ADOPT on a variety of large DCOP problems.BnB-ADOPT can also be inferred to be faster than SBB since ADOPT is faster than SBB (Modi et al., 2005).BnB-ADOPT is also as fast as NCBB on most of these DCOP problems.Our results for the suboptimal variants of BnB-ADOPT show that the Weighted Heuristics Mechanism dominates both the Absolute Error Mechanism and Relative Error Mechanism.

Article Structure
This article is organized as follows: We formalize DCOP problems in Section 2 and describe our DCOP search algorithm, BnB-ADOPT, in Section 3. We describe approximation mechanisms that allow BnB-ADOPT to find solutions with bounded error in Section 4. We outline correctness and completeness proofs of BnB-ADOPT in Section 5. Lastly, we present our experimental evaluations in Section 6 and our conclusions in Section 7.

DCOP Problems
In this section, we formally define distributed constraint optimization (DCOP) problems and describe their solution space.

Definition of DCOP Problems
A DCOP problem is defined by the following elements: • a finite set of agents A = {a 1 , a 2 , ..., a n }; • a set of finite domains D = {Dom(a 1 ), Dom(a 2 ), ..., Dom(a n )}, where Dom(a i ) is the domain of possible floating point values of agent a i ∈ A; and specifies its non-negative constraint cost as a function of the values of the distinct agents a i1 and a i2 that share the constraint.
The above definition assumes that each agent takes on one value rather than multiple values, for example, a different value for each constraint that it is involved in.These DCOP problems are more commonly formulated as each agent being responsible for the assignments of values to multiple variables.However, there exist techniques that reduce such DCOP problems to our DCOP problems (Burke & Brown, 2006).Thus, we use the terms agent and variable interchangeably.The above definition also assumes that constraints are binary (= between two agents) rather than n-ary (= between n agents).One should be able to extend BnB-ADOPT to solve DCOP problems with nary constraints by using the same techniques that were proposed to extend ADOPT to solve DCOP problems with n-ary constraints (Modi et al., 2005).Additionally, we assume that the messages sent between agents can be delayed by a finite amount of time but are never lost.

Search Trees
The solution space of DCOP problems can be visualized with search trees.Traditional search trees or, synonymously, OR search trees (Marinescu & Dechter, 2009) assign values to agents sequentially.They do not utilize the fact that the values of agents that belong to independent DCOP subproblems do not have to be assigned sequentially.AND/OR search trees are based on pseudo-trees and remedy this issue (Marinescu & Dechter, 2009).Thus, we use AND/OR search trees and refer to them as search trees in this article.Their depth is bounded by (twice) the number of agents.
Figure 3(a) shows the search tree that is based on the pseudo-tree in Figure 1(b).Figure 3(b) labels each node of the search tree with an identifier to allow us to refer to the nodes easily.Circular nodes are OR nodes (labeled with upper-case letters) and correspond to agents.For example, the agent of node C is agent a 2 .Left branches of OR nodes correspond to the agents taking on value 0 and right branches correspond to the agents taking on value 1. Square nodes are AND nodes (labeled with lower-case letters) and correspond to the partial solutions from the root node to those nodes.For example, the partial solution of node f is {(a 1 , 1), (a 2 , 1)}.The subtree rooted at an AND node represents the DCOP subproblem that assumes the partial solution of the AND node.For example, the subtree rooted at node f represents the DCOP subproblem of assigning values to agents a 3 and a 4 given that {(a 1 , 1), (a 2 , 1)}.The number of independent DCOP subproblems within this DCOP subproblem is indicated by the number of branches exiting the AND node.For example, there are two branches exiting node f , indicating that there are two independent DCOP subproblems, namely of assigning values to agents a 3 and a 4 .The numbers in the AND nodes are the delta costs of the nodes.The delta cost of an AND node is defined to be the sum of the constraint costs of all constraints in its partial solution that involve the agent of its parent OR node.
For example, the partial solution of node v is {(a 1 , 1), (a 2 , 1), (a 4 , 1)}.There are two constraints in this partial solution, namely the constraint between agents a 1 and a 2 , which has constraint cost 3, and the constraint between agents a 2 and a 4 , which also has constraint cost 3.Since the parent node of node v is node K with agent a 4 , the delta cost of node v is 3, namely the constraint cost of the latter constraint.The former constraint is not included since it does not involve agent a 4 .The solution cost of a partial solution of an AND node is the sum of the delta costs of all AND nodes along the branch from the root node to that node.For example, the solution cost of the partial solution of node v (= 6) is the sum of the delta costs of nodes b, f and v.In our example DCOP problem, a cost-minimal solution is the union of the partial solutions of nodes t and v (all agents take on value 1).Thus, the minimal solution cost (= 12) is the sum of the delta costs of nodes b, f , t and v.

BnB-ADOPT
In this section, we present Branch-and-Bound ADOPT (BnB-ADOPT).We do not describe BnB-ADOPT as a modification of ADOPT since this approach requires the readers to have an in-depth understanding of ADOPT.Instead, we give a stand-alone description of BnB-ADOPT that requires no knowledge of ADOPT, with the intention of creating a self-contained and hopefully easy-to-read description.

Search Strategies of ADOPT and BnB-ADOPT
We first describe centralized versions of the search strategies of ADOPT and BnB-ADOPT and omit technical details since these are described in more detail in later sections.

Search Strategy of ADOPT
ADOPT (Modi et al., 2005) is a popular DCOP search algorithm (Modi & Ali, 2004;Ali, Koenig, & Tambe, 2005;Bowring, Tambe, & Yokoo, 2006;Davin & Modi, 2006;Pecora, Modi, & Scerri, 2006;Choxi & Modi, 2007;Silaghi & Yokoo, 2009;Matsui, Silaghi, Hirayama, Yokoo, & Matsuo, 2009) that traverses the search tree in a best-first search order.We now describe a simplified version of best-first search.The complete version can be found in (Marinescu & Dechter, 2007).Bestfirst search maintains a list that initially contains only the child AND nodes of the root node.It repeatedly performs the following operations: It expands the AND node with the smallest solution cost in the list by removing that node from the list and adding the grandchild AND nodes of that node into the list.For our example DCOP problem, best-first search expands the AND nodes in the search tree in Figure 3 for the first time in the following order, where the numbers in parentheses indicate the solution costs of the partial solutions of the expanded nodes: a (0), b (0), f (3), c (5), v (6), i (8), d (8) and t (9).
Figure 4 shows a simplified trace of ADOPT on our example DCOP problem.ADOPT terminates after fifteen steps with minimal solution cost 12.The numbers in the AND nodes are the delta costs of the nodes.The lower bound LB r X r is an optimistic estimate of the minimal solution cost.It is the smallest underestimated solution cost, over all solutions.The underestimated solution cost of a solution is the sum of the delta costs of all AND nodes of that solution whose parent OR node is the root node or whose grandparent AND node is expanded.For example, the underestimated solution cost of the solution {(a 1 , 1), (a 2 , 1), (a 3 , 1), (a 4 , 1)} is 3 if node b is expanded and nodes f , t and v are not expanded.The upper bound U B r X r is a pessimistic estimate of the minimal solution cost.It is the solution cost of the solution with the smallest solution cost found so far.ADOPT terminates when the upper bound U B r X r is no larger than the lower bound LB r X r .In order to be memory-bounded, ADOPT maintains only one branch of the search tree (shaded grey in the figure) from the root node to the currently expanded node and thus needs to repeatedly reconstruct nodes Step 14 Step 12 that it purged from memory.For example, in Step 3, ADOPT has the branch to node f in memory.
The next node that best-first search expands is node c, and ADOPT discards the branch to node f in Step 4. In Steps 6 and 7, it then needs to reconstruct the discarded branch to node f in order to expand node v in Step 8.

Search Strategy of BnB-ADOPT
We now describe a simplified version of depth-first branch-and-bound search.The complete version can be found in (Marinescu & Dechter, 2009).We use the same definitions of LB r X r and U B r X r as described earlier for Figure 4. Depth-first branch-and-bound search maintains a stack that initially contains only the child AND nodes of the root node.It expands the AND node on top of the stack by removing that node from the stack and performing the following check.If the solution cost of that node is no smaller than the upper bound U B r X r , it prunes that node and repeats the operation.Otherwise, it adds the grandchild AND nodes of that node to the top of the stack and repeats the operation.It terminates when the upper bound U B r X r is no larger than lower bound LB r X r .Depth-first branch-and-bound search can add the grandchild AND nodes of an expanded AND node (and the child AND nodes of the root node) in decreasing order of their solution costs instead of a random order to the top of the stack.This ordering ensures that depth-first branchand-bound search expands the grandchild AND node with the smallest solution cost first.We use this improvement throughout the article.For our example DCOP problem, depth-first branch-andbound search expands the AND nodes in the search tree in the following order, where it prunes the nodes in brackets: a (0), c (5), i (8), j (13), g (15), [h (19)], d (8), n (11), k (16), [m (18)], [l (21)], b (0), f (3), v (6) and t (9). Figure 5 shows a trace of depth-first branch-and-bound search for our example DCOP problem.It is memory-bounded without having to repeatedly reconstruct nodes that it purged from memory but expands some nodes that a best-first search does not expand, such as node j in Step 4. The depth-first branch-and-bound search terminates after twelve steps with minimal solution cost 12, which is three steps fewer than ADOPT.

Description of BnB-ADOPT
We now provide an incremental description of BnB-ADOPT.First, we provide the notations and key terms of BnB-ADOPT.Then, we describe how BnB-ADOPT updates its bounds, adheres to memory limitations, performs depth-first search and performs branch-and-bound.Finally, we introduce our enhanced final version of BnB-ADOPT and show both its pseudocode and its trace for our example DCOP problem.

Notation and Key Terms
We adopt the following notation from ADOPT to describe BnB-ADOPT: • CD(a) ⊆ A is the set of child and pseudo-child agents of agent a ∈ A; • C(a) ⊆ CD(a) is the set of child agents of agent a ∈ A; • pa(a) ∈ A is the parent agent of agent a ∈ A except for the root agent; • P (a) ⊆ A is the set of ancestor agents (including the parent agent) of agent a ∈ A; • SCP (a) ⊆ P (a) is the set of ancestor agents (including the parent agent) of agent a ∈ A that are parent or pseudo-parent agents of agent a or one (or more) of its descendant agents; and • CP (a) ⊆ SCP (a) is the set of ancestor agents (including the parent agent) of agent a ∈ A that are parent or pseudo-parent agents of agent a.
We adopt the following key terms from ADOPT to describe BnB-ADOPT: The context X a of agent a is the set of values of all ancestor agents of agent a.The context X r of the root agent r is always equal to {}.
• Delta cost (δ): The delta cost δ a X a (d) is the sum of the constraint costs of all constraints that involve both agent a and one of its ancestor agents, under the assumption that agent a takes on value d and its ancestor agents take on the values in context X a .In the search tree, δ a X a (d) is the delta cost of the AND node that has partial solution X a ∪ (a, d).For example, δ a2 {(a1,1)} (1) is the delta cost of node f in Figure 3.
• Gamma cost (γ): The gamma costs γ a X a (d) and γ a X a are defined as follows: for all agents a, all values d and all contexts X a .Thus, the gamma cost γ a X a (d) is the sum of the constraint costs of all constraints that involve agent a or one of its descendant agents (that is, either both agent a and one of its ancestor agents, both agent a and one of its descendant agents, both a descendant agent and an ancestor agent of agent a or two descendant agents of agent a) minimized over all possible values of its descendant agents, under the assumption that agent a takes on value d and its ancestor agents take on the values in context X a .In the search tree, γ a X a (d) is the gamma cost of the AND node that has partial solution X a ∪ (a, d).For example, γ a2 {(a1,1)} (1) is the gamma cost of node f in Figure 3.The gamma cost γ a X a is the sum of the constraint costs of all constraints that involve agent a or one of its descendant agents minimized over all possible values of agent a and its descendant agents, under the assumption that the ancestor agents of agent a take on the values in context X a .In the search tree, the gamma cost γ a X a is the gamma cost of the OR node whose agent is agent a and whose parent AND node has partial solution X a .For example, γ a2 {(a1,1)} is the gamma cost of node C in Figure 3. Therefore, the gamma cost of an AND node is the sum of its delta cost and the gamma costs of its child OR nodes, and the gamma cost of an OR node is the minimum of the gamma costs of its child AND nodes.For example, the gamma cost of node f in Figure 3 is the sum of its delta cost and the gamma costs of nodes J and K, and the gamma cost of node C in Figure 3 is the minimum of the gamma costs of nodes e and f .Solving a DCOP problem optimally means to determine γ r X r for the root agent r or, equivalently, the gamma cost of the root node since γ r X r is the minimal solution cost.It is not difficult for the agents to cache information that allows them to determine a cost-minimal solution.

Updating the Bounds
Every agent a of BnB-ADOPT stores and updates several bounds on the gamma costs, namely and U B a X a for all values d, all child agents c and all contexts X a , maintaining the following bound property: In the search tree, • LB a X a and U B a X a are lower and upper bounds, respectively, (on the gamma cost) of the OR node whose agent is agent a and whose parent AND node has partial solution X a ; • LB a X a (d) and U B a X a (d) are lower and upper bounds, respectively, (on the gamma cost) of the AND node that has partial solution X a ∪ (a, d); and • lb a,c X a (d) and ub a,c X a (d) are lower and upper bounds, respectively, (on the gamma cost) of the OR node whose agent is agent c and whose parent AND node has partial solution X a ∪ (a, d).
Each agent a uses the following update equations for all values d, all child agents c and all contexts X a to initialize its bounds lb a,c X a (d) and ub a,c X a (d), where the heuristic values h a,c X a (d) are floating point numbers that are admissible and thus satisfy 0 Agent a then uses repeatedly the following update equations for all values d, all child agents c, all contexts X a and all contexts X c (= X a ∪ (a, d)) to tighten the bounds: The updates maintain the bound property and improve the bounds monotonically, that is, the lower bounds are monotonically non-decreasing and the upper bounds are monotonically nonincreasing.1After a finite amount of time, U B a X a ≤ LB a X a for all agents a and all contexts X a .BnB-ADOPT terminates when its termination condition U B r X r ≤ LB r X r for the root agent r is satisfied.Then, U B r X r ≤ LB r X r and the bound property , and the DCOP problem is solved optimally.Figure 6 shows a simplified trace of the updates of the (lower and upper) bounds for our example DCOP problem.We assume that the updates proceed sequentially from the leaf agents to the root agent.Due to this simplification, the lower and upper bounds of each node are identical to its gamma cost and independent of the heuristic values.The numbers in the nodes are their bounds.Two agents maintain the bounds of OR nodes except for the root node.The figure shows the bounds that the parent agent maintains rather than the bounds that the child agent maintains.For example, the number in node B is the bounds that agent a 1 rather than agent a 2 maintains.The bounds that the child agent maintains can be computed by taking the minimum of the bounds of the child AND nodes of the OR node.Agents update the bound of an AND node to the sum of its delta cost and the bounds of its child OR nodes according to update equations 9 and 12.They update the bound of an OR node to the minimum of the bounds of its child AND nodes according to update equations 10 and 13.A more detailed description of the trace is as follows: • Step 1: Leaf agent a 3 updates the bounds of AND nodes g, h, k, l, o, p, s and t to their delta costs according to update equations 9 and 12 and the bounds of OR nodes D, F , H and J to the minimum of the bounds of their child AND nodes according to update equations 10 and 13.Similarly, leaf agent a 4 updates the bounds of AND nodes i, j, m, n, q, r, u and v to their delta costs according to update equations 9 and 12 and the bounds of OR nodes E, G, I and K to the minimum of the bounds of their child AND nodes according to update equations 10 and 13.The bounds of OR nodes D to K are not shown in the figure since they are not (yet) maintained by agent a 2 .
• Step 2: Agent a 2 updates the bounds of OR nodes D to K that it maintains to the bounds of the same OR nodes that leaf agents a 3 and a 4 maintain according to update equations 8 and 11, the bounds of AND nodes c to f to the sum of their delta costs and the bounds of their child OR nodes according to update equations 9 and 12 and the bounds of OR nodes B and C to the minimum of the bounds of their child AND nodes according to update equations 10 and 13.The bounds of OR nodes B and C are not shown in the figure since they are not (yet) maintained by agent a 1 .
• Step 3: Agent a 1 updates the bounds of OR nodes B and C that it maintains to the bounds of the same OR nodes that agent a 2 maintains according to update equations 8 and 11, the bounds of AND nodes a and b to the sum of their delta costs and the bounds of their child OR nodes according to update equations 9 and 12 and the bounds of OR node A to the minimum of the bounds of its child AND nodes according to update equations 10 and 13.Since the lower and upper bounds of a node are equal to its gamma cost, the lower and upper bounds of the root node are equal to its gamma cost, which in turn is equal to the minimal solution cost.The propagation terminates after three steps with minimal solution cost 12.

Adhering to Memory Limitations
Our description of BnB-ADOPT so far assumes no memory limitations.However, BnB-ADOPT is a memory-bounded DCOP search algorithm with memory requirements per agent that are linear in the number of agents.We now describe how BnB-ADOPT adheres to these memory limitations using techniques that were introduced for ADOPT but apply to BnB-ADOPT as well.
The simplified trace in Figure 6 assumes that every agent a maintains its bounds for all values d, all child agents c and all contexts X a .The number of contexts can be exponential in the depth of the agent in the pseudo-tree.For our example DCOP problem, agent a 3 has four different contexts for the four different combinations of values of its ancestor agents a 1 and a 2 .An agent cannot maintain an exponential number of bounds due to the memory limitations.Therefore, every agent maintains its bounds for only one context at any given time.This context is stored in the variable X a for agent a.The size of the context is at most linear in the number of agents.The number of bounds of an agent is now linear in the product of its domain cardinality and the number of its child agents.Thus, the memory requirements per agent are only linear in the number of agents if the domain cardinality and the magnitude of the bounds (and the other variables) are constant for each agent.

Performing Depth-First Search
Our description of BnB-ADOPT so far applies to ADOPT as well.However, BnB-ADOPT uses depth-first branch-and-bound search and ADOPT uses best-first search.We now describe how BnB-ADOPT implements depth-first search.
Agents of BnB-ADOPT send messages that are similar to that of ADOPT but processes them differently.They send messages of three different types, namely VALUE, COST and TERMINATE messages.At the start, every agent a initializes its context X a , uses update equations 6, 9, 10, 7, 12 and 13 to initialize its bounds and takes on its best value d a := arg min d∈Dom(a) {LB a X a (d)}.It sends VALUE messages to all child agents and a COST message to its parent agent.It then repeatedly waits for incoming messages, processes them, possibly takes on a different value and again sends VALUE messages to all child agents and a COST message to its parent agent.A description of the three message types and how agents process them is as follows: • VALUE messages: An agent a with context X a and value d a sends VALUE messages to all child agents with the desired context X a ∪ (a, d a ), which is its context augmented with its value.Leaf agents do not have child agents and thus do not send VALUE messages.VALUE messages thus propagate contexts down the pseudo-tree.
When an agent receives a VALUE message, it checks whether its context is identical to the desired context in the VALUE message.If it is not, then the agent changes its context to the desired context in the VALUE message.In either case, it then executes the common program (see below).
• COST messages: An agent a sends COST messages to its parent agent with its identity a, its context X a and its bounds LB a X a and U B a X a .The root agent does not have a parent agent and thus does not send COST messages.COST messages thus propagate bounds up the pseudo-tree.
When an agent receives a COST message, it checks whether its context and the context in the COST message are compatible.Two contexts are compatible if no agent takes on different values in the two contexts.If they are, then the agent uses update equations 8 to 13 with the bounds in the COST message to improve its bounds for the value in the message.In either case, it then executes the common program (see below).
• TERMINATE messages: When the termination condition U B r X r ≤ LB r X r is satisfied, the root agent r sends TERMINATE messages (without parameters) to all child agents to inform them that the search is complete and then terminates.When an agent receives such a TERMINATE message, it sends TERMINATE messages to all child agents and terminates as well.Leaf agents do not have child agents and thus do not send TERMINATE messages.TERMINATE messages thus propagate down the pseudo-tree until all agents terminate.The common program is as follows: • Context change: If an agent a changed its context X a , it executes the following statements: It uses update equations 6, 9, 10, 7, 12 and 13 to initialize its bounds and takes on its best value d a := arg min d∈Dom(a) {LB a X a (d)}.It then sends VALUE messages to all child agents and a COST message to its parent agent.
for its value d a , then the context of the agent augmented with its value cannot be completed to a solution whose solution cost is smaller than the solution cost of the best solution found so far for its context X a (= U B a X a ) and the agent thus takes on its best value d a := arg min d∈Dom(a) {LB a X a (d)}.It then sends VALUE messages to all child agents and a COST message to its parent agent.
Assume that the context X a of an agent a does not change.After a finite amount of time, U B a X a ≤ LB a X a (d a ) for its value d a .The agent then takes on its best value and repeats the procedure.After a finite amount of time, U B a X a ≤ LB a X a (d) for all values d, which implies that U B a X a ≤ LB a X a .The agent takes on every value d at most once until U B a X a ≤ LB a X a since LB a X a (d) remains unchanged and U B a X a is monotonically non-increasing once the agent changes its value from d to a different value, which prevents the agent from changing its value back to d before U B a X a ≤ LB a X a .BnB-ADOPT thus performs depth-first search.Then, after a finite amount of time, U B r X r ≤ LB r X r and the bound property for the root agent r, and the DCOP problem is solved optimally.Figures 7 and 8 show traces of the updates of the lower and upper bounds, respectively, for our example DCOP problem.BnB-ADOPT uses the zero heuristic values.The initial context of every Cycle 7 agent assigns value 0 to all ancestor agents of the agent.We partition time into cycles.Agents maintain their bounds for only one context at any given time.Nodes in the figures are crossed out if their agent does not maintain their bounds.AND nodes are shaded if their partial solution is equal to the context of the agent of their parent OR node augmented with its value.For example, agents a 1 , a 3 and a 4 take on value 0 in Cycle 2, and agent a 2 takes on value 1.The context of agent a 1 is {}, the context of agent a 2 is {(a 1 , 0)} and the contexts of agents a 3 and a 4 are {(a 1 , 0), (a 2 , 0)}.A description of the trace is as follows: • Cycle 1: Root agent a 1 initializes its context X a1 to {}.It initializes the lower bounds of nodes B (= lb a1,a2 X a 1 (0)) and C (= lb a1,a2 X a 1 (1)) to 0 since it uses the zero heuristic values.It updates the lower bound of node a (= LB a1 X a 1 (0)) to the sum of its delta cost (= 0) and the lower bound of node B (= 0) according to the update equations.It updates the lower bound of node b (= LB a1 X a 1 (1)) to the sum of its delta cost (= 0) and the lower bound of node C (= 0) according to the update equations.It updates the lower bound of node A (= LB a1 X a 1 ) to the minimum of the lower bound of node a (= 0) and the lower bound of node b (= 0) according to the update equations.It initializes the upper bounds of nodes B and C to infinity.It updates the upper bounds of nodes a, b and A to infinity according to the update equations.It takes on its best value.It can take on either value 0 or value 1 since the lower bounds of nodes a and b are both 0. It takes on value 0 and sends a VALUE message to its child agent a 2 .
Agent a 2 initializes its context X a2 to {(a 1 , 0)}.It initializes the lower bounds of nodes D, E, F and G to 0. It updates the lower bounds of nodes c, d and B to 5, 8 and 5, respectively.It initializes the upper bounds of nodes D, E, F and G to infinity.It updates the upper bounds of nodes c, d and B to infinity.The bounds of node B that agent a 2 maintains are not shown in the figures.It takes on its best value 0, sends VALUE messages to its child agents a 3 and a 4 and sends a COST message to its parent agent a 1 .
Leaf agent a 3 initializes its context X a3 to {(a 1 , 0), (a 2 , 0)}.It updates the lower bounds of nodes g and h to their delta costs 10 and 14, respectively, since leaf agents do not have child agents.It updates the lower bound of node D to 10.It updates the upper bounds of nodes g and h to their delta costs 10 and 14, respectively, since leaf agents do not have child agents.It updates the upper bound of node D to 10.The bounds of node D that leaf agent a 3 maintains are not shown in the figures.It takes on its best value 0 and sends a COST message to its parent agent a 2 .
Leaf agent a 4 initializes its context X a4 to {(a 1 , 0), (a 2 , 0)}.It updates the lower bounds of nodes i and j to their delta costs 3 and 8, respectively.It updates the lower bound of node E to 3. It updates the upper bounds of nodes i and j to their delta costs 3 and 8, respectively.It updates the upper bound of node E to 3. The bounds of node E that leaf agent a 4 maintains are not shown in the figures.It takes on its best value 0 and sends a COST message to its parent agent a 2 .
• Cycle 2: Root agent a 1 receives the COST message sent by its child agent a 2 in Cycle 1.Since the context of agent a 1 (= {}) is compatible with the context in the message (= {(a 1 , 0)}), it improves its bounds.It updates the bounds of node B to the bounds in the message (= 5 and infinity, respectively).It updates the bounds of nodes a, b and A. It does not change its value since the lower bound of node a (= LB a1 X a 1 (d a1 ) = 5 for its value d a1 = 0) is still smaller than the upper bound of node Agent a 2 receives the VALUE message sent by its parent agent a 1 in Cycle 1.Its context (= {(a 1 , 0)}) remains unchanged since it is the same as the desired context in the message (= {(a 1 , 0)}).Agent a 2 also receives the COST messages sent by its child agents a 3 and a 4 in Cycle 1.Since the context of agent a 2 (= {(a 1 , 0)}) is compatible with the contexts in the messages (= {(a 1 , 0), (a 2 , 0)}), it improves its bounds.It updates the bounds of node D to the bounds in the first message (= 10 and 10, respectively) and the bounds of node E to the bounds in the second message (= 3 and 3, respectively).It updates the bounds of nodes c, d and B. It changes its value since the lower bound of node c (= LB a2 X a 2 (d a2 ) = 18 for its value d a2 = 0) is no longer smaller than the upper bound of node B (= U B a2 X a 2 = 18).It takes on its best value 1, sends VALUE messages to its child agents a 3 and a 4 and sends a COST message to its parent agent a 1 .
Leaf agents a 3 and a 4 receive the VALUE messages sent by their parent agent a 2 in Cycle 1. Their contexts (= {(a 1 , 0), (a 2 , 0)}) remain unchanged since they are the same as the desired context in the message (= {(a 1 , 0), (a 2 , 0)}).They send the same COST messages as before to their parent agent a 2 .
The VALUE messages are different because agent a 2 changed its value from 0 to 1.The COST message is different because agent a 2 changed its bounds.
The COST messages are different because agents a 3 and a 4 changed their contexts.The termination condition holds after a finite amount of time when the upper bound of node A (= U B a1 X a 1 = 12) is no larger than the lower bound of node A (= LB a1 X a 1 = 12).Root agent a 1 sends TERMINATE messages to all child agents, and the TERMINATE messages propagate down the pseudo-tree until all agents terminate.BnB-ADOPT terminates after nine cycles with minimal solution cost 12.

Performing Branch-and-Bound
We now refine our description of BnB-ADOPT by explaining how the agents implement branchand-bound search to make BnB-ADOPT faster.Every agent a of BnB-ADOPT now also maintains the variable threshold T H a X a , which it initializes to infinity.The threshold of the root agent always remains infinity.Every other agent uses its threshold for pruning, meaning that it can change its value earlier than previously.
• First change: If an agent a did not change its context X a , it previously executed the following statements: If U B a X a ≤ LB a X a (d a ) for its value d a , then the agent took on its best value.It then sent VALUE messages to all child agents and a COST message to its parent agent.Now, if T H a X a ≤ LB a X a (d a ), then the agent also takes on its best value.Thus, if min{T , then the agent takes on its best value and thus potentially changes its value, which is earlier than previously.min{T H a X a , UB a X a } is the pruning quantity.• Second change: An agent a with context X a and value d a sends VALUE messages to all its child agents, which previously contained only the desired context X a ∪ (a, d a ).VALUE messages now also contain the desired threshold min{T X a (d a ) for the child agent c.When agent c receives a VALUE message, it sets its threshold to the desired threshold and then proceeds as described earlier.The desired threshold is set such that the lower bound LB a X a (d a ) of agent a for its value d a reaches its pruning quantity (and agent a thus potentially changes its value) when the lower bound LB c X c of agent c reaches the desired threshold.This property can be verified as follows:

Further Enhancements
We continue to refine our description of BnB-ADOPT by explaining a number of additional enhancements, which were introduced for ADOPT.
• Reduced contexts: The agents now use reduced contexts, which are subsets of the contexts described previously.The reduced context X a 1 of agent a contains the values of all ancestor agents p ∈ SCP (a), while the context X a 2 described previously contains the values of all ancestor agents p ∈ P (a).The agents can use reduced contexts since γ a for all values d.Agents now use reduced contexts because they need to change their contexts and thus initialize their bounds less often when they receive VALUE messages since their contexts are then more often identical to the desired contexts in the VALUE messages.For our example DCOP problem, the reduced context of agent a 4 contains the values of only agent a 2 rather than the values of agents a 1 and a 2 .Therefore, the following pairs of nodes in the search tree are actually the same node: nodes i and q, nodes j and r, nodes m and u, and nodes n and v.
• VALUE and COST messages: An agent sends VALUE messages to all child agents, which previously contained the desired context and the desired threshold.The desired context is the context of the agent augmented with its value.When an agent receives a VALUE message, it previously checked whether its context is identical to the desired context in the VALUE message.If it was not, then the agent changed its context to the desired context in the VALUE message.Agents now update their contexts differently to reduce the size of the VALUE messages.An agent sends VALUE messages to all child and pseudo-child agents with its identity, value and desired threshold, which is infinity for its pseudo-child agents.When an agent receives a VALUE message, it sets its threshold to the desired threshold if the message is from its parent agent.It also checks whether the value of the ancestor agent in the VALUE message is more recent than the value of the ancestor agent in its context.If it is, then the agent changes the value of the ancestor agent in its context to the value of the ancestor agent in the VALUE message.However, the context of an agent does not only contain the values of its parent and pseudo-parent agents but also the values of its ancestor agents that are the parent or pseudo-parent agents of one (or more) of its descendant agents, and ancestor agents that are not constrained with the agent cannot send VALUE messages to the agent.However, they send VALUE messages to their pseudo-child agents, at least one of which is a descendant agent of the agent, and the information then propagates up the pseudo-tree with COST messages until it reaches the agent.When an agent receives a COST message, it now checks whether the value of an ancestor agent in the context of the COST message is more recent than the value of the ancestor agent in its context.If it is, then the agent changes the value of the ancestor agent in its context to the value of the ancestor agent in the context of the COST message.Our example DCOP problem is too simple to allow us to illustrate the propagation of the information up the pseudo-tree.However, imagine that a new agent a 5 is a child agent of agent a 4 and is constrained with agents a 1 and a 4 .The context of agent a 4 then contains the value of agent a 1 but agent a 1 cannot send VALUE messages to agent a 4 .However, agent a 1 sends VALUE messages to agent a 5 .Agent a 5 changes the value of agent a 1 in its context and sends COST messages with its context to agent a 4 , which then changes the value of agent a 1 in its context as well.
The agents now need to determine whether the value of an agent in VALUE messages or in the contexts of COST messages is more recent than the value of the agent in their contexts.Every agent a therefore now also maintains a counter ID a and increments it whenever it changes its value.Therefore, a larger ID indicates a more recent value.The values of agents in contexts are now labeled with their IDs, and VALUE messages contain the identity of the sending agent, its value, its ID and the desired threshold.
• Bounds: Whenever an agent changes its context X a , it previously initialized its bounds and took on its best value.The (reduced) context of a child agent of an agent can now be a strict subset of the (reduced) context of the agent since the parent or some pseudo-parent agents of the agent might not be (parent or) pseudo-parent agents of the child agent or its descendant agents.If the context of child agent c does not contain the values of any agents whose values changed in the context of agent a, then agent a does not initialize its lower bounds lb a,c X a (d) and upper bounds ub a,c X a (d) for agent c and all values d before it takes on its best value.Agents use this optimization because they need to initialize their bounds less often this way.For our example DCOP problem, if agent a 2 changes its context from {(a 1 , 0)} to {(a 1 , 1)} (where the IDs are omitted for simplicity), then it does not initialize its lower bounds lb a2,a4 X a 2 (d) and upper bounds ub a2,a4 X a 2 (d) for child agent a 4 and all values d since the context of agent a 4 does not contain the value of agent a 1 .
Additionally, if an agent a changes its context due to a COST message from its child agent c and its new context X a is compatible with the context in the COST message, then agent a can set its lower bound lb a,c X a (d) and upper bound ub a,c X a (d) for agent c and the value d of agent a in the COST message to the bounds in the COST message before it takes on its best value.Agents use this optimization because the bounds in the COST message are more informed than the initialized bounds.Our example DCOP problem is too simple to allow us to illustrate this optimization.However, imagine again that a new agent a 5 is a child agent of agent a 4 and is constrained with agents a 1 and a 4 .Assume that the context of agent a 4 is {(a 1 , 0), (a 2 , 0)} (where the IDs are again omitted for simplicity) and it receives a COST message from agent a 5 with context {(a 1 , 1), (a 4 , 0)}.Agent a 4 then changes its context to {(a 1 , 1), (a 2 , 0)}, sets its lower bound lb a4,a5 {(a1,1),(a2,0)} (0) and its upper bound ub a4,a5 {(a1,1),(a2,0)} (0) to the bounds in the COST message and initializes all other bounds before it takes on its best value.

Pseudocode
Figure 9 shows the BnB-ADOPT pseudocode of every agent.The pseudocode does not index variables with the context since this context is implicitly given by the variable X a .It uses the predicate Compatible(X, X ) = ¬∃ (a,d,ID)∈X,(a ,d ,ID )∈X (a = a ∧ d = d ) that determines if two contexts X and X are compatible, that is, if no agent takes on two different values in the two contexts [Lines 35,44,46,48 and 51].The pseudocode also uses the procedure PriorityMerge(X, X ) that executes of agents in context X with more recent values, if available, of the same agents in context X [Lines 34 and 43].
The code is identical for every agent except that the variable a is a "self" variable that points to the agent itself.At the start, BnB-ADOPT calls Start() for every agent.When an agent a receives a VALUE message from an ancestor agent, then the "When Received" handler for VALUE messages is called with p being the ancestor agent, d p being the value of the ancestor agent, ID p being the ID of the ancestor agent and T H p being the desired threshold for agent a if the ancestor agent is its parent agent (and infinity otherwise) [Line 11].When agent a receives a COST message from a child agent, then the "When Received" handler for COST messages is called with c being the child agent, X c being the context of the child agent, LB c being the lower bound LB c X c of the child agent and U B c being the upper bound U B c X c of the child agent [Line 11].Finally, when agent a receives a TERMINATE message from its parent agent, then the "When Received" handler for TERMINATE messages is called without any arguments [Line 11].
BnB-ADOPT uses the same message passing and communication framework as ADOPT and has the same memory requirements.It uses similar VALUE, COST and TERMINATE messages, a similar strategy to update the context of an agent based on VALUE messages from its ancestor agents and COST messages from its child agents, the same semantics for the bounds and the same update equations to update these bounds.BnB-ADOPT and ADOPT both use thresholds but BnB-ADOPT uses the thresholds for pruning while ADOPT uses them to reconstruct partial solutions that were purged from memory.Thus, BnB-ADOPT uses a different threshold initialization [Line 17], different desired threshold calculation [Line 30] and different termination condition [Line 27].BnB-ADOPT also differs from ADOPT in that it maintains IDs that agents use to indicate the recency of their values and labels the values of agents in contexts with their IDs.

Trace
Figures 10 and 11 show traces of the updates of the lower and upper bounds, respectively, for our example DCOP problem, and Table 2 shows a trace of the update of all variables.BnB-ADOPT uses the heuristic values h a1,a2 X a 1 (0) := 3, h a1,a2 X a 1 (1) := 6, h a2,a3 X a 2 (0) := 2, h a2,a3 X a 2 (1) := 2, h a2,a4 X a 2 (0) := 2 and h a2,a4 X a 2 (1) := 2 for all contexts X a1 and X a2 .These heuristic values were chosen by hand.Every agent assigns the value of all its ancestor agents in its initial context to 0. We partition time into cycles as in Figures 7 and 8 and continue to use the conventions made in the context of those figures.
• It does not change its value since the lower bound of node a (= LB a1 X a 1 (d a1 ) = 9 for its value d a1 = 0) is still smaller than its pruning quantity (= min{T H a1 X a 1 , UB a1 X a 1 } = min(∞, ∞) = ∞).It sends VALUE messages to its child agent a 2 and pseudo-child agent a 3 [Lines 30-31].
Agent a 2 receives the VALUE message sent by its parent agent a 1 in Cycle 1.It updates its context from {(a 1 , 0, 0)} to {(a 1 , 0, 1)} since the ID of agent a 1 in its context (= 0) is smaller than the ID in the message (= 1) [Line 34].Its threshold (= ∞) remains unchanged since it is the same as the desired threshold (= ∞) in the message.Agent a 2 also receives the COST messages sent by its child agents a 3 and a 4 in Cycle 1.Since its context (= {(a 1 , 0, 1)}) is compatible with the contexts in the messages (= {(a 1 , 0, 0), (a 2 , 0, 0)} and {(a 2 , 0, 0)}, respectively), it improves its bounds.It updates the bounds of node D to the bounds in the first message (= 10 and 10, respectively) and the bounds of node E to the bounds in the second message (= 3 and 3, respectively)  X a 2 (d a2 ) = 18 for its value d a2 = 0) is no longer smaller than its pruning quantity (= min{T H a2 X a 2 , UB a2 X a 2 } = min(∞, 18) = 18).It takes on its best value 1 [Line 24], increments its ID to 2 [Lines 25-26], sends VALUE messages to its child agents a 3 and a 4 [Lines 30-31] and sends a COST message to its parent agent a 1 [Line 32].
Leaf agent a 3 receives the VALUE messages sent by its parent agent a 2 and pseudo-parent agent a 1 in Cycle 1.It updates its context from {(a 1 , 0, 0), (a 2 , 0, 0)} to {(a 1 , 0, 1), (a 2 , 0, 1)} since the IDs of agents a 1 and a 2 in its context (= 0 and 0, respectively) are smaller than the IDs in the messages (= 1 and 1, respectively) In summary, the messages sent during Cycle 2 are identical to the ones sent during Cycle 1, except for the messages sent by agents a 2 , a 3 and a 4 , which are as follows: message (VALUE, a 2 , 1, 2, 8) from agent a 2 to agent a 3 ; -message (VALUE, a 2 , 1, 2, 8) from agent a 2 to agent a 4 ; and message (COST, a 2 , {(a 1 , 0, 1)}, 12, 18) from agent a 2 to agent a 1 .
The VALUE messages are different because agent a 2 changed its value from 0 to 1.The COST messages are different because agent a 2 changed its bounds and its context and agents a 3 and a 4 changed their contexts.
The COST messages are different because agents a 3 and a 4 changed their contexts.The termination conditions holds after a finite amount of time when the upper bound of node A (= U B a1 X a 1 = 12) is no larger than the lower bound of node A (= LB a1 X a 1 = 12) [Line 27].Root agent a 1 sends TERMINATE messages to all child agents [Line 28], and the TERMINATE messages propagate down the pseudo-tree [Line 28] until all agents terminate.BnB-ADOPT terminates after nine cycles with minimal solution cost 12.

Bounded-Error Approximations
In this section, we present three approximation mechanisms that allow BnB-ADOPT to trade off solution cost for a smaller runtime.They bound the error on the solution cost by a user-defined error bound.First, we modify the Absolute Error Mechanism of ADOPT (Modi et al., 2005) to work with BnB-ADOPT.This approximation mechanism allows users to specify an absolute error bound on the solution cost (for example, that the solution cost should be at most 10 larger than the minimal solution cost).However, it is often much more desirable to specify a relative error bound on the solution cost (for example, that the solution cost should be at most 10 percent larger than the minimal solution cost or, equivalently, 1.1 times larger than the minimal solution cost).This cannot be done with the Absolute Error Mechanism without knowing the minimal solution cost a priori.Thus, we introduce two approximation mechanisms that allow users to specify a relative error bound on the solution cost, namely the Relative Error Mechanism and the Weighted Heuristics Mechanism.
All approximation mechanisms let the root agent r (and only the root agent) maintain the limit lim r .The root agent uses this limit in the same way in the termination condition for all approximation mechanisms but updates it differently.The termination condition U B r X r ≤ LB r X r on Line 27 of the pseudocode of BnB-ADOPT is replaced with U B r X r ≤ lim r .The root agent updates the limit between Lines 26 and 27 in the pseudocode, outside of the preceding if statement.

Absolute Error Mechanism
The Absolute Error Mechanism of ADOPT requires a user-defined absolute error bound 0 ≤ b < ∞ that specifies that the solution cost should be at most b larger than the minimal solution cost.This approximation mechanism can easily be modified for BnB-ADOPT by setting the limit as follows: BnB-ADOPT AEM is the resulting variant of BnB-ADOPT with the Absolute Error Mechanism.BnB-ADOPT AEM terminates once the upper bound of the root node (which is equal to the solution cost of the solution with the smallest solution cost found so far) is no larger than the limit (which is equal to the absolute error bound b plus the lower bound of the root node, which is a lower bound on the minimal solution cost).BnB-ADOPT AEM terminates with a solution cost that is equal to the upper bound of the root node although the minimal solution cost could be as small as the lower bound of the root node.It thus terminates with a solution cost that is at most b larger than the minimal solution cost. Figure 12 shows a trace of BnB-ADOPT AEM with absolute error bound b = 24 for our example DCOP problem.BnB-ADOPT AEM terminates after three cycles with suboptimal solution cost 18, which is six cycles faster than BnB-ADOPT.

Relative Error Mechanism
It is often much more desirable to specify a relative error bound on the solution cost rather than an absolute error bound.Fortunately, the Absolute Error Mechanism of BnB-ADOPT can easily be changed to the Relative Error Mechanism by setting the limit as follows.The Relative Error     Mechanism requires a user-defined relative error bound 1 ≤ p < ∞ that specifies that the solution cost should be at most p times larger than the minimal solution cost: BnB-ADOPT REM is the resulting variant of BnB-ADOPT with the Relative Error Mechanism.BnB-ADOPT REM terminates once the upper bound of the root node (which is equal to the solution cost of the solution with the smallest solution cost found so far) is no larger than the limit (which is equal to the relative error bound p times the lower bound of the root node, which is a lower bound on the minimal solution cost).BnB-ADOPT REM terminates with a solution cost that is equal to the upper bound of the root node although the minimal solution cost could be as small as the lower bound of the root node.It thus terminates with a solution cost that is at most p times larger than the minimal solution cost. Figure 13 shows a trace of BnB-ADOPT REM with relative error bound p = 3 for our example DCOP problem.BnB-ADOPT REM terminates after three cycles with suboptimal solution cost 18, which is six cycles faster than BnB-ADOPT.

Weighted Heuristics Mechanism
There is a second way of implementing a relative error bound for BnB-ADOPT since BnB-ADOPT uses admissible heuristic values.It is common practice in the context of A* to trade off solution cost for a smaller runtime by using weighted heuristic values (Pohl, 1973), which are derived from admissible heuristic values by multiplying them with a user-defined weight 1 ≤ w < ∞.The resulting heuristic values can be inadmissible.A* is then no longer guaranteed to find cost-minimal solutions but it is guaranteed to terminate with a solution cost that is at most w times larger than the minimal solution cost (Pohl, 1970).This approximation mechanism can easily be modified for BnB-ADOPT by setting the limit as follows:  and by initializing the lower bounds lb a,c X a (d) as follows: for all agents a, all values d, all child agents c and all contexts X a .BnB-ADOPT W HM is the resulting variant of BnB-ADOPT with the Weighted Heuristics Mechanism.BnB-ADOPT W HM terminates once the upper bound of the root node (which is equal to the solution cost of the solution with the smallest solution cost found so far) is no larger than the limit (which is equal to the lower bound of the root node, which is a lower bound on w times the minimal solution cost).BnB-ADOPT W HM terminates with a solution cost that is equal to the upper bound of the root node although the minimal solution cost could be as small as the lower bound of the root node divided by w.It thus terminates with a solution cost that is at most w times larger than the minimal solution cost. Figure 14 shows a trace of BnB-ADOPT W HM with w = 3 for our example DCOP problem.BnB-ADOPT W HM terminates after three cycles with suboptimal solution cost 18, which is six cycles faster than BnB-ADOPT.

Correctness and Completeness
In this section, we prove the correctness and completeness of BnB-ADOPT and its suboptimal variants.All definitions, lemmata, theorems and corollaries hold for BnB-ADOPT and its suboptimal variants except when mentioned otherwise.Therefore, each agent a uses the following update equation for all values d, all child agents c and all contexts X a to initialize its bounds lb a,c X a (d): where the weight w is a floating point number that satisfies 1 ≤ w < ∞ and the heuristic values h a,c X a (d) are floating point numbers that satisfy Messages are sent at the end of a cycle and received in the beginning of a cycle.∆ is the largest duration between the time a message is sent and the time it is processed, and is the largest duration of a cycle.Lemma 1.If two contexts X and X of an arbitrary agent a ∈ A agree on the values of all ancestor agents p ∈ SCP (a) of agent a, then γ a X = γ a X .Proof.By definition, X a ⊆ X is the (reduced) context that contains the values of all ancestor agents p ∈ SCP (a) of agent a.The gamma cost γ a X is the sum of the constraint costs of all constraints that involve agent a or one of its descendant agents minimized over all possible values of agent a and its descendant agents, under the assumption that the ancestor agents of agent a take on the values in context X.The gamma cost γ a X thus depends only on the values of the ancestor agents (including the parent agent) of agent a that are the parent or pseudo-parent agents of agent a or one (or more) of its descendant agents, that is, the values of all ancestor agents p ∈ SCP (a) of agent a.Therefore, Definition 1. Contexts are correct iff the IDs of the values of all agents in the contexts are equal to the IDs of the agents, which implies that the values of all agents in the contexts are equal to the values of the agents.
Lemma 2. If the context X a of an arbitrary agent a ∈ A does not change for a period of time, then the lower bounds lb a,c X a (d), LB a X a (d) and LB a X a are monotonically non-decreasing and the upper bounds ub a,c X a (d), U B a X a (d) and U B a X a are monotonically non-increasing during that period of time for all values d ∈ Dom(a) and all child agents c ∈ C(a).
Proof.Since the context X a does not change, the delta values δ a X a (d) are constant and the bounds (once initialized) are updated according to update equations 8 to 13.Thus, the lower bounds are monotonically non-decreasing and the upper bounds are monotonically non-increasing.
Lemma 3. If the value of an arbitrary ancestor agent p ∈ SCP (a) of an arbitrary agent a ∈ A does not change between the current time T and a future time t with t ≥ T + |A| • (∆ + ) + , then the value of agent p and its ID in the context of agent a are equal to the value of agent p and its ID, respectively, between some time t and time t with t ≤ t.
Proof.Assume that the value of an arbitrary ancestor agent p ∈ SCP (a) of an arbitrary agent a ∈ A does not change between the current time T and a future time t with t ≥ T + |A| • (∆ + ) + .There are the following two cases.
• Case 1: If agent p is a parent or pseudo-parent agent of agent a, then it sent a VALUE message to agent a with its value and ID at time t ≤ T + , that is, in the same cycle in which it took on the value that it has at time T since the duration of that cycle is no larger than .(The agents send VALUE messages at the end of every cycle.)Agent a receives the VALUE message by time t + ∆ since messages are delivered with finite delay ∆.It then updates the value of agent p and its ID in its context by time t + ∆ + since the update is done in the same cycle and the duration of that cycle is no larger than .Thus, the value of agent p and its ID in the context of agent a are equal to the value of agent p and its ID, respectively, between some time t and time t with t ≤ t ≤ t + ∆ + ≤ T + ∆ + 2 • ≤ t since agent p does not change its value between time t and time t.
• Case 2: If agent p is not a parent or pseudo-parent agent of agent a, then one of its pseudo-child agents c is a descendant agent of agent a.  Proof.We prove the lemma by induction on the number of times that agent a changes its context or updates its bounds lb a,c X a (d) and ub a,c X a (d) for an arbitrary value d and an arbitrary child agent c after agent a initializes its bounds.The conclusion of the lemma holds after agent a with context X a initializes its bounds since for the (unchanged or new) context X a of agent a (induction basis).Now assume that the lemma holds after agent a changed its context or updated its bounds a number of times (induction assumption).We show that it then also holds after agent a changes its context or updates its bounds one more time (induction step).There are the following two cases (where we split the operations after receiving a COST message into two parts).
• Case 1: The conclusion of the lemma holds when agent a changes its context from X a to Xa after receiving a VALUE or COST message and the two contexts agree on the values of all ancestor agents p ∈ SCP (c) since agent a then does not change its bounds and thus after receiving the VALUE or COST message (since contexts X a and Xa agree on the values of all ancestor agents p ∈ SCP (c)).
• Case 2: The conclusion of the lemma holds when agent a updates its bounds from lb a,c X a (d) and ub a,c X a (d) to lb a,c X a (d) and ûb a,c X a (d), respectively, after receiving a COST message from some child agent c with bounds LB c X c and U B c X c and context X c that is compatible with its context X a and in which agent a has value d since lb a,c for all values d at all times.Thus, {LB a X a (d)} (Eq.10) = min X a at all times (induction basis).Now assume that the lemma holds for all agents of depth d in the pseudo-tree (induction assumption).We show that it then also holds for all agents of depth d − 1 in the pseudo-tree each time after they update their bounds (induction step).The lemma holds for agent a with context X a since (a,d) (induction assumption and Lemma 4) (induction assumption and Lemma 4) {LB a X a (d)} (Eq.10)

Definition 2. The potential of an agent
X a (d)}.Lemma 6.If the context X a of an arbitrary agent a ∈ A no longer changes, then the potential of the agent is monotonically non-increasing and decreases by more than a positive constant every time the agent changes its value.
Proof.The lower bounds LB a X a (d) are monotonically non-decreasing and the upper bounds U B a X a (d) are monotonically non-increasing for all values d according to Lemma 2 since the context X a of agent a no longer changes.Therefore, the potential of agent a is monotonically non-increasing.Furthermore, agent a changes its value d to a new value only if min d∈Dom(a) {LB a X a (d)} < LB a X a (d) [Line 24].Thus, the lower bound LB a X a (d) must have strictly increased between the time when the agent changed its value to d and the time when it changes its value d to a new value.Thus, its potential has decreased by more than a positive constant, namely the smallest possible increase of the lower bound LB a X a (d).Assume that all constraint costs, weights and heuristic values are integers.Then, the smallest possible increase is bounded from below by one because the only possible values of LB a X a (d) are combinations of all constraint costs and weighted heuristic values.A similar statement holds if all constraint costs, weights and heuristic values are floating point numbers since they can then all be transformed into integers by multiplying them with the same sufficiently large integer.

Lemma 7. All agents change their values only a finite number of times.
Proof.Assume that the lemma does not hold and choose an agent a that changes its value an infinite number of times but whose ancestor agents p ∈ SCP (a) change their values only a finite number of times.Then, there exists a time when the ancestor agents do not change their values any longer.There exists a (later) time when agent a no longer changes its context X a according to Corollary 1.Every time agent a changes its value afterwards, its potential decreases by more than a positive constant according to Lemma 6, towards minus infinity.However, its potential cannot become negative since LB a X a (d) ≤ w • U B a X a (d) for all values d according to Lemma 5, which is a contradiction.Thus, all agents change their values only a finite number of times.
Lemma 8.If BnB-ADOPT and its suboptimal variants do not terminate earlier, then U B a X a ≤ LB a X a after a finite amount of time for all agents a ∈ A and their contexts X a .Proof.We prove the lemma by induction on the depth of an agent in the pseudo-tree.There exists a time when no agent changes its value any longer according to Lemma 7.There exists a (later) time when the contexts of all agents are correct and no longer change according to Corollary 1.Let X a be the context of agent a at this point in time for all agents a.There exists an (even later) time when the bounds lb X a for all agents a and all values d according to Lemma 5, (3) lb a,c X a (d) ≤ w • ub a,c X a (d) for all agents a, all values d and all child agents c according to Lemma 4 and (4) the smallest possible increases of the lower bounds and the smallest possible decreases of the upper bounds are larger than a positive constant since the only possible values of the bounds are combinations of all constraint costs and heuristic values, as explained in more detail in the proof of Lemma 6.Consider the first COST message that each agent sends after this time and the earliest time when all of these COST messages have been processed by their receiving agents.The lemma holds for a leaf agent a in the pseudo-tree with context X a since for all values d after the considered time.Furthermore, {LB a X a (d)} (Eq.10) = min after the considered time.Thus, U B a X a = LB a X a after the considered time (induction basis).Now assume that the lemma holds for all agents of depth d in the pseudo-tree after the considered time (induction assumption).We show that it then also holds for all agents of depth d − 1 in the pseudotree after the considered time (induction step).For agent a with context for its value d after the considered time since all bounds no longer change.Thus, U B a X a (d) ≤ LB a X a (d) for its value d after the considered time.Since agent a does not change its value d after the considered time, it must hold that LB a for its value d, which is a contradiction.The second disjunct implies that (Eq. 10) for its value d and thus that U B a X a ≤ LB a X a .
Theorem 1. BnB-ADOPT and its suboptimal variants terminate after a finite amount of time.
Proof.If BnB-ADOPT and its suboptimal variants do not terminate earlier, then U B a X a ≤ LB a X a after a finite amount of time for all agents a ∈ A and their contexts X a according to Lemma 8.In particular, U B r X r ≤ LB r X r ≤ lim r for the root agent r, where lim r = LB r X r for BnB-ADOPT and BnB-ADOPT W HM , lim r = b + LB r X r with b ≥ 0 for BnB-ADOPT AEM and lim r = p • LB r X r with p ≥ 1 for BnB-ADOPT REM according to Section 4. Thus, both the termination condition U B r X r ≤ LB r X r of BnB-ADOPT and the termination condition U B r X r ≤ lim r of its suboptimal variants are satisfied.• Cycles: Cycles are time slices.A cycle is the time required for an agent to process all incoming messages in its queue and send all outgoing messages, which are then processed by the receiving agents in the next cycle.Thus, the number of cycles indicates the length of the longest chain of messages between agents.Cycles are a good runtime metric if the communication time is much larger than the processing time.Cycles will become a better and better runtime metric in the future since the communication time is expected to remain relatively stable while the processing time is expected to decrease (Silaghi, Lass, Sultanik, Regli, Matsui, & Yokoo, 2008).

DCOP Problem Types
We use three DCOP problem types in our experiments, namely graph coloring problems, sensor network problems and meeting scheduling problems.
• Graph coloring: Graph coloring problems involve coloring the vertices of a graph, taking restrictions between the colors of adjacent vertices into account.The agents are the vertices, their domains are the colors, and the constraints are between adjacent vertices.We vary the number of vertices from 5 to 15, the constraint density (= the ratio between the number of constraints and the number of agents) from 2 (sparse graphs) to 3 (dense graphs) and the range of constraint costs from a range of 0 to 1 (small range) to a range of 0 to 10,000 (large range).Each agent always has three possible values.We average the experimental results over 50 DCOP problem instances with randomly generated constraints and randomly generated integer constraint costs.
• Sensor network: Sensor network problems involve assigning targets to sensors in a sensor network, taking restrictions in the availability of the sensors, restrictions in the number of sensors that need to track each target and the priorities of the targets into account.The agents are the targets, their domains are the time slots when they can be tracked, and the constraints are between adjacent targets (Maheswaran et al., 2004b).Figure 15 shows a sensor network where the targets are located on a grid and each target is surrounded by four sensors, all of which are needed to track the target.We vary the number of targets from 4 to 15.We always use 8 time slots.The cost of assigning a time slot to a target that is also assigned to an adjacent target is infinity (to be precise: 1,000,000) since the same sensor cannot track both targets during the same time slot.The cost of targets that are not tracked during any time slot is 100.All other costs are in the range of 0 to 100.We average the experimental results over 50 DCOP problem instances with randomly generated integer constraint costs.
• Meeting scheduling: Meeting scheduling problems involve scheduling meetings between the employees of a company, taking restrictions in their availability as well as their priorities into account.The agents are the meetings, their domains are the time slots when they can be held, and the constraints are between meetings that share participants (  16 shows a hierarchical organization with 4 units of a supervisor and its three subordinates.For example, supervisor 2 has three subordinates 5, 6 and 7.In each unit, we assume five possible meetings: one of the entire unit (e.g., 2, 5, 6, 7), two parent-child meetings (e.g., 2, 5 and 2, 7) and two sibling-sibling meetings (e.g., 5, 6 and 6, 7).We vary the number of meetings from 5 (1 unit) to 20 (4 units).We always use 8 time slots.The cost of assigning a time slot to a meeting that has at least one participant who has another meeting during the same time slot is infinity (to be precise: 1,000,000) since the same person cannot attend more than one meeting at a time.The cost of a non-scheduled meeting is 100.All other costs are in the range of 0 to 100.We average the experimental results over 50 DCOP problem instances with randomly generated integer constraint costs.

Experimental Results: Optimal DCOP Search Algorithms
We first compare BnB-ADOPT to ADOPT and NCBB. Figure 17 shows our experimental results for graph coloring problems with constraint costs ranging from 0 to 10,000, where we varied the number of vertices, while Figure 18 shows our experimental results for graph coloring problems with 10 vertices, where we varied the range of constraint costs.Figures 17(a-c) and 18(a-c) show the results for coloring sparse graphs, and Figures 17(d-f) and 18(d-f) show the results for coloring dense graphs.The y-axes are in log scale and show the runtimes in NCCCs or cycles.DCOP search algorithms on sparse graphs are faster than on dense graphs because, for example, there is a larger likelihood of independent DCOP subproblems in sparse graphs.BnB-ADOPT is generally faster than NCBB on sparse graphs but not on dense graphs because BnB-ADOPT allows agents to send messages only to their parent agents in the pseudo-tree (along edges of the pseudo-tree) but NCBB  allows agents also to send messages to their pseudo-parent agents (along backedges of the pseudotree).Thus, agents of NCBB receive updates faster than agents of BnB-ADOPT.This effect is more prevalent in dense graphs since there are more backedges in dense graphs.However, the difference between BnB-ADOPT and NCBB becomes negligible when communication is slow.
Figure 17 shows that BnB-ADOPT is at least half an order of magnitude faster than ADOPT when the number of vertices is small.The speedup over ADOPT increases as the number of vertices gets larger and the DCOP problems thus become more complex.Similarly, Figure 18 shows that the speedup over ADOPT increases as the range of constant costs increases and the DCOP problems thus become more complex.However, ADOPT can be faster than BnB-ADOPT for simple DCOP problems.For example, ADOPT requires fewer cycles than BnB-ADOPT for DCOP problems with constraint costs ranging from 0 to 1. Figure 19 shows the same trend for sensor network and meeting scheduling problems.The reason for this behavior is as follows.ADOPT uses memory-bounded best-(= different) and repeated contexts per agent for different weights c.When the heuristic values are well informed (large weights), agents of ADOPT explore fewer unique contexts than agents of BnB-ADOPT since they are more focused in their search.However, when the heuristic values are poorly informed (small weights), they explore more unique contexts.Agents of ADOPT explore many more repeated contexts than agents of BnB-ADOPT since they need to reconstruct partial solutions that they purged from memory.Agents of BnB-ADOPT explore a few repeated contexts even though it does not have to reconstruct partial solutions.The reason for this behavior is the distributed nature of BnB-ADOPT.For example, assume that the context of an agent is {(a 1 , 0), (a 2 , 0)} and the next context of a centralized variant of BnB-ADOPT would be {(a 1 , 1), (a 2 , 1)} (where the IDs are omitted for simplicity).The agent updates its context to {(a 1 , 1), (a 2 , 0)} when it receives the message from agent a 1 that it takes on value 1.The agent then updates its context to {(a 1 , 1), (a 2 , 1)} when it receives the message from agent a 2 that it takes on value 1.Thus, the agent explores the intermediate context {(a 1 , 1), (a 2 , 0)} that a centralized variant of BnB-ADOPT would not explore.It counts as a repeated context if the agent explores this context intentionally in the future.Overall, BnB-ADOPT tends to be faster than ADOPT if the heuristic values are poorly informed (small weights).Thus, BnB-ADOPT has great potential as a DCOP search algorithm since heuristic values are often poorly informed for complex DCOP problems, such as DCOP problems with large numbers of agents, large domains, large numbers of constraints or large ranges of constraint costs.

Experimental Results: Suboptimal Variants of BnB-ADOPT
We now compare the three suboptimal variants of BnB-ADOPT to each other.The experimental setup is identical to the one for the optimal DCOP search algorithms, except as follows: For graph coloring problems, the number of vertices is 10, the range of constraint costs is 0 to 10,000 and the constraint density is 2; for sensor network problems, the number of targets is 9; and for meeting scheduling problems, the number of meetings is 10.We measure the runtimes in cycles.(The results for NCCCs are similar.)However, we report normalized runtimes, that is, the runtimes divided by the runtime for finding a cost-minimal solution with BnB-ADOPT.Thus, the normalized runtime 0.25 refers to one quarter of the number of cycles that it takes to find a cost-minimal solution with BnB-ADOPT.Similarly, we report normalized solution costs, that is, the solution costs divided by the minimal solution costs.Thus, the normalized solution cost 2.5 refers to a solution cost that is two and a half times larger than the minimal solution cost.We vary the relative error bound (which is the worst acceptable normalized solution cost) from 1.0 to 4.0.The relative error bound is p for BnB-ADOPT REM and w for BnB-ADOPT W HM .We pre-calculate the minimal solution costs to set the correct value of b for BnB-ADOPT AEM .For example, if the minimal solution cost is 100 and the relative error bound is 2.5, then p = 2.5 for BnB-ADOPT REM , w = 2.5 for BnB-ADOPT W HM and b = (2.5 − 1) • 100 = 150 for BnB-ADOPT AEM .
Figure 21(a-c) shows our experimental results for graph coloring problems.Figure 21(a) shows that the normalized solution costs of all three suboptimal variants increase as the relative error bound increases.However, the solution costs remain much smaller than the error bound.For example, the normalized solution costs of all three suboptimal variants are less than 1.3 (rather than 3) when the relative error bound is 3.The normalized solution costs of BnB-ADOPT AEM are usually larger than the normalized solution costs of BnB-ADOPT REM for the same relative error bound.The reason for this behavior is that BnB-ADOPT AEM terminates when U B r X r ≤ lim r = b + LB r X r = (p − 1) • γ r X r + LB r X r , where γ r X r is the minimal solution cost.Thus, the solution cost of BnB-ADOPT AEM can be at most U B r X r − LB r X r ≤ (p − 1) • γ r X r larger than the minimal solution cost.On the other hand, BnB-ADOPT REM terminates when U B r X r ≤ lim r = p • LB r X r .Thus, the solution cost of BnB-ADOPT REM can be at most U B r X r − LB r X r ≤ (p − 1) • LB r X r larger than the minimal solution cost.The absolute error bound of BnB-ADOPT AEM is thus no smaller than the absolute error bound of BnB-ADOPT REM since γ r X r ≥ LB r X r but is initially strictly greater than the absolute error bound of BnB-ADOPT REM since γ r X r > LB r X r during most of the search.shows that the normalized runtimes of all three suboptimal variants decrease as the relative error bound increases.They decrease to almost 0 when the relative error bound is about 2.0.Therefore, all three suboptimal variants terminate almost immediately after finding the first solution.The normalized runtimes of BnB-ADOPT AEM are usually smaller than the normalized runtimes of BnB-ADOPT REM for the same relative error bound since BnB-ADOPT AEM can terminate with a suboptimal solution cost that is within its absolute error bound but not yet within the absolute error bound of BnB-ADOPT REM if the absolute error bound of BnB-ADOPT AEM is strictly greater than the absolute error bound of BnB-ADOPT REM .In other words, BnB-ADOPT AEM can terminate with a suboptimal solution cost (p − 1) • LB r X r < UB r X r ≤ (p − 1) • γ r X r while BnB-ADOPT REM can not.
Figure 21(c) shows the normalized runtimes needed to achieve a given normalized solution cost.BnB-ADOPT W HM terminates faster than BnB-ADOPT AEM , which in turn terminates faster than BnB-ADOPT REM .For example, the normalized runtime needed to achieve the normalized solu-tion cost 1.05 is about 0.18 for BnB-ADOPT W HM , 0.30 for BnB-ADOPT AEM and 0.35 for BnB-ADOPT REM .Thus, BnB-ADOPT W HM is the suboptimal variant of BnB-ADOPT with the best performance.Figures 21(d-e) show the same trend for sensor network and meeting scheduling problems.

Conclusions
In this article, we introduced Branch-and-Bound ADOPT (BnB-ADOPT), a memory-bounded DCOP search algorithm.BnB-ADOPT uses the message passing and communication framework of ADOPT but changes the search strategy of ADOPT from best-first search to depth-first branchand-bound search to make ADOPT faster by taking advantage of the fact that DCOP problems have depth-bounded search trees.The other properties of BnB-ADOPT are similar to those of ADOPT.BnB-ADOPT allows agents to operate concurrently (in order to decrease the runtime) and asynchronously (in order to increase robustness).BnB-ADOPT restricts communication to agents that share constraints (in order to fit the restrictions of applications such as sensor networks).Finally, BnB-ADOPT orders agents into a pseudo-tree (in order to take advantage of independent DCOP subproblems).Our experimental results showed that BnB-ADOPT finds cost-minimal solutions up to one order of magnitude faster than ADOPT for a variety of large DCOP problems and is as fast as NCBB for most of these DCOP problems.The reason for this behavior is the following: Agents of NCBB operate sequentially and are thus often idle.ADOPT can construct fewer partial solutions than BnB-ADOPT but has to reconstruct some partial solutions that it purged from memory.The advantage of ADOPT with respect to the number of constructed partial solutions decreases and its disadvantage with respect to the number of reconstructed partial solutions increases as heuristic values become more poorly informed.Thus, BnB-ADOPT has great potential as a DCOP search algorithm since heuristic values are often poorly informed for complex DCOP problems such as DCOP problems with large numbers of agents, large domains, large numbers of constraints or large ranges of constraint costs.
We also investigated three approximation mechanisms that trade off the solution cost of BnB-ADOPT for a smaller runtime, namely the Absolute Error Mechanism from ADOPT (resulting in BnB-ADOPT AEM ), the new Relative Error Mechanism (resulting in BnB-ADOPT REM ) and the new Weighted Heuristics Mechanism (resulting in BnB-ADOPT W HM ).The two new approximation mechanisms allow users to specify a relative error bound, which is often more meaningful than an absolute error bound.The Weighted Heuristics Mechanism dominated both the Absolute Error Mechanism and the Relative Error Mechanism in our experiments and should apply to other DCOP search algorithms as well since they all benefit from using heuristic values to focus their searches (Yeoh, Koenig, & Sun, 2008b).
In the future, we plan to improve BnB-ADOPT in the following ways: First, we would like to reduce the number of sent messages and handle lost messages.Second, we would like to study how different pseudo-tree arrangements (Atlas & Decker, 2007;Sultanik, Lass, & Regli, 2009) and pre-processing techniques (Matsui et al., 2009) affect the efficiency of BnB-ADOPT.Finally, we would like to compare BnB-ADOPT and its approximation mechanisms to other DCOP algorithms, including OptAPO, DPOP and their variants (Petcu & Faltings, 2005a, 2006).
Figure 1: Example DCOP Problem

Figure 3 :
Figure 3: AND/OR Search Tree

Figure 4 :
Figure 4: Trace of Simplified Memory-Bounded Best-First Search (Centralized ADOPT)

Figure 6 :
Figure 6: Simplified Trace of the Updates of the (Lower and Upper) Bounds

Figure 7 :
Figure 7: Trace of the Updates of the Lower Bounds

Figure 8 :
Figure 8: Trace of the Updates of the Upper Bounds [Line 13].It updates the lower bound of node a (= LB a1 X a 1 (0)) to the sum of its delta cost (= 0) and the lower bound of node B (= 3) according to the update equations [Line 19].It updates the lower bound of node b (= LB a1 X a 1 (1)) to the sum of its delta cost (= 0) and the lower bound of node C (= 6) according to the update equations [Line 19].It updates the lower bound of node A (= LB a1 X a 1 ) to the minimum of the lower bound of node a (= 3) and the lower bound of node b (= 6) according to the update equations [Line 21].It initializes the upper bounds of nodes B and C to infinity [Line 14].It updates the upper bounds of nodes a, b and A to infinity according to the update equations [Lines 20 and 22].It takes on its best value 0 since the lower bound of node a (= 3) is smaller than the lower bound of node b (= 6) [Line 15], initializes its ID ID a1 to 1 [Lines 2 and 16], initializes its threshold T H a1 to infinity [Line 17] and sends VALUE messages to its child agent a 2 and pseudo-child agent a 3 [Lines 30 and 31].Agent a 2 initializes its context X a2 to {(a 1 , 0, 0)} [Line 1].It initializes the lower bounds of nodes D, E, F and G to their heuristic value 2 [Line 13].It updates the lower bounds of nodes c, d and B to 9, 12 and 9, respectively [Lines 19 and 21].It initializes the upper bounds of nodes D, E, F and G to infinity [Line 14].It updates the upper bounds of nodes c, d and B to infinity [Lines 20 and 22].The bounds of node B that agent a 2 maintains are not shown

Figure 10 :
Figure 10: Trace of the Update of the Lower Bounds of BnB-ADOPT

Figure 11 :
Figure 11: Trace of the Update of the Upper Bounds of BnB-ADOPT

Cycle 2 :
Root agent a 1 receives the COST message sent by its child agent a 2 in Cycle 1.Since the context of agent a 1 (= {}) is compatible with the context in the message (= {(a 1 , 0, 0)}), it improves its bounds.It updates the bounds of node B to the bounds in the message (= 9 and infinity, respectively) [Lines 48-50].It updates the bounds of nodes a, b and A [Lines 18-22].
[Lines 48-50].It updates the bounds of nodes c, d and B [Lines 18-22].It changes its value since the lower bound of node c (= LB a2 [Line 34].Its threshold (= ∞) remains unchanged since it is the same as the desired threshold (= ∞) in the message.Its bounds are not reinitialized since its context is compatible with its previous context [Line 35].It sends the same COST message as before to its parent agent a 2 [Line 32].Leaf agent a 4 receives the VALUE message sent by its parent agent a 2 in Cycle 1.It updates its contexts from {(a 2 , 0, 0)} to {(a 2 , 0, 1)} since the ID of agent a 2 in its context (= 0) is smaller than the ID in the message (= 1) [Line 34].Its threshold (= ∞) remains unchanged since it is the same as the desired threshold (= ∞) in the message.Its bounds are not reinitialized since its context is compatible with its previous context [Line 35].It sends the same COST message as before to its parent agent a 2 [Line 32].

Figure 12 :
Figure 12: Trace of the Update of the Lower Bounds of BnB-ADOPT AEM with b = 24

Figure 13 :
Figure 13: Trace of the Update of the Lower Bounds of BnB-ADOPT REM with p = 3

Figure 14 :
Figure 14: Trace of the Update of the Lower Bounds of BnB-ADOPT W HM with w = 3 at all times for all child agents c ∈ C(a) of an arbitrary agent a and their contextsX c , then lb a,c X a (d) ≤ w • γ c X a ∪(a,d) ≤ w • ub a,cX a (d) at all times for the context X a of agent a, all values d ∈ Dom(a) and all child agents c ∈ C(a).
lb a,c Xa (d) = lb a,c X a (d) ( p r e m i s e o f c a s e ) ≤ w • γ c X a ∪(a,d) (induction assumption) = w • γ c Xa ∪(a,d) (Lemma 1) ub a,c Xa (d) = ub a,c X a (d) ( pr e m i s e o f c a s e )

Figure 18 :
Figure 18: Experimental Results Comparing ADOPT, BnB-ADOPT and NCBB for Graph Coloring Problems with 10 Vertices

Figure 19 :
Figure 19: Experimental Results Comparing ADOPT, BnB-ADOPT and NCBB for Sensor Network and Meeting Scheduling Problems

Figure 21 :
Figure 21: Experimental Results Comparing the Suboptimal Variants of BnB-ADOPT

Table 1 :
Properties of DCOP Search Algorithms

Table 2 :
Trace of the Update of all Variables of BnB-ADOPT Agent p sent a VALUE message to agent c with its value and ID at time t ≤ T + .Agent c receives the VALUE message by time t + ∆.It then updates the value of agent p and its ID in its context and sends a COST message to its parent agent pa(c) with its updated context by time t + ∆ + .(The agents send COST messages at the end of every cycle.)Agent pa(c) receives the COST message by time t + 2 • ∆ + .It then updates the value of agent p and its ID in its context and sends a COST message to its parent agent pa(pa(c)) with its updated context by time t + 2 • (∆ + ).This process continues until agent a updates the value of agent p and its ID in its context by time t + n • (∆ + ), where n ≤ |A| is the number of messages in the chain of messages.Thus, the value of agent p and its ID in the context of agent a are equal to the value of agent p and its ID, respectively, between some time t and time t with t ≤ t ≤ t + n • (∆ + ) ≤ T + |A| • (∆ + ) + ≤ t since agent p does not change its value between time t and time t.If the values of all ancestor agents p ∈ SCP (a) of an arbitrary agent a ∈ A do not change between the current time T and a future time t with t ≥ T + |A| • (∆ + ) + , then the context of agent a is correct between some time t and time t with t ≤ t.