DYNAMIC S ERVICE P LACEMENT IN M OBILE M ICRO -C LOUDS S HIQIANG WANG A Thesis Submitted in Fulfilment of Requirements for the Degree of Doctor of Philosophy of Imperial College London and Diploma of Imperial College Communications and Signal Processing Group Department of Electrical and Electronic Engineering Imperial College London 2015 Copyright Notice The copyright of this thesis rests with the author and is made available under a Creative Commons Attribution Non-Commercial No Derivatives licence. Researchers are free to copy, distribute or transmit the thesis on the condition that they attribute it, that they do not use it for commercial purposes and that they do not alter, transform or build upon it. For any reuse or redistribution, researchers must make clear to others the licence terms of this work. Declaration of Originality I certify that the intellectual content of this thesis is the product of my own work and that all the assistance received in preparing this thesis and sources have been acknowledged. Shiqiang Wang Abstract Cloud computing is an important enabling technique for running complicated applications on resource-limited handheld devices, personal computers, or small enterprise servers, by offloading part of the computation and storage to the cloud. However, traditional centralized cloud architectures are incapable of coping with many emerging applications that are delay-sensitive and require large amount of data exchange between the front-end and back-end components of the application. To tackle these issues, the concept of mobile micro-cloud (MMC) has recently emerged. An MMC is typically connected directly to a network component, such as a wireless basestation, at the edge of the network and provides services to a small group of users. In this way, the communication distances between users and the cloud(s) hosting their services are reduced, and thus users can have more instantaneous access to cloud services. Several new challenges arise in the MMC context, which are mainly caused by the limited coverage area of basestations and the dynamic nature of mobile users, network background traffic, etc. Among these challenges, one important problem is where (on which cloud) to place the services (or, equivalently, to execute the service applications) to cope with the user demands and network dynamics. We focus on this problem in this thesis, and consider both the initial placement and subsequent migration of services, where migration may occur when the user location or network conditions change. The problem is investigated from a theoretical angle with practical considerations. We first abstract the service application and the physical cloud system as graphs, and propose online approximation algorithms for finding the placement of iv an incoming stream of application graphs onto the physical graph. Then, we consider the dynamic service migration problem, which we model as a Markov decision process (MDP). The state space of the MDP is large, making it difficult to solve in real-time. Therefore, we propose simplified solution approaches as well as approximation methods to make the problem tractable. Afterwards, we consider more general non-Markovian scenarios but assume that we can predict the future costs with a known accuracy. We propose a method of dynamically placing each service instance upon its arrival and a way of finding the optimal look-ahead window size for cost prediction. The results are verified using simulations driven by both synthetic and real-world data traces. Finally, a framework for emulating MMCs in a more practical setting is proposed. In our view, the proposed solutions can enrich the fundamental understanding of the service placement problem. It can also path the way for practical deployment of MMCs. Furthermore, various solution approaches proposed in this thesis can be applicable or generalized for solving a larger set of problems beyond the context of MMC. Acknowledgments Looking back to the years of my Ph.D. study, there are many colleagues and friends whom I would like to thank. First, I would like to deeply thank my Ph.D. supervisor Prof. Kin K. Leung, who has been continuously supporting and guiding me throughout these years. Prof. Leung sets an excellent example as a world-class researcher. From him, I have learned not only how to do research, but also how to behave in the scientific community, which I believe will be my standard throughout the rest of my career. Furthermore, he has provided me with great opportunities of working with other outstanding researchers in different parts of the world – a very valuable experience for my professional development. I sincerely thank my close collaborators, Dr. Rahul Urgaonkar and Dr. Ting He, at IBM T.J. Watson Research Center in New York, Dr. Murtaza Zafer at Nyansa Inc. in California, and Dr. Kevin Chan at the U.S. Army Research Laboratory (ARL) in Maryland. Their superb technical skills and dedication to research greatly impressed me and helped me carry out the research work in this thesis. I have always been enjoying the fruitful collaborations with them. I would also like to thank Dr. Seraphin Calo (IBM) and Dr. Kang-Won Lee (formerly at IBM) who provided me with precious internship opportunities in the summers of 2013 and 2014. I also thank many other collaborators on the U.S./U.K. International Technology Alliance (ITA) Project, including Dr. Theodoros Salonidis (IBM), Dr. Bong-Jun Ko (IBM), Dr. Raghu Ganti (IBM) and Katy Warr (Roke Manor Research), as well as other collaborators of mine. Discussions with them have always been enlightening and bringing new ideas. With thanks, I would like to acknowledge that my Ph.D. work was funded in vi part by the ITA Project and a Scholarship from the Department of Electrical and Electronic Engineering at Imperial College London. Sincere thanks to my viva examiners Dr. Moez Draief (Imperial College London) and Dr. Ananthram Swami (ARL) for their time in attending my viva and providing insightful feedback. I finally thank my family and friends, for their endless support and encouragement, which makes me feel happy wherever I am and whatever I encounter. Shiqiang Wang Related Publications The work reported in this thesis has been published/submitted as the following papers. 1. Application Graph Placement (Chapters 2–3) (a) S. Wang, M. Zafer, and K. K. Leung, “Online workload placement with provable performance guarantees in cloud environments,” preprint available, to be submitted for journal publication. (b) S. Wang, G.-H. Tu, R. Ganti, T. He, K. K. Leung, H. Tripp, K. Warr, and M. Zafer, “Mobile micro-cloud: application classification, mapping, and deployment,” in Proc. of Annual Fall Meeting of ITA 2013, Oct. 2013. (c) S. Wang, M. Zafer, K. K. Leung, and T. He, “Security-aware application placement in a mobile micro-cloud,” in Proc. of Annual Fall Meeting of ITA 2013, Oct. 2013. 2. MDP-Based Approach to Dynamic Service Migration (Chapter 4) (a) S. Wang, R. Urgaonkar, M. Zafer, T. He, K. Chan, and K. K. Leung, “Dynamic service migration in mobile edge-clouds based on Markov decision processes,” submitted to IEEE Transactions on Mobile Computing. (b) S. Wang, R. Urgaonkar, M. Zafer, T. He, K. Chan, and K. K. Leung, “Dynamic service migration in mobile edge-clouds,” in Proc. of IFIP Networking 2015, May 2015. viii (c) S. Wang, R. Urgaonkar, T. He, M. Zafer, K. Chan, and K. K. Leung, “Mobility-induced service migration in mobile micro-clouds,” in Proc. of IEEE Military Communications Conference (MILCOM) 2014, Oct. 2014. (d) S. Wang, R. Urgaonkar, M. Zafer, K. Chan, T. He, and K. K. Leung, “Mobility-driven service migration in mobile micro-clouds,” in Proc. of Annual Fall Meeting of ITA 2014, Sept. 2014. 3. Dynamic Service Placement with Predicted Future Costs (Chapter 5) (a) S. Wang, R. Urgaonkar, T. He, K. Chan, M. Zafer, and K. K. Leung, “Dynamic service placement for mobile micro-clouds with predicted future costs,” submitted to IEEE Transactions on Parallel and Distributed Systems. (b) S. Wang, R. Urgaonkar, T. He, K. Chan, M. Zafer, and K. K. Leung, “Dynamic service placement for mobile micro-clouds with predicted future costs,” in Proc. of Annual Fall Meeting of the ITA 2015, Sept. 2015. (c) S. Wang, R. Urgaonkar, K. Chan, T. He, M. Zafer, and K. K. Leung, “Dynamic service placement for mobile micro-clouds with predicted future costs,” in Proc. of IEEE International Conference on Communications (ICC) 2015, Jun. 2015. 4. Emulation-Based Study (Chapter 6) (a) S. Wang, K. Chan, R. Urgaonkar, T. He, and K. K. Leung, “Emulationbased study of dynamic service placement in mobile micro-clouds,” in Proc. of IEEE Military Communications Conference (MILCOM) 2015, Oct. 2015. ix (b) S. Wang, K. Chan, R. Urgaonkar, T. He, and K. K. Leung, “Emulationbased study of dynamic service placement in mobile micro-clouds,” in Proc. of Annual Fall Meeting of the ITA 2015, Sept. 2015. The following publications were completed as part of my Ph.D. study, but its contents are not included in this thesis for compactness. 1. Dynamic Service Migration and Workload Scheduling Based on Lyapunov Optimization (a) R. Urgaonkar, S. Wang, T. He, M. Zafer, K. Chan, and K. K. Leung, “Dynamic service migration and workload scheduling in edge-clouds,” in Proc. of IFIP Performance 2015, Oct. 2015. (b) R. Urgaonkar, S. Wang, T. He, M. Zafer, K. Chan, and K. K. Leung, “Dynamic service migration and workload scheduling in micro-clouds,” in Proc. of Annual Fall Meeting of the ITA 2015, Sept. 2015. (c) R. Urgaonkar, S. Wang, K. Chan, and K. K. Leung, “Dynamic service migration and workload scheduling using Lyapunov optimization,” in Proc. of Annual Fall Meeting of ITA 2014, Sept. 2014. 2. Other Topics Related to Mobile Micro-Clouds (a) S. Wang, R. Urgaonkar, T. He, K. Chan, and K. K. Leung, “Distributed workload scheduling with limited control information exchange in mobile micro-clouds,” in Proc. of Annual Fall Meeting of the ITA 2015, Sept. 2015. (b) A. Freeman, K. Warr, H. Tripp, S. Wang, K. Leung, “Assessing the applicability of commercial cloud distributed processing techniques to mobile micro-cloud deployments,” in Proc. of Annual Fall Meeting of ITA 2014, Sept. 2014. x 3. Other Topics (a) S. Wang, L. Le, N. Zahariev, and K. K. Leung, “Centralized rate control mechanism for cellular-based vehicular networks,” in Proc. of IEEE Global Communications Conference (GLOBECOM) 2013, Dec. 2013. Contents Abstract iii Acknowledgments v Related Publications vii List of Figures xvi List of Tables xx List of Algorithms xxi Abbreviations xxii Mathematical Notations xxiii 1 2 Introduction 1 1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1.1 Initial Service Placement . . . . . . . . . . . . . . . . . . . 4 1.1.2 Real-Time Service Migration . . . . . . . . . . . . . . . . . 5 1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.3 Summary of Contributions . . . . . . . . . . . . . . . . . . . . . . 7 1.4 Organization of the Thesis . . . . . . . . . . . . . . . . . . . . . . 10 Background of Dynamic Service Placement 11 2.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.2 A Mixed-Integer Linear Program (MILP) Approach to Offline Service Placement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.2.1 15 Additional Definitions . . . . . . . . . . . . . . . . . . . . Contents 2.3 3 2.2.2 MILP Formulation . . . . . . . . . . . . . . . . . . . . . . 16 2.2.3 Example Mapping Result . . . . . . . . . . . . . . . . . . . 18 Approximation Algorithms . . . . . . . . . . . . . . . . . . . . . . 19 Online Placement of Application Graphs 23 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.1.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.1.2 Our Approach . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.1.3 Motivations and Main Results . . . . . . . . . . . . . . . . 26 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.2.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.2.2 Objective Function . . . . . . . . . . . . . . . . . . . . . . 32 Basic Assignment Unit: Single Linear Application Graph Placement 33 3.3.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . 34 3.3.2 Decomposing the Objective Function . . . . . . . . . . . . 35 3.3.3 Optimal Algorithm . . . . . . . . . . . . . . . . . . . . . . 37 3.3.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3.3.5 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Online Placement Algorithms for Tree Application Graphs . . . . . 40 3.4.1 Hardness Result . . . . . . . . . . . . . . . . . . . . . . . 41 3.4.2 When All Junction Node Placements Are Given . . . . . . . 41 3.4.3 When at Least One Junction Node Placement Is Not Given . 49 3.5 Numerical Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 54 3.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 3.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 3.2 3.3 3.4 4 xii An MDP-Based Approach to Dynamic Service Migration 64 4.1 64 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Contents 4.2 4.3 4.4 5 xiii 4.1.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . 65 4.1.2 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . 66 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . 68 4.2.1 Control Decisions and Costs . . . . . . . . . . . . . . . . . 69 4.2.2 Performance Objective . . . . . . . . . . . . . . . . . . . . 70 4.2.3 Characteristics of Optimal Policy . . . . . . . . . . . . . . 71 4.2.4 Generic Notations . . . . . . . . . . . . . . . . . . . . . . 72 Constant Cost Model under 1-D Mobility . . . . . . . . . . . . . . 72 4.3.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . 72 4.3.2 Optimal Threshold Policy . . . . . . . . . . . . . . . . . . 76 4.3.3 Simplifying the Cost Calculation . . . . . . . . . . . . . . 78 4.3.4 Algorithm for Finding the Optimal Thresholds . . . . . . . 79 4.3.5 Simulation Results . . . . . . . . . . . . . . . . . . . . . . 83 Constant-Plus-Exponential Cost Model under 2-D Mobility . . . . . 91 4.4.1 Simplifying the Search Space . . . . . . . . . . . . . . . . 91 4.4.2 Optimal Policy for Distance-Based MDP . . . . . . . . . . 92 4.4.3 Approximate Solution for 2-D Mobility Model . . . . . . . 100 4.4.4 Application to Real-World Scenarios . . . . . . . . . . . . 110 4.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 Dynamic Service Placement with Predicted Future Costs 5.1 5.2 132 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 5.1.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . 133 5.1.2 Main Contributions . . . . . . . . . . . . . . . . . . . . . 133 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . 135 5.2.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . 136 5.2.2 Actual and Predicted Costs . . . . . . . . . . . . . . . . . . 139 Contents 5.2.3 5.3 5.4 5.5 5.6 5.7 6 xiv Our Goal . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 Offline Service Placement with Given Look-Ahead Window Size . . 141 5.3.1 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . 142 5.3.2 Equivalence to Shortest-Path Problem . . . . . . . . . . . . 143 5.3.3 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 Complexity Reduction and Online Service Placement . . . . . . . . 144 5.4.1 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . 144 5.4.2 Performance Analysis . . . . . . . . . . . . . . . . . . . . 148 Optimal Look-Ahead Window Size . . . . . . . . . . . . . . . . . . 161 5.5.1 Upper Bound on Cost Difference . . . . . . . . . . . . . . 163 5.5.2 Characteristics of the Problem in (5.26) . . . . . . . . . . . 165 5.5.3 Finding the Optimal Solution . . . . . . . . . . . . . . . . 167 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . 168 5.6.1 Synthetic Arrivals and Departures . . . . . . . . . . . . . . 170 5.6.2 Real-World Traces . . . . . . . . . . . . . . . . . . . . . . 171 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 Emulation-Based Study 175 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 6.2 System Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . 177 6.3 6.2.1 Network Connection and User Mobility . . . . . . . . . . . 177 6.2.2 Service Model . . . . . . . . . . . . . . . . . . . . . . . . 178 Packet Exchange and Placement Control . . . . . . . . . . . . . . . 180 6.3.1 Control Messages . . . . . . . . . . . . . . . . . . . . . . . 180 6.3.2 Packet Exchange and Control Procedure . . . . . . . . . . . 182 6.3.3 Service Placement Decisions . . . . . . . . . . . . . . . . . 183 6.4 Emulation Scenario and Results . . . . . . . . . . . . . . . . . . . 185 6.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 Contents 7 xv Conclusions and Future Work 7.1 7.2 192 Contributions and Conclusions . . . . . . . . . . . . . . . . . . . . 192 7.1.1 Application Graph Placement . . . . . . . . . . . . . . . . 192 7.1.2 MDP-Based Approach to Dynamic Service Migration . . . 193 7.1.3 Dynamic Service Placement with Predicted Future Costs . . 195 7.1.4 Emulation-Based Study . . . . . . . . . . . . . . . . . . . 195 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 Bibliography 198 A Approximation Ratio for Cycle-free Mapping 206 B Constant-Plus-Exponential Cost Approximation to General Cost Functions 210 C Proofs 214 C.1 Proof of Proposition 3.3 . . . . . . . . . . . . . . . . . . . . . . . . 214 C.2 Proofs of Proposition 4.1 and Corollary 4.1 . . . . . . . . . . . . . 217 C.2.1 Proof of Proposition 4.1 . . . . . . . . . . . . . . . . . . . 217 C.2.2 Proof of Corollary 4.1 . . . . . . . . . . . . . . . . . . . . 219 C.3 Proof of Proposition 4.5 . . . . . . . . . . . . . . . . . . . . . . . . 220 C.4 Proof of Proposition 5.2 . . . . . . . . . . . . . . . . . . . . . . . . 230 C.5 Proof of Proposition 5.4 . . . . . . . . . . . . . . . . . . . . . . . . 236 List of Figures 1.1 Application scenario with mobile micro-clouds (MMCs). . . . . . . 1.2 Example scenario with face recognition application, where the red 2 arrows show the data transmission path: (a) user connected to MMC 1, (b) user connected to MMC 2. . . . . . . . . . . . . . . . . . . . 3 2.1 The service placement problem. . . . . . . . . . . . . . . . . . . . 11 2.2 Domain and conflict constraints: (a) domain constraint – application node 1 can only be mapped to physical nodes A, B, and C and application node 2 can only be mapped to physical nodes D and E; (b) conflict constraint – application nodes 1 and 3 cannot be mapped onto the same physical node. . . . . . . . . . . . . . . . . . . . . . 2.3 15 Example of service placement: (a) problem setting, (b) mapping result. In (a), the numbers besides nodes and edges in the application graph are resource demands, and the numbers in the physical graph are capacity values, the underlined numbers correspond to edge values. In (b), the underlined numbers are the bandwidth consumption on the corresponding communication links. . . . . . . . . . . . . . 3.1 19 Mapping with and without cycles. In this example, the path in the application graph is between application node 1 and application node 5. 28 3.2 Auxiliary graph and algorithm procedure for the placement of a linear application graph onto a tree physical graph. . . . . . . . . . . . 39 List of Figures 3.3 xvii Example of application graph with given placement of junction nodes. Junction node 2 is placed on physical node B and junction node 5 is placed on physical node E. The algorithm needs to decide the placement of the remaining nodes, subject to the cycle-free constraint. 3.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 Example of application graphs with some unplaced junction nodes, the nodes and edges within each dashed boundary form a general branch: (a) nodes 2 and 5 are both unplaced, (b) node 2 is placed, node 5 is unplaced, (c) node 2 is placed, nodes 5 and 6 are unplaced. 3.5 Maximum resource utilization when junction node placements are pre-specified. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6 57 Maximum resource utilization when junction node placements are not pre-specified. . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7 50 57 Example where application and physical graphs are not trees: (a) application graph, (b) physical graph, (c) restricted physical graph with pre-specified placement of application nodes 1 and 2. . . . . . 60 4.1 Timing of the proposed service migration mechanism. . . . . . . . . 69 4.2 MDP model for service migration. The solid lines denote transition under action a = 0 and the dotted lines denote transition under action a = 1. When taking action a = 1 from any state, the next state is e = −1 with probability q, e = 0 with probability 1 − p − q, or e = 1 with probability p. . . . . . . . . . . . . . . . . . . . . . . . . . . 74 4.3 Frequency of different optimal threshold values under γ = 0.9. . . . 84 4.4 Performance under different ξ with γ = 0.5. . . . . . . . . . . . . . 86 4.5 Performance under different ξ with γ = 0.9. . . . . . . . . . . . . . 87 4.6 Performance under different ξ with γ = 0.99. . . . . . . . . . . . . 88 List of Figures 4.7 xviii An example of distance-based MDP with the distances {d(t)} (before possible migration) as states. In this example, migration is only performed at state N , and only the possible action of a(N ) = 1 is shown for compactness. The solid lines denote state transitions without migration. . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 4.8 Example of constant-plus-exponential cost function cd (y). . . . . . 94 4.9 Example of 2-D offset model on hexagon cells, where N = 3. . . . 100 4.10 Simulation result for 2-D random walk with γ = 0.5. . . . . . . . . 106 4.11 Simulation result for 2-D random walk with γ = 0.9. . . . . . . . . 107 4.12 Simulation result for 2-D random walk with γ = 0.99. . . . . . . . 108 4.13 Instantaneous average cost per user in each timeslot over a day in trace-driven simulation, where Rt = Rp = 1.5. An enlarged plot of the circled area is shown on the top-right of the plot. The arrows annotated with (A), (B), (C), and (D) point to the average values over the whole day of the corresponding policy. . . . . . . . . . . . . . . 127 4.14 Cost reduction (averaged over the entire day) compared to alternative policies in trace-driven simulation, the error bars denote the standard deviation (where we regard the instantaneous cost at different time of the day as samples): (a)–(b) cost reduction vs. different Rp , (c)– (d) cost reduction vs. different Rt , (e) cost reduction vs. different number of cells with MMC, (f) cost reduction vs. different capacity limit of each MMC (expressed as the maximum number of services allowed per MMC). . . . . . . . . . . . . . . . . . . . . . . . . . . 128 5.1 Timing of the proposed approach. . . . . . . . . . . . . . . . . . . 136 5.2 Shortest-path formulation with N = 2, M = 2, and T = 3. Instance i = 1 is running in all slots, instance i = 2 arrives at the beginning of slot t0 + 1 and is running in slots t0 + 1 and t0 + 2. . . . . . . . . 143 List of Figures 5.3 xix Illustration of the performance gap for t = 1, T = 1, and N = 1, where amax denotes the maximum resource consumption of a single instance. In this example, (5.12) becomes φ ≥ becomes ψ ≥ 5.4 ψnum . ψdenom φnum , φdenom and (5.13) . . . . . . . . . . . . . . . . . . . . . . . . . . 158 Results with synthetic traces: (a) objective function value, (b) average performance ratio. . . . . . . . . . . . . . . . . . . . . . . . . 171 5.5 Results with real-world traces (where the costs are summed over all clouds, i.e., the A(t) values): (a) Actual costs at different time of a day, where β = 0.4 for the proposed method E. The arrows point to the average values over the whole day of the corresponding policy. (b) Actual costs averaged over the whole day. . . . . . . . . . . . . 173 6.1 System architecture for CORE emulation. . . . . . . . . . . . . . . 177 6.2 Emulation scenario (source of map: https://maps.google.com/). . . . 186 6.3 Instantaneous round-trip delays of service packets for the first 3, 000 s of emulation with T = 2 s: (a) AM policy, (b) IM policy, (c) MAH policy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188 6.4 Moving average results for the first 3, 000 s of emulation with T = 2 s: (a) round-trip delay of service packets, (b) number of migrations. 189 6.5 Overall results: (a) average round-trip delay of service packets (error bars denote the standard deviation), (b) average number of migrations per second, (c) total number of received service packets. . . . . 190 B.1 Examples of approximating a general cost function with exponential √ cost function: (a) f (x) = ln(x + 1) + 10, (b) f (x) = x + 1 + 5, (c) f (x) = x2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212 C.1 Illustration of original and modified 2-D MDPs, only some exemplar states and transition probabilities are shown: (a) original, (b) modified.221 List of Tables 6.1 Emulation setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 List of Algorithms 3.1 Placement of a linear application graph onto a tree physical graph . 3.2 Online placement of a service that is either a simple branch or a set 38 of nodes with given placement . . . . . . . . . . . . . . . . . . . . 45 3.3 High-level procedure for multiple arriving tree application graphs . 47 3.4 Tree-to-tree placement when some junction nodes are not placed . . 52 4.1 Modified policy iteration algorithm for finding the optimal thresholds 81 4.2 Modified policy-iteration algorithm based on difference equations . 5.1 Procedure for offline service placement . . . . . . . . . . . . . . . 142 5.2 Algorithm for solving (5.3) . . . . . . . . . . . . . . . . . . . . . . 145 5.3 Procedure for online service placement . . . . . . . . . . . . . . . . 147 5.4 Binary search for finding optimal window size . . . . . . . . . . . . 168 6.1 Procedure at the core cloud . . . . . . . . . . . . . . . . . . . . . . 182 6.2 Procedure at each MMC . . . . . . . . . . . . . . . . . . . . . . . 183 6.3 Procedure at each user . . . . . . . . . . . . . . . . . . . . . . . . 183 99 Abbreviations 1-D One-Dimensional 2-D Two-Dimensional AM Always Migrate CORE Common Open Research Emulator EMANE Extendable Mobile Ad-hoc Network Emulator FLOPs Floating-Point Operations IM Infrequently Migrate LP Linear Program MAH Moving Average + Hysteresis MDP Markov Decision Process MILP Mixed-Integer Linear Program MMC Mobile Micro-Cloud OPT True Optimal Result vs. Versus w.r.t. With respect to Mathematical Notations Below are some generic notations used in this thesis, specific notations will be introduced in each chapter. , Is defined to be equal to |V| Number of elements in set V R = (V, E) Graph R with nodes V and edges E e = (v1 , v2 ) Edge e connects nodes v1 and v2 v → n, e → l Application node v (or link e) is mapped to physical node n (or link l) kϕ1 − ϕ2 k (or |ϕ1 − ϕ2 | Distance between locations ϕ1 and ϕ2 for 1-D cases) E {·} Pij or Pi,j Expected value Transition probability from state i to state j Pr{X} Probability of random event X a·b Dot-product of vectors a and b (g)m1 m2 (or (g)m1 m2 m3 ) dun,t (a) dy ∂wnh,t ∂znh (a, b, c) (y, z) ∇x (or ∇y,z ) The (m1 , m2 )th (or (m1 , m2 , m3 )th) element in vector or matrix g Derivative of un,t (y) w.r.t. y evaluated at y = a Partial derivative of wnh,t (yn , yh , znh ) w.r.t. znh evaluated at yn = a, yh = b, znh = c Vector that concatenates vectors y and z Gradient w.r.t. (y, z)) each element in vector x (or C HAPTER 1 Introduction 1.1 Overview Mobile applications have become increasingly popular over recent years, with examples including data streaming, real-time video processing, etc. These applications generally require high data processing capability. However, portable devices (e.g. smartphones) are limited by their size and battery life, which makes them incapable of performing complicated computational tasks. A solution to this problem is to utilize cloud computing techniques [1, 2], where the cloud provides additional data processing and computational capabilities. Such cloud-based applications usually consist of a front-end component running on the mobile device and one or multiple back-end components running on the cloud [3, 4]. This architecture makes it possible to instantiate complicated applications from handheld devices that have limited processing power. Traditionally, cloud services are provided by centralized data-centers that may be located far away from end-users, which can be inefficient because the user may suffer from long latency and poor connectivity due to long-distance communication [5]. The question on how to provide readily accessible cloud services has been an ongoing challenge, and it has become particularly important in recent years as delaysensitive and bandwidth-consuming applications emerge. As a result, the concept of mobile micro-cloud (MMC) has recently emerged. The core idea of MMC is to move computation closer to users, where small servers or data-centers that can 1.1. Overview 2 Centralized (core) cloud Backhaul network MMC 1 MMC 2 Area 1 Area 2 … MMC n Area n … … Figure 1.1: Application scenario with mobile micro-clouds (MMCs). host cloud applications are distributed across the network and connected directly to entities (such as cellular basestations) at the network edge [6, 7]. Fig. 1.1 shows an application scenario where MMCs coexist with the centralized core cloud. The idea of MMCs is also known as cloudlets [8], edge computing [9], fog computing [10], small cell cloud [11], follow me cloud [12], etc. We use the term “MMC” in this thesis. MMCs can be used for many applications that require low latency, high data processing capability, or high reliability [4, 5, 13, 14, 15]. They are expected to develop rapidly with the growth of new mobile applications and more advanced smartphones, and are also more robust than traditional cloud computing systems [8]. One important problem in MMCs is to decide where the computation for each user should be performed, with the presence of user mobility and other dynamic changes in the network. When a user requests for a service provided by the cloud, the service can run either in the centralized cloud or in one of the MMCs. There can also be multiple servers or datacenters within the centralized cloud or within each MMC. The question is how to choose the optimal location to run the service. In addition, the user may move across different geographical areas, thus another question is whether and where should we migrate (move) the service when the user location or network condition changes. We refer to the above problems as the dynamic service placement problem, which is the focus of this thesis. 1.1. Overview 3 Centralized (core) cloud Face recognition Database Backhaul network MMC 1 Image processing & Feature extraction MMC 2 User Face detection Video source (a) Centralized (core) cloud Face recognition Database Backhaul network Image processing & Feature extraction MMC 1 MMC 2 Face detection User Video source (b) Figure 1.2: Example scenario with face recognition application, where the red arrows show the data transmission path: (a) user connected to MMC 1, (b) user connected to MMC 2. 1.1. Overview 4 We illustrate the service placement problem with an example face recognition application. This application recognizes faces from a real-time video stream captured by the camera of a hand-held device, as shown in Fig. 1.2. Such applications typically consist of at least three modules: face detection; image processing and feature extraction; and face recognition. Although one can further break each module into smaller building blocks, we consider each module as a whole for simplicity. The service placement problem aims to find the location to place each module. As mentioned above, the problem includes two subproblems: initial service placement and real-time service migration. 1.1.1 Initial Service Placement The initial service placement refers to the placement of different modules at the time the service is initialized, i.e., when the face recognition application starts. One possible result of initial service placement is shown in Fig. 1.2(a). Here, the face detection module is placed at the user, meaning that this module runs directly on the user device. The purpose of face detection is to find areas of an image that contain faces. This task requires relatively low processing capability, and once finished, the application only needs to send specific parts of the image that contain faces for next stage processing. Therefore, it can be beneficial to place this module on the user device, because it saves the necessary communication bandwidth between user and cloud, and at the same time does not consume much processing power of the user device. The image processing and feature extraction stage usually requires more processing than face detection. Hence, it can be placed on the MMC closest to the user, in order not to overload the user device’s processing resource. After processing at this stage, only the extracted features need to be sent to the next stage (face recognition, which is placed on the centralized cloud), thus such a placement reduces the amount 1.1. Overview 5 of data sent through the backhaul network. The reason for placing the face recognition module on the centralized cloud is that this module often needs to frequently look up a large database, which contains features of different people’s faces. It is often impractical to transfer the whole database to MMCs due to its size. Thus this placement appears to be good from our reasoning. Most realistic applications are more complex than our example application here. Therefore, a conceptual analysis such as the above is not always possible. We need to develop algorithms for finding optimal placement. Multiple services may also arrive to the cloud system over time, thus we need to consider such online service arrivals when developing placement algorithms. 1.1.2 Real-Time Service Migration After the initial service placement, the mobile user may move to a different location. One possibility after such a movement is that the user is now associated to a different MMC, and it has to send data via the backhaul network if the related application module is still running in the previous MMC, as shown in Fig. 1.2(b). Obviously, in Fig. 1.2(b), the communication overhead can be reduced if we migrate the image processing and feature extraction module from MMC 1 to MMC 2. However, as part of migration, data related to the state of the application module usually needs to be transmitted from the original MMC 1 to the new MMC 2. Migration decisions need to be made in a considerate way to achieve a good tradeoff between the migration cost and benefits after migration. A well-designed algorithm is usually required for making such decisions. Except for user mobility, other factors that may trigger migration include change in resource availability (caused by factors other than user mobility) and mobility of MMCs (for scenarios where MMCs are installed on moving vehicles, for example). 1.2. Motivation 6 We mainly focus on user mobility as a dominating cause for migration in this thesis, but our results can be extended to more general cases by incorporating parameters in the cost functions to reflect resource availability and considering the relative mobility of users with respect to MMCs. 1.2 Motivation The previous section has shown the importance of the service placement problem. This section outlines the main gaps between existing literature and what we would like to achieve in this thesis. Detailed literature review on specific aspects are included in each chapter later on. Cloud computing is a form of distributed computing, where computation is distributed across multiple machines and data exchange is necessary over the communication network. Such a distributed system has many benefits. For example, as mentioned before, resource intensive processing tasks can be offloaded to more powerful computers; processing components that require access to large databases can be placed close to the database, to reduce the network communication overhead. In the traditional setting, cloud services are provided by large datacenters, in which the allocation of jobs to servers (i.e., service placement) and communication within the datacenter are generally predictable and can usually be scheduled by a centralized scheduler. The idea of MMC was introduced only a few years ago, driven by the increasing popularity and demand of highly reliable and low-latency cloud applications, such as the face recognition application mentioned in Section 1.1. Compared to traditional centralized clouds, MMCs have the following features: 1. Computation is distributed across a large geographical area, whereas in the centralized cloud, computation is usually only distributed within the datacen- 1.3. Summary of Contributions 7 ter. 2. There is a hierarchy from the core centralized cloud down to the MMCs at the network edge. When we allow multiple layers of MMCs, those MMCs that are closer to the network edge serve a smaller area. 3. Compared to centralized clouds, MMCs exhibit much more dynamics in resource availability, due to user mobility and other uncontrollable factors in the network and the cloud (see Section 1.1.2). The above features cause many existing approaches for service placement in centralized cloud computing [16, 17] inapplicable to MMC, because those approaches do not consider the presence of dynamically changing resource availability related to user mobility. This is a fundamental challenge in MMCs. Conversely, the hierarchical nature of MMCs can enable more efficient service placement algorithms. We also note that most existing literatures on MMC focus on system aspects [4, 5, 8, 9, 10, 12, 13, 14, 15], while only a few consider its theoretical aspects under simplified settings [11, 18, 19]. The goal of our research is therefore to enrich the theoretical understanding of MMCs, with focus on the dynamic service placement problem. Our results include a set of efficient placement algorithms with provable performance guarantees, which provide insights for practical deployment and are also directly applicable in practice. 1.3 Summary of Contributions As illustrated by the example in Section 1.1, the service placement problem contains two subproblems. One is the initial placement of services, and the other is the realtime service migration due to dynamic changes. We consider both aspects in this thesis. 1.3. Summary of Contributions 8 The main contributions of this thesis are summarized as follows: (a) We first model the resource demand of a service with an application/service graph and the resource availability of the physical computing system with a physical graph. The resource demand/availability are annotated on each node and link of these graphs, and there can be multiple types of computational resources at nodes. The service placement problem is then essentially the problem of placing the application graph onto the physical graph, where the node and link assignment1 are jointly considered. With this model, we formulate the offline placement problem for general graphs as a mixed-integer linear program (MILP) in Section 2.2, with consideration of practical domain and conflict constraints related to security and access-control policies. (b) After proving that the general graph placement problem is NP-hard even in the offline case, we consider in Section 3.3 the placement of a linear application graph onto a tree physical graph and propose an algorithm for finding its optimal solution. (c) Based on the line-to-tree placement algorithm proposed in (b), in Section 3.4, we generalize the formulation and propose online approximation algorithms with poly-log competitive ratio2 for the placement of multiple tree application graphs that arrive over time. (d) After the initial service placement in Chapters 2 and 3, service migration may be needed due to user and network dynamics. Thus, we consider the dynamic service migration problem under real-time variations in the user location and 1 We exchangeably use the terms “placement”, “assignment”, “mapping”, and “configuration” in this thesis. 2 See Section 2.3 for definition of competitive ratio. The term “poly-log” means “polynomial logarithmic”. For example, if an algorithm’s competitive ratio is log2 (N ), where N is the problem size, we can say that the competitive ratio is poly-log in N . 1.3. Summary of Contributions 9 resource availability, where we first model the problem as a Markov decision process (MDP) in Section 4.2. (e) We show in Section 4.3 that under one-dimensional (1-D) user mobility and constant cost values, the optimal policy of the MDP (defined in (d)) is a threshold policy. An algorithm is proposed to find this optimal threshold policy, which is more efficient than standard MDP solution approaches. (f) To incorporate more general cases, we consider two-dimensional (2-D) user mobility with a constant-plus-exponential cost model in Section 4.4. We approximate the state space of the MDP (defined in (d)) by the distance between the user and service locations. We show that the resulting MDP is exact for uniform 1-D random walk user mobility while it provides a close approximation for uniform 2-D random walk mobility with a constant additive error term. A new algorithm and a numerical technique is proposed for computing the optimal policy, which is significantly faster than traditional approaches. We also discuss how to apply the proposed algorithm in real-world scenarios where many theoretical assumptions are relaxed. (g) For more general cases, the MDP-based approach can be inapplicable. However, if there is an underlying mechanism to predict the future costs, we can find an approximately optimal service placement sequence that minimizes the average cost over time. In Chapter 5, we first propose a method which solves for the optimal placement sequence for a specific look-ahead time window, based on the predicted costs in this time window. We show that when there exist multiple services, the problem is NP-hard, and propose an online approximation algorithm with provable performance guarantee to find the solution. Then, we propose a method to find the optimal look-ahead window size based on the prediction error, which minimizes an upper bound of the average cost. 1.4. Organization of the Thesis 10 (h) The above study is theoretical in nature where realistic aspects such as delay in control message exchange are not considered. To study the performance of dynamic service placement in a more practical setting, we propose an emulation framework in Chapter 6, in which physical nodes are encapsulated into virtual containers that are connected via emulated physical links. A real computer program with some basic packet exchange functionalities runs in each node in the emulation. Emulation results of different service placement policies are shown and their insights are discussed. 1.4 Organization of the Thesis This thesis is organized as follows. Chapter 2 introduces the general problem and related techniques that are used in this thesis. Chapter 3 presents the optimal algorithm for line-to-tree placement and online approximation algorithms for the placement of an incoming stream of tree application graphs. The MDP-based approach for dynamic service migration is presented in Chapter 4. Chapter 5 presents algorithms for dynamic service placement with predicted future costs. An emulation framework is presented in Chapter 6. Chapter 7 draws conclusions and discusses some future directions. C HAPTER 2 Background of Dynamic Service Placement 2.1 Definitions We can abstract the service placement problem as a graph (or, as a special case, node) placement problem, as illustrated in Fig. 2.1. In the following, we introduce some concepts that will be used in this thesis. Application/Service Graph: A service or a service instance1 can be abstracted as a graph (referred to as the application/service graph2 ) in which the nodes represent processing/computational modules in the service, and the edges represent communication demand between the nodes. Each node v ∈ V in the application graph R = (V, E) is associated with parameters that represent the computational resource 1 2 For simplicity, we exchangeably use the terms “service” and “service instance” in this thesis. We exchangeably use the notions application graph and service graph in this thesis. 1 CPU, storage, I/O requirements 2 Communication bandwidth requirement 3 Map A B C D Current utilization and total amount of resources 4 E 6 5 F H G Physical graph (cloud environment) Application graph Figure 2.1: The service placement problem. Application graph Mapping result 1 2 A 1 2 A 1 3 2.1. Definitions 12 (of K different types) demands of node v. Similarly, each edge e ∈ E is associated with a communication bandwidth demand. The notation e = (v1 , v2 ) denotes that application edge e connects application nodes v1 and v2 . The application graph R can be either a directed or an undirected graph. If it is a directed graph, the direction of edges specify the direction of data communication; if it is an undirected graph, data communication can occur in either direction along application edges. Physical Graph: The physical computing system can also be abstracted as a graph (referred to as the physical graph) with nodes denoting network elements such as data-centers3 , servers, routers, etc. and edges denoting communication links between the nodes. Each node n ∈ N in the physical graph Y = (N , L) has K different types of computational resources, and each edge l ∈ L has a communication capability. For nodes in the physical network that do not have capability of hosting computational modules (such as routers), we can regard their computational capacity as zero. We use the notation l = (n1 , n2 ) to denote that physical link l connects physical nodes n1 and n2 . Similar to the application graph, the physical graph can be either directed or undirected, depending on whether the physical links are bidirectional (i.e., communication in both directions share the same link) or singledirectional (i.e., communication in each direction has a separate link). Policy: A policy specifies how to place the application graph R onto the physical graph Y. In a dynamic setting, the policy can also depend on the current state of the system (defined below), so that a new placement decision can be made based on the current placement. It can also be dependent on the time of performing the placement. 3 A physical node can either represent a data-center (which is a collection of network elements) or a single network element, depending on the granularity we consider in practical cases. 2.1. Definitions 13 State: The state of the cloud system specifies where each application node and link is placed, and the current status (including topology, resource demand/availability, etc.) of the application and physical graphs. Cost: Each possible way of service placement incurs some costs. Generally, there are two types of costs. The first type is the cost of running services on the cloud system when the placement of these services remain unchanged. The second type is the migration cost, which is incurred when we change the placement of the service during its operation. We consider the migration cost because when the service placement is altered, we may need to restart the service or transfer some state information to the new physical machine that runs the service, which incurs some cost. The explicit cost definition will be discussed in details for each specific problem in the remainder of this thesis. Constraints: There can be constraints such as the resource capacity of each physical node and link, the location that each application node and link can be placed at, as well as other constraints including those that are related to access or security restrictions. Offline and Online Service Placement: Throughout this thesis, we say that a ser- vice placement is offline when our goal is to place a single or a set of application graphs “in one shot”. In contrast, an online service placement is the case where we have an incoming stream of application graphs, which have to be sequentially placed onto the physical graph as each application graph arrives. There may or may not exist some application graphs that depart while others are arriving. A Note on Graphs: For the problems discussed Chapters 4, 5, and 6, the notion of graphs is not important for explaining the problems. In those chapters, we will 2.2. A Mixed-Integer Linear Program (MILP) Approach to Offline Service Placement 14 not explicitly use the notion of graphs in the problem formulation, but one can easily define those problems based on application and physical graphs. 2.2 A Mixed-Integer Linear Program (MILP) Approach to Offline Service Placement To provide a basic understanding to service placement, in this section, we illustrate how an offline service placement problem can be formulated as an MILP, and show an example of the placement result. The goal of service placement is to place the service onto the physical cloud. With the notion of application and physical graphs, the service placement problem can be seen as a graph mapping problem, which maps each application node to one physical node and each application link to at least one physical path connecting the two application nodes. A feasible mapping is usually subject to several constraints that need to be satisfied. For example, the resource capacity at a physical node or link can be finite, and the sum resource consumption of application nodes/links that the physical node/link is hosting cannot exceed this limit. In addition, there can be constraints on where the application nodes can be placed and which set of application nodes cannot be hosted on the same physical node. We call these two types of constraints respectively as domain and conflict constraints as illustrated in Fig. 2.2. These constraints can be related to security policies, database locations, etc. All the above-mentioned constraints can be expressed as a set of linear constraints with some binary variables. As a result, the service placement problem can be formulated as an MILP, as described next. 2.2. A Mixed-Integer Linear Program (MILP) Approach to Offline Service Placement 15 1 2 Conflict fl 1 2 3 ((a)) 4 ((b)) Figure 2.2: Domain and conflict constraints: (a) domain constraint – application node 1 can only be mapped to physical nodes A, B, and C and application node 2 can only be mapped to physical nodes D and E; (b) conflict constraint – application nodes 1 and 3 cannot be mapped onto the same physical node. 2.2.1 Additional Definitions In the following, we summarize several definitions in addition to those introduced in Section 2.1, which are used in the MILP formulation. We assume that the application graph R is a directed graph (where the direction of each edge specifies the data flow direction) and the physical graph Y is an undirected graph. Other cases can also be formulated using MILP but we omit the discussion here for simplicity. Each node v ∈ V is associated with demands φv,1 , φv,2 , ..., φv,K which represent the computation resource (of K different types) demand of node v. Similarly, each edge e ∈ E is associated with a communication bandwidth demand ψe . The capacity of type k resource at node n ∈ N is cn,k , and the communication bandwidth capacity of edge l ∈ L is cl . For nodes in the physical network that do not have capability of hosting service modules (such as routers), we can set cn,k = 0 for ∀k = {1, 2, ..., K}. Let Nm (v) ⊆ N for ∀v ∈ V denote the subset of physical nodes that v can be mapped to, and Nm2 (v1 , v2 ) ⊆ N × N for ∀v1 , v2 ∈ V denote the subset of physical node-pairs that the application node-pair (v1 , v2 ) can be mapped to. We also use the notation v → n to denote that the application node v is mapped to physical node n, and we use e → l to denote that the application link e is mapped to physical link l. 2.2. A Mixed-Integer Linear Program (MILP) Approach to Offline Service Placement 16 Define the normalized demand of type k resource of placing v to n as dv→n,k = φv,k , if n ∈ Nm (v) cn,k ∞, . (2.1) otherwise Define the normalized resource demand of placing e = (v1 , v2 ) to l = (n1 , n2 ) as be→l = ψe , cl if (n1 , n2 ) ∈ Nm2 (v1 , v2 ) . (2.2) ∞, otherwise 2.2.2 MILP Formulation We first define the decision variables in our optimization problem. The flow variables fe→(n1 ,n2 ) for ∀e ∈ E and ∀n1 , n2 ∈ N that have a communication link connecting them denote the amount of data sent from n1 to n2 , for the application edge e. The binary variables xv→n for ∀v ∈ V, ∀n ∈ Nm (v) indicate whether v is mapped to n. Auxiliary variables yv→n for ∀v ∈ V, ∀n ∈ Nm (v) indicate whether a node that has a conflict with v is mapped to n, which is used to assist the optimization problem formulation. We focus on the case where K = 1 in our formulation in this section, but it can be easily extended to other values of K. To jointly consider the load balancing of servers and communication links, as well as minimizing the total amount of communication bandwidth consumption, we 2.2. A Mixed-Integer Linear Program (MILP) Approach to Offline Service Placement 17 consider the following objective function: ( min ! max αn n∈N + X max l=(n1 ,n2 )∈L X + dv→n,1 xv→n v∈V βl X fe→(n1 ,n2 ) + fe→(n2 ,n1 ) cl e∈E βl0 l=(n1 ,n2 )∈L ! X fe→(n1 ,n2 ) + fe→(n2 ,n1 ) cl e∈E , (2.3) where the edge l connects n1 and n2 , and αn , βl , and βl0 are weighting factors. This is not the only possible form of objective function. One can define other types of objective functions and still formulate the problem as an MILP as long as the objective function is linear in the decision variables. The objective function (2.3) can be rewritten as a linear objective function, by adding two additional sets of constraints for the maximum operations, as follows: min X t1 + t2 + βl0 l=(n1 ,n2 )∈L s.t. αn X X fe→(n1 ,n2 ) + fe→(n2 ,n1 ) cl e∈E dv→n,1 xv→n ≤ t1 , ∀n ∈ N v∈V max l=(n1 ,n2 )∈L βl X fe→(n1 ,n2 ) + fe→(n2 ,n1 ) cl e∈E ≤ t2 , ∀l = (n1 , n2 ) ∈ L Other constraints. (2.4) The other constraints are discussed in the following. The node and edge capacity constraints are: X dv→n,1 xv→n ≤ 1, ∀n ∈ N , (2.5) fe→(n1 ,n2 ) + fe→(n2 ,n1 ) ≤ cl , ∀l = (n1 , n2 ) ∈ L, (2.6) v∈V X e∈E 2.2. A Mixed-Integer Linear Program (MILP) Approach to Offline Service Placement 18 The flow conservation constraint is: X X fe→(n1 ,n2 ) − n2 :l=(n1 ,n2 )∈L fe→(n2 ,n1 ) n2 :l=(n1 ,n2 )∈L = cl be→l xv1 →n1 1n1 ∈Nm (v1 ) − xv2 →n1 1n1 ∈Nm (v2 ) , ∀n1 ∈ N , ∀e ∈ E, (2.7) where e is the directed application edge from v1 to v2 , and the indicator 1 indicates whether the condition in its subscript is satisfied. Note that these conditions are static and do not change with the decision variables in the optimization. The following constraint guarantees that each application node is mapped to exactly one physical node: X xv→n = 1, ∀v ∈ V. (2.8) n∈Nm (v) The node conflict constraints are: X yv→n ≤ xv1 →n , ∀v ∈ V, ∀n ∈ N , (2.9) 2 (v,v ) (n,n)∈N / m 1 P yv→n ≥ 2 (v,v ) (n,n)∈N / m 1 xv1 →n |V| , ∀v ∈ V, ∀n ∈ N , xv→n + yv→n ≤ 1, ∀v ∈ V, ∀n ∈ N . (2.10) (2.11) We also require that fe→(n1 ,n2 ) ≥ 0, xv→n ∈ {0, 1}, and yv→n ∈ {0, 1}. 2.2.3 Example Mapping Result Our optimization problem is an MILP, which can be solved with IBM CPLEX [20], or OPTI Toolbox [21], etc. Fig. 2.3 shows an example scenario and its mapping result which has been obtained with OPTI Toolbox. In the example, application 2.3. Approximation Algorithms 19 Application graph: 1 Physical graph: h 1 05 0.5 A 3 1 15 B 1.5 05 0.5 1 2 01 4 0.1 15 1.5 05 0.5 05 5 0.5 15 C 1.5 05 0.5 2 2 1 1 2 2 1 05 0.5 A 15 D 1.5 B 2 05 0.5 0 83 0.83 C 3 5 D 0 67 0.67 4 Conflict ((a)) (b) Figure 2.3: Example of service placement: (a) problem setting, (b) mapping result. In (a), the numbers besides nodes and edges in the application graph are resource demands, and the numbers in the physical graph are capacity values, the underlined numbers correspond to edge values. In (b), the underlined numbers are the bandwidth consumption on the corresponding communication links. node 1 can only be mapped to physical node A, and application nodes 4 and 5 can only be mapped to physical nodes C and D but not on the same node. We set αn = βl = βl0 = 1, the demands and capacities are indicated in Fig. 2.3(a) and the mapping result is shown in Fig. 2.3(b). 2.3 Approximation Algorithms The MILP formulated in the above section is NP-hard. In fact, the service placement problem is NP-hard even for simple graphs as we will discuss later in Section 3.4.1. The term “NP-hard” refers to a particular problem class for which it is generally believed (although a formal proof is still an open question in algorithms research) that the simplest algorithms for solving such problems require exponential time-complexity. If an algorithm has exponential complexity, the complexity of the algorithm scales badly with the size of problem input. For example, in the service placement problem, the input size can be reflected by the number of nodes in both the application and physical graphs. It turns out that for large graphs, an algorithm 2.3. Approximation Algorithms 20 with exponential complexity requires a substantial amount of time to find the optimal solution, which can easily become unacceptably time consuming. Therefore, a common practice is to employ heuristic algorithms with polynomial time-complexity to solve such problems [16, 22]. Such algorithms find suboptimal solutions instead of optimal solutions. For most existing heuristic algorithms, the performance is evaluated via simulation or experimental studies under a limited set of application scenarios. A natural question is: Do such heuristics perform well enough in all the scenarios that possibly occur? Obviously, an empirical study based on simulations or experiments is not sufficient to answer this question. We need to seek theoretical foundations to justify the performance of these heuristic algorithms. One theoretical basis to study the performance of algorithms that approximate the solution to NP-hard problems is the field of approximation algorithms [23], wherein the worst-case difference between the solution from the approximation algorithm and the true optimal result (OPT) is quantified. We define the approximation ratio of a minimization problem as the maximum (worst-case) ratio (either exact or in an upper bound sense) of the result provided by the algorithm to OPT. Consider the service placement problem introduced earlier in this chapter. For the offline placement problem, we would like to place one or multiple application graphs at a single time. Suppose that we have defined a proper objective (cost) function (such as the one in (2.3)) and would like to minimize this cost function. Since this problem is NP-hard, we may want to find an approximately optimal solution to it using algorithms with polynomial time-complexity. Now, assume that we are given a problem input I. This problem input I specifies the topology of the application and physical graphs and all parameters associated with them. It also includes any additional input parameters to the problem. Assume we know that the optimal placement incurs a cost equal to OPTI . If our algorithm 2.3. Approximation Algorithms 21 incurs a cost equal to ALGI , we say that the approximation ratio of this algorithm is ΓA if ΓA ≤ max I ALGI OPTI (2.12) where the maximization is over all possible problem inputs. We say the approximation ratio is tight if equality is attained in (2.12). For the online service placement problem, multiple application graphs arrive over time, where there may or may not exist application graphs leaving while others are arriving. Here, instances (i.e., application graphs in our problem) arrive in real-time and an online algorithm needs to provide a solution at every time when an instance arrives. The competitive ratio is defined as the maximum (worst-case) ratio (either exact or in an upper bound sense) of the result from the online algorithm to the offline OPT, where the offline OPT is the true optimum by assuming that all the instances are known in advance [24]. We also introduce the notion of an online algorithm being c-competitive if its competitive ratio is c. Let us denote the problem input in the online case as I(M ), where M is the number of application graphs that have arrived to the system, among which some graphs may have already departed from the system. This input contains M instances (application graphs) that arrive one-by-one, and the online algorithm places them onto the physical graph right after an application graph arrives, without prior knowledge on future arrivals. We define OPTI(M ) as the cost of an optimal offline placement, and define ALGI(M ) as the cost of the placement found by the online algorithm. We say that the competitive ratio in placing M instances of this algorithm is ΓC if ΓC ≤ max I(M ) ALGI(M ) OPTI(M ) (2.13) where the maximization is over all possible problem inputs that have M instances. In some cases, the competitive ratio holds for an arbitrary value of M , and we may 2.3. Approximation Algorithms 22 omit the discussion on M in such cases. The main difference between the competitive and approximation ratios is that the competitive ratio considers the optimality gap with respect to (w.r.t.) an online algorithm, whereas the approximation ratio considers the optimality gap w.r.t. an offline algorithm. The competitive ratio is the same as the approximation ratio if we solve an offline problem in an online way, e.g., breaking the application graph into smaller subgraphs, and assuming online arrival of each of these subgraphs. Besides using the notions of approximation and competitive ratios, the optimality guarantee can also be quantified in many other ways, such as in the form of an additive error term that specifies the difference between the optimum and its approximate value. We will be considering different forms of optimality guarantees in this thesis. C HAPTER 3 Online Placement of Application Graphs 3.1 Introduction Due to the potential hardness of solving the MILP in Section 2.2, we consider approximation algorithms with polynomial time-complexity and provable performance guarantees in this chapter. We start with a basic building block that solves for the placement of a linear application graph. We show that for this case, it is actually possible to find the optimal placement (under some constraints) and we propose an algorithm for finding this placement. We then show that for more general cases, for example when the application graph is a tree, the problem is NP-hard. Afterwards, we generalize the algorithm for linear graph placement and obtain online approximation algorithms with poly-log1 competitive ratio for tree application graph placement. In our formulation, similar to Section 2.2, we jointly consider node and link assignment and incorporate multiple types of computational resources at nodes. 1 The term “poly-log” means “polynomial logarithmic”. For example, if an algorithm’s competitive ratio is log2 (N ), where N is the problem size, we can say that the competitive ratio is poly-log in N . 3.1. Introduction 3.1.1 24 Related Work In the literature, there is limited work on approximation algorithms with provable approximation/competitive ratios for the service placement problem, especially when it involves both server (node) and network (link) optimization. In [17], the authors proposed an algorithm for minimizing the sum cost with some considerations on load balancing, which has an approximate approximation ratio of O(N ), where N is the number of nodes in the physical graph. The algorithm is based on linear program (LP) relaxation2 , and only allows one node in each application graph to be placed on a particular physical node; thus, excluding server resource sharing among different nodes in one application graph. A drawback of this approach is that the approximation ratio of O(N ) is trivial, in the sense that one would achieve the same approximation ratio when placing the whole application graph onto a single physical node instead of spreading it across the whole physical graph. A theoretical work in [25] proposed an algorithm with N O(D) time-complexity and an approximation ratio of δ = O(D2 log(N D)) for placing a tree application graph with D levels of nodes onto a physical graph. It uses LP relaxation and its goal is to minimize the sum cost. Based on this algorithm, the authors showed an online algorithm for minimizing the maximum load on each node and link, which is O(δ log(N ))-competitive when the service durations are equal. The LP formulation in [25] is complex and requires N O(D) variables and constraints. This means when D is not a constant, the space-complexity (specifying the required memory size of the algorithm) is exponential in D. Another related theoretical work which proposed an LP-based method for offline placement of paths into trees was reported in [26], where the application nodes can 2 LP relaxation is a common technique used for approximating MILPs. It first replaces the integer variable with a continuous variable, so that the resulting problem becomes an LP which can be solved in polynomial time. Then, a rounding procedure is usually applied to round the values of relaxed integer variables to integers. 3.1. Introduction 25 only be placed onto the leaves of a tree physical graph, and the goal is to minimize link congestion. In our problem, the application nodes are distributed across users, MMCs, and core cloud, thus they should not be only placed at the leaves of a tree so the problem formulation in [26] is inapplicable to our scenario. Additionally, [26] only focuses on minimizing link congestion. The load balancing of nodes is not considered as part of the objective, whereas only the capacity limits of nodes are considered. Some other related work focuses on graph partitioning, such as [27] and [28], where the physical graph is defined as a complete graph with edge costs associated with the distance or latency between physical servers. Such an abstraction may combine multiple actual network links into one (abstract) physical edge. Thus, it can be inappropriate when we would like to consider load balancing across all actual network links, which is what we consider in this chapter. One important aspect is that most existing work, including [17, 26, 27], and [28], do not specifically consider the online operation of the algorithms. Although some of them implicitly claim that one can apply the algorithm repeatedly for each newly arriving service/application, the competitive ratio for such application is not clear. To the best of our knowledge, [25] is the only work that studied the competitive ratio of the online service placement problem that considers both node and link optimization. 3.1.2 Our Approach In this chapter, we propose algorithms for solving the online service placement problem with provable competitive ratios. Different from [25], our approach is not based on LP relaxation. Instead, our algorithms are built upon a baseline algorithm that provides an optimal solution to the placement of a linear application graph, i.e., an application graph that is a line. This is the main novelty in contrast to [25], because no optimal solution for linear application graph placement was presented in [25]. 3.1. Introduction 26 Many applications expected to run in an MMC environment can be abstracted as hierarchical graphs, and the simplest case of such a hierarchical graph is a line (see the example in Section 1.1). Therefore, the placement of a linear application graph is an important problem in the context of MMCs. Another novelty in our work, compared to [25] and most other approaches based on LP relaxation, is that our solution approach is decomposable into multiple small building blocks. This makes it easy to extend our proposed algorithms to a distributed solution in the future, which would be very beneficial for reducing the amount of necessary control information exchange among different cloud entities in a distributed cloud environment containing MMCs. This decomposable feature of algorithms also makes it easier to use these algorithms as a sub-procedure for solving a larger problem. It is also worth noting that the analytical methodology we use in this chapter is new compared to existing techniques such as LP relaxation, thus we enhance the set of tools for online optimization. The theoretical analysis in this chapter also provides insights on the features and difficulty of the problem, which can guide future practical implementations. In addition, the proposed algorithms themselves are relatively easy to implement in practice. 3.1.3 Motivations and Main Results We propose non-LP based approximation algorithms for online service placement in this chapter, which have both theoretical and practical relevance. The general problem of service placement is hard to approximate even from a theoretical point of view [25, 26, 29]. Therefore, similar to related work [17, 25, 26, 27, 28], we make a few simplifications to make the problem tractable. These simplifications and their motivations are described as follows. Throughout this chapter, we focus on application and physical graphs that have 3.1. Introduction 27 tree topologies. This restriction makes the problem mathematically tractable which allows us to obtain exact and approximation algorithms with provable guarantees. Thus, from a theoretical perspective, this work attempts to provide rigor to the service placement problem in a cloud environment. In terms of practical applicability, a tree application graph models a wide range of practical applications that involve a hierarchical set of processes (or virtual machines). For example, a virtual graph for applications involving streaming, multicasting, or data aggregation is typically represented as a hierarchical set of operators forming a tree topology [30, 31, 32]. Similarly, a web-application topology consisting of a front-end web-server connected to a set of load balancers which are then connected to databases has a tree topology. For the physical system, the main motivation for us to consider tree physical graphs is the hierarchical nature of MMCs, as discussed in Section 1.2. For a general connected network, a tree physical graph can also be regarded as a subgraph of the original physical network topology, which would arise from restrictions imposed by spanning-tree based routing mechanisms [33] etc. Furthermore, different services could be assigned on different tree subgraphs of the physical network, which makes this assumption less restrictive. In the tree application-graph topology, if we consider any path from the root to a leaf, we only allow those assignments where the application nodes along this path are assigned in their respective order on a sub-path of the physical topology (multiple application nodes may still be placed onto one physical node), thus, creating a “cycle-free” placement. Figure 3.1 illustrates this placement. Let nodes 1 to 5 denote the application nodes along a path in the application-graph topology. The cycle-free placement of this path onto a sub-path of the physical network ensures the order is preserved (as shown in Fig. 3.1(b)), whereas the order is not preserved in Fig. 3.1(c). A cycle-free placement has a clear motivation of avoiding cyclic commu- 2 Communication bandwidth requirement Map and total amount of resources D C 4 3 E 6 5 F H G Physical graph (cloud environment) Application graph 3.1. Introduction 28 Application graph Mapping result 1 A 2 B 3 1 A 1 2 3 4 4 C 5 Cycle-free (a) D 5 (b) B 2 4 3 C D 5 With cycles (c) Figure 3.1: Mapping with and without cycles. In this example, the path in the application graph is between application node 1 and application node 5. nication among the application nodes. For example, for the placement in Fig. 3.1(c), application nodes 2 and 4 are placed on physical node B, while application node 3 is placed on physical node C. In this case, the physical link B–C carries the data demand of application links 2–3 and 3–4 in a circular fashion. Such traffic can be naturally avoided with a cycle-free mapping (Fig. 3.1(b)), thus relieving congestion on the communication links. As we will see in the simulations in Section 3.5, the cycle-free constraint still allows the proposed scheme to outperform some other comparable schemes that allow cycles. Further discussion on the approximation ratio associated with the cycle-free restriction is given in Appendix A. In this chapter, for the purpose of describing the algorithms, we classify an application node as a junction node in the application graph when it is a root node and is connected to two or more edges, or when it is not a root node and is connected to three or more edges (among which one edge connects to its parent node). These junction nodes can be more significant than other nodes, because they represent data splitting or joining processes for multiple data streams. In some cases, the junction nodes may have pre-specified placement, because they serve multiple data streams that may be associated with different end-users, and individual data streams may arrive dynamically in an online fashion. Our work first considers cases where the placements of these junction nodes are pre-specified, and then extends the results to 3.1. Introduction 29 the general case where some junction nodes are not placed beforehand. For the aforementioned scenarios, we obtain the following main results for the problem of service placement with the goal of load balancing among the physical nodes and edges: 1. An optimal offline algorithm for placing a single application graph which is a linear graph, with O(V 3 N 2 ) time-complexity and O(V N (V + N )) spacecomplexity, where the application graph has V nodes and the physical graph has N nodes. 2. An online approximation algorithm for placing single or multiple tree application graphs, in which the placements of all junction nodes are pre-specified, i.e., their placements are given. This algorithm has a time-complexity of O(V 3 N 2 ) and a space-complexity of O(V N (V + N )) for each application graph placement, and its competitive ratio is O(log N ). 3. An online approximation algorithm for placing single or multiple tree application graphs, in which the placements of some junction nodes are not prespecified. This algorithm has a time-complexity of O(V 3 N 2+H ) and a spacecomplexity of O(V N 1+H (V + N )) for each application graph placement, and its competitive ratio is O(log1+H N ), where H is the maximum number of junction nodes without given placement on any single path from the root to a leaf in the application graph. Note that we always have H ≤ D, where D is the depth of the tree application graph. We consider undirected application and physical graphs in this chapter, which means that data can flow in any direction on an edge, but the proposed algorithms can be easily extended to some types of directed graphs. For example, when the tree application graph is directed and the tree physical graph is undirected, we can merge the two application edges that share the same end nodes in different directions into 3.2. Problem Formulation 30 one edge, and focus on the merged undirected application graph for the purpose of finding optimal placement. This does not affect the optimality because for any placement of application nodes, there is a unique path connecting two different application nodes due to the cycle-free constraint and the tree structure of physical graphs. Thus, application edges in both directions connecting the same pair of application nodes have to be placed along the same path on the physical graph. Our work also considers multiple types of resources on each physical node, such as CPU, storage, and I/O resources. The proposed algorithms can also work with domain constraints which restrict the set of physical nodes that a particular application node can be assigned to. The exact algorithm for single line placement can also incorporate conflict constraints where some assignments are not allowed for a pair of adjacent application nodes that are connected by an application edge; such constraints naturally arise in practice due to security policies as discussed in Section 2.2. 3.2 Problem Formulation 3.2.1 Definitions We consider the placement of application graphs onto a physical graph, where the application graphs represent services that may arrive in an online fashion. In this chapter, we reuse the definitions given in Section 2.1. In addition, we introduce some additional definitions as follows. Graphs: Because we consider multiple application graphs in this chapter, we de- note the tree application graph for the ith service arrival as R(i) = (V(i), E(i)). Throughout this chapter, we define V = |V|, E = |E|, N = |N |, and L = |L|, where | · | denotes the number of elements in the corresponding set. 3.2. Problem Formulation Costs: 31 For the ith service, the weighted cost (where the weighting factor can serve as a normalization to the total resource capacity) for type k ∈ {1, 2, ..., K} resource of placing v to n is denoted by dv→n,k (i). Similarly, the weighted communication bandwidth cost of assigning e to l is denoted by be→l (i). This edge cost is also defined for a dummy link l = (n, n), namely a non-existing link that connects the same node, to take into account the additional cost when placing two application nodes on one physical node. It is also worth noting that an application edge may be placed onto multiple physical links that form a path. Remark: The cost of placing an application node (or edge) onto different physical nodes (or edges) can be different. This is partly because different physical nodes and edges may have different resource capacities, and therefore different weighting factors for cost computation. It can also be due to the domain and conflict constraints as mentioned earlier. If some mapping is not allowed, then we can set the corresponding mapping cost to infinity. Hence, our cost definitions allow us to model a wide range of access-control/security policies. Mapping: A mapping is specified by π : V → N . Because we consider tree physical graphs with the cycle-free restriction, there exists only one path between two nodes in the physical graph, and we use (n1 , n2 ) to denote either the link or path between nodes n1 and n2 . We use the notation l ∈ (n1 , n2 ) to denote that link l is included in path (n1 , n2 ). The placement of nodes automatically determines the placement of edges. In a successive placement of the 1st up to the ith service, each physical node n ∈ N has an aggregated weighted cost of pn,k (i) = i X X dv→n,k (j), (3.1) j=1 v:π(v)=n where the second sum is over all v that are mapped to n. Equation (3.1) gives the 3.2. Problem Formulation 32 total cost of type k resource requested by all application nodes that are placed on node n, upto the ith service. Similarly, each physical edge l ∈ L has an aggregated weighted cost of ql (i) = i X X be→l (j), (3.2) j=1 e=(v1 ,v2 ):(π(v1 ),π(v2 ))3l where the second sum is over all application edges e = (v1 , v2 ) for which the path between the physical nodes π(v1 ) and π(v2 ) (which v1 and v2 are respectively mapped to) includes the link l. 3.2.2 Objective Function The optimization objective in this chapter is load balancing for which the objective function is defined as min max max pn,k (M ); max ql (M ) , π k,n l (3.3) where M is the total number of services (application graphs). The function in (3.3) aims to minimize the maximum weighted cost on each physical node and link, ensuring that no single element gets overloaded and becomes a point of failure, which is important especially in the presence of bursty traffic. Such an objective has been widely used in the literature [34, 35]. While we choose the objective function (3.3) in this chapter, we do realize that there can be other objectives as well, such as minimizing the total resource consumption. We note that the exact algorithm for the placement of a single linear application graph can be generalized to a wide class of other objective functions as will be discussed in Section 3.3.5, but for simplicity, we restrict our attention to the objective function in (3.3) in the following discussion. 3.3. Basic Assignment Unit: Single Linear Application Graph Placement A Note on Capacity Limit: 33 For simplicity, we do not impose capacity constraints on physical nodes and links in the optimization problem defined in (3.3), because even without the capacity constraint, the problem is very hard as we will see later in this chapter. However, because the resource demand of each application node and link is specified in every application graph, the total resource consumption at a particular physical node/link can be calculated by summing up the resource demands of application nodes/links that are placed on it. Therefore, an algorithm can easily check within polynomial time whether the current placement violates the capacity limits. If such a violation occurs, it can simply reject the newly arrived application graph. In most practical cases, the costs of node and link placements should be defined as proportional to the resource occupation when performing such placement, with weights inversely proportional to the capacity of the particular type of resource, such as the cost definitions in Section 2.2. With such a definition, the objective function (3.3) essentially tries to place as many application graphs as possible without increasing the maximum resource occupation (normalized by the resource capacity) among all physical nodes and links. Thus, the placement result should utilize the available resource reasonably well. A more rigorous analysis on the impact of capacity limit is left as future work. 3.3 Basic Assignment Unit: Single Linear Application Graph Placement We first consider the assignment of a single linear application graph (i.e., the application nodes are connected in a line), where the goal is to find the best placement of application nodes onto a path in the tree physical graph under the cycle-free constraint (as shown in Fig. 3.1). The solution to this problem forms the building block 3.3. Basic Assignment Unit: Single Linear Application Graph Placement 34 of other more sophisticated algorithms presented later. As discussed next, we develop an algorithm that can find the optimal solution to this problem. We omit the service index i in this section because we focus on a single service, i.e. M = 1, here. 3.3.1 Problem Formulation Without loss of generality, we assume that V and N are indexed sets, and we use v to exchangeably denote elements and indices of application nodes in V, and use n to exchangeably denote elements and indices of physical nodes in N . This index (starting from 1 for the root node) is determined by the topology of the graph. In particular, it can be determined via a breadth-first or depth-first indexing on the tree graph (note that linear graphs are a special type of tree graphs). From this it follows that, if n1 is a parent of n2 , then we must have n1 < n2 . The same holds for the application graph with nodes V. With this setting, the edge cost can be combined together with the cycle-free constraint into a single definition of pairwise costs. The weighted pairwise cost of placing v−1 to n1 and v to n2 is denoted by c(v−1,v)→(n1 ,n2 ) , and it takes the following values with v ≥ 2: 1. If the path from n1 to n2 traverses some n < n1 , in which case the cycle-free assumption is violated, then c(v−1,v)→(n1 ,n2 ) = ∞. 2. Otherwise, c(v−1,v)→(n1 ,n2 ) = max b(v−1,v)→l l∈(n1 ,n2 ) . (3.4) (π(v−1),π(v))3l The maximum operator in (3.4) follows from the fact that, in the single line placement, at most one application edge can be placed onto a physical link. Also recall that the edge cost definition incorporates dummy links such as l = (n, n), thus there always exists l ∈ (n1 , n2 ) even if n1 = n2 . 3.3. Basic Assignment Unit: Single Linear Application Graph Placement 35 Then, the optimization problem (3.3) with M = 1 becomes min max π X max k,n dv→n,k ; max c(v−1,v)→(π(v−1),π(v)) (v−1,v)∈E v:π(v)=n . (3.5) The last maximum operator in (3.5) takes the maximum among all application edges (rather than physical links), because when combined with the maximum in (3.4), it essentially computes the maximum among all physical links that are used for data transmission under the mapping π. 3.3.2 Decomposing the Objective Function In this subsection, we decompose the objective function in (3.5) to obtain an iterative solution. Note that the objective function (3.5) already incorporates all the constraints as discussed earlier. Hence, we only need to focus on the objective function itself. When only considering a subset of application nodes 1, 2, ..., v1 ≤ V , for a given mapping π, the value of the objective function for this subset of application nodes is Jπ (v1 ) = max max k,n X dv→n,k ; v≤v1 :π(v)=n max (v−1,v)∈E,v≤v1 c(v−1,v)→(π(v−1),π(v)) . (3.6) Compared with (3.5), the only difference in (3.6) is that we consider the first v1 application nodes and the mapping π is assumed to be given. The optimal cost for application nodes 1, 2, ..., v1 ≤ V is then Jπ∗ (v1 ) = min Jπ (v1 ), π where π ∗ denotes the optimal mapping. (3.7) 3.3. Basic Assignment Unit: Single Linear Application Graph Placement 36 Proposition 3.1. (Decomposition of the Objective Function): Let Jπ∗ |π(v1 ) (v1 ) denote the optimal cost under the condition that π(v1 ) is given, i.e. Jπ∗ |π(v1 ) (v1 ) = minπ(1),...,π(v1 −1) Jπ (v1 ) with given π(v1 ). When π(v1 ) = π(v1 − 1) = ... = π(v s ) > π(v s − 1) ≥ π(v s − 2) ≥ ... ≥ π(1), where 1 ≤ vs ≤ v1 , which means that v s is mapped to a different physical node from v s − 1 and nodes vs , ..., v1 are mapped onto the same physical node3 , then we have Jπ∗ |π(v1 ) (v1 )= min min max Jπ∗ |π(vs −1) (vs − 1); v s =1,...,v1 π(vs −1) X max dv→π(v1 ),k ; k=1,...,K v=v s ...v1 max (v−1,v)∈E,vs ≤v≤v1 c(v−1,v)→(π(v−1),π(v)) . (3.8) The optimal mapping for v1 can be found by Jπ∗ (v1 ) = min Jπ∗ |π(v1 ) (v1 ). (3.9) π(v1 ) Proof. Because π(vs ) = π(vs + 1) = ... = π(v1 ), we have ( Jπ (v1 ) = max Jπ (vs − 1); max k=1,...,K max (v−1,v)∈E,vs ≤v≤v1 X dv→π(v1 ),k ; v=v s ...v1 c(v−1,v)→(π(v−1),π(v)) . (3.10) The three terms in the maximum operation in (3.10) respectively correspond to: 1) the cost at physical nodes and edges that the application nodes 1, ..., vs − 1 (and their connecting edges) are mapped to, 2) the costs at the physical node that vs , ..., v1 are mapped to, and 3) the pairwise costs for connecting vs − 1 and vs as well as 3 Note that when vs = 1, then vs − 1 does not exist, which means that all nodes 1, ..., v1 are placed onto the same physical node. For convenience, we define Jπ (0) = 0. 3.3. Basic Assignment Unit: Single Linear Application Graph Placement 37 interconnections4 of nodes in vs , ..., v1 . Taking the maximum of these three terms, we obtain the cost function in (3.6). In the following, we focus on finding the optimal mapping based on the cost decomposition in (3.10). We note that the pairwise cost between vs − 1 and vs depends on the placements of both vs − 1 and vs . Therefore, in order to find the optimal Jπ (v1 ) from Jπ (vs −1), we need to find the minimum cost among all possible placements of vs − 1 and vs , provided that nodes vs , ..., v1 are mapped onto the same physical node and vs and vs − 1 are mapped onto different physical nodes. For a given v1 , node vs may be any node that satisfies 1 ≤ vs ≤ v1 . Therefore, we also need to search through all possible values of vs . This can then be expressed as the proposition, where we first find Jπ∗ |π(v1 ) (v1 ) as an intermediate step. Equation (3.8) is the Bellman’s equation [36] for problem (3.5). Using dynamic programming [36], we can solve (3.5) by iteratively solving (3.8). In each iteration, the algorithm computes new costs Jπ∗ |π(v1 ) (v1 ) for all possible mappings π(v1 ), based on the previously computed costs Jπ∗ |π(v) (v) where v < v1 . For the final application node v1 = V , we use (3.9) to compute the final optimal cost Jπ∗ (V ) and its corresponding mapping π ∗ . 3.3.3 Optimal Algorithm The pseudocode of the exact optimal algorithm is shown in Algorithm 3.1. It computes V · N number of Jπ∗ |π(v)=n (v) values, and we take the minimum among no more than V · N values in (3.8). The terms in (3.8) include the sum or maximum of no more than V values and the maximum of K values. Because K is a con4 Note that, although vs , ..., v1 are mapped onto the same physical node, their pairwise costs may be non-zero if there exists additional cost when placing different application nodes onto the same physical node. In the extreme case where adjacent application nodes are not allowed to be placed onto the same physical node (i.e. conflict constraint), their pairwise cost when placing them on the same physical node becomes infinity. 3.3. Basic Assignment Unit: Single Linear Application Graph Placement 38 Algorithm 3.1 Placement of a linear application graph onto a tree physical graph 1: Given linear application graph R, tree physical graph Y 2: Given V × N × K matrix D, its entries represent the weighted type k node cost dv→n,k 3: Given (V − 1) × N × N matrix C, its entries represent the weighted pairwise cost c(v−1,v)→(n1 ,n2 ) 4: Define V × N matrix J to keep the costs Jπ∗ |π(v)=n (v) for each node (v, n) in the auxiliary graph 5: Define V × N × V matrix Π to keep the mapping corresponding to its cost Jπ∗ |π(v)=n (v) for each node (v, n) in the auxiliary graph 6: for v = 1...V do 7: for n = 1...N do 8: Compute Jπ∗ |π(v)=n (v) from (3.8), put the result into J and the corresponding mapping into Π 9: end for 10: end for 11: Compute Jπ∗ (V ) ← minn Jπ∗ |π(V )=n (V ) 12: return the final mapping result π ∗ and final optimal cost Jπ∗ (V ) stant in practical systems, we conclude that the time-complexity of this algorithm is O(V 3 N 2 ). The space-complexity of Algorithm 3.3 is O(V N (V + N )), which is related to the memory required for storing matrices D, C, J, and Π in the algorithm, where K is also regarded as a constant here. Also note that the optimality of the result from Algorithm 3.1 is subject to the cycle-free constraint, and the sequence of nodes is always preserved in each iteration. 3.3.4 Example To illustrate the procedure of the algorithm, we construct an auxiliary graph from the given application and physical graphs, as shown in Fig. 3.2. Each node (v1 , n1 ) in the auxiliary graph represents a possible placement of a particular application node, and is associated with the cost value Jπ∗ |π(v1 )=n1 (v1 ), where v1 is the application node index and n1 is the physical node index in the auxiliary graph. When computing the cost at a particular node, e.g. the cost Jπ∗ |π(4)=C (4) at node (4,C) in Fig. 3.2, the 3.3. Basic Assignment Unit: Single Linear Application Graph Placement 1 1,B 1,C 1,D 2,A 2,B 2,C 2,D 3,A 3,B 3,C 3,D 4,A 4,B 4,C 4,D 5,A 5,B 5,C 5,D A 2 3 1,A 39 B Map C 4 D Physical graph 5 Application graph Auxiliary graph Figure 3.2: Auxiliary graph and algorithm procedure for the placement of a linear application graph onto a tree physical graph. 1 A Map B (vs − 1) where the tuple (vs − algorithm starts from2 the “earlier” costs Jπ∗ |π(vs −1) C (3,A), or (3,B). From each of these 4 1, π(vs − 1)) is either (1,A), (1,B), (2,A), (2,B), 3 6 nodes, the subsequent application nodes (i.e.D from vs to node 4) are all mapped onto 5 E F such “path” with the maximum physical node 7 C,8and we compute the cost for each Physical graph the values operations in Application (3.8), by assuming of vsgraph − 1 and π(vs − 1) are given by its originating node in the auxiliary graph. For example, one path can be (2,B) – (3,C) – (4,C) where vs − 1 = 2 and π(vs − 1) = B, another path can be (1,A) – (2,C) – (3,C) – (4,C) where vs − 1 = 1 and π(vs − 1) = A. Then, the algorithm takes the minimum of the costs for all paths, which corresponds to the minimum operations in (3.8) and gives Jπ∗ |π(4)=C (4). In the end, the algorithm searches through all the possible mappings for the final application node (node 5 in Fig. 3.2) and chooses the mapping that results in the minimum cost, which corresponds to the procedure in (3.9). 3.3.5 Extensions The placement algorithm for single linear application graph can also be used when the objective function (in the form of (3.3) with M = 1) is modified to one of the 3.4. Online Placement Algorithms for Tree Application Graphs 40 following: min max π X max fn,k k,n dv→n,k ; max gl l v:π(v)=n X e=(v1 ,v2 ):(π(v1 ),π(v2 ))3l be→l (3.11) or min π X k,n fn,k X v:π(v)=n dv→n,k + X l gl X be→l , (3.12) e=(v1 ,v2 ):(π(v1 ),π(v2 ))3l where fn,k (·) and gl (·) are increasing functions with fn,k (0) = 0, gl (0) = 0, fn,k (∞) = ∞, and gl (∞) = ∞. The algorithm and its derivation follow the same procedure as discussed above. These alternative objective functions can be useful for scenarios where the goal of optimization is other than min-max. The objective function in (3.12) will also be used later for solving the online placement problem. 3.4 Online Placement Algorithms for Tree Application Graphs Using the optimal algorithm for the single linear application graph placement as a sub-routine, we now present algorithms for the generalized case; namely, placement of an arriving stream of application graphs with tree topology. We first show that even the offline placement of a single tree is NP-hard. Then, we propose online algorithms to approximate the optimal placement with provable competitive ratio, by first considering the case where junction nodes in the application graph have prespecified placements that are given beforehand, and later relax this assumption. 3.4. Online Placement Algorithms for Tree Application Graphs 3.4.1 41 Hardness Result Proposition 3.2. (NP-hardness) Placement of a tree application graph onto a tree physical graph for the objective function defined in (3.3), with or without pre-specified junction node placement, is NP-hard. Proof. To show that the given problem is NP-hard, we show that the problem can be reduced from the NP-hard problem of minimum makespan scheduling on unrelated parallel machines (MMSUPM) [23], which minimizes the maximum load (or job processing time) on each machine. Consider a special case in our problem where the application graph has a star topology with two levels (one root and multiple leaf nodes), and the physical graph is a line with multiple nodes. Assume that the number of resource types in the nodes is K = 1, the application edge resource demand is zero, and the application node resource demand is non-zero. Then, the problem is essentially the MMSUPM problem. It follows that the MMSUPM problem reduces to our problem. In other words, if we can solve our problem in polynomial time, then we can also solve the MMSUPM problem in polynomial time. Because MMSUPM is NP-hard, our problem is also NP-hard. The above result holds no matter whether the root node (junction node) of the application graph has pre-specified placement or not. Note that although the objective functions (3.3) and (2.3) are slightly different, the above proof essentially shows that the problem in (2.3) is also NP-hard. This is because when setting application edge resource demands to zero as done in the proof, the problems (3.3) and (2.3) become the same. 3.4.2 When All Junction Node Placements Are Given We consider tree application graphs for which the placements of junction nodes are given, and focus on placing the remaining non-junction nodes which are connected C 4 D Physical graph 5 3,A 3,B 3,C 3,D 4,A 4,B 4,C 4,D 5,A 5,B 5,C 5,D Application graph Auxiliary graph 3.4. Online Placement Algorithms for Tree Application Graphs 1 A Map B 2 3 7 C 4 D 6 5 42 8 Application graph E F Physical graph Figure 3.3: Example of application graph with given placement of junction nodes. Junction node 2 is placed on physical node B and junction node 5 is placed on physical node E. The algorithm needs to decide the placement of the remaining nodes, subject to the cycle-free constraint. to at most two edges. An example is shown in Fig. 3.3. Given the placed junction nodes, we name the set of application edges and nodes that form a chain between the placed nodes (excluding each placed node itself, but including each edge that is connected to a placed node) as a simple branch, where the notion “simple” is opposed to the general branch which will be defined in Section 3.4.3. A simple branch can also be a chain starting from an edge that connects a placed node and ending at a leaf node, such as the nodes and edges within the dashed boundary in the application graph in Fig. 3.3. Each node in a simple branch is connected to at most two edges. 3.4.2.1 Algorithm Design In the following, we propose an online placement algorithm, where we make use of some ideas from [37]. Different from [37] which focused on routing and job scheduling problems, our work considers more general graph mapping. When a service (represented by a tree application graph) arrives, we split the whole application graph into simple branches, and regard each simple branch as an independent application graph. All the nodes with given placement can also be regarded as an application/service that is placed before placing the individual simple 3.4. Online Placement Algorithms for Tree Application Graphs 43 branches. After placing those nodes, each individual simple branch is placed using the online algorithm that we describe below. In the remaining of this section, by service we refer to the service after splitting, i.e. each service either consists of a simple branch or a set of nodes with given placement. How to Place Each Simple Branch: While our ultimate goal is to optimize (3.3), we use an alternative objective function to determine the placement of each newly arrived service i (after splitting). Such an indirect approach provides performance guarantee with respect to (3.3) in the long run. We will first introduce the new objective function and then discuss its relationship with the original objective function (3.3). We first define a variable Jˆ as a reference cost. The reference cost may be an estimate of the true optimal cost (defined as in (3.3)) from the optimal offline placement, and the method to determine this value will be discussed later. Then, for placing the ith service, we use an objective function which has the same form as (3.12), with fn,k (·) and gl (·) defined as pn,k (i − 1) + x pn,k (i − 1) fn,k (x) , expα − expα , ˆ ˆ J J pl (i − 1) + x pl (i − 1) gl (x) , expα − expα , Jˆ Jˆ (3.13a) (3.13b) subject to the cycle-free placement constraint, where we define expα (y) , αy and α , 1 + 1/γ, in which γ > 1 is a design parameter. Why We Use an Alternative Objective Function: The objective function (3.12) with (3.13a) and (3.13b) is the increment of the sum exponential values of the original costs, given all the previous placements. With this objective function, the performance bound of the algorithm can be shown analytically (see Proposition 3.3 below). Intuitively, the new objective function (3.12) serves the following purposes: 3.4. Online Placement Algorithms for Tree Application Graphs 44 1. “Guide” the system into a state such that the maximum cost among physical links and nodes is not too high, thus approximating the original objective function (3.3). This is because when the existing cost at a physical link or node (for a particular resource type k) is high, the incremental cost (following (3.12)) of placing the new service i on this link or node (for the same resource type k) is also high, due to the fact that expα (y) is convex increasing and the cost definitions in (3.13a) and (3.13b). 2. While (3.3) only considers the maximum cost, (3.12) is also related to the sum cost, because we sum up all the exponential cost values at different physical nodes and links together. This “encourages” a low resource consuming placement of the new service i (which is reflected by low sum cost), thus leaving more available resources for future services. In contrast, if we use (3.3) directly for each newly arrived service, the placement may greedily take up too much resource, so that future services can no longer be placed with a low cost. In practice, we envision that objective functions with a shape similar to (3.12) can also serve our purpose. How to Solve It: Because each service either obeys a pre-specified placement or consists of a simple branch, we can use Algorithm 3.1 with appropriately modified cost functions to find the optimal solution to (3.12) with (3.13a) and (3.13b). For the case of a simple branch having an open edge, such as edge (2, 4) in Fig. 3.3, we connect an application node that has zero resource demand to extend the simple branch to a graph, so that Algorithm 3.1 is applicable. Algorithm 3.2 summarizes the above argument as a formal algorithm for each service placement, where πi denotes the mapping for the ith service. Define a pa K+L) rameter, β = logα γ(Nγ−1 , then Algorithm 3.2 performs the placement as long ˆ otherwise it returns FAIL. as the cost on each node and link is not bigger than β J, The significance of the parameter β is in calculating the competitive ratio, i.e. the 3.4. Online Placement Algorithms for Tree Application Graphs 45 Algorithm 3.2 Online placement of a service that is either a simple branch or a set of nodes with given placement 1: Given the ith service that is either a set of nodes with given placement or a simple branch 2: Given tree physical graph Y 3: Given pn,k (i − 1), q l (i − 1), and placement costs 4: Given Jˆ and β 5: if service is a set of nodes with given placement then 6: Obtain πi based on given placement 7: else if service is a simple branch then 8: Extend simple branch to linear graph R(i), by connecting zero-resourcedemand application nodes to open edges, and the placements of these zeroresource-demand application nodes are given 9: Run Algorithm 3.1 with objective function (3.12) with (3.13a) and (3.13b), for R(i), to obtain πi 10: end if P ˆ 11: if ∃n, k : pn,k (i − 1) + v:πi (v)=n dv→n,k (i) > β J or ∃l : ql (i − 1) + P ˆ e=(v1 ,v2 ):(πi (v1 ),πi (v2 ))3l be→l (i) > β J then 12: return FAIL 13: else 14: return πi 15: end if maximum ratio of the cost resulting from Algorithm 3.2 to the optimal cost from an equivalent offline placement, as shown in Proposition 3.3. ˆ The reference cost Jˆ is an input parameWhy We Need the Reference Cost J: ter of the objective function (3.12) and Algorithm 3.2, which enables us to show a performance bound for Algorithm 3.2, as shown in Proposition 3.3. Proposition 3.3. If there exists an offline mapping π o that considers all M appliˆ then Algorithm 3.2 never cation graphs and brings cost Jπo , such that Jπo ≤ J, ˆ The cost Jπo is fails, i.e., pn,k (M ) and ql (M ) from Algorithm 3.2 never exceeds β J. defined in (3.3). Proof. See Appendix C.1. Proposition 3.3 guarantees a bound for the cost resulting from Algorithm 3.2. We note that the optimal offline mapping π o∗ produces cost Jπo∗ , which is smaller 3.4. Online Placement Algorithms for Tree Application Graphs 46 than or equal to the cost of an arbitrary offline mapping. It follows that for any π o , ˆ then we we have Jπo∗ ≤ Jπo . This means that if there exists π o such that Jπo ≤ J, ˆ If we can set Jˆ = Jπo∗ , then from Proposition 3.3 we have must have Jπo∗ ≤ J. max {maxk,n pn,k (M ); maxl ql (M )} ≤ βJπo∗ , which means that the competitive ratio is β. ˆ How to Determine the Reference Cost J: Because the value of Jπo∗ is unknown, we cannot always set Jˆ exactly to Jπo∗ . Instead, we need to set Jˆ to an estimated value that is not too far from Jπo∗ . We achieve this by using the doubling technique, which is widely used in online approximation algorithms. The idea is to double the value of Jˆ every time Algorithm 3.2 fails. After each doubling, we ignore all the previous placements when calculating the objective function (3.12) with (3.13a) and (3.13b), i.e. we assume that there is no existing service, and we place the subsequent ˆ with the new services (including the one that has failed with previous value of J) ˆ At initialization, the value of Jˆ is set to a reasonably small number Jˆ0 . value of J. In Algorithm 3.3, we summarize the high-level procedure that includes the splitting of the application graph, the calling of Algorithm 3.2, and the doubling process, with multiple application graphs that arrive over time. 3.4.2.2 Complexity and Competitive Ratio In the following, we discuss the complexity and competitive ratio of Algorithm 3.3. Because the value of Jπo∗ is finite5 , the doubling procedure in Algorithm 3.3 only contains finite steps. The remaining part of the algorithm mainly consists of calling Algorithm 3.2 which then calls Algorithm 3.1 for each simple branch. Because nodes 5 The value of Jπo∗ is finite unless the placement cost specification does not allow any placement with finite cost. We do not consider this case here because it means that the placement is not realizable under the said constraints. In practice, the algorithm can simply reject such application graphs when the mapping cost resulting from Algorithm 3.2 is infinity, regardless of what value of Jˆ has been chosen. 3.4. Online Placement Algorithms for Tree Application Graphs 47 Algorithm 3.3 High-level procedure for multiple arriving tree application graphs 1: Initialize Jˆ ← Jˆ0 2: Define index i as the service index, which automatically increases by 1 for each new service (after splitting) 3: Initialize i ← 1 4: Initialize i0 ← 1 5: loop 6: if new application graph has arrived then 7: Split the application graph into simple branches and a set of nodes with given placement, assume that each of them constitute a service 8: for all service i do 9: repeat 10: Call Algorithm 3.2 for service i with pn,k (i − 1) = max {0, pn,k (i − 1) − pn,k (i0 − 1)} and q l (i − 1) = max {0, q l (i − 1) − q l (i0 − 1)} 11: if Algorithm 3.2 returns FAIL then 12: Set Jˆ ← 2Jˆ 13: Set i0 ← i 14: end if 15: until Algorithm 3.2 does not return FAIL 16: Map service i according to πi resulting from Algorithm 3.2 17: end for 18: end if 19: end loop 3.4. Online Placement Algorithms for Tree Application Graphs 48 and links in each simple branch together with the set of nodes with given placement add up to the whole application graph, similar to Algorithm 3.1, the time-complexity of Algorithm 3.3 is O(V 3 N 2 ) for each application graph arrival. Similarly, when combining the procedures in Algorithms 3.1–3.3, we can see that the space-complexity of Algorithm 3.3 is O(V N (V + N )) for each application graph arrival, which is in the same order as Algorithm 3.1. For the competitive ratio, we have the following result. Proposition 3.4. (Competitive Ratio): Algorithm 3.3 is 4β = 4 logα γ(N K+L) γ−1 - competitive. Proof. If Algorithm 3.2 fails, then we know that Jπo∗ > Jˆ according to Proposition 3.3. Hence, by doubling the value of Jˆ each time Algorithm 3.2 fails, we have Jˆf < 2Jπo∗ , where Jˆf is the final value of Jˆ after placing all M services. Because we ignore all previous placements and only consider the services i0 , ..., i for a particular ˆ it follows that value of J, max max{pn,k (i) − pn,k (i0 − 1)}; k,n o max{ql (i) − ql (i0 − 1)} ≤ β Jˆ l (3.14) ˆ for the particular value of J. When we consider all the placements of M services, by summing up (3.14) for ˆ we have all values of J, max max pn,k (M ); max ql (M ) k,n l 1 1 1 ≤ 1 + + + + · · · β Jˆf 2 4 8 1 1 1 < 2 1 + + + + · · · βJπo∗ 2 4 8 = 4βJπo∗ . 3.4. Online Placement Algorithms for Tree Application Graphs 49 Hence, the proposition follows. The variables α, γ and K are constants, and L = N − 1 because the physical graph is a tree. Hence, the competitive ratio of Algorithm 3.3 can also be written as O(log N ). It is also worth noting that, for each application graph, we may have different tree physical graphs that are extracted from a general physical graph, and the above conclusions still hold. 3.4.3 When at Least One Junction Node Placement Is Not Given In this subsection, we focus on cases where the placements of some or all junction nodes are not given. For such scenarios, we first extend our concept of branches to incorporate some unplaced junction nodes. The basic idea is that each general branch is the largest subset of nodes and edges that are interconnected with each other not including any of the nodes with pre-specified placement, but (as with our previous definition of simple branches) the subset includes the edges connected to placed nodes. A simple branch (see definition in Section 3.4.2) is always a general branch, but a general branch may or may not be a simple branch. Examples of general branches are shown in Fig. 3.4. 3.4.3.1 Algorithm Design The main idea behind the algorithm is to combine Algorithm 3.2 with the enumeration of possible placements of unplaced junction nodes. When there is only a constant number of such nodes on any path from the root to a leaf, the algorithm remains polynomial in time-complexity while guaranteeing a poly-log competitive ratio. To illustrate the intuition, consider the example application graph shown in Fig. 3.4(a), where nodes 2 and 5 are both initially unplaced. We follow a hierarchical 3.4. Online Placement Algorithms for Tree Application Graphs 1 1 1 2 2 2 4 3 6 5 7 (a) (b) 6 5 7 8 4 3 6 5 7 8 4 3 50 9 8 10 (c) Maximum resource utilization Figure 3.4: Example of application graphs with some unplaced junction nodes, the nodes and form a general branch: (a) nodes 2 0.9 0.9 edges within each dashed boundary and 5 are0.8both unplaced, (b) node 2 is 0.8 placed, node 5 is unplaced, (c) node 2 is 0.7 0.7 placed, nodes 5 and 6 are unplaced. 0.6 0.6 0.5 0.5 determination of the placement of unplaced nodes starting with the nodes in the 0.4 0.4 0.3 0.3 Proposed 0.2 deepest level. For theProposed example in Fig. 3.4(a), we first determine the placement of MILP w. min-max obj. 0.2 MILP w. min-max obj. MILP w. min-sum obj. 0.1 0.1 MILP w. min-sum obj. 0 node 5, given each possible placement of 0node 2; and then determine the placement 0 10 20 30 40 50 0.005 0.015 0.025 Number of nodes in physical graph Maximum resource demand of node 2. Recall that we use the cost function in (3.12) with (3.13a) and (3.13b) to determine the placement of each simple branch when all the junction nodes are Maximum resource utilization placed. 0.9 We use the same cost function 0.9 (with slightly modified parameters) for the 0.8 0.8 0.7 0.7 placement of nodes 2 and 5. However, when determining the placement of node 5, 0.6 0.6 0.5 the general branch that includes 0.5 node 5 (which contains nodes 3, 5, 7, we regard 0.4 0.4 0.3 0.3 and 8 and the corresponding edges as shown in Fig. Proposed 3.4(b)) as one single service, Proposed 0.2 0.2 MILP w. min-max obj. MILP w. min-max obj. 0.1 MILP w. min-sum obj. i.e. the 0.1 values of pn,kMILP (i −w.1)min-sum and qobj. l (i − 1) in (3.13a) and (3.13b) correspond to the 0 0 0 10 20 30 40 50 0.005 0.015 0.025 resource utilization at nodes and links before placing this whole general branch, and Maximum resource demand Number of nodes in physical graph the service i contains all the nodes and edges in this general branch. Similarly, when determining the placement of node 2, we consider the whole application graph as a single service. It is worth noting that in many cases we may not need to enumerate all the possible combinations of the placement of unplaced junction nodes. For example, in Fig. 3.4(c), when the placement of node 2 is given, the placement of nodes 5 and 6 does not impose additional restrictions upon each other (i.e., the placement of node 5 does 3.4. Online Placement Algorithms for Tree Application Graphs 51 not affect where node 6 can be placed, for instance). Hence, the general branches that respectively include node 5 and node 6 can be placed in a subsequent order using the online algorithm. Based on the above examples, we summarize the procedure as Algorithm 3.4, where we solve the problem recursively and determine the placement of one junction node that has not been placed before in each instance of the function Unplaced(v, h). The parameter v is initially set to the top-most unplaced junction node (node 2 in Fig. 3.4(a)), and h is initially set to H (the maximum number of unplaced junction nodes on any path from the root to a leaf in the application graph). Algorithm 3.4 can be embedded into Algorithm 3.3 to handle multiple arriving ˆ The only part that needs to be application graphs and unknown reference cost J. modified in Algorithm 3.3 is that it now splits the whole application graph into general branches (rather than simple branches without unplaced junction nodes), and it either calls Algorithm 3.2 or Algorithm 3.4 depending on whether there are unplaced junction nodes in the corresponding general branch. When there are such nodes, it calls Unplaced(v, h) with the aforementioned initialization parameters. 3.4.3.2 Complexity and Competitive Ratio The time-complexity of Algorithm 3.4 together with its high-level procedure that is a modified version of Algorithm 3.3 is O(V 3 N 2+H ) for each application graph arrival, as explained below. Note that H is generally not the total number of unplaced nodes. Obviously, when H = 0, the time-complexity is the same as the case where all junction nodes are placed beforehand. When there is only one unplaced junction node (in which case H = 1), Algorithm 3.4 considers all possible placements for this vertex, which has at most N choices. Hence, its time-complexity becomes N times the time-complexity with all placed junction nodes. When there are multiple unplaced junction nodes, we can see from Algorithm 3.4 that it only increases its re- 3.4. Online Placement Algorithms for Tree Application Graphs 52 Algorithm 3.4 Tree-to-tree placement when some junction nodes are not placed 1: function Unplaced(v, h) ˆ and β 2: Given the ith service that is a general branch, tree physical graph Y, J, 3: Given pn,k (i − 1) and q l (i − 1) which is the current resource utilization on nodes and links 4: Define Π to keep the currently obtained mappings, its entry π|π(v)=n0 for all n0 represents the mapping, given that v is mapped to n0 5: Define pn,k (i)|π(v)=n0 and q l (i)|π(v)=n0 for all n0 as the resource utilization after placing the ith service, given that v is mapped to n0 6: Initialize pn,k (i)|π(v)=n0 ← pn,k (i − 1) and q l (i)|π(v)=n0 ← q l (i − 1) for all n0 7: for all n0 that v can be mapped to do 8: Assume v is placed at n0 9: for all general branch that is connected with v do 10: if the general branch contains unplaced junction nodes then 11: Find the top-most unplaced vertex v 0 within this general branch 12: Call Unplaced(v 0 , h − 1) while assuming v is placed at n0 , and with pn,k (i − 1) = pn,k (i)|π(v)=n0 and q l (i − 1) = q l (i)|π(v)=n0 13: else 14: (in which case the general branch is a simple branch without unplaced junction nodes) Run Algorithm 3.2 for this branch 15: end if 16: Put mappings resulting from Unplaced(v 0 , h − 1) or Algorithm 3.2 into π|π(v)=n0 17: Update pn,k (i)|π(v)=n0 and q l (i)|π(v)=n0 to incorporate new mappings 18: end for 19: end for P p (i)| pn,k (i−1) 0 −exp + 20: Find minn0 k,n expα n,k β hπ(v)=n α Jˆ β h Jˆ P q l (i)|π(v)=n0 − expα qlβ(i−1) , h Jˆ l expα β h Jˆ ∗ returning the optimal placement of v as n0 . ˆ 21: if h = H and (∃n, k : pn,k (i)|π(v)=n∗0 > β 1+H Jˆ or ∃l : q l (i)|π(v)=n∗0 > β 1+H J) then 22: return FAIL 23: else 24: return π|π(v)=n∗0 25: end if 3.4. Online Placement Algorithms for Tree Application Graphs 53 cursion depth when some lower level unplaced junction nodes exist. In other words, parallel general branches (such as the two general branches that respectively include node 5 and node 6 in Fig. 3.4(c)) do not increase the recursion depth, because the function Unplaced(v, h) for these general branches is called in a sequential order. Therefore, the time-complexity depends on the maximum recursion depth which is H; thus, the overall time-complexity is O(V 3 N 2+H ). The space-complexity of Algorithm 3.4 is O(V N 1+H (V + N )) for each application graph arrival, because in every recursion, the results for all possible placements of v are stored, and there are at most N such placements for each junction node. Regarding the competitive ratio, similar to Proposition 3.3, we can obtain the following result. Proposition 3.5. If there exists an offline mapping π o that considers all M applicaˆ then Algorithm 3.4 never fails, tion graphs and brings cost Jπo , such that Jπo ≤ J, ˆ i.e., pn,k (M ) and ql (M ) resulting from Algorithm 3.4 never exceeds β 1+H J. Proof. When H = 0, the claim is the same as Proposition 3.3. When H = 1, there is at most one unplaced junction node in each general branch. Because Algorithm 3.4 operates on each general branch, we can regard that we have only one unplaced junction node when running Algorithm 3.4. In this case, there is no recursive calling of Unplaced(v, h). Recall that v is the top-most unplaced junction node. The function Unplaced(v, h) first fixes the placement of this unplaced junction node v to a particular physical node n0 , and finds the placement of the remaining nodes excluding v. It then finds the placement of v. From Proposition 3.3, we know that when we fix the placement of v, the cost resulting from the algorithm never exceeds β Jˆ if there exists a mapping π o |π(v)=n0 (under the constraint that v is placed at n0 ) that brings cost Jπo |π(v)=n0 with Jπo |π(v)=n0 ≤ ˆ J. To find the placement of v, Algorithm 3.4 finds the minimum cost placement 3.5. Numerical Evaluation 54 from the set of placements that have been obtained when the placement of v is given. ˆ we Reapplying Proposition 3.3 for the placement of v, by substituting Jˆ with β J, ˆ provided that there exists a know that the cost from the algorithm never exceeds β 2 J, mapping, which is within the set of mappings produced by the algorithm with given ˆ Such a mapping exists and can v placements6 , that has a cost not exceeding β J. be produced by the algorithm if there exists an offline mapping π o (thus a mapping ˆ Hence, π o |π(v)=n0 for a particular placement of v) that brings cost Jπo with Jπo ≤ J. the claim follows for H = 1. When H > 1, because we decrease the value of h by one every time we recursively call Unplaced(v, h), the same propagation principle of the bound applies as for the case with H = 1. Hence, the claim follows. Using the same reasoning as for Proposition 3.4, it follows that Algorithm 3.4 in combination with the extended version of Algorithm 3.3 1+H γ(N K+L) 1+H -competitive, thus its competitive ratio is is 4β = 4 logα γ−1 O(log1+H N ) . 3.5 Numerical Evaluation In this section, we evaluate the performance of the proposed algorithm via simulation. For comparison, we consider online algorithms that utilize a mixed-integer linear program (MILP) solution. Specifically, a MILP optimization is solved for each new service arrival, and the service is placed according to the solution from the 6 Note that, as shown in Line 20 of Algorithm 3.4, to determine the placement of v, we only take the minimum cost (expressed as the difference of exponential functions) with respect to those mappings that were obtained with given placement of v. It follows that the minimization is only taken among a subset of all the possible mappings. This restricts the reference mapping to be within the set of mappings that the minimization operator operates on. Because, only in this way, the inequality (C.5) in the proof of Proposition 3.3 can be satisfied. On the contrary, Algorithm 3.2 considers all possible mappings that a particular simple branch can be mapped to, by calling Algorithm 3.1 as its subroutine. 3.5. Numerical Evaluation 55 MILP. By service, we refer to non-split services, i.e. whole application graphs, in this section. MILPs are generally not solvable in polynomial-time, but we use them for comparison to capture the best possible result for a larger class of online algorithms (including LP-relaxation or other heuristics) that place each service upon its arrival. Also note that these MILP results do not represent the optimal offline solution, because an optimal offline solution needs to consider all services at the same time, whereas the methods that we use for comparison only solve the MILP for each newly arrived service. Obtaining the optimal offline solution would require excessive computational time, hence we do not consider it here. We use the optimization toolbox CPLEX [20] to solve the MILPs in the simulation. Recall that the goal of the proposed algorithm is to minimize the maximum resource utilization on nodes and links after hosting a number of services, as given in (3.3), and the (non-linear) exponential-difference cost function in (3.12) with (3.13a) and (3.13b) is used to determine the placement of each newly arrived service. In the methods for comparison, we also consider (3.3) as the “long-term” goal, but we consider two different MILP formulations with different objective functions for placing each newly arrived service. The first formulation greedily minimizes the maximum resource utilization for each service placement, with a min-max objective function. Such a formulation represents a general class of greedy mechanisms. The second formulation has a min-sum objective function as defined in [17]. The coefficients in the cost function are inversely proportional to the amount of available resource. Hence, this method also considers load balancing in its formulation. We do not impose the cycle-free constraint in the MILP formulations. When the placements of junction nodes are given, the children of this junction node can only be placed onto the physical node, on which the junction node has been placed, or onto the children of this physical node. This is considering that a placed junction node may be 3.5. Numerical Evaluation 56 responsible for nodes in a specific area. We consider randomly generated tree application and physical graphs7 . The number of application nodes for each service is randomly chosen from the interval [3, 10], and the number of physical nodes ranges from 2 to 50. This setting is similar to that in related work such as [17]. We use a sequential approach to assign connections between nodes. Namely, we first label the nodes with indices. Then, we start from the lowest index, and connect each node m to those nodes that have indices 1, 2, ..., m − 1. Node m connects to node m − 1 with probability 0.7, and connects to nodes 1, 2, ..., m − 2 each with probability 0.3/(m − 2). We restrict the application root node to be placed onto the physical root node, which is due to the consideration that some portion of processing has to be performed on the core cloud in an MMC environment. We consider 100 service arrivals and simulate with 100 different random seeds to obtain the overall performance. The placement cost of a single node or link is uniformly distributed between 0 and a maximum value. For the root application node, the cost is divided by a factor of 10. We set the design parameter γ = 2 in the simulation. Figures 3.5 and 3.6 show the average maximum resource utilization (averaged over results from different random seeds8 ), respectively with and without pre-specified placement of junction nodes. In Figs. 3.5(a) and 3.6(a), the number of physical nodes is randomly chosen from the interval [2, 50]; and in Figs. 3.5(b) and 3.6(b), the maximum resource demand (per application node/link) is set to 0.015. It is evident that the proposed method outperforms the methods in comparison. The 7 Because the focus of our work is on MMCs, which have not been widely deployed in practice and thus realistic graphs are not available to us. Therefore, we use randomly generated graphs in our evaluation. 8 We only consider those random seeds which produce a maximum resource utilization that is smaller than one, because otherwise, the physical network is overloaded after hosting 100 services. We also observed in the simulations that the number of accepted services is similar when using different methods. The relative degradation in the number of accepted services of the proposed method compared with other methods never exceeds 2% in the simulations. 3 5 4 6 1 7 8 3.5. Numerical Evaluation 2 (a) 4 3 5 4 3 6 6 5 7 8 1 6 5 7 8 5 1 57 9 10 2 (c) 2 (b) 3 4 3 4 3 6 4 5 Maximum resource utilization Maximum resource utilization Maximum resource utilization Maximum resource utilization 0.9 0.9 9 70.8 8 7 7 8 80.8 0.7 0.7 (a) (c) 0.6(b) 0.6 0.5 0.5 0.4 0.4 0.3 0.3 Proposed Proposed 0.9 0.9 0.2 MILP w. min-max obj. 0.2 MILP w. min-max obj. 0.8 0.8 MILP w. min-sum obj. 0.1 0.1 MILP w. min-sum obj. 0.7 0.7 0 0 0.6 0.6 0 10 20 30 40 50 0.005 0.015 0.025 0.5 Number of nodes in physical graph 0.5 Maximum resource demand 0.4 0.4 Figure 3.5: Maximum resource utilization when placements are pre0.3 junction nodeProposed 0.3 Proposed specified. 0.2 MILP w. min-max obj. 0.2 MILP w. min-max obj. MILP w. min-sum obj. 0.1 MILP w. min-sum obj. 0.90.1 0.9 0 0 0.8 0.8 0 10 20 30 40 50 0.005 0.015 0.025 0.7 0.7 Number of nodes in physical graph Maximum resource demand 0.6 0.6 0.5 0.5 0.4 0.4 0.3 0.3 0.9 0.9 Proposed Proposed 0.2 0.2 0.8 MILP w. min-max obj. 0.8 MILP w. min-max obj. 0.1 0.1 MILP w. min-sum obj. 0.7 0.7 MILP w. min-sum obj. 0 0 0.6 0.6 10 20 30 40 50 0.005 0.015 0.025 0.5 0.5 0 Maximum resource demand Number of nodes in physical graph 0.4 0.4 0.3 0.3 Proposed Proposed 0.2 0.2 MILP w. min-max obj. MILP w. min-max obj. 0.1 0.1 MILP w. min-sum obj. MILP w. min-sum obj. 0 0 0 10 20 30 40 50 0.005 0.015 0.025 Maximum resource demand Number of nodes in physical graph Figure 3.6: Maximum resource utilization when junction node placements are not pre-specified. 6 10 3.5. Numerical Evaluation 58 resource utilization tends to converge when the number of physical nodes is large because of the fixed root placement. As mentioned earlier, the MILP solution is the best possible solution among a set of existing approaches for online placement, which use the same objective function for each service as in the corresponding MILP formulation. Realistic online algorithms in the same set would not perform better than the MILP result, whereas solving MILP requires non-polynomial time. The reason why the proposed method outperforms other methods is described below. We should first note that the uniqueness in the proposed algorithm is that it uses a non-linear objective function for each service, whereas the MILP methods all use linear objective functions (otherwise it is not possible to formulate with MILP, and obtaining exact solutions to non-linear programs with integer variables generally requires excessive computational time9 , thus we do not compare against exact solutions for the exponential-difference cost). The exponential-difference cost used in the proposed algorithm is indeed related to both load balancing and reducing sum resource utilization. It leaves more available resources for services that arrive in the future, compared to the MILP with min-max objective which greedily optimizes (3.3) for each new service. The MILP with min-sum objective function does not strongly enforce load balancing unless operating closely to the resource saturation point, because of the characteristics of its objective function. When comparing Fig. 3.5 with Fig. 3.6, we can find that the performance gaps between the proposed method and other methods are larger when the junction nodes are not placed beforehand. This is mainly because the judgment of whether Algorithm 3.4 has failed is based on the factor β 1+H , and for Algorithm 3.2 it is based on β. It follows that Algorithm 3.4 is less likely to fail when H > 0. In this case, the value of Jˆ is generally set to a smaller value by the doubling procedure in Algorithm 9 This is by noting that even linear programs with integer variables are generally NP-hard, and the best known algorithms for solving NP-hard problems have exponential time-complexity. A non-linear program is usually more difficult to solve than a linear program. 3.6. Discussion 59 3.3. A smaller value of Jˆ also results in a larger change in the exponential-difference cost when the amount of existing load changes10 . This brings a better load balancing on average (but not for the worst-case, the worst-case result is still bounded by the bounds derived earlier in this chapter). 3.6 Discussion Is the Tree Assumption Needed? For ease of presentation and considering the practical relevance to MMC applications, we have focused on tree-to-tree placements in this chapter. However, the tree assumption is not absolutely necessary for our algorithm to be applicable. For example, consider the placement problem shown in Fig. 3.7, where the application graph consists of two junction nodes (nodes 1 and 2) and multiple simple branches (respectively including nodes 3, 4, 5, and 6) between these two junction nodes. Such an application graph is common in applications where processing can be parallelized at some stage. The physical graph shown in Fig. 3.7(b) still has a hierarchy, but we now have connections between all pairs of nodes at two adjacent levels. Obviously, neither the application nor the physical graph in this problem has a tree structure. Let us assume that junction node 1 has to be placed at the top level of the physical graph (containing nodes A, B, C, D, E), junction node 2 has to be placed at the bottom level of the physical graph (containing nodes K, L, M, N, O), and application nodes 3, 4, 5, 6 have to be placed at the middle level of the physical graph (containing nodes F, G, H, I, J). One possible junction node placement under this restriction is shown in Fig. 3.7(c). With this pre-specified junction node placement, the mapping of each application node in {3, 4, 5, 6} can be found by the simple branch placement algorithm (Algorithm 3.3 which embeds Algorithm 3.2) introduced earlier, because This is except for the top-level instance of Unplaced(v, h) due to the division by β h in Line 20 of Algorithm 3.4. 10 3.6. Discussion 60 1 3 4 5 6 2 A B C D E F G H I J K L M N O (a) Application graph (b) Physical graph 1 A B C D E F G H I J K L M N O 2 (c) With pre‐specified placement Figure 3.7: Example where application and physical graphs are not trees: (a) application graph, (b) physical graph, (c) restricted physical graph with pre-specified placement of application nodes 1 and 2. 3.6. Discussion 61 it only needs to map each application node in {3, 4, 5, 6} onto each physical node in {F, G, H, I, J}, and find the particular assignment that minimizes (3.12) with (3.13a) and (3.13b). Therefore, in this example, when the junction node placements are prespecified, the proposed algorithm can find the placement of other application nodes with O(V 3 N 2 ) time-complexity, which is the complexity of Algorithm 3.3 as discussed in Section 3.4.2.2. When the junction node placements are not pre-specified, the proposed algorithm can find the placement of the whole application graph with O(V 3 N 4 ) time-complexity, because here H = 2 (recall that the complexity result was derived in Section 3.4.3.2). We envision that this example can be generalized to a class of application and physical graphs where there exist a limited number of junction nodes (where we can define an arbitrary node in the application graph as the root node) that are not placed beforehand. The algorithms proposed in this chapter should still be applicable to such cases, as long as we can find a limited number of cycle-free paths between two junction nodes when they are placed on the physical graph. We leave a detailed discussion to this aspect as future work. Practical Implications: Beside the proposed algorithms themselves, the results of this chapter can also provide the following insights that may guide future implementation: 1. The placement is easier when the junction nodes are placed beforehand. This is obvious when comparing the time-complexity and competitive ratio for cases with and without unplaced junction nodes. 2. There is a trade-off between instantaneously satisfying the objective function and leaving more available resources for future services. Leaving more available resources may cause the system to operate in a sub-optimal state for the short-term, but future services may benefit from it. This trade-off can be con- 3.7. Summary 62 trolled by defining an alternative objective function which is different from (but related to) the overall objective that the system tries to achieve (see Section 3.4.2.1). Comparison with [25]: As mentioned in Section 3.1, [25] is the only work which we know that has studied the competitive ratio of online service placement considering both node and link optimization. Our approach has several benefits compared to [25] as discussed in Section 3.1.2. Beside those benefits, we would like to note that the proposed algorithm outperforms [25] in time-complexity, space-complexity, and competitive ratio when the junction node placements are pre-specified (the performance bounds of these two approaches can be found in Sections 3.1.1 and 3.1.3, respectively). When some junction node placements are not pre-specified, our approach provides a performance bound comparable to that in [25], where we also note that H ≤ D. Moreover, [25] does not consider exact optimal solutions for the placement of a single linear application graph, and it also does not have simulations to show the average performance of the algorithm. Tightness of Competitive Ratio: By comparing the competitive ratio result of our approach to that in [25], we see that both approaches provide poly-log competitive ratios for the general case. It is however unclear whether this is the best performance bound one can achieve for the service placement problem. This is an interesting but difficult aspect worth studying in the future. 3.7 Summary We have focused on the placement of an incoming stream of application graphs onto a physical graph in this chapter. We started with a basic assignment module for placing a single application graph that is a line, based on which we developed online 3.7. Summary 63 algorithms for placing tree application graphs. The performance of the proposed algorithms has been studied both analytically and via simulation. Until now, we have proposed algorithms for initial service placement as discussed in Section 1.1.1. The next question we need to address is whether/where to migrate services in real time when mobile users move after the initial placement. This is discussed in the next two chapters. C HAPTER 4 An MDP-Based Approach to Dynamic Service Migration 4.1 Introduction We considered the initial placement of an arriving stream of application graphs onto the physical graph in the previous chapter. Because mobile users may move after initial placement (as discussed in Section 1.1.2) and such a movement essentially changes the physical graph topology, the initial placement may no longer be optimal or near-optimal after such a change. Thus, in this chapter, we consider the case where the cost of a particular choice of service placement varies over time, and this cost variation is mainly due to user mobility. In this case, we need to make decisions related to whether/where we should migrate the service. As illustrated in the example given in Section 1.1.2, application components running on MMCs are the most prone to user movements, due to the relatively small coverage area of network entities (such as basestations) MMCs are directly connected to. Thus, for simplicity, in this and remaining chapters we only consider a star (two-level tree) application graph with one root node running on an MMC and one or multiple leaf nodes running on users. When the application graph only has one leaf node, it corresponds to the case where the service only serves one user. When it has multiple leaf nodes, the service serves multiple users simultaneously. We will mainly focus on the first case where each user independently runs its own 4.1. Introduction 65 service instance on an MMC, but several approaches we present can be directly applied or extended to the second case where multiple users are served by one service instance. The service placement and migration problem with user dynamics mainly focuses on where (on which cloud) to place the root application node. We will soon see that this problem is already very complex even with these simplifications, where user dynamics is the main cause for complexity. 4.1.1 Related Work Most existing work on service migration focuses on wide-area migrations in response to workload and energy costs that vary slowly [38, 39]. In MMCs, user mobility is the driving factor of migration, which varies much faster than parameters in conventional clouds. The performance of MMCs with the presence of user mobility is studied in [18], but decisions on whether and where to migrate the service is not considered. A preliminary work on mobility-driven service migration based on Markov Decision Processes (MDPs) is given in [19], which mainly considers 1-D mobility patterns with a specifically defined cost function. Standard solution procedures are used to solve this MDP, which can be time-consuming especially when the MDP has a large number of states. Due to real-time dynamics, the cost functions and transition probabilities of the MDP may change rapidly over time, thus it is desirable to solve the MDP in an effective manner. Furthermore, 2-D mobility has not been considered in the literature to the best of our knowledge, which is a much more realistic case compared to 1-D mobility. We also note that a related area of work, which is relevant to the user mobility, studies handover policies in the context of cellular networks [40]. However, the notion of service migration is very different from cellular handover. Handover is usually decided by signal strengths from different basestations, and a handover must 4.1. Introduction 66 occur if a user’s signal is no longer provided satisfactorily by its original basestation. In the service migration context, a user may continue receiving service from an MMC even if it is no longer associated with that basestation, because the user can communicate with a remote MMC via its currently associated basestation and the backhaul network. As a result, the service for a particular user can potentially be placed on any MMC, and the service migration problem has a much larger decision space than the cellular handover problem. 4.1.2 Main Results In this chapter, we use the MDP framework to study service migration in MMCs. We provide novel contributions beyond [19], by considering more efficient solution methods, general cost models, 2-D user mobility, and application to real-world scenarios. The details are described as follows. 1. We formulate the mobility-driven dynamic service placement/migration problem with general cost models and provide a mathematical framework to design optimal service migration policies. 2. We consider 1-D user mobility1 with a constant cost model, and propose an optimal threshold policy to solve for the optimal action of the MDP, which is more efficient than standard solution techniques. A threshold policy means that we always migrate the service for a user from one micro-cloud to another when the user is in states bounded by a particular set of thresholds, and do not migrate otherwise. We first prove the existence of an optimal threshold policy and then propose an algorithm with polynomial time-complexity for finding the optimal thresholds. The analysis with 1-D mobility model can also help 1 The 1-D mobility is an important practical scenario often encountered in transportation networks, such as vehicles moving along a road. 4.1. Introduction 67 us gain new insights into the migration problem, which set the foundation for more complicated scenarios studied next. 3. We consider 2-D user mobility with a more general cost model, where the cost can be expressed in a constant-plus-exponential form. For this case, we propose the following: (a) We note that the resulting problem becomes difficult to solve due to the large state space. In order to overcome this challenge, we propose an approximation of the underlying state space where we define the states as the distance between the user and the service locations2 . This approximation becomes exact for uniform 1-D mobility. We prove several structural properties of the distance-based MDP, which includes a closedform solution to the discounted sum cost. We leverage these properties to develop an algorithm for computing the optimal policy, which reduces the per-iteration complexity from O(N 3 ) (by policy iteration [41, Chapter 6]) to O(N 2 ), where N is the number of states in the distance-based MDP excluding the state corresponding to zero distance. (b) We show how to use the distance-based MDP to approximate the solution for 2-D mobility models, which allows us to efficiently compute a service migration policy for 2-D mobility. For uniform 2-D mobility, the approximation error is bounded by a constant. Simulation results comparing our approximation solution to the optimal solution (where the optimal solution is obtained from a 2-D MDP) suggest that it performs very close to optimal, and the proposed approximation approach obtains the solution significantly faster. 2 Throughout this chapter, we mean by user location the location of the basestation that the user is associated to. 4.2. Problem Formulation 68 (c) We demonstrate how to apply our algorithms in a practical scenario driven by real mobility traces of taxis in San Francisco which involve multiple users and services. The practical scenario includes realistic factors, e.g., not every basestation has an MMC connected to it, and each MMC can only host a limited number of services. We compare the proposed policy with several baseline strategies that include myopic, never-migrate, and always-migrate policies. It is shown that the proposed approach offers significant gains over these baseline approaches. 4.2 Problem Formulation Consider a mobile user that accesses a cloud-based service hosted on the MMCs. The set of possible user locations is given by Q, where Q is assumed to be finite (but arbitrarily large). We consider a time-slotted model where the user’s location remains fixed for the duration of one slot and changes from one slot to the next according to a Markovian mobility model. The time-slotted model can be regarded as a sampled version of a continuous-time model, and the sampling can be performed either at equal intervals over time or occur right after a cellular handover instance. In addition, we assume that each location ϕ ∈ Q is associated with an MMC that can host the service for the user. The locations in Q are represented as 2-D vectors and there exists a distance metric kϕ1 − ϕ2 k that can be used to calculate the distance between locations ϕ1 and ϕ2 . Note that the distance metric may not be Euclidean distance. An example of this model is a cellular network in which the user’s location is taken as the location of its current basestation and the MMCs are co-located with the basestations. As shown in Section 4.4.3, these locations can be represented as 2-D vectors (i, j) with respect to a reference location (represented by (0, 0)) and the 4.2. Problem Formulation Observe Ob u(t) ((t)) and d h(t) () … Possible migration g changing h gi g h(t) ( ) to t h'(t) () 69 Operate p with states u(t) (t) and d hh'(t) (t) Ti Timeslot l tt We always have h(t+1) h(t+1)= ( ) h'(t) () … Time i Figure 4.1: Timing of the proposed service migration mechanism. distance between any two locations can be calculated in terms of the number of hops to reach from one cell to another cell. We denote the user and service locations at timeslot t as u(t) and h(t) respectively. Remark: The problem is formulated for the case of a single user accessing a single service, i.e., the application graph only contains two nodes, one running at one of the MMCs, and the other running at the user device. However, our solution can be applied to manage services for multiple users in a heuristic manner. We will illustrate such an application in Section 4.4.4, where we also consider aspects including that MMCs are only deployed at a subset of basestations and a limited number of services can be hosted at each MMC. We assume in this chapter that different services are independent of each other, and one service serves a single user. The notion of “service” in this chapter can also stand for an instance of a particular type of service, but we do not distinguish between services and instances in this chapter for simplicity. 4.2.1 Control Decisions and Costs At the beginning of each slot, the MMC controller can choose from one of the following control options: 1. Migrate the service from location h(t) to some other location h0 (t) ∈ Q. This incurs a migration cost cm (x) that is assumed to be a non-decreasing function 4.2. Problem Formulation 70 of x, where x is the distance between h(t) and h0 (t), i.e., x = kh(t) − h0 (t)k. Once the migration is completed, the system operates under state (u(t), h0 (t)). We assume that the time to perform migration is negligible compared to the time scale of user mobility (as shown in Fig. 4.1). 2. Do not migrate the service. In this case, we have h0 (t) = h(t) and the migration cost is cm (0) = 0. In addition to the migration cost, there is a data transmission cost incurred by the user for connecting to the currently active service instance. The transmission cost is related to the distance between the service and the user after possible migration, and it is defined as a general non-decreasing function cd (y), where y = ku(t) − h0 (t)k. We set cd (0) = 0. 4.2.2 Performance Objective Let us denote the overall system state at the beginning of each timeslot (before possible migration) by s(t) = (u(t), h(t)). The state s(t) is named as the initial state of slot t. Consider any policy π that makes control decisions based on the state s(t) of the system, and we use aπ (s(t)) to represent the control action taken when the system is in state s(t). This action causes the system to transition to a new intermediate state s0 (t) = (u(t), h0 (t)) = aπ (s(t)). We use Caπ (s(t)) to denote the sum of migration and transmission costs incurred by a control aπ (s(t)) in slot t, and we have Caπ (s(t)) = cm (kh(t) − h0 (t)k) + cd (ku(t) − h0 (t)k). Starting from any initial state s(0) = s0 (assuming that the current slot has index 0), the long-term expected discounted sum cost incurred by policy π is given by Vπ (s0 ) = lim E t→∞ ) γ τ Caπ (s(τ ))s(0) = s0 τ =0 ( t X (4.1) 4.2. Problem Formulation 71 where 0 < γ < 1 is a discount factor which is related to how far we look-ahead in the cost computation. A large value of γ gives more weight to future costs, whereas a small value of γ gives less weight to future costs. Our objective is to design a control policy that minimizes the long-term expected discounted sum total cost starting from any initial state, i.e., V ∗ (s0 ) = min Vπ (s0 ) ∀s0 . π (4.2) This problem falls within the class of MDPs with infinite horizon discounted cost. It is well known that the optimal solution is given by a stationary policy and can be obtained as the unique solution to the Bellman’s equation: n o X V ∗ (s0 ) = min Ca (s0 ) + γ Pa(s0 ),s1 V ∗ (s1 ) a (4.3) s1 ∈Q×Q where Pa(s0 ),s1 denotes the probability of transitioning from state s0 (0) = s00 = a(s0 ) to s(1) = s1 . Note that the intermediate state s0 (t) has no randomness when s(t) and a(·) are given, thus we only consider the transition probability from s0 (t) to the next state s(t + 1) in (4.3). Also note that we always have h(t + 1) = h0 (t). 4.2.3 Characteristics of Optimal Policy We next characterize some structural properties of the optimal solution. The following proposition states that it is not optimal to migrate the service to a location that is farther away from the user than the current service location, as one would intuitively expect. Proposition 4.1. Let a∗ (s) = (u, h0 ) denote the optimal action at any state s = (u, h). Then, we have ku − h0 k ≤ ku − hk. (If the optimal action is not unique, then there exists at least one such optimal action.) 4.3. Constant Cost Model under 1-D Mobility 72 Corollary 4.1. If cm (x) and cd (y) are both constants (possibly of different values) for x, y > 0, and cm (0) < cm (x) and cd (0) < cd (y) for x, y > 0, then migrating to locations other than the current location of the mobile user is not optimal. See Appendix C.2 for the proofs of Proposition 4.1 and Corollary 4.1. 4.2.4 Generic Notations In the remainder of this chapter, where there is no ambiguity, we reuse the notations P , Ca (·), V (·), and a(·) to respectively represent transition probabilities, onetimeslot costs, discounted sum costs, and actions of different MDPs. 4.3 Constant Cost Model under 1-D Mobility In this section, we consider the simple case where users follow a 1-D mobility model. 4.3.1 Definitions We consider a 1-D region partitioned into a discrete set of areas, each of which is served by an MMC, as shown in Fig. 1.1. Such a scenario models user mobility on roads, for instance. Mobile users are assumed to follow a 1-D asymmetric random walk mobility model. In every new timeslot, a user moves with probability p (or q) to the area that is on the right (or left) of its previous area, it stays in the same area with probability 1 − p − q. If the system is sampled at handoff instances, then 1 − p − q = 0, but we consider the general case with 0 ≤ 1 − p − q ≤ 1. Obviously, this mobility model can be described as a Markov chain. The state of the user is defined as the offset between the mobile user location and the location of the MMC running the service at the beginning of a slot, before 4.3. Constant Cost Model under 1-D Mobility 73 possible service migration, i.e., the state in slot t is defined as e(t) = u(t) − h(t), where u(t) is the location (index of area) of the mobile user, and h(t) the location (index of area) of the MMC hosting the service. Note that e(t) can be zero, positive, or negative. For simplicity, we omit the variable t and the policy subscript π in the following. Because we consider a scenario where all MMCs are connected via the backhaul (as shown in Fig. 1.1), and the backhaul can be regarded as a central entity (which resembles the case for cellular networks, for example), in this section, we define the migration and transmission costs respectively as follows: cm (x) = 0, if x = 0 (4.4) 1, if x > 0 cd (y) = 0, if y = 0 ξ, . (4.5) if y > 0 This gives the following one-timeslot cost function for taking action a in state e: Ca (e) = 0, ξ, if no migration or data transmission, i.e., e = a(e) = 0 if only data transmission, i.e., e = a(e) 6= 0 if only migration, i.e., e 6= a(e) = 0 1, ξ + 1, if both migration and data transmission, i.e., e 6= a(e) 6= 0 (4.6) Equation (4.6) is explained as follows. If the action a under state e causes no migration or data transmission (e.g., if the user and the MMC hosting the service are in the same location, i.e., e = 0, and we do not migrate the service to another location), we do not need to communicate via the backhaul network, and the cost is zero. A non- … t Operating after possible migration at time t t+1 Time 4.3. Constant Cost Model under 1-D Mobility 1-p-q 1-p-q M M+1 … q q 1-p-q -2 -1 q 1-p-q 1-p-q p p p p q 1 p p 2 q q 1-p-q 1-p-q 1-p-q p 0 q 74 … q N-1 N p Figure 4.2: MDP model for service migration. The solid lines denote transition under action a = 0 and the dotted lines denote transition under action a = 1. When taking action a = 1 from any state, the next state is e = −1 with probability q, e = 0 with probability 1 − Backhaul p − q, or enetwork = 1 with probability p. zero cost is incurred when the user and the MMC hosting the service are in different MicroMicroMicro- cloud 1 cloud 2 cloud i Area 1 Area 2 Area i locations (i.e., u 6= h). In this case, if we do not migrate the service to the current 0 user location at the beginning of the timeslot … (i.e., u 6= h = h ), the…data between the MMC and mobile user need to be transmitted via the backhaul network. This data transmission incurs a cost of ξ. When migrating to the current user location (i.e., u = h0 6= h), we need resources to support migration, incurring a migration cost that is assumed to be 1, i.e., the cost Ca (e) is normalized by the migration cost. Finally, if we migrate to a location other than the current user location, so that data transmission still occurs after migration (i.e., u 6= h0 6= h), the cost is ξ + 1. 4.3.1.1 Characteristics of the Optimal Solution Note that the cost definitions in (4.4) and (4.5), thus (4.6), satisfy the conditions for Corollary 4.1. Therefore, we only have two candidates for optimal actions, which are migrating to the current user location and not migrating. This simplifies the action space to two actions: a migration action, denoted as a = “migrate”; and a nomigration action, denoted as a = “not migrate”. In practice, there is usually a limit on the maximum allowable distance between the mobile user and the MMC hosting its service for the service to remain usable. We model this limitation by a maximum negative offset M and a maximum positive offset N (where M < 0, N > 0), such that the service must be migrated (a = “migrate”) when the system enters state M 4.3. Constant Cost Model under 1-D Mobility 75 or N . This implies that, although the user can move in an unbounded space, the state space of our MDP for service control is finite. The overall transition diagram of the resulting MDP is illustrated in Fig. 4.2. Note that because each state transition is the concatenated effect of (possible) migration and user movement, and the states are defined as the offset between user and service locations, the next state after taking a migration action is either −1, 0, or 1. 4.3.1.2 Modified Cost Function With the above considerations, the cost function in (4.6) can be expressed as the following: Ca (e) = 4.3.1.3 0, if e = 0 ξ, if e 6= 0, M < e < N, a = “not migrate” 1, if e 6= 0, M ≤ e ≤ N, a = “migrate” (4.7) Modified Bellman’s Equations With the one-timeslot cost defined as in (4.7), we obtain the following Bellman’s equations for the discounted sum cost when respectively taking actions a = “not migrate” and a = “migrate” at state e: V (e|a = “not migrate”) = P γ 1 j=−1 P0j V (j), P ξ + γ e+1 (4.8) j=e−1 V (e|a = “migrate”) = P γ 1 j=−1 Pej V (j), if e 6= 0, M<e<N P0j V (j), P 1 + γ 1 j=−1 if e = 0 if e = 0 (4.9) P0j V (j), if e 6= 0, M≤e≤N 4.3. Constant Cost Model under 1-D Mobility 76 where the transition probability Pij is related to parameters p and q as defined earlier. The discounted sum cost V (e) when following the optimal policy is V (e) = min{V (e|a = “not migrate”), V (e|a = “migrate”)}, if M < e < N V (e|a = “migrate”), if e = M or e = N (4.10) 4.3.2 Optimal Threshold Policy 4.3.2.1 Existence of Optimal Threshold Policy We first show that there exists a threshold policy which is optimal for the MDP in Fig. 4.2. Proposition 4.2. There exists a threshold policy (k1 , k2 ), where M < k1 ≤ 0 and 0 ≤ k2 < N , such that when k1 ≤ e ≤ k2 , the optimal action for state e is a∗ (e) = “not migrate”, and when e < k1 or e > k2 , a∗ (e) = “migrate”. Proof. It is obvious that different actions for state zero a(0) = “not migrate” and a(0) = “migrate” are essentially the same, because the mobile user and the MMC hosting its service are in the same location under state zero, either action does not incur cost and we always have Ca(0) (0) = 0. Therefore, we can conveniently choose a∗ (0) = “not migrate”. In the following, we show that, if it is optimal to migrate at e = k1 − 1 and e = k2 + 1, then it is optimal to migrate at all states e with M ≤ e ≤ k1 − 1 or k2 + 1 ≤ e ≤ N . We relax the restriction that we always migrate at states M and N for now, and later discuss that the results also hold for the unrelaxed case. We only focus on k2 + 1 ≤ e ≤ N , because the case for M ≤ e ≤ k1 − 1 is similar. 4.3. Constant Cost Model under 1-D Mobility 77 If it is optimal to migrate at e = k2 + 1, we have V (k2 + 1|a = “migrate”) ≤ ξ ∞ X γt = t=0 ξ 1−γ (4.11) where the right hand-side of (4.11) is the discounted sum cost of a never-migrate policy supposing that the user never returns back to state zero when starting from state e = k2 + 1. This cost is an upper bound of the cost incurred from any possible state-transition path without migration, and migration cannot bring higher cost than this because otherwise it contradicts the presumption that it is optimal to migrate at e = k2 + 1. Suppose we do not migrate at a state e0 where k2 + 1 < e0 ≤ N , then we have a (one-timeslot) cost of ξ in each timeslot until the user reaches a migration state (i.e., a state at which we perform migration). From (4.9), we know that V (e|a = “migrate”) is constant for any e 6= 0. Therefore, any state-transition path L starting from state e0 has a discounted sum cost of 0 VL (e ) = ξ tX m −1 γ t + γ tm V (k2 + 1|a = “migrate”) t=0 where tm > 0 is a parameter representing the first timeslot that the user is in a migration state after reaching state e0 (assuming that we reach state e0 at t = 0), which is dependent on the state-transition path L. We have VL (e0 ) − V (k2 + 1|a = “migrate”) (1 − γ tm ) =ξ − 1 − γ tm V (k2 + 1|a = “migrate”) 1−γ ξ tm = 1−γ − V (k2 + 1|a = “migrate”) ≥ 0 1−γ where the last inequality follows from (4.11). It follows that, for any possible statetransition path L, VL (e0 ) ≥ V (k2 + 1|a = “migrate”). Hence, it is always optimal 4.3. Constant Cost Model under 1-D Mobility 78 to migrate at state e0 , which brings cost V (e0 |a = “migrate”) = V (k2 + 1|a = “migrate”). The result also holds with the restriction that we always migrate at states M and N , because no matter what thresholds (k1 , k2 ) we have for the relaxed problem, migrating at states M and N always yield a threshold policy. Proposition 4.2 shows the existence of an optimal threshold policy. The optimal threshold policy exists for arbitrary values of M , N , p, and q. 4.3.3 Simplifying the Cost Calculation The existence of the optimal threshold policy allows us simplify the cost calculation, which helps us develop an algorithm that has lower complexity than standard MDP solution algorithms. For the policy specified by the thresholds (k1 , k2 ), the value updating function (4.10) can be changed to the following: V (e) = V (e|a = “not migrate”), if k1 ≤ e ≤ k2 V (e|a = “migrate”), (4.12) otherwise From (4.8) and (4.9), we know that, for a given policy with thresholds (k1 , k2 ), we only need to compute V (e) with k1 − 1 ≤ e ≤ k2 + 1, because the values of V (e) with e ≤ k1 − 1 are identical, and the values of V (e) with e ≥ k2 − 1 are also identical. Note that we always have k1 − 1 ≥ M and k2 + 1 ≤ N , because k1 > M and k2 < N as we always migrate when at states M and N . Define v(k1 ,k2 ) = [V (k1 − 1) V (k1 ) · · · V (0) · · · V (k2 ) V (k2 + 1)]T (4.13) 4.3. Constant Cost Model under 1-D Mobility 79 T c(k1 ,k2 ) = 1 ξ ··· ξ 0 ξ ··· ξ 1 | {z } | {z } −k1 elements k2 elements P0(k1 ,k2 ) P0,k1 −1 Pk1 ,k1 −1 .. . = P0,k1 −1 .. . Pk ,k −1 2 1 P0,k1 −1 (4.14) ··· P00 ··· · · · Pk1 ,0 · · · .. . ··· P00 .. . ··· · · · Pk2 ,0 · · · ··· P00 ··· P0,k2 +1 Pk1 ,k2 +1 .. . P0,k2 +1 .. . Pk2 ,k2 +1 P0,k2 +1 (4.15) where superscript “T” denotes the transpose of the matrix. Then, (4.8) and (4.9) can be rewritten as v(k1 ,k2 ) = c(k1 ,k2 ) + γP0(k1 ,k2 ) v(k1 ,k2 ) (4.16) The value vector v(k1 ,k2 ) can be obtained by v(k1 ,k2 ) = I − γP0(k1 ,k2 ) −1 c(k1 ,k2 ) (4.17) The matrix I − γP0(k1 ,k2 ) is invertible for 0 < γ < 1, because in this case there exists a unique solution for v(k1 ,k2 ) from (4.16), which is a known property for MDPs. Equation (4.17) can be computed using Gaussian elimination that has a complexity of O (|M | + N )3 . 4.3.4 Algorithm for Finding the Optimal Thresholds To find the optimal thresholds, we can perform a search on values of (k1 , k2 ). Further, because an increase/decrease in V (e) for some e increases/decreases each ele- 4.3. Constant Cost Model under 1-D Mobility 80 ment in the cost vector v due to cost propagation following balance equations (4.8) and (4.9), we only need to minimize V (e) while considering a specific state e. We propose an algorithm for finding the optimal thresholds, as shown in Algorithm 4.1, which is a modified version of the standard3 policy iteration mechanism [41, Chapter 6]. Algorithm 4.1 is explained as follows. We keep iterating until the thresholds no longer change, which implies that the optimal thresholds have been found. The thresholds (k1∗ , k2∗ ) are those obtained from each iteration. Lines 4–6 compute V (e) for all e under the given thresholds (k1∗ , k2∗ ). Then, Lines 8–22 determine the search direction for k1 and k2 . Because V (e) in each iteration is computed using the current thresholds (k1∗ , k2∗ ), we have actions a(k1∗ ) = a(k2∗ ) = “not migrate”, and (4.8) is automatically satisfied when replacing its left hand-side with V (k1∗ ) or V (k2∗ ). Lines 9 and 16 check whether iterating according to (4.9) can yield lower cost. If it does, it means that migrating is a better action at state k1∗ (or k2∗ ), which also implies that we should migrate at states e with M ≤ e ≤ k1∗ (or k2∗ ≤ e ≤ N ) according to Proposition 4.2. In this case, k1∗ (or k2∗ ) should be set closer to zero, and we search through those thresholds that are closer to zero than k1∗ (or k2∗ ). If Line 9 (or Line 16) is not satisfied, according to Proposition 4.2, it is good not to migrate at states e with k1∗ ≤ e ≤ 0 (or 0 ≤ e ≤ k2∗ ), so we search k1 (or k2 ) to the direction approaching M (or N ), to see whether it is good not to migrate under those states. Lines 23–37 adjust the thresholds. If we are searching toward state M or N and Line 25 is satisfied, it means that it is better not to migrate under this state (ki ), and we update the threshold to ki . When Line 27 is satisfied, it means that it is better to migrate at state ki . According to Proposition 4.2, we should also migrate at any state 3 We use the term “standard” here to distinguish with the modified policy iteration mechanism proposed in Algorithm 4.1. 4.3. Constant Cost Model under 1-D Mobility 81 Algorithm 4.1 Modified policy iteration algorithm for finding the optimal thresholds 1: Initialize k1∗ ← 0, k2∗ ← 0 2: repeat 3: k10∗ ← k1∗ , k20∗ ← k2∗ //record previous thresholds 4: Construct c(k∗ ,k∗ ) and P0 k∗ ,k∗ according to (4.14) and (4.15) 1 2 ( 1 2) 5: Evaluate v(k∗ ,k∗ ) from (4.17) 1 2 6: Extend v(k∗ ,k∗ ) to obtain V (e) for all M ≤ e ≤ N 1 2 7: for i = 1, 2 do 8: if i = 1 then P 9: if 1 + γ 1j=−1 P0j V (j) < V (k1∗ ) then 10: dir ← 1, loopVec ← [k1∗ + 1, k1∗ + 2, ..., 0] 11: k1∗ ← k1∗ + 1 12: else 13: dir ← 0, loopVec ← [k1∗ − 1, k1∗ − 2, ..., M + 1] 14: end if 15: else if i = P 2 then 16: if 1 + γ 1j=−1 P0j V (j) < V (k2∗ ) then 17: dir ← 1, loopVec ← [k2∗ − 1, k2∗ − 2, ..., 0] 18: k2∗ ← k2∗ − 1 19: else 20: dir ← 0, loopVec ← [k2∗ + 1, k2∗ + 2, ..., N − 1] 21: end if 22: end if 23: for ki = each value in loopVec do 24: if dir = 0 then P i +1 Pki ,j V (j) < V (ki ) then 25: if ξ + γ kj=k i −1 ∗ 26: ki ← ki P i +1 27: else if ξ + γ kj=k P V (j) > V (ki ) then i −1 ki ,j 28: exit for 29: end if 30: else if dir = P11 then 31: if 1 + γ j=−1 P0j V (j) < V (ki ) then 32: ki∗ ← ki −P sign(ki ) 33: else if 1 + γ 1j=−1 P0j V (j) > V (ki ) then 34: exit for 35: end if 36: end if 37: end for 38: end for 39: until k1∗ = k10∗ and k2∗ = k20∗ 40: return k1∗ , k2∗ 4.3. Constant Cost Model under 1-D Mobility 82 closer to M or N than state ki , thus we exit the loop. If we are searching toward state zero and Line 31 is satisfied, it is good to migrate under this state (ki ), therefore the threshold is set to one state closer to zero (ki − sign(ki )). When Line 33 is satisfied, we should not migrate at state ki . According to Proposition 4.2, we should also not migrate at any state closer to zero than state ki , and we exit the loop. Proposition 4.3. The threshold-pair (k1∗ , k2∗ ) is different in every iteration of the loop starting at Line 2, otherwise the loop terminates. Proof. The loop starting at Line 2 changes k1∗ and k2∗ in every iteration so that V (e) for all e become smaller. It is therefore impossible that k1∗ and k2∗ are the same as in one of the previous iterations and at the same time reduce the value of V (e), because V (e) computed from (4.17) is the stationary cost value for thresholds (k1∗ , k2∗ ). The only case when (k1∗ , k2∗ ) are the same as in the previous iteration (which does not change V (e)) terminates the loop. Corollary 4.2. The number of iterations in Algorithm 4.1 is O(|M |N ). Proof. According to Proposition 4.3, there can be at most |M |N + 1 iterations in the loop starting at Line 2. If we use Gaussian elimination (with complexity O (|M | + N )3 ) to compute (4.17), the overall time-complexity of Algorithm 4.1 is O |M |N (|M | + N )3 . This is a promising result because the overall complexity of standard value or policy iteration methods for solving MDPs are generally dependent on the discount factor γ [42]. Regarding the space-complexity issue, i.e., necessary memory storage, we can see that the non-constant size arrays in Algorithm 4.1 include v(k∗ ,k∗ ) , c(k∗ ,k∗ ) , and 1 2 1 2 2 0 P k∗ ,k∗ , so the space complexity of this algorithm is O (|M | + N ) , where the ( 1 2) square is due to matrix P0 . Note that we also need to store non-truncated versions of v, c, and P0 , but the order of complexity remains the same. This space-complexity 4.3. Constant Cost Model under 1-D Mobility 83 is also in the same order as standard value and policy iteration algorithms which also need to store non-truncated arrays of costs and transition probabilities. We do not analyze the detailed change in memory occupation of different approaches because it can largely depend on how well the code is optimized for memory. 4.3.5 Simulation Results In the simulations, we set the number of states |M | = N = 10. The transition probabilities p and q are randomly generated. Simulations are run with 1000 different random seeds in each setting to obtain the overall performance. Distribution of Optimal Thresholds: We first study the distribution of optimal thresholds found from the proposed threshold-based method under different values of ξ. Recall that as defined in (4.6), ξ stands for the data transmission cost, while the migration cost is defined to be always equal to 1. A larger value of ξ stands for a larger transmission cost. The frequency of different optimal threshold values is shown in Fig. 4.3, where the frequency is defined as the percentage that each threshold value is found as optimum, among all 1000 simulation instances. Note that the thresholds k1 and k2 can only take integer values within the interval of [−10, 10], but we have connected these discrete points in the plot to illustrate the overall distribution. Several interesting observations can be seen in Fig. 4.3. First, because we always migrate at states M = −10 and N = 10 as discussed in Section 4.3.1.1, we always have k1∗ ≥ −9 and k2∗ ≤ 9. Thus the frequencies of thresholds k1 = −10 and k2 = 10 are always zero. Next, when ξ = 0, we always have the optimal thresholds k1∗ = −9 and k2∗ = 9, which means that we never migrate except at states M and N . We refer to such a policy as “never-migrate” in later discussion. The reason for having such optimal thresholds is because the transmission cost is zero in this case, 4.3. Constant Cost Model under 1-D Mobility 84 k*1 ξ=0 ξ=0.2 1 1 0.8 0.8 0.8 0.6 0.6 0.6 0.4 0.4 0.4 0.2 0.2 0.2 0 10 0 −10 ξ=0.6 Frequency of occurrence ξ=0.4 1 0 −10 0 10 0 −10 ξ=0.8 1 1 0.8 0.8 0.8 0.6 0.6 0.6 0.4 0.4 0.4 0.2 0.2 0.2 0 10 0 −10 ξ=1.5 0 10 0 −10 ξ=2 1 1 0.8 0.8 0.8 0.6 0.6 0.6 0.4 0.4 0.4 0.2 0.2 0.2 0 10 0 −10 0 10 0 10 ξ=2.5 1 0 −10 0 ξ=1 1 0 −10 k*2 10 0 −10 0 Threshold value Figure 4.3: Frequency of different optimal threshold values under γ = 0.9. 10 4.3. Constant Cost Model under 1-D Mobility 85 and there is no benefit from performing migration, except at states M and N which is enforced in the problem formulation. When ξ increases, the optimal thresholds tend to become closer to zero, because migration is more beneficial for reducing the overall cost when the transmission cost is large. We see a spike in the frequencies for thresholds k1 = −9 and k2 = 9 when ξ ≤ 0.8, due to the truncation effect of states |M | and N . The simulation instances that have these optimal threshold values correspond to those that would have optimal thresholds at or beyond the [−9, 9] limit if we would have larger values of |M | and N . When ξ ≥ 2, the optimal thresholds k1∗ and k2∗ are always zero, because the transmission cost is so large in this case so that it is beneficial to always migrate to the user location. We refer to such a migration policy as “always-migrate” in later discussion. Time-Complexity and Discounted Sum Costs: We now study the time-complexity of different MDP solution approaches and discounted sum costs of different migration policies. We use two measures of time-complexity. One is the physical time of running the algorithms in MATLAB (referred to as computation time), on a computer with 64-bit Windows 7, Intel Core i7-2600 CPU, and 8GB memory, without any other foreground tasks running simultaneously (but there might be some background tasks running which are hard to control). The other measure is the number of floating-point operations (FLOPs), which is independent of background tasks thus more objective, but less intuitive than the computation time. The number of FLOPs has been collected using the toolbox in [43]. Figs. 4.4–4.6. show the computation time, number of FLOPs, and discounted sum costs under different values of ξ, respectively with γ = 0.5, 0.9, 0.99. For time complexity evaluation, we compare the proposed threshold approach against standard value and policy iteration methods [41, Chapter 6]. The value iteration terminates according to an error bound of ε = 0.1 in the discounted sum cost. Note 4.3. Constant Cost Model under 1-D Mobility 86 −2 10 Computation time (s) Proposed Policy iteration Value iteration −3 10 −4 10 0 0.5 1 1.5 2 2.5 5 Number of FLOPs 10 4 10 Proposed Policy iteration Value iteration 3 10 2 10 0 0.5 1 1.5 2 2.5 1.5 2 2.5 Discounted sum cost 5 Optimal Never migrate Always migrate 4 3 2 1 0 0 0.5 1 ξ Figure 4.4: Performance under different ξ with γ = 0.5. 4.3. Constant Cost Model under 1-D Mobility 87 −1 Computation time (s) 10 −2 10 Proposed Policy iteration Value iteration −3 10 −4 10 0 0.5 1 1.5 2 2.5 5 Number of FLOPs 10 4 10 Proposed Policy iteration Value iteration 3 10 2 10 0 0.5 1 1.5 2 2.5 1.5 2 2.5 Discounted sum cost 25 Optimal Never migrate Always migrate 20 15 10 5 0 0 0.5 1 ξ Figure 4.5: Performance under different ξ with γ = 0.9. 4.3. Constant Cost Model under 1-D Mobility 88 0 Computation time (s) 10 −1 10 Proposed Policy iteration Value iteration −2 10 −3 10 −4 10 0 0.5 1 1.5 2 2.5 8 Number of FLOPs 10 6 10 Proposed Policy iteration Value iteration 4 10 2 10 0 0.5 1 1.5 2 2.5 1.5 2 2.5 Discounted sum cost 250 Optimal Never migrate Always migrate 200 150 100 50 0 0 0.5 1 ξ Figure 4.6: Performance under different ξ with γ = 0.99. 4.3. Constant Cost Model under 1-D Mobility 89 that the proposed method and the standard policy iteration method always incur the optimal cost. For discounted sum cost evaluation, we compare the cost of the optimal migration policy to that of never-migrate and always-migrate policies. The results show that the proposed method always has lowest computation time and almost always has lowest number of FLOPs. This is because the proposed algorithm simplifies the solution search procedure compared to standard mechanisms. Specifically, the computation time of the standard policy iteration method is 2 to 5 times larger than that of the proposed algorithm, while the value iteration approach consumes longer time. Compared to the computation time, the trend of the number of FLOPs varies more with different values of ξ. Particularly for the proposed approach, the number of FLOPs is high when ξ is small, and it decreases substantially when ξ is large. We think this is because the complexity of matrix inversion is adequately weighted when counting the number of FLOPs, but less weighted when measuring the computation time, because MATLAB generally computes matrix operations significantly faster than other procedures written in MATLAB code. When ξ is small, the optimal policy becomes close to a never-migrate policy (see earlier discussion on Fig. 4.3). In this case, the reduced transition probability matrix P0(k1 ,k2 ) defined in (4.15) is usually large, thus a large number of FLOPs is needed to compute the inverse in (4.17). We also note here that the initial policies in all three algorithms are set as the alwaysmigrate policy. Conversely, when ξ is large, the optimal policy is close to an alwaysmigrate policy and the size of P0(k1 ,k2 ) is usually small, resulting in a small number of FLOPs required for matrix inversion. We can also see a similar but much less obvious trend for the standard policy iteration approach, which we think is also due to the complexity of matrix inversion, because MATLAB may have mechanisms to simplify the inversion procedure according to the matrix structure. When comparing the figures for different values of γ, we can see a clear increase 4.3. Constant Cost Model under 1-D Mobility 90 in the complexity of value iteration when γ increases. This is because for a larger γ, it takes a longer time for the discounted sum cost to converge when iterating according to (4.8)–(4.10), and value iteration is based on such iterations. The third subfigures in each of Figs. 4.4–4.6 show the optimal cost compared to a never-migrate or always-migrate policy, where we note that the optimal cost can be found from the proposed threshold-based algorithm. As discussed earlier, we also see here that the optimal cost approaches the cost of a never-migrate policy when ξ is small, and it approaches the cost of an always-migrate policy when ξ is large. There is an intersection point of the costs of never-migrate and always-migrate policies. When γ = 0.5, the intersection point is at around ξ = 0.9, and it is close to the cost of optimal policy under the same value of ξ. We can infer that the optimal policy approaches either a never-migrate policy or an always-migrate policy in this case. When γ is larger, the gap between the intersection point and optimal cost is also larger. In this case, the optimal policy is likely to be neither never-migrate nor always-migrate, meaning that the optimal thresholds lie somewhere between 0 and ±9, as shown in Fig. 4.3 when ξ takes a proper value. The reason for this phenomenon is that with a small γ, the future costs have low weights in the discounted sum cost expression (4.1), so migration decision is made mainly based on the instantaneous cost. In the extreme case where γ = 0, the optimal policy would only minimize the one-slot cost Ca (e) defined in (4.7), which means it would never migrate if ξ < 1 and always migrate if ξ > 1. Therefore, the gap between the intersection point and optimal cost is small when γ is small, and large when γ is large. 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 4.4 91 Constant-Plus-Exponential Cost Model under 2D Mobility In this section, we consider 2-D mobility models with more general cost functions. We first outline the necessity of simplifying the search space, then introduce the distance-based MDP, which will be used to approximate the 2-D MDP later in this section. 4.4.1 Simplifying the Search Space Because the optimality of threshold policy discussed in Section 4.3 does not directly apply to the more general case involving 2-D mobility and non-constant transmission/migration costs, we need to find alternative ways to simplify the solution space. Note that Proposition 4.1 simplifies the search space for the optimal policy considerably. However, it is still very challenging to derive the optimal control policy for the general model presented in Section 4.2, particularly when the state space {s(t)} is large. One possible approach to address this challenge is to re-define the state space to represent only the distance between the user and service locations d(t) = ku(t) − h(t)k. The motivation for this comes from the observation that the cost functions in our model depend only on the distance (see cost definition in Section 4.2.1). Note that in general, the optimal control actions can be different for two states s0 and s1 that have the same user-service distance. However, it is reasonable to use the distance as an approximation of the state space for many practical scenarios of interest, and this simplification allows us to formulate a far more tractable MDP. We discuss the distance-based MDP in the next section, and show how the results on the distance-based MDP can be applied to 2-D mobility models and real-world mobility traces in Sections 4.4.3 and 4.4.4. 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 1-p-q 1 pq 1 0 1-p p0 0 1 q q 1-p-q 1 pq 11-p-q pq 1 1-p-q p p p 2 q p 92 … q N-1 N 1 N a(N) (N) = 1 Figure 4.7: An example of distance-based MDP with the distances {d(t)} (before possible migration) as states. In this example, migration is only performed at state N , and only the possible action of a(N ) = 1 is shown for compactness. The solid lines denote state transitions without migration. 4.4.2 Optimal Policy for Distance-Based MDP In this section, we consider a distance-based4 MDP where the states {d(t)} represent N lithe distances Nonlinear cost, t single ibetween l sided id the d user and the service before possible migration (an example is shown in Fig. 4.7), i.e., d(t) = ku(t) − h(t)k. Similar to Section 4.3, we define the parameter N as an application-specific maximum allowed distance, and we always perform migration when d(t) ≥ N . We set the actions a(d(t)) = a(N ) for d(t) > N , so that we only need to focus on the states d(t) ∈ [0, N ]. After taking action a(d(t)), the system operates in the intermediate state d0 (t) = a(d(t)), and the value of the next state d(t + 1) follows the transition probability Pd0 (t),d(t+1) which is related to the mobility model of the user. To simplify the solution, we restrict the transition probabilities Pd0 (t),d(t+1) according to the parameters p0 , p, and q as shown in Fig. 4.7. Such a restriction is sufficient when the underlying mobility model is a uniform 1-D random walk where the user moves one step to the left or right with equal probability r1 and stays in the same location with probability 1 − 2r1 , in which case we can set p = q = r1 and p0 = 2r1 . This model is also sufficient to approximate the uniform 2-D random walk model, as will be discussed in Section 4.4.3.2. For an action of d0 (t) = a(d(t)), the new service location h0 (t) is chosen such 4 We assume that the distance is quantized, as it will be the case with the 2-D model discussed in later sections. 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 93 that x = kh(t) − h0 (t)k = |d(t) − d0 (t)| and y = ku(t) − h0 (t)k = d0 (t). This means that migration happens along the shortest path that connects u(t) and h(t), and h0 (t) is on this shortest path (also note that d0 (t) ≤ d(t) according to Proposition 4.1). Such a migration is possible for the 1-D case where u(t), h(t), and h0 (t) are all scalar values. It is also possible for the 2-D case if the distance metric is properly defined (see Section 4.4.3.2 for details). The one-timeslot cost is then Ca (d(t)) = cm (|d(t) − d0 (t)|) + cd (d0 (t)). 4.4.2.1 Constant-Plus-Exponential Cost Functions To simplify the analysis later, we define the cost functions cm (x) and cd (y) in a constant-plus-exponential form: cm (x) = 0, if x = 0 (4.18) βc + βl µx , if x > 0 cd (y) = 0, if y = 0 (4.19) δc + δl θy , if y > 0 where βc , βl , δc , δl , µ, and θ are real-valued parameters. The functions cm (x) and cd (y) defined above can have different shapes and are thus applicable to many realistic scenarios (see Fig. 4.8 for an example). They can approximate an arbitrary cost function as discussed in Appendix B. They also have nice properties allowing us to obtain a closed-form solution to the discounted sum cost, based on which we design an efficient algorithm for finding the optimal policy. The parameters βc , βl , δc , δl , µ, and θ are selected such that cm (x) ≥ 0, cd (y) ≥ 0, and both cm (x) and cd (y) are non-decreasing respectively in x and y for x, y ≥ 0. Explicitly, we have µ ≥ 0; βl ≤ 0 when µ ≤ 1; βl ≥ 0 when µ ≥ 1; βc ≥ −βl ; θ ≥ 0; δl ≤ 0 when θ ≤ 1; δl ≥ 0 when θ ≥ 1; and δc ≥ −δl . The definition that 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 2 cd (y) 1.5 94 δc=1, δl= −1, θ=0.9 δc= −0.4, δl=0.4, θ=1.03 δc=3000, δl= −3000, θ=0.99999 1 0.5 0 0 10 20 y 30 40 50 Figure 4.8: Example of constant-plus-exponential cost function cd (y). cm (0) = cd (0) = 0 is for convenience, because a non-zero cost for x = 0 or y = 0 can be offset by the values of βc and δc , thus setting cm (0) = cd (0) = 0 does not affect the optimal decision. With this definition, the values of βc + βl and δc + δl can be regarded as constant terms of the costs, at least such an amount of cost is incurred when x > 0 and y > 0, respectively. The parameters µ and θ specify the impact of the distance x and y, respectively, to the costs, and their values can be related to the network topology and routing mechanism of the network. The parameters βl and δl further adjust the costs proportionally. 4.4.2.2 Closed-Form Solution to Discounted Sum Cost Problem Formulation with Difference Equations: From (4.1), we can get the following balance equation on the discounted sum cost for a given policy π: aπ (d(0))+1 Vπ (d(0)) = Caπ (d(0)) + γ X Paπ (d(0)),d(1) Vπ (d(1)). (4.20) d(1)=aπ (d(0))−1 In the following, we will omit the subscript π and write d(0) as d for short. Proposition 4.4. For a given policy π, let {ñk : k ≥ 0} denote the series of all migration states (such that a(ñk ) 6= ñk ) as specified by policy π, where 0 ≤ ñk ≤ N . 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 95 The discounted sum cost V (d) for d ∈ [ñk−1 , ñk ] (where we define ñ−1 , 0 for convenience) when following policy π can be expressed as V (d) = Ak ζ1d + Bk ζ2d +D+ H · θ d if 1 − φ1 θ − φ2 θ 6= 0 Hd · θd if 1 − φ1 θ − φ2 θ = 0 (4.21) where Ak and Bk are constants corresponding to the interval [ñk−1 , ñk ], the coefficients ζ1 , ζ2 , D, and H are expressed as follows: 1+ ζ1 = √ 1 − 4φ1 φ2 1− , ζ2 = 2φ2 D= H= 1− φ4 1 −φ θ 2 θ where we define φ1 , δl , 1−γ(1−p−q) φ3 1 − φ1 − φ2 φ4 φ1 −φ2 θ θ φ γq , 1−γ(1−p−q) √ 1 − 4φ1 φ2 2φ2 (4.23) if 1 − φ1 θ − φ2 θ 6= 0 if 1 − φ1 θ − φ2 θ = 0 φ2 , (4.22) (4.24) γp , 1−γ(1−p−q) φ3 , δc , 1−γ(1−p−q) and φ4 , and assume that p 6= 0 and q 6= 0. Proof. Note that (4.20) is a difference equation [44]. Because we only migrate at states {ñk }, we have a(d) = d for d ∈ (ñk−1 , ñk ) (∀k). From (4.20), for d ∈ (ñk−1 , ñk ), we have V (d) = δc + δl θd + γ d+1 X Pdd1 V (d1 ) . (4.25) d1 =d−1 We can rewrite (4.25) as V (d) = φ1 V (d − 1) + φ2 V (d + 1) + φ3 + φ4 θd . (4.26) This difference function has characteristic roots as expressed in (4.22). When 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 96 0 < γ < 1, we always have ζ1 6= ζ2 under the assumption that p 6= 0 and q 6= 0. Fixing an index k, for d ∈ (ñk−1 , ñk ), the homogeneous equation of (4.26) has general solution Vh (d) = Ak ζ1d + Bk ζ2d . (4.27) To solve the non-homogeneous equation (4.26), we try a particular solution in the form of Vp (d) = D + H · θ d if 1 − D + Hd · θd if 1 − φ1 θ − φ2 θ 6= 0 φ1 θ − φ2 θ = 0 (4.28) where D and H are constant coefficients. By substituting (4.28) into (4.26), we get (4.23) and (4.24). Because the expression in (4.26) is related to d−1 and d+1, the result also holds for the closed interval. Therefore, V (d) can be expressed as (4.21) for d ∈ [ñk−1 , ñk ] (∀k). The above proposition is subject to the assumption that p 6= 0 and q 6= 0. When p = 0 or q = 0, we will have m1 = m2 , in which case we can still find the solution to the difference equation (4.20) using a similar approach, but we omit the discussion for brevity. We also note that for two different states d1 and d2 , if policy π has actions aπ (d1 ) = d2 and aπ (d2 ) = d2 , then Vπ (d1 ) = cm (|d1 − d2 |) + Vπ (d2 ) . (4.29) Finding the Coefficients: The coefficients Ak and Bk are unknowns in the solution (4.21) that need to be found using additional constraints. Their values may be different for different k. After Ak and Bk are determined, (4.21) holds for all d ∈ [0, N ]. 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility We assume 1 − φ1 θ − φ2 θ 6= 0 and 1 − φ2 θ 97 − φ1 θ 6= 0 in the following, the other cases can be derived in a similar way and are omitted for brevity. Coefficients for interval [0, ñ0 ]: We have one constraint from the balance equation (4.20) for d = 0, which is V (0) = γp0 V (1) + γ(1 − p0 )V (0). (4.30) By substituting (4.21) into (4.30), we get A0 (1 − φ0 ζ1 ) + B0 (1 − φ0 ζ2 ) = D(φ0 − 1) + H(φ0 θ − 1) where φ0 , γp0 . 1−γ(1−p0 ) (4.31) We have another constraint by substituting (4.21) into (4.29), which gives a(ñ ) a(ñ ) A0 ζ1ñ0 − ζ1 0 + B0 ζ2ñ0 − ζ2 0 = βc + βl µñ0 −a(ñ0 ) − H θñ0 − θa(ñ0 ) . (4.32) We can find A0 and B0 from (4.31) and (4.32). Coefficients for interval [ñk−1 , ñk ]: Assume that we have found V (d) for all d ≤ ñk−1 . By letting d = ñk−1 in (4.21), we have the first constraint given by ñ ñ Ak ζ1 k−1 + Bk ζ2 k−1 = V (ñk−1 ) − D − H · θñk−1 . (4.33) For the second constraint, we consider two cases. If a(ñk ) ≤ ñk−1 , then Ak ζ1ñk + Bk ζ2ñk = βc + βl µñk −a(ñk ) + V (a(ñk )) − D − H · θñk . (4.34) 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 98 If ñk−1 < a(ñk ) ≤ ñk − 1, then Ak ζ1ñk − a(ñ ) ζ1 k + Bk ζ2ñk − a(ñ ) ζ2 k = βc + βl µñk −a(ñk ) − H θñk − θa(ñk ) . (4.35) The values of Ak and Bk can be solved from (4.33) together with either (4.34) or (4.35). Solution is in Closed-Form: We note that A0 and B0 can be expressed in closed- form, and Ak and Bk for all k can also be expressed in closed-form by substituting (4.21) into (4.33) and (4.34) where necessary. Therefore, (4.21) is a closed-form solution for all d ∈ [0, N ]. Numerically, we can find V (d) for all d ∈ [0, N ] in O(N ) time. 4.4.2.3 Algorithm for Finding the Optimal Policy Standard approaches to solving for the optimal policy of an MDP include value iteration and policy iteration [41, Chapter 6]. Value iteration finds the optimal policy from Bellman’s equation (4.3) iteratively, which may require a large number of iterations before converging to the optimal result. Policy iteration generally requires a smaller number of iterations, because, in each iteration, it finds the exact values of the discounted sum cost V (d) for the policy resulting from the previous iteration, and performs the iteration based on the exact V (d) values. However, in general, the exact V (d) values are found by solving a system of linear equations, which has a time-complexity of O(N 3 ) when using Gaussian-elimination. We propose a modified policy-iteration approach for finding the optimal policy, which uses the above result instead of Gaussian-elimination to compute V (d), and also only checks for migrating to lower states or not migrating (according to Proposition 4.1). The algorithm is shown in Algorithm 4.2, where Lines 4–9 find the values 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 99 Algorithm 4.2 Modified policy-iteration algorithm based on difference equations 1: Initialize a(d) = 0 for all d = 0, 1, 2, ..., N 2: Find constants φ0 , φ1 , φ2 , φ3 , φ4 , ζ1 , ζ2 , D, and H 3: repeat 4: k←0 5: for d = 1...N do 6: if a(d) 6= d then 7: ñk ← d, k ← k + 1 8: end if 9: end for 10: for all ñk do 11: if k = 0 then 12: Solve for A0 and B0 from (4.31) and (4.32) 13: Find V (d) with 0 ≤ d ≤ ñk from (4.21) with A0 and B0 found above 14: else if k > 0 then 15: if a(ñk ) ≤ ñk−1 then 16: Solve for Ak and Bk from (4.33) and (4.34) 17: else 18: Solve for Ak and Bk from (4.33) and (4.35) 19: end if 20: Find V (d) with ñk−1 < d ≤ ñk from (4.21) with Ak and Bk found above 21: end if 22: end for 23: for d = 1...N do 24: aprev (d) = a(d) o n P P V (j) 25: a(d) = arg mina≤d Ca (d) + γ a+1 aj j=a−1 26: end for 27: until aprev (d) = a(d) for all d 28: return a(d) for all d of ñk , Lines 10–22 find the discounted sum cost values, and Lines 23–26 update the optimal policy. The overall time-complexity for each iteration is O (N 2 ) in Algorithm 4.2, which reduces time-complexity because standard5 policy iteration has complexity O(N 3 ), and the standard value iteration approach does not compute the exact value function in each iteration and generally has long convergence time. 5 We use the term “standard” here to distinguish from the modified policy iteration mechanism proposed in Algorithm 4.2. 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 100 (3 0) (3,0) (3,17)) ((3,17) ( (3,16) ) (3,15) (3,1)) ((3,1) ( ) (2,0) (2 11) (2,11) (2 10) (2,10) (3 14) (3,14) (1 0) (1,0) (1 5) (1,5) (2,9)) ((2,9) (3,13)) ((3,13) (2 2) (2,2) (0,0)) ((0,0) (2 8) (2,8) (3 4)) ((3,4) (2,3) ((2,3) 3) (1 2) (1,2) (1 3) (1,3) (2 7) (2,7) (3 11) (3,11) (3 3) (3,3) (1 1)) ((1,1) (1 4) (1,4) (3 12) (3,12) (3,2) (2 1) (2,1) (3 5) (3,5) (2 4) (2,4) (2 5) (2,5) (2,6)) ((2,6) (3,10)) ((3,10) (3 6) (3,6) (3,7)) ((3,7) (3,8) (3 9) (3,9) Figure 4.9: Example of 2-D offset model on hexagon cells, where N = 3. 4.4.3 Approximate Solution for 2-D Mobility Model In this section, we show that the distance-based MDP can be used to find a nearoptimal service migration policy, where the user conforms to a uniform 2-D random walk mobility model in an infinite space. This mobility model can be used as an approximation to real-world mobility traces (see Section 4.4.4). We consider a hexagonal cell structure, but the approximation procedure can also be used for other 2-D mobility models (such as Manhattan grid) with some parameter changes. The user is assumed to transition to one of its six neighboring cells at the beginning of each timeslot with probability r ≤ 61 , and stay in the same cell with probability 1 − 6r. 4.4.3.1 Offset-Based MDP Similar to 4.3, we define the offset of the user from the service as a 2-D vector e(t) = u(t) − h(t) (recall that u(t) and h(t) are also 2-D vectors). Due to the spacehomogeneity of the mobility model, it is sufficient to model the state of the MDP by e(t) rather than s(t). The distance metric kϕ1 − ϕ2 k is defined as the minimum number of hops that are needed to reach cell ϕ2 from cell ϕ1 on the hexagon model. We name the states with the same value of ke(t)k as a ring, and express the states 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 101 {e(t)} with polar indices (i, j), where the first index i refers to the ring index, and the second index j refers to each of the states within the ring, as shown in Fig. 4.9. For e(t) = (i, j), we have ke(t)k = i. If u(t) = h(t) (i.e., the actual user and service locations (cells) are the same), then we have e(t) = (0, 0) and ke(t)k = 0. As in the distance-based MDP, we assume in the 2-D MDP that we always migrate when ke(t)k ≥ N , where N is a design parameter, and we only consider the state space {e(t)} with ke(t)k ≤ N . The system operates in the intermediate state e0 (t) = u(t) − h0 (t) = a(e(t)) after taking action a(e(t)). The next state e(t + 1) is determined probabilistically according to the transition probability Pe0 (t),e(t+1) . We have Pe0 (t),e(t+1) = 1 − 6r when e(t + 1) = e0 (t); Pe0 (t),e(t+1) = r when e(t + 1) is a neighbor of e0 (t); and Pe0 (t),e(t+1) = 0 otherwise. Note that we always have e(t) − e0 (t) = h0 (t) − h(t), so the one-timeslot cost is Ca (e(t)) = cm (ke(t) − e0 (t)k) + cd (ke0 (t)k). We note that, even after simplification with the offset model, the 2-D offset-based MDP has a significantly larger number of states compared with the distance-based MDP, because for a distance-based model with N states (excluding state zero), the 2-D offset model has 3N 2 + 3N states (excluding state (0, 0)). Therefore, we use the distance-based MDP proposed in Section 4.4.2 to approximate the 2-D offsetbased MDP, which significantly reduces the computational time as shown in Section 4.4.3.4. 4.4.3.2 Approximation by Distance-based MDP In the approximation, the parameters of the distance-based MDP are chosen as p0 = 6r, p = 2.5r, and q = 1.5r. The intuition behind the parameter choice is that, at state (i00 , j00 ) = (0, 0) in the 2-D MDP, the aggregate probability of transitioning to any state in ring i1 = 1 is 6r, so we set p0 = 6r; at any other state (i00 , j00 ) 6= (0, 0), the aggregate probability of transitioning to any state in the higher ring i1 = i00 + 1 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 102 is either 2r or 3r, and the aggregate probability of transitioning to any state in the lower ring i1 = i00 − 1 is either r or 2r, so we set p and q to the median value of these transition probabilities. To find the optimal policy for the 2-D MDP, we first find the optimal policy for the distance-based MDP with the parameters defined above. Then, we map the optimal policy from the distance-based MDP to a policy for the 2-D MDP. To explain this mapping, we note that, in the 2-D hexagon offset model, there always exists at least one shortest path from any state (i, j) to an arbitrary state in ring i0 , the length of this shortest path is |i − i0 |, and each ring between i and i0 is traversed once on the shortest path. For example, one shortest path from state (3, 2) to ring i0 = 1 is {(3, 2), (2, 1), (1, 0)}. When the system is in state (i, j) and the optimal action from the distance-based MDP is a∗ (i) = i0 , we perform migration on the shortest path from (i, j) to ring i0 . If there exist multiple shortest paths, one path is arbitrarily chosen. For example, if a(3) = 2 in the distance-based MDP, then we have either a(3, 2) = (2, 1) or a(3, 2) = (2, 2) in the 2-D MDP. With this mapping, the one-timeslot cost Ca (d(t)) for the distance-based MDP and the one-timeslot cost Ca (e(t)) for the 2-D MDP are the same, because the migration distances in the distance-based MDP and 2-D MDP are the same (thus same migration cost) and all states in the same ring i0 = ke0 (t)k = d0 (t) have the same transmission cost cd (ke0 (t)k) = cd (d0 (t)). 4.4.3.3 Bound on Approximation Error Error arises from the approximation because the transition probabilities in the distance-based MDP are not exactly the same as that in the 2-D MDP (there is at most a difference of 0.5r). In this subsection, we study the difference in the discounted sum costs when using the policy obtained from the distance-based MDP and the true optimal policy for the 2-D MDP. The result is summarized as 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 103 Proposition 4.5. Proposition 4.5. Let Vdist (e) denote the discounted sum cost when using the policy from the distance-based MDP, and let V ∗ (e) denote the discounted sum cost when using true optimal policy of the 2-D MDP, then we have Vdist (e) − V ∗ (e) ≤ γrk 1−γ for all e, where k , maxx {cm (x + 2) − cm (x)}. Proof. Details of the proof are presented in Appendix C.3. An outline is given here. The proof is completed in three steps. First, we modify the states of the 2-D MDP in such a way that the aggregate transition probability from any state (i00 , j00 ) 6= (0, 0) to ring i1 = i00 + 1 (correspondingly, i1 = i00 − 1) is 2.5r (correspondingly, 1.5r). We assume that we use a given policy on both the original and modified 2-D MDPs, and show a bound on the difference in the discounted sum costs for these two MDPs. In the second step, we show that the modified 2-D MDP is equivalent to the distancebased MDP. This can be intuitively explained by the reason that the modified 2-D MDP has the same transition probabilities as the distance-based MDP when only considering the ring index i, and also, the one-timeslot cost Ca (e(t)) only depends on ke(t) − a(e(t))k and ka(e(t))k, both of which can be determined from the ring indices of e(t) and a(e(t)). The third step uses the fact that the optimal policy for the distance-based MDP cannot bring a higher discounted sum cost for the distancebased MDP (and hence the modified 2-D MDP) than any other policy. By utilizing the error bound found in the first step twice, we prove the result. The error bound is a constant value when all the related parameters are given. It increases with γ. However, the absolute value of the discounted sum cost also increases with γ, so the relative error can remain low. 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 4.4.3.4 104 Numerical Evaluation The error bound derived in Section 4.4.3.3 is a worst-case upper bound of the error. In this subsection, we evaluate the performance of the proposed approximation method numerically, and focus on the average performance of the approximation. We consider 2-D random walk mobility with randomly chosen parameter r. The maximum user-service distance is set as N = 10. The transmission cost function parameters are selected as θ = 0.8, δc = 1, and δl = −1. With these parameters, we have δc + δl = 0, which means that there is no constant portion in the cost function. For the migration cost, we choose µ = 0.8 and fix βc + βl = 1 to represent a constant server processing cost for migration. The parameter βl ≤ 0 takes different values in the simulations, to represent different sizes of data to be migrated. Similar to Section 4.3.5, we perform simulations in MATLAB on a computer with Intel Core i7-2600 CPU, 8GB memory, and 64-bit Windows 7. We study the computation time, the number of FLOPs, and the discounted sum cost of the proposed approach that is based on approximating the original 2-D MDP with the distance-based MDP. For the computation time and number of FLOPs, standard value and policy iteration approaches [41, Chapter 6] are used to solve the original 2-D MDP for comparison. The value iteration procedure terminates when the difference in the discounted sum cost between two iterations is smaller than 0.1 or when a maximum number of iterations has been reached. The maximum number of iterations is set to 5, which is a relatively small number in order to avoid excessive simulation time. We will see later that even with this restriction on maximum number of iterations, value iteration performs worse than standard policy iteration in terms of both computation time and number of FLOPs. The standard policy iteration procedure terminates when the optimal actions from two consecutive iterations are the same, and it always returns the true optimal policy and its corresponding cost values. 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 105 The discounted sum cost from the proposed approach is compared with the costs from alternative policies, including the true optimal policy from standard policy iteration on the 2-D model, the never-migrate policy which never migrates except when at states in ring i ≥ N (in which case the service is migrated to the current location of the user), the always-migrate policy which always migrates to the current user location when the user and the service are at different locations, and the myopic policy that chooses actions to minimize the one-timeslot cost. The simulations are run with 50 different random seeds, which generate 50 different values of r, and the overall results are shown in Figs. 4.10–4.12 with different values of the discount factor γ. Reduction in Time-Complexity: Figs. 4.10–4.12 show that the computation time of the proposed method is only about 0.1% of that of standard value or policy iteration. This time reduction is explained as follows. As discussed in Section 4.4.3.2, for a distance-based MDP with N states (excluding state zero), the 2-D MDP has N2-D , 3N 2 + 3N states (excluding state (0, 0)). When we ignore the complexity of matrix inversion6 in the policy iteration procedure, the standard value and policy it2 eration approaches on the 2-D MDP have a complexity of O(N2-D ) in each iteration, because the optimal action needs to be found for each state, which requires enumerating all the states and all possible actions for each state (similar to Lines 23–26 of Algorithm 4.2). In the simulations, N = 10, so we have N2-D = 330. Recall that the complexity of Algorithm 4.2 used in the proposed approach is O(N 2 ), so the ratio of the computational complexities of different approaches can be approximated by 2 N2-D N2 ≈ 103 . Therefore, the standard value and policy iteration approaches consume about 103 times more computation time compared to the proposed approach. When considering the number of FLOPs, the magnitude of reduction of the pro6 We ignore the complexity of matrix inversion here because MATLAB performs matrix operations significantly faster than other operations, as discussed in Section 4.3.5. 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 106 2 Computation time (s) 10 0 10 Proposed Policy iteration Value iteration −2 10 −4 10 0 0.5 1 1.5 2 10 10 Number of FLOPs 8 10 Proposed Policy iteration Value iteration 6 10 4 10 2 10 0 0.5 1 1.5 2 Discounted sum cost 2.5 2 1.5 Proposed Optimal Never migrate Always migrate Myopic 1 0.5 0 0 0.5 1 1.5 2 Migration cost parameter, −βl Figure 4.10: Simulation result for 2-D random walk with γ = 0.5. 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 107 2 Computation time (s) 10 0 10 Proposed Policy iteration Value iteration −2 10 −4 10 0 1 2 3 4 5 6 7 8 10 Number of FLOPs 10 8 10 Proposed Policy iteration Value iteration 6 10 4 10 2 10 0 1 2 3 4 5 6 7 8 4 5 6 7 8 Discounted sum cost 20 Proposed Optimal Never migrate Always migrate Myopic 15 10 5 0 0 1 2 3 Migration cost parameter, −βl Figure 4.11: Simulation result for 2-D random walk with γ = 0.9. 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 108 2 Computation time (s) 10 Proposed Policy iteration Value iteration 0 10 −2 10 −4 10 0 5 10 15 20 10 Number of FLOPs 10 8 10 Proposed Policy iteration Value iteration 6 10 4 10 2 10 0 5 10 15 20 10 15 20 Discounted sum cost 250 Proposed Optimal Never migrate Always migrate Myopic 200 150 100 50 0 0 5 Migration cost parameter, −βl Figure 4.12: Simulation result for 2-D random walk with γ = 0.99. 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 109 posed method compared to standard policy and value iteration approaches ranges between 104 and 105 . This reduction is larger than the reduction in computation time. Reduction in computation time is smaller presumably because of some running time optimization mechanisms applied in MATLAB for performing matrix operations (as discussed in Section 4.3.5) and perhaps also some other operations. Note that the standard policy iteration includes a matrix inversion operation, which has a com3 plexity of O(N2-D ) when using Gaussian elimination. This brings an approximate complexity ratio of 3 N2-D N2 ≈ 105 . Different from Section 4.3.5, we no longer see a substantial change in the number of FLOPs under different values of |βl |, because our proposed approach here does not include matrix inversion, and the transition probability matrix of the 2-D MDP is less well-structured compared to that of the 1-D MDP (in Section 4.3) so presumably no efficient matrix inversion mechanism exists in this case. Another aspect different from Section 4.3.5 is that, here in Figs. 4.10–4.12, we do not see a substantial change in the time-complexity of value iteration under different values of γ. The main reason is that we only allow at most 5 iterations in the value iteration approach here as discussed earlier in this section. This is mainly to avoid excessive time we need to run the simulation, because the 2-D scenario is much more complex than the 1-D scenario. Near-Optimal Cost: We can also see from Figs. 4.10–4.12 that the proposed method yields a discounted sum cost that is very close to the optimal cost. The results also provide several insights into the performance of the baseline policies, which are similar to our observations in the 1-D scenario in Section 4.3.5. Specifically, the cost of the always-migrate policy approximates the optimal cost when |βl | is small, and the cost of the never-migrate policy approximates the optimal cost when |βl | is large. This is because, when |βl | is small, the migration 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 110 cost is relatively small, and migration can be beneficial for most cases; when |βl | is large, the migration cost is large, and it is better not to migrate in most cases. The myopic policy is the same as the never-migrate policy when |βl | ≥ 0.5, because cm (x) and cd (y) are both concave according to the simulation settings and we always have cm (x) ≥ cd (y) when |βl | ≥ 0.5, where we recall that the myopic policy does not consider the future impact of actions. There is an intersection of the costs from never-migrate and always-migrate policies. When |βl | takes values that are larger than the value at the intersection point, the optimal cost is close to the cost from the never-migrate policy when γ is small, and the gap becomes larger with a larger γ. The reason is that the benefits of migration become more significant when we look farther ahead into the future. We also note that the cost of never-migrate policy slightly increases as |βl | increases, because the never-migrate policy also occasionally migrates when the user-service distance is greater than or equal to N (see earlier definition). 4.4.4 Application to Real-World Scenarios In this section, we discuss how the aforementioned approaches can be applied to service migration in the real world, where multiple users and services co-exist in the cloud system. We note that in practical scenarios, MMCs may not be deployed at every basestation, and each MMC may have a capacity limit that restricts the number of services it can host. Theoretically, it is still possible to formulate the service migration problem with these additional constraints as an MDP. However, the resulting MDP will have a significantly larger state space than our current model, and it is far more difficult to solve or approximate this new MDP. While we leave the theoretical analysis of this new MDP as future work, we propose a heuristic approach in this section to handle these additional constraints. The proposed approach is largely based on the results and intuitions obtained in previous sections. The 2-D MDP 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 111 approximation approach proposed in Section 4.4.3.2 is used as a subroutine in the heuristic scheme, and the distance-based MDP resulting from the approximation is solved using Algorithm 4.2. 4.4.4.1 Mapping between Real-World and MDP-Model The mapping between the real-world and the MDP model is discussed as follows. MMC Controller: We assume that there exists a control entity which we refer to as the MMC controller. The MMC controller does not need to be a separate cloud entity. Rather, it can be a service running at one of the MMCs. Basestations: Each basestation (which may or may not have an MMC connected to it) is assumed to have some basic capability of keeping records on arriving and departing users as well as performing simple monitoring and computational operations. Timeslots: The physical time length corresponding to one timeslot in the MDP model is a pre-specified parameter, which is a constant for ease of presentation. This parameter can be regarded as a protocol parameter, and it is not necessary for all basestations to precisely synchronize on individual timeslots. Transition Probability: The transition probability parameter r is estimated from the sample paths of multiple users, using the procedure described in Section 4.4.4.2 below. We define a window length Tw ≥ 1 (represented as the number of timeslots), which specifies the amount of timeslots to look back to estimate the parameter r. We consider the case where r is the same across the whole geographical area, which is a reasonable assumption when different locations within the geographical area under consideration have similarities (for example, they all belong to an urban area). More 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 112 sophisticated cases can be studied in the future. Cost Parameters: The cost parameters βc , βl , µ, δc , δl , and θ are selected based on the actual application scenario, and their values may vary with the background traffic load of the network and MMC servers. Discount Factor: The discount factor γ can be selected based on the duration of services, and a larger γ corresponds to a longer service duration. Such a selection is because the discount factor γ determines the amount of time to look ahead, if a user only requires the service for a short time, then there is no need to consider the cost for the long-term future. For ease of presentation, we set γ as a constant value. In practice, the value of γ can be different for different users or services. Policy Update Interval: A policy update interval Tu is defined (represented as the number of timeslots), at which a new migration policy is computed by the MMC controller. 4.4.4.2 Overall Procedure The data collection, estimation, and service placement procedure is described below. 1. At the beginning of each timeslot, the following is performed: (a) Each basestation obtains the identities of its associated users. Based on this information, the basestation computes the number of users that have left the cell (compared to the beginning of the previous timeslot) and the total number of users that are currently in the cell. This information is saved for each timeslot for the duration of Tw and will be used in step 2b. (b) The MMC controller collects information on currently active services on each MMC, computes the new placement of services according to the 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 113 procedure described in Section 4.4.4.4, and sends the resulting placement instructions to each MMC. The placements of all services are updated based on these instructions. 2. At every interval Tu , the following is performed: (a) The MMC controller sends a request to all basestations to collect the current statistics. (b) After receiving the request, each basestation n computes the empirical probability of users moving outside of the cell as t−1 1 X m0n (τ ) fn = Tw τ =t−T mn (τ ) (4.36) w where the total number of users that are associated to basestation n in slot τ is mn (τ ), among which m0n (τ ) users have disconnected from basestation n at the end of slot τ and these users are associated to a different basestation in slot τ + 1; and t denotes the current timeslot index. These empirical probabilities are sent together with other monitored information, such as the current load of the network and MMC server (if the basestation has an MMC connected to it), to the MMC controller. (c) After the controller receives responses from all basestations, it performs the following: i. Compute the transmission and migration cost parameters βc , βl , µ, δc , δl , and θ based on the measurements at basestations. ii. Compute the average of empirical probabilities fn by f= 1 X fn NBS n∈N BS (4.37) 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 114 where NBS is the set of basestations and NBS = |NBS | is the total number of basestations. Then, estimate the parameter r by r̂ = f /6. (4.38) iii. In the distance-based MDP, set p0 = 6r̂, p = 2.5r̂, and q = 1.5r̂ (as discussed in Section 4.4.3.2), compute and save the optimal distance-based policy from Algorithm 4.2. Also save the estimated cost parameters and the optimal discounted sum costs V ∗ (d) for all distances d for later use. Remark: In the procedure presented above, we have assumed that mn (τ ) 6= 0 for all n and τ . This is only for ease of presentation. When there exist some n and τ such that mn (τ ) = 0, we can simply ignore those terms (set the corresponding terms to zero) in the sums of (4.36) and (4.37), and set the values of Tw and NBS to the actual number of terms that are summed up in (4.36) and (4.37), respectively. This simplification does not affect our analysis below if we make a similar substitution in the analysis for cases with mn (τ ) = 0. Thus, we still assume that mn (τ ) 6= 0 for all n and τ in the discussion presented next. 4.4.4.3 Discussion on the Estimation of Parameter r As introduced in Section 4.4.3, at every timeslot, each user randomly moves to one of its neighboring cells with probability r and stays in the same cell with probability 1 − 6r. In the real world, the parameter r is unknown a priori and needs to be estimated based on observations of user movement. Equations (4.36)–(4.38) in the above procedure serve for this estimation purpose, and the resulting r̂ is an estimator of r. In the following, we analyze some statistical properties of r̂ and discuss the 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 115 rationale for using such an estimation approach. We note that the mobility model presented in Section 4.4.3 is for an infinite 2-D space with an infinite number of cells. In reality, the number of cells is finite. We assume in our analysis that each user stays in the same cell with probability 1 − 6r (r ≤ 16 ) and moves out of its current cell with probability 6r, no matter whether the cell is at the boundary (such as cells in the outer ring i = 3 in Fig. 4.9) or not. When a cell is at the boundary, its transition probability to each of its neighboring cells is larger than r, because it has less than six neighbors. For example, in Fig. 4.9, a user in cell (3, 0) moves to each of its neighboring cells (including (3, 17), (2, 0), (3, 1)) with probability 2r, and the total probability of moving out of the cell is still 6r. We also assume that the mobility patterns of different users are independent of each other. Proposition 4.6. Assume that each user follows 2-D random walk (defined above) with parameter r, then r̂ is an unbiased estimator of r, i.e., E {r̂} = r. Proof. We note that in (4.36), mn (τ ) and m0n (τ ) are both random variables respectively representing the total number of users associated to basestation (located in cell) n in slot τ and the number of users that have moved out of cell n at the end of slot τ . The values of mn (τ ) and m0n (τ ) are random due to the randomness of user mobility. According to the mobility model, each user moves out of its current cell at the end of a timeslot with probability 6r, where we recall that r ≤ 1 6 by definition. Hence, under the condition that mn (τ ) = k, m0n (τ ) follows the binomial distribution with parameters k and 6r. Thus, the conditional probability k k−k0 k0 Pr {m0n (τ ) = k 0 |mn (τ ) = k} = (6r) (1 − 6r) k0 (4.39) 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 116 for all 0 ≤ k 0 ≤ k. From the expression of the mean of binomially distributed random variables, we know that the conditional expectation E {m0n (τ )|mn (τ )} = 6r · mn (τ ). (4.40) Combining (4.36)–(4.38), we have r̂ = 1 X 6NBS Tw n∈NBS t−1 X m0n (τ ) . mn (τ ) τ =t−T (4.41) w We then have ) t−1 X m0n (τ ) E {r̂} = E 6NBS Tw n∈N τ =t−T mn (τ ) w BS t−1 X X E {m0n (τ )|mn (τ )} 1 E = 6NBS Tw n∈N τ =t−T mn (τ ) w BS t−1 X X 1 6r · mn (τ ) = E 6NBS Tw n∈N τ =t−T mn (τ ) ( 1 X BS = r NBS Tw w NBS Tw =r where the second equality follows from the law of iterated expectations. The fact that r̂ is an unbiased estimator of r justifies our estimation approach, which intuitively means that the long-term average of the estimated value r̂ should not be too far away from the true value of r. We analyze the variance of the estimator next. Such analysis is not very easy due to the dependency among different random variables. To make the analysis theoretically tractable, we introduce the following assumption. 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 117 Assumption 4.1. We assume that m0n (τ ) is independent of mn (τ̃ ), mñ (τ ), mñ (τ̃ ), m0n (τ̃ ), m0ñ (τ ), and m0ñ (τ̃ ) (where ñ 6= n and τ̃ 6= τ ) when mn (τ ) is given. Essentially, this assumption says that m0n (τ ) is only dependent on mn (τ ). In practice when the number of users is large, this assumption can become close to reality, because the locations of different users are independent of each other, and also because of the Markovian property which says that future locations of a user only depends on its present location (and independent of all past locations when the present location is given). We define M , {mn (τ ) : ∀n ∈ NBS , τ ∈ [t − Tw , t − 1]} as the set of number of users at all basestations and all timeslots considered in the estimation. Proposition 4.7. Assume that Assumption 4.1 is satisfied and each user follows 2-D random walk (defined at the beginning of Section 4.4.4.3) with parameter r. The variance of estimator r̂, under the condition that M is given, is r(1 − 6r) Var{r̂|M} = 2 6NBS Tw2 X t−1 X n∈NBS τ =t−Tw 1 mn (τ ) ! (4.42) where the conditional variance Var{r̂|M} is defined as Var{r̂|M} , E r̂2 M − (E {r̂|M})2 . Proof. Taking the conditional expectation on both sides of (4.41), we have ( E {r̂|M} = E = = 1 X 6NBS Tw 1 6NBS Tw 1 6NBS Tw n∈NBS X n∈NBS X ) t−1 X m0n (τ ) M mn (τ ) τ =t−T w t−1 X E {m0n (τ )|mn (τ )} mn (τ ) τ =t−T w t−1 X n∈NBS τ =t−Tw 6r · mn (τ ) mn (τ ) (4.43) 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility = r NBS Tw 118 NBS Tw =r (4.44) where the second equality is because m0n (τ ) is independent of mn (τ̃ ), mñ (τ ), and mñ (τ̃ ) (where ñ 6= n and τ̃ 6= τ ) when mn (τ ) is given, according to Assumption 4.1. We thus have (E {r̂|M})2 = r2 . We focus on evaluating E {r̂2 |M} in the following. From (4.41), we have 1 r̂2 = 2 36NBS Tw2 X n∈NBS t−1 X m0n (τ ) mn (τ ) τ =t−T !2 w 1 = 2 36NBS Tw2 t−1 X X (m0n (τ ))2 + 2 n∈NBS τ =t−Tw (mn (τ )) X n1 ,n2 ∈NBS ; τ1 ,τ2 ∈[t−Tw ,t−1]; n1 6=n2 and/or τ1 6=τ2 m0n1 (τ1 ) m0n2 (τ2 ) . · mn1 (τ1 ) mn2 (τ2 ) (4.45) We now consider the two parts in (4.45). From the proof of Proposition 4.6, we know that m0n (τ ) follows the binomial distribution when mn (τ ) is given. Further, when mn (τ ) is given, m0n (τ ) is independent of mn (τ̃ ), mñ (τ ), and mñ (τ̃ ) (where ñ 6= n and τ̃ 6= τ ), according to Assumption 4.1. Thus, we have ( E ) E (m0n (τ ))2 mn (τ ) (m0n (τ ))2 M = (mn (τ ))2 (mn (τ ))2 mn (τ ) · 6r · (1 − 6r) + 36r2 (mn (τ ))2 (mn (τ ))2 6r · (1 − 6r) = + 36r2 mn (τ ) = (4.46) where the second equality is a known result for binomially distributed random vari- 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 119 ables. We also have ) m0n1 (τ1 ) m0n2 (τ2 ) E · M mn1 (τ1 ) mn2 (τ2 ) ) ( ) ( m0n2 (τ2 ) m0n1 (τ1 ) =E mn (τ1 ) · E mn (τ2 ) mn1 (τ1 ) 1 mn2 (τ2 ) 2 E m0n1 (τ1 )|mn1 (τ1 ) E m0n2 (τ2 )|mn2 (τ2 ) = · mn1 (τ1 ) mn2 (τ2 ) ( = 36r2 (4.47) for n1 6= n2 and/or τ1 6= τ2 , where the first equality follows from the fact that m0n1 (τ1 ) and m0n2 (τ2 ) are independent when mn1 (τ1 ) and mn2 (τ2 ) are given (according to Assumption 4.1), the last equality follows from (4.40). We now take the conditional expectation on both sides of (4.45), and substitute corresponding terms with (4.46) and (4.47). This yields E r̂2 M 1 = 2 36NBS Tw2 X ( t−1 X E n∈NBS τ =t−Tw ) (m0n (τ ))2 M (mn (τ ))2 ( X + E n1 ,n2 ∈NBS ; τ1 ,τ2 ∈[t−Tw ,t−1]; n1 6=n2 and/or τ1 6=τ2 ) m0n1 (τ1 ) m0n2 (τ2 ) · M mn1 (τ1 ) mn2 (τ2 ) 1 = 2 36NBS Tw2 t−1 X X 6r · (1 − 6r) 2 + 36r + mn (τ ) n∈NBS τ =t−Tw 1 = 2 36NBS Tw2 X r(1 − 6r) = 2 6NBS Tw2 6r · (1 − 6r) · t−1 X n∈NBS τ =t−Tw X t−1 X n∈NBS τ =t−Tw 1 mn (τ ) 1 mn (τ ) X 36r 2 n1 ,n2 ∈NBS ; τ1 ,τ2 ∈[t−Tw ,t−1]; n1 6=n2 and/or τ1 6=τ2 ! ! 2 + NBS Tw2 · 36r2 ! + r2 . (4.48) 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 120 Subtracting (E {r̂|M})2 = r2 from the above yields the result. Proposition 4.7 gives the conditional variance of estimator r̂ when the values of mn (τ ) are given. It is not straightforward to remove the condition, because it is hard to find the stationary distribution of user locations in a hexagonal 2-D mobility model with a finite number of cells. We note that the set of mn (τ ) values represents the set of samples in our estimation problem. In standard estimation problems, the sample size is usually deterministic, while it is random in our problem due to random user locations. This causes the difficulty in finding the unconditional variance of our estimator. However, Proposition 4.7 is important because it gives us a sense on how large the gap between r̂ and r is, provided that each user precisely follows the random walk mobility model. We also have the following corollary which gives an upper bound of the unconditional variance. Corollary 4.3. Assume that Assumption 4.1 is satisfied and each user follows 2-D random walk (defined at the beginning of Section 4.4.4.3) with parameter r. The unconditional variance of estimator r̂ has the following upper bound: Var{r̂} ≤ 1 . 144NBS Tw (4.49) Proof. As discussed in Section 4.4.4.2, we assume that mn (τ ) 6= 0 for all n and τ . Thus, we always have 1 mn (τ ) ≤ 1 (since mn (τ ) is a positive integer) and X t−1 X n∈NBS τ =t−Tw 1 ≤ NBS Tw . mn (τ ) (4.50) 1 24 (4.51) We also have the following bound: r(1 − 6r) ≤ 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 121 for any r ∈ [0, 16 ]. The law of total variance gives Var{r̂} = E {Var{r̂|M}} + Var{E {r̂|M}} According to (4.44), E {r̂|M} = r which is a constant, thus Var{E {r̂|M}} = 0. Therefore, we have Var{r̂} = E {Var{r̂|M}} ≤ max Var{r̂|M} ≤ M 1 144NBS Tw where the last inequality follows from substituting (4.50) and (4.51) into (4.42). We see from Corollary 4.3 that the upper bound of variance is inversely proportional to Tw . This is an intuitive but also very favorable property, which says that when users follow an ideal random walk mobility model with parameter r, we can estimate the value of r as accurate as possible if we have a sufficient amount of samples. Different from many estimation problems where it is costly (requiring human participation, data communication, etc.) to obtain samples, it is not too difficult to collect samples in our case, because each basestation can save user records at every timeslot, and we can easily adjust the number of samples (proportional to Tw ) by changing the amount of timeslots to search back in the record. Therefore, unlike many other estimation problems where the goal is to minimize the variance of estimator under a given sample size, we do not target this goal here. A much more important issue in our problem is that the ideal random walk mobility model may not hold in practice. The parameter estimation procedure in Section 4.4.4.2 conceptually considers such potential model violations, and the rationale of this estimation approach is explained below. We see in the procedure that we first focus on a particular cell n and compute 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 122 the empirical probability of users moving out of that cell in (4.36), by treating each timeslot with equal weight. Then, we take the average of such empirical probabilities of all cells in (4.37). We note that the number of users in different cells and timeslots may be imbalanced. Thus, in (4.36), the empirical probabilities for different cells and slots may be computed with different number of users (samples), i.e., different values of mn (τ ). Suppose we think of an alternative approach which first computes the total numP Pt−1 ber of users in all cells and all slots (i.e., n∈NBS τ =t−Tw mn (τ )) and the aggregated total number of users leaving their current cells at the end of slots (i.e., P Pt−1 0 τ =t−Tw mn (τ )). Then, this approach estimates r by the overall empirical n∈NBS P probability of users moving out of their current cells (i.e., 6· n∈NBS P Pt−1 n∈NBS τ =t−Tw Pt−1 m0n (τ ) τ =t−Tw mn (τ ) ). Intuitively, this alternative estimator may bring a lower variance compared to the current estimator r̂, because it treats all the samples as a whole. However, note that the uniform random walk mobility model may not hold precisely in practice; even if it holds, the parameter r may be time-varying or different in different geographical areas. Therefore, the proposed approach computes the empirical probability for each cell in each slot first, so that different cells and slots are equally weighted in the estimation, although some cells may have more users than others in certain slots. This is for the consideration of fairness among different cells, so that the performance at different cells remains similar. It is also to avoid statistics at a single slot dominating the overall result. Since r may be time-varying in practice, it is possible that single-slot statistics do not represent the overall case. 4.4.4.4 Service Placement Update The service placement is updated at the beginning of every timeslot. For each service, if its new location is different from its previous location, a migration of this service is carried out. 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 123 The policy found from the MDP model specifies which cell to migrate to when the system is in a particular state (u(t), h(t)). However, we may not be able to apply the policy directly, because not every basestation has an MMC connected to it and each MMC has a capacity limit. We may need to make some modifications to the service placement specified by the policy, so that the practical constraints are not violated. The MDP model also does not specify where to place the service if it was not present in the system before. In the following, we present a method to determine the service placement with these practical considerations, which is guided by the optimal policy obtained from the MDP model and at the same time satisfies the practical constraints. In the proposed procedure, we first ignore the MMC capacity limit and repeat the process presented in steps I and II below for every service. Then, we take into account the capacity limit, and reassign the locations of some services (in step III) that were previously assigned to an MMC whose capacity is exceeded. (I) Initial Service Placement: When the service was not running in the system before (i.e., it is being initialized), the service is placed onto an MMC that has the smallest distance to the user. The intuition for this rule is that the initialization cost and the cost of further operation is usually small with such a placement. (II) Dynamic Service Migration: When the service has been initialized earlier and is currently running in the system, a decision on whether and where to migrate the service is made. Without loss of generality, assume that the current timeslot is t = 0. We would like to find an action a that is the solution to the following problem: a(d(0))+1 min Ca (d(0)) + γ a X Pa(d(0)),d(1) V ∗ (d(1)) d(1)=a(d(0))−1 (4.52) 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 124 s.t. there exists an MMC such that the user-service distance is a(d(0)) after migration where V ∗ (d) stands for the optimal discounted sum cost found from step 2(c)iii in Section 4.4.4.2. We note that (4.52) is a one-step value iteration following the balance equation (4.20). Intuitively, it means that assuming the optimal actions are taken in future slots, find the action for the current slot that incurs the lowest discounted sum cost (including both immediate and future cost). When all basestations have MMCs connected to them, the solution a to problem (4.52) is the same as the optimal action found from step 2(c)iii in Section 4.4.4.2. However, a may be different from the optimal action when some basestations are not connected to MMCs, because there may not exist an MMC that satisfies the distance specification according to action a. The resulting distance-based migration action can be mapped to a migration action on 2-D space using the procedure in Section 4.4.3.2. (III) Relocate Service(s) if MMC’s Capacity Exceeded: The placement decisions in steps I and II above do not consider the capacity limit of each MMC, so it is possible that we find the MMC capacity is exceeded after following the steps in the above sections. When this happens, we start with an arbitrary MMC (denoted by i0 ) whose capacity constraint is violated. We rank all services in this MMC according to the objective function in (4.52), and start to relocate the service with the highest objective function value. This service is placed on an MMC that still has capacity for hosting it, where the placement decision is also made according to (4.52) but only the subset of MMCs that are still capable of hosting this service are considered. The above process is repeated until the number of services hosted at MMC i0 is within its capacity limit. Then, this whole process is repeated for other MMCs that have violated capacity 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 125 constraints. Remark: We note that service relocation does not really occur in the system. It is only an intermediate step in the process of finding new service locations. We use this two-step approach involving temporary placement and relocation instead of an alternative one-step approach that checks for MMC capacity when performing the placement/migration in steps I and II, because with such a two-step approach, we can leave the low-cost services within the MMC and move high-cost services to an alternative location. 4.4.4.5 Trace-Driven Simulation We perform simulation with real-world mobility traces of 536 taxis in San Francisco, collected on May 31, 2008 [45, 46], where different number of taxis are operating (i.e., active) at different time of the day. Each active taxi is modeled as a user that requests a service that is independent of the services of the other taxis. A hexagonal cell structure with 500 m cell separation and NBS = 331 cells (basestations) in total is assumed and each taxi is assumed to be connected to the nearest basestation. We set the physical time length for each timeslot as 60 s. The parameters for data collection and policy update are set as Tu = 1 slot, and Tw = 60 slots. We choose N = 10, µ = θ = 0.8, and γ = 0.9. From Corollary 4.3, we know that the standard deviation of estimator r̂ (for an ideal random walk mobility model) is upper-bounded by q 1 = 0.00059, which is reasonably small. Unless otherwise specified, there 144NBS Tw are 100 basestations that have MMCs connected to them and each MMC can host at most 50 services. The basestations with MMCs are evenly distributed among all basestations. Note that the locations of taxis in the dataset are unevenly distributed and the density of taxis can be very high in some areas at certain times of the day. Similar to Section 4.4.3.4, we compare the performance of the proposed method with baseline policies including always/never-migrate and myopic. To cope with 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 126 the nature of the policies, the objective function in (4.52) is modified accordingly for these baseline policies. The objective in (4.52) is defined as the user-service distance for the always- or never-migrate policies (recall that migration also happens with never-migrate policy when the user-service distance is N or larger), and it is defined as the one-timeslot cost for the myopic policy. Cost definition: It is assumed that the system load is proportional to the number of taxis in operation, and we define parameters Rt and Rp as weighting factors respectively for transmission bandwidth and MMC processing resources. Then, we . . mcur cur define Gt , 1 1 − Rm and G , 1 1 − , where mcur denotes the p Rp mmax t mmax number of taxis in operation at the time when the optimal policy is being computed, and mmax denotes the maximum number of taxis that may simultaneously operate at any time instant in the considered dataset. The expressions for Gt and Gp have the same form as the average queuing delay expression in queuing theory [47], and they mimic the delay of data transmission (Gt ) and processing (Gp ). The cost parameters are then defined as βc = Gp + Gt , βl = −Gt , δc = Gt , and δl = −Gt . With such a definition, we have βc + βl = Gp and δc + δl = 0, which means that the constant part of the migration cost is Gp (to represent the processing cost for migration) and there is no constant part in the cost for data transmission. We set βl = δl = Gt because this part of cost can be regarded as related to data transmission in both cm (x) and cd (y). Results: The instantaneous cost in each time slot (i.e., the Ca (s(t)) values) averaged over all users that are active in that slot is collected and shown in Fig. 4.13. The fluctuation in cost values is due to different system load over the day, and we can see that the proposed method almost always outperforms the baseline approaches. To consider the overall performance under different parameter settings, we denote the cost of the proposed method as C and the cost of the baseline method under 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility 127 Actual cost from traces 3 2.5 2 Never Migrate (A) 1.23 Always Migrate (B) 1.22 (B) Myopic (C) 1.21 Proposed (D) (Average values) (C) (A) 1.5 1 (D) 0.5 00:00 06:00 12:00 18:00 00:00 Time Figure 4.13: Instantaneous average cost per user in each timeslot over a day in tracedriven simulation, where Rt = Rp = 1.5. An enlarged plot of the circled area is shown on the top-right of the plot. The arrows annotated with (A), (B), (C), and (D) point to the average values over the whole day of the corresponding policy. comparison as C0 , and we define the cost reduction as (C0 − C)/C0 . Fig. 4.14 shows the cost reductions (averaged over the entire day) under different parameter settings. From Figs. 4.14(a)–(d), we can see that under the default number of MMCs and capacity limit at each MMC, the proposed approach is beneficial with cost reductions ranging from 9% to 44% compared to the never/always-migrate or myopic policies. From Figs. 4.14(e)–(f), we see that the cost reductions compared to never-migrate and myopic policies become small in the case where either the number of MMCs is small or the capacity of each MMC is low. In this case, it is hardly possible to migrate the service to a better location because the action space for migration is very small. Therefore, the proposed approach gives a similar cost as baseline policies, but still outperforms them as indicated by a positive cost reduction on average. The cost reduction compared to the always-migrate policy becomes slightly higher in the case of small action space, because the always-migrate policy always incurs migration cost even though the benefit of such migration is not obvious. The fact that costs fluctuate over the day and vary with different amount of avail- 4.4. Constant-Plus-Exponential Cost Model under 2-D Mobility R =1.1 R =1.5 t t 0.7 0.7 Over never migrate Over always migrate Over myopic Over never migrate Over always migrate Over myopic 0.6 0.5 Cost reduction Cost reduction 0.6 0.4 0.3 0.2 0.1 0 128 0.5 0.4 0.3 0.2 0.1 1.2 1.4 1.6 1.8 0 2 1.2 1.4 1.6 Rp 1.8 2 Rp (a) (b) Rp=1.1 Rp=1.5 0.6 Over never migrate Over always migrate Over myopic Over never migrate Over always migrate Over myopic 0.5 0.6 Cost reduction Cost reduction 0.8 0.4 0.2 0.4 0.3 0.2 0.1 0 1.2 1.4 1.6 1.8 0 2 1.2 1.4 1.6 Rt (c) Rt=Rp=1.5 Rt=Rp=1.5 0.6 Over never migrate Over always migrate Over myopic 0.5 Cost reduction 0.3 Cost reduction 2 (d) 0.4 0.2 0.1 Over never migrate Over always migrate Over myopic 0 −0.1 1.8 Rt 0 50 100 150 Number of cells with MMC (e) 0.4 0.3 0.2 0.1 0 200 −0.1 0 50 100 150 200 Maximum number of services per MMC (f) Figure 4.14: Cost reduction (averaged over the entire day) compared to alternative policies in trace-driven simulation, the error bars denote the standard deviation (where we regard the instantaneous cost at different time of the day as samples): (a)– (b) cost reduction vs. different Rp , (c)–(d) cost reduction vs. different Rt , (e) cost reduction vs. different number of cells with MMC, (f) cost reduction vs. different capacity limit of each MMC (expressed as the maximum number of services allowed per MMC). 4.5. Discussion 129 able resources (which is adjusted by Rt and Rp ) also implies that it is necessary to compute the optimal policy in real-time, based on recent observations of the system condition. 4.5 Discussion Some assumptions have been made in this chapter to make the problem theoretically tractable and also for ease of presentation. In this section, we justify these assumptions from a practical point of view and discuss possible extensions. Cost Functions: To ease our discussion, we have limited our attention to trans- mission and migration costs in this chapter. This can be extended to include more sophisticated cost models. For example, the transmission cost can be extended to include the computational cost of hosting the service at an MMC, by adding a constant value to the transmission cost expression. As in Section 4.4.4.5, the cost values can also be time-varying and related to the background system load. Furthermore, the cost definition can be practically regarded as the average cost over multiple locations, which means that when seen from a single location, the monotonicity of cost values with distances does not need to apply. This makes the proposed approach less restrictive in terms of practical applicability. We also note that it is generally possible to formulate an MDP with additional dimensions in cost modeling, such as one that includes the state of the network, load at each specific MMC, state of the service to avoid service interruption when in critical state, etc. However, this requires a significantly larger state space compared to our formulation in this chapter, as we need to include those network/MMC/service states in the state space of the MDP. There is a tradeoff between the complexity of solving the problem and accuracy of cost modeling. Such issues can be studied in the future, where we envision similar approximation techniques as in this chapter 4.5. Discussion 130 can be used to approximately solve the resulting MDP. Single/Multiple Users: Although we have focused on a single user in our problem modeling, practical cases involving multiple users running independent services can be considered by setting cost functions related to the background traffic generated by other users, then considering each user independently, as in the cost definition of the simulation in Section 4.4.4.5. More general cases where each MMC has a capacity limit and not all basestations have MMCs attached to it can be tackled using the heuristic approach presented in Section 4.4.4. From a theoretical perspective, one can also formulate this and other generalized problems as an MDP with larger state space (similar to the generalized cost model discussed above), but we leave it as future work. Random Walk: The random walk mobility model is used as a modeling assump- tion, which not only simplifies the theoretical analysis, but also makes the practical implementation of the proposed method fairly simple in the sense that only the empirical probability of users moving outside of the cell needs to be recorded (see Section 4.4.4.2). This model can capture the average mobility of a large number of users. The simulation results in Section 4.4.4.5 confirm that this model provides good performance, even though individual users do not necessarily follow a random walk. Centralized/Distributed Control: We have focused on a centralized control mechanism in this chapter for ease of presentation. However, many parts of the proposed approach can be performed in a distributed manner. For example, in step 1b in Section 4.4.4.2, the service placement decision can be made among a smaller group of MMCs if the controller sends the results from step 2(c)iii to these MMCs. In particular, the temporary service placement decision in Sections I and II can be 4.6. Summary 131 made locally on each MMC (provided that it knows the locations of other MMCs). The capacity violation check in Section III can also be performed locally. If some MMCs are in excess of capacity, service relocation can be performed using a few control messages exchanged between MMCs, where the number of necessary messages is proportional to the number of services to relocate. Relocation would rarely occur if the system load is not very high. The computation of average empirical probability in step 2(c)ii in Section 4.4.4.2 can also be distributed in the sense that a subset of MMCs compute local averages, which are subsequently sent to the MMC controller that computes the global average. MMC-Basestation Co-location: For ease of presentation, we have assumed that MMCs are co-located with basestations. However, our proposed approach is not restricted to such cases and can easily incorporate scenarios where MMCs are not co-located with basestations as long as the costs are geographically dependent. 4.6 Summary We have taken an MDP-based approach to dynamic service migration in this chapter. Noting that the state space of the MDP for the original problem can be arbitrarily large, we have made simplifications that are either provably optimal or suboptimal with a provable optimality gap. We have also discussed ways to implement the proposed algorithms in practical systems, and evaluated their performance with both synthetic mobility traces and real-world mobility traces of San Francisco taxis. C HAPTER 5 Dynamic Service Placement with Predicted Future Costs 5.1 Introduction The previous chapter presented an MDP-based approach to dynamic service migration. Although the proposed approach is simple, it is constrained to cases where the user mobility follows or can be approximated by a mobility model that can be described by a Markov chain. We note that there are also practical cases where the Markovian assumption is not valid [48]. Furthermore, the definition of costs related to the locations of users and service instances may be inapplicable if the load on different MMCs are imbalanced or if we consider the centralized cloud as a placement option. In this chapter, we assume that there is an underlying mechanism to predict the future costs to some known accuracy. Using these predicted costs, we consider a more general setting that may have different forms of cost functions, as well as heterogeneous network structure and mobility models. Meanwhile, different from Chapter 4 where we mainly consider services that are constantly running for each user, we consider service instances that may arrive and depart over time. 5.1. Introduction 5.1.1 133 Related Work Systems with online (and usually unpredictable) arrivals and departures have been studied in the field of online approximation algorithms [24, 49]. The goal is to design efficient algorithms (usually with polynomial time-complexity) that have reasonable competitive ratios. However, most existing work focuses on problems that can be formulated as integer linear programs. Problems that have convex but non-linear objective functions have attracted attention only very recently [50, 51], where the focus is on online covering problems in which new constraints arrive over time. Our problem is different from the existing work in the sense that the online arrivals in our problem are abstracted as change in constraints (or, with a slightly different but equivalent formulation, adding new variables) instead of adding new constraints, and we consider the average cost over multiple timeslots. Meanwhile, online departures are not considered in [50, 51]. We also note that existing online algorithms with provable performance guarantees are often of theoretical nature [24, 49, 50, 51], which may not be straightforward to apply in practical systems. At the same time, most online algorithms applied in practice are of heuristic nature without provable performance guarantees, which may perform poorly particularly under critical settings. We propose a simple and practically applicable online algorithm with provable performance guarantees in this chapter, and also verify its performance with simulation using both synthetic arrivals and real-world user traces. 5.1.2 Main Contributions In this chapter, we consider a general setting which allows heterogeneity in cost values, network structure, and mobility models. We assume that the cost is related to a finite set of parameters, which can include the locations and preferences of users, load in the system, database locations, etc. We focus on the case where there is an 5.1. Introduction 134 underlying mechanism to predict the future values of these parameters1 , and also assume that the prediction mechanism provides the most likely future values and an upper bound on possible deviation of the actual value from the predicted value. Such an assumption is valid for many prediction methods that provide guarantees on prediction accuracy. Based on the predicted parameters, the (predicted) future costs of each configuration can be found, in which each configuration represents one particular placement sequence of service instances. While we do not deal with specific cost prediction methods in this thesis, a brief summary of possible approaches we can use is given in Section 5.2.2. With the above assumption, we formulate a problem of finding the optimal placement sequence of service instances that minimizes the average cost over time. We define a look-ahead window to represent the amount of time that we look (predict) into the future. The main contributions of this chapter are summarized as follows: 1. We first focus on the offline problem of service instance placement using the predicted costs within a specific look-ahead window, where we assume that the service instance arrivals and departures within this look-ahead window are known beforehand. We show that this problem is equivalent to a shortest-path problem and propose an algorithm (Algorithm 5.2 in Section 5.3.3) to find its optimal solution, by leveraging dynamic programming techniques. 2. We note that it is often practically infeasible to know in advance about when a service instance will arrive or depart. Meanwhile, Algorithm 5.2 may have exponential time-complexity when there exist multiple service instances, which means that it can be very time consuming to find the solution. Therefore, we propose an online approximation algorithm that finds the placement of each 1 We regard these cost parameters as predictable because they are generally related to the overall state of the system or historical pattern of users, which are unlikely to vary significantly from its previous state or pattern within a short time. This is different from arrivals and departures of service instances, which can be spontaneous and unlikely to follow a predictable pattern. 5.2. Problem Formulation 135 service instance at the time it arrives. This online algorithm has polynomial time-complexity and can find the solution within a reasonably short time. It calls Algorithm 5.2 as a subroutine on each service instance arrival. We analytically evaluate the performance of this online algorithm compared to the optimal offline placement. The proposed online algorithm is O(1)-competitive for certain types of cost functions (including those which are linear, polynomial, or in some other specific form), under some mild assumptions. 3. Considering the existence of prediction errors, we propose a method to find the optimal look-ahead window size, such that an upper bound on the actual placement cost is minimized. 4. The effectiveness of the proposed approach is evaluated by simulations with both synthetic service instance arrivals and real-world mobility traces of San Francisco taxis. 5.2 Problem Formulation We consider a cloud computing system as shown in Fig. 1.1, where the clouds are indexed by n ∈ {1, 2, ..., N }. Each cloud n can be either an MMC or a centralized cloud. All MMCs together with the centralized cloud can host service instances that may arrive and leave the system over time. A service instance is a process that is executed for a particular task of a particular cloud service, which may or may not be embedded into a containing environment (such as virtual machine or Linux container). Each service instance may serve one or multiple users, where there usually exists data transfer between the instance and the users it is serving. A time-slotted system as shown in Fig. 5.1 is considered, in which the actual physical time interval corresponding to each slot t = 1, 2, 3, ... can be either the same or different. 5.2. Problem Formulation 136 Possible migration … … 1 2 3 T slots … … … … T slots T slots … Timeslot t Cost prediction for next T slots Figure 5.1: Timing of the proposed approach. We consider a window-based control framework, where every T slots, a controller performs cost prediction and computes the service instance placement sequence for the next T slots. We define these T consecutive slots as a look-ahead window. Service instance placement within each window are found either at the beginning of the window (in the offline case) or whenever an instance arrives (in the online case). We limit ourselves within one look-ahead window when finding the placement sequence. In other words, we do not attempt to find the placement in the next window until the time for the current window has elapsed and the next window starts. Our solution can also be extended to a slot-based control framework where the controller computes the next T -slot service placement sequence at the beginning of every slot, based on predicted cost parameters for the next T slots. We leave the detailed comparison of these frameworks and their variations for future work. 5.2.1 Definitions We introduce some definitions in the following. 5.2.1.1 Service Instances Service instances may arrive and depart over time. We keep an index counter to assign an index for each new instance. The counter is initialized to zero when the cloud system starts to operate2 . Upon a service instance arrival, we increment the 2 This is only for ease of presentation. In practice, the index can be reset when the maximum possible number of the counter is reached. 5.2. Problem Formulation 137 counter by one, so that if the previously arrived instance has index i, a newly arrived instance will have index i + 1. With this definition, if i < i0 , instance i arrives no later than instance i0 . A particular instance i can only arrive once, and we assume that arrivals always occur at the beginning of a slot and departures always occur at the end of a slot. For example, consider timeslots t = 1, 2, 3, 4, 5, instance i = 2 may arrive at the beginning of slot t = 2, and depart at the end of slot t = 4. At any timeslot t, instance i can have one of the following states: not arrived, running, or departed. For the above example, instance i = 2 has not yet arrived in slot t = 1, it is running in slots t = 2, 3, 4, and it has already departed in slot t = 5. Note that an instance can be running across multiple windows each containing T slots before it departs. 5.2.1.2 Configurations Consider an arbitrary sequence of consecutive timeslots t ∈ {t0 , t0 +1, ..., t0 +Q−1}, where Q is an integer. Assume that the instance with the smallest index running in slot t0 has index i0 , and the instance with the largest index running in any of the slots within {t0 , ..., t0 + Q − 1} has index i0 + M − 1. According to the index assignment discussed in Section 5.2.1.1, there can be at most M instances running in any slot t ∈ {t0 , ..., t0 + Q − 1}. We define a Q-by-M matrix denoted by π, where its (q, i)th (q ∈ {1, ..., Q}) element (π)qi ∈ {0, 1, 2, ..., N } denotes the location of service instance i in slot tq , t0 + q − 1, in which “,” stands for “is defined to be equal to”. We set (π)qi according to the state of instance i in slot tq , as follows (π)qi = 0, if i is not running in slot tq n, if i is running in cloud n in slot tq 5.2. Problem Formulation 138 where instance i is not running if it has not yet arrived or has already departed. The matrix π is called the configuration of instances in slots {t0 , ..., t0 +Q−1}. Throughout this chapter, we use matrix π to represent configurations in different subsets of timeslots. We will write π(t0 , t1 , ..., tm ) to explicitly denote the configuration in slots {t0 , t1 , ..., tm } (and we have Q = tm − t0 + 1), and we write π for short where the considered slots can be inferred from the context. When considering a single slot t, π(t) becomes a vector (i.e., Q = 1). Remark: Note that the configurations in different slots can appear either in the same matrix or in different matrices. This means, from π(t0 , ..., t0 + Q − 1), we can get π(t) for any t ∈ {t0 , ..., t0 + Q − 1}, as well as π(t − 1, t) for any t ∈ {t0 + 1, ..., t0 + Q − 1}, etc., and vice versa. For ease of presentation later in this chapter, we define (π(0))i = 0 for any i. 5.2.1.3 Costs Similar to Chapter 4, we consider two types of costs. The local cost U (t, π(t)) specifies the cost of data transmission (e.g., between each pair of user and service instance) and processing in slot t when the configuration in slot t is π(t). Its value can depend on many factors, including user location, network condition, load of clouds, etc., as discussed in Section 5.1.2. When a service instance is initiated in slot t, the local cost in slot t also includes the cost of initial placement of the corresponding service instance(s). We then define the migration cost W (t, π(t − 1), π(t)), which specifies the cost related to migration between slots t − 1 and t, which respectively have configurations π(t − 1) and π(t). There is no migration cost in the very first timeslot (start of the system), thus we define W (1, ·, ·) = 0. The sum of local and migration costs in slot t when following configuration π(t − 1, t) is given by Cπ(t−1,t) (t) , U (t, π(t)) + W (t, π(t − 1), π(t)). (5.1) 5.2. Problem Formulation 139 The above defined costs are aggregated costs for all clouds and transmission links in the system. 5.2.2 Actual and Predicted Costs To distinguish between the actual and predicted cost values, for a given configuration π(t − 1, t), we let Aπ (t) denote the actual value of Cπ (t), and let Dπt0 (t) denote the predicted most likely value of Cπ (t), when cost-parameter prediction is performed at the beginning of slot t0 . For completeness of notations, we define Dπt0 (t) = Aπ (t) for t < t0 , because at the beginning of t0 , the costs of all past timeslots are known. For t ≥ t0 , we assume that the absolute difference between Aπ (t) and Dπt0 (t) is at most ε(τ ) , t0 max Aπ(t−1,t) (t0 + τ ) − Dπ(t−1,t) (t0 + τ ) π(t−1,t),t0 which represents the maximum error when looking ahead for τ slots, among all possible configurations π and all possible prediction time instant t0 . The function ε(τ ) is assumed to be non-decreasing with τ , because we generally cannot have lower error when we look farther ahead into the future. The specific value of ε(τ ) is assumed to be provided by the cost prediction module. How to Obtain and Predict Cost Parameters: We note that specific methods for obtaining current cost parameters and predicting their future values are beyond the scope of this thesis, but we anticipate that existing approaches for cloud monitoring and future cost prediction can be applied. We provide a brief discussion on possible approaches one can use. We start with monitoring the current cloud performance and user locations. The simplest way for cloud monitoring is to send the current parameters that can be directly measured on the system (such as CPU and memory utilization) to the cloud controller. However, such simple methods may either generate high overhead for 5.2. Problem Formulation 140 control information exchange or be inaccurate due to the possibly abrupt change in computational resource occupation of cloud applications. More systematic approaches for cloud monitoring are discussed in the survey in [52], where it is shown that different monitoring mechanisms target slightly different goals, for example some focus on the accuracy of monitored parameters while others focus on the adaptability to a change in resource occupation. The user locations can be monitored by determining which basestation each user is connected to. This granularity of location is sufficient for the case where MMCs are connected directly to basestations. A simple monitoring approach can notify the cloud controller of user handoff instances between different basestations, so that the controller always keeps track of which basestation each user is associated to. After the current state of the system has been monitored, the future states (parameters in the cost model) need to be predicted to determine the predicted future cost. One simple approach for this purpose is to measure cost parameters based on current observation, and regard them as parameters for the future cost until the next measurement is taken. The prediction accuracy in this case is related to how fast the cost parameters vary, which can be estimated from historical records of these parameters. There are more intelligent ways for making predictions, which are expected to be more accurate than the simple approach which takes current measurements as future values. For example, for predicting future cloud performance, one can use linear models such as Kalman filters, discrete models such as hidden Markov chains, or ideas from expert systems [53]. For predicting future user locations, one can make use of social network information of users to derive each user’s intent of moving to a particular area, and some periodicity of user mobility that can be derived from the user’s historical mobility traces [54], for instance. 5.3. Offline Service Placement with Given Look-Ahead Window Size 5.2.3 141 Our Goal Our ultimate goal is to find an optimal configuration π ∗ (1, ..., ∞) that minimizes the actual average cost over sufficiently long time, i.e. PTmax ∗ π (1, ..., ∞) = arg min lim π(1,...,∞) Tmax →∞ t=1 Aπ(t−1,t) (t) . Tmax (5.2) However, it is impractical to find the optimal solution to (5.2), because we cannot precisely predict the future costs and also do not have exact knowledge on instance arrival and departure events in the future. Therefore, we focus on obtaining an approximate solution to (5.2) by utilizing predicted cost values that are collected every T slots. Now, the service placement problem includes two separable parts: one is finding the look-ahead window size T , which will be discussed in Section 5.5; the other is finding the placement sequence (configuration) within each window, where we consider both offline and online placements and will be discussed in Section 5.3 (for offline placement) and Section 5.4 (for online placement). The offline placement assumes that at the beginning of window T , we know the exact time of when each instance arrives and departs within the rest of window T . The online placement does not assume this knowledge. We note that the notion of “offline” here does not imply exact knowledge of future costs. The predicted costs are used in both offline and online placements. 5.3 Offline Service Placement with Given Look-Ahead Window Size In this section, we focus on the offline placement problem. We denote the configuration found for this problem by π off . 5.3. Offline Service Placement with Given Look-Ahead Window Size 142 Algorithm 5.1 Procedure for offline service placement 1: Initialize t0 = 1 2: loop 3: At the beginning of slot t0 , find the solution to (5.3) 4: Apply placements π off (t0 , ..., t0 + T − 1) in timeslots t0 , ..., t0 + T − 1 5: t0 ← t0 + T 6: end loop 5.3.1 Procedure We start by illustrating the high-level procedure of finding π off . When the lookahead window size T is given, the placement sequence π off is found sequentially for each window (containing timeslots t0 , ..., t0 + T − 1), by solving the following optimization problem: π off (t0 , ..., t0 +T −1) = arg min t0X +T −1 π(t0 ,...,t0 +T −1) t0 Dπ(t−1,t) (t) (5.3) t=t0 where Dπt0 (t) can be found based on the parameters obtained from the cost prediction module. The procedure is shown in Algorithm 5.1. In Algorithm 5.1, whenever we solve (5.3), we get the value of π off for additional T slots. Such a solution is sufficient in practice because we only need to know t0 where to place the instances in the current slot. The value of Dπ(t−1,t) (t) in (5.3) depends on the configuration in slot t0 − 1, i.e. π(t0 − 1), according to (5.1). When t0 = 1, π(t0 − 1) can be regarded as an arbitrary value, because the migration cost W (t, ·, ·) = 0 for t = 1. The intuitive explanation of (5.3) is that, at the beginning of slot t0 , it finds the optimal configuration that minimizes the predicted cost over the next slots (including t0 ) up to t0 + T − 1, given the locations of instances in the previous slot t0 − 1. We focus on solving (5.3) next. 5.3. Offline Service Placement with Given Look-Ahead Window Size Instance i=2 arrives in slot t0+1 Dtπ0(t0) π(t0+1)=(1,1) π(t0-1) π(t0)=(1,0) π(t0)=(2,0) π(t0+1)=(1,2) π(t0+1)=(2,1) π(t0+1)=(2,2) π(t0+2)=(1,2) π(t0+2)=(2,1) π(t0+2)=(2,2) Dtπ0(t0+1) 143 Dtπ0(t0+2) π(t0+2)=(1,1) 0 B Figure 5.2: Shortest-path formulation with N = 2, M = 2, and T = 3. Instance i = 1 is running in all slots, instance i = 2 arrives at the beginning of slot t0 + 1 and is running in slots t0 + 1 and t0 + 2. 5.3.2 Equivalence to Shortest-Path Problem t0 Problem (5.3) is equivalent to a shortest-path problem with Dπ(t−1,t) (t) as weights, as shown in Fig. 5.2. Each edge represents one possible combination of configurations in adjacent timeslots, and the weight on each edge is the predicted cost for such configurations. The configuration in slot t0 − 1 is always given, and the number of possible configurations in subsequent timeslots is at most N M , where M is defined as in Section 5.2.1.2 for the current window {t0 , ..., t0 + T − 1}. Node B is a dummy node to ensure that we find a single shortest path, and the edges connecting node B have zero weights. It is obvious that the optimal solution to (5.3) can be found by taking the shortest (minimum-weighted) path from node π(t0 − 1) to node B in Fig. 5.2, and the nodes that the shortest path traverses correspond to the optimal solution π off (t0 , ..., t0 + T − 1) for (5.3). 5.3.3 Algorithm We can solve the abovementioned shortest-path problem by means of dynamic programming [36]. The algorithm is shown in Algorithm 5.2, where we use Up (t, m) 5.4. Complexity Reduction and Online Service Placement 144 and Wp (t, m̃, m) to respectively denote the predicted local and migration costs, when π(t) = m and π(t−1) = m̃. In the algorithm, Lines 5–16 iteratively finds the shortest path (minimum objective function) for each timeslot. In each iteration, the optimal solution for every possible (single-slot) configuration m is found by solving Bellman’s equation of the problem (Line 11). Then, the final optimal configuration is found in Lines 17 and 18. It is obvious that output of this algorithm satisfies Bellman’s principle of optimality, so the result is the shortest path and hence the optimal solution to (5.3). Complexity: When the vectors π m and ξ m are stored as linked-lists, Algo rithm 5.2 has time-complexity O N 2M T . This is because the minimization in Line 11 requires enumerating at most N M possible configurations, and there can be at most N M T possible combinations of values of t and m. 5.4 Complexity Reduction and Online Service Placement The complexity of Algorithm 5.2 is exponential unless the number of service instances M is a constant. Therefore, it is desirable to reduce its complexity. In this section, we propose a method that can find an approximate solution to (5.3) and, at the same time, handle online service instance arrivals and departures. We will also show that (5.3) is NP-hard when the number of service instances M is non-constant, which justifies the need to solve (5.3) approximately in an efficient manner. 5.4.1 Procedure In the online case, we modify the procedure given in Algorithm 5.1 so that instances are placed one-by-one, where each placement greedily minimizes the objective function given in (5.3), while the configurations of previously placed instances remain 5.4. Complexity Reduction and Online Service Placement 145 Algorithm 5.2 Algorithm for solving (5.3) 1: Define variables m and m̃ to represent configurations respectively in the current and previous iteration 2: Define vectors π m and ξ m for all m, where π m (correspondingly, ξ m ) records the optimal configuration given that the configuration at the current (correspondingly, previous) timeslot of iteration is m 3: Define variables µm and νm for all m to record the sum cost values from slot t0 respectively to the current and previous slot of iteration, given that the configuration is m in the current or previous slot 4: Initialize µm ← 0 and π m ← ∅ for all m 5: for t = t0 , ..., t0 + T − 1 do 6: for all m do 7: νm ← µ m 8: ξm ← πm 9: end for 10: for all m do 11: m̃∗ ← arg minm̃ {νm̃ + Up (t, m) + Wp (t, m̃, m)} 12: π m (t0 , ..., t − 1) ← ξm̃∗ (t0 , ..., t − 1) 13: πm (t) ← m 14: µm ← νm̃∗ + Up (t, m) + Wp (t, m̃∗ , m) 15: end for 16: end for 17: m∗ ← arg minm µm 18: π off (t0 , ..., t0 + T − 1) ← π m∗ (t0 , ..., t0 + T − 1) 19: return π off (t0 , ..., t0 + T − 1) 5.4. Complexity Reduction and Online Service Placement 146 unchanged. We assume that each service instance i has a maximum lifetime Tlife (i), denoting the maximum number of remaining timeslots (including the current slot) that the instance remains in the system. The value of Tlife (i) may be infinity for instances that can potentially stay in the system for an arbitrary amount of time. The actual time that the instance stays in the system may be shorter than Tlife (i), but it cannot be longer than Tlife (i). When an instance leaves the system before its maximum lifetime has elapsed, we say that such a service instance departure is unpredictable. We denote the configuration matrix π following online placement by π on . The configuration π on is updated every time when an instance arrives or unpredictably departs. At the beginning of the window (before any instance has arrived), it is initiated as an all-zero matrix. For a specific look-ahead window {t0 , ..., t0 + T − 1}, when service instance i arrives in slot t ∈ {t0 , ..., t0 + T − 1}, we assume that this instance stays in the system until slot te = min {t + Tlife (i) − 1; t0 + T − 1}, and accordingly update the configuration matrix by π on (t, ..., te ) = arg min π(ta ,...,te ) te X t0 Dπ(t−1,t) (t) (5.4) t=ta s.t. π(t, ..., te ) = π on (t, ..., te ) except for column i. Note that only the configuration of service instance i (which is stored in the ith column of π) is found and updated in (5.4), the configurations of all other instances i0 6= i remain unchanged. The solution to (5.4) can still be found with Algorithm 5.2. The only difference is that vectors m and m̃ now become scalar values within {1, ..., N }, because we only consider the configuration of a single instance i. The complexity in this case becomes O(N 2 T ). At the beginning of the window, all the instances that have not departed after slot t0 −1 are seen as arrivals in slot t0 , because 5.4. Complexity Reduction and Online Service Placement 147 Algorithm 5.3 Procedure for online service placement 1: Initialize t0 = 1 2: loop 3: Initialize π on (t0 , ..., t0 + T − 1) as an all-zero matrix 4: for each timeslot t = t0 , ..., t0 + T − 1 do 5: for each instance i arriving at the beginning of slot t do 6: te ← min {t + Tlife (i) − 1; t0 + T − 1} 7: Update π on (t, ..., te ) with the result from (5.4) 8: Apply configurations specified in the ith column of π on (t, ..., te ) for service instance i in timeslots t, ..., te until instance i departs 9: end for 10: for each instance i departing at the end of slot t do 11: Set the ith column of π on (t + 1, ..., t0 + T − 1) to zero 12: end for 13: end for 14: t0 ← t0 + T 15: end loop we independently consider the placements in each window of size T . When multiple instances arrive simultaneously, an arbitrary arrival sequence is assigned to them, and the instances are still placed one-by-one by greedily minimizing (5.4). When an instance i unpredictably departs at the end of slot t ∈ {t0 , ..., t0 +T −1}, we update π on such that the ith column of π on (t + 1, ..., t0 + T − 1) is set to zero. The online procedure described above is shown in Algorithm 5.3. Recall that π on (t, ..., te ) and π on (t + 1, ..., t0 + T − 1) are both part of a larger configuration matrix π on (t0 , ..., t0 + T − 1) (see Section 5.2.1.2). Complexity: When placing a total of M instances, for a specific look-ahead window with size T , we can find the configurations of these M instances with complexity O(N 2 T M ), because (5.4) is solved M times, each with complexity O(N 2 T ). Remark: It is important to note that in the above procedure, the configuration t0 matrix π on (and thus the cost value Dπ(t−1,t) (t) for any t ∈ {t0 , ..., t0 + T − 1}) may vary upon instance arrival or departure. It follows that the T -slot sum cost Pt0 +T −1 t0 Dπon (t−1,t) (t) may vary whenever an instance arrives or departs at an arbit=t0 P 0 +T −1 t0 trary slot t ∈ {t0 , ..., t0 + T − 1}, and the value of tt=t Dπon (t−1,t) (t) stands 0 5.4. Complexity Reduction and Online Service Placement 148 for the predicted sum cost (over the current window containing T slots) under the current configuration, assuming that no new instance arrives and no instance unpredictably departs in the future. This variation in configuration and cost upon instance arrival and departure is frequently noted in the performance analysis presented next. 5.4.2 Performance Analysis It is clear that for a single look-ahead window, Algorithm 5.3 has polynomial timecomplexity while Algorithm 5.1 has exponential time-complexity. In this subsection, we show the NP-hardness of the offline service placement problem, and discuss the optimality gap between the online algorithm and the optimal offline placement. Note that we only focus on a single look-ahead window in this subsection. The interplay of multiple look-ahead windows and the impact of the window size will be considered in Section 5.5. 5.4.2.1 Definitions For simplicity, we analyze the performance for a slightly restricted (but still general) class of cost functions. We introduce some additional definitions next. Indexing of Instances: Without loss of generality, we assume that the instance with lowest index in the current window {t0 , ..., t0 + T − 1} has index i = 1, and the last instance that arrives before the current time of interest has index i = M , where the current time of interest can be any time within the current window. With this definition, M does not need to be the largest index in window {t0 , ..., t0 + T − 1}. Instead, it can be the index of any instance that arrives within {t0 , ..., t0 + T − 1} The cost of placing up to (and including) instance M is considered, where some instances i ≤ M may have already departed from the system. 5.4. Complexity Reduction and Online Service Placement Configuration Sequence: 149 When considering a window of T slots, we define a set of T -dimensional vectors Λ , {(λ1 , ..., λT ) : λm ∈ {0, 1, ..., N }, ∀m ∈ {1, ..., T }, where λm is non-zero for at most one block of consecutive values of m}. We also define a vector λ ∈ Λ to represent one possible configuration sequence of a single service instance across these T consecutive slots. For any instance i, the ith column of configuration matrix π(t0 , ..., t0 + T − 1) is equal to one particular value of λ. We also define a binary variable xiλ , where xiλ = 1 if service instance i operates in configuration sequence λ across slots {t0 , ..., t0 + T − 1} (i.e., the ith column of π(t0 , ..., t0 + T − 1) is equal to λ), and xiλ = 0 otherwise. We always have P λ∈Λ xiλ = 1 for all i ∈ {1, ..., M }. We note that the values of xiλ may vary over time due to arrivals and unpredictable departures of service instances, which can be seen from Algorithm 5.3 and by noting the relationship between λ and π. Before instance i arrives, xiλ0 = 1 for λ0 = [0, ..., 0] which contains all zeros, and xiλ = 0 for λ 6= λ0 . Upon arrival of instance i, we have xiλ0 = 0 and xiλ1 = 1 for a particular λ1 . When instance i unpredictably departs, its configuration sequence switches from λ1 to an alternative (but partly correlated) sequence λ01 according to Line 11 in Algorithm 5.3, after which xiλ1 = 0 and xiλ01 = 1. Resource Consumption: We assume that the costs are related to the resource consumption, and for ease of presentation, we consider two types of resource consumptions. The first type is associated with serving user requests, i.e., data transmission and processing when a cloud is running a service instance, which we refer to as local resource consumption. The second type is associated with migration, i.e., migrating a service instance from one cloud to another cloud, which we refer to as migration resource consumption. If we know that instance i operates under configuration sequence λ, then we 5.4. Complexity Reduction and Online Service Placement 150 know whether instance i is placed on cloud n in slot t, for any n ∈ {1, ..., N } and t ∈ {t0 , ..., t0 +T −1}. We also know whether instance i is migrated from cloud n to cloud h (h ∈ {1, 2, ..., N }) between slots t − 1 and t. We use aiλn (t) ≥ 0 to denote the local resource consumption at cloud n in slot t when instance i is operating under λ, and use biλnh (t) ≥ 0 to denote the migration resource consumption when instance i operating under λ is assigned to cloud n in slot t − 1 and to cloud h in slot t, where we note that the configuration in slot t0 − 1 (before the start of the current window) is assumed to be given and thus independent of λ. The values of aiλn (t) and biλnh (t) are either service-specific parameters that are known beforehand, or they can be found as part of cost prediction. denote the sum local resource consumption at cloud n by PM P yn (t) , i=1 λ∈Λ aiλn (t)xiλ , and denote the sum migration resource PM P consumption from cloud n to cloud h by znh (t) , i=1 λ∈Λ biλnh (t)xiλ . We We may omit the argument t in the following discussion. Remark: The local and migration resource consumptions defined above can be related to CPU occupation, communication bandwidth, etc., or the sum of them. We only consider these two types of resource consumption for ease of presentation. By applying the same theoretical framework, the performance gap results (presented later) can be extended to incorporate multiple types of resources and more sophisticated cost functions, and similar results are obtained for the general case. Costs: We refine the costs defined in Section 5.2.1.3 by considering the cost for each cloud or each pair of clouds. The local cost at cloud n in timeslot t is denoted by un,t (yn (t)). When a service instance is initiated in slot t, the local cost in slot t also includes the cost of initial placement of the corresponding service instance(s). The migration cost from cloud n to cloud h between slots t − 1 and t is denoted by wnh,t (yn (t − 1), yh (t), znh (t)). Besides znh (t), the migration cost is also related 5.4. Complexity Reduction and Online Service Placement 151 to yn (t − 1) and yh (t), because additional processing may be needed for migration, and the cost for such processing can be related to the current load at clouds n and h. The functions un,t (y) and wnh,t (yn , yh , znh ) can be different for different timeslots t and different clouds n and h, and they can depend on many factors, such as user location, network condition, background load of the cloud, etc. Noting that any constant term added to the cost function does not affect the optimal configuration, we set un,t (0) = 0 and wnh,t (0, 0, 0) = 0. We also set wnh,t (·, ·, 0) = 0, because there is no migration cost if we do not migrate. There is also no migration cost at the start of the first timeslot, thus we set wnh,t (·, ·, ·) = 0 for t = 1. With the above definition, the aggregated costs U (t, π(t)) and W (t, π(t − 1), π(t)) can be explicitly expressed as U (t, π(t)) , N X un,t (yn (t)) (5.5) n=1 N X N X W(t, π(t−1),π(t)), wnh,t (yn (t−1), yh (t), znh (t)) . (5.6) n=1 h=1 We then assume that the following assumption is satisfied for the cost functions, which holds for a large class of practical cost functions, such as those related to delay performance or load balancing [17]. Assumption 5.1. Both un,t (y) and wnh,t (yn , yh , znh ) are convex non-decreasing functions of y (or yn , yh , znh ), satisfying: • dun,t (0) dy • ∂wnh,t ∂znh >0 (·, ·, 0) > 0 for t ≥ 2 for all t, n, and h (unless stated otherwise), where dun,t (0) dy of un,t with respect to (w.r.t.) y evaluated at y = 0, and denotes the derivative ∂wnh,t ∂znh (·, ·, 0) denotes the partial derivative of wnh,t w.r.t. znh evaluated at znh = 0 and arbitrary yn and yh . 5.4. Complexity Reduction and Online Service Placement Vector Notation: 152 To simplify the presentation, we use vectors to denote a col- lection of variables across multiple clouds, slots, or configuration sequences. For simplicity, we index each element in the vector with multiple indices that are related to the index of the element, and use the general notion (g)m1 m2 (or (g)m1 m2 m3 ) to denote the (m1 , m2 )th (or (m1 , m2 , m3 )th) element in an arbitrary vector g. Because we know the range of each index, multiple indices can be easily mapped to a single index (such as using the vectorization operation when g is a matrix). We regard each vector as a single-indexed vector for the purpose of vector concatenation (i.e., joining two vectors into one vector) and gradient computation later. We define vectors y (with N T elements), z (with N 2 T elements), x (with M (N + 1)T elements), aiλ (with N T elements), and biλ (with N 2 T elements), for every value of i ∈ {1, 2, ..., M } and λ ∈ Λ. Different values of i and λ correspond to different vectors aiλ and biλ . The elements in these vectors are defined as follows: (y)nt , yn (t), (z)nht , znh (t), (x)iλ , xiλ , (aiλ )nt , aiλn (t), (biλ )nht , biλnh (t). As discussed earlier in this section, xiλ may unpredictably change over time due to arrivals and departures of service instances. It follows that the vectors x, y, and z may vary over time (recall that y and z are dependent on x by definition). The vectors aiλ and biλ are constant. Alternative Cost Expression: Using the above definitions, we can write the sum cost of all T slots as follows e (x) , D e (y, z) , D t0X +T −1 t=t0 + N X N X n=1 h=1 " N X un,t (yn (t)) n=1 # wnh,t (yn (t − 1), yh (t), znh (t)) (5.7) 5.4. Complexity Reduction and Online Service Placement 153 e can be expressed either in terms of x or in terms of where the cost function D(·) P 0 +T −1 t0 Dπ(t−1,t) (t), read(y, z). The cost function defined in (5.7) is equivalent to tt=t 0 ers are also referred to the per-slot cost defined in (5.1) for comparison. The value e (x) or, equivalently, D e (y, z) may vary over time due to service arrivals and of D unpredictable service instance departures as discussed above. 5.4.2.2 Equivalent Problem Formulation With the above definitions, the offline service placement problem in (5.3) can be equivalently formulated as the following, where our goal is to find the optimal configuration for all service instances 1, 2, ..., M (we consider the offline case here where we know when each instance arrives and no instance will unpredictably leave after they have been placed): e (x) min D x s.t. X (5.8) xiλ = 1, ∀i ∈ {1, 2, ..., M } λ∈Λi xiλ ∈ {0, 1}, ∀i ∈ {1, 2, ..., M }, λ ∈ Λi where Λi ⊆ Λ is a subset of configuration sequences that contains those vectors whose elements are non-zero starting from the slot at which i arrives and ending at the slot at which i departs, while all other elements of the vectors are zero. We now show that (5.8), and thus (5.3), is NP-hard even in the offline case, which further justifies the need for an approximation algorithm for solving the problem. Proposition 5.1. (NP-Hardness) The problem in (5.8), and thus (5.3), is NP-hard. Proof. We show that problem (5.8) can be reduced from the partition problem, which is known to be NP-complete [55, Corollary 15.28]. The partition problem is defined as follows. 5.4. Complexity Reduction and Online Service Placement 154 Definition of Partition Problem: Given positive integers v1 , v2 , ..., vM , is there a P P subset S ⊆ {1, 2, ..., M } such that j∈S vj = j∈S c vj , where S c is the complement set of S? Similar to the proof of [55, Theorem 18.1], we define a decision version of the bin packing problem, where we assume that there are M items each with size 2vi ai , PM j=1 vj for all i ∈ {1, 2, ..., M }, and the problem is to determine whether these M items can be packed into two bins each with unit size (i.e., its size is equal to one). It is obvious that this bin packing decision problem is equivalent to the partition problem. To solve the above defined bin packing decision problem, we can set t0 = 1, T = 1, and N = 2 in (5.8). Because we attempt to place all items, we set Λi = {1, 2} for all i. By definition, wkl,t (·, ·, ·) = 0 for t = 1. We omit the subscript t in the following as we only consider a single slot. We define aiλn = ai for all λ, n, and define un (y) = εy, if y ≤ 1 (5.9) 2ε (y − 1) + ε, if y > 1 c where c , 1 PM j=1 vj , and ε > 0 is an arbitrary constant. Because vi is a positive integer for any i, we have that aci = 2vi is always a P positive integer, and 1c = M j=1 vj is also always a positive integer. It follows that y can only be integer multiples of c (where we recall that y is the sum of ai for those P items i that are placed in the bin), and there exists a positive integer c0 , M j=1 vj such that c0 c = 1. Thus, when y > 1, we always have y − 1 ≥ c. Therefore, the choice of un (y) in (5.9) guarantees that un (yn ) ≥ 3ε > 2ε whenever bin n P (n ∈ {1, 2}) exceeds its size, and 2n=1 un (yn ) ≤ 2ε when no bin has exceeded its size. At the same time, un (y) satisfies Assumption 5.1 as long as c ≤ 2. 5.4. Complexity Reduction and Online Service Placement By the definition of c, we always have c ≤ 2 because 155 PM j=1 vj ≥ 1. To solve the bin packing decision problem defined above (thus the partition problem), we can solve (5.8) with the above definitions. If the solution is not larger than 2ε, the packing is feasible and the answer to the partition problem is “yes”; otherwise, the packing is infeasible and the answer to the partition problem is “no”. It follows that problem (5.8) is “at least as hard as” the partition problem, which proves that (5.8) is NP-hard. An online version of problem (5.8) can be constructed by updating Λi over time. When an arbitrary instance i has not yet arrived, we define Λi as the set containing an all-zero vector. After instance i arrives, we assume that it will run in the system until te (defined in Section 5.4.1), and update Λi to conform to the arrival and departure times of instance i (see above). After instance i departs, Λi can be further updated so that the configurations corresponding to all remaining slots are zero. 5.4.2.3 Performance Gap As discussed earlier, Algorithm 5.3 solves (5.8) in a greedy manner, where each service instance i is placed to greedily minimize the objective function in (5.8). In the following, we compare the result from Algorithm 5.3 with the true optimal result, where the optimal result assumes offline placement. We use x and (y, z) to denote the result from Algorithm 5.3, and use x∗ and (y∗ , z∗ ) to denote the optimal result. e Lemma 5.1. (Convexity of D(·)) When Assumption 5.1 is satisfied, the cost function e (x) or, equivalently, D e (y, z) is a non-decreasing convex function w.r.t. x, and it D is also a non-decreasing convex function w.r.t. y and z. Proof. According to Assumption 5.1, un,t (yn (t)) and wnh,t (yn (t − 1), yh (t), znh (t)) are non-decreasing convex functions. Because yn (t) and znh (t) are linear mappings of xiλ with non-negative weights for any t, n, and h, and also because the sum 5.4. Complexity Reduction and Online Service Placement 156 of non-decreasing convex functions is still a non-decreasing convex function, the lemma holds [56, Section 3.2]. In the following, we use ∇x to denote the gradient w.r.t. each element in vector e x, i.e., the (i, λ)th element of ∇x D(x) is e ∂ D(x) . ∂xiλ Similarly, we use ∇y,z to denote the gradient w.r.t. each element in vector (y, z), where (y, z) is a vector that concatenates vectors y and z. Proposition 5.2. (Performance Gap) When Assumption 5.1 is satisfied, we have ∗ e e D(x) ≤ D(φψx ) (5.10) e (y, z) ≤ D e (φψy∗ , φψz∗ ) D (5.11) or, equivalently, where φ and ψ are constants satisfying φ≥ e (ymax + aiλ , zmax + biλ ) · (aiλ , biλ ) ∇y,z D e (y, z) · (aiλ , biλ ) ∇y,z D (5.12) e (x) · x e (y, z) · (y, z) ∇x D ∇y,z D = e e (y, z) D(x) D (5.13) ψ≥ for any i and λ ∈ Λi , in which ymax and zmax respectively denote the maximum values of y and z (the maximum is taken element-wise) at any time within slots {t0 , ..., t0 + T − 1} until the current time of interest, (aiλ , biλ ) is a vector that concatenates aiλ and biλ , and “·” denotes the dot-product. Proof. See Appendix C.4. Remark: We note that according to the definition of M in Section 5.4.2.1, the bound given in Proposition 5.2 holds at any time of interest within slots {t0 , ..., t0 + T − 1}, i.e., for any number of instances that has arrived to the system, where some of them may have already departed. 5.4. Complexity Reduction and Online Service Placement 5.4.2.4 157 Intuitive Explanation to the Constants φ and ψ The constants φ and ψ in Proposition 5.2 are related to “how convex” the cost function is. Here, we use the second order derivative as a measure of convexity, and we say that a function is more convex if it has a larger second order derivative. In other words, they are related to how fast the cost of placing a single instance changes under different amount of existing resource consumption. Figure 5.3 shows an illustrative example, where we only consider one cloud and one timeslot (i.e., t = 1, T = 1, and N = 1). In this case, setting φ = dDe e dD (y + a ) (y) satisfies (5.12), where amax denotes the maximum resource max max dy dy e e consumption of a single instance. Similarly, setting ψ = ddyD (y) · y D(y) satisfies (5.13). We can see that the values of φ and ψ need to be larger when the cost function is more convex. For the general case, there is a weighted sum in both the numerator and denominator in (5.12) and (5.13). However, when we look at a single cloud (for the local cost) or a single pair of clouds (for the migration cost) in a single timeslot, the above intuition still applies. So, why is the optimality gap larger when the cost functions are more convex, i.e., have a larger second order derivative? We note that in the greedy assignment procedure in Algorithm 5.3, we choose the configuration of each instance i by minimizing the cost under the system state at the time when instance i arrives, where the system state represents the local and migration resource consumptions as specified by vectors y and z. When cost functions are more convex, for an alternative system state (y0 , z0 ), it is more likely that the placement of instance i (which was determined at system state (y, z)) becomes far from optimum. This is because if cost functions are more convex, the cost increase of placing a new instance i (assuming the same configuration for i) varies more when (y, z) changes. This intuition is also confirmed by formal results as described next. 5.4. Complexity Reduction and Online Service Placement 158 ~ D(y) ψdenom φdenom ψnum φnum y ymax+amax Figure 5.3: Illustration of the performance gap for t = 1, T = 1, and N = 1, where amax denotes the maximum resource consumption of a single instance. In this num num , and (5.13) becomes ψ ≥ ψψdenom . example, (5.12) becomes φ ≥ φφdenom 5.4.2.5 Performance with Linear Cost Functions When the cost functions are linear, in the form of un,t (y) = γn,t y (5.14) (1) (2) (3) wnh,t (yn , yh , znh ) = κnh,t yn + κnh,t yh + κnh,t znh (1) (2) (5.15) (3) where the constants γn,t ≥ 0 and κnh,t , κnh,t , κnh,t ≥ 0, we have the following result. Proposition 5.3. When the cost functions are defined as in (5.14) and (5.15) while satisfying Assumption 5.1, Algorithm 5.3 provides the optimal solution. Proof. We have e (ymax + aiλ , zmax + biλ ) = ∇y,z D e (y, z) ∇y,z D e (y, z) · (y, z) = D e (y, z) ∇y,z D because the gradient in this case is a constant. Hence, choosing φ = ψ = 1 satise e ∗ ) which means that the solution from fies (5.12) and (5.13), yielding D(x) ≤ D(x 5.4. Complexity Reduction and Online Service Placement 159 Algorithm 5.3 is not worse than the optimal solution. This implies that the greedy service placement is optimal for linear cost functions, which is intuitive because the previous placements have no impact on the cost of later placements when the cost function is linear. 5.4.2.6 Performance with Polynomial Cost Functions Consider polynomial cost functions in the form of un,t (y) = X (ρ) γn,t y ρ (5.16) ρ wnh,t (yn , yh , znh ) = XXX ρ1 ρ2 (ρ ,ρ ,ρ3 ) ρ1 ρ2 ρ3 yn yh znh 1 2 κnh,t (5.17) ρ3 where ρ, ρ1 , ρ2 , ρ3 are integers satisfying ρ ≥ 1, ρ1 + ρ2 + ρ3 ≥ 1 and the constants (ρ) (ρ ,ρ ,ρ3 ) 1 2 γn,t ≥ 0, κnh,t ≥ 0. We first introduce the following assumption which can be satisfied in most practical systems with an upper bound on resource consumptions and departure rates. Assumption 5.2. The following is satisfied: • For all i, λ, n, h, t, there exists a constants amax and bmax , such that aiλn (t) ≤ amax and biλnh (t) ≤ bmax • The number of instances that unpredictably leave the system in each slot is upper bounded by a constant Bd . Proposition 5.4. Assume that the cost functions are defined as in (5.16) and (5.17) while satisfying Assumptions 5.1 and 5.2. Let Ω denote the highest order of the polynomial cost functions. Specifically, (ρ) (ρ ,ρ ,ρ3 ) 1 2 Ω , max{ρ; ρ1 + ρ2 + ρ3 }, subject to γn,t > 0 and κnh,t > 0. 5.4. Complexity Reduction and Online Service Placement 160 . e I(M ) ) D(x e ∗ ), where I(M ) is a problem input3 conDefine Γ(I(M )) , D(x I(M ) taining M instances, and xI(M ) and x∗I(M ) are respectively the online and offline (optimal) results for input I(M ). We say that Algorithm 5.3 is c-competitive in placing M instances if Γ , maxI(M ) Γ(I(M )) ≤ c for a given M . We have: • Algorithm 5.3 is O(1)-competitive. • In particular, for any δ > 0, there exists a sufficiently large M , such that Algorithm 5.3 is ΩΩ + δ -competitive. Proof. See Appendix C.5. Proposition 5.4 states that the competitive ratio does not indefinitely increase with increasing number of instances (specified by M ). Instead, it approaches a constant value when M becomes large. When the cost functions are linear as in (5.14) and (5.15), we have Ω = 1. In this case, Proposition 5.4 gives a competitive ratio upper bound of 1 + δ (for sufficiently large M ) where δ > 0 can be arbitrarily small, while Proposition 5.3 shows that Algorithm 5.3 is optimal. This means that the competitive ratio upper bound given in Proposition 5.4 is asymptotically tight as M goes to infinity. 5.4.2.7 Performance with Other Forms of Cost Functions Algorithm 5.3 is also O(1)-competitive for some more general forms of cost functions. For example, consider a simple case where there is no migration resource consumption, i.e. biλnh (t) = 0 for all i, λ, n, h. Define un0 ,t (y) = γy for some cloud n0 and all t, where γ > 0 is a constant. For all other clouds n 6= n0 , define un,t (y) as a general cost function while satisfying Assumption 5.1 and some additional mild 3 A particular problem input specifies the time each instance arrives/departs as well as the values of aiλ and biλ for each i, λ. 5.5. Optimal Look-Ahead Window Size 161 assumptions presented below. Assume that there exists a constant amax such that aiλn (t) ≤ amax for all i, λ, n, t. Because un,t (y) is convex non-decreasing and Algorithm 5.3 operates in a greedy manner, if dudyn,t (y) > γ, no new instance will be placed on cloud n, as it incurs higher cost than placing it on n0 . As a result, the maximum value of yn (t) is bounded, let us denote this upper bound by ynmax (t). We note that ynmax (t) is only dependent on the cost function definition and is independent of the number of arrived instances. Assume un,t (ynmax (t)) < ∞ and dun,t (ynmax (t) dy + amax ) < ∞ for all n 6= n0 and t. When ignoring the cost at cloud n0 , the ratio Γ(I(M )) does not indefinitely grow with incoming instances, because among all yn (t) ∈ [0, ynmax (t)] for all t and n 6= n0 , we can find φ and ψ that satisfy (5.12) and (5.13), we can also find the competitive ratio Γ , maxI(M ) Γ(I(M )). The resulting Γ is only dependent on the cost function definition, hence it does not keep increasing with M . Taking into account the cost at cloud n0 , the above result still applies, because the cost at n0 is linear in yn0 (t), so that in either of (5.12), (5.13), or in the expression of Γ(I(M )), the existence of this linear cost only adds a same quantity (which might be different in different expressions though) to both the numerator and denominator, which does not increase Γ (because Γ ≥ 1). The cloud n0 can be considered as the backend cloud, which usually has abundant resources thus its cost-per-unit-resource often remains unchanged. This example can be generalized to cases with non-zero migration resource consumption, and we will illustrate such an application in the simulations in Section 5.6. 5.5 Optimal Look-Ahead Window Size In this section, we study how to find the optimal window size T to look-ahead. When there are no errors in the cost prediction, setting T as large as possible can 5.5. Optimal Look-Ahead Window Size 162 potentially bring the best long-term performance. However, the problem becomes more complicated when we consider the prediction error, because the farther ahead we look into the future, the less accurate the prediction becomes. When T is large, the predicted cost value may be far away from the actual cost, which can cause the configuration obtained from the predicted cost with size-T windows (denoted by π p ) deviate significantly from the true optimal configuration (obtained from actual costs) π ∗ . Note that π p and π ∗ specify the configurations for an arbitrarily large number of timeslots, as in (5.2). Conversely, when T is small, the solution may not perform well in the long-term, because the look-ahead window is small and the long-term effect of service placement is not considered. Therefore, we have to find the optimal value of T which minimizes both the impact of prediction error and the impact of truncating the look-ahead time-span. We first define a constant σ, which satisfies max Wa (t, π(t − 1), π(t)) ≤ σ π(t−1,t) (5.18) for any t, to represent the maximum value of the actual migration cost in any slot, where Wa (t, π(t − 1), π(t)) denotes the actual migration cost. The value of σ is system-specific and is related to the cost definition. To help with our analysis below, we define the sum-error starting from slot t0 up to slot t0 + T − 1 as F (T ) , t0X +T −1 ε(t − t0 ). (5.19) t=t0 Because ε(t − t0 ) ≥ 0 and ε(t − t0 ) is non-decreasing with t, it is obvious that F (T + 2) − F (T + 1) ≥ F (T + 1) − F (T ). Hence, F (T ) is a convex non-decreasing function for T ≥ 0, where we define F (0) = 0. 5.5. Optimal Look-Ahead Window Size 5.5.1 163 Upper Bound on Cost Difference In the following, we focus on the objective function given in (5.2), and study how worse the configuration π p can perform, compared to the optimal configuration π ∗ . Proposition 5.5. For look-ahead window size T , suppose that we can solve (5.3) with competitive ratio Γ ≥ 1, the upper bound on the cost difference (while taking the competitive ratio Γ into account) from placement sequences π p and π ∗ is given by PTmax lim Tmax →∞ t=1 Aπp (t) Tmax PTmax −Γ t=1 ! Aπ∗ (t) Tmax ≤ (Γ+1)F (T )+σ . T (5.20) Proof. Define Tmax > 1 as an arbitrarily large timeslot index. We note that there are Tmax full look-ahead windows of size T within timeslots from 1 to Tmax , where bxc T denotes the integral part of x. In the last window, there are Tmax − T · TTmax slots. We have Tmax − T · TTmax Tmax F (T ) F Tmax − T · ≤ T T (5.21) because F (T ) is convex non-decreasing and F (0) = 0. For the true optimal configuration π ∗ , according to the definitions of ε(τ ) and F (T ), the difference in the predicted and actual sum-costs satisfies Tmax X Dπ∗ (t) − Tmax X Aπ∗ (t) t=1 t=1 Tmax Tmax ≤ F (T ) + F Tmax − T · T T Tmax ≤ F (T ) T (5.22) where the last inequality follows from (5.21). Similarly, for the configuration π p obtained from predicted costs, we have Tmax X t=1 Aπp (t) − Tmax X t=1 Dπp (t) ≤ Tmax F (T ). T (5.23) 5.5. Optimal Look-Ahead Window Size 164 In the following, we establish the relationship between π ∗ and π p . Assume that, in (5.3), we neglect the migration cost at the beginning of each look-ahead window, i.e. we consider each window independently and there is no migration cost in the first timeslot of each window, then we have Tmax X Dπp (t) ≤ Γ t=1 Tmax X Dπ∗ (t) t=1 where the constant Γ ≥ 1 is the competitive ratio of solving (5.3). This holds because there is no connection between different windows, thus the optimal sequences (considering predicted costs) obtained from (5.3) constitute the optimal sequence up to a factor Γ for all timeslots [1, Tmax ]. Now we relax the assumption and consider the existence of migration cost in the first slot of each window. Note that we cannot have more than TTmax + 1 windows and the first timeslot t = 1 does not have migration cost. Thus, Tmax X Dπp (t) ≤ Γ Tmax X Dπ∗ (t) + t=1 t=1 Tmax σ. T (5.24) The bound holds because regardless of the configuration in slot t0 − 1, the migration cost in slot t0 cannot exceed σ. By multiplying Γ on both sides of (5.22) and summing up the result with (5.24), we get Tmax X Dπp (t) − Γ t=1 Tmax X t=1 Aπ∗ (t) ≤ Tmax (ΓF (T ) + σ) . T (5.25) Summing up (5.23) with (5.25), dividing both sides by Tmax , and taking the limit on both sides yields the proposition. We assume in the following that the competitive ratio Γ is independent of the choice of T , and regard it as a given parameter in the problem of finding optimal T . This assumption is justified because the performance gap results in Propositions 5.2, 5.3, and 5.4 are not related to the window size T . We define the optimal look-ahead 5.5. Optimal Look-Ahead Window Size 165 window size as the solution to the following optimization problem: (Γ + 1)F (T ) + σ T min T (5.26) s.t. T ≥ 1. Considering the original objective in (5.2), the problem (5.26) can be regarded as finding the optimal look-ahead window size such that an upper bound of the objective function in (5.2) is minimized (according to Proposition 5.5). The solution to (5.26) is the optimal window size to look-ahead so that (in the worst case) the cost is closest to the cost from the optimal configuration π ∗ . 5.5.2 Characteristics of the Problem in (5.26) In the following, we study the characteristics of (5.26). To help with the analysis, we interchangeably use variable T to represent either a discrete or a continuous variable. We define a continuous convex function G(T ), where T ≥ 1 is a continuous variable. The function G(T ) is defined in such a way that G(T ) = F (T ) for all the discrete values T ∈ {1, 2, ...}, i.e. G(T ) is a continuous time extension of F (T ). Such a definition is always possible by connecting the discrete points in F (T ). Note that we do not assume the continuity of the derivatives of G(T ), which means that may be non-continuous and d2 G(T ) dT 2 dG(T ) dT may be +∞. However, these do not affect our analysis below. We will work with continuous values of T in some parts and will discretize it when appropriate. We define a function θ(T ) , (Γ+1)G(T )+σ T to represent the objective function in (5.26) after replacing F (T ) with G(T ), where T is regarded as a continuous variable. We take the logarithm of θ(T ), yielding ln θ = ln ((Γ + 1)G(T ) + σ) − ln T. (5.27) 5.5. Optimal Look-Ahead Window Size 166 Taking the derivative of ln θ, we have ) (Γ + 1) dG(T d ln θ 1 dT = − . dT (Γ + 1)G(T ) + σ T (5.28) We set (5.28) equal to zero, and rearrange the equation, yielding Φ(T ) , (Γ + 1)T dG(T ) − (Γ + 1)G(T ) − σ = 0. dT (5.29) Proposition 5.6. Let T0 denote a solution to (5.29), if the solution exists, then the optimal look-ahead window size T ∗ for problem (5.26) is either bT0 c or dT0 e, where bxc and dxe respectively denote the floor (rounding down to integer) and ceiling (rounding up to integer) of x . Proof. Taking the derivative of Φ(T ), we get dΦ d2 G(T ) = (Γ + 1)T ≥0 dT dT 2 (5.30) where the last inequality is because G(T ) is convex. This implies that Φ(T ) is nondecreasing with T . Hence, there is at most one consecutive interval of T (the interval may only contain one value) such that (5.29) is satisfied. We denote this interval by [T− , T+ ], and a specific solution to (5.29) is T0 ∈ [T− , T+ ]. We note that d ln θ dT and Φ(T ) have the same sign, because d ln θ dT ≶ 0 yields Φ(T ) ≶ 0 and vice versa, which can be seen from (5.28) and (5.29). When T < T− , we have Φ(T ) < 0 and hence hence d ln θ dT d ln θ dT < 0; when T > T+ , we have Φ(T ) > 0 and > 0. This implies that ln θ, thus θ(T ), keeps decreasing with T until the optimal solution is reached, and afterwards it keeps increasing with T . It follows that the minimum value of θ(T ) is attained at T ∈ [T− , T+ ]. Because T0 ∈ [T− , T+ ] and T ∗ is a discrete variable, we complete the proof. Note that we do not consider the convexity of θ(T ) in the above analysis. From 5.5. Optimal Look-Ahead Window Size 167 the proof of Proposition 5.6, we can also conclude the following corollary. Corollary 5.1. For window sizes T and T + 1, if θ(T ) < θ(T + 1), then the optimal size T ∗ ≤ T ; if θ(T ) > θ(T + 1), then T ∗ ≥ T + 1; if θ(T ) = θ(T + 1), then T∗ = T. 5.5.3 Finding the Optimal Solution According to Proposition 5.6, we can solve (5.29) to find the optimal look-ahead window size. When G(T ) (and F (T )) can be expressed in some specific analytical forms, the solution to (5.29) can be found analytically. For example, consider α1 σ G(T ) = F (T ) = βT α , where β > 0 and α > 1. In this case, T0 = (Γ+1)β(α−1) , and T ∗ = arg minT ∈{bT0 c,dT0 e} θ (T ). One can also use such specific forms as an upper bound for a general function. When G(T ) (and F (T )) have more general forms, we can perform a search on the optimal window size according to the properties discussed in Section 5.5.2. Because we do not know the convexity of θ(T ) or Φ(T ), standard numerical methods for solving (5.26) or (5.29) may not be efficient. However, from Corollary 5.1, we know that the local minimum point of θ(T ) is the global minimum point, so we can develop algorithms that use this property. Because the optimal window size T ∗ takes discrete values, we can perform a discrete search on T ∈ {1, 2, ..., Tm }, where Tm > 1 is a pre-specified upper limit on the search range. We then compare θ(T ) with θ(T + 1) and determine the optimal solution according to Corollary 5.1. One possible approach is to use binary search, as shown in Algorithm 5.4, which has time-complexity of O (log Tm ). Remark: We note that the exact value of Γ may be difficult to find in practice, and (5.20) is an upper bound which may have a gap from the actual value of the left hand-side of (5.20). Therefore, in practice, we can regard Γ as a tuning parameter, which can be tuned so that the resulting window size T ∗ yields good performance. 5.6. Simulation Results 168 Algorithm 5.4 Binary search for finding optimal window size 1: Initialize variables T− ← 1 and T+ ← Tm 2: repeat 3: T ← b(T− + T+ ) /2c 4: if θ(T ) < θ(T + 1) then 5: T+ ← T 6: else if θ(T ) > θ(T + 1) then 7: T− ← T + 1 8: else if θ(T ) = θ(T + 1) then 9: return T //Optimum found 10: end if 11: until T− = T+ 12: return T− For a similar reason, the parameter σ can also be regarded as a tuning parameter in practice. 5.6 Simulation Results In this section, we evaluate the performance of the proposed approach with simulations. We assume that there exist a backend cloud (with index n0 ) and multiple MMCs. A service instance can be placed either on one of the MMCs or on the backend cloud. We first define R(y) , 1 1− Yy , if y < Y +∞, (5.31) if y ≥ Y where Y denotes the capacity of a single MMC. Then, we define the local and migration costs as in (5.5), (5.6), with un,t (yn (t)) , g̃yn (t), if n = n0 yn (t)R(yn (t)) + grn (t), if n 6= n0 (5.32) 5.6. Simulation Results wnh,t (yn (t − 1), yh (t), znh (t)) h̃znh (t), if n = n0 or/and h = n0 , znh (t) (R(yn (t)) + R(yh (t))) + hsnh (t), else 169 (5.33) where yn (t) and znh (t) are defined in the same way as in Section 5.4.2.1, rn (t) is the sum of the communication distances between each instance running on cloud n all users connected to this instance, snh (t) is the communication distance between clouds n and h multiplied by the number migrated instances from cloud n to cloud h, and g̃, g, h̃, h are simulation parameters. The communication distance is expressed in the number of hops on the communication network. In the cost functions defined in (5.32) and (5.33), as in Section 5.4.2.7 (but with migration cost here), the local and migration costs involving the backend cloud n0 are linear in the corresponding resource consumptions yn (t) and znh (t). When not involving the backend cloud, the cost functions have two terms. The first term containing R(·) can be explained as related to the queuing delay of data processing/transmission, where we note that the function R(·) has a similar form as the average queueing delay expression from queueing theory, and the additional coefficient yn (t) or znh (t) scales the delay by the total amount of workload so that the experience of all service instances (hosted at the cloud or being migrated) are considered. This expression is also a widely used objective (such as in [17]) which pushes the system towards a load-balanced state. The second term is related to the distance of data transmission or migration. Note that the above defined cost functions are heterogeneous, because the cost definitions are different depending on whether the backend cloud is involved or not. Therefore, we cannot directly apply the existing MDP-based approaches to solve this problem. 5.6. Simulation Results 5.6.1 170 Synthetic Arrivals and Departures To evaluate how much worse the online placement (presented in Section 5.4) performs compared to the optimal offline placement (presented in Section 5.3), we first consider a setting with synthetic instance arrivals and departures. For simplicity, we ignore the migration cost and set g = 0 to make the local cost independent of the distance rn (t). We set Y = 5, g̃ = 3, and the total number of clouds N = 5 among which one is the backend cloud. We simulate 4000 arrivals, where the local resource consumption of each arrival is uniformly distributed within interval [0.5, 1.5]. Before a new instance arrives, we generate a random variable ζ that is uniformly distributed within [0, 1]. If ζ < 0.1, one randomly selected instance that is currently running in the system (if any) departs. We only focus on the cost in a single timeslot and assume that arrival and departure events happen within this slot. The online placement greedily places each instance, while the offline placement considers all instances as an entirety. We compare the cost of the proposed online placement algorithm with a lower bound on the cost of the optimal placement. The optimal lower bound is obtained by solving an optimization problem that allows every instance to be arbitrarily split across multiple clouds, in which case the problem becomes a convex optimization problem due to the relaxation of integer constraints. The simulation is run with 100 different random seeds. Fig. 5.4 shows the overall results. We see that the cost is convex increasing when the number of arrived instances is small, and it increases linearly when the number of instances becomes large, because in the latter case, the MMCs are close to being overloaded and most instances are placed at the backend cloud. In Fig. 5.4(b), we show the average performance ratio, defined as Γavg e mean D(x) , e ∗) mean D(x (5.34) 5.6. Simulation Results 171 1.5 Proposed Optimal lower bound 2000 Upper bound of avg. perf. ratio Objective function value 2500 15 1500 10 5 1000 0 0 10 20 500 0 0 1000 2000 3000 Number of arrived instances (a) 4000 1.45 1.4 1.35 1.3 1.25 1.2 1.15 1.1 0 1000 2000 3000 4000 Number of arrived instances (b) Figure 5.4: Results with synthetic traces: (a) objective function value, (b) average performance ratio. which is an analogy for the competitive ratio. The difference is that the average performance ratio quantifies the performance gap in an average sense, while competitive ratio quantifies the worst-case performance gap. Because we cannot find the real worst case from a limited number of simulation instances, we plot the average performance here. We see that the average performance ratio converges with increasing number of instances, which supports our analysis in Section 5.4.2.7. 5.6.2 Real-World Traces To further evaluate the performance while considering the impact of prediction errors and look-ahead window size, we perform simulations using real-world San Francisco taxi traces obtained on May 31, 2008 [45, 46]. We assume that the MMCs are deployed according to a hexagonal cellular structure, and the distance between the center points of adjacent cells is 1000 m. We consider N − 1 = 91 cells (thus MMCs), one backend cloud, and 50 users (taxis) in total and not all the users are active (which can be derived from the dataset) at a given time. A user may require at most one service at a time from the cloud when it is active, where the duration that 5.6. Simulation Results 172 each active user requires (or, does not require) service is exponentially distributed with a mean value of 50 slots (or, 10 slots). When a user requires service, we assume that there is a service instance for this particular request (independent from other users) running on one of the clouds. The local and migration (if migration occurs) resource consumptions of each such instance are set to 1. We assume that the online algorithm has no knowledge on the departure time of instances and set Tlife = ∞ for all instances. Note that the taxi locations in the dataset are unevenly distributed, with most taxis in the central area of San Francisco, so it is still possible that one MMC hosts multiple services although the maximum possible number of instances (50) is smaller than the number of MMCs (91). The distance metric (for evaluating rn (t) and snh (t)) is defined as the minimum number of hops between two locations on the cellular structure. The physical time corresponding to each slot is set to 60 s. We set the parameters Γ = 1.5, σ = 2, Y = 5, g̃ = h̃ = 3, g = h = 0.2. The cost prediction error is assumed to have an upper bound in the form of F (T ) = βT α (see Section 5.5.3), where we fix α = 1.1. The prediction error is generated randomly while ensuring that the upper bound is satisfied. The simulation results are shown in Fig. 5.5. In Fig. 5.5(a), we can see that the result of the proposed online placement approach (E) performs close to the case of online placement with precise future knowledge (D), where approach D assumes that all the future costs as well as instance arrival and departure times are precisely known, but we still use the online algorithm to determine the placement (i.e., we greedily place each instance), because the offline algorithm is too time consuming due to its high complexity. The proposed method E also outperforms alternative methods including only placing on MMCs and never migrate the service instance after initialization (A), always following the user when the user moves to a different cell (B), as well as always placing the service instance on the backend cloud (C). In approaches A and B, the instance placement is 5.6. Simulation Results 173 51 200 160 Actual cost 140 120 100 Proposed (E), β=0.4 Proposed (E), β=0.6 49 Proposed (E), β=0.8 Proposed (E), β=1.0 48 (C) (A) (B) 80 60 Precise (D) T* 47 46 T* 45 44 40 T* 43 (D) (E) 20 0 00:00 Proposed (E), β=0.2 50 Actual cost 180 Only MMC, never migrate (A) Only MMC, always migrate (B) Only centralized cloud (C) Precise future knowledge (D) Proposed (E) (Average values) 06:00 12:00 Time (a) 18:00 00:00 T* T* 42 41 0 20 40 60 80 100 Look−ahead window size T (b) Figure 5.5: Results with real-world traces (where the costs are summed over all clouds, i.e., the A(t) values): (a) Actual costs at different time of a day, where β = 0.4 for the proposed method E. The arrows point to the average values over the whole day of the corresponding policy. (b) Actual costs averaged over the whole day. determined greedily so that the distance between the instance and its corresponding user is the shortest, subject to the MMC capacity constraint Y so that the costs are finite (see (5.31)). The fluctuation of the cost during the day is because of different number of users that require the service (thus different system load). In Fig. 5.5(b), we show the average cost over the day with different look-ahead window sizes and β values, where the average results from 8 different random seeds are shown. Recall that β quantifies the prediction error upper bound, and a large β indicates a large prediction error. The main observation from Fig. 5.5(b) is that for each value of β we consider in the simulation, the optimal window size (T ∗ , pointed by arrows in the figure) found from the method proposed in Section 5.5 is close to the actual optimal window size that brings the lowest cost (which can be seen in the cost plot for the particular value of β). This implies that the proposed method for finding T ∗ is reasonably accurate. We also see that different β values yield different optimal window sizes; the optimal window size is small when β is large. This is because a 5.7. Summary 174 larger β corresponds to a larger prediction error. When the prediction error is large, the algorithm should not look too far ahead into the future, to avoid bringing in a large error which would bias the optimal placement decision in both short and long term. 5.7 Summary In this chapter, we have studied the dynamic service placement problem utilizing predicted future costs. Noting that the actual future costs cannot be obtained precisely, we have proposed a method that utilizes predicted future costs with a known upper bound on the prediction error to make placement decisions that are close to the true optimum. The method includes finding the optimal window size to look ahead as well as finding the optimal configuration within each look-ahead window. For the latter aspect, we have considered both the optimal offline placement and an approximate online placement with provable performance guarantee. C HAPTER 6 Emulation-Based Study 6.1 Introduction Summing up all the previous chapters, we have proposed algorithms for both initial service placement and real-time service migration (see Section 1.1 for an overview on these two subproblems). The analysis started from a theoretical perspective, with practical considerations discussed subsequently in some chapters. The performances of algorithms have also been evaluated using real-world user mobility traces in Chapters 4 and 5. These contributions and discussions are from both theoretical and practical perspectives, which justify the reasoning of algorithm design, performances of proposed algorithms, and practical applicability of the algorithms. In this chapter, we aim to further push the theoretical results one step closer to practice. We present a framework for conducting experiments in an emulated cloud environment that contains one centralized core cloud and multiple MMCs, which jointly cover an area that contains multiple mobile users. The emulation is performed using the Common Open Research Emulator (CORE) [57, 58] which embeds the Extendable Mobile Ad-hoc Network Emulator (EMANE) [59]. In CORE/EMANE, each network (physical) node is encapsulated into a virtual container which can run a piece of code independent of other nodes. These network (physical) nodes are then connected via emulated network (physical) links for which the quality may vary over time and may be related to the location of nodes (for wireless links). Implementing algorithms in CORE enables more realistic experimentation compared to simulation, 6.1. Introduction 176 thus providing insight into what behavior or performance we can expected when implementing these algorithms in real systems. The emulation-based study we consider in this chapter has the following key differences from the theoretical work in previous chapters: 1. In the emulation, the parameters related to the transmission and migration costs are estimated based on real-time network measurements, which can be subject to fluctuations and inaccuracies. 2. A fully distributed system is considered in the emulation. No node (i.e., core cloud, MMC, or user) in the network has full control over other nodes. They can only communicate with each other by exchanging control messages over the communication network. 3. The emulation considers realistic communication links that may experience loss and delay as a result of congestion and/or physical layer effects. Control messages may also be delayed or lost. The ultimate goal of emulation study is to evaluate the theoretical findings (such as those presented in previous chapters) in realistic settings where some assumptions made for theoretical tractability may not hold. However, non-trivial effort is needed to achieve this ultimate goal, because we need to properly translate the theoretical setting into a practical setting. For example, we have to collect all the information an algorithm needs for decision making, which by itself requires notable implementation effort because the transmission cost, for example, can be related to the users’ associated basestations, signal strength of wireless channels, topology of backhaul network, etc. In a practical emulation environment such as CORE/EMANE, developing a program for obtaining all such information is not straightforward but doable if sufficient time is allowed. For simplicity, we propose an initial emulation framework in this chapter, and perform emulation with some simple service migration 6.2. System Architecture 177 Figure 6.1: System architecture for CORE emulation. algorithms. This sets the foundation of emulating more sophisticated algorithms with realistic applications in the future. 6.2 System Architecture In the CORE emulation, we abstract the application scenario depicted in Fig. 1.1 into the system architecture shown in Fig. 6.1. In Fig. 6.1, node n1 is the core centralized cloud; nodes n2, n3, and n4 are MMCs; and nodes n5 and n6 are users. Note that this is only an example and the number of MMCs and users can be arbitrary in the emulation. 6.2.1 Network Connection and User Mobility The core cloud is connected with each MMC via a persistent link configured with a relative low bandwidth and a relatively large delay. Each MMC and each user has a wireless interface, which has an IP address of 20.0.0.x and is configured by the configuration node wlan7 in Fig. 6.1. Throughout the emulation, we use a fixed range radio propagation model. A pair of nodes within the specified range can communicate with each other, and they can- 6.2. System Architecture 178 not communicate if they are outside the specified range, where a node can be either a user or an MMC. More realistic propagation models can be considered in the future using functionalities of EMANE. The OSPF-MDR routing protocol [60] is used for routing among the wireless nodes. A pair of nodes may communicate either in single-hop or multi-hop depending on their distance. We consider in the emulation the case where MMCs are static and users are mobile. This is only to ease the emulation setup, and the same design principle can be applied to scenarios where MMCs are also mobile. We employ the EMANE event service to use existing traces to govern the mobility of users. The mobility is described by an Emulation Event Log (EEL) file. To receive messages from EMANE, each user has an additional wireless interface with an IP address of 30.0.0.x, which is configured by the configuration node wlan8. This interface is used only for communicating with EMANE (which is executed outside the emulator in the emulator’s host machine) so that the user locations remain updated in real time according to the trace file. It does not participate in any actual communication in the emulated network. With the above setting, the core cloud, MMCs, and users can communicate with each other. The wireless connections are also updated in real-time based on locations of users. Users do not have a direct connection with the core cloud, but they can connect to the core via an MMC. 6.2.2 Service Model We consider a relatively simple service model as presented next, while noting that more sophisticated cases can be built based on the emulation framework we present here. We assume that each user is running one (independent) service instance in the cloud, and the terms “service” and “service instance” are exchangeably used. Ser- 6.2. System Architecture 179 vice components can only be placed on MMCs and no service component is placed on the core cloud. This simplifies the control procedure of service placement, while more sophisticated cases involving the core cloud can be considered in the future by extending the current framework. For now, the role of the core cloud is to act as a controller for service placement, which is a relatively robust setup for a centralized control approach because the system can still work even if some MMCs fail to function. We consider a delay-sensitive situational awareness application, such as the face recognition application introduced in Section 1.1. In such applications, each user regularly sends data (e.g., images as in the face recognition application) describing its current surrounding to the MMC that is hosting its service. The MMC analyzes user data1 to extract user situation (e.g., detecting and tagging objects and faces in images) and sends the results back to the user, also in a regular manner. We abstract this process as UDP packet transfer between the user and the MMC, and we name these packets as service packets to distinguish them from control packets introduced later. The service packet transmission is initiated by the MMC. The MMC sends a MMC_SERVICE_PACKET to each user (for which it is hosting a service) at a pre-specified interval. After receiving this packet, the user responds with its own USER_SERVICE_PACKET that contains newly measured data to the MMC. The performance metric of the service is the round-trip delay of transmitting service packets from the MMC to the user and then back to the MMC. This mimics situational awareness applications that need to collect user data as quickly as possible so that rapid decisions can be made. This round-trip delay can be seen as the transmission cost between the user and the MMC. 1 Note that this is a simplified version of the face recognition example presented in Section 1.1. We assume here that the face recognition module and the database are also placed on the MMC, which can be a feasible configuration when the database is not too large. This is a simplification and more general cases can be considered in the future. 6.3. Packet Exchange and Placement Control 180 For simplicity, in the emulation framework proposed in this chapter, we assume that each MMC can host any number of services. In more realistic scenarios, different services may consume different amount of resources of the hosting MMC, and the number of services that a MMC can host will be subject to its capacity limit. We envision that algorithms presented in previous chapters can be used for making placement decisions in such capacity-limiting scenarios. Most of these algorithms either explicitly or implicitly incorporate capacity constraints, and are able to avoid placing new services on MMCs that result in violation of the capacity constraints. We also assume that service migration can be completed instantaneously, and leave the more realistic case where there is a cost (such as interruption of service, bandwidth consumption for transferring application data) associated with migration as future work. As an indicator of the migration cost, we record the number of migrations in the emulation results in Section 6.4. We also note that for real applications, the migration cost can be related to the state of the application at the time when migration is performed. While the algorithms presented in previous chapters have considered the migration cost, they have not considered the application state which could impact the migration cost. This aspect is worth studying in the future. The emulation framework we present in this chapter can be an essential tool for such a study, because we can run real applications on our emulated MMC platform, which would be very helpful for studying the interplay between application state and migration cost. 6.3 6.3.1 Packet Exchange and Placement Control Control Messages Control messages need to be exchanged for performance measurement and controlling the placement of services. All messages are sent in UDP packets. The different 6.3. Packet Exchange and Placement Control 181 types of messages are described as follows. 6.3.1.1 Beacon Messages from Core Cloud At a pre-specified interval, the centralized core cloud sends out a beacon message CORE_BEACON_MMC to each MMC, and it also sends out a beacon message CORE_BEACON_USER to each user. This is to notify MMCs and users of some status information, including the set of users each MMC should currently serve, the set of users each MMC should probe for delay measurement, etc. Before sending these beacon messages, the core cloud makes decisions such as where to place the service for each user. 6.3.1.2 Connection Request In order for the core cloud to know which MMCs and users are currently present in the system, each MMC sends MMC_CONNECT and each user sends USER_CONNECT at a pre-specified interval. 6.3.1.3 Delay Probing Probing for delay measurement is initiated by each MMC at a pre-specified interval. Every MMC probes a set of users as instructed by the core cloud. This set of users should be within a specific proximity of the MMC, so that the MMC is potentially suitable of running services for these users. An MMC first sends MMC_DELAY_PROBE to all the users it intends to probe. Upon receiving this message, each user immediately replies with a USER_DELAY_PROBE to the MMC that sent this message. The MMC calculates the round-trip delay for the particular user and sends the result to the core cloud with an MMC_USER_DELAY_MEASURED message. After the core cloud has received this delay information, it stores it in a list so that it can be used for making service placement decisions before sending beacon 6.3. Packet Exchange and Placement Control 182 Algorithm 6.1 Procedure at the core cloud 1: loop 2: if timer t(CORE_BEACON_MMC) expired then 3: Update beacon information, such as the placement of services for all users (obtained based on the recorded delay measurements) 4: Send CORE_BEACON_MMC to each MMC 5: Reset timer t(CORE_BEACON_MMC) 6: end if 7: if timer t(CORE_BEACON_USER) expired then 8: Update beacon information 9: Send CORE_BEACON_USER to each user 10: Reset timer t(CORE_BEACON_USER) 11: end if 12: if received MMC_USER_DELAY_MEASURED then 13: Update the recorded delay statistics 14: end if 15: if received MMC_CONNECT or USER_CONNECT then 16: Update the MMC or user record 17: end if 18: end loop messages. We note that one may think of using service packets for delay measurement. However, service packets are only transmitted between a user and the particular MMC that is hosting its service, and the delay of transmitting packets between the user and other MMCs cannot be obtained from service packets. Therefore, we introduce additional packets for delay probing, which is usually substantially shorter than service packets. 6.3.2 Packet Exchange and Control Procedure The detailed procedures that are executed at the core cloud, MMC, and user are respectively shown in Algorithms 6.1, 6.2, and 6.3, where we use the notation t(A) to denote the timer for packet A, the packet A is sent when t(A) expires. 6.3. Packet Exchange and Placement Control Algorithm 6.2 Procedure at each MMC 1: loop 2: if timer t(MMC_CONNECT) expired then 3: Send MMC_CONNECT to the core cloud 4: Reset timer t(MMC_CONNECT) 5: end if 6: if timer t(MMC_DELAY_PROBE) expired then 7: Send MMC_DELAY_PROBE to each user it intends to probe 8: Reset timer t(MMC_DELAY_PROBE) 9: end if 10: if timer t(MMC_SERVICE_PACKET) expired then 11: Send MMC_SERVICE_PACKET to each user it is serving 12: Reset timer t(MMC_SERVICE_PACKET) 13: end if 14: if received USER_DELAY_PROBE then 15: Calculate the round-trip delay and send the MMC_USER_DELAY_MEASURED to the core cloud 16: end if 17: end loop 183 result via Algorithm 6.3 Procedure at each user 1: loop 2: if timer t(USER_CONNECT) expired then 3: Send USER_CONNECT to the core cloud 4: Reset timer t(USER_CONNECT) 5: end if 6: if received t(MMC_DELAY_PROBE) then 7: Send USER_DELAY_PROBE to the originating MMC 8: end if 9: if received t(MMC_SERVICE_PACKET) then 10: Send USER_SERVICE_PACKET to the originating MMC 11: end if 12: end loop 6.3.3 Service Placement Decisions In the emulation, we consider three different policies for deciding the service placement. These are described as follows. 6.3. Packet Exchange and Placement Control 6.3.3.1 184 Always Migrate (AM) In the AM policy, the core cloud always looks at its most recently received delay measurements, and places the service for each user to the MMC that has the lowest delay as measured by the latest probe. This policy puts strong emphasis on minimizing the transmission cost (round-trip delay), and does not consider the migration cost (number of migrations). 6.3.3.2 Infrequently Migrate (IM) The IM policy is similar to the AM policy, except that migration can only occur at an interval denoted by τ . The value of τ is normally large compared to the delay probing interval, so that migration happens infrequently. The initial placement of services is not restricted by τ , which means that if a user is not served by any cloud previously, its service can be placed immediately after the core cloud (controller) recognizes that it has established connection with an MMC. The IM policy attempts to bound the migration cost, while the transmission cost may be large because services may not be migrated soon enough after the transmission delay has changed. The IM policy mimics the never migrate policy used for comparison in previous chapters. We use IM instead of never migrate here to avoid the system being trapped in the initial state. 6.3.3.3 Moving Average + Hysteresis (MAH) The MAH policy does not use the instantaneous delay measurement for service placement and migration decisions. Instead, it uses an exponentially-weighted moving averaged delay for decision making. For each MMC i and user j, the moving average is performed according to Dij (m) = αDij (m − 1) + (1 − α)dij (m) (6.1) 6.4. Emulation Scenario and Results 185 where dij (m) is the mth round-trip delay for MMC-user pair (i, j) that has been received by the core cloud, Dij (m) is the delay used for decision making after receiving the mth and before receiving the (m + 1)th measurement, 0 ≤ α ≤ 1 is a controllable parameter for moving average computation. The MAH policy also has a hysteresis delay value ε ≥ 0. If the service for user j is previously running at MMC i, a migration to MMC i0 only occurs when Di0 j (m) < Dij (m) − ε. This prevents frequent migration between different MMCs. We can see that the MAH policy can be configured to achieve various tradeoffs between the transmission and migration costs by tuning the values of α and ε. It is used here as an alternative representative of the theoretical results in previous chapters, which jointly consider the transmission and migration costs. The MAH policy is simpler than the algorithms in previous chapters, but also less robust due to the existence of parameters α and ε that are subject to tuning. We also note that the AM policy is a special case of the MAH policy under α = ε = 0. 6.4 Emulation Scenario and Results We created an emulation scenario that consists of 20 users and 16 MMCs distributed in the San Francisco area, as shown in Fig. 6.2. We assume that the MMCs are static. The user mobility is generated using the San Francisco taxi dataset [45, 46] that has also been used in earlier chapters. To reduce the required emulation time and see the impact of user mobility, we compressed the timescale by a factor of 6, so that 6 seconds in the original trace is one second in the emulation. We note that the San Francisco taxi traces is only one representative of a real-world mobility pattern, which is the mobility of vehicles moving in a city area. Some other mobility patterns that can be helpful for evaluating the performance of MMCs include vehicles moving on highways, pedestrians moving in cities, etc. It can be worthwhile to carry out an 6.4. Emulation Scenario and Results 186 Figure 6.2: Emulation scenario (source of map: https://maps.google.com/). in-depth study using different mobility patterns in the future. We ran the emulation for 15, 000 seconds, in which we applied either the AM, IM, or MAH policy for service placement. The interval of sending CORE_BEACON_MMC, CORE_BEACON_USER, and MMC_DELAY_PROBE is specified by parameter T which take different values in the emulation. The value of T specifies the interval of delay probing and service placement update. Other parameter settings in the emulation are summarized in Table 6.1. We first set T = 2 s and focus on the instantaneous results in the first 3, 000 seconds of emulation. The instantaneous round-trip delay of service packets under different policies is shown in Fig. 6.3, where the results for service packets corresponding to different MMC-user pairs are merged into the same timeline and plotted in one graph. We can see that there are many spikes in the figure, which indicates that the delay exhibits large variation. Reasons for this include variation in network traffic, diversity of user and service locations, as well as the change of user and service locations over time. The last reason is obvious from Fig. 6.3(b), where the delay has 6.4. Emulation Scenario and Results 187 Table 6.1: Emulation setup Parameter name Wireless communication bandwidth One-hop wireless communication range One-hop wireless communication delay Bandwidth of link connecting MMC and core cloud Delay of link connecting MMC and core cloud Size of MMC_SERVICE_PACKET Size of USER_SERVICE_PACKET Interval of sending MMC_SERVICE_PACKET Interval of sending MMC_CONNECT Interval of sending CORE_BEACON_MMC, CORE_BEACON_USER, and MMC_DELAY_PROBE Parameter τ in IM policy Parameter α in MAH policy Parameter ε in MAH policy Value 10 Mbps 6, 000 m 20 ms 10 Mbps 500 ms 1, 000 Bytes 50, 000 Bytes 2s 10 s T (variable) 100 s 0.5 10 ms a block pattern because the IM policy only migrates (and thereby changing service locations) once in 100 seconds. To obtain a more comprehensible set of results, we perform cumulative moving average with a window size of 50 s on the instantaneously measured data. At each time instant (in the resolution of one second), we look back to the past 50 s (or up to the emulation start time, whichever is later) and plot the average result within this window. The moving averaged delay is shown in Fig. 6.4(a), and Fig. 6.4(b) shows the average number of service migrations within the 50 s window size. We can see that compared to the AM and MAH policies, the IM policy may cause blocks of large delays, because services may be placed at non-optimal locations for a long time. The overall performance with different values of T is shown in Fig. 6.5, where the results are collected starting from 1, 000 s emulation time to remove the impact of variation in the initialization time under different T . We see that as expected, the AM policy always has the largest number of migrations. It is interesting that round-trip delay of the AM policy is also generally larger than that of the MAH policy, mainly because the AM policy uses instantaneous delay measurements for 188 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 1000 2000 Time (seconds) 3000 Round−trip delay of service packets (seconds) Round−trip delay of service packets (seconds) 6.4. Emulation Scenario and Results 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 Round−trip delay of service packets (seconds) (a) 1000 2000 Time (seconds) 3000 (b) 0.5 0.4 0.3 0.2 0.1 0 0 1000 2000 Time (seconds) 3000 (c) Figure 6.3: Instantaneous round-trip delays of service packets for the first 3, 000 s of emulation with T = 2 s: (a) AM policy, (b) IM policy, (c) MAH policy. 0.35 AM IM MAH 0.3 0.25 0.2 0.15 0.1 0.05 0 0 1000 2000 Time (seconds) (a) 189 Average number of migrations per second Average round−trip delay of service packets (seconds) 6.4. Emulation Scenario and Results 3000 8 7 6 AM IM MAH 5 4 3 2 1 0 0 1000 2000 Time (seconds) 3000 (b) Figure 6.4: Moving average results for the first 3, 000 s of emulation with T = 2 s: (a) round-trip delay of service packets, (b) number of migrations. service placement decisions, which may fluctuate substantially over time and cause the placement decision deviate away from the optimum. Instead, the MAH policy is more stable because it has the moving average and hysteresis building blocks. The delay performance of the MAH policy is also generally better than the IM policy, because services are migrated to good locations more frequently. The performance is related to the value of the interval T . It is clear that T = 0.5 s brings the worst performance, in which case the control messages overload the network. For the MAH policy, its lowest delay and highest number of received service packets (which represents the packet success rate) is attained at T = 2 s, while giving an intermediate number of migrations. We also note that the average delay of the MAH policy is the lowest among all the policies when T = 2 s, also with lower standard deviation than all other cases. This implies that the MAH policy can be beneficial for delay-sensitive applications. When T is large, the delay performance does not vary significantly with different T , but the number of received service pack- Round−trip delay of service packets (s) 6.4. Emulation Scenario and Results 190 0.8 AM IM MAH 0.6 0.4 0.2 0 −0.2 0.5 1 2 3 5 10 20 50 100 Probing and placement update interval T (s) 5 14 AM IM MAH 12 10 8 6 4 2 0 0.5 1 2 3 5 10 20 50 100 Probing and placement update interval T (s) (b) Number of received service packets Average number of migrations per second (a) 2.8 x 10 AM IM MAH 2.75 2.7 2.65 2.6 2.55 2.5 0.5 1 2 3 5 10 20 50 100 Probing and placement update interval T (s) (c) Figure 6.5: Overall results: (a) average round-trip delay of service packets (error bars denote the standard deviation), (b) average number of migrations per second, (c) total number of received service packets. 6.5. Summary 191 ets decreases with T . The network in this case is not overloaded, thus the delay is not very large. However, due to obsolete delay measurement and prolonged service placement update, the service location may be far away from user location, so that the connection between the MMC and the user has multiple hops, causing higher packet loss. The slight fluctuation in the performance with different values of T is due to randomness in the emulation. 6.5 Summary In this chapter, we have taken an initial step to an emulation-based study of service placement/migration in an MMC environment containing mobile users. We have proposed a simple emulation framework that can be used as a foundation for more sophisticated emulations in the future. C HAPTER 7 Conclusions and Future Work 7.1 Contributions and Conclusions This thesis has focused on service placement in a networked cloud environment containing mobile micro-clouds (MMCs). The hierarchical structure of MMCs allows us to exploit and develop new algorithms for service graph placement, which are more efficient than existing approaches. Meanwhile, MMCs bring in much more user and infrastructure dynamics compared to a traditional cloud environment, and we have therefore proposed mechanisms for controlling the service placement/migration to cope with these dynamics. The main contributions of this thesis are summarized as follows. 7.1.1 Application Graph Placement In Chapter 3, we have studied the placement of an incoming stream of application graphs onto a physical graph. With the goal of minimizing the maximum resource utilization at physical nodes and links, we have proposed an exact optimal algorithm for placing one linear application graph onto a tree physical graph, as well as approximation algorithms for placing tree application graphs onto tree physical graphs. When the maximum number of unplaced junction nodes on any path from the root to a leaf (in the application graph) is a constant, the proposed algorithm has polynomial time-complexity and provides polynomial-logarithmic (poly-log) worst-case performance bound. Besides the theoretical evaluation of the worst-case performance, we 7.1. Contributions and Conclusions 193 have also shown the average performance via simulation. A combination of these results implies that the proposed method performs reasonably well on average and it is also robust in extreme cases. Moreover, the method we propose is relatively easy to implement in practice. The results in Chapter 3 can be regarded as an initial step towards a more comprehensive study in this direction. Many constraints in the problem formulation are for simplicity of presentation, and can be readily relaxed for a more general problem. For example, the tree application and physical graph restriction is not absolutely essential for the applicability of the key concepts in our proposed algorithms. Modified algorithms based on similar ideas would work for more general graphs as long as the cycle-free constraint is satisfied. While we have not considered services leaving at some time after their arrival, our algorithm can be extended to incorporate such cases using the idea in [61]. The algorithm for cases with unplaced junction nodes is essentially considering the scenario where there exists some low-level placement (for each of the branches) followed by some high level placement (for the junction nodes). Such ideas may also be useful in developing practical distributed algorithms with provable performance guarantees. 7.1.2 MDP-Based Approach to Dynamic Service Migration In Chapter 4, we have studied service migration in MMCs to cope with user mobility and network changes. The problem is formulated as an MDP, but its state space can be arbitrarily large. To make the problem tractable, we have considered reasonable simplifications for 1-D and 2-D user mobility models. For the 1-D case, we have considered a constant cost model and proposed a threshold policy-based mechanism for service migration in MMCs. We have shown the existence of a threshold policy that is an optimal policy, and have proposed an algorithm for finding the optimal thresholds. The proposed algorithm has polynomial 7.1. Contributions and Conclusions 194 time-complexity, which is independent of the discount factor γ. This is promising because the time-complexity of standard algorithms for solving MDPs, such as the value or policy iteration methods, is generally dependent on the discount factor, and those solutions can only be shown to have polynomial time-complexity when the discount factor is regarded as a constant [42]. For the 2-D case, we have considered a constant-plus-exponential cost model. We have reduced the general problem into an MDP that only considers a meaningful parameter, namely the distance between the user and service locations. The distancebased MDP has several structural properties that allow us to develop an efficient algorithm for finding its optimal policy. We have then shown that the distance-based MDP is a good approximation to scenarios where the users move in a 2-D space, which is confirmed by analytical and numerical evaluations. Afterwards, we have illustrated the application of our solution in practical scenarios where many theoretical assumptions are relaxed. Our evaluation based on real-world mobility traces of San Francisco taxis shows the superior performance of the proposed solution compared to baseline solutions. The results of Chapter 4 provide an efficient solution to service migration in MMCs. Further, we envision that the approaches used in this chapter can be extended to a range of other problems that share similar properties. The highlights of our approaches include: a closed-form solution to the discounted sum cost of a particular class of MDPs, which can be used to simplify the procedure for finding the optimal policy; a method to approximate an MDP (in a particular class) with one that has much smaller state space, where the approximation error can be quantified analytically; and a method to collect statistics from the real-world environment to serve as parameters of the MDP. 7.1. Contributions and Conclusions 7.1.3 195 Dynamic Service Placement with Predicted Future Costs Noting that the MDP-based approach can be difficult to apply to cases with more general user mobility and cost functions, we have assumed the ability of predicting future costs to some known accuracy in Chapter 5. This allows us to develop a mechanism for dynamic service placement in more general scenarios, including those where service instances arrive and depart without prior knowledge. We have defined a window to specify the time to look (or predict) ahead. Cost prediction is performed at the beginning of each look-ahead window. Based on the predicted cost, we find the service placement sequence, where we have considered both the optimal offline placement and approximate online placement. The online placement algorithm has polynomial time-complexity and we have analytically shown its optimality gap. Its competitive ratio is a constant for certain types of cost functions. We have then studied how to find the optimal look-ahead window size. The performance of the proposed approach is verified via simulations using both synthetic mobility traces and real-world mobility traces of San Francisco taxis. The theoretical framework used for analyzing the performance of online placement can be extended to incorporate more general cases, such as those where there exist multiple types of resources in each cloud. We envision that the performance results will be qualitatively similar. We also note that our framework can be applied to analyzing a large class of online resource allocation problems that have convex objective functions. 7.1.4 Emulation-Based Study With the goal of studying the performance of the above theoretical results in a more practical setting, we have proposed an emulation framework based on CORE/EMANE in Chapter 6. Different from its previous chapters, Chapter 6 7.2. Future Work 196 considers realistic exchange of control and service packets. The emulation results show several insightful observations, such as the impact of randomness and delay on the service placement performance in a realistic network setting. 7.2 Future Work Some possible areas of future work are summarized as follows. Queuing Aware Scheduling of Service Requests: The approaches presented in this thesis have focused on scenarios where users are continuously connected to services. In reality, user requests may be buffered at the cloud for some time before they are served. Service placement with consideration of queuing/buffering can be studied in the future. We envision that queuing-aware scheduling techniques, such as Lyapunov optimization [62], can be used for such study. One such attempt is recently reported in [63]. Distributed Decision Making: In the approaches presented in this thesis, deci- sions on service placement are made by a centralized controller. This is usually feasible in practice because we can always see the controller as a service running at one or multiple clouds. However, it is desirable to have a distributed control mechanism for the sake of robustness. Issues regarding distributed service placement can be studied in the future, where randomization techniques such as in [64] and [65] can be used. Practical Aspects: As MMCs have not yet been largely deployed in practice, it is definitely worth studying its implementation issues and identifying new challenges from a practical angle, which in turn can introduce new problems that are worthwhile for theoretical study. The emulation framework proposed in Chapter 6 can be useful 7.2. Future Work 197 for such study. For example, we can implement the theoretical results in Chapters 3–5 on the emulator and investigate how they perform. We can also connect real applications to the emulator. Several new challenges arise with running real applications on MMCs, such as: What is the best way of performing service migration and what are the (real) costs associated with it? Service migration is expected to occur more frequently in MMCs than in centralized clouds, because otherwise the benefits of MMCs can no longer be fully realized as users tend to move out of their original area in mobile environments. Bibliography Note: The numbers at the end of each reference stand for the page numbers where the reference has been cited. [1] M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, and M. Zaharia, “A view of cloud computing,” Commun. ACM, vol. 53, no. 4, pp. 50–58, Apr. 2010. 1 [2] P. Bahl, R. Y. Han, L. E. Li, and M. Satyanarayanan, “Advancing the state of mobile cloud computing,” in Proceedings of the third ACM workshop on Mobile cloud computing and services. ACM, 2012, pp. 21–28. 1 [3] Y. Abe, R. Geambasu, K. Joshi, H. A. Lagar-Cavilla, and M. Satyanarayanan, “vTube: efficient streaming of virtual appliances over last-mile networks,” in Proceedings of the 4th annual Symposium on Cloud Computing. ACM, 2013, p. 16. 1 [4] K. Ha, Z. Chen, W. Hu, W. Richter, P. Pillai, and M. Satyanarayanan, “Towards wearable cognitive assistance,” in Proc. of ACM MobiSys, 2014. 1, 2, 7 [5] M. Satyanarayanan, Z. Chen, K. Ha, W. Hu, W. Richter, and P. Pillai, “Cloudlets: at the leading edge of mobile-cloud convergence,” in Proc. of MobiCASE 2014, Nov. 2014. 1, 2, 7 [6] “Smarter wireless networks,” IBM Whitepaper No. WSW14201USEN, Feb. 2013. [Online]. Available: www.ibm.com/services/multimedia/Smarter_ wireless_networks.pdf 2 [7] M. Satyanarayanan, R. Schuster, M. Ebling, G. Fettweis, H. Flinck, K. Joshi, and K. Sabnani, “An open ecosystem for mobile-cloud convergence,” IEEE Communications Magazine, vol. 53, no. 3, pp. 63–70, Mar. 2015. 2 Bibliography 199 [8] M. Satyanarayanan, G. Lewis, E. Morris, S. Simanta, J. Boleng, and K. Ha, “The role of cloudlets in hostile environments,” IEEE Pervasive Computing, vol. 12, no. 4, pp. 40–49, Oct. 2013. 2, 7 [9] S. Davy, J. Famaey, J. Serrat-Fernandez, J. Gorricho, A. Miron, M. Dramitinos, P. Neves, S. Latre, and E. Goshen, “Challenges to support edge-as-a-service,” IEEE Communications Magazine, vol. 52, no. 1, pp. 132–139, Jan. 2014. 2, 7 [10] F. Bonomi, R. Milito, J. Zhu, and S. Addepalli, “Fog computing and its role in the internet of things,” in Proceedings of the first edition of the MCC workshop on Mobile cloud computing. ACM, 2012, pp. 13–16. 2, 7 [11] Z. Becvar, J. Plachy, and P. Mach, “Path selection using handover in mobile networks with cloud-enabled small cells,” in Proc. of IEEE PIMRC 2014, Sept. 2014. 2, 7 [12] T. Taleb and A. Ksentini, “Follow me cloud: interworking federated clouds and distributed mobile networks,” IEEE Network, vol. 27, no. 5, pp. 12–19, Sept. 2013. 2, 7 [13] K. Ha, P. Pillai, G. Lewis, S. Simanta, S. Clinch, N. Davies, and M. Satyanarayanan, “The impact of mobile multimedia applications on data center consolidation,” in Cloud Engineering (IC2E), 2013 IEEE International Conference on, March 2013, pp. 166–176. 2, 7 [14] M. Satyanarayanan, “A brief history of cloud offload: A personal journey from odyssey through cyber foraging to cloudlets,” SIGMOBILE Mob. Comput. Commun. Rev., vol. 18, no. 4, pp. 19–23, Jan. 2015. 2, 7 [15] S. Wang, L. Le, N. Zahariev, and K. K. Leung, “Centralized rate control mechanism for cellular-based vehicular networks,” in Proc. of IEEE GLOBECOM 2013, 2013. 2, 7 Bibliography 200 [16] A. Fischer, J. Botero, M. Beck, H. De Meer, and X. Hesselbach, “Virtual network embedding: A survey,” IEEE Communications Surveys and Tutorials, vol. 15, no. 4, pp. 1888–1906, 2013. 7, 20 [17] M. Chowdhury, M. Rahman, and R. Boutaba, “Vineyard: Virtual network embedding algorithms with coordinated node and link mapping,” IEEE/ACM Transactions on Networking, vol. 20, no. 1, pp. 206–219, 2012. 7, 24, 25, 26, 55, 56, 151, 169 [18] T. Taleb and A. Ksentini, “An analytical model for follow me cloud,” in Proc. of IEEE GLOBECOM 2013, Dec. 2013. 7, 65 [19] A. Ksentini, T. Taleb, and M. Chen, “A Markov decision process-based service migration procedure for follow me cloud,” in Proc. of IEEE ICC 2014, June 2014. 7, 65, 66 [20] IBM CPLEX Optimizer. [Online]. Available: http://www-01.ibm.com/ software/commerce/optimization/cplex-optimizer/ 18, 55 [21] OPTI Toolbox. [Online]. Available: http://www.i2c2.aut.ac.nz/Wiki/OPTI/ 18 [22] I. Giurgiu, C. Castillo, A. Tantawi, and M. Steinder, “Enabling efficient placement of virtual infrastructures in the cloud,” in Proceedings of the 13th International Middleware Conference, ser. Middleware ’12, 2012, pp. 332–353. 20 [23] V. V. Vazirani, Approximation Algorithms. Springer, 2001. 20, 41, 208 [24] A. Borodin and R. El-Yaniv, Online Computation and Competitive Analysis. Cambridge University Press, 1998. 21, 133 [25] N. Bansal, K.-W. Lee, V. Nagarajan, and M. Zafer, “Minimum congestion mapping in a cloud,” in Proceedings of the 30th annual ACM SIGACT-SIGOPS Bibliography 201 symposium on Principles of distributed computing, ser. PODC ’11, 2011, pp. 267–276. 24, 25, 26, 62 [26] D. Dutta, M. Kapralov, I. Post, and R. Shinde, “Embedding paths into trees: VM placement to minimize congestion,” in Algorithms – ESA 2012, ser. Lecture Notes in Computer Science, L. Epstein and P. Ferragina, Eds. Springer Berlin Heidelberg, 2012, vol. 7501, pp. 431–442. 24, 25, 26 [27] M. Alicherry and T. V. Lakshman, “Network aware resource allocation in distributed clouds,” in Proc. of IEEE INFOCOM 2012, Mar. 2012, pp. 963–971. 25, 26 [28] J.-J. Kuo, H.-H. Yang, and M.-J. Tsai, “Optimal approximation algorithm of virtual machine placement for data latency minimization in cloud systems,” in Proc. of IEEE INFOCOM 2014, 2014. 25, 26 [29] R. Hassin, A. Levin, and M. Sviridenko, “Approximating the minimum quadratic assignment problems,” ACM Trans. Algorithms, vol. 6, no. 1, pp. 18:1–18:10, Dec. 2009. 26 [30] Y.-h. Chu, S. G. Rao, and H. Zhang, “A case for end system multicast (keynote address),” SIGMETRICS Perform. Eval. Rev., vol. 28, no. 1, pp. 1–12, June 2000. 27 [31] B. Krishnamachari, D. Estrin, and S. Wicker, “The impact of data aggregation in wireless sensor networks,” in Proc. of 22nd International Conference on Distributed Computing Systems Workshops, 2002, pp. 575–578. 27 [32] D. Westhoff, J. Girao, and M. Acharya, “Concealed data aggregation for reverse multicast traffic in sensor networks: Encryption, key distribution, and routing adaptation,” IEEE Trans. on Mobile Computing, vol. 5, no. 10, pp. 1417–1431, Oct. 2006. 27 Bibliography 202 [33] IEEE 802.1D, IEEE Standard for Local and metropolitan area networks: Media Access Control (MAC) Bridges, 2004. 27 [34] A. R. Choudhury, S. Das, N. Garg, and A. Kumar, “Rejecting jobs to minimize load and maximum flow-time,” in Proc. of ACM-SIAM Symposium on Discrete Algorithms (SODA), Jan. 2015. 32 [35] Y. Azar, “On-line load balancing,” Theoretical Computer Science, pp. 218–225, 1992. 32 [36] W. B. Powell, Approximate Dynamic Programming: Solving the curses of dimensionality. John Wiley & Sons, 2007. 37, 143 [37] J. Aspnes, Y. Azar, A. Fiat, S. Plotkin, and O. Waarts, “On-line routing of virtual circuits with applications to load balancing and machine scheduling,” J. ACM, vol. 44, no. 3, pp. 486–504, May 1997. 42, 214 [38] M. Bienkowski, A. Feldmann, J. Grassler, G. Schaffrath, and S. Schmid, “The wide-area virtual service migration problem: A competitive analysis approach,” IEEE/ACM Trans. on Networking, vol. 22, no. 1, pp. 165–178, Feb. 2014. 65 [39] U. Mandal, M. Habib, S. Zhang, B. Mukherjee, and M. Tornatore, “Greening the cloud using renewable-energy-aware service migration,” IEEE Network, vol. 27, no. 6, pp. 36–43, Nov. 2013. 65 [40] D. Xenakis, N. Passas, L. Merakos, and C. Verikoukis, “Mobility management for femtocells in LTE-advanced: Key aspects and survey of handover decision algorithms,” IEEE Communications Surveys Tutorials, vol. 16, no. 1, pp. 64– 91, 2014. 65 Bibliography 203 [41] M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, 2009, vol. 414. 67, 80, 85, 98, 104, 218, 224, 225 [42] I. Post and Y. Ye, “The simplex method is strongly polynomial for deterministic markov decision processes.” in Proc. of ACM-SIAM Symposium on Discrete Algorithms (SODA), 2013, pp. 1465–1473. 82, 194 [43] H. Qian. (2015, Apr.) Counting the floating point operations (FLOPS). MATLAB Central File Exchange, No. 50608, Ver. 1.0. [Online]. Available: http://www.mathworks.com/matlabcentral/fileexchange/50608 85 [44] S. Elaydi, An Introduction to Difference Equations, Third Edition. Springer, 2005. 95 [45] M. Piorkowski, N. Sarafijanovoc-Djukic, and M. Grossglauser, “A parsimonious model of mobile partitioned networks with clustering,” in Proc. of COMSNETS, Jan. 2009. 125, 171, 185 [46] M. Piorkowski, N. Sarafijanovic-Djukic, and M. Grossglauser, “CRAWDAD data set epfl/mobility (v. 2009-02-24),” Downloaded from http://crawdad.org/epfl/mobility/, Feb. 2009. 125, 171, 185 [47] L. Kleinrock, Queuing Systems, Volume II: Computer Applications. Hoboken, NJ: John Wiley and Sons, 1976. 126 [48] M. Srivatsa, R. Ganti, J. Wang, and V. Kolar, “Map matching: Facts and myths,” in Proc. of ACM SIGSPATIAL 2013, 2013, pp. 484–487. 132 [49] S. O. Krumke, Online optimization: Competitive analysis and beyond. Habilitationsschrift Technische Universitaet Berlin, 2001. 133 Bibliography 204 [50] Y. Azar, I. R. Cohen, and D. Panigrahi, “Online covering with convex objectives and applications,” CoRR, vol. abs/1412.3507, Dec. 2014. [Online]. Available: http://arxiv.org/abs/1412.3507 133 [51] N. Buchbinder, S. Chen, A. Gupta, V. Nagarajan, and J. Naor, “Online packing and covering framework with convex objectives,” CoRR, vol. abs/1412.8347, Dec. 2014. [Online]. Available: http://arxiv.org/abs/1412.8347 133 [52] G. Aceto, A. Botta, W. de Donato, and A. Pescape, “Cloud monitoring: A survey,” Computer Networks, vol. 57, no. 9, pp. 2093 – 2115, 2013. 140 [53] K. LaCurts, J. Mogul, H. Balakrishnan, and Y. Turner, “Cicada: Introducing predictive guarantees for cloud networks,” Jun. 2014. 140 [54] E. Cho, S. A. Myers, and J. Leskovec, “Friendship and mobility: User movement in location-based social networks,” in Proc. of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ser. KDD ’11, 2011, pp. 1082–1090. 140 [55] B. Korte and J. Vygen, Combinatorial optimization. Springer, 2002. 153, 154 [56] S. Boyd and L. Vandenberghe, Convex optimization. Cambridge university press, 2004. 156, 234 [57] J. Ahrenholz. (2010) CORE download. [Online]. Available: http://downloads. pf.itd.nrl.navy.mil/core/ 175 [58] ——, “Comparison of core network emulation platforms,” in Proc. of IEEE MILCOM 2010, Oct.-Nov. 2010, pp. 166–171. 175 [59] N. Ivanic, B. Rivera, and B. Adamson, “Mobile ad hoc network emulation environment,” in Proc. of IEEE MILCOM 2009, Oct. 2009. 175 Bibliography 205 [60] R. Ogier and P. Spagnolo. (2009, Aug.) RFC 5614, Mobile ad hoc network (MANET) extension of OSPF using connected dominating set (CDS) flooding. [Online]. Available: http://www.ietf.org/rfc/rfc5614.txt 178 [61] Y. Azar, B. Kalyanasundaram, S. Plotkin, K. R. Pruhs, and O. Waarts, “On-line load balancing of temporary tasks,” Journal of Algorithms, vol. 22, no. 1, pp. 93–110, 1997. 193 [62] M. J. Neely, “Stochastic network optimization with application to communication and queueing systems,” Synthesis Lectures on Communication Networks, vol. 3, no. 1, 2010. 196 [63] R. Urgaonkar, S. Wang, T. He, M. Zafer, K. Chan, and K. K. Leung, “Dynamic service migration and workload scheduling in edge-clouds,” in Proc. of IFIP Performance 2015, Oct. 2015. 196 [64] M. Chen, S. C. Liew, Z. Shao, and C. Kai, “Markov approximation for combinatorial network optimization,” IEEE Trans. on Information Theory, vol. 59, no. 10, pp. 6301–6327, Oct 2013. 196 [65] M. J. Neely, “Distributed stochastic optimization via correlated scheduling,” IEEE/ACM Trans. on Networking, 2015, to appear. 196 A PPENDIX A Approximation Ratio for Cycle-free Mapping In the following, we focus on how well the cycle-free restriction approximates the more general case which allows cycles, for the placement of a single linear application graph. We first show that with the objective of load balancing, the problem of placing a single linear application graph onto a linear physical graph when allowing cycles is NP-hard, and then discuss the approximation ratio of the cycle-free restriction. Proposition A.1. The line-to-line placement problem while allowing cycles is NPhard. Proof. The proof is similar to the proof of Proposition 3.2 in Section 3.4.1, namely the problem can be reduced from the minimum makespan scheduling on unrelated parallel machines (MMSUPM) problem. Consider the special case where the edge demand is zero, then the problem is the same with the MMSUPM problem, which deals with placing V jobs onto N machines without restriction on their ordering, with the goal of minimizing the maximum load on each machine. To discuss the approximation ratio of the cycle-free assignment, we separately consider edge costs and node costs. The worst case ratio is then the maximum among these two ratios, because we have max {r1 x1 , r2 x2 } ≤ max {r1 , r2 } max {x1 , x2 }, where r1 , r2 , x1 , x2 ≥ 0. The variables x1 and x2 can respectively denote the true 207 optimal maximum costs at nodes and links, and the variables r1 and r2 can be their corresponding approximation ratios. Then, max {x1 , x2 } is the true optimal maximum cost when considering nodes and links together, and max {r1 , r2 } is their joint approximation ratio. The joint approximation ratio max {r1 , r2 } is tight when r1 and r2 are tight, because we can construct worst-case examples, one with zero node demand and another with zero link demand, and there must exist one worst-case example which has approximation ratio max {r1 , r2 }. The following proposition shows that cycle-free placement is always optimal when only the edge cost is considered. Proposition A.2. Cycle-free placement on tree physical graphs always brings lower or equal maximum edge cost compared with placement that allows cycles. Proof. Suppose a placement that contains cycles produces a lower maximum edge cost than any cycle-free placement, then there exists v and v1 (v1 > v + 1) both placed on a particular node n, while nodes v + 1, ..., v1 − 1 are placed on some nodes among n + 1, ..., N . In this case, placing nodes v + 1, ..., v1 − 1 all onto node n never increases the maximum edge cost, which shows a contradiction. For the node cost, we first consider the case where the physical graph is a single line. We note that in this case the cycle-free placement essentially becomes an “ordered matching”, which matches V items into N bins, where the first bin may contain items 1, ..., v1 , the second bin may contain items v1 + 1, ..., v2 , and so on. We can also see the problem as partitioning the ordered set V into N subsets, and each subset contains consecutive elements from V. Proposition A.3. When each application node has the same cost when placing it on any physical node, then the cycle-free line-to-line placement brings a tight approximation ratio of 2. 208 Proof. Suppose we have V items that can be packed into N bins by an optimal algorithm (which does not impose ordering on items), and the optimal cost at each bin is OPT. To show that the worst case cost ratio resulting from the ordering cannot be larger than 2, we consider a bin packing where the size of each bin is OPT. (Note that the bin packing problem focuses on minimizing the number of bins with given bin size, which is slightly different from our problem.) Because an optimal solution can pack our V items into N bins with maximum cost OPT, when we are given that the size of each bin is OPT, we can also pack all the V items into N bins. Hence, the optimal solution to the related bin packing problem is N . When we have an ordering, we can do the bin packing by the first-fit algorithm which preserves our ordering. The result of the first-fit algorithm has been proven to be at most 2N bins [23]. Now we can combine two neighboring bins into one bin. Because we have at most 2N bins from the first-fit algorithm, we will have at most N bins after combination. Also because each bin has size OPT in the bin packing problem, the cost after combination will be at most 2 · OPT for each bin. This shows that the worst-case cost for ordered items is at most 2 · OPT. To show that the approximation ratio of 2 is tight, we consider the following tight example. Suppose V = 2N . Among the 2N items, N of them have cost of (1 − ε)·OPT, where ε > 1 , 1+N the remaining N have cost of ε · OPT. Obviously, an optimal allocation will put one (1 − ε)·OPT item and one ε·OPT item into one bin, and the resulting maximum cost at each bin is OPT. A bad ordering could have all (1 − ε)·OPT items coming first, and all ε·OPT items coming afterwards. In this case, if we would like the maximum cost to be smaller than (2 − 2ε)·OPT, it would be impossible to fit all the items into N bins, because all the (1 − ε)·OPT items will already occupy N bins, as it is impossible to put more than one (1 − ε)·OPT item into each bin if the cost is smaller than 209 (2 − 2ε)·OPT, and also because N ε·OPT > 1 ε − 1 ε·OPT = (1 − ε)·OPT, which means that it is also impossible to put all ε·OPT into the last bin on top of the existing (1 − ε)·OPT item. As N becomes arbitrarily large and ε becomes arbitrarily small, we can conclude that the worst-case ordered assignment has at least a cost of 2·OPT. Corollary A.1. When the physical graph is a tree, and for each application node, its maximum to minimum cost when placing it on any physical node is d%,v , then the cycle-free line-to-line placement has an approximation ratio of 2V · maxv d%,v = O(V ). Proof. This follows from the fact that OPT may choose the minimum cost for each v while the ordered assignment may have to choose the maximum cost for some v, and also, in the worst case, the cycle-free placement may place all application nodes onto one physical node. The factor 2 follows from Proposition A.3. It is not straightforward to find out whether the bound in the above corollary is tight or not, thus we do not discuss it here. We conclude that the cycle-free placement always brings optimal link cost, which is advantageous because the communication bandwidth is often the more scarce resource in cloud environments. The approximation ratio of node costs can be O(V ) in some extreme cases. However, the cycle-free restriction is still reasonable in many practical scenarios. Basically, in these scenarios, one cannot split the whole workload onto all the available servers without considering the total link resource consumption. The analysis in this section is also aimed to provide some further insights that helps to justify in what practical scenarios the proposed work is applicable, while further study is worthwhile for some other scenarios. A PPENDIX B Constant-Plus-Exponential Cost Approximation to General Cost Functions We only focus on the migration cost function cm (x), because cm (x) and cd (y) have the same form. For a given cost function f (x), where x ≥ 0, we would like to approximate it with the constant-plus-exponential cost function given by cm (x) = βc + βl µx . Note that although we force cm (0) = 0 by definition, we relax that restriction here and only consider the smooth part of the cost function for simplicity. We assume that this smoothness can be extended to x = 0, so that f (x) remains smooth at x = 0. Under this definition, f (x) may be non-zero at x = 0. Because cm (x) includes both a constant term and an exponential term, we cannot obtain an exact analytical expression to minimize a commonly used error function, such as the mean squared error. We can, however, solve for the parameters that minimize the error cost function numerically. To reduce the computational complexity compared to a fully numerical solution, we propose an approximate solution in this section. We are given an integer w that may be chosen according to practical considerations, and we solve for the parameters βc , βl , and µ according to the following 211 system of equations: βc + βl = f (0) (B.1) βc + βl µw = f (w) (B.2) βc + βl µ2w = f (2w) (B.3) Subtracting (B.1) respectively from (B.2) and (B.3) and dividing the two results gives µ2w − 1 f (2w) − f (0) = w µ −1 f (w) − f (0) (B.4) subject to µw 6= 1. We can then solve µw from (B.4) which gives w µ = where R , f (2w)−f (0) f (w)−f (0) R± p R2 − 4(R − 1) 2 (B.5) ≥ 1. It follows that we always have R2 − 4(R − 1) ≥ 0 and µw ≥ 0. To guarantee that µw 6= 1, we set µw = 1 ± ε0 if |µw − 1| < ε0 , where ε0 is a small number and the sign is the same as the sign of µw − 1. Then, we have µ= R± ! w1 p R2 − 4(R − 1) 2 (B.6) We can then obtain f (0)µw − f (w) βc = µw − 1 βl = f (w) − f (0) µw − 1 (B.7) (B.8) According to (B.6), we have two possible values of µ, and correspondingly two possible sets of values of βc and βl . To determine which is better, we compute the P x 2 sum squared error 2w x=0 (f (x) − (βc + βl µ )) for each parameter set, and choose the parameter set that produces the smaller sum squared error. 212 13.5 10 Original Approximated 12.5 12 11.5 11 9 8.5 8 7.5 10.5 7 10 6.5 9.5 0 5 Original Approximated 9.5 Value of cost function Value of cost function 13 10 15 x 6 20 0 5 10 x (a) 15 20 (b) 400 Original Approximated Value of cost function 350 300 250 200 150 100 50 0 0 5 10 x 15 20 (c) Figure B.1: Examples of approximating a general cost √ function with exponential cost function: (a) f (x) = ln(x + 1) + 10, (b) f (x) = x + 1 + 5, (c) f (x) = x2 . 213 Some examples of approximation results are shown in Fig. B.1. A PPENDIX C Proofs C.1 Proof of Proposition 3.3 The proof presented here borrows ideas from [37], but is applied here to the generalized case of graph mappings and arbitrary reference offline costs Jπo . For a given ˆ we define p̃n,k (i) = pn,k (i)/J, ˆ d˜v→n,k (i) = dv→n,k (i)/J, ˆ q̃ l (i) = ql (i)/J, ˆ and J, ˆ To simplify the proof structure, we first introduce some nob̃e→l (i) = be→l (i)/J. tations so that the link and node costs can be considered in an identical manner, because it is not necessary to distinguish them in the proof of this proposition. We refer to each type of resources as an element, i.e. the type k resource at node n is an element, the resource at link l is also an element. Then, we can define the aggregated cost up to service i for element r as z̃r (i). The value of z̃r (i) can be either p̃n,k (i) or q̃ l (i) depending on the resource type under consideration. Similarly, we define w̃r|π (i) as the incremental cost that service i brings to element r P under the mapping π. The value of w̃r|π (i) can be either ∀v:π(v)=n d˜v→n,k (i) or P ∀e=(v1 ,v2 ):(π(v1 ),π(v2 ))3l b̃e→l (i). Both z̃r (i) and w̃ r|π (i) are normalized by the referˆ ence cost J. Using the above notations, the objective function in (3.12) with (3.13a) and (3.13b) becomes min πi X αz̃r (i−1)+w̃r|πi (i) − αz̃r (i−1) . (C.1) r Note that due to the notational equivalence, (C.1) is the same as (3.12) with (3.13a) C.1. Proof of Proposition 3.3 215 and (3.13b). Recall that π o denotes the reference offline mapping result, let πio denote the offline mapping result for nodes that correspond to the ith service, and z̃ro (i) denote the corresponding aggregated cost until service i. Define the following potential function: Φ(i) = X αz̃r (i) (γ − z̃ro (i)) , (C.2) r which helps us prove the proposition. Note that variables without superscript “o” correspond to the values resulting from Algorithm 3.2 that optimizes the objective function (C.1) for each service independently. The change in Φ(i) after new service arrival is Φ(i) − Φ(i − 1) X = αz̃r (i) − αz̃r (i−1) (γ − z̃ro (i − 1)) − X γ αz̃r (i−1)+w̃r|πi (i) − αz̃r (i−1) − X (C.3) X αz̃r (i−1) w̃r|πio (i) (C.4) r:∃πio (·)=r r:∃πi (·)=r ≤ αz̃r (i) w̃r|πio (i) r:∃πio (·)=r r:∃πi (·)=r ≤ X z̃ (i−1)+w̃r|πo (i) i γ αr − αz̃r (i−1) − αz̃r (i−1) w̃r|πio (i) (C.5) n o w̃ o (i) αz̃r (i−1) γ α r|πi − 1 − w̃r|πio (i) , (C.6) r:∃πio (·)=r = X r:∃πio (·)=r where the notation πi (·) = r or πio (·) = r means that service i has occupied some resource from element r when respectively using the mapping from Algorithm 3.2 or the reference offline mapping. We explain the relationships in (C.3) to (C.6) in the following. Equality (C.3) follows from Φ(i) − Φ(i − 1) X X z̃r (i−1) α (γ − z̃ro (i − 1)) = αz̃r (i) γ − z̃ro (i − 1) + w̃r|πio (i) − r r C.1. Proof of Proposition 3.3 = X 216 X αz̃r (i) w̃r|πio (i) αz̃r (i) − αz̃r (i−1) (γ − z̃ro (i − 1)) − r r = X X αz̃r (i) − αz̃r (i−1) (γ − z̃ro (i − 1)) − αz̃r (i) w̃r|πio (i), r:∃πio (·)=r r:∃πi (·)=r where the last equality follows from the fact that αz̃r (i) − αz̃r (i−1) = 0 for all r such that ∀πi (·) 6= r, and w̃r|πio (i) = 0 for all r such that ∀πio (·) 6= r. Inequality (C.4) follows from z̃ro (i − 1) ≥ 0 and z̃r (i) = z̃r (i − 1) + w̃r|πi (i). Note that the first term in (C.4) is the same as the objective function (C.1). Because the mapping πi results from Algorithm 3.2 which optimizes (C.1), we know that the reference mapping π0 must produce a cost α z̃r (i−1)+w̃r|πo (i) i − αz̃r (i−1) that is greater than or equal to the optimum, hence following (C.5). Equality (C.6) is obvious. Now we proof that the potential function Φ(i) does not increase with i, by proving that (C.6) is not larger than zero. For the ith request, the reference offline mapping produces the mapping result πio . Therefore, for all r such that ∃πio (·) = r, we have 0 ≤ w̃r|πio (i) ≤ Jπo /Jˆ ≤ 1. Hence, we only need to show that w̃ o (i) γ α r|πi − 1 − w̃r|πio (i) ≤ 0 for w̃r|πio (i) ∈ [0, 1], which is true for α ≤ 1 + 1/γ. From (C.3)–(C.6), it follows that Φ(i) ≤ Φ(i − 1). (We take α = 1 + 1/γ because this results in the smallest value of β.) Because z̃r (0) = z̃ro (0) = 0, we have Φ(0) = γ(N K + L). Because Φ(i) does ˆ we have not increase, α > 1, and z̃ro (i) ≤ 1 due to Jπo ≤ J, (γ − 1) αmaxr z̃r (i) ≤ (γ − 1) X αz̃r (i) ≤ Φ(i) ≤ Φ(0) = γ(N K + L). (C.7) r Taking the logarithm on the left and right side of (C.7), we have max z̃r (i) ≤ logα r γ(N K + L) γ−1 ˆ which proves the result because zr (i) = z̃r (i) · J. = β, (C.8) C.2. Proofs of Proposition 4.1 and Corollary 4.1 C.2 Proofs of Proposition 4.1 and Corollary 4.1 C.2.1 Proof of Proposition 4.1 217 Suppose that we are given a service migration policy π such that the service can be migrated to a location that is farther away from the user, i.e., ku−h0 k > ku−hk. We will show that, for an arbitrary sample path of the user locations u(t), we can find a (possibly history dependent) policy ψ that does not migrate to locations farther away from the user in any timeslot, which performs not worse than policy π. For an arbitrary sample path of user locations u(t), denote the timeslot t0 as the first timeslot (starting from t = 0) in which the service is migrated to somewhere farther away from the user when following policy π. The initial state at timeslot t0 is denoted by s(t0 ) = (u(t0 ), h(t0 )). When following π, the state shifts from s(t0 ) to s0π (t0 ) = (u(t0 ), h0π (t0 )), where ku(t0 ) − h0π (t0 )k > ku(t0 ) − h(t0 )k. The subsequent states in this case are denoted by sπ (t) = (u(t), hπ (t)) for t > t0 . Now, we define a policy ψ such that the following conditions are satisfied for the given sample path of user locations u(t): • The migration actions in timeslots t < t0 are the same when following either ψ or π. • The policy ψ specifies that there is no migration within the timeslots t ∈ [t0 , tm − 1], where tm > t0 is a timeslot index that is defined later. • The policy ψ is defined such that, at timeslot tm , the service is migrated to h0π (tm ), where h0π (tm ) is the service location (after possible migration at timeslot tm ) when following π. • For timeslots t > tm , the migration actions for policies ψ and π are the same. For t > t0 , the states when following ψ are denoted by sψ (t) = (u(t), hψ (t)). C.2. Proofs of Proposition 4.1 and Corollary 4.1 218 The timeslot tm is defined as the first timeslot after t0 such that the following condition is satisfied: 1. ku(tm ) − hψ (tm )k > ku(tm ) − h0π (tm )k, i.e., the transmission cost (before migration at timeslot tm ) when following ψ is larger than the transmission cost (after possible migration at timeslot tm ) when following π. Accordingly, within the interval [t0 + 1, tm − 1], the transmission cost when following ψ is always less than or equal to the transmission cost when following π, and there is no migration when following ψ. Therefore, for timeslots t ∈ [t0 , tm − 1], policy π cannot bring lower cost than policy ψ. In timeslot tm , we can always choose a migration action for ψ where the migration cost is smaller than or equal to the sum of the migration costs of π within [t0 , tm ]. The reason is that ψ can migrate (in timeslot tm ) following the same migration path as π within [t0 , tm ]. It follows that, for timeslots within [t0 , tm ], policy π cannot perform better than policy ψ, and both policies have the same costs for timeslots within [0, t0 − 1] and [tm + 1, ∞). The above procedure can be repeated so that all the migration actions to a location farther away from the user can be removed without increasing the overall cost, thus we can redefine ψ to be a policy that removes all such migrations. We note that the policy ψ can be constructed based on policy π without prior knowledge of the user’s future locations. For any policy π, the policy ψ is a policy that does not migrate whenever π migrates to a location farther away from the user (corresponding to timeslot t0 ). Then, it migrates to h0π (tm ) when condition 1 is satisfied (this is the timeslot tm in the above discussion). The policy ψ is a history dependent policy, because its actions depend on the past actions of the underlying policy π. From [41, Chapter 6], we know that history dependent policies cannot outperform Markovian policies for our problem, assuming that the action spaces of both policies are identical for every possible state. There- C.2. Proofs of Proposition 4.1 and Corollary 4.1 219 fore, there exists a Markovian policy that does not migrate to a location farther away from the user, which does not perform worse than π. Noting that the optimal policy found from (4.2) and (4.3) are Markovian policies, we have proved the proposition. C.2.2 Proof of Corollary 4.1 The proof follows the same procedure as the proof of Proposition 4.1. For any given policy π that migrates to locations other than the user location, we show that there exists a policy ψ that does not perform such migration, which performs not worse than the original policy π. The difference is that t0 is defined as the first timeslot such that u(t0 ) 6= h0π (t0 ), and tm is defined as the first timeslot after t0 such that u(tm ) = h0π (tm ). Due to the strict inequality relationship of the cost functions given in the corollary, and because 0 < γ < 1, we can conclude that π is not optimal. C.3. Proof of Proposition 4.5 C.3 220 Proof of Proposition 4.5 Recall that among the neighbors (i0 , j 0 ) of a cell (i, j) in the 2-D offset-based MDP {e(t)}, when i > 0, we have two (or, correspondingly, one) cells with i0 = i − 1, and two (or, correspondingly, three) cells with i0 = i + 1. To “even out” the different number of neighboring cells, we define a new (modified) MDP {g(t)} for the 2-D offset-based model, where the states are connected as in the original 2-D MDP, but the transition probabilities are different, as shown in Fig. C.1. In the modified MDP {g(t)}, the transition probabilities starting from state (0, 0) to each of its neighboring cells have the same value r. For all the other states (i, j) 6= (0, 0), they are defined as follows: • The transition probability to each of its neighbors with the same ring index i is r. • If state (i, j) has two (correspondingly, one) neighbors in the lower ring i − 1, then the transition probability to each of its neighbors in the lower ring is 1.5 r 2 (correspondingly, 1.5r). • if state (i, j) has two (correspondingly, three) neighbors in the higher ring i+1, then the transition probability to each of its neighbors in the higher ring is (correspondingly, 2.5 r 2 2.5 r). 3 We denote the discounted sum cost from the original MDP {e(t)} by V (i, j), and denote that from the modified MDP {g(t)} by U (i, j), where (i, j) stands for the initial state in the discounted sum cost definition. C.3. Proof of Proposition 4.5 221 (3 1) (3,1) (2,0) (2 11) (2,11) r r r r r r r (2 1) (2,1) r r (1 0) (1,0) r (3 2) (3,2) (1 1) (1,1) r ( ) (0,0) (a) (3 1) (3,1) (2,0) (2 11) (2,11) 1 .5 r 2 .5 3 2 .5 3 r 2 .5 2 r 2 .5 3 1 .5 2 (1 0) (1,0) 1 .5 r ( ) (0,0) r r r (3 2) (3,2) r 2 .5 2 r (2 1) (2,1) 1 .5 2 r (1 1) (1,1) r (b) Figure C.1: Illustration of original and modified 2-D MDPs, only some exemplar states and transition probabilities are shown: (a) original, (b) modified. C.3. Proof of Proposition 4.5 222 Part I – Upper bound on the difference between V (i, j) and U (i, j) for a given policy π Assume we have the same policy π for the original and modified MDPs. Then, the balance equation of V (i, j) is V (i, j) = Ca (i, j) + γ (1 − 6r) V (a(i, j)) + r X V (i0 , j 0 ) (C.9) (i0 ,j 0 )∈N (a(i,j)) where a(i, j) is the new state after possible migration at state (i, j), and N (a(i, j)) is the set of states that are neighbors of state a(i, j). For U (i, j), we have U (i, j) = Ca (i, j) + γ (1 − 6r) U (a(i, j)) ! + X Pg0 (t)=a(i,j),g(t+1)=(i0 ,j 0 ) U (i0 , j 0 ) (C.10) (i0 ,j 0 )∈N (a(i,j)) where Pg0 (t)=a(i,j),g(t+1)=(i0 ,j 0 ) is the transition probability of the modified MDP {g(t)} as specified earlier. In the following, let ia(i,j) denote the ring index of a(i, j). We define sets N − (a(i, j)) = (i0 , j 0 ) ∈ N (a(i, j)) : i0 = ia(i,j) − 1 N + (a(i, j)) = (i0 , j 0 ) ∈ N (a(i, j)) : i0 = ia(i,j) + 1 to represent the neighboring states of a(i, j) that are respectively in the lower and higher rings. We use |·| to denote the number of elements in a set. Assume |U (i, j) − V (i, j)| ≤ ε for all i and j, and the value of ε is unknown for now. We subtract (C.9) from (C.10), and then take the absolute value, yielding (C.11), (C.12), and (C.13) which are explained below, where the set N2 (i, j) is the C.3. Proof of Proposition 4.5 223 |U (i, j) − V (i, j)| = γ (1 − 6r) (U (a(i, j)) − V (a(i, j))) X + Pg0 (t)=a(i,j),g(t+1)=(i0 ,j 0 ) (U (i0 , j 0 ) − V (i0 , j 0 )) (i0 ,j 0 )∈N (a(i,j)) P ± 0.5λia(i,j) r (i0 ,j 0 )∈N + (a(i,j)) V (i0 , j 0 ) |N + (a(i, j))| P − (i0 ,j 0 )∈N − (a(i,j)) V (i0 , j 0 ) |N − (a(i, j))| ! (C.11) P P 0 0 0 0 V (i , j ) V (i , j ) 0 0 + 0 0 − (i ,j )∈N (a(i,j)) (i ,j )∈N (a(i,j)) − ≤ γε + 0.5γr |N + (a(i, j))| |N − (a(i, j))| ≤ γε + 0.5γr max i,j,j 0 :(i+1,j)∈N 2 (i−1,j 0 ) |(V (i + 1, j) − V (i − 1, j 0 ))| (C.12) (C.13) set of states that are two-hop neighbors of state (i, j), the variable λia(i,j) = 0 when ia(i,j) = 0, and λia(i,j) = 1 when ia(i,j) > 0. The first two terms of (C.11) subtract the discounted sum cost of the original MDP {e(t)} from that of the modified MDP {g(t)}, by assuming that both chains have the same transition probabilities specified by the modified MDP. The difference in their transition proabilities is captured by the last term of (C.11). There is no difference in the transition probabilities when ia(i,j) = 0, thus λia(i,j) = 0 when ia(i,j) = 0. In the following, we consider ia(i,j) > 0 and further explain the last term of (C.11). We first note that there is difference in the transition probabilities only when moving to the lower or higher ring: • The sum probability of moving to the lower ring in {g(t)} is by 0.5r smaller (or, correspondingly, greater) than that in {e(t)}. • The sum probability of moving to the higher ring in {g(t)} is by 0.5r greater (or, correspondingly, smaller) than that in {e(t)}. Therefore, the transition probablity difference for each neighboring state in the lower C.3. Proof of Proposition 4.5 224 (or higher) ring is ±0.5r divided by the number of neighbors in the lower (or higher) ring. Also note that the probablity difference for lower and higher rings have opposite signs. This explains the last term of (C.11), which captures the difference in the transition probabilities and its impact on the discounted sum costs. The inequality in (C.12) is from the triangle inequality. We note that the subtraction in the last term of (C.11) only occurs on V (i0 , j 0 ) values that are two-hop neighbors, so we have the inequality in (C.13) by replacing the value with the maximum. From (C.13), we can obtain a balance equation for the upper bound of |U (i, j) − V (i, j)|, which is ε = γε + 0.5γr max |(V (i+1, j)−V (i−1, j 0 ))| i,j,j 0 :(i+1,j)∈N2 (i−1,j 0 ) (C.14) Because 0 < γ < 1, the value of V (i, j) converges after a number of iterations according to (C.9) [41, Chapter 6], so |(V (i + 1, j) − V (i − 1, j))| also converges, and the value of ε can be solved by εV = γr maxi,j,j 0 :(i+1,j)∈N2 (i−1,j 0 ) |(V (i+1, j)−V (i−1, j 0 ))| 2(1 − γ) (C.15) Note that the above argument also applies when interchanging V and U , which means that an alternative upper bound of the cost is εU = γr maxi,j,j 0 :(i+1,j)∈N2 (i−1,j 0 ) |(U (i+1, j)−U (i−1, j 0 ))| 2(1 − γ) (C.16) The upper bound can also be expressed as ε = min {εV , εU }, but either εV or εU may have the smaller value. C.3. Proof of Proposition 4.5 225 Part II – Optimal policy for the modified 2-D MDP {g(t)} is equivalent to the optimal policy for the distance-based MDP {d(t)} We note that the optimal policy of an MDP can be found from value iteration [41, Chapter 6]. For the modified MDP {g(t)}, we initialize with a never migrate policy, which gives the initial value function U0 (i, j) = cd (i), satisfying U0 (i, j) = U0 (i, j 0 ) for all i and j 6= j 0 . Suppose Un (i, j) = Un (i, j 0 ) for all i and j 6= j 0 . In each iteration, we use the following equation to obtain the new value function and the corresponding actions for each i and j: ( Un+1 (i, j) = min Ca (i, j) a ) +γ XX i0 Pg0 (t)=a(i,j),g(t+1)=(i0 ,j 0 ) Un (i0 , j 0 ) (C.17) j0 From the hexagon model in Fig. 4.9, we can see that for ring indices i and i0 , where i0 < i, we can always reach from state (i, j) to a state in ring i0 with i − i0 hops, regardless of the index j. In the (n + 1)th iteration, if it is optimal to migrate from state (i, j) to a state with ring index i0 , then the migration destination must be i − i0 hops away from origin state, because Un (i0 , j) = Un (i0 , j 0 ) for all i0 and j 6= j 0 , it cannot be beneficial to migrate to somewhere farther away. Further, if it is optimal to migrate at a state (i, j) to a state in ring i0 , it must be optimal to migrate at states (i, j) for all j to a state (which may not be the same state) in ring i0 , bringing the same cost, i.e. Un+1 (i, j) = Un+1 (i, j 0 ) for j 6= j 0 . This is due to the symmetry of cost functions (in the sense that Un (i, j) = Un (i, j 0 ) for all i and j 6= j 0 ) and symmetry of transition probabilities (in the sense that the sum probability of reaching ring i − 1 from any state in ring i is the same, and the sum probability of reaching ring i + 1 from any state in ring i is also the same). Similarly, C.3. Proof of Proposition 4.5 226 if it is optimal not to migrate at a state (i, j), then it is optimal not to migrate at states (i, j) for all j, which also brings Un+1 (i, j) = Un+1 (i, j 0 ) for any j 6= j 0 . Because the value iteration converges to the optimal policy and its corresponding cost as n → ∞, for the optimal policy of the modified MDP {g(t)}, we have the same discounted sum cost for states in the same ring, i.e. U ∗ (i, j) = U ∗ (i, j 0 ) for all i and j 6= j 0 . Meanwhile, for a given i, the optimal actions a∗ (i, j) and a∗ (i, j 0 ) for any j 6= j 0 have the same ring index ia∗ (i,j) . Since the optimal actions a∗ (i, j) only depend on the ring index i and the ring index of a∗ (i, j) does not change with j, the optimal policy for {g(t)} can be directly mapped to a policy for the distance-based MDP {d(t)}. A policy for {d(t)} can also be mapped to a policy for {g(t)} by considering the shortest path between different states in {g(t)}, as discussed in Section 4.4.3.2. This implies that there is a one-toone mapping between the optimal policy for {g(t)} and a policy for {d(t)}, because the optimal policy for {g(t)} also only migrates along the shortest path between states. We now show that the policy for {d(t)} obtained from the optimal policy for {g(t)} is optimal for {d(t)}. To find the optimal policy for {d(t)}, we can perform value iteration according to the following update equation: ( Un+1 (i) = min Ca (i) a ! +γ X X i0 Pg(t)=a(i),g(t+1)=(i0 ,j 0 ) ) Un (i0 ) (C.18) j0 The difference between (C.17) and (C.18) is that (C.18) does not distinguish the actions and value functions with different j indices. Recall that for the modified 2-D MDP {g(t)}, we have Un (i, j) = Un (i, j 0 ) for all n, i and j 6= j 0 , so the index j can be natually removed from the value functions. Further, if it is optimal to migrate at state (i, j) to a state in ring i0 , it must be optimal to migrate at states (i, j) for all j to C.3. Proof of Proposition 4.5 227 a state (which may not be the same state) in ring i0 . The migration cost for different j are the same because they all follow the shortest path from state (i, j) to ring i0 . If it is optimal not to migrate at state (i, j), then it is optimal not to migrate at states (i, j) for all j. Therefore, we can also remove the j index associated with the actions, without affecting the value function. It follows that the optimal policy for {g(t)} is equivalent to the optimal policy for {d(t)}, both bringing the same value functions (discounted sum costs). Part III – Error bound for distance-based approximation By now, we have shown the upper bound on the discounted sum cost difference between the original and modified 2-D MDPs {e(t)} and {g(t)}, when both MDPs use the same policy. We have also shown that the optimal policy for the modified 2-D MDP {g(t)} is equivalent to the optimal policy for the distance-based MDP {d(t)}. Note that the true optimal cost is obtained by solving for the optimal policy for the original 2-D MDP {e(t)}, and the approximate optimal cost is obtained by applying the optimal policy for {d(t)} to {e(t)}. In the following, we consider the upper bound on the difference between the true and approximate optimal discounted sum costs. ∗ for {e(t)}, denote the discounted sum We start with a (true) optimal policy πtrue ∗ (i, j). When using the same policy on {g(t)}, the costs from this policy as Vπtrue ∗ (i, j) and Vπ ∗ (i, j) satisfies the upper bound given difference between the costs Uπtrue true in (C.15), i.e. ∗ (i, j) − Vπ ∗ (i, j) ≤ εV ∗ Uπtrue true π true ∗ (i, j) are the optimal costs, we have Since Vπtrue max i,j,j 0 :(i+1,j)∈N2 (i−1,j 0 ) Vπ∗ (i+1, j)−Vπ∗ (i−1, j 0 ) true true (C.19) C.3. Proof of Proposition 4.5 ≤ max {cm (x + 2) − cm (x)} x 228 (C.20) because, otherwise, there exists at least one pair of states (i + 1, j) and (i − 1, j 0 ) for which it is beneficial to migrate from state (i + 1, j) to state (i − 1, j 0 ), according to a 2-D extension of (4.29). The cost after performing such migration is upper bounded ∗ by (C.20), which contradicts with the fact that πtrue is optimal. Define εc = γr maxx {cm (x + 2) − cm (x)} 2(1 − γ) (C.21) From (C.15), (C.19) and (C.20), we have ∗ (i, j) − Vπ ∗ (i, j) ≤ εc Uπtrue true (C.22) According to the equivalence of {g(t)} and {d(t)}, we know that the optimal ∗ policy πappr of {d(t)} is also optimal for {g(t)}. Hence, we have ∗ (i, j) ≤ Uπ ∗ (i, j) Uπappr true (C.23) because the cost from the optimal policy cannot be higher than the cost from any other policy. ∗ ∗ (i, j). From (C.16), and on {e(t)}, we get costs Vπappr When using the policy πappr ∗ (i, j) is the optimal cost for {g(t)}, we have because Uπappr ∗ (i, j) − Uπ ∗ (i, j) ≤ εU ∗ Vπappr ≤ εc appr πappr (C.24) From (C.22), (C.23), and (C.24), we get ∗ (i, j) − Vπ ∗ (i, j) ≤ 2εc Vπappr true (C.25) C.3. Proof of Proposition 4.5 which completes the proof. 229 C.4. Proof of Proposition 5.2 C.4 230 Proof of Proposition 5.2 We first introduce a few lemmas, with results used later in the proof. Lemma C.1. For any instance j and configuration sequence λ, we have e ∂D e (y, z) · (ajλ , bjλ ) (x) = ∇y,z D ∂xjλ (C.26) Proof. t +T −1 " N X e ∂D ∂yn (t) (x) · (x) + ∂y (t) ∂x n jλ n=1 0 # N N X X ∂D e ∂znh (t) (x) · (x) ∂znh (t) ∂xjλ n=1 h=1 " N t0X +T −1 X e ∂D = (x) · ajλn (t)+ ∂yn (t) t=t0 n=1 # N X N X e ∂D (x) · bjλnh (t) ∂z (t) nh n=1 h=1 0X e ∂D (x) = ∂xjλ t=t e (y, z) · (ajλ , bjλ ) = ∇y,z D where we recall that yn (t) and znh (t) are functions of xjλ for all j and λ, thus they are also functions of vector x. Lemma C.2. For any instance j and configuration sequence λ, we have e (x) · x = ∇y,z D e (y, z) · (y, z) ∇x D Proof. e (x) · x ∇x D M X X e ∂D (x) · xjλ = ∂x jλ j=1 λ∈Λ (C.27) C.4. Proof of Proposition 5.2 = M X X 231 e (y, z) · (ajλ , bjλ ) · xjλ ∇y,z D j=1 λ∈Λ e (y, z) · = ∇y,z D M X X ! (ajλ , bjλ ) · xjλ j=1 λ∈Λ e (y, z) · (y, z) = ∇y,z D where the second step follows from Lemma C.1, the last step follows from the definition of vectors y, z, ajλ , bjλ . We introduce some additional notations that are used in the proof below. Recall that the values of vectors x, y, and z may vary over time due to service arrivals (j) (j) (j) and departures. Let xj , yj , and zj respectively denote the values of x, y, and (j) (j) (j) z immediately after instance j is placed; and let xj−1 , yj−1 , and zj−1 respectively denote the values of x, y, and z immediately before instance j is placed. We note that the values of x, y, and z may change after placing each instance. Therefore, the notions of “before”, “after”, and “time” (used below) here correspond to the sequence of service instance placement, instead of the actual physical time. We then introduce vectors that only consider the placement up to the jth service instance, which are necessary because the proof below uses an iterative approach. Let xj , yj , and zj respectively denote the values of x, y, and z at any time after placing instance j (where instance j can be either still running in the system or already departed) while ignoring the placement of any subsequent instances j 0 > j (if any). This means, in vector xj , we set (xj )iλ , xiλ for any i ≤ j and λ, and set (xj )iλ , 0 for any i > j and λ, although the value of xiλ at the current time of interest may be non-zero for some i > j and λ. Similarly, in vectors yj and zj , we only consider the resource consumptions up to instance j, i.e., (yj )nt , Pj P Pj P i=1 λ∈Λ aiλn (t)xiλ and (zj )nht , i=1 λ∈Λ biλnh (t)xiλ for any n, h, and t. We assume that the last service instance that has arrived before the current time C.4. Proof of Proposition 5.2 232 of interest has index M , thus x = xM , y = yM , and z = zM . Because an instance will never come back after it has departed (even if an instance of the same type comes back, it will be given a new index), we have yj−1 ≤ (j) (j) yj−1 and zj−1 ≤ zj−1 , where the inequalities are defined element-wise for the vector. e y(j) , z(j) − D e y(j) , z(j) to denote the increase in the sum Define vj , D j j j−1 j−1 e e cost D(y, z) (or equivalently, D(x)) at the time when placing service j. Note that e (yj , zj ) − D e (yj−1 , zj−1 ) may vary over time, after this placement, the value of D because some services i ≤ j may leave the system, but the value of vj is only taken when service j is placed upon its arrival. Lemma C.3. When Assumption 5.1 is satisfied, for any M , we have e (xM ) ≤ D M X vj (C.28) j=1 Proof. Assume that service j takes configuration λ0 after its placement (and before it (j) (j) (j) (j) possibly unpredictably departs), then yj − yj−1 = ajλ0 and zj − zj−1 = bjλ0 . For any time after placing instance j we define ∆yj , yj − yj−1 and ∆zj , zj − zj−1 . We always have ∆yj = ajλ0 , ∆zj = bjλ0 , if instance j has not yet departed from the system, and ∆yj = ∆zj = 0 if j has already departed from the system. e (yj , zj ) is convex non-decreasing (from Lemma 5.1), we have Noting that D e (yj , zj ) − D e (yj−1 , zj−1 ) D e (yj−1 + ∆yj , zj−1 + ∆zj ) − D e (yj−1 , zj−1 ) =D (j) (j) (j) (j) e e ≤ D yj−1 + ∆yj , zj−1 + ∆zj − D yj−1 , zj−1 e y(j) + ajλ0 , z(j) + bjλ0 − D e y0j−1 , z0j−1 ≤D j−1 j−1 e y(j) , z(j) − D e y(j) , z(j) =D j j j−1 j−1 = vj (C.29) (C.30) C.4. Proof of Proposition 5.2 233 (j) (j) where inequality (C.29) is because yj−1 ≤ yj−1 , zj−1 ≤ zj−1 (see discussion above) e (yj , zj ); inequality (C.30) is and due to the convex non-decreasing property of D because ∆yj ≤ ajλ0 , ∆zj ≤ bjλ0 and also due to the non-decreasing property of e (yj , zj ). D e (x0 ) = 0, where x0 = 0 and 0 is defined as a vector with all We now note that D zeros, thus y0 = z0 = 0. We have M X j=1 M h i X e e vj ≥ D (yj , zj ) − D (yj−1 , zj−1 ) j=1 e (xM ) − D e (x0 ) = D e (xM ) =D Lemma C.4. When Assumption 5.1 is satisfied, for any j and λ, we have vj ≤ φ e ∂D (xM ) ∂xjλ (C.31) where φ is a constant satisfying (5.12). Proof. Assume that service j takes configuration λ0 after its placement (and before it possibly unpredictably departs). Because we perform a greedy assignment in Algorithm 5.3, we have e y(j) , z(j) − D e y(j) , z(j) vj = D j j j−1 j−1 (j) (j) (j) (j) e e = D yj−1 + ajλ0 , zj−1 + bjλ0 − D yj−1 , zj−1 e y(j) + ajλ , z(j) + bjλ − D e y(j) , z(j) ≤D j−1 j−1 j−1 j−1 for any λ ∈ Λi . C.4. Proof of Proposition 5.2 234 Then, we have (j) (j) (j) (j) e e D yj−1 + ajλ , zj−1 + bjλ − D yj−1 , zj−1 e y(j) + ajλ , z(j) + bjλ · (ajλ , bjλ ) ≤ ∇y,z D j−1 j−1 (C.32) e (ymax + ajλ , zmax + bjλ ) · (ajλ , bjλ ) ≤ ∇y,z D (C.33) e (yM , zM ) · (ajλ , bjλ ) ≤ φ∇y,z D (C.34) =φ e ∂D (xM ) ∂xjλ (C.35) where “·” denotes the dot-product. The above relationship is explained as follows. Inequality (C.32) follows from the first-order conditions of convex functions [56, Section 3.1.3]. The definition of ymax and zmax in Proposition 5.2 gives (C.33). The definition of φ in (5.12) gives (C.34). Equality (C.35) follows from Lemma C.1. This completes the proof. Using the above lemmas, we now proof Proposition 5.2. e Proof. (Proposition 5.2) Due to the convexity of D(x), from the first-order conditions of convex functions [56, Section 3.1.3], we have ∗ e e D(φψx M ) − D(xM ) e (xM ) · (φψx∗M − xM ) ≥ ∇x D (C.36) e (xM ) · x∗ − ∇x D e (xM ) xM = φψ∇x D M (C.37) = M X X φψx∗iλ i=1 λ∈Λ =ψ M X X i=1 e ∂D e (xM ) · xM (xM ) − ∇x D ∂xiλ e (xM ) · xM e ∂D ∇x D (xM ) − x∗iλ φ ∂xiλ ψ λ∈Λ (C.38) ! where x∗iλ is the (i, λ)th element of vector x∗M . From Lemma C.4, we have (C.39) C.4. Proof of Proposition 5.2 Eq. (C.39) ≥ ψ 235 i=1 =ψ M X i=1 From the constraint P λ∈Λ e (xM ) · xM ∇x D x∗iλ vi − ψ λ∈Λ ! e (xM ) · xM ∇x D vi x∗iλ − ψ λ∈Λ ! M X X (C.40) X (C.41) x∗iλ = 1 and the definition of ψ, we get M X e (xM ) · xM ∇x D Eq. (C.41) = ψ vi − ψ i=1 ! M X e M) ≥ψ vi − D(x ! (C.42) (C.43) i=1 ≥0 (C.44) where the last equality follows from Lemma C.3. This gives (5.10). Equation (5.11) follows from the fact that yn,j (t) and znh,j (t) are both linear in xiλ . e (x) = The last equality in (5.13) follows from Lemma C.2 and the fact that D e (y, z) as well as x = xM , y = yM , and z = zM . D C.5. Proof of Proposition 5.4 C.5 236 Proof of Proposition 5.4 Lemma C.5. For polynomial functions Ξ1 (y) and Ξ2 (y) in the general form: Ξ1 (y) , Ω X (ρ) ω1 y ρ ρ=0 Ξ2 (y) , Ω X (ρ) ω2 y ρ ρ=0 (ρ) where the constants ω1 (Ω) ω2 (ρ) ≥ 0 and ω2 (Ω) ≥ 0 for 0 ≤ ρ < Ω, while ω1 > 0 and > 0, we have (Ω) Ξ1 (y) ω lim = 1(Ω) y→+∞ Ξ2 (y) ω2 Proof. When Ω = 0, we have (0) Ξ1 (y) ω lim = 1(0) y→+∞ Ξ2 (y) ω2 When Ω > 0, we note that limy→+∞ Ξ1 (y) = +∞ and limy→+∞ Ξ2 (y) = +∞, (Ω) because ω1 (Ω) > 0 and ω2 > 0. We apply the L’Hospital’s rule and get dΞ1 (y) dy dΞ2 (y) dy PΩ ρ=1 lim PΩ y→+∞ ρ=1 Ξ1 (y) lim = lim y→+∞ Ξ2 (y) y→+∞ = (ρ) ρω1 y ρ−1 (ρ) ρω2 y ρ−1 (C.45) Suppose we have PΩ Qk−1 (ρ) ρ−k ρ=k m=0 (ρ − m) ω1 y Ξ1 (y) Q lim = lim P (ρ) Ω k−1 y→+∞ Ξ2 (y) y→+∞ (ρ − m) ω2 y ρ−k ρ=k m=0 (C.46) which equals to (C.45) for k = 1. For 1 ≤ k < Ω, we note that Ω − k > 0, hence the numerator and denominator in the right hand-side (RHS) of (C.46) still respectively C.5. Proof of Proposition 5.4 237 (Ω) (Ω) approach +∞ when y → +∞ (because ω1 > 0 and ω2 > 0). Let Ψ(k) denote the RHS (C.46), we can reapply the L’Hospital’s rule on Ψ(k), yielding (ρ) (ρ − m) ω1 y ρ−(k+1) ρ=k+1 m=0 Q Ψ(k) = lim P (ρ) (k+1)−1 Ω y→+∞ (ρ − m) ω2 y ρ−(k+1) m=0 ρ=k+1 PΩ Q (k+1)−1 = Ψ(k + 1) which proofs that (C.46) holds for 1 ≤ k ≤ Ω. Therefore, (Ω) (Ω) Ξ1 (y) ρ!ω1 ω = Ψ(Ω) = = 1(Ω) (Ω) y→+∞ Ξ2 (y) ρ!ω2 ω2 lim 0 Lemma C.6. For variables 0 ≤ y ≤ y 0 , 0 ≤ yn ≤ yn0 , 0 ≤ yh ≤ yh0 , 0 ≤ znh ≤ znh , we always have dun,t 0 dun,t (y) ≤ (y ) dy dy ∂wnh,t ∂wnh,t 0 0 0 (yn , yh , znh ) ≤ (yn , yh , znh ) ∂Υ ∂Υ (C.47) (C.48) where Υ stands for either yn , yh , or znh . Proof. We note that X (ρ) dun,t (y) = ργn,t y ρ−1 dy ρ (ρ) from which (C.47) follows directly because γn,t ≥ 0. We then note that XXX ∂wnh,t (ρ1 ,ρ2 ,ρ3 ) ρ1 −1 ρ2 ρ3 (yn , yh , znh ) = ρ1 κnh,t yn yh znh ∂yn ρ ρ ρ 1 2 3 (ρ ,ρ ,ρ3 ) 1 2 from which (C.48) follows for Υ = yn because κnh,t follows for Υ = yh and Υ = znh . ≥ 0. Similarly, (C.48) also C.5. Proof of Proposition 5.4 238 Lemma C.7. Let Ω denote the highest order of the polynomial cost functions. Specif(ρ) (ρ ,ρ ,ρ3 ) 1 2 ically, Ω , max{ρ; ρ1 + ρ2 + ρ3 }, subject to γn,t > 0 and κnh,t > 0. Assume that the cost functions are defined as in (5.16) and (5.17), then for any constants δ > 0, B ≥ 0, there exist sufficiently large values of y, yn , yh , znh , such that dun,t (y + B) dy dun,t (y) dy dun,t (y) · y dy un,t (y) ∂wnh,t (yn + B, yh + B, znh + B) ∂Υ ∂wnh,t (yn , yh , znh ) ∂Υ ∂wnh,t (yn , yh , znh ) · Υ ∂Υ wn,t (yn , yh , znh ) ≤1+δ (C.49) ≤Ω+δ (C.50) ≤1+δ (C.51) ≤Ω+δ (C.52) for any n, h, t, where Υ stands for either yn , yh , or znh . (ρ) Proof. Let Ω0 denote the maximum value of ρ such that γn,t > 0, we always have Ω0 ≤ Ω. We note that dun,t (y + B) dy dun,t (y) dy dun,t (y) dy PΩ0 (ρ) ρ=1 = ργn,t (y + B)ρ−1 PΩ0 (ρ) ρ=1 ·y un,t (y) ργn,t y ρ−1 (C.53) PΩ0 (ρ) ρ ρ=1 ργn,t y PΩ0 (ρ) ρ ρ=1 γn,t y = (C.54) According to Lemma C.5, we have dun,t (y + B) dy lim dun,t y→+∞ (y) dy dun,t (y) · y dy lim y→+∞ un,t (y) (Ω0 ) = Ω0 γn,t (Ω0 ) Ω0 γn,t =1 (C.55) = Ω0 (C.56) (Ω0 ) = Ω0 γn,t (Ω0 ) γn,t where we note that after expanding the numerator in the RHS of (C.53), the constant 0 B does not appear in the coefficient of y Ω −1 . C.5. Proof of Proposition 5.4 239 Now, define a variable q > 0, and we let yn = ζ1 q, yh = ζ2 q, znh = ζ3 q, where ζ1 , ζ2 , ζ3 > 0 are arbitrary constants. We also define ρ , ρ1 +ρ2 +ρ3 . Using ζ1 , ζ2 , ζ3 , and q, we can represent any value of (yn , yh , znh ) > 0. With this definition, we have wnh,t (q) , wnh,t (ζ1 q, ζ2 q, ζ3 q) X X X (ρ ,ρ ,ρ ) ρ ρ ρ 1 2 3 = κnh,t ζ1 1 ζ2 2 ζ3 3 q ρ1 +ρ2 +ρ3 ρ1 ρ2 ρ3 Ω00 = X (ρ) (κ0 )nh,t q ρ (C.57) ρ=1 where the constant X (ρ) (κ0 )nh,t , (ρ ,ρ ,ρ3 ) ρ1 ρ2 ρ3 ζ1 ζ2 ζ3 1 2 κnh,t {(ρ1 ,ρ2 ,ρ3 ):ρ1 +ρ2 +ρ3 =ρ} (ρ) and Ω00 is defined as the maximum value of ρ such that (κ0 )nh,t > 0. We always have Ω00 ≤ Ω. Note that (C.57) is in the same form as (5.16). Following the same procedure as for obtaining (C.55) and (C.56), we get dwnh,t (q + B 0 ) dq lim dwnh,t q→+∞ (q) dq dwnh,t (q) · q dq lim q→+∞ where B 0 , wn,t (q) (Ω00 ) = Ω00 γn,t (Ω00 ) Ω00 γn,t =1 (C.58) = Ω00 (C.59) (Ω00 ) = Ω00 γn,t (Ω00 ) γn,t B . min{ζ1 ;ζ2 ;ζ3 } According to the definition of limits, for any δ > 0, there exist sufficiently large values of y and q (thus yn , yh , znh ), such that dun,t (y + B) dy dun,t (y) dy dun,t (y) · y dy un,t (y) ≤1+δ (C.60) ≤ Ω0 + δ (C.61) C.5. Proof of Proposition 5.4 240 dwnh,t (q + B 0 ) dq dwnh,t (q) dq dwnh,t (q) · q dq wn,t (q) ≤1+δ (C.62) ≤ Ω00 + δ (C.63) for any n, h, t. Because dwnh,t (q + B 0 ) dq dwnh,t (q) dq = dwnh,t (q + B 0 ) d(ζq) dwnh,t (q) d(ζq) dwnh,t dwnh,t (q) · q = (q) · ζq dq d(ζq) for any ζ > 0, we can also express the bounds (C.62) and (C.63) in terms of yn , yh , znh , yielding ∂wnh,t (yn ∂Υ + ζ1 B 0 , yh + ζ2 B 0 , znh + ζ3 B 0 ) ∂wnh,t (yn , yh , znh ) ∂Υ ∂wnh,t (yn , yh , znh ) ∂Υ ≤1+δ ·Υ ≤ Ω00 + δ wn,t (yn , yh , znh ) (C.64) (C.65) where Υ stands for either yn , yh , or znh . According to the definition of B 0 , we have B ≤ ζ1 B 0 , B ≤ ζ2 B 0 , B ≤ ζ3 B 0 . From Lemma C.6, we have ∂wnh,t (yn + B, yh + B, znh + B) ∂Υ ∂wnh,t ≤ (yn + ζ1 B 0 , yh + ζ2 B 0 , znh + ζ3 B 0 ) ∂Υ (C.66) Combining (C.66) with (C.64) and noting that Ω0 ≤ Ω and Ω00 ≤ Ω, together with (C.60), (C.61), and (C.65), we get (C.49)–(C.52). Lemma C.8. For arbitrary values ϑ1,k ≥ 0 and ϑ2,k ≥ 0 for all k = 1, 2, ..., K, where ϑ1,k and ϑ2,k are either both zero or both non-zero and there exists k such that C.5. Proof of Proposition 5.4 241 ϑ1,k and ϑ2,k are non-zero, if the following bound is satisfied: ϑ1,k ≤Θ {k∈{1,...,K}:ϑ1,k 6=0,ϑ2,k 6=0} ϑ2,k max then we have PK k=1 PK ωk ϑ1,k k=1 ωk ϑ2,k ≤Θ for any ωk ≥ 0. Proof. Because ϑ1,k ≤ Θϑ2,k for all k, we have K X ωk ϑ1,k ≤ k=1 K X ωk Θϑ2,k k=1 yielding the result. Lemma C.9. When Assumption 5.2 is satisfied and the window size T is a constant, there exists a constant B ≥ 0 such that (ymax + aiλ , zmax + biλ ) − (y, z) ≤ Be (C.67) for any i and any λ ∈ Λi , where e , [1, ..., 1] is a vector of all ones that has the same dimension as (y, z). Proof. We note that (ymax + aiλ , zmax + biλ ) − (y, z) ≤ (ymax + amax ey , zmax + bmax ez ) − (y, z) (C.68) ≤ (amax (Bd T + 1) ey , bmax (Bd T + 1) ez ) (C.69) ≤ max {amax (Bd T + 1) ; bmax (Bd T + 1)} · e (C.70) where ey , [1, ..., 1] and ez , [1, ..., 1] are vectors of all ones that respectively have C.5. Proof of Proposition 5.4 242 the same dimensions as y and z. Inequality (C.68) follows from the boundedness assumption in Assumption 5.2. Inequality (C.69) follows by noting that the gap between (ymax , zmax ) and (y, z) is because of instances unpredictably leaving the system before their maximum lifetime, and that there are at most T slots, at most Bd instances unpredictably leave the system in each slot (according to Assumption 5.2). Inequality (C.70) is obvious (note that the maximum is taken element-wise). By setting B = max {amax (Bd T + 1) ; bmax (Bd T + 1)}, we prove the result. We now proof Proposition 5.4. e (y, z) sums up un,t (yn ) and Proof. (Proposition 5.4) We note that D wnh,t (yn , yh , znh ) over t, n, h, as defined in (5.7). The numerator in the RHS of (5.12) can be expanded into a sum containing terms of either dun,t ((ymax + aiλ )nt ) dy or ∂wnh,t ((ymax + aiλ )nt , (ymax + aiλ )ht , (zmax + biλ )nht ) ∂Υ where Υ stands for either yn (t), yh (t), or znh (t), with either aiλn (t) or biλnh (t) as weights. Because Assumption 5.2 is satisfied, according to (C.67) in Lemma C.9, there exists a constant B ≥ 0 such that (ymax + aiλ )nt ≤ yn (t) + B (zmax + biλ )nht ≤ znh (t) + B C.5. Proof of Proposition 5.4 243 for all n, h, t. From Lemma C.6, we have dun,t dun,t (yn (t) + B) ≥ ((ymax + aiλ )nt ) dy dy and ∂wnh,t (yn (t) + B, yh (t) + B, znh (t) + B) ∂Υ ∂wnh,t ≥ ((ymax + aiλ )nt , (ymax + aiλ )ht , (zmax + biλ )nht ) ∂Υ Therefore, if φ≥ e ((y, z) + Be) · (aiλ , biλ ) ∇y,z D e (y, z) · (aiλ , biλ ) ∇y,z D (C.71) then (5.12) is always satisfied. Similar to the above, the numerator in the RHS of (C.71) can be expanded into a sum containing terms of either ∂wnh,t ∂Υ dun,t dy (yn (t) + B) and (yn (t) + B, yh (t) + B, znh (t) + B) with either aiλn (t) or biλnh (t) as weights. Again, the denominator in the RHS of (5.12) (or equivalently, (C.71)) can be expanded into a sum containing terms of either ∂wnh,t (yn (t), yh (t), znh (t)), ∂Υ or with either aiλn (t) or biλnh (t) as weights. For any given i, λ, the terms same weight aiλn (t), and ∂wnh,t (yn (t), yh (t), znh (t)) ∂Υ dun,t (y(t)) dy dun,t (yn (t) dy ∂wnh,t (yn (t) ∂Υ + B) and dun,t (yn (t)) dy have the + B, yh (t) + B, znh (t) + B) and have the same weight biλnh (t). According to Lemmas C.7 and C.8, for any δ > 0, there exist sufficiently large values of y and z, such that RHS of (C.71) ≤ 1 + δ Following a similar reasoning, we know that, for any δ > 0, there exist sufficiently large values of y and z, such that RHS of (5.13) ≤ Ω + δ C.5. Proof of Proposition 5.4 244 where we recall that Ω denotes the highest order of the polynomial cost functions, (ρ) (ρ ,ρ ,ρ3 ) 1 2 i.e., Ω , max{ρ; ρ1 + ρ2 + ρ3 }, subject to γn,t > 0 and κnh,t > 0. We assume sufficiently large y, z in the following, in which case we can set φ = 1 + δ and ψ = Ω + δ while satisfying (C.71) (thus (5.12)) and (5.13). We then note that from (5.16), (5.17), and the definition of Ω, we have ∗ e e ∗) D(φψx ) ≤ (φψ)Ω D(x e ∗) = ((1 + δ)(Ω + δ))Ω D(x e ∗) = ΩΩ + δ 0 D(x where δ 0 , δΩ + δ + δ 2 > 0 is an arbitrary constant (because δ is an arbitrary ∗ e constant). The first inequality is because of φ, ψ ≥ 1 and D(φψx ) is a polynomial of φψx∗ with maximum order of Ω, where we note that y and z are both linear in x. We then have ∗ e e D(φψx ) D(x) ≤ = ΩΩ + δ 0 ∗ ∗ e e D(x ) D(x ) (C.72) Until now, we have shown that (C.72) holds for sufficiently large y and z. According to Assumption 5.2, the number of instances that unpredictably leave the system in each slot is upper bounded by a constant Bd . It follows that y and z increase with M when M is larger than a certain threshold. Therefore, there exists a sufficiently large M , so that we have a sufficiently large y and z that satisfies (C.72). Hence, the competitive ratio upper bound can be expressed as Γ , max Γ(I(M )) ≤ ΩΩ + δ 0 I(M ) for sufficiently large M . (C.73) C.5. Proof of Proposition 5.4 245 According to the definition of the big-O notation, we can also write Γ = O(1) because Ω and δ 0 are both constants in M . (C.74) C.5. Proof of Proposition 5.4 246

- Similar pages
- [pdf]
- Presentation