AmoebaNet : an efficient and scalable SDN-enabled network service for extreme-scale distributed science = AmoebaNet : 대규모 분산 과학을 위한 효율적이고 확장 가능한 SDN 기반 네트워크 서비스

다국어 초록 (Multilingual Abstract)

The extreme-scale distributed science workflows play an essential function for scientific discoveries. Today’s large scientific experimental facilities are generating tremendous amount of data. In recent years, the significant growth of scientific data analysis has been observed across scientific centers. The scientific experimental facilities are producing unprecedented amount of data and scientific community encounters new challenges to data intensive computing. The performance of extreme-scale distributed science is highly depends on high-performance, adaptive, and robust network service infrastructures. To support data transfer for extreme-scale distributed science, there is the need of high performance, scalable, end-to-end, and programmable networks that enable scientific applications to use network efficiently.
The existing network paradigm that support extreme-scale distributed science workflows consists of three major components: terabit networks that provide high network bandwidths, Data Transfer Nodes (DTNs) and Science DMZ architecture that bypasses the performance hotspots in typical campus networks, and on-demand secure circuits/paths reservation systems, such as ESNet OSCARS and Internet2 AL2S, which provides automated, guaranteed bandwidth service in WAN. This network paradigm has proven to be very successful. However, to reach its full potentials, we claim that existing network paradigm for extreme-scale distributed science must address three major problems: last mile problem; scalability problem; and the agility, automation and programmability problem.
The recently emerged concept in network world is called Software-Defined Networking (SDN). This latest technology introduced the new methods of configuration and management of networking. In SDN, the underlying network devices are simply considered as packets forwarding elements and control logic of network is managed centrally by using a software program that dictates the entire network behavior. To address above mentioned problems, this thesis proposed a solution called AmoebaNet. AmoebaNet applies SDN technology to provide “QoS-guaranteed” network services in campus or local area networks. AmoebaNet complements existing network paradigm for extreme-scale distributed science: it allows application to program networks at run-time for optimum performance; and, in conjunction with WAN circuits/paths reservation system such as ESNet OSCARS and Internet2 AL2S; in result, it solves the problems of last mile, scalability, and the agility, automation and programmability. In this thesis, we also presented Congestion Aware Multipath Optimal Routing (CAMOR) solution which can be an additional service for AmoebaNet.

목차 (Table of Contents)

• Acknowledgement i
• Abstract iii
• Table of Contents viii
• List of Figures xi
• List of Tables xiii
• List of Algorithms xiv
• 1. Introduction 1
• 1.1 Overview 1
• 1.2 Motivation 2
• 1.3 Research problem 5
• 1.4 Contribution 9
• 1.5 Thesis organization 11
• 2. Background and Related Work 13
• 2.1 DTNs and science DMZ approach 13
• 2.2 Terabit networks 14
• 2.3 ESNet OSCARS and Internet2 AL2S 16
• 2.4 Software-Defined Networking (SDN) technology 17
• 2.4.1 SDN architecture 17
• 2.4.2 Open Network Operating System (ONOS) 20
• 2.5 Literature review 22
• 2.5.1 Software-Defined Networking (SDN) 22
• 2.5.2 End-to-End collaborative efforts 23
• 2.5.3 QoS-guaranteed end-to-end network path 24
• 3. Proposed Solution: AmoebaNet 26
• 3.1 An overview of AmoebaNet 26
• 3.2 Design goals 26
• 3.3 AmoebaNet Implementation and Architecture 27
• 3.3.1 Data plane architecture 28
• 3.3.2 Control plane architecture 29
• 3.3.2.1 AMQP based APIs 30
• 3.3.2.2 REST APIs 32
• 3.3.2.3 AmoebaNet manager 32
• 3.3.2.4 QoS-based routing & path computation 33
• 3.3.2.5 Flow manager 35
• 3.3.2.6 Resource monitoring 35
• 3.3.2.7 Topology manager 35
• 3.3.3 AmoebaNet primitives 36
• 3.4 BigData Express (BDE) 40
• 3.4.1 Introduction to BDE 40
• 3.4.2 BDE system design and architecture 42
• 3.5 Role of AmoebaNet in BDE 45
• 3.5.1 Provisioning QoS-guaranteed End-to-End network path 46
• 3.6 A new network paradigm for extreme-scale science 48
• 4. Evaluation and Results 50
• 4.1 End-to-End network path provisioning within a domain 50
• 4.2 QoS-guaranteed network slicing with a domain 53
• 4.3 Cross domain end-to-end network path provisioning 55
• 4.4 Cross-pacific QoS-guaranteed end-to-end network path 60
• 5. CAMOR: An Additional Solution for AmoebaNet 65
• 5.1 CAMOR: Congestion Aware Multipath Optimal Routing 65
• 5.2 CAMOR implementation 66
• 5.2.1 Optimal path selection service 68
• 5.2.2 Congestion service 69
• 5.2.3 Flow installation service 70
• 5.2.4 Topology service 70
• 5.3 Evaluation and results 70
• 5.3.1 Test cases 72
• 5.3.2 Results 73
• 6. Conclusion and Future Directions 77
• 6.1 Conclusion 77
• 6.2 Future directions 78
• References 79
• Appendix A: List of Publications 86