The sharing and transfer of huge amounts of data across borders and continents are key foundations for the advancement of global science. Such data is essential in advanced, leading-edge research that run extreme data-intensive calculations to solve today’s most complex research puzzles for tomorrow’s innovations. Applications for the data are wide-ranging including the Internet of Things (IoT), Artificial Intelligence (AI), advanced manufacturing, drug discovery, weather monitoring, big data analysis and many others.
Over the past decade, such transfers have progressed beyond research and into applications, and at increasingly larger data amounts as innovations and standards of the data resolution being captured continues to evolve. The network bandwidth to support the growing demand has gone from Mega to Giga, and to hundreds of Gbps in just a few years. Despite these advancements in high bandwidth capacity, scientists and researchers are still not experiencing the corresponding increase in the “speed” of data transfer. This is due to multiple factors, including server and storage hardware limitations, operating system tuning, and data transfer tools used, to name a few.
The Data Mover Challenge (DMC), which is organised by the National Supercomputing Centre (NSCC), Singapore brings together experts from industry and academia, to push the envelope in high speed, big data transfer capabilities, and across vast distances. Participants for DMC19 included international teams from USA, Japan, Taiwan, Indonesia, Thailand, Vietnam and South Korea. The Data Transfer Nodes (DTNs) for DMC19 that were set up globally in Singapore, Australia, USA, South Korea and Japan were connected by 100Gbps international research and education networks.
The next edition of the Data Mover Challenge, DMC20, has been launched. This year’s edition will challenge participants to send test data effectively around the globe!
Submissions for DMC20 are open from 01 June 2019 till 31 July 2019.
The lead of the winning team will have the opportunity to showcase the team’s solutions at the SupercomputingAsia conference in Singapore, which includes return coach airfare and free admission passes to the event that gathers the best of HPC in Asia!