You are here: Home / Events / SC15 PetaTransfer Experiment The Pacific Research Platform (PRP) Booth: SDSC #823

SC15 PetaTransfer Experiment The Pacific Research Platform (PRP) Booth: SDSC #823

by John Graham published Nov 08, 2015 02:52 PM, last modified Feb 07, 2017 06:35 PM
SDSC / Calit2 PetaTransfer Experiment The Pacific Research Platform (PRP) Booth: SDSC #823
When Nov 16, 2015 03:00 PM to
Nov 20, 2015 03:00 PM
Where SC15 Austin Texas
Contact Name
Contact Phone 858-692-3396
Add event to calendar vCal
iCal

You can’t rsync a petabyte, but data centers are dealing with the need to move and replicate petabyte-scale file systems on a regular basis. New supercomputers, large disk-based parallel file systems, lack of  sustainable archives, and maintenance requirements all drive the need to take a massive non-uniform distribution of file and folders and put them somewhere else.

The PetaTransfer Experiment will demonstrate an efficient parallel recursive algorithm for file system replication from the San Diego Supercomputer Center (SDSC) to our booth at SC15 in Austin over a 100Gbps link provided via the NSF-funded Pacific Research Platform (PRP).

The connection via the PRP makes the SC15 show floor another campus on the PRP, and the SDSC booth represents a Science DMZ on the SC15 campus. The storage at SDSC is a petabyte-sized randomized clone of a real Lustre file system, with all the small files and deep directory trees that come with it, which will be pushed to flash storage arrays using ZFS in the booth.

Thank you to Arista Networks for providing 100Gigabit Ethernet networking and SanDisk for providing the storage equipment. And thanks to Calit2's Qualcomm Institute for providing the servers and system integration. More information at: http://ucsdnews.ucsd.edu/pressrelease/nsf_gives_green_light_to_pacific_research_platform.

More information about this event…