Earlier this week shared some of the announcements Seagate has made, from SC14 in New Orleans. Well we’ve made another – this time addressing three key items that continue the company’s commitment to the open source community.
-
Seagate has finalized its contribution of an Apache Hadoop on Lustre Connector, which improves workflow efficiency by eliminating the need to copy data to the Hadoop Distributed File System (HDFS) prior to running Apache Hadoop jobs. It also provides an alternative to Hadoop’s reliance on the HDFS file system and enables Hadoop ecosystem tools such as Mahout, Hive and Pig to take advantage of the Lustre file system.
-
Seagate has released source code for a patch to Hadoop that allows Map and Reduce processes to share files and enables the use of “diskless” Hadoop compute clusters. This allows Hadoop to function with HPC architectures that use Lustre for storage. HPC customers in the Life Science and Energy fields are increasingly using Hadoop and Lustre together as part of their data analysis workflows.
The Hadoop on Lustre Connector helps HPC customers streamline their Hadoop workflows and accelerate time to results.
-
Further Seagate announced that it has agreed to transfer assets relating to Lustre.org to Open Scalable File Systems, Inc. (OpenSFS) and European Open Filesystem SCE (EOFS). OpenSFS and EOFS are trusted stewards of the Lustre distributed file software community and will jointly manage Lustre.org.
Seagate’s commitment to Lustre is at the highest ‘Promoter’ level and as an active board member as such and as one of the largest code contributors to the Lustre code tree we have deep involvement with OpenSFS and EOFS, on all levels.
For the next three days Seagate is demonstrating the Hadoop on Lustre Connector from booth 3239 at SC14. Those interested in a meeting should contact a member of the Seagate team or stop by the booth during the event. However, if that doesn’t work for you there always another way to contact Seagate.
