E-MapReduce - JindoData Released to Support OSS-HDFS
Apr 15 2022
E-MapReduceContent
Target customers: all users. Features released: JindoData 4.0.0 is the first version released after the architecture of SmartData 3.8.0 is upgraded. JindoData connects to Alibaba Cloud Object Storage Service (OSS) and the Alibaba Cloud OSS-HDFS service, which is a cloud-native data lake storage service and is also called JindoFS. SmartData is a core self-developed component of E-MapReduce (EMR). You can use JindoSDK to access data in OSS-HDFS. OSS-HDFS is built based on unified metadata management capabilities and is fully compatible with HDFS APIs. The Portable Operating System Interface (POSIX) is supported in OSS-HDFS. This way, OSS-HDFS can be used to manage data in data lake-based computing scenarios in big data and AI fields. You can use OSS-HDFS without the need to modify configurations of the Hadoop and Spark applications. You can configure OSS-HDFS with ease to access and manage data in a similar way as in HDFS. In addition, you can take advantage of the characteristics of OSS such as unlimited storage space, elastic scalability, and high security, reliability, and availability. OSS-HDFS serves as the foundation of a cloud-native data lake. It can be used to analyze exabytes of data, manage hundreds of millions of objects, and achieve terabytes of throughput. OSS-HDFS provides flat and hierarchical namespace features to meet the requirements for big data storage. The hierarchical namespace feature allows you to manage objects in a hierarchical directory structure. In addition, unified metadata management allows automatic switchover between OSS and HDFS. Users of Hadoop can access their objects in OSS-HDFS without the need to copy or convert the format of the objects. This improves the performance of jobs and reduces maintenance costs.