Hadoop Application Architectures - Mark Grover, Ted Malaska, Jonathan Seidman, Gwen Shapira

Hadoop Application Architectures

Mark Grover, Ted Malaska, Jonathan Seidman, Gwen Shapira

出版时间

2015-04-01

ISBN

9781491900086

评分

★★★★★
书籍介绍
With Early Release ebooks, you get books in their earliest form — the author's raw and unedited content as he or she writes — so you can take advantage of these technologies long before the official release of these titles. You'll also receive updates when significant changes are made, new chapters as they're written, and the final ebook bundle. Get expert guidance on architecting end-to-end data management solutions with Apache Hadoop. While many sources explain how to use various components in the Hadoop ecosystem, this practical book takes you through architectural considerations necessary to tie those components together into a complete tailored application, based on your particular use case. To reinforce those lessons, the book’s second section provides detailed examples of architecture used in some of the most commonly found Hadoop applications. Whether you’re designing and implementing a new Hadoop application, or planning to integrate Hadoop into your existing data infrastructure, Hadoop Application Architectures will skillfully guide you through the process. The Early Release edition begins with chapters that concentrate on design considerations for Data Modeling and Data Movement in Hadoop: Explore whether your application should store data on Hadoop Distributed File System (HDFS) or HBase Get best practices for designing an HDFS or HBase schema Learn how to design schemas for SQL-on-Hadoop (e.g. Hive, Impala, HCatalog) tables
精彩摘录
  • "如果关联的数据集恰好按照关联的键分桶,而且一个数据集中桶的数量是另一个的倍数,那么就足够单独关联相应的桶,而不需要关联整个数据集了。着显著降低了两个数据集执行 Reduce 端关联(Reduce-side join) 的时间复杂度。这是因为 Reduce 端的关联非常消耗资源。但是,如果关联的是两个桶数据集,而不是两个整数数据集,那么关联相应的桶即可。这样就可以减少关联消耗。当然,来自两个表的不同的桶可以并行关联。另外,分桶之后的数据量通常都比较小,一般能够放入内存。所以整个关联操作可以在 Map-Reduce 任务的 Map 阶段将小桶加载到内存中进行。这就是所谓的 Map 端关联(Map-"
用户评论
不错o
看的影印版,覆盖的比较全面,具体的技术还要自己更深入的看
2016年读过影印版,当时很好的Hadoop概览类图书,高屋建瓴
对比了常用的hadoop组件。描述了两个应用场景。内容比较新,但也缺乏很新的技术,例如Kudu+impala,kappa架构等。
大数据系统工程化,很赞
收藏