In-Memory Analytics with Apache Arrow

Matthew Topol

出版时间

未知

ISBN

9781801071031

评分

★★★★★
书籍介绍
Apache Arrow is designed to accelerate analytics and allow the exchange of data across big data systems easily. In-Memory Analytics with Apache Arrow begins with a quick overview of the Apache Arrow format, before moving on to helping you to understand Arrow’s versatility and benefits as you walk through a variety of real-world use cases. You'll cover key tasks such as enhancing data science workflows with Arrow, using Arrow and Apache Parquet with Apache Spark and Jupyter for better performance and hassle-free data translation, as well as working with Perspective, an open source interactive graphical and tabular analysis tool for browsers. As you advance, you'll explore the different data interchange and storage formats and become well-versed with the relationships between Arrow, Parquet, Feather, Protobuf, Flatbuffers, JSON, and CSV. In addition to understanding the basic structure of the Arrow Flight and Flight SQL protocols, you'll learn about Dremio’s usage of Apache Arrow to enhance SQL analytics and discover how Arrow can be used in web-based browser apps. Finally, you'll get to grips with the upcoming features of Arrow to help you stay ahead of the curve. By the end of this book, you will have all the building blocks to create useful, efficient, and powerful analytical services and utilities with Apache Arrow.
用户评论
arrow本身就是一个可以跨语言通用的列存格式,本身的东西不多,主要是各种memory layout和一套自己的RPC框架(避免了序列化开销),这本书也都提供了比较完整的介绍,其实个人认为最有趣的topic还是他们总结的各种arrow的应用部分,比如query execute结果下载这类用arrow来传输性能可能有10倍以上的提升,另外flight + arrow的SQL 落地也可参考一下databend 计算引擎的实现https://github.com/datafuselabs/databend/tree/main/query, 也算是用rust做arrow最多的项目了
Arrow 的一本菜单。作者 Matt Topol 前段时间刚刚晋升 arrow PMC。书里面没有什么特别的 insight,但胜在所有组件讲的还算详细和图画的很好。文章介绍了 Arrow Columnar Format, IPC, compute, dataset, flight, flight sql 这些组件。然后略过了一些扩展项目(gandiva)。像 Acero, datafusion 这些项目其实还没咋介绍。本书还是比较易懂的,一下午应该能翻完,里面的图非常有参考价值,看完确实对 arrow 有些基础了解了
主要讲了各种类型的memory layout、arrow的应用场景和flight。讲arrow应用场景的那部分让我收获还挺大,大概知道了arrow适合干什么不适合干什么。flight这部分结合官方文档flight的python样例来看更清晰。总的来说还是这本书还是不错
下载
收藏