Data pipelines are the foundation for success in data analytics. Moving data from numerous diverse sources and transforming it to provide context is the difference between having data and actually gaining value from it. This pocket reference defines data pipelines and explains how they work in today's modern data stack.

You'll learn common considerations and key decision points when implementing pipelines, such as batch versus streaming data ingestion and build versus buy. This book addresses the most common decisions made by data professionals and discusses foundational concepts that apply to open source frameworks, commercial products, and homegrown solutions.

You'll learn:
• What a data pipeline is and how it works
• How data is moved and processed on modern data infrastructure, including cloud platforms
• Common tools and products used by data engineers to build pipelines
• How pipelines support analytics and reporting needs

• Considerations for pipeline maintenance, testing, and alerting

数据管线是数据分析成功的基础。从众多不同的来源移动数据并对其进行转换以提供背景，是拥有数据和真正从中获得价值之间的区别。这本袖珍参考书定义了数据管道，并解释了它们如何在当今的现代数据堆栈中工作。

你将学习实施管道时的常见考虑因素和关键决策点，如批处理与流式数据摄取以及构建与购买。本书讨论了数据专业人士最常见的决定，并讨论了适用于开源框架、商业产品和自制解决方案的基础概念。

你将会学到。

- 什么是数据管道，它是如何工作的

- 数据如何在现代数据基础设施（包括云平台）上移动和处理

- 数据工程师用来建立管道的常用工具和产品

- 管道如何支持分析和报告需求

- 管线维护、测试和警报的考虑因素