Hadoop in Practice

Summary

Hadoop in Practice collects 85 Hadoop examples and presents them in a problem/solution format. Each technique addresses a specific task you'll face, like querying big data using Pig or writing a log file loader. You'll explore each problem step by step, learning both how to build and deploy that specific solution along with the thinking that went into its design. As you work through the tasks, you'll find yourself growing more comfortable with Hadoop and at home in the world of big data. About the Technology

Hadoop is an open source MapReduce platform designed to query and analyze data distributed across large clusters. Especially effective for big data systems, Hadoop powers mission-critical software at Apple, eBay, LinkedIn, Yahoo, and Facebook. It offers developers handy ways to store, manage, and analyze data. About the Book

Hadoop in Practice collects 85 battle-tested examples and presents them in a problem/solution format. It balances conceptual foundations with practical recipes for key problem areas like data ingress and egress, serialization, and LZO compression. You'll explore each technique step by step, learning how to build a specific solution along with the thinking that went into it. As a bonus, the book's examples create a well-structured and understandable codebase you can tweak to meet your own needs.

This book assumes the reader knows the basics of Hadoop.

Purchase of the print book comes with an offer of a free PDF, ePub, and Kindle eBook from Manning. Also available is all code from the book. What's Inside

• Conceptual overview of Hadoop and MapReduce
• 85 practical, tested techniques
• Real problems, real solutions
• How to integrate MapReduce and R

PART 1 BACKGROUND AND FUNDAMENTALS

1. Hadoop in a heartbeat

PART 2 DATA LOGISTICS

2. Moving data in and out of Hadoop
3. Data serialization?working with text and beyond

PART 3 BIG DATA PATTERNS



4. Applying MapReduce patterns to big data
5. Streamlining HDFS for big data


6. Diagnosing and tuning performance problems

PART 4 DATA SCIENCE

7. Utilizing data structures and algorithms
8. Integrating R and Hadoop for statistics and more
9. Predictive analytics with Mahout

PART 5 TAMING THE ELEPHANT

10. Hacking with Hive
11. Programming pipelines with Pig


12. Crunch and other technologies
13. Testing and debugging

Hadoop is a framework organizing large clusters of computers around a problem. Because it's especially effective for big data, many well-known companies use Hadoop, including Facebook, Yahoo, Apple, eBay and LinkedIn. Hadoop in Practice collects nearly 100 Hadoop examples and presents them in a problem/solution format. Each technique explores a problem step-by-step, teaching both how to build that specific solution and the thinking that went into its design. As readers work through the tasks, they'll find themselves growing more comfortable with Hadoop and at home in the world of big data.

Alex Holmes is a Software Engineer with over a decade of experience developing large scale distributed Java systems. He currently is a technical lead at VeriSign, using Hadoop as a Big Data platform. Alex previously developed an Internet crawl, analysis, and search system using Hadoop and machine classification algorithms.