13 posts
A detailed account of upgrading a large Telco Hadoop cluster from HDP 2.6.4 to 3.1.5, covering practice runs, planning strategies, and lessons from executing the upgrade during COVID remote work.
A practical troubleshooting guide for common Hive and Sqoop job failures on YARN, covering memory tuning, CPU allocation, and where to find the right logs.
Key design considerations for storing time series data, including access pattern analysis, windowed storage strategies, and the trade-offs between granularity and performance.
Lessons learned from migrating a large Hadoop cluster over a single weekend -- planning, data migration with distcp, code migration of 300+ Oozie jobs, and HBase migration.
A comparison of the new HDFS centralized cache management feature in Hadoop 2.3 with Spark RDDs, and why Spark still held the edge for in-memory processing.
Approaches for handling schema evolution in Hadoop using Avro and ORC file formats, including a practical workflow for managing schema changes with Hive.
How to use the ChainMapper class in Hadoop to call multiple mappers in sequence, with a working example and key points about configuration and type compatibility.
The small files problem in Hadoop and five approaches to solve it: HDFSConcat, IdentityMapper/Reducer, FileUtil.copyMerge, Hadoop File Crush, and Hive concatenate.
To do bench marks of Hadoop cluster is an ongoing process as we use it inside the organization. The main thing which we don't know when we buy new cluste...
At last its 1 May 2012 Cloudera has opened certifications through vue to worldwide people Details are as follows from Developer Exam Exam...
How to set up a Hadoop development environment in Eclipse with the WordCount MapReduce example.
Error java.io.IOException: Failed to set permissions of path: file:/tmp/hadoopjj/mapred/staging/jj1931875024/.staging to 0700 The details for my Hadoop we...
Quick guide to installing Hadoop on Windows with Cygwin, including Java configuration and verification steps.