HDFS & Pig Notes

By | March 1st, 2016|Categories: Hadoop, Map Reduce|

Some pretty scrappy notes that got me through my time learning Pig & HDFS Setup the sshfs on local machine <> sshfs -p 22 hsohn@4.26.4.XX:/ebs/user/hsohn /Users/hsohn/Documents/remoteHome/dev/ -o auto_cache,reconnect,defer_permissions,negative_vncache,volname=dev Hadoop For dev access, do ssh bi@4.26.4.XX For prod access do ssh bi@4.26.4.XX Run a file in pig      pig -f query_20150917_test.pig Run locally      pig -x local script.pig Retrieving Results from HDFS      hdfs dfs -text [path / directory]*.gz Redirect [...]