Thursday, July 28, 2011

How to monitor your EMR job

So, you just launch your EMR instances and executed your M/R job.  Now, how do you monitor your job?

One way is to create an ssh tunnel.

Open a terminal console and execute the following commands:

#set the key variable to the location of your key
key=~/amazon/emr.pem
#set the dns variable to the name of your EMR master public DNS name
dns=ec2-10-20-30-400.us-west-1.compute.amazonaws.com

# Start your tunnel and leave it running
ssh -L 9100:localhost:9100 -L 9101:localhost:9101 -i $key hadoop@$dns

# Launch a browser
# To view your JobTracker
http://localhost:9100/

# To view the your NameNode
http://localhost:9101/

Have fun!

No comments:

Post a Comment