Hadoop notes
From Simson Garfinkel
Jump to navigationJump to search
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.
Run a command with Apache Yarn as the **yarn** user:
export DSJAR=/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell-2.8.3-amzn-0.jar yarn jar $DSJAR -jar $DSJAR org.apache.hadoop.yarn.applications.distributedshell.Client -master_memory 1000 -container_memory 1000 -shell_command 'hostname' yarn jar $DSJAR -jar $DSJAR org.apache.hadoop.yarn.applications.distributedshell.Client -num_containers 32 -master_memory 1000 -container_memory 1000 -shell_command 'hostname'
Output:
.... 18/05/03 21:35:32 INFO distributedshell.Client: Got application report from ASM for, appId=7, clientToAMToken=null, appDiagnostics=, appMasterHost=host/127.0.0.1, appQueue=default, appMasterRpcPort=-1, appStartTime=1525383325669, yarnAppState=FINISHED, distributedFinalState=SUCCEEDED, appTrackingUrl=http://host:20888/proxy/application_1525380541355_0007/, appUser=hadoop ....
See the apps that ran with:
yarn application -list -appStates ALL
view the logs with:
yarn logs -applicationId application_1525380541355_0007
