Difference between revisions of "Hadoop notes"
From Simson Garfinkel
Jump to navigationJump to search
(Created page with "Run a command with Apache Yarn: export DSJAR=/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell-2.8.3-amzn-0.jar yarn jar $DSJAR -jar $DSJAR org.apache.ha...") |
m |
||
| (4 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
Run a command with Apache Yarn: | Run a command with Apache Yarn as the **yarn** user: | ||
export DSJAR=/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell-2.8.3-amzn-0.jar | export DSJAR=/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell-2.8.3-amzn-0.jar | ||
yarn jar $DSJAR -jar $DSJAR org.apache.hadoop.yarn.applications.distributedshell.Client -master_memory 1000 -container_memory 1000 -shell_command 'hostname' | yarn jar $DSJAR -jar $DSJAR org.apache.hadoop.yarn.applications.distributedshell.Client -master_memory 1000 -container_memory 1000 -shell_command 'hostname' | ||
yarn jar $DSJAR -jar $DSJAR org.apache.hadoop.yarn.applications.distributedshell.Client -num_containers 32 -master_memory 1000 -container_memory 1000 -shell_command 'hostname' | |||
Output: | Output: | ||
.... | .... | ||
18/05/03 21:35:32 INFO distributedshell.Client: Got application report from ASM for, appId=7, clientToAMToken=null, appDiagnostics=, appMasterHost= | 18/05/03 21:35:32 INFO distributedshell.Client: Got application report from ASM for, appId=7, clientToAMToken=null, appDiagnostics=, appMasterHost=host/127.0.0.1, appQueue=default, appMasterRpcPort=-1, appStartTime=1525383325669, yarnAppState=FINISHED, distributedFinalState=SUCCEEDED, appTrackingUrl=http://host:20888/proxy/application_1525380541355_0007/, appUser=hadoop | ||
.... | .... | ||
See the apps that ran with: | |||
yarn application -list -appStates ALL | |||
Latest revision as of 10:45, 2 August 2018
Run a command with Apache Yarn as the **yarn** user:
export DSJAR=/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell-2.8.3-amzn-0.jar yarn jar $DSJAR -jar $DSJAR org.apache.hadoop.yarn.applications.distributedshell.Client -master_memory 1000 -container_memory 1000 -shell_command 'hostname' yarn jar $DSJAR -jar $DSJAR org.apache.hadoop.yarn.applications.distributedshell.Client -num_containers 32 -master_memory 1000 -container_memory 1000 -shell_command 'hostname'
Output:
.... 18/05/03 21:35:32 INFO distributedshell.Client: Got application report from ASM for, appId=7, clientToAMToken=null, appDiagnostics=, appMasterHost=host/127.0.0.1, appQueue=default, appMasterRpcPort=-1, appStartTime=1525383325669, yarnAppState=FINISHED, distributedFinalState=SUCCEEDED, appTrackingUrl=http://host:20888/proxy/application_1525380541355_0007/, appUser=hadoop ....
See the apps that ran with:
yarn application -list -appStates ALL
view the logs with:
yarn logs -applicationId application_1525380541355_0007
