2017-03-10 12:58:10,580 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-03-10 12:58:11,075 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2017-03-10 12:58:11,087 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-03-10 12:58:11,561 WARN org.apache.hadoop.mapreduce.JobResourceUploader: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2017-03-10 12:58:11,594 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1
2017-03-10 12:58:11,707 INFO org.apache.hadoop.mapreduce.JobSubmitter: number of splits:1
2017-03-10 12:58:11,996 INFO org.apache.hadoop.mapreduce.JobSubmitter: Submitting tokens for job: job_local1458741767_0001
2017-03-10 12:58:12,393 INFO org.apache.hadoop.mapreduce.Job: The url to track the job: http://localhost:8080/
2017-03-10 12:58:12,394 INFO org.apache.hadoop.mapreduce.Job: Running job: job_local1458741767_0001
2017-03-10 12:58:12,399 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter set in config null
2017-03-10 12:58:12,419 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 12:58:12,422 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2017-03-10 12:58:12,540 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for map tasks
2017-03-10 12:58:12,543 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local1458741767_0001_m_000000_0
2017-03-10 12:58:12,607 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 12:58:12,632 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 12:58:12,640 INFO org.apache.hadoop.mapred.MapTask: Processing split: file:/home/cloudera/workspace/WordCount/input/pg100.txt:0+5589886
2017-03-10 12:58:12,811 INFO org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-03-10 12:58:12,823 INFO org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-03-10 12:58:12,824 INFO org.apache.hadoop.mapred.MapTask: soft limit at 83886080
2017-03-10 12:58:12,824 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-03-10 12:58:12,824 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600
2017-03-10 12:58:12,832 INFO org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-03-10 12:58:13,397 INFO org.apache.hadoop.mapreduce.Job: Job job_local1458741767_0001 running in uber mode : false
2017-03-10 12:58:13,399 INFO org.apache.hadoop.mapreduce.Job:  map 0% reduce 0%
2017-03-10 12:58:14,983 INFO org.apache.hadoop.mapred.LocalJobRunner: 
2017-03-10 12:58:14,985 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output
2017-03-10 12:58:14,985 INFO org.apache.hadoop.mapred.MapTask: Spilling map output
2017-03-10 12:58:14,985 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 8925696; bufvoid = 104857600
2017-03-10 12:58:14,985 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396(104857584); kvend = 22120424(88481696); length = 4093973/6553600
2017-03-10 12:58:16,014 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2017-03-10 12:58:16,798 INFO org.apache.hadoop.mapred.MapTask: Finished spill 0
2017-03-10 12:58:16,800 INFO org.apache.hadoop.mapred.Task: Task:attempt_local1458741767_0001_m_000000_0 is done. And is in the process of committing
2017-03-10 12:58:16,806 INFO org.apache.hadoop.mapred.LocalJobRunner: map
2017-03-10 12:58:16,806 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local1458741767_0001_m_000000_0' done.
2017-03-10 12:58:16,806 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local1458741767_0001_m_000000_0
2017-03-10 12:58:16,806 INFO org.apache.hadoop.mapred.LocalJobRunner: map task executor complete.
2017-03-10 12:58:16,814 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for reduce tasks
2017-03-10 12:58:16,814 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local1458741767_0001_r_000000_0
2017-03-10 12:58:16,819 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 12:58:16,819 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 12:58:16,823 INFO org.apache.hadoop.mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7f792cec
2017-03-10 12:58:16,833 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: MergerManager: memoryLimit=679778688, maxSingleShuffleLimit=169944672, mergeThreshold=448653952, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2017-03-10 12:58:16,839 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_local1458741767_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2017-03-10 12:58:16,862 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.deflate]
2017-03-10 12:58:16,862 INFO org.apache.hadoop.mapreduce.task.reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1458741767_0001_m_000000_0 decomp: 491464 len: 167092 to MEMORY
2017-03-10 12:58:16,869 INFO org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 491464 bytes from map-output for attempt_local1458741767_0001_m_000000_0
2017-03-10 12:58:16,869 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 491464, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->491464
2017-03-10 12:58:16,870 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: EventFetcher is interrupted.. Returning
2017-03-10 12:58:16,871 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 12:58:16,871 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2017-03-10 12:58:16,876 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 12:58:16,876 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 491459 bytes
2017-03-10 12:58:17,106 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merged 1 segments, 491464 bytes to disk to satisfy reduce memory limit
2017-03-10 12:58:17,107 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 1 files, 167100 bytes from disk
2017-03-10 12:58:17,107 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce
2017-03-10 12:58:17,107 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 12:58:17,108 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 491459 bytes
2017-03-10 12:58:17,108 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 12:58:17,113 INFO org.apache.hadoop.conf.Configuration.deprecation: mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords
2017-03-10 12:58:17,416 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 0%
2017-03-10 12:58:17,746 INFO org.apache.hadoop.mapred.Task: Task:attempt_local1458741767_0001_r_000000_0 is done. And is in the process of committing
2017-03-10 12:58:17,747 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 12:58:17,751 INFO org.apache.hadoop.mapred.Task: Task attempt_local1458741767_0001_r_000000_0 is allowed to commit now
2017-03-10 12:58:17,752 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Saved output of task 'attempt_local1458741767_0001_r_000000_0' to file:/home/cloudera/workspace/WordCount/output/_temporary/0/task_local1458741767_0001_r_000000
2017-03-10 12:58:17,756 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce > reduce
2017-03-10 12:58:17,756 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local1458741767_0001_r_000000_0' done.
2017-03-10 12:58:17,757 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local1458741767_0001_r_000000_0
2017-03-10 12:58:17,758 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce task executor complete.
2017-03-10 12:58:18,417 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 100%
2017-03-10 12:58:18,418 INFO org.apache.hadoop.mapreduce.Job: Job job_local1458741767_0001 completed successfully
2017-03-10 12:58:18,427 INFO org.apache.hadoop.mapreduce.Job: Counters: 33
	File System Counters
		FILE: Number of bytes read=11514350
		FILE: Number of bytes written=1395729
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
	Map-Reduce Framework
		Map input records=124787
		Map output records=1023494
		Map output bytes=8925696
		Map output materialized bytes=167092
		Input split bytes=120
		Combine input records=1023494
		Combine output records=34513
		Reduce input groups=34513
		Reduce shuffle bytes=167092
		Reduce input records=34513
		Reduce output records=34513
		Spilled Records=69026
		Shuffled Maps =1
		Failed Shuffles=0
		Merged Map outputs=1
		GC time elapsed (ms)=109
		CPU time spent (ms)=0
		Physical memory (bytes) snapshot=0
		Virtual memory (bytes) snapshot=0
		Total committed heap usage (bytes)=331227136
	Shuffle Errors
		BAD_ID=0
		CONNECTION=0
		IO_ERROR=0
		WRONG_LENGTH=0
		WRONG_MAP=0
		WRONG_REDUCE=0
	File Input Format Counters 
		Bytes Read=5589886
	File Output Format Counters 
		Bytes Written=363879
2017-03-10 14:05:48,287 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-03-10 14:05:48,833 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2017-03-10 14:05:48,841 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-03-10 14:05:49,279 WARN org.apache.hadoop.mapreduce.JobResourceUploader: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2017-03-10 14:05:49,288 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1
2017-03-10 14:05:49,331 INFO org.apache.hadoop.mapreduce.JobSubmitter: number of splits:1
2017-03-10 14:05:49,610 INFO org.apache.hadoop.mapreduce.JobSubmitter: Submitting tokens for job: job_local1590990832_0001
2017-03-10 14:05:50,040 INFO org.apache.hadoop.mapreduce.Job: The url to track the job: http://localhost:8080/
2017-03-10 14:05:50,042 INFO org.apache.hadoop.mapreduce.Job: Running job: job_local1590990832_0001
2017-03-10 14:05:50,046 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter set in config null
2017-03-10 14:05:50,059 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:05:50,067 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2017-03-10 14:05:50,190 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for map tasks
2017-03-10 14:05:50,192 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local1590990832_0001_m_000000_0
2017-03-10 14:05:50,229 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:05:50,244 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:05:50,247 INFO org.apache.hadoop.mapred.MapTask: Processing split: file:/home/cloudera/workspace/WordCount/input/pg100.txt:0+5589886
2017-03-10 14:05:50,339 INFO org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-03-10 14:05:50,339 INFO org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-03-10 14:05:50,339 INFO org.apache.hadoop.mapred.MapTask: soft limit at 83886080
2017-03-10 14:05:50,339 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-03-10 14:05:50,339 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600
2017-03-10 14:05:50,344 INFO org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-03-10 14:05:51,047 INFO org.apache.hadoop.mapreduce.Job: Job job_local1590990832_0001 running in uber mode : false
2017-03-10 14:05:51,050 INFO org.apache.hadoop.mapreduce.Job:  map 0% reduce 0%
2017-03-10 14:05:52,504 INFO org.apache.hadoop.mapred.LocalJobRunner: 
2017-03-10 14:05:52,504 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output
2017-03-10 14:05:52,504 INFO org.apache.hadoop.mapred.MapTask: Spilling map output
2017-03-10 14:05:52,504 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 8925696; bufvoid = 104857600
2017-03-10 14:05:52,504 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396(104857584); kvend = 22120424(88481696); length = 4093973/6553600
2017-03-10 14:05:53,471 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2017-03-10 14:05:54,161 INFO org.apache.hadoop.mapred.MapTask: Finished spill 0
2017-03-10 14:05:54,164 INFO org.apache.hadoop.mapred.Task: Task:attempt_local1590990832_0001_m_000000_0 is done. And is in the process of committing
2017-03-10 14:05:54,170 INFO org.apache.hadoop.mapred.LocalJobRunner: map
2017-03-10 14:05:54,170 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local1590990832_0001_m_000000_0' done.
2017-03-10 14:05:54,170 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local1590990832_0001_m_000000_0
2017-03-10 14:05:54,170 INFO org.apache.hadoop.mapred.LocalJobRunner: map task executor complete.
2017-03-10 14:05:54,178 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for reduce tasks
2017-03-10 14:05:54,178 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local1590990832_0001_r_000000_0
2017-03-10 14:05:54,183 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:05:54,183 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:05:54,185 INFO org.apache.hadoop.mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7f792cec
2017-03-10 14:05:54,198 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: MergerManager: memoryLimit=679778688, maxSingleShuffleLimit=169944672, mergeThreshold=448653952, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2017-03-10 14:05:54,206 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_local1590990832_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2017-03-10 14:05:54,277 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.deflate]
2017-03-10 14:05:54,277 INFO org.apache.hadoop.mapreduce.task.reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1590990832_0001_m_000000_0 decomp: 491464 len: 167092 to MEMORY
2017-03-10 14:05:54,297 INFO org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 491464 bytes from map-output for attempt_local1590990832_0001_m_000000_0
2017-03-10 14:05:54,298 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 491464, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->491464
2017-03-10 14:05:54,302 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: EventFetcher is interrupted.. Returning
2017-03-10 14:05:54,305 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:05:54,305 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2017-03-10 14:05:54,315 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 14:05:54,319 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 491459 bytes
2017-03-10 14:05:54,702 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merged 1 segments, 491464 bytes to disk to satisfy reduce memory limit
2017-03-10 14:05:54,702 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 1 files, 167100 bytes from disk
2017-03-10 14:05:54,702 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce
2017-03-10 14:05:54,703 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 14:05:54,704 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 491459 bytes
2017-03-10 14:05:54,704 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:05:54,714 INFO org.apache.hadoop.conf.Configuration.deprecation: mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords
2017-03-10 14:05:55,077 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 0%
2017-03-10 14:05:55,853 INFO org.apache.hadoop.mapred.Task: Task:attempt_local1590990832_0001_r_000000_0 is done. And is in the process of committing
2017-03-10 14:05:55,858 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:05:55,859 INFO org.apache.hadoop.mapred.Task: Task attempt_local1590990832_0001_r_000000_0 is allowed to commit now
2017-03-10 14:05:55,859 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Saved output of task 'attempt_local1590990832_0001_r_000000_0' to file:/home/cloudera/workspace/WordCount/output/_temporary/0/task_local1590990832_0001_r_000000
2017-03-10 14:05:55,861 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce > reduce
2017-03-10 14:05:55,861 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local1590990832_0001_r_000000_0' done.
2017-03-10 14:05:55,861 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local1590990832_0001_r_000000_0
2017-03-10 14:05:55,861 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce task executor complete.
2017-03-10 14:05:56,079 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 100%
2017-03-10 14:05:56,080 INFO org.apache.hadoop.mapreduce.Job: Job job_local1590990832_0001 completed successfully
2017-03-10 14:05:56,090 INFO org.apache.hadoop.mapreduce.Job: Counters: 33
	File System Counters
		FILE: Number of bytes read=11514350
		FILE: Number of bytes written=1395729
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
	Map-Reduce Framework
		Map input records=124787
		Map output records=1023494
		Map output bytes=8925696
		Map output materialized bytes=167092
		Input split bytes=120
		Combine input records=1023494
		Combine output records=34513
		Reduce input groups=34513
		Reduce shuffle bytes=167092
		Reduce input records=34513
		Reduce output records=34513
		Spilled Records=69026
		Shuffled Maps =1
		Failed Shuffles=0
		Merged Map outputs=1
		GC time elapsed (ms)=80
		CPU time spent (ms)=0
		Physical memory (bytes) snapshot=0
		Virtual memory (bytes) snapshot=0
		Total committed heap usage (bytes)=331227136
	Shuffle Errors
		BAD_ID=0
		CONNECTION=0
		IO_ERROR=0
		WRONG_LENGTH=0
		WRONG_MAP=0
		WRONG_REDUCE=0
	File Input Format Counters 
		Bytes Read=5589886
	File Output Format Counters 
		Bytes Written=363879
2017-03-10 14:07:44,622 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-03-10 14:07:45,122 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2017-03-10 14:07:45,129 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-03-10 14:07:45,628 WARN org.apache.hadoop.mapreduce.JobResourceUploader: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2017-03-10 14:07:45,645 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1
2017-03-10 14:07:45,678 INFO org.apache.hadoop.mapreduce.JobSubmitter: number of splits:1
2017-03-10 14:07:45,909 INFO org.apache.hadoop.mapreduce.JobSubmitter: Submitting tokens for job: job_local1520504035_0001
2017-03-10 14:07:46,336 INFO org.apache.hadoop.mapreduce.Job: The url to track the job: http://localhost:8080/
2017-03-10 14:07:46,338 INFO org.apache.hadoop.mapreduce.Job: Running job: job_local1520504035_0001
2017-03-10 14:07:46,337 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter set in config null
2017-03-10 14:07:46,344 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:07:46,346 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2017-03-10 14:07:46,457 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for map tasks
2017-03-10 14:07:46,460 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local1520504035_0001_m_000000_0
2017-03-10 14:07:46,523 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:07:46,542 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:07:46,545 INFO org.apache.hadoop.mapred.MapTask: Processing split: file:/home/cloudera/workspace/WordCount/input/pg100.txt:0+5589886
2017-03-10 14:07:46,634 INFO org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-03-10 14:07:46,634 INFO org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-03-10 14:07:46,634 INFO org.apache.hadoop.mapred.MapTask: soft limit at 83886080
2017-03-10 14:07:46,634 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-03-10 14:07:46,635 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600
2017-03-10 14:07:46,639 INFO org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-03-10 14:07:47,343 INFO org.apache.hadoop.mapreduce.Job: Job job_local1520504035_0001 running in uber mode : false
2017-03-10 14:07:47,344 INFO org.apache.hadoop.mapreduce.Job:  map 0% reduce 0%
2017-03-10 14:07:48,802 INFO org.apache.hadoop.mapred.LocalJobRunner: 
2017-03-10 14:07:48,802 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output
2017-03-10 14:07:48,802 INFO org.apache.hadoop.mapred.MapTask: Spilling map output
2017-03-10 14:07:48,802 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 8925696; bufvoid = 104857600
2017-03-10 14:07:48,802 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396(104857584); kvend = 22120424(88481696); length = 4093973/6553600
2017-03-10 14:07:49,778 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2017-03-10 14:07:50,507 INFO org.apache.hadoop.mapred.MapTask: Finished spill 0
2017-03-10 14:07:50,510 INFO org.apache.hadoop.mapred.Task: Task:attempt_local1520504035_0001_m_000000_0 is done. And is in the process of committing
2017-03-10 14:07:50,516 INFO org.apache.hadoop.mapred.LocalJobRunner: map
2017-03-10 14:07:50,516 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local1520504035_0001_m_000000_0' done.
2017-03-10 14:07:50,516 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local1520504035_0001_m_000000_0
2017-03-10 14:07:50,516 INFO org.apache.hadoop.mapred.LocalJobRunner: map task executor complete.
2017-03-10 14:07:50,523 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for reduce tasks
2017-03-10 14:07:50,523 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local1520504035_0001_r_000000_0
2017-03-10 14:07:50,529 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:07:50,529 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:07:50,531 INFO org.apache.hadoop.mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7f792cec
2017-03-10 14:07:50,541 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: MergerManager: memoryLimit=679778688, maxSingleShuffleLimit=169944672, mergeThreshold=448653952, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2017-03-10 14:07:50,548 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_local1520504035_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2017-03-10 14:07:50,573 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.deflate]
2017-03-10 14:07:50,574 INFO org.apache.hadoop.mapreduce.task.reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1520504035_0001_m_000000_0 decomp: 491464 len: 167092 to MEMORY
2017-03-10 14:07:50,580 INFO org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 491464 bytes from map-output for attempt_local1520504035_0001_m_000000_0
2017-03-10 14:07:50,581 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 491464, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->491464
2017-03-10 14:07:50,582 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: EventFetcher is interrupted.. Returning
2017-03-10 14:07:50,582 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:07:50,582 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2017-03-10 14:07:50,587 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 14:07:50,588 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 491459 bytes
2017-03-10 14:07:50,811 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merged 1 segments, 491464 bytes to disk to satisfy reduce memory limit
2017-03-10 14:07:50,811 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 1 files, 167100 bytes from disk
2017-03-10 14:07:50,812 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce
2017-03-10 14:07:50,812 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 14:07:50,813 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 491459 bytes
2017-03-10 14:07:50,813 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:07:50,818 INFO org.apache.hadoop.conf.Configuration.deprecation: mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords
2017-03-10 14:07:51,355 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 0%
2017-03-10 14:07:51,511 INFO org.apache.hadoop.mapred.Task: Task:attempt_local1520504035_0001_r_000000_0 is done. And is in the process of committing
2017-03-10 14:07:51,512 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:07:51,515 INFO org.apache.hadoop.mapred.Task: Task attempt_local1520504035_0001_r_000000_0 is allowed to commit now
2017-03-10 14:07:51,516 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Saved output of task 'attempt_local1520504035_0001_r_000000_0' to file:/home/cloudera/workspace/WordCount/output/_temporary/0/task_local1520504035_0001_r_000000
2017-03-10 14:07:51,516 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce > reduce
2017-03-10 14:07:51,518 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local1520504035_0001_r_000000_0' done.
2017-03-10 14:07:51,518 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local1520504035_0001_r_000000_0
2017-03-10 14:07:51,518 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce task executor complete.
2017-03-10 14:07:52,356 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 100%
2017-03-10 14:07:52,356 INFO org.apache.hadoop.mapreduce.Job: Job job_local1520504035_0001 completed successfully
2017-03-10 14:07:52,367 INFO org.apache.hadoop.mapreduce.Job: Counters: 33
	File System Counters
		FILE: Number of bytes read=11514350
		FILE: Number of bytes written=1397073
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
	Map-Reduce Framework
		Map input records=124787
		Map output records=1023494
		Map output bytes=8925696
		Map output materialized bytes=167092
		Input split bytes=120
		Combine input records=1023494
		Combine output records=34513
		Reduce input groups=34513
		Reduce shuffle bytes=167092
		Reduce input records=34513
		Reduce output records=34513
		Spilled Records=69026
		Shuffled Maps =1
		Failed Shuffles=0
		Merged Map outputs=1
		GC time elapsed (ms)=86
		CPU time spent (ms)=0
		Physical memory (bytes) snapshot=0
		Virtual memory (bytes) snapshot=0
		Total committed heap usage (bytes)=331227136
	Shuffle Errors
		BAD_ID=0
		CONNECTION=0
		IO_ERROR=0
		WRONG_LENGTH=0
		WRONG_MAP=0
		WRONG_REDUCE=0
	File Input Format Counters 
		Bytes Read=5589886
	File Output Format Counters 
		Bytes Written=363879
2017-03-10 14:08:46,208 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-03-10 14:08:46,725 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2017-03-10 14:08:46,726 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-03-10 14:08:46,791 WARN org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:cloudera (auth:SIMPLE) cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory file:/home/cloudera/workspace/WordCount/output already exists
2017-03-10 14:09:00,496 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-03-10 14:09:00,991 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2017-03-10 14:09:00,992 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-03-10 14:09:01,486 WARN org.apache.hadoop.mapreduce.JobResourceUploader: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2017-03-10 14:09:01,504 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1
2017-03-10 14:09:01,622 INFO org.apache.hadoop.mapreduce.JobSubmitter: number of splits:1
2017-03-10 14:09:01,930 INFO org.apache.hadoop.mapreduce.JobSubmitter: Submitting tokens for job: job_local1768387477_0001
2017-03-10 14:09:02,340 INFO org.apache.hadoop.mapreduce.Job: The url to track the job: http://localhost:8080/
2017-03-10 14:09:02,341 INFO org.apache.hadoop.mapreduce.Job: Running job: job_local1768387477_0001
2017-03-10 14:09:02,345 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter set in config null
2017-03-10 14:09:02,348 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:09:02,366 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2017-03-10 14:09:02,467 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for map tasks
2017-03-10 14:09:02,468 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local1768387477_0001_m_000000_0
2017-03-10 14:09:02,532 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:09:02,561 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:09:02,564 INFO org.apache.hadoop.mapred.MapTask: Processing split: file:/home/cloudera/workspace/WordCount/input/pg100.txt:0+5589886
2017-03-10 14:09:02,644 INFO org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-03-10 14:09:02,645 INFO org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-03-10 14:09:02,645 INFO org.apache.hadoop.mapred.MapTask: soft limit at 83886080
2017-03-10 14:09:02,645 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-03-10 14:09:02,645 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600
2017-03-10 14:09:02,652 INFO org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-03-10 14:09:03,343 INFO org.apache.hadoop.mapreduce.Job: Job job_local1768387477_0001 running in uber mode : false
2017-03-10 14:09:03,344 INFO org.apache.hadoop.mapreduce.Job:  map 0% reduce 0%
2017-03-10 14:09:04,790 INFO org.apache.hadoop.mapred.LocalJobRunner: 
2017-03-10 14:09:04,792 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output
2017-03-10 14:09:04,792 INFO org.apache.hadoop.mapred.MapTask: Spilling map output
2017-03-10 14:09:04,792 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 8925696; bufvoid = 104857600
2017-03-10 14:09:04,792 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396(104857584); kvend = 22120424(88481696); length = 4093973/6553600
2017-03-10 14:09:05,819 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2017-03-10 14:09:06,544 INFO org.apache.hadoop.mapred.MapTask: Finished spill 0
2017-03-10 14:09:06,546 INFO org.apache.hadoop.mapred.Task: Task:attempt_local1768387477_0001_m_000000_0 is done. And is in the process of committing
2017-03-10 14:09:06,552 INFO org.apache.hadoop.mapred.LocalJobRunner: map
2017-03-10 14:09:06,552 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local1768387477_0001_m_000000_0' done.
2017-03-10 14:09:06,552 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local1768387477_0001_m_000000_0
2017-03-10 14:09:06,552 INFO org.apache.hadoop.mapred.LocalJobRunner: map task executor complete.
2017-03-10 14:09:06,560 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for reduce tasks
2017-03-10 14:09:06,561 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local1768387477_0001_r_000000_0
2017-03-10 14:09:06,569 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:09:06,570 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:09:06,572 INFO org.apache.hadoop.mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7f792cec
2017-03-10 14:09:06,582 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: MergerManager: memoryLimit=679778688, maxSingleShuffleLimit=169944672, mergeThreshold=448653952, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2017-03-10 14:09:06,588 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_local1768387477_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2017-03-10 14:09:06,616 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.deflate]
2017-03-10 14:09:06,617 INFO org.apache.hadoop.mapreduce.task.reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1768387477_0001_m_000000_0 decomp: 491464 len: 167092 to MEMORY
2017-03-10 14:09:06,626 INFO org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 491464 bytes from map-output for attempt_local1768387477_0001_m_000000_0
2017-03-10 14:09:06,627 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 491464, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->491464
2017-03-10 14:09:06,628 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: EventFetcher is interrupted.. Returning
2017-03-10 14:09:06,629 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:09:06,629 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2017-03-10 14:09:06,636 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 14:09:06,636 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 491459 bytes
2017-03-10 14:09:06,896 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merged 1 segments, 491464 bytes to disk to satisfy reduce memory limit
2017-03-10 14:09:06,896 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 1 files, 167100 bytes from disk
2017-03-10 14:09:06,897 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce
2017-03-10 14:09:06,897 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 14:09:06,898 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 491459 bytes
2017-03-10 14:09:06,898 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:09:06,903 INFO org.apache.hadoop.conf.Configuration.deprecation: mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords
2017-03-10 14:09:07,352 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 0%
2017-03-10 14:09:07,776 INFO org.apache.hadoop.mapred.Task: Task:attempt_local1768387477_0001_r_000000_0 is done. And is in the process of committing
2017-03-10 14:09:07,792 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:09:07,792 INFO org.apache.hadoop.mapred.Task: Task attempt_local1768387477_0001_r_000000_0 is allowed to commit now
2017-03-10 14:09:07,792 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Saved output of task 'attempt_local1768387477_0001_r_000000_0' to file:/home/cloudera/workspace/WordCount/output/_temporary/0/task_local1768387477_0001_r_000000
2017-03-10 14:09:07,793 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce > reduce
2017-03-10 14:09:07,793 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local1768387477_0001_r_000000_0' done.
2017-03-10 14:09:07,793 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local1768387477_0001_r_000000_0
2017-03-10 14:09:07,793 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce task executor complete.
2017-03-10 14:09:08,353 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 100%
2017-03-10 14:09:08,354 INFO org.apache.hadoop.mapreduce.Job: Job job_local1768387477_0001 completed successfully
2017-03-10 14:09:08,363 INFO org.apache.hadoop.mapreduce.Job: Counters: 33
	File System Counters
		FILE: Number of bytes read=11514350
		FILE: Number of bytes written=1397073
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
	Map-Reduce Framework
		Map input records=124787
		Map output records=1023494
		Map output bytes=8925696
		Map output materialized bytes=167092
		Input split bytes=120
		Combine input records=1023494
		Combine output records=34513
		Reduce input groups=34513
		Reduce shuffle bytes=167092
		Reduce input records=34513
		Reduce output records=34513
		Spilled Records=69026
		Shuffled Maps =1
		Failed Shuffles=0
		Merged Map outputs=1
		GC time elapsed (ms)=76
		CPU time spent (ms)=0
		Physical memory (bytes) snapshot=0
		Virtual memory (bytes) snapshot=0
		Total committed heap usage (bytes)=331227136
	Shuffle Errors
		BAD_ID=0
		CONNECTION=0
		IO_ERROR=0
		WRONG_LENGTH=0
		WRONG_MAP=0
		WRONG_REDUCE=0
	File Input Format Counters 
		Bytes Read=5589886
	File Output Format Counters 
		Bytes Written=363879
2017-03-10 14:10:49,958 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-03-10 14:10:50,420 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2017-03-10 14:10:50,423 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-03-10 14:10:50,893 WARN org.apache.hadoop.mapreduce.JobResourceUploader: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2017-03-10 14:10:50,902 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1
2017-03-10 14:10:50,935 INFO org.apache.hadoop.mapreduce.JobSubmitter: number of splits:1
2017-03-10 14:10:51,165 INFO org.apache.hadoop.mapreduce.JobSubmitter: Submitting tokens for job: job_local321623198_0001
2017-03-10 14:10:51,576 INFO org.apache.hadoop.mapreduce.Job: The url to track the job: http://localhost:8080/
2017-03-10 14:10:51,577 INFO org.apache.hadoop.mapreduce.Job: Running job: job_local321623198_0001
2017-03-10 14:10:51,586 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter set in config null
2017-03-10 14:10:51,596 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:10:51,607 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2017-03-10 14:10:51,726 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for map tasks
2017-03-10 14:10:51,728 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local321623198_0001_m_000000_0
2017-03-10 14:10:51,786 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:10:51,801 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:10:51,805 INFO org.apache.hadoop.mapred.MapTask: Processing split: file:/home/cloudera/workspace/WordCount/input/pg100.txt:0+5589886
2017-03-10 14:10:51,887 INFO org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-03-10 14:10:51,887 INFO org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-03-10 14:10:51,887 INFO org.apache.hadoop.mapred.MapTask: soft limit at 83886080
2017-03-10 14:10:51,888 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-03-10 14:10:51,888 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600
2017-03-10 14:10:51,891 INFO org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-03-10 14:10:52,584 INFO org.apache.hadoop.mapreduce.Job: Job job_local321623198_0001 running in uber mode : false
2017-03-10 14:10:52,587 INFO org.apache.hadoop.mapreduce.Job:  map 0% reduce 0%
2017-03-10 14:10:54,371 INFO org.apache.hadoop.mapred.LocalJobRunner: 
2017-03-10 14:10:54,373 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output
2017-03-10 14:10:54,373 INFO org.apache.hadoop.mapred.MapTask: Spilling map output
2017-03-10 14:10:54,374 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 8925696; bufvoid = 104857600
2017-03-10 14:10:54,374 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396(104857584); kvend = 22120424(88481696); length = 4093973/6553600
2017-03-10 14:10:55,678 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2017-03-10 14:10:56,656 INFO org.apache.hadoop.mapred.MapTask: Finished spill 0
2017-03-10 14:10:56,659 INFO org.apache.hadoop.mapred.Task: Task:attempt_local321623198_0001_m_000000_0 is done. And is in the process of committing
2017-03-10 14:10:56,666 INFO org.apache.hadoop.mapred.LocalJobRunner: map
2017-03-10 14:10:56,666 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local321623198_0001_m_000000_0' done.
2017-03-10 14:10:56,666 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local321623198_0001_m_000000_0
2017-03-10 14:10:56,667 INFO org.apache.hadoop.mapred.LocalJobRunner: map task executor complete.
2017-03-10 14:10:56,676 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for reduce tasks
2017-03-10 14:10:56,676 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local321623198_0001_r_000000_0
2017-03-10 14:10:56,684 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:10:56,685 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:10:56,689 INFO org.apache.hadoop.mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@405d65c3
2017-03-10 14:10:56,704 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: MergerManager: memoryLimit=679778688, maxSingleShuffleLimit=169944672, mergeThreshold=448653952, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2017-03-10 14:10:56,710 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_local321623198_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2017-03-10 14:10:56,742 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.deflate]
2017-03-10 14:10:56,742 INFO org.apache.hadoop.mapreduce.task.reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local321623198_0001_m_000000_0 decomp: 491464 len: 167092 to MEMORY
2017-03-10 14:10:56,749 INFO org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 491464 bytes from map-output for attempt_local321623198_0001_m_000000_0
2017-03-10 14:10:56,749 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 491464, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->491464
2017-03-10 14:10:56,750 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: EventFetcher is interrupted.. Returning
2017-03-10 14:10:56,752 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:10:56,752 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2017-03-10 14:10:56,756 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 14:10:56,757 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 491459 bytes
2017-03-10 14:10:57,102 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merged 1 segments, 491464 bytes to disk to satisfy reduce memory limit
2017-03-10 14:10:57,102 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 1 files, 167100 bytes from disk
2017-03-10 14:10:57,103 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce
2017-03-10 14:10:57,103 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 14:10:57,104 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 491459 bytes
2017-03-10 14:10:57,105 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:10:57,109 INFO org.apache.hadoop.conf.Configuration.deprecation: mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords
2017-03-10 14:10:57,598 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 0%
2017-03-10 14:10:57,797 INFO org.apache.hadoop.mapred.Task: Task:attempt_local321623198_0001_r_000000_0 is done. And is in the process of committing
2017-03-10 14:10:57,801 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:10:57,802 INFO org.apache.hadoop.mapred.Task: Task attempt_local321623198_0001_r_000000_0 is allowed to commit now
2017-03-10 14:10:57,802 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Saved output of task 'attempt_local321623198_0001_r_000000_0' to file:/home/cloudera/workspace/WordCount/output/_temporary/0/task_local321623198_0001_r_000000
2017-03-10 14:10:57,803 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce > reduce
2017-03-10 14:10:57,803 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local321623198_0001_r_000000_0' done.
2017-03-10 14:10:57,803 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local321623198_0001_r_000000_0
2017-03-10 14:10:57,803 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce task executor complete.
2017-03-10 14:10:58,598 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 100%
2017-03-10 14:10:58,599 INFO org.apache.hadoop.mapreduce.Job: Job job_local321623198_0001 completed successfully
2017-03-10 14:10:58,612 INFO org.apache.hadoop.mapreduce.Job: Counters: 33
	File System Counters
		FILE: Number of bytes read=11514350
		FILE: Number of bytes written=1394261
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
	Map-Reduce Framework
		Map input records=124787
		Map output records=1023494
		Map output bytes=8925696
		Map output materialized bytes=167092
		Input split bytes=120
		Combine input records=1023494
		Combine output records=34513
		Reduce input groups=34513
		Reduce shuffle bytes=167092
		Reduce input records=34513
		Reduce output records=34513
		Spilled Records=69026
		Shuffled Maps =1
		Failed Shuffles=0
		Merged Map outputs=1
		GC time elapsed (ms)=82
		CPU time spent (ms)=0
		Physical memory (bytes) snapshot=0
		Virtual memory (bytes) snapshot=0
		Total committed heap usage (bytes)=331227136
	Shuffle Errors
		BAD_ID=0
		CONNECTION=0
		IO_ERROR=0
		WRONG_LENGTH=0
		WRONG_MAP=0
		WRONG_REDUCE=0
	File Input Format Counters 
		Bytes Read=5589886
	File Output Format Counters 
		Bytes Written=363879
2017-03-10 14:11:49,324 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-03-10 14:11:49,809 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2017-03-10 14:11:49,819 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-03-10 14:11:50,294 WARN org.apache.hadoop.mapreduce.JobResourceUploader: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2017-03-10 14:11:50,309 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1
2017-03-10 14:11:50,418 INFO org.apache.hadoop.mapreduce.JobSubmitter: number of splits:1
2017-03-10 14:11:50,734 INFO org.apache.hadoop.mapreduce.JobSubmitter: Submitting tokens for job: job_local1130190814_0001
2017-03-10 14:11:51,124 INFO org.apache.hadoop.mapreduce.Job: The url to track the job: http://localhost:8080/
2017-03-10 14:11:51,125 INFO org.apache.hadoop.mapreduce.Job: Running job: job_local1130190814_0001
2017-03-10 14:11:51,125 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter set in config null
2017-03-10 14:11:51,137 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:11:51,145 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2017-03-10 14:11:51,262 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for map tasks
2017-03-10 14:11:51,262 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local1130190814_0001_m_000000_0
2017-03-10 14:11:51,319 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:11:51,345 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:11:51,348 INFO org.apache.hadoop.mapred.MapTask: Processing split: file:/home/cloudera/workspace/WordCount/input/pg100.txt:0+5589886
2017-03-10 14:11:51,448 INFO org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-03-10 14:11:51,448 INFO org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-03-10 14:11:51,448 INFO org.apache.hadoop.mapred.MapTask: soft limit at 83886080
2017-03-10 14:11:51,448 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-03-10 14:11:51,448 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600
2017-03-10 14:11:51,451 INFO org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-03-10 14:11:52,138 INFO org.apache.hadoop.mapreduce.Job: Job job_local1130190814_0001 running in uber mode : false
2017-03-10 14:11:52,139 INFO org.apache.hadoop.mapreduce.Job:  map 0% reduce 0%
2017-03-10 14:11:53,548 INFO org.apache.hadoop.mapred.LocalJobRunner: 
2017-03-10 14:11:53,549 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output
2017-03-10 14:11:53,549 INFO org.apache.hadoop.mapred.MapTask: Spilling map output
2017-03-10 14:11:53,549 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 8925696; bufvoid = 104857600
2017-03-10 14:11:53,549 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396(104857584); kvend = 22120424(88481696); length = 4093973/6553600
2017-03-10 14:11:54,505 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2017-03-10 14:11:55,315 INFO org.apache.hadoop.mapred.MapTask: Finished spill 0
2017-03-10 14:11:55,318 INFO org.apache.hadoop.mapred.Task: Task:attempt_local1130190814_0001_m_000000_0 is done. And is in the process of committing
2017-03-10 14:11:55,323 INFO org.apache.hadoop.mapred.LocalJobRunner: map
2017-03-10 14:11:55,324 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local1130190814_0001_m_000000_0' done.
2017-03-10 14:11:55,324 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local1130190814_0001_m_000000_0
2017-03-10 14:11:55,324 INFO org.apache.hadoop.mapred.LocalJobRunner: map task executor complete.
2017-03-10 14:11:55,331 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for reduce tasks
2017-03-10 14:11:55,332 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local1130190814_0001_r_000000_0
2017-03-10 14:11:55,337 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:11:55,337 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:11:55,339 INFO org.apache.hadoop.mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@33c40638
2017-03-10 14:11:55,352 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: MergerManager: memoryLimit=679778688, maxSingleShuffleLimit=169944672, mergeThreshold=448653952, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2017-03-10 14:11:55,358 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_local1130190814_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2017-03-10 14:11:55,386 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.deflate]
2017-03-10 14:11:55,387 INFO org.apache.hadoop.mapreduce.task.reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1130190814_0001_m_000000_0 decomp: 491464 len: 167092 to MEMORY
2017-03-10 14:11:55,397 INFO org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 491464 bytes from map-output for attempt_local1130190814_0001_m_000000_0
2017-03-10 14:11:55,397 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 491464, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->491464
2017-03-10 14:11:55,398 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: EventFetcher is interrupted.. Returning
2017-03-10 14:11:55,399 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:11:55,399 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2017-03-10 14:11:55,403 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 14:11:55,404 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 491459 bytes
2017-03-10 14:11:55,647 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merged 1 segments, 491464 bytes to disk to satisfy reduce memory limit
2017-03-10 14:11:55,647 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 1 files, 167100 bytes from disk
2017-03-10 14:11:55,648 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce
2017-03-10 14:11:55,648 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 14:11:55,649 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 491459 bytes
2017-03-10 14:11:55,649 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:11:55,654 INFO org.apache.hadoop.conf.Configuration.deprecation: mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords
2017-03-10 14:11:56,162 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 0%
2017-03-10 14:11:56,278 INFO org.apache.hadoop.mapred.Task: Task:attempt_local1130190814_0001_r_000000_0 is done. And is in the process of committing
2017-03-10 14:11:56,289 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:11:56,292 INFO org.apache.hadoop.mapred.Task: Task attempt_local1130190814_0001_r_000000_0 is allowed to commit now
2017-03-10 14:11:56,294 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Saved output of task 'attempt_local1130190814_0001_r_000000_0' to file:/home/cloudera/workspace/WordCount/output/_temporary/0/task_local1130190814_0001_r_000000
2017-03-10 14:11:56,294 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce > reduce
2017-03-10 14:11:56,295 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local1130190814_0001_r_000000_0' done.
2017-03-10 14:11:56,295 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local1130190814_0001_r_000000_0
2017-03-10 14:11:56,295 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce task executor complete.
2017-03-10 14:11:57,163 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 100%
2017-03-10 14:11:57,163 INFO org.apache.hadoop.mapreduce.Job: Job job_local1130190814_0001 completed successfully
2017-03-10 14:11:57,176 INFO org.apache.hadoop.mapreduce.Job: Counters: 33
	File System Counters
		FILE: Number of bytes read=11514350
		FILE: Number of bytes written=1397073
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
	Map-Reduce Framework
		Map input records=124787
		Map output records=1023494
		Map output bytes=8925696
		Map output materialized bytes=167092
		Input split bytes=120
		Combine input records=1023494
		Combine output records=34513
		Reduce input groups=34513
		Reduce shuffle bytes=167092
		Reduce input records=34513
		Reduce output records=34513
		Spilled Records=69026
		Shuffled Maps =1
		Failed Shuffles=0
		Merged Map outputs=1
		GC time elapsed (ms)=84
		CPU time spent (ms)=0
		Physical memory (bytes) snapshot=0
		Virtual memory (bytes) snapshot=0
		Total committed heap usage (bytes)=331227136
	Shuffle Errors
		BAD_ID=0
		CONNECTION=0
		IO_ERROR=0
		WRONG_LENGTH=0
		WRONG_MAP=0
		WRONG_REDUCE=0
	File Input Format Counters 
		Bytes Read=5589886
	File Output Format Counters 
		Bytes Written=363879
2017-03-10 14:12:54,192 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-03-10 14:12:54,666 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2017-03-10 14:12:54,677 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-03-10 14:12:54,726 WARN org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:cloudera (auth:SIMPLE) cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory file:/home/cloudera/workspace/WordCount/output already exists
2017-03-10 14:13:16,264 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-03-10 14:13:16,742 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2017-03-10 14:13:16,746 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-03-10 14:13:17,210 WARN org.apache.hadoop.mapreduce.JobResourceUploader: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2017-03-10 14:13:17,218 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1
2017-03-10 14:13:17,249 INFO org.apache.hadoop.mapreduce.JobSubmitter: number of splits:1
2017-03-10 14:13:17,469 INFO org.apache.hadoop.mapreduce.JobSubmitter: Submitting tokens for job: job_local1259538348_0001
2017-03-10 14:13:17,909 INFO org.apache.hadoop.mapreduce.Job: The url to track the job: http://localhost:8080/
2017-03-10 14:13:17,911 INFO org.apache.hadoop.mapreduce.Job: Running job: job_local1259538348_0001
2017-03-10 14:13:17,915 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter set in config null
2017-03-10 14:13:17,923 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:13:17,937 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2017-03-10 14:13:18,056 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for map tasks
2017-03-10 14:13:18,059 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local1259538348_0001_m_000000_0
2017-03-10 14:13:18,117 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:13:18,128 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:13:18,132 INFO org.apache.hadoop.mapred.MapTask: Processing split: file:/home/cloudera/workspace/WordCount/input/pg100.txt:0+5589886
2017-03-10 14:13:18,215 INFO org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-03-10 14:13:18,216 INFO org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-03-10 14:13:18,216 INFO org.apache.hadoop.mapred.MapTask: soft limit at 83886080
2017-03-10 14:13:18,216 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-03-10 14:13:18,216 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600
2017-03-10 14:13:18,226 INFO org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-03-10 14:13:18,912 INFO org.apache.hadoop.mapreduce.Job: Job job_local1259538348_0001 running in uber mode : false
2017-03-10 14:13:18,913 INFO org.apache.hadoop.mapreduce.Job:  map 0% reduce 0%
2017-03-10 14:13:20,442 INFO org.apache.hadoop.mapred.LocalJobRunner: 
2017-03-10 14:13:20,443 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output
2017-03-10 14:13:20,443 INFO org.apache.hadoop.mapred.MapTask: Spilling map output
2017-03-10 14:13:20,443 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 8925696; bufvoid = 104857600
2017-03-10 14:13:20,443 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396(104857584); kvend = 22120424(88481696); length = 4093973/6553600
2017-03-10 14:13:21,376 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2017-03-10 14:13:22,115 INFO org.apache.hadoop.mapred.MapTask: Finished spill 0
2017-03-10 14:13:22,117 INFO org.apache.hadoop.mapred.Task: Task:attempt_local1259538348_0001_m_000000_0 is done. And is in the process of committing
2017-03-10 14:13:22,124 INFO org.apache.hadoop.mapred.LocalJobRunner: map
2017-03-10 14:13:22,124 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local1259538348_0001_m_000000_0' done.
2017-03-10 14:13:22,124 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local1259538348_0001_m_000000_0
2017-03-10 14:13:22,124 INFO org.apache.hadoop.mapred.LocalJobRunner: map task executor complete.
2017-03-10 14:13:22,132 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for reduce tasks
2017-03-10 14:13:22,132 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local1259538348_0001_r_000000_0
2017-03-10 14:13:22,137 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:13:22,137 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:13:22,140 INFO org.apache.hadoop.mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7f792cec
2017-03-10 14:13:22,151 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: MergerManager: memoryLimit=679778688, maxSingleShuffleLimit=169944672, mergeThreshold=448653952, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2017-03-10 14:13:22,156 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_local1259538348_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2017-03-10 14:13:22,209 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.deflate]
2017-03-10 14:13:22,209 INFO org.apache.hadoop.mapreduce.task.reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1259538348_0001_m_000000_0 decomp: 491464 len: 167092 to MEMORY
2017-03-10 14:13:22,215 INFO org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 491464 bytes from map-output for attempt_local1259538348_0001_m_000000_0
2017-03-10 14:13:22,217 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 491464, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->491464
2017-03-10 14:13:22,219 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: EventFetcher is interrupted.. Returning
2017-03-10 14:13:22,220 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:13:22,224 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2017-03-10 14:13:22,229 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 14:13:22,230 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 491459 bytes
2017-03-10 14:13:22,458 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merged 1 segments, 491464 bytes to disk to satisfy reduce memory limit
2017-03-10 14:13:22,459 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 1 files, 167100 bytes from disk
2017-03-10 14:13:22,459 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce
2017-03-10 14:13:22,459 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 14:13:22,460 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 491459 bytes
2017-03-10 14:13:22,460 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:13:22,465 INFO org.apache.hadoop.conf.Configuration.deprecation: mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords
2017-03-10 14:13:22,936 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 0%
2017-03-10 14:13:23,199 INFO org.apache.hadoop.mapred.Task: Task:attempt_local1259538348_0001_r_000000_0 is done. And is in the process of committing
2017-03-10 14:13:23,200 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:13:23,201 INFO org.apache.hadoop.mapred.Task: Task attempt_local1259538348_0001_r_000000_0 is allowed to commit now
2017-03-10 14:13:23,201 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Saved output of task 'attempt_local1259538348_0001_r_000000_0' to file:/home/cloudera/workspace/WordCount/output/_temporary/0/task_local1259538348_0001_r_000000
2017-03-10 14:13:23,202 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce > reduce
2017-03-10 14:13:23,202 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local1259538348_0001_r_000000_0' done.
2017-03-10 14:13:23,202 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local1259538348_0001_r_000000_0
2017-03-10 14:13:23,202 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce task executor complete.
2017-03-10 14:13:23,936 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 100%
2017-03-10 14:13:23,937 INFO org.apache.hadoop.mapreduce.Job: Job job_local1259538348_0001 completed successfully
2017-03-10 14:13:23,948 INFO org.apache.hadoop.mapreduce.Job: Counters: 33
	File System Counters
		FILE: Number of bytes read=11514350
		FILE: Number of bytes written=1397073
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
	Map-Reduce Framework
		Map input records=124787
		Map output records=1023494
		Map output bytes=8925696
		Map output materialized bytes=167092
		Input split bytes=120
		Combine input records=1023494
		Combine output records=34513
		Reduce input groups=34513
		Reduce shuffle bytes=167092
		Reduce input records=34513
		Reduce output records=34513
		Spilled Records=69026
		Shuffled Maps =1
		Failed Shuffles=0
		Merged Map outputs=1
		GC time elapsed (ms)=79
		CPU time spent (ms)=0
		Physical memory (bytes) snapshot=0
		Virtual memory (bytes) snapshot=0
		Total committed heap usage (bytes)=331227136
	Shuffle Errors
		BAD_ID=0
		CONNECTION=0
		IO_ERROR=0
		WRONG_LENGTH=0
		WRONG_MAP=0
		WRONG_REDUCE=0
	File Input Format Counters 
		Bytes Read=5589886
	File Output Format Counters 
		Bytes Written=363879
2017-03-10 14:15:07,671 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-03-10 14:15:08,143 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2017-03-10 14:15:08,146 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-03-10 14:15:08,597 WARN org.apache.hadoop.mapreduce.JobResourceUploader: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2017-03-10 14:15:08,610 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1
2017-03-10 14:15:08,649 INFO org.apache.hadoop.mapreduce.JobSubmitter: number of splits:1
2017-03-10 14:15:08,885 INFO org.apache.hadoop.mapreduce.JobSubmitter: Submitting tokens for job: job_local744282859_0001
2017-03-10 14:15:09,357 INFO org.apache.hadoop.mapreduce.Job: The url to track the job: http://localhost:8080/
2017-03-10 14:15:09,358 INFO org.apache.hadoop.mapreduce.Job: Running job: job_local744282859_0001
2017-03-10 14:15:09,361 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter set in config null
2017-03-10 14:15:09,374 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:15:09,376 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2017-03-10 14:15:09,515 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for map tasks
2017-03-10 14:15:09,516 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local744282859_0001_m_000000_0
2017-03-10 14:15:09,574 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:15:09,584 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:15:09,587 INFO org.apache.hadoop.mapred.MapTask: Processing split: file:/home/cloudera/workspace/WordCount/input/pg100.txt:0+5589886
2017-03-10 14:15:09,674 INFO org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-03-10 14:15:09,674 INFO org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-03-10 14:15:09,674 INFO org.apache.hadoop.mapred.MapTask: soft limit at 83886080
2017-03-10 14:15:09,674 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-03-10 14:15:09,674 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600
2017-03-10 14:15:09,679 INFO org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-03-10 14:15:10,364 INFO org.apache.hadoop.mapreduce.Job: Job job_local744282859_0001 running in uber mode : false
2017-03-10 14:15:10,366 INFO org.apache.hadoop.mapreduce.Job:  map 0% reduce 0%
2017-03-10 14:15:11,981 INFO org.apache.hadoop.mapred.LocalJobRunner: 
2017-03-10 14:15:11,982 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output
2017-03-10 14:15:11,982 INFO org.apache.hadoop.mapred.MapTask: Spilling map output
2017-03-10 14:15:11,982 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 8925696; bufvoid = 104857600
2017-03-10 14:15:11,982 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396(104857584); kvend = 22120424(88481696); length = 4093973/6553600
2017-03-10 14:15:13,023 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2017-03-10 14:15:13,726 INFO org.apache.hadoop.mapred.MapTask: Finished spill 0
2017-03-10 14:15:13,728 INFO org.apache.hadoop.mapred.Task: Task:attempt_local744282859_0001_m_000000_0 is done. And is in the process of committing
2017-03-10 14:15:13,739 INFO org.apache.hadoop.mapred.LocalJobRunner: map
2017-03-10 14:15:13,739 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local744282859_0001_m_000000_0' done.
2017-03-10 14:15:13,739 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local744282859_0001_m_000000_0
2017-03-10 14:15:13,739 INFO org.apache.hadoop.mapred.LocalJobRunner: map task executor complete.
2017-03-10 14:15:13,748 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for reduce tasks
2017-03-10 14:15:13,749 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local744282859_0001_r_000000_0
2017-03-10 14:15:13,753 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:15:13,754 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:15:13,756 INFO org.apache.hadoop.mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@534d4113
2017-03-10 14:15:13,766 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: MergerManager: memoryLimit=679778688, maxSingleShuffleLimit=169944672, mergeThreshold=448653952, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2017-03-10 14:15:13,772 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_local744282859_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2017-03-10 14:15:13,800 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.deflate]
2017-03-10 14:15:13,800 INFO org.apache.hadoop.mapreduce.task.reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local744282859_0001_m_000000_0 decomp: 491464 len: 167092 to MEMORY
2017-03-10 14:15:13,807 INFO org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 491464 bytes from map-output for attempt_local744282859_0001_m_000000_0
2017-03-10 14:15:13,807 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 491464, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->491464
2017-03-10 14:15:13,809 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: EventFetcher is interrupted.. Returning
2017-03-10 14:15:13,813 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:15:13,814 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2017-03-10 14:15:13,819 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 14:15:13,819 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 491459 bytes
2017-03-10 14:15:14,078 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merged 1 segments, 491464 bytes to disk to satisfy reduce memory limit
2017-03-10 14:15:14,079 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 1 files, 167100 bytes from disk
2017-03-10 14:15:14,079 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce
2017-03-10 14:15:14,079 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 14:15:14,080 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 491459 bytes
2017-03-10 14:15:14,080 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:15:14,085 INFO org.apache.hadoop.conf.Configuration.deprecation: mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords
2017-03-10 14:15:14,378 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 0%
2017-03-10 14:15:14,760 INFO org.apache.hadoop.mapred.Task: Task:attempt_local744282859_0001_r_000000_0 is done. And is in the process of committing
2017-03-10 14:15:14,761 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:15:14,761 INFO org.apache.hadoop.mapred.Task: Task attempt_local744282859_0001_r_000000_0 is allowed to commit now
2017-03-10 14:15:14,762 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Saved output of task 'attempt_local744282859_0001_r_000000_0' to file:/home/cloudera/workspace/WordCount/output/_temporary/0/task_local744282859_0001_r_000000
2017-03-10 14:15:14,762 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce > reduce
2017-03-10 14:15:14,762 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local744282859_0001_r_000000_0' done.
2017-03-10 14:15:14,762 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local744282859_0001_r_000000_0
2017-03-10 14:15:14,762 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce task executor complete.
2017-03-10 14:15:15,379 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 100%
2017-03-10 14:15:15,380 INFO org.apache.hadoop.mapreduce.Job: Job job_local744282859_0001 completed successfully
2017-03-10 14:15:15,390 INFO org.apache.hadoop.mapreduce.Job: Counters: 33
	File System Counters
		FILE: Number of bytes read=11514350
		FILE: Number of bytes written=1394261
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
	Map-Reduce Framework
		Map input records=124787
		Map output records=1023494
		Map output bytes=8925696
		Map output materialized bytes=167092
		Input split bytes=120
		Combine input records=1023494
		Combine output records=34513
		Reduce input groups=34513
		Reduce shuffle bytes=167092
		Reduce input records=34513
		Reduce output records=34513
		Spilled Records=69026
		Shuffled Maps =1
		Failed Shuffles=0
		Merged Map outputs=1
		GC time elapsed (ms)=90
		CPU time spent (ms)=0
		Physical memory (bytes) snapshot=0
		Virtual memory (bytes) snapshot=0
		Total committed heap usage (bytes)=331227136
	Shuffle Errors
		BAD_ID=0
		CONNECTION=0
		IO_ERROR=0
		WRONG_LENGTH=0
		WRONG_MAP=0
		WRONG_REDUCE=0
	File Input Format Counters 
		Bytes Read=5589886
	File Output Format Counters 
		Bytes Written=363879
2017-03-10 14:16:55,128 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-03-10 14:16:55,605 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2017-03-10 14:16:55,614 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-03-10 14:16:55,649 WARN org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:cloudera (auth:SIMPLE) cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory file:/home/cloudera/workspace/WordCount/output already exists
2017-03-10 14:17:14,700 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-03-10 14:17:15,157 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2017-03-10 14:17:15,169 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-03-10 14:17:15,624 WARN org.apache.hadoop.mapreduce.JobResourceUploader: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2017-03-10 14:17:15,635 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1
2017-03-10 14:17:15,668 INFO org.apache.hadoop.mapreduce.JobSubmitter: number of splits:1
2017-03-10 14:17:15,901 INFO org.apache.hadoop.mapreduce.JobSubmitter: Submitting tokens for job: job_local1420822781_0001
2017-03-10 14:17:16,319 INFO org.apache.hadoop.mapreduce.Job: The url to track the job: http://localhost:8080/
2017-03-10 14:17:16,321 INFO org.apache.hadoop.mapreduce.Job: Running job: job_local1420822781_0001
2017-03-10 14:17:16,323 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter set in config null
2017-03-10 14:17:16,335 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:17:16,340 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2017-03-10 14:17:16,441 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for map tasks
2017-03-10 14:17:16,442 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local1420822781_0001_m_000000_0
2017-03-10 14:17:16,518 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:17:16,531 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:17:16,534 INFO org.apache.hadoop.mapred.MapTask: Processing split: file:/home/cloudera/workspace/WordCount/input/pg100.txt:0+5589886
2017-03-10 14:17:16,616 INFO org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-03-10 14:17:16,617 INFO org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-03-10 14:17:16,617 INFO org.apache.hadoop.mapred.MapTask: soft limit at 83886080
2017-03-10 14:17:16,617 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-03-10 14:17:16,617 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600
2017-03-10 14:17:16,621 INFO org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-03-10 14:17:16,682 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output
2017-03-10 14:17:16,697 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2017-03-10 14:17:16,704 INFO org.apache.hadoop.mapred.LocalJobRunner: map task executor complete.
2017-03-10 14:17:16,705 WARN org.apache.hadoop.mapred.LocalJobRunner: job_local1420822781_0001
java.lang.Exception: java.lang.ArrayIndexOutOfBoundsException: 1
	at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 1
	at similarity.WordSort$Map.loadWordFreq(WordSort.java:87)
	at similarity.WordSort$Map.setup(WordSort.java:118)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
	at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2017-03-10 14:17:17,328 INFO org.apache.hadoop.mapreduce.Job: Job job_local1420822781_0001 running in uber mode : false
2017-03-10 14:17:17,329 INFO org.apache.hadoop.mapreduce.Job:  map 0% reduce 0%
2017-03-10 14:17:17,331 INFO org.apache.hadoop.mapreduce.Job: Job job_local1420822781_0001 failed with state FAILED due to: NA
2017-03-10 14:17:17,336 INFO org.apache.hadoop.mapreduce.Job: Counters: 0
2017-03-10 14:26:12,465 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-03-10 14:26:12,973 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2017-03-10 14:26:12,986 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-03-10 14:26:13,468 WARN org.apache.hadoop.mapreduce.JobResourceUploader: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2017-03-10 14:26:13,490 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1
2017-03-10 14:26:13,599 INFO org.apache.hadoop.mapreduce.JobSubmitter: number of splits:1
2017-03-10 14:26:13,979 INFO org.apache.hadoop.mapreduce.JobSubmitter: Submitting tokens for job: job_local1562971559_0001
2017-03-10 14:26:14,383 INFO org.apache.hadoop.mapreduce.Job: The url to track the job: http://localhost:8080/
2017-03-10 14:26:14,385 INFO org.apache.hadoop.mapreduce.Job: Running job: job_local1562971559_0001
2017-03-10 14:26:14,384 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter set in config null
2017-03-10 14:26:14,391 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:26:14,399 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2017-03-10 14:26:14,512 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for map tasks
2017-03-10 14:26:14,513 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local1562971559_0001_m_000000_0
2017-03-10 14:26:14,576 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:26:14,613 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:26:14,617 INFO org.apache.hadoop.mapred.MapTask: Processing split: file:/home/cloudera/workspace/WordCount/input/pg100.txt:0+5589886
2017-03-10 14:26:14,762 INFO org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-03-10 14:26:14,762 INFO org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-03-10 14:26:14,763 INFO org.apache.hadoop.mapred.MapTask: soft limit at 83886080
2017-03-10 14:26:14,763 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-03-10 14:26:14,763 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600
2017-03-10 14:26:14,766 INFO org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-03-10 14:26:15,398 INFO org.apache.hadoop.mapreduce.Job: Job job_local1562971559_0001 running in uber mode : false
2017-03-10 14:26:15,398 INFO org.apache.hadoop.mapreduce.Job:  map 0% reduce 0%
2017-03-10 14:26:15,473 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output
2017-03-10 14:26:15,481 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2017-03-10 14:26:15,488 INFO org.apache.hadoop.mapred.LocalJobRunner: map task executor complete.
2017-03-10 14:26:15,490 WARN org.apache.hadoop.mapred.LocalJobRunner: job_local1562971559_0001
java.lang.Exception: java.lang.NullPointerException
	at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.lang.NullPointerException
	at similarity.WordSort$Map$1.compare(WordSort.java:135)
	at similarity.WordSort$Map$1.compare(WordSort.java:1)
	at java.util.TimSort.countRunAndMakeAscending(TimSort.java:324)
	at java.util.TimSort.sort(TimSort.java:189)
	at java.util.TimSort.sort(TimSort.java:173)
	at java.util.Arrays.sort(Arrays.java:659)
	at java.util.Collections.sort(Collections.java:217)
	at similarity.WordSort$Map.map(WordSort.java:131)
	at similarity.WordSort$Map.map(WordSort.java:1)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
	at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2017-03-10 14:26:16,400 INFO org.apache.hadoop.mapreduce.Job: Job job_local1562971559_0001 failed with state FAILED due to: NA
2017-03-10 14:26:16,402 INFO org.apache.hadoop.mapreduce.Job: Counters: 0
2017-03-10 14:29:37,807 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-03-10 14:29:38,356 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2017-03-10 14:29:38,361 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-03-10 14:29:38,865 WARN org.apache.hadoop.mapreduce.JobResourceUploader: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2017-03-10 14:29:38,874 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1
2017-03-10 14:29:38,909 INFO org.apache.hadoop.mapreduce.JobSubmitter: number of splits:1
2017-03-10 14:29:39,202 INFO org.apache.hadoop.mapreduce.JobSubmitter: Submitting tokens for job: job_local705841975_0001
2017-03-10 14:29:39,678 INFO org.apache.hadoop.mapreduce.Job: The url to track the job: http://localhost:8080/
2017-03-10 14:29:39,680 INFO org.apache.hadoop.mapreduce.Job: Running job: job_local705841975_0001
2017-03-10 14:29:39,685 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter set in config null
2017-03-10 14:29:39,703 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:29:39,705 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2017-03-10 14:29:39,828 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for map tasks
2017-03-10 14:29:39,829 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local705841975_0001_m_000000_0
2017-03-10 14:29:39,879 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:29:39,891 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:29:39,896 INFO org.apache.hadoop.mapred.MapTask: Processing split: file:/home/cloudera/workspace/WordCount/input/pg100.txt:0+5589886
2017-03-10 14:29:39,983 INFO org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-03-10 14:29:39,984 INFO org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-03-10 14:29:39,984 INFO org.apache.hadoop.mapred.MapTask: soft limit at 83886080
2017-03-10 14:29:39,984 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-03-10 14:29:39,984 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600
2017-03-10 14:29:39,988 INFO org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-03-10 14:29:40,687 INFO org.apache.hadoop.mapreduce.Job: Job job_local705841975_0001 running in uber mode : false
2017-03-10 14:29:40,690 INFO org.apache.hadoop.mapreduce.Job:  map 0% reduce 0%
2017-03-10 14:29:40,701 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output
2017-03-10 14:29:40,708 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2017-03-10 14:29:40,717 INFO org.apache.hadoop.mapred.LocalJobRunner: map task executor complete.
2017-03-10 14:29:40,718 WARN org.apache.hadoop.mapred.LocalJobRunner: job_local705841975_0001
java.lang.Exception: java.lang.NullPointerException
	at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.lang.NullPointerException
	at similarity.WordSort$Map$1.compare(WordSort.java:138)
	at similarity.WordSort$Map$1.compare(WordSort.java:1)
	at java.util.TimSort.countRunAndMakeAscending(TimSort.java:324)
	at java.util.TimSort.sort(TimSort.java:189)
	at java.util.TimSort.sort(TimSort.java:173)
	at java.util.Arrays.sort(Arrays.java:659)
	at java.util.Collections.sort(Collections.java:217)
	at similarity.WordSort$Map.map(WordSort.java:134)
	at similarity.WordSort$Map.map(WordSort.java:1)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
	at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2017-03-10 14:29:41,693 INFO org.apache.hadoop.mapreduce.Job: Job job_local705841975_0001 failed with state FAILED due to: NA
2017-03-10 14:29:41,695 INFO org.apache.hadoop.mapreduce.Job: Counters: 0
2017-03-10 14:34:26,674 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-03-10 14:34:27,174 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2017-03-10 14:34:27,179 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-03-10 14:34:27,223 WARN org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:cloudera (auth:SIMPLE) cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory file:/home/cloudera/workspace/WordCount already exists
2017-03-10 14:34:55,125 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-03-10 14:34:55,598 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2017-03-10 14:34:55,612 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-03-10 14:34:56,114 WARN org.apache.hadoop.mapreduce.JobResourceUploader: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2017-03-10 14:34:56,121 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1
2017-03-10 14:34:56,259 INFO org.apache.hadoop.mapreduce.JobSubmitter: number of splits:1
2017-03-10 14:34:56,554 INFO org.apache.hadoop.mapreduce.JobSubmitter: Submitting tokens for job: job_local1013376941_0001
2017-03-10 14:34:56,980 INFO org.apache.hadoop.mapreduce.Job: The url to track the job: http://localhost:8080/
2017-03-10 14:34:56,982 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter set in config null
2017-03-10 14:34:56,983 INFO org.apache.hadoop.mapreduce.Job: Running job: job_local1013376941_0001
2017-03-10 14:34:56,990 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:34:56,995 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2017-03-10 14:34:57,121 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for map tasks
2017-03-10 14:34:57,124 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local1013376941_0001_m_000000_0
2017-03-10 14:34:57,181 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:34:57,198 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:34:57,202 INFO org.apache.hadoop.mapred.MapTask: Processing split: file:/home/cloudera/workspace/WordCount/input/pg100.txt:0+5589886
2017-03-10 14:34:57,295 INFO org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-03-10 14:34:57,298 INFO org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-03-10 14:34:57,298 INFO org.apache.hadoop.mapred.MapTask: soft limit at 83886080
2017-03-10 14:34:57,298 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-03-10 14:34:57,298 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600
2017-03-10 14:34:57,300 INFO org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-03-10 14:34:57,967 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output
2017-03-10 14:34:57,974 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2017-03-10 14:34:57,988 INFO org.apache.hadoop.mapred.LocalJobRunner: map task executor complete.
2017-03-10 14:34:57,990 INFO org.apache.hadoop.mapreduce.Job: Job job_local1013376941_0001 running in uber mode : false
2017-03-10 14:34:57,991 INFO org.apache.hadoop.mapreduce.Job:  map 0% reduce 0%
2017-03-10 14:34:57,993 WARN org.apache.hadoop.mapred.LocalJobRunner: job_local1013376941_0001
java.lang.Exception: java.lang.NullPointerException
	at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.lang.NullPointerException
	at similarity.WordSort$Map$1.compare(WordSort.java:146)
	at similarity.WordSort$Map$1.compare(WordSort.java:1)
	at java.util.TimSort.countRunAndMakeAscending(TimSort.java:324)
	at java.util.TimSort.sort(TimSort.java:189)
	at java.util.TimSort.sort(TimSort.java:173)
	at java.util.Arrays.sort(Arrays.java:659)
	at java.util.Collections.sort(Collections.java:217)
	at similarity.WordSort$Map.map(WordSort.java:142)
	at similarity.WordSort$Map.map(WordSort.java:1)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
	at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2017-03-10 14:34:57,995 INFO org.apache.hadoop.mapreduce.Job: Job job_local1013376941_0001 failed with state FAILED due to: NA
2017-03-10 14:34:57,998 INFO org.apache.hadoop.mapreduce.Job: Counters: 0
2017-03-10 14:38:51,972 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-03-10 14:38:52,515 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2017-03-10 14:38:52,527 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-03-10 14:38:53,098 WARN org.apache.hadoop.mapreduce.JobResourceUploader: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2017-03-10 14:38:53,128 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1
2017-03-10 14:38:53,247 INFO org.apache.hadoop.mapreduce.JobSubmitter: number of splits:1
2017-03-10 14:38:53,578 INFO org.apache.hadoop.mapreduce.JobSubmitter: Submitting tokens for job: job_local1410438889_0001
2017-03-10 14:38:54,016 INFO org.apache.hadoop.mapreduce.Job: The url to track the job: http://localhost:8080/
2017-03-10 14:38:54,017 INFO org.apache.hadoop.mapreduce.Job: Running job: job_local1410438889_0001
2017-03-10 14:38:54,024 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter set in config null
2017-03-10 14:38:54,064 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:38:54,065 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2017-03-10 14:38:54,219 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for map tasks
2017-03-10 14:38:54,220 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local1410438889_0001_m_000000_0
2017-03-10 14:38:54,279 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:38:54,293 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:38:54,296 INFO org.apache.hadoop.mapred.MapTask: Processing split: file:/home/cloudera/workspace/WordCount/input/pg100.txt:0+5589886
2017-03-10 14:38:54,396 INFO org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-03-10 14:38:54,396 INFO org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-03-10 14:38:54,396 INFO org.apache.hadoop.mapred.MapTask: soft limit at 83886080
2017-03-10 14:38:54,397 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-03-10 14:38:54,397 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600
2017-03-10 14:38:54,400 INFO org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-03-10 14:38:55,024 INFO org.apache.hadoop.mapreduce.Job: Job job_local1410438889_0001 running in uber mode : false
2017-03-10 14:38:55,027 INFO org.apache.hadoop.mapreduce.Job:  map 0% reduce 0%
2017-03-10 14:38:55,119 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output
2017-03-10 14:38:55,119 INFO org.apache.hadoop.mapred.MapTask: Spilling map output
2017-03-10 14:38:55,119 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 325; bufvoid = 104857600
2017-03-10 14:38:55,119 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214376(104857504); length = 21/6553600
2017-03-10 14:38:55,128 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2017-03-10 14:38:55,137 INFO org.apache.hadoop.mapred.MapTask: Finished spill 0
2017-03-10 14:38:55,141 INFO org.apache.hadoop.mapred.LocalJobRunner: map task executor complete.
2017-03-10 14:38:55,141 WARN org.apache.hadoop.mapred.LocalJobRunner: job_local1410438889_0001
java.lang.Exception: java.lang.NullPointerException
	at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.lang.NullPointerException
	at java.lang.Integer.compareTo(Integer.java:1003)
	at similarity.WordSort$Map$1.compare(WordSort.java:144)
	at similarity.WordSort$Map$1.compare(WordSort.java:1)
	at java.util.TimSort.countRunAndMakeAscending(TimSort.java:324)
	at java.util.TimSort.sort(TimSort.java:189)
	at java.util.TimSort.sort(TimSort.java:173)
	at java.util.Arrays.sort(Arrays.java:659)
	at java.util.Collections.sort(Collections.java:217)
	at similarity.WordSort$Map.map(WordSort.java:140)
	at similarity.WordSort$Map.map(WordSort.java:1)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
	at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2017-03-10 14:38:56,030 INFO org.apache.hadoop.mapreduce.Job: Job job_local1410438889_0001 failed with state FAILED due to: NA
2017-03-10 14:38:56,035 INFO org.apache.hadoop.mapreduce.Job: Counters: 0
2017-03-10 14:40:16,992 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-03-10 14:40:17,522 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2017-03-10 14:40:17,536 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-03-10 14:40:18,047 WARN org.apache.hadoop.mapreduce.JobResourceUploader: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2017-03-10 14:40:18,055 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1
2017-03-10 14:40:18,089 INFO org.apache.hadoop.mapreduce.JobSubmitter: number of splits:1
2017-03-10 14:40:18,383 INFO org.apache.hadoop.mapreduce.JobSubmitter: Submitting tokens for job: job_local483843492_0001
2017-03-10 14:40:18,855 INFO org.apache.hadoop.mapreduce.Job: The url to track the job: http://localhost:8080/
2017-03-10 14:40:18,856 INFO org.apache.hadoop.mapreduce.Job: Running job: job_local483843492_0001
2017-03-10 14:40:18,860 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter set in config null
2017-03-10 14:40:18,887 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:40:18,889 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2017-03-10 14:40:19,021 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for map tasks
2017-03-10 14:40:19,023 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local483843492_0001_m_000000_0
2017-03-10 14:40:19,078 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:40:19,106 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:40:19,109 INFO org.apache.hadoop.mapred.MapTask: Processing split: file:/home/cloudera/workspace/WordCount/input/pg100.txt:0+5589886
2017-03-10 14:40:19,243 INFO org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-03-10 14:40:19,243 INFO org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-03-10 14:40:19,243 INFO org.apache.hadoop.mapred.MapTask: soft limit at 83886080
2017-03-10 14:40:19,244 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-03-10 14:40:19,244 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600
2017-03-10 14:40:19,249 INFO org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-03-10 14:40:19,852 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output
2017-03-10 14:40:19,852 INFO org.apache.hadoop.mapred.MapTask: Spilling map output
2017-03-10 14:40:19,852 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 325; bufvoid = 104857600
2017-03-10 14:40:19,852 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214376(104857504); length = 21/6553600
2017-03-10 14:40:19,858 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2017-03-10 14:40:19,860 INFO org.apache.hadoop.mapreduce.Job: Job job_local483843492_0001 running in uber mode : false
2017-03-10 14:40:19,861 INFO org.apache.hadoop.mapreduce.Job:  map 0% reduce 0%
2017-03-10 14:40:19,866 INFO org.apache.hadoop.mapred.MapTask: Finished spill 0
2017-03-10 14:40:19,869 INFO org.apache.hadoop.mapred.LocalJobRunner: map task executor complete.
2017-03-10 14:40:19,870 WARN org.apache.hadoop.mapred.LocalJobRunner: job_local483843492_0001
java.lang.Exception: java.lang.NullPointerException
	at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.lang.NullPointerException
	at java.lang.Integer.compareTo(Integer.java:1003)
	at similarity.WordSort$Map$1.compare(WordSort.java:144)
	at similarity.WordSort$Map$1.compare(WordSort.java:1)
	at java.util.TimSort.countRunAndMakeAscending(TimSort.java:324)
	at java.util.TimSort.sort(TimSort.java:189)
	at java.util.TimSort.sort(TimSort.java:173)
	at java.util.Arrays.sort(Arrays.java:659)
	at java.util.Collections.sort(Collections.java:217)
	at similarity.WordSort$Map.map(WordSort.java:140)
	at similarity.WordSort$Map.map(WordSort.java:1)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
	at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2017-03-10 14:40:20,864 INFO org.apache.hadoop.mapreduce.Job: Job job_local483843492_0001 failed with state FAILED due to: NA
2017-03-10 14:40:20,871 INFO org.apache.hadoop.mapreduce.Job: Counters: 0
2017-03-10 14:41:43,006 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-03-10 14:41:43,480 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2017-03-10 14:41:43,485 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-03-10 14:41:44,008 WARN org.apache.hadoop.mapreduce.JobResourceUploader: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2017-03-10 14:41:44,042 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1
2017-03-10 14:41:44,079 INFO org.apache.hadoop.mapreduce.JobSubmitter: number of splits:1
2017-03-10 14:41:44,328 INFO org.apache.hadoop.mapreduce.JobSubmitter: Submitting tokens for job: job_local1726509137_0001
2017-03-10 14:41:44,799 INFO org.apache.hadoop.mapreduce.Job: The url to track the job: http://localhost:8080/
2017-03-10 14:41:44,800 INFO org.apache.hadoop.mapreduce.Job: Running job: job_local1726509137_0001
2017-03-10 14:41:44,806 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter set in config null
2017-03-10 14:41:44,835 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:41:44,838 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2017-03-10 14:41:44,977 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for map tasks
2017-03-10 14:41:44,978 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local1726509137_0001_m_000000_0
2017-03-10 14:41:45,030 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:41:45,051 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:41:45,058 INFO org.apache.hadoop.mapred.MapTask: Processing split: file:/home/cloudera/workspace/WordCount/input/pg100.txt:0+5589886
2017-03-10 14:41:45,138 INFO org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-03-10 14:41:45,142 INFO org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-03-10 14:41:45,143 INFO org.apache.hadoop.mapred.MapTask: soft limit at 83886080
2017-03-10 14:41:45,143 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-03-10 14:41:45,143 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600
2017-03-10 14:41:45,149 INFO org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-03-10 14:41:45,805 INFO org.apache.hadoop.mapreduce.Job: Job job_local1726509137_0001 running in uber mode : false
2017-03-10 14:41:45,806 INFO org.apache.hadoop.mapreduce.Job:  map 0% reduce 0%
2017-03-10 14:41:48,519 INFO org.apache.hadoop.mapred.LocalJobRunner: 
2017-03-10 14:41:48,519 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output
2017-03-10 14:41:48,519 INFO org.apache.hadoop.mapred.MapTask: Spilling map output
2017-03-10 14:41:48,519 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 4776007; bufvoid = 104857600
2017-03-10 14:41:48,519 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396(104857584); kvend = 25753980(103015920); length = 460417/6553600
2017-03-10 14:41:48,780 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2017-03-10 14:41:49,395 INFO org.apache.hadoop.mapred.MapTask: Finished spill 0
2017-03-10 14:41:49,398 INFO org.apache.hadoop.mapred.Task: Task:attempt_local1726509137_0001_m_000000_0 is done. And is in the process of committing
2017-03-10 14:41:49,404 INFO org.apache.hadoop.mapred.LocalJobRunner: map
2017-03-10 14:41:49,408 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local1726509137_0001_m_000000_0' done.
2017-03-10 14:41:49,409 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local1726509137_0001_m_000000_0
2017-03-10 14:41:49,409 INFO org.apache.hadoop.mapred.LocalJobRunner: map task executor complete.
2017-03-10 14:41:49,411 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for reduce tasks
2017-03-10 14:41:49,411 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local1726509137_0001_r_000000_0
2017-03-10 14:41:49,416 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:41:49,417 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:41:49,419 INFO org.apache.hadoop.mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@70d586bc
2017-03-10 14:41:49,429 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: MergerManager: memoryLimit=679778688, maxSingleShuffleLimit=169944672, mergeThreshold=448653952, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2017-03-10 14:41:49,435 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_local1726509137_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2017-03-10 14:41:49,461 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.deflate]
2017-03-10 14:41:49,461 INFO org.apache.hadoop.mapreduce.task.reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1726509137_0001_m_000000_0 decomp: 5006219 len: 2376884 to MEMORY
2017-03-10 14:41:49,517 INFO org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 5006219 bytes from map-output for attempt_local1726509137_0001_m_000000_0
2017-03-10 14:41:49,521 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 5006219, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->5006219
2017-03-10 14:41:49,523 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: EventFetcher is interrupted.. Returning
2017-03-10 14:41:49,524 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:41:49,524 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2017-03-10 14:41:49,529 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 14:41:49,529 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 5006209 bytes
2017-03-10 14:41:49,818 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 0%
2017-03-10 14:41:50,226 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merged 1 segments, 5006219 bytes to disk to satisfy reduce memory limit
2017-03-10 14:41:50,227 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 1 files, 2376892 bytes from disk
2017-03-10 14:41:50,227 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce
2017-03-10 14:41:50,228 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 14:41:50,228 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 5006209 bytes
2017-03-10 14:41:50,230 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:41:50,235 INFO org.apache.hadoop.conf.Configuration.deprecation: mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords
2017-03-10 14:41:50,888 INFO org.apache.hadoop.mapred.Task: Task:attempt_local1726509137_0001_r_000000_0 is done. And is in the process of committing
2017-03-10 14:41:50,898 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:41:50,899 INFO org.apache.hadoop.mapred.Task: Task attempt_local1726509137_0001_r_000000_0 is allowed to commit now
2017-03-10 14:41:50,900 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Saved output of task 'attempt_local1726509137_0001_r_000000_0' to file:/home/cloudera/workspace/WordCount/output/_temporary/0/task_local1726509137_0001_r_000000
2017-03-10 14:41:50,901 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce > reduce
2017-03-10 14:41:50,901 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local1726509137_0001_r_000000_0' done.
2017-03-10 14:41:50,901 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local1726509137_0001_r_000000_0
2017-03-10 14:41:50,901 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce task executor complete.
2017-03-10 14:41:51,822 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 100%
2017-03-10 14:41:51,822 INFO org.apache.hadoop.mapreduce.Job: Job job_local1726509137_0001 completed successfully
2017-03-10 14:41:51,834 INFO org.apache.hadoop.mapreduce.Job: Counters: 33
	File System Counters
		FILE: Number of bytes read=16657772
		FILE: Number of bytes written=12451545
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
	Map-Reduce Framework
		Map input records=124787
		Map output records=115105
		Map output bytes=4776007
		Map output materialized bytes=2376884
		Input split bytes=120
		Combine input records=0
		Combine output records=0
		Reduce input groups=115105
		Reduce shuffle bytes=2376884
		Reduce input records=115105
		Reduce output records=115105
		Spilled Records=230210
		Shuffled Maps =1
		Failed Shuffles=0
		Merged Map outputs=1
		GC time elapsed (ms)=140
		CPU time spent (ms)=0
		Physical memory (bytes) snapshot=0
		Virtual memory (bytes) snapshot=0
		Total committed heap usage (bytes)=331227136
	Shuffle Errors
		BAD_ID=0
		CONNECTION=0
		IO_ERROR=0
		WRONG_LENGTH=0
		WRONG_MAP=0
		WRONG_REDUCE=0
	File Input Format Counters 
		Bytes Read=5589886
	File Output Format Counters 
		Bytes Written=4789771
2017-03-10 14:54:30,023 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-03-10 14:54:30,544 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2017-03-10 14:54:30,565 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-03-10 14:54:31,208 WARN org.apache.hadoop.mapreduce.JobResourceUploader: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2017-03-10 14:54:31,222 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1
2017-03-10 14:54:31,311 INFO org.apache.hadoop.mapreduce.JobSubmitter: number of splits:1
2017-03-10 14:54:31,680 INFO org.apache.hadoop.mapreduce.JobSubmitter: Submitting tokens for job: job_local520002813_0001
2017-03-10 14:54:32,173 INFO org.apache.hadoop.mapreduce.Job: The url to track the job: http://localhost:8080/
2017-03-10 14:54:32,174 INFO org.apache.hadoop.mapreduce.Job: Running job: job_local520002813_0001
2017-03-10 14:54:32,184 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter set in config null
2017-03-10 14:54:32,195 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:54:32,197 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2017-03-10 14:54:32,352 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for map tasks
2017-03-10 14:54:32,354 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local520002813_0001_m_000000_0
2017-03-10 14:54:32,427 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:54:32,457 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:54:32,463 INFO org.apache.hadoop.mapred.MapTask: Processing split: file:/home/cloudera/workspace/WordCount/input/pg100.txt:0+5589886
2017-03-10 14:54:32,548 INFO org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-03-10 14:54:32,549 INFO org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-03-10 14:54:32,550 INFO org.apache.hadoop.mapred.MapTask: soft limit at 83886080
2017-03-10 14:54:32,550 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-03-10 14:54:32,550 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600
2017-03-10 14:54:32,559 INFO org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-03-10 14:54:33,178 INFO org.apache.hadoop.mapreduce.Job: Job job_local520002813_0001 running in uber mode : false
2017-03-10 14:54:33,179 INFO org.apache.hadoop.mapreduce.Job:  map 0% reduce 0%
2017-03-10 14:54:35,891 INFO org.apache.hadoop.mapred.LocalJobRunner: 
2017-03-10 14:54:35,891 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output
2017-03-10 14:54:35,891 INFO org.apache.hadoop.mapred.MapTask: Spilling map output
2017-03-10 14:54:35,891 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 5541084; bufvoid = 104857600
2017-03-10 14:54:35,891 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396(104857584); kvend = 25753980(103015920); length = 460417/6553600
2017-03-10 14:54:36,206 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2017-03-10 14:54:37,261 INFO org.apache.hadoop.mapred.MapTask: Finished spill 0
2017-03-10 14:54:37,264 INFO org.apache.hadoop.mapred.Task: Task:attempt_local520002813_0001_m_000000_0 is done. And is in the process of committing
2017-03-10 14:54:37,275 INFO org.apache.hadoop.mapred.LocalJobRunner: map
2017-03-10 14:54:37,280 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local520002813_0001_m_000000_0' done.
2017-03-10 14:54:37,280 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local520002813_0001_m_000000_0
2017-03-10 14:54:37,280 INFO org.apache.hadoop.mapred.LocalJobRunner: map task executor complete.
2017-03-10 14:54:37,284 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for reduce tasks
2017-03-10 14:54:37,284 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local520002813_0001_r_000000_0
2017-03-10 14:54:37,289 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:54:37,289 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:54:37,293 INFO org.apache.hadoop.mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@22c4d1da
2017-03-10 14:54:37,303 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: MergerManager: memoryLimit=679778688, maxSingleShuffleLimit=169944672, mergeThreshold=448653952, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2017-03-10 14:54:37,312 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_local520002813_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2017-03-10 14:54:37,342 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.deflate]
2017-03-10 14:54:37,342 INFO org.apache.hadoop.mapreduce.task.reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local520002813_0001_m_000000_0 decomp: 5771296 len: 2500118 to MEMORY
2017-03-10 14:54:37,405 INFO org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 5771296 bytes from map-output for attempt_local520002813_0001_m_000000_0
2017-03-10 14:54:37,408 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 5771296, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->5771296
2017-03-10 14:54:37,410 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: EventFetcher is interrupted.. Returning
2017-03-10 14:54:37,410 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:54:37,411 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2017-03-10 14:54:37,420 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 14:54:37,420 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 5771286 bytes
2017-03-10 14:54:38,188 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 0%
2017-03-10 14:54:38,209 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merged 1 segments, 5771296 bytes to disk to satisfy reduce memory limit
2017-03-10 14:54:38,209 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 1 files, 2500126 bytes from disk
2017-03-10 14:54:38,210 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce
2017-03-10 14:54:38,210 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 14:54:38,210 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 5771286 bytes
2017-03-10 14:54:38,211 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:54:38,219 INFO org.apache.hadoop.conf.Configuration.deprecation: mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords
2017-03-10 14:54:39,174 INFO org.apache.hadoop.mapred.Task: Task:attempt_local520002813_0001_r_000000_0 is done. And is in the process of committing
2017-03-10 14:54:39,181 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:54:39,181 INFO org.apache.hadoop.mapred.Task: Task attempt_local520002813_0001_r_000000_0 is allowed to commit now
2017-03-10 14:54:39,187 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Saved output of task 'attempt_local520002813_0001_r_000000_0' to file:/home/cloudera/workspace/WordCount/output/_temporary/0/task_local520002813_0001_r_000000
2017-03-10 14:54:39,188 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce > reduce
2017-03-10 14:54:39,188 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local520002813_0001_r_000000_0' done.
2017-03-10 14:54:39,188 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local520002813_0001_r_000000_0
2017-03-10 14:54:39,188 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce task executor complete.
2017-03-10 14:54:39,190 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 100%
2017-03-10 14:54:40,191 INFO org.apache.hadoop.mapreduce.Job: Job job_local520002813_0001 completed successfully
2017-03-10 14:54:40,203 INFO org.apache.hadoop.mapreduce.Job: Counters: 34
	File System Counters
		FILE: Number of bytes read=16904240
		FILE: Number of bytes written=13589484
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
	Map-Reduce Framework
		Map input records=124787
		Map output records=115105
		Map output bytes=5541084
		Map output materialized bytes=2500118
		Input split bytes=120
		Combine input records=0
		Combine output records=0
		Reduce input groups=115105
		Reduce shuffle bytes=2500118
		Reduce input records=115105
		Reduce output records=115105
		Spilled Records=230210
		Shuffled Maps =1
		Failed Shuffles=0
		Merged Map outputs=1
		GC time elapsed (ms)=124
		CPU time spent (ms)=0
		Physical memory (bytes) snapshot=0
		Virtual memory (bytes) snapshot=0
		Total committed heap usage (bytes)=331227136
	Shuffle Errors
		BAD_ID=0
		CONNECTION=0
		IO_ERROR=0
		WRONG_LENGTH=0
		WRONG_MAP=0
		WRONG_REDUCE=0
	File Input Format Counters 
		Bytes Read=5589886
	File Output Format Counters 
		Bytes Written=5560824
	similarity.WordSort$DocLineCounter
		NUM=124787
2017-03-10 14:58:53,626 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-03-10 14:58:54,092 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2017-03-10 14:58:54,097 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-03-10 14:58:54,564 WARN org.apache.hadoop.mapreduce.JobResourceUploader: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2017-03-10 14:58:54,572 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1
2017-03-10 14:58:54,619 INFO org.apache.hadoop.mapreduce.JobSubmitter: number of splits:1
2017-03-10 14:58:54,850 INFO org.apache.hadoop.mapreduce.JobSubmitter: Submitting tokens for job: job_local544686432_0001
2017-03-10 14:58:55,274 INFO org.apache.hadoop.mapreduce.Job: The url to track the job: http://localhost:8080/
2017-03-10 14:58:55,275 INFO org.apache.hadoop.mapreduce.Job: Running job: job_local544686432_0001
2017-03-10 14:58:55,284 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter set in config null
2017-03-10 14:58:55,298 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:58:55,301 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2017-03-10 14:58:55,409 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for map tasks
2017-03-10 14:58:55,411 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local544686432_0001_m_000000_0
2017-03-10 14:58:55,473 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:58:55,485 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:58:55,487 INFO org.apache.hadoop.mapred.MapTask: Processing split: file:/home/cloudera/workspace/WordCount/input/pg100.txt:0+5589886
2017-03-10 14:58:55,566 INFO org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-03-10 14:58:55,567 INFO org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-03-10 14:58:55,567 INFO org.apache.hadoop.mapred.MapTask: soft limit at 83886080
2017-03-10 14:58:55,567 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-03-10 14:58:55,567 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600
2017-03-10 14:58:55,569 INFO org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-03-10 14:58:56,279 INFO org.apache.hadoop.mapreduce.Job: Job job_local544686432_0001 running in uber mode : false
2017-03-10 14:58:56,280 INFO org.apache.hadoop.mapreduce.Job:  map 0% reduce 0%
2017-03-10 14:58:58,471 INFO org.apache.hadoop.mapred.LocalJobRunner: 
2017-03-10 14:58:58,472 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output
2017-03-10 14:58:58,472 INFO org.apache.hadoop.mapred.MapTask: Spilling map output
2017-03-10 14:58:58,472 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 5541084; bufvoid = 104857600
2017-03-10 14:58:58,472 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396(104857584); kvend = 25753980(103015920); length = 460417/6553600
2017-03-10 14:58:58,671 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2017-03-10 14:58:59,336 INFO org.apache.hadoop.mapred.MapTask: Finished spill 0
2017-03-10 14:58:59,339 INFO org.apache.hadoop.mapred.Task: Task:attempt_local544686432_0001_m_000000_0 is done. And is in the process of committing
2017-03-10 14:58:59,344 INFO org.apache.hadoop.mapred.LocalJobRunner: map
2017-03-10 14:58:59,344 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local544686432_0001_m_000000_0' done.
2017-03-10 14:58:59,344 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local544686432_0001_m_000000_0
2017-03-10 14:58:59,345 INFO org.apache.hadoop.mapred.LocalJobRunner: map task executor complete.
2017-03-10 14:58:59,347 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for reduce tasks
2017-03-10 14:58:59,348 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local544686432_0001_r_000000_0
2017-03-10 14:58:59,353 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 14:58:59,353 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 14:58:59,355 INFO org.apache.hadoop.mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@8a93430
2017-03-10 14:58:59,364 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: MergerManager: memoryLimit=679778688, maxSingleShuffleLimit=169944672, mergeThreshold=448653952, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2017-03-10 14:58:59,370 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_local544686432_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2017-03-10 14:58:59,395 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.deflate]
2017-03-10 14:58:59,395 INFO org.apache.hadoop.mapreduce.task.reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local544686432_0001_m_000000_0 decomp: 5771296 len: 2405517 to MEMORY
2017-03-10 14:58:59,447 INFO org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 5771296 bytes from map-output for attempt_local544686432_0001_m_000000_0
2017-03-10 14:58:59,447 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 5771296, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->5771296
2017-03-10 14:58:59,448 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: EventFetcher is interrupted.. Returning
2017-03-10 14:58:59,450 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:58:59,450 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2017-03-10 14:58:59,454 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 14:58:59,455 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 5771286 bytes
2017-03-10 14:59:00,203 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merged 1 segments, 5771296 bytes to disk to satisfy reduce memory limit
2017-03-10 14:59:00,203 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 1 files, 2405525 bytes from disk
2017-03-10 14:59:00,204 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce
2017-03-10 14:59:00,204 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 14:59:00,205 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 5771286 bytes
2017-03-10 14:59:00,205 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:59:00,210 INFO org.apache.hadoop.conf.Configuration.deprecation: mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords
2017-03-10 14:59:00,290 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 0%
2017-03-10 14:59:00,890 INFO org.apache.hadoop.mapred.Task: Task:attempt_local544686432_0001_r_000000_0 is done. And is in the process of committing
2017-03-10 14:59:00,895 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 14:59:00,895 INFO org.apache.hadoop.mapred.Task: Task attempt_local544686432_0001_r_000000_0 is allowed to commit now
2017-03-10 14:59:00,896 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Saved output of task 'attempt_local544686432_0001_r_000000_0' to file:/home/cloudera/workspace/WordCount/output/_temporary/0/task_local544686432_0001_r_000000
2017-03-10 14:59:00,897 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce > reduce
2017-03-10 14:59:00,897 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local544686432_0001_r_000000_0' done.
2017-03-10 14:59:00,901 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local544686432_0001_r_000000_0
2017-03-10 14:59:00,901 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce task executor complete.
2017-03-10 14:59:01,290 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 100%
2017-03-10 14:59:01,291 INFO org.apache.hadoop.mapreduce.Job: Job job_local544686432_0001 completed successfully
2017-03-10 14:59:01,312 INFO org.apache.hadoop.mapreduce.Job: Counters: 34
	File System Counters
		FILE: Number of bytes read=16715038
		FILE: Number of bytes written=13110192
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
	Map-Reduce Framework
		Map input records=124787
		Map output records=115105
		Map output bytes=5541084
		Map output materialized bytes=2405517
		Input split bytes=120
		Combine input records=0
		Combine output records=0
		Reduce input groups=115105
		Reduce shuffle bytes=2405517
		Reduce input records=115105
		Reduce output records=115105
		Spilled Records=230210
		Shuffled Maps =1
		Failed Shuffles=0
		Merged Map outputs=1
		GC time elapsed (ms)=113
		CPU time spent (ms)=0
		Physical memory (bytes) snapshot=0
		Virtual memory (bytes) snapshot=0
		Total committed heap usage (bytes)=331227136
	Shuffle Errors
		BAD_ID=0
		CONNECTION=0
		IO_ERROR=0
		WRONG_LENGTH=0
		WRONG_MAP=0
		WRONG_REDUCE=0
	File Input Format Counters 
		Bytes Read=5589886
	File Output Format Counters 
		Bytes Written=5365335
	similarity.WordSort$DocLineCounter
		NUM=124787
2017-03-10 16:04:16,924 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-03-10 16:04:17,595 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2017-03-10 16:04:17,605 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-03-10 16:04:17,661 WARN org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:cloudera (auth:SIMPLE) cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory file:/home/cloudera/workspace/WordCount/output already exists
2017-03-10 16:04:38,662 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-03-10 16:04:39,236 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2017-03-10 16:04:39,243 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-03-10 16:04:39,836 WARN org.apache.hadoop.mapreduce.JobResourceUploader: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2017-03-10 16:04:39,846 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1
2017-03-10 16:04:39,896 INFO org.apache.hadoop.mapreduce.JobSubmitter: number of splits:1
2017-03-10 16:04:40,232 INFO org.apache.hadoop.mapreduce.JobSubmitter: Submitting tokens for job: job_local2107526736_0001
2017-03-10 16:04:40,744 INFO org.apache.hadoop.mapreduce.Job: The url to track the job: http://localhost:8080/
2017-03-10 16:04:40,745 INFO org.apache.hadoop.mapreduce.Job: Running job: job_local2107526736_0001
2017-03-10 16:04:40,753 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter set in config null
2017-03-10 16:04:40,762 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 16:04:40,772 INFO org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2017-03-10 16:04:40,922 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for map tasks
2017-03-10 16:04:40,925 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local2107526736_0001_m_000000_0
2017-03-10 16:04:40,998 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 16:04:41,021 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 16:04:41,024 INFO org.apache.hadoop.mapred.MapTask: Processing split: file:/home/cloudera/workspace/WordCount/input/pg100.txt:0+5589886
2017-03-10 16:04:41,117 INFO org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-03-10 16:04:41,117 INFO org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-03-10 16:04:41,117 INFO org.apache.hadoop.mapred.MapTask: soft limit at 83886080
2017-03-10 16:04:41,117 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-03-10 16:04:41,117 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600
2017-03-10 16:04:41,121 INFO org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-03-10 16:04:41,747 INFO org.apache.hadoop.mapreduce.Job: Job job_local2107526736_0001 running in uber mode : false
2017-03-10 16:04:41,749 INFO org.apache.hadoop.mapreduce.Job:  map 0% reduce 0%
2017-03-10 16:04:44,514 INFO org.apache.hadoop.mapred.LocalJobRunner: 
2017-03-10 16:04:44,515 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output
2017-03-10 16:04:44,515 INFO org.apache.hadoop.mapred.MapTask: Spilling map output
2017-03-10 16:04:44,515 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 5541084; bufvoid = 104857600
2017-03-10 16:04:44,515 INFO org.apache.hadoop.mapred.MapTask: kvstart = 26214396(104857584); kvend = 25753980(103015920); length = 460417/6553600
2017-03-10 16:04:44,850 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2017-03-10 16:04:45,645 INFO org.apache.hadoop.mapred.MapTask: Finished spill 0
2017-03-10 16:04:45,648 INFO org.apache.hadoop.mapred.Task: Task:attempt_local2107526736_0001_m_000000_0 is done. And is in the process of committing
2017-03-10 16:04:45,653 INFO org.apache.hadoop.mapred.LocalJobRunner: map
2017-03-10 16:04:45,654 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local2107526736_0001_m_000000_0' done.
2017-03-10 16:04:45,654 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local2107526736_0001_m_000000_0
2017-03-10 16:04:45,654 INFO org.apache.hadoop.mapred.LocalJobRunner: map task executor complete.
2017-03-10 16:04:45,658 INFO org.apache.hadoop.mapred.LocalJobRunner: Waiting for reduce tasks
2017-03-10 16:04:45,659 INFO org.apache.hadoop.mapred.LocalJobRunner: Starting task: attempt_local2107526736_0001_r_000000_0
2017-03-10 16:04:45,664 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-03-10 16:04:45,664 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2017-03-10 16:04:45,666 INFO org.apache.hadoop.mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@a9b03af
2017-03-10 16:04:45,680 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: MergerManager: memoryLimit=679778688, maxSingleShuffleLimit=169944672, mergeThreshold=448653952, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2017-03-10 16:04:45,687 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_local2107526736_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2017-03-10 16:04:45,730 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.deflate]
2017-03-10 16:04:45,730 INFO org.apache.hadoop.mapreduce.task.reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local2107526736_0001_m_000000_0 decomp: 5771296 len: 2404026 to MEMORY
2017-03-10 16:04:45,758 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 0%
2017-03-10 16:04:45,813 INFO org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 5771296 bytes from map-output for attempt_local2107526736_0001_m_000000_0
2017-03-10 16:04:45,813 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 5771296, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->5771296
2017-03-10 16:04:45,817 INFO org.apache.hadoop.mapreduce.task.reduce.EventFetcher: EventFetcher is interrupted.. Returning
2017-03-10 16:04:45,818 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 16:04:45,818 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2017-03-10 16:04:45,825 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 16:04:45,826 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 5771286 bytes
2017-03-10 16:04:46,671 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merged 1 segments, 5771296 bytes to disk to satisfy reduce memory limit
2017-03-10 16:04:46,672 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 1 files, 2404034 bytes from disk
2017-03-10 16:04:46,672 INFO org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce
2017-03-10 16:04:46,672 INFO org.apache.hadoop.mapred.Merger: Merging 1 sorted segments
2017-03-10 16:04:46,673 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 5771286 bytes
2017-03-10 16:04:46,674 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 16:04:46,680 INFO org.apache.hadoop.conf.Configuration.deprecation: mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords
2017-03-10 16:04:47,525 INFO org.apache.hadoop.mapred.Task: Task:attempt_local2107526736_0001_r_000000_0 is done. And is in the process of committing
2017-03-10 16:04:47,532 INFO org.apache.hadoop.mapred.LocalJobRunner: 1 / 1 copied.
2017-03-10 16:04:47,532 INFO org.apache.hadoop.mapred.Task: Task attempt_local2107526736_0001_r_000000_0 is allowed to commit now
2017-03-10 16:04:47,533 INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Saved output of task 'attempt_local2107526736_0001_r_000000_0' to file:/home/cloudera/workspace/WordCount/output/_temporary/0/task_local2107526736_0001_r_000000
2017-03-10 16:04:47,533 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce > reduce
2017-03-10 16:04:47,534 INFO org.apache.hadoop.mapred.Task: Task 'attempt_local2107526736_0001_r_000000_0' done.
2017-03-10 16:04:47,534 INFO org.apache.hadoop.mapred.LocalJobRunner: Finishing task: attempt_local2107526736_0001_r_000000_0
2017-03-10 16:04:47,534 INFO org.apache.hadoop.mapred.LocalJobRunner: reduce task executor complete.
2017-03-10 16:04:47,766 INFO org.apache.hadoop.mapreduce.Job:  map 100% reduce 100%
2017-03-10 16:04:47,767 INFO org.apache.hadoop.mapreduce.Job: Job job_local2107526736_0001 completed successfully
2017-03-10 16:04:47,785 INFO org.apache.hadoop.mapreduce.Job: Counters: 34
	File System Counters
		FILE: Number of bytes read=16712056
		FILE: Number of bytes written=13108535
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
	Map-Reduce Framework
		Map input records=124787
		Map output records=115105
		Map output bytes=5541084
		Map output materialized bytes=2404026
		Input split bytes=120
		Combine input records=0
		Combine output records=0
		Reduce input groups=115105
		Reduce shuffle bytes=2404026
		Reduce input records=115105
		Reduce output records=115105
		Spilled Records=230210
		Shuffled Maps =1
		Failed Shuffles=0
		Merged Map outputs=1
		GC time elapsed (ms)=142
		CPU time spent (ms)=0
		Physical memory (bytes) snapshot=0
		Virtual memory (bytes) snapshot=0
		Total committed heap usage (bytes)=331227136
	Shuffle Errors
		BAD_ID=0
		CONNECTION=0
		IO_ERROR=0
		WRONG_LENGTH=0
		WRONG_MAP=0
		WRONG_REDUCE=0
	File Input Format Counters 
		Bytes Read=5589886
	File Output Format Counters 
		Bytes Written=5365335
	similarity.WordSort$DocLineCounter
		NUM=124787