Test5

This is the page of Test5.

Goal

The goal of this test is to measure the elapsed times, bytes per sec, records per sec processed when the mapper outputs three times as many records as input records has been read. We compare the result with Test3, Test4.

##Setup Input data: 8 GB data in 8 text files. 1 GB each. The block size is 256 MB. Each file contains 16 bytes long rows. That is 16777216 rows (records). TextInputFormat is used for the mappers

Mapper: The mappers processes the input records and produces three records for each input records as output.

Reducers: There are no reducers.

Test and runs: 32 map tasks were running in one mrrun that is less the available tasks slots (19*4). 10 mrruns (run41...run50) where used in mrtest (test5).

##Observations

We can see from the picture below that the elapsed times are higher than in Test4.

Here we can see the mean elapsed times and standard deviation (also min and max) for the mrruns.

Here we have something about mbytes processes per sec for each run.

And heres is the number of input records processed for each run by per second. We can see that is less than in test4.