class Minitest::Reporters::MeanTimeReporter

This reporter creates a report providing the average (mean), minimum and maximum times for a test to run. Running this for all your tests will allow you to:

1) Identify the slowest running tests over time as potential candidates

for improvements or refactoring.

2) Identify (and fix) regressions in test run speed caused by changes to

your tests or algorithms in your code.

3) Provide an abundance of statistics to enjoy.

This is achieved by creating a (configurable) 'previous runs' statistics file which is parsed at the end of each run to provide a new (configurable) report. These statistics can be reset at any time by using a simple rake task:

rake reset_statistics

Attributes

all_suite_times[RW]

Public Class Methods

new(options = {}) click to toggle source

@param options [Hash] @option previous_runs_filename [String] Contains the times for each test

by description. Defaults to '/tmp/minitest_reporters_previous_run'.

@option report_filename [String] Contains the parsed results for the

last test run. Defaults to '/tmp/minitest_reporters_report'.

@option show_count [Fixnum] The number of tests to show in the report

summary at the end of the test run. Default is 15.

@option show_progress [Boolean] If true it prints pass/skip/fail marks.

Default is true.

@option show_all_runs [Boolean] If true it shows all recorded suit results.

Default is true.

@option sort_column [Symbol] One of :avg (default), :min, :max, :last.

Determines the column by which the report summary is sorted.

@option order [Symbol] One of :desc (default), or :asc. By default the

report summary is listed slowest to fastest (:desc). :asc will order
the report summary as fastest to slowest.

@return [Minitest::Reporters::MeanTimeReporter]

# File lib/minitest/reporters/mean_time_reporter.rb, line 55
def initialize(options = {})
  super

  @all_suite_times = []
end
reset_statistics!() click to toggle source

Reset the statistics file for this reporter. Called via a rake task:

rake reset_statistics

@return [Boolean]

# File lib/minitest/reporters/mean_time_reporter.rb, line 34
def self.reset_statistics!
  new.reset_statistics!
end

Public Instance Methods

after_suite(suite) click to toggle source

Copies the suite times from the {Minitest::Reporters::DefaultReporter#after_suite} method, making them available to this class.

@return [Hash<String => Float>]

# File lib/minitest/reporters/mean_time_reporter.rb, line 66
def after_suite(suite)
  super

  @all_suite_times = @suite_times
end
on_record(test) click to toggle source
# File lib/minitest/reporters/mean_time_reporter.rb, line 91
def on_record(test)
  super if options[:show_progress]
end
on_report() click to toggle source
# File lib/minitest/reporters/mean_time_reporter.rb, line 95
def on_report
  super if options[:show_progress]
end
on_start() click to toggle source
# File lib/minitest/reporters/mean_time_reporter.rb, line 87
def on_start
  super if options[:show_progress]
end
report() click to toggle source

Runs the {Minitest::Reporters::DefaultReporter#report} method and then enhances it by storing the results to the 'previous_runs_filename' and outputs the parsed results to both the 'report_filename' and the terminal.

# File lib/minitest/reporters/mean_time_reporter.rb, line 77
def report
  super

  create_or_update_previous_runs!

  create_new_report!

  write_to_screen!
end
reset_statistics!() click to toggle source

Resets the 'previous runs' file, essentially removing all previous statistics gathered.

@return [void]

# File lib/minitest/reporters/mean_time_reporter.rb, line 103
def reset_statistics!
  File.open(previous_runs_filename, 'w+') { |f| f.write('') }
end

Private Instance Methods

asc?() click to toggle source

@return [Boolean] Whether the given :order option is :asc.

# File lib/minitest/reporters/mean_time_reporter.rb, line 349
def asc?
  order == :asc
end
avg_label() click to toggle source

@return [String] A yellow 'Avg:' label.

# File lib/minitest/reporters/mean_time_reporter.rb, line 304
def avg_label
  ANSI::Code.yellow('Avg:')
end
column_sorted_body() click to toggle source

@return [Array<Hash<Symbol => String>>] All of the results sorted by

the :sort_column option. (Defaults to :avg).
# File lib/minitest/reporters/mean_time_reporter.rb, line 173
def column_sorted_body
  runs = options[:show_all_runs] ? previous_run : current_run
  runs.keys.each_with_object([]) do |description, obj|
    timings = previous_run[description]
    size = Array(timings).size
    sum  = Array(timings).inject { |total, x| total + x }
    obj << {
      avg:  (sum / size).round(9),
      min:  Array(timings).min.round(9),
      max:  Array(timings).max.round(9),
      last: Array(timings).last.round(9),
      desc: description,
    }
  end.sort_by { |k| k[sort_column] }
end
create_new_report!() click to toggle source

Creates a new report file in the 'report_filename'. This file contains a line for each test of the following example format: (this is a single line despite explicit wrapping)

Avg: 0.0555555 Min: 0.0498765 Max: 0.0612345 Last: 0.0499421 Description: The test name

Note however the timings are to 9 decimal places, and padded to 12 characters and each label is coloured, Avg (yellow), Min (green), Max (red), Last (multi), and Description (blue). It looks pretty!

The 'Last' label is special in that it will be colour coded depending on whether the last run was faster (bright green) or slower (bright red) or inconclusive (purple). This helps to identify changes on a per run basis.

@return [void]

# File lib/minitest/reporters/mean_time_reporter.rb, line 289
def create_new_report!
  File.write(report_filename, report_title + report_body)
end
create_or_update_previous_runs!() click to toggle source

Creates a new 'previous runs' file, or updates the existing one with the latest timings.

@return [void]

# File lib/minitest/reporters/mean_time_reporter.rb, line 249
def create_or_update_previous_runs!
  if previously_ran?
    current_run.each do |description, elapsed|
    new_times = if previous_run["#{description}"]
                  Array(previous_run["#{description}"]) << elapsed

                else
                  Array(elapsed)

                end

      previous_run.store("#{description}", new_times)
    end

    File.write(previous_runs_filename, previous_run.to_yaml)

  else

    File.write(previous_runs_filename, current_run.to_yaml)

  end
end
current_run() click to toggle source

@return [Hash<String => Float>]

# File lib/minitest/reporters/mean_time_reporter.rb, line 114
def current_run
  Hash[all_suite_times]
end
defaults() click to toggle source

@return [Hash] Sets default values for the filenames used by this class,

and the number of tests to output to output to the screen after each
run.
# File lib/minitest/reporters/mean_time_reporter.rb, line 121
def defaults
  {
    order:                  :desc,
    show_count:             15,
    show_progress:          true,
    show_all_runs:          true,
    sort_column:            :avg,
    previous_runs_filename: '/tmp/minitest_reporters_previous_run',
    report_filename:        '/tmp/minitest_reporters_report',
  }
end
des_label() click to toggle source

@return [String] A blue 'Description:' label.

# File lib/minitest/reporters/mean_time_reporter.rb, line 309
def des_label
  ANSI::Code.blue('Description:')
end
desc?() click to toggle source

@return [Boolean] Whether the given :order option is :desc (default).

# File lib/minitest/reporters/mean_time_reporter.rb, line 354
def desc?
  order == :desc
end
max_label() click to toggle source

@return [String] A red 'Max:' label.

# File lib/minitest/reporters/mean_time_reporter.rb, line 314
def max_label
  ANSI::Code.red('Max:')
end
min_label() click to toggle source

@return [String] A green 'Min:' label.

# File lib/minitest/reporters/mean_time_reporter.rb, line 319
def min_label
  ANSI::Code.green('Min:')
end
options() click to toggle source

@return [Hash]

# File lib/minitest/reporters/mean_time_reporter.rb, line 190
def options
  defaults.merge!(@options)
end
order() click to toggle source

@raise [Minitest::Reporters::MeanTimeReporter::InvalidOrder]

When the given :order option is invalid.

@return [Symbol] The :order option, or by default; :desc.

# File lib/minitest/reporters/mean_time_reporter.rb, line 361
def order
  orders = [:desc, :asc]

  if orders.include?(options[:order])
    options[:order]

  else
    fail Minitest::Reporters::MeanTimeReporter::InvalidOrder,
         "`:order` option must be one of #{orders.inspect}."

  end
end
order_sorted_body() click to toggle source

@return [String] All of the column-sorted results sorted by the :order

option. (Defaults to :desc).
# File lib/minitest/reporters/mean_time_reporter.rb, line 161
def order_sorted_body
  if desc?
    column_sorted_body.reverse

  elsif asc?
    column_sorted_body

  end
end
previous_run() click to toggle source

@return [Hash<String => Array<Float>]

# File lib/minitest/reporters/mean_time_reporter.rb, line 201
def previous_run
  @previous_run ||= YAML.load_file(previous_runs_filename)
end
previous_runs_filename() click to toggle source

@return [String] The path to the file which contains all the durations

for each test run. The previous runs file is in YAML format, using the
test name for the key and an array containing the time taken to run
this test for values.
# File lib/minitest/reporters/mean_time_reporter.rb, line 209
def previous_runs_filename
  options[:previous_runs_filename]
end
previously_ran?() click to toggle source

Returns a boolean indicating whether a previous runs file exists.

@return [Boolean]

# File lib/minitest/reporters/mean_time_reporter.rb, line 216
def previously_ran?
  File.exist?(previous_runs_filename)
end
rate(run, min, max) click to toggle source

@param run [Float] The last run time. @param min [Float] The minimum run time. @param max [Float] The maximum run time. @return [Symbol] One of :faster, :slower or :inconclusive.

# File lib/minitest/reporters/mean_time_reporter.rb, line 338
def rate(run, min, max)
  if run == min
    :faster
  elsif run == max
    :slower
  else
    :inconclusive
  end
end
report_body() click to toggle source

The report itself. Displays statistics about all runs, ideal for use with the Unix 'head' command. Listed in slowest average descending order.

@return [String]

# File lib/minitest/reporters/mean_time_reporter.rb, line 147
def report_body
  order_sorted_body.each_with_object([]) do |result, obj|
    rating = rate(result[:last], result[:min], result[:max])

    obj << "#{avg_label} #{result[:avg].to_s.ljust(12)} " \
           "#{min_label} #{result[:min].to_s.ljust(12)} " \
           "#{max_label} #{result[:max].to_s.ljust(12)} " \
           "#{run_label(rating)} #{result[:last].to_s.ljust(12)} " \
           "#{des_label} #{result[:desc]}\n"
  end.join
end
report_filename() click to toggle source

@return [String] The path to the file which contains the parsed test

results. The results file contains a line for each test with the
average time of the test, the minimum time the test took to run,
the maximum time the test took to run and a description of the test
(which is the test name as emitted by Minitest).
# File lib/minitest/reporters/mean_time_reporter.rb, line 225
def report_filename
  options[:report_filename]
end
report_title() click to toggle source

Added to the top of the report file and to the screen output.

@return [String]

# File lib/minitest/reporters/mean_time_reporter.rb, line 136
def report_title
  "\n\e[4mMinitest Reporters: Mean Time Report\e[24m " \
  "(Samples: #{samples}, Order: #{sort_column.inspect} " \
  "#{order.inspect})\n"
end
run_label(rating) click to toggle source

@param rating [Symbol] One of :faster, :slower or :inconclusive. @return [String] A purple 'Last:' label.

# File lib/minitest/reporters/mean_time_reporter.rb, line 325
def run_label(rating)
  case rating
  when :faster then ANSI::Code.green('Last:')
  when :slower then ANSI::Code.red('Last:')
  else
    ANSI::Code.magenta('Last:')
  end
end
samples() click to toggle source

A barbaric way to find out how many runs are in the previous runs file; this method takes the first test listed, and counts its samples trusting (naively) all runs to be the same number of samples. This will produce incorrect averages when new tests are added, so it is advised to restart the statistics by removing the 'previous runs' file. A rake task is provided to make this more convenient.

rake reset_statistics

@return [Fixnum]

# File lib/minitest/reporters/mean_time_reporter.rb, line 239
def samples
  return 1 unless previous_run.first[1].is_a?(Array)

  previous_run.first[1].size
end
show_count() click to toggle source

@return [Fixnum] The number of tests to output to output to the screen

after each run.
# File lib/minitest/reporters/mean_time_reporter.rb, line 196
def show_count
  options[:show_count]
end
sort_column() click to toggle source

@raise [Minitest::Reporters::MeanTimeReporter::InvalidSortColumn]

When the given :sort_column option is invalid.

@return [Symbol] The :sort_column option, or by default; :avg.

# File lib/minitest/reporters/mean_time_reporter.rb, line 377
def sort_column
  sort_columns = [:avg, :min, :max, :last]

  if sort_columns.include?(options[:sort_column])
    options[:sort_column]

  else
    fail Minitest::Reporters::MeanTimeReporter::InvalidSortColumn,
         "`:sort_column` option must be one of #{sort_columns.inspect}."

  end
end
write_to_screen!() click to toggle source

Writes a number of tests (configured via the 'show_count' option) to the screen after creating the report. See '#create_new_report!' for example output information.

@return [void]

# File lib/minitest/reporters/mean_time_reporter.rb, line 298
def write_to_screen!
  puts report_title
  puts report_body.lines.take(show_count)
end