Class: Datadog::CI::Test

Inherits:
Span
  • Object
show all
Defined in:
lib/datadog/ci/test.rb,
sig/datadog/ci/test.rbs

Overview

Represents a single part of a test run.

Constant Summary

Constants inherited from Span

Span::EMPTY_TAGS, Span::ENVIRONMENT_RUNTIME_TAGS

Instance Attribute Summary collapse

Attributes inherited from Span

#tracer_span

Instance Method Summary collapse

Methods inherited from Span

#base_commit_sha, #clear_internal_tag, #clear_tag, #failed?, #get_metric, #get_tag, #git_branch, #git_commit_message, #git_commit_sha, #git_repository_url, #git_tag, #id, #original_git_commit_message, #original_git_commit_sha, #os_architecture, #os_platform, #os_version, #passed?, #runtime_name, #runtime_version, #service, #set_default_tags, #set_environment_runtime_tags, #set_internal_metric, #set_internal_tag, #set_internal_tags, #set_metric, #set_tag, #set_tags, #shared_tags, #skipped?, #source_file, #status, #test_tracing, #to_s, #trace_id, #type, #undefined?

Constructor Details

#initialize(tracer_span) ⇒ Test

Returns a new instance of Test.

Parameters:

  • tracer_span (Datadog::Tracing::SpanOperation) 


15
16
17
18
19
20
 
# File 'lib/datadog/ci/test.rb', line 15

def initialize(tracer_span)
  super

  @custom_impacted_files = []
  @inherited_custom_impacted_files = []
end

Instance Attribute Details

#context_ids ⇒ Object

Context IDs for this test (used for TIA context coverage merging)



25
26
27
 
# File 'lib/datadog/ci/test.rb', line 25

def context_ids
  @context_ids
end

Instance Method Details

#add_impacted_files(file_paths) ⇒ void

This method returns an undefined value.

Adds custom files that Test Impact Analysis should consider capable of impacting this test.

This is useful for files that Datadog's native Ruby coverage cannot observe, such as JavaScript executed in a browser during a Capybara test. Paths must resolve inside the Git repository. Relative paths must be relative to the repository root. Absolute paths are also accepted.

If paths are relative to the current working directory, convert them to absolute paths with +File.expand_path+ before calling this method. Submitted paths must be lexically normalized without redundant +.+ or +..+ components.

Path resolution does not access the filesystem, so files do not need to exist when they are added. Paths outside the repository are ignored when the coverage event is serialized. Calls are incremental: every call adds files for the remainder of the test.

Examples:

Register JavaScript files loaded during an RSpec test

RSpec.configure do |config|
  config.after do
    Datadog::CI.active_test&.add_impacted_files(
      javascript_files_loaded_by_browser.map do |path|
        File.expand_path(path, Dir.pwd)
      end
    )
  end
end

Parameters:

  • file_paths (Array<String>) —

    custom paths that can impact this test

Raises:

  • (ArgumentError) —

    if file_paths is not an Array



242
243
244
245
246
247
 
# File 'lib/datadog/ci/test.rb', line 242

def add_impacted_files(file_paths)
  raise ArgumentError, "file_paths must be an Array" unless file_paths.is_a?(Array)

  @custom_impacted_files.concat(file_paths)
  nil
end

#all_executions_failed? ⇒ Boolean

Returns:

  • (Boolean) 


323
324
325
 
# File 'lib/datadog/ci/test.rb', line 323

def all_executions_failed?
  !!test_suite&.all_executions_failed?(local_test_id)
end

#all_executions_passed? ⇒ Boolean

Returns:

  • (Boolean) 


328
329
330
 
# File 'lib/datadog/ci/test.rb', line 328

def all_executions_passed?
  !!test_suite&.all_executions_passed?(local_test_id)
end

#any_retry_passed? ⇒ Boolean

Returns:

  • (Boolean) 


318
319
320
 
# File 'lib/datadog/ci/test.rb', line 318

def any_retry_passed?
  !!test_suite&.any_test_retry_passed?(local_test_id)
end

#apply_automation_exclusions(prevent_failure_suppression:, no_auto_retry:) ⇒ Object

Records execution exclusions from the framework's test declaration. Replaces any user-provided initial tags before execution policy is evaluated.



142
143
144
145
146
147
148
149
150
151
152
153
154
 
# File 'lib/datadog/ci/test.rb', line 142

def apply_automation_exclusions(prevent_failure_suppression:, no_auto_retry:)
  if prevent_failure_suppression
    set_internal_tag(Ext::Test::TAG_PREVENT_FAILURE_SUPPRESSION, "true")
  else
    clear_internal_tag(Ext::Test::TAG_PREVENT_FAILURE_SUPPRESSION)
  end

  if no_auto_retry
    set_internal_tag(Ext::Test::TAG_NO_AUTO_RETRY, "true")
  else
    clear_internal_tag(Ext::Test::TAG_NO_AUTO_RETRY)
  end
end

#attempt_to_fix? ⇒ Boolean

Returns "true" if this flaky test has fixing attempts (determined by Datadog backend).

Returns:

  • (Boolean) —

    true if this test is attempted to be fixed.



135
136
137
 
# File 'lib/datadog/ci/test.rb', line 135

def attempt_to_fix?
  get_tag(Ext::Test::TAG_IS_ATTEMPT_TO_FIX) == "true"
end

#compute_final_status(status) ⇒ String

Parameters:

  • status (String)

Returns:

  • (String) 


394
395
396
397
398
399
400
401
402
403
404
405
406
407
 
# File 'lib/datadog/ci/test.rb', line 394

def compute_final_status(status)
  # Skip status is always preserved
  return status if status == Ext::Test::Status::SKIP

  # attempt_to_fix takes precedence over quarantine/disabled flags:
  # any failure across attempts means the fix didn't work.
  if should_retry_attempt_to_fix?
    return all_executions_passed? ? Ext::Test::Status::PASS : Ext::Test::Status::FAIL
  end

  return status if status == Ext::Test::Status::PASS

  should_ignore_failures? ? Ext::Test::Status::PASS : Ext::Test::Status::FAIL
end

#datadog_skip_reason ⇒ String?

Returns:

  • (String, nil) 


333
334
335
336
337
338
339
 
# File 'lib/datadog/ci/test.rb', line 333

def datadog_skip_reason
  if skipped_by_test_impact_analysis?
    Ext::Test::SkipReason::TEST_IMPACT_ANALYSIS
  elsif should_disable? || should_quarantine?
    Ext::Test::SkipReason::TEST_MANAGEMENT_DISABLED
  end
end

#datadog_test_id ⇒ String

Returns the test id according to Datadog's test impact analysis.

Returns:

  • (String) —

    the test id according to Datadog's test impact analysis.



33
34
35
 
# File 'lib/datadog/ci/test.rb', line 33

def datadog_test_id
  @datadog_test_id ||= Utils::TestRun.datadog_test_id(name, test_suite_name, parameters)
end

#disabled? ⇒ Boolean

Returns "true" if this test is disabled by Datadog test management.

Returns:

  • (Boolean) —

    true if this test is disabled, false otherwise.



129
130
131
 
# File 'lib/datadog/ci/test.rb', line 129

def disabled?
  get_tag(Ext::Test::TAG_IS_TEST_DISABLED) == "true"
end

#end_line ⇒ Integer?

Returns the ending line number of the test in the source file.

Returns:

  • (Integer) —

    the ending line number

  • (nil) —

    if the ending line is not available



98
99
100
101
 
# File 'lib/datadog/ci/test.rb', line 98

def end_line
  line = get_tag(Ext::Test::TAG_SOURCE_END)
  line&.to_i
end

#failed!(exception: nil) ⇒ void

This method returns an undefined value.

Sets the status of the span to "fail".

Parameters:

  • exception (Exception) (defaults to: nil) —

    the exception that caused the test to fail.



281
282
283
284
285
 
# File 'lib/datadog/ci/test.rb', line 281

def failed!(exception: nil)
  super

  record_test_result(Ext::Test::Status::FAIL)
end

#finish ⇒ void

This method returns an undefined value.

Finishes the current test.



39
40
41
42
43
44
45
46
47
 
# File 'lib/datadog/ci/test.rb', line 39

def finish
  if is_retry? && retry_reason.nil?
    set_internal_tag(Ext::Test::TAG_RETRY_REASON, Ext::Test::RetryReason::RETRY_EXTERNAL)
  end

  test_tracing.deactivate_test

  super
end

#inherit_impacted_files(custom_impacted_files) ⇒ void

This method returns an undefined value.

Inherits suite-level custom impacted files without copying them.



253
254
255
256
 
# File 'lib/datadog/ci/test.rb', line 253

def inherit_impacted_files(custom_impacted_files)
  @inherited_custom_impacted_files = custom_impacted_files
  nil
end

#is_new? ⇒ Boolean

Returns "true" if this span represents a test that wasn't known to Datadog before.

Returns:

  • (Boolean) —

    true if this test is a new one, false otherwise.



117
118
119
 
# File 'lib/datadog/ci/test.rb', line 117

def is_new?
  get_tag(Ext::Test::TAG_IS_NEW) == "true"
end

#is_retry? ⇒ Boolean

Returns "true" if test span represents a retry.

Returns:

  • (Boolean) —

    true if this test is a retry, false otherwise.



105
106
107
 
# File 'lib/datadog/ci/test.rb', line 105

def is_retry?
  get_tag(Ext::Test::TAG_IS_RETRY) == "true"
end

#itr_unskippable! ⇒ void

This method returns an undefined value.

Marks this test as unskippable by the Test Impact Analysis. This must be done before the test execution starts.

Examples of tests that should be unskippable:

  • tests that read files from disk
  • tests that make network requests
  • tests that call external processes
  • tests that use forking


198
199
200
201
202
203
204
205
206
207
208
 
# File 'lib/datadog/ci/test.rb', line 198

def itr_unskippable!
  Utils::Telemetry.itr_unskippable
  set_internal_tag(Ext::Test::TAG_ITR_UNSKIPPABLE, "true")

  if skipped_by_test_impact_analysis?
    clear_internal_tag(Ext::Test::TAG_ITR_SKIPPED_BY_ITR)

    Utils::Telemetry.itr_forced_run
    set_internal_tag(Ext::Test::TAG_ITR_FORCED_RUN, "true")
  end
end

#local_test_id ⇒ Object

Distinct declarations can share the backend ID but must not share retry history.



381
382
383
384
385
386
387
388
389
390
391
392
 
# File 'lib/datadog/ci/test.rb', line 381

def local_test_id
  @local_test_id ||= begin
    source_file = get_tag(Ext::Test::TAG_SOURCE_FILE)
    source_start = get_tag(Ext::Test::TAG_SOURCE_START)

    if source_file && source_start
      [datadog_test_id, source_file, source_start]
    else
      datadog_test_id
    end
  end
end

#lock_custom_impacted_files ⇒ Array<String>

Locks custom impacted files against further changes and returns them to Test Impact Analysis.

Returns:

  • (Array<String>) 


263
264
265
266
267
268
 
# File 'lib/datadog/ci/test.rb', line 263

def lock_custom_impacted_files
  return @custom_impacted_files if @custom_impacted_files.frozen?

  @custom_impacted_files =
    (@inherited_custom_impacted_files | @custom_impacted_files).freeze
end

#modified? ⇒ Boolean

Returns "true" if this test is marked as modified (e.g., impacted by code changes).

Returns:

  • (Boolean) —

    true if this test is modified, false otherwise.



184
185
186
 
# File 'lib/datadog/ci/test.rb', line 184

def modified?
  get_tag(Ext::Test::TAG_TEST_IS_MODIFIED) == "true"
end

#name ⇒ String

Returns the name of the test.

Returns:

  • (String) —

    the name of the test.



28
29
30
 
# File 'lib/datadog/ci/test.rb', line 28

def name
  get_tag(Ext::Test::TAG_NAME)
end

#no_auto_retry? ⇒ Boolean

Returns:

  • (Boolean) 


162
163
164
 
# File 'lib/datadog/ci/test.rb', line 162

def no_auto_retry?
  get_tag(Ext::Test::TAG_NO_AUTO_RETRY) == "true"
end

#parameters ⇒ String?

Gets the parameters for this test (e.g. Cucumber example or RSpec specs) as a serialized JSON.

Returns:

  • (String) —

    the serialized JSON of the parameters

  • (nil) —

    if this test does not have parameters



313
314
315
 
# File 'lib/datadog/ci/test.rb', line 313

def parameters
  get_tag(Ext::Test::TAG_PARAMETERS)
end

#passed! ⇒ void

This method returns an undefined value.

Sets the status of the span to "pass".



272
273
274
275
276
 
# File 'lib/datadog/ci/test.rb', line 272

def passed!
  super

  record_test_result(Ext::Test::Status::PASS)
end

#peek_duration ⇒ Float

Returns:

  • (Float) 


370
371
372
373
374
375
376
 
# File 'lib/datadog/ci/test.rb', line 370

def peek_duration
  end_time = Core::Utils::Time.now.utc
  start_time = tracer_span.start_time

  return 0.0 if start_time.nil? || end_time.nil?
  end_time - start_time
end

#prevent_failure_suppression? ⇒ Boolean

Returns:

  • (Boolean) 


157
158
159
 
# File 'lib/datadog/ci/test.rb', line 157

def prevent_failure_suppression?
  get_tag(Ext::Test::TAG_PREVENT_FAILURE_SUPPRESSION) == "true"
end

#quarantined? ⇒ Boolean

Returns "true" if this test is quarantined by Datadog test management.

Returns:

  • (Boolean) —

    true if this test is quarantined, false otherwise.



123
124
125
 
# File 'lib/datadog/ci/test.rb', line 123

def quarantined?
  get_tag(Ext::Test::TAG_IS_QUARANTINED) == "true"
end

#record_final_status ⇒ void

This method returns an undefined value.



360
361
362
363
364
365
366
367
 
# File 'lib/datadog/ci/test.rb', line 360

def record_final_status
  status = get_tag(Ext::Test::TAG_STATUS)
  return if status.nil?

  final_status = compute_final_status(status)
  set_internal_tag(Ext::Test::TAG_FINAL_STATUS, final_status)
  test_suite&.record_test_final_status(local_test_id, final_status)
end

#record_test_result(datadog_status) ⇒ void

This method returns an undefined value.

Parameters:

  • datadog_status (String) 


409
410
411
412
413
414
415
416
 
# File 'lib/datadog/ci/test.rb', line 409

def record_test_result(datadog_status)
  # if this test was already executed in this test suite, mark it as retried
  if test_suite&.test_executed?(local_test_id)
    set_internal_tag(Ext::Test::TAG_IS_RETRY, "true")
  end

  test_suite&.record_test_result(local_test_id, datadog_status)
end

#retry_reason ⇒ String

Returns string with a reason why test was retried

Returns:

  • (String) —

    retry reason



111
112
113
 
# File 'lib/datadog/ci/test.rb', line 111

def retry_reason
  get_tag(Ext::Test::TAG_RETRY_REASON)
end

#set_parameters(arguments, metadata = {}) ⇒ void

This method returns an undefined value.

Sets the parameters for this test (e.g. Cucumber example or RSpec specs). Parameters are needed to compute test fingerprint to distinguish between different tests having same names.

Parameters:

  • arguments (Hash) —

    the arguments that test accepts as key-value hash

  • metadata (Hash) (defaults to: {}) —

    optional metadata



303
304
305
306
307
 
# File 'lib/datadog/ci/test.rb', line 303

def set_parameters(arguments, metadata = {})
  return if arguments.nil?

  set_internal_tag(Ext::Test::TAG_PARAMETERS, Utils::TestRun.test_parameters(arguments: arguments, metadata: metadata))
end

#should_disable? ⇒ Boolean

Returns:

  • (Boolean) 


173
174
175
 
# File 'lib/datadog/ci/test.rb', line 173

def should_disable?
  disabled? && !prevent_failure_suppression?
end

#should_ignore_failures? ⇒ Boolean

Returns:

  • (Boolean) 


347
348
349
350
351
352
 
# File 'lib/datadog/ci/test.rb', line 347

def should_ignore_failures?
  return false if should_retry_attempt_to_fix?
  return true if should_quarantine? || should_disable?

  any_retry_passed?
end

#should_quarantine? ⇒ Object

Assigned properties stay observable independently of effective policy.



168
169
170
 
# File 'lib/datadog/ci/test.rb', line 168

def should_quarantine?
  quarantined? && !prevent_failure_suppression?
end

#should_retry_attempt_to_fix? ⇒ Boolean

Returns:

  • (Boolean) 


178
179
180
 
# File 'lib/datadog/ci/test.rb', line 178

def should_retry_attempt_to_fix?
  attempt_to_fix? && !no_auto_retry?
end

#should_skip? ⇒ Boolean

Returns:

  • (Boolean) 


342
343
344
 
# File 'lib/datadog/ci/test.rb', line 342

def should_skip?
  skipped_by_test_impact_analysis? || (should_disable? && !should_retry_attempt_to_fix?)
end

#skipped!(exception: nil, reason: nil) ⇒ void

This method returns an undefined value.

Sets the status of the span to "skip".

Parameters:

  • exception (Exception) (defaults to: nil) —

    the exception that caused the test to fail.

  • reason (String) (defaults to: nil) —

    the reason why the test was skipped.



291
292
293
294
295
 
# File 'lib/datadog/ci/test.rb', line 291

def skipped!(exception: nil, reason: nil)
  super

  record_test_result(Ext::Test::Status::SKIP)
end

#skipped_by_test_impact_analysis? ⇒ Boolean

Returns:

  • (Boolean) 


355
356
357
 
# File 'lib/datadog/ci/test.rb', line 355

def skipped_by_test_impact_analysis?
  get_tag(Ext::Test::TAG_ITR_SKIPPED_BY_ITR) == "true"
end

#start_line ⇒ Integer?

Returns the starting line number of the test in the source file.

Returns:

  • (Integer) —

    the starting line number

  • (nil) —

    if the starting line is not available



90
91
92
93
 
# File 'lib/datadog/ci/test.rb', line 90

def start_line
  line = get_tag(Ext::Test::TAG_SOURCE_START)
  line&.to_i
end

#test_module_id ⇒ String

Span id of the running test module this test belongs to.

Returns:

  • (String) —

    the span id of the test module.



71
72
73
 
# File 'lib/datadog/ci/test.rb', line 71

def test_module_id
  get_tag(Ext::Test::TAG_TEST_MODULE_ID)
end

#test_module_name ⇒ String

Name of the running test module this test belongs to.

Returns:

  • (String) —

    the name of the test module.



77
78
79
 
# File 'lib/datadog/ci/test.rb', line 77

def test_module_name
  get_tag(Ext::Test::TAG_MODULE)
end

#test_session_id ⇒ String

Span id of the running test session this test belongs to.

Returns:

  • (String) —

    the span id of the test session.



83
84
85
 
# File 'lib/datadog/ci/test.rb', line 83

def test_session_id
  get_tag(Ext::Test::TAG_TEST_SESSION_ID)
end

#test_suite ⇒ Datadog::CI::TestSuite?

Running test suite that this test is part of (if any).

Returns:



52
53
54
55
 
# File 'lib/datadog/ci/test.rb', line 52

def test_suite
  suite_name = test_suite_name
  CI.active_test_suite(suite_name) if suite_name
end

#test_suite_id ⇒ String

Span id of the running test suite this test belongs to.

Returns:

  • (String) —

    the span id of the test suite.



59
60
61
 
# File 'lib/datadog/ci/test.rb', line 59

def test_suite_id
  get_tag(Ext::Test::TAG_TEST_SUITE_ID)
end

#test_suite_name ⇒ String

Name of the running test suite this test belongs to.

Returns:

  • (String) —

    the name of the test suite.



65
66
67
 
# File 'lib/datadog/ci/test.rb', line 65

def test_suite_name
  get_tag(Ext::Test::TAG_SUITE)
end