Michael Achenbach
e3b750c335
This turns all paths in the test frameworks and all its suite implementations from strings into pathlib paths. This makes the framework more future-proof, simplifies a lot of code and enhances readability. It comes with the risk of introducing new bugs. This is lowered using an experimental test comparison script: https://crrev.com/c/4648372. Some detailed changes: - The "path" of a test case is now a pathlib path (also in cases where the path doesn't correspond to a location on the file system). This path is always relative to the test root. - The "name" of a test case is the posix string of that path used in status files and at the command line. - All other paths are absolute. - We use some more wrapping with Path() than absolutely necessary. Reducing such a wrapping where it's not needed is a micro-optimization and the potential by introducing a bug by forgetting one is worse. - Some modules are not happy with pathlib objects, hence a few explicit str() conversions are added. Thoughts on risk: - The highest risk is that due to the newly introduced logic, some test cases are silently filtered and not run now. We'll manually compare runs before and after this change. - Each test attempted to run will either pass or fail, failures won't be silent. E.g. a wrong path passed to the cctest executable is flagged as a failure. - Wrongly constructed paths to JS test cases lead to failures (tested several manually). Other failure types: - Some string path not turned into pathlib: Will typically fail when calling any pathlib function. - Some pathlib object treated as a string: Will typically raise an exception, e.g. using in-operator. - Logic bugs in pathlib functions behaving differently than anticipated, e.g. with_suffix(): Will typically result in wrong paths, resulting in errors reading a test case. Though worst case could be silent ignoring. Cq-Include-Trybots: luci.v8.try:v8_numfuzz_dbg Cq-Include-Trybots: luci.v8.try:v8_numfuzz_rel Cq-Include-Trybots: luci.v8.try:v8_numfuzz_tsan_rel Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel Cq-Include-Trybots: luci.v8.try:v8_android_arm64_n5x_rel Cq-Include-Trybots: luci.v8.try:v8_linux64_msan_rel Cq-Include-Trybots: luci.v8.try:v8_linux64_ubsan_rel Cq-Include-Trybots: luci.v8.try:v8_android_arm64_n5x_rel Bug: chromium:1132088 Change-Id: Ifc94feb292174935db969dcf5d174d2eedcf3dfd Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4538761 Reviewed-by: Liviu Rau <liviurau@google.com> Commit-Queue: Michael Achenbach <machenbach@chromium.org> Cr-Commit-Position: refs/heads/main@{#88502}
76 lines
2.1 KiB
Python
76 lines
2.1 KiB
Python
#!/usr/bin/env python3
|
|
# Copyright 2022 the V8 project authors. All rights reserved.
|
|
# Use of this source code is governed by a BSD-style license that can be
|
|
# found in the LICENSE file.
|
|
|
|
import os
|
|
import sys
|
|
import unittest
|
|
|
|
from pathlib import Path
|
|
|
|
# Needed because the test runner contains relative imports.
|
|
TOOLS_PATH = Path(__file__).resolve().parent.parent.parent
|
|
sys.path.append(str(TOOLS_PATH))
|
|
|
|
from testrunner.objects.testcase import TestCase
|
|
|
|
|
|
class TestCaseTest(unittest.TestCase):
|
|
|
|
def testSubtestsProperties(self):
|
|
test = TestCase(
|
|
suite=FakeSuite(),
|
|
path='far/away',
|
|
name='parent')
|
|
self.assertEqual(test.rdb_test_id, 'fakeSuite/parent')
|
|
# provide by DuckProcessor
|
|
self.assertEqual(test.processor.name, None)
|
|
self.assertEqual(test.procid, 'fakeSuite/parent')
|
|
self.assertEqual(test.keep_output, False)
|
|
|
|
subtest = test.create_subtest(FakeProcessor(), 0, keep_output=True)
|
|
self.assertEqual(subtest.rdb_test_id, 'fakeSuite/parent//fakep/0')
|
|
# provide by FakeProcessor
|
|
self.assertEqual(subtest.processor.name, 'fake_processor1')
|
|
self.assertEqual(subtest.procid, 'fakeSuite/parent.fake_processor1-0')
|
|
self.assertEqual(subtest.keep_output, True)
|
|
|
|
subsubtest = subtest.create_subtest(FakeProcessor(), 1)
|
|
self.assertEqual(subsubtest.rdb_test_id,
|
|
'fakeSuite/parent//fakep/0/fakep/1')
|
|
# provide by FakeProcessor
|
|
self.assertEqual(subsubtest.processor.name, 'fake_processor2')
|
|
self.assertEqual(subsubtest.procid,
|
|
'fakeSuite/parent.fake_processor1-0.fake_processor2-1')
|
|
self.assertEqual(subsubtest.keep_output, True)
|
|
|
|
|
|
class FakeSuite:
|
|
|
|
@property
|
|
def name(self):
|
|
return 'fakeSuite'
|
|
|
|
def statusfile_outcomes(self, name, variant):
|
|
return []
|
|
|
|
|
|
class FakeProcessor:
|
|
instance_count = 0
|
|
|
|
def __init__(self):
|
|
FakeProcessor.instance_count += 1
|
|
self.idx = FakeProcessor.instance_count
|
|
|
|
@property
|
|
def name(self):
|
|
return f'fake_processor{self.idx}'
|
|
|
|
def test_suffix(self, test):
|
|
return f'fakep/{test.subtest_id}'
|
|
|
|
|
|
if __name__ == '__main__':
|
|
unittest.main()
|