ios – Does XCTest strategies generated dynamically by testInvocations work with xcodebuild’s -only-testing?

Spread the love


I’ve an app which must dynamically generate take a look at strategies in its XCTestCase subclass. I am utilizing +(NSArray<NSInvocation*>*)testInvocations to dynamically generate take a look at strategies at runtime with the next (dummy) names: example_test, permissions_location_test and permissions_many_test. Beneath is a simplified, ready-to-be-pasted-into-Xcode code.

@import XCTest;
@import ObjectiveC.runtime;

@interface ParametrizedTests : XCTestCase
@finish

@implementation ParametrizedTests
+ (NSArray<NSInvocation *> *)testInvocations {
  NSLog(@"testInvocations() known as");

  /* Put together dummy enter */
  __block NSMutableArray<NSString *> *dartTestFiles = [[NSMutableArray alloc] init];
  [dartTestFiles addObject:@"example_test"];
  [dartTestFiles addObject:@"permissions_location_test"];
  [dartTestFiles addObject:@"permissions_many_test"];

  NSMutableArray<NSInvocation *> *invocations = [[NSMutableArray alloc] init];

  NSLog(@"Earlier than the loop, %lu components within the array", (unsigned lengthy)dartTestFiles.depend);

  for (int i = 0; i < dartTestFiles.depend; i++) {
    /* Step 1 */

    NSString *title = dartTestFiles[i];

    void (^anonymousFunc)(ParametrizedTests *) = ^(ParametrizedTests *occasion) {
      NSLog(@"anonymousFunc known as!");
    };

    IMP implementation = imp_implementationWithBlock(anonymousFunc);
    NSString *selectorStr = [NSString stringWithFormat:@"test_%@", name];
    SEL selector = NSSelectorFromString(selectorStr);
    class_addMethod(self, selector, implementation, "v@:");

    /* Step 2 */

    NSMethodSignature *signature = [self instanceMethodSignatureForSelector:selector];
    NSInvocation *invocation = [NSInvocation invocationWithMethodSignature:signature];
    invocation.selector = selector;

    NSLog(@"RunnerUITests.testInvocations(): selectorStr = %@", selectorStr);

    [invocations addObject:invocation];
  }

  NSLog(@"After the loop");

  return invocations;
}

@finish

I can run all these exams directly utilizing:

xcodebuild take a look at 
  -scheme Landmarks 
  -destination 'platform=iOS Simulator,title=iPhone 15'

Excerpt from above command’s stdout:

Take a look at Suite 'Chosen exams' handed at 2023-11-16 13:44:59.148.
     Executed 3 exams, with 0 failures (0 surprising) in 0.246 (0.248) seconds

Drawback

The issue I am going through now could be that I can’t choose only one take a look at to run utilizing xcodebuild‘s -only-testing flag. For instance:

xcodebuild take a look at 
  -scheme Landmarks 
  -destination 'platform=iOS Simulator,title=iPhone 15' 
  -only-testing 'LandmarksUITests/ParametrizedTests/example_test'

doesn’t work – no exams are executed:

Take a look at Suite 'ParametrizedTests' handed at 2023-11-16 13:45:58.472.
     Executed 0 exams, with 0 failures (0 surprising) in 0.000 (0.000) seconds

I additionally tried doing:

xcodebuild take a look at 
  -scheme Landmarks 
  -destination 'platform=iOS Simulator,title=iPhone 15' 
  -only-testing 'LandmarksUITests/ParametrizedTests/testInvocations'

however the outcome is identical.

So the query is: how can I choose a subset of exams (that had been generated dynamically at runtime utilizing testInvocations) with the -only-testing choice?. Is is even doable?

Leave a Reply

Your email address will not be published. Required fields are marked *