Worker immediately respond with PASS

[DamianLion]

Hey Guys,

This is our current magellan setup:

"dependencies": {	
    "nightwatch": "0.9.15",
    "selenium-server": "3.4.0",
    "testarmada-magellan": "10.0.7",
    "testarmada-magellan-admiral2-plugin": "4.0.2",
    "testarmada-magellan-local-executor": "2.0.0",
    "testarmada-magellan-nightwatch-plugin": "6.0.1",
    "testarmada-magellan-saucelabs-executor": "1.0.5"
}

So we encounterd some issue that just appeared after we had an npm cache clean.
All of our test are immediately reported as successful even though they have not been executed on saucelabs yet. We see that a connection is established to sauce labs but no test commands get executed there, this results in a timeout after 90 seconds.

Even with debug mode enabled we were not able to analyze where the error occured.
Maybe you have an idea?

[INFO] [Magellan] Will try to load configuration from default of ./magellan.json
Loaded configuration from:  /user/E2E/magellan.json
[INFO] [Magellan] Magellan is creating temporary files at: /user/E2E/temp
[INFO] [Magellan] Magellan 10.0.7
[INFO] [Magellan] Loaded test framework:
[INFO] [Magellan]   testarmada-magellan-nightwatch-plugin
[INFO] [Magellan] Loaded test executors:
[INFO] [Magellan]   testarmada-magellan-local-executor
[INFO] [Magellan]   testarmada-magellan-sauce-executor
[INFO] [Magellan Nightwatch Plugin] test iterator found nightwatch configuration at: ./nightwatch.json
[INFO] [Magellan Nightwatch Plugin] Scanning tests/nightwatch/tests for test files ...
[INFO] [Magellan Nightwatch Plugin] Found 2 test files in tests/nightwatch/tests
[INFO] [Magellan] Requested profile(s): all
[WARN] [Saucelabs Executor] Warning! No connectVersion is set. This is set via the environment variable $SAUCE_CONNECT_VERSION . This isn't required, but can cause problems with Sauce if not set
[INFO] [Saucelabs Executor] Sauce configuration OK
[INFO] [Saucelabs Executor] Connected to sauce without tunnel
[INFO] [Magellan] Running 4 tests with 20 workers with [env:chrome_58_Windows_10_Desktop|executor:sauce, env:firefox_53_Windows_10_Desktop|executor:sauce]
[INFO] [Magellan] --> Worker 4, port range: [12009, 12011], running test: tests/nightwatch/tests/e2eWithApproval.js @env:firefox_53_Windows_10_Desktop|executor:sauce
[INFO] [Magellan] --> Worker 3, port range: [12006, 12008], running test: tests/nightwatch/tests/e2eWithApproval.js @env:chrome_58_Windows_10_Desktop|executor:sauce
[INFO] [Magellan] --> Worker 2, port range: [12003, 12005], running test: tests/nightwatch/tests/e2e.test.js @env:firefox_53_Windows_10_Desktop|executor:sauce
[INFO] [Magellan] --> Worker 1, port range: [12000, 12002], running test: tests/nightwatch/tests/e2e.test.js @env:chrome_58_Windows_10_Desktop|executor:sauce
[WARN] [Saucelabs Executor] No meta data is found, executor will not report result to saucelabs
[INFO] [Magellan] (1 / 4) <-- Worker 4 PASS  tests/nightwatch/tests/e2eWithApproval.js @env:firefox_53_Windows_10_Desktop|executor:sauce
[WARN] [Saucelabs Executor] No meta data is found, executor will not report result to saucelabs
[INFO] [Magellan] (2 / 4) <-- Worker 3 PASS  tests/nightwatch/tests/e2eWithApproval.js @env:chrome_58_Windows_10_Desktop|executor:sauce
[WARN] [Saucelabs Executor] No meta data is found, executor will not report result to saucelabs
[INFO] [Magellan] (3 / 4) <-- Worker 2 PASS  tests/nightwatch/tests/e2e.test.js @env:firefox_53_Windows_10_Desktop|executor:sauce
[WARN] [Saucelabs Executor] No meta data is found, executor will not report result to saucelabs
[INFO] [Magellan] (4 / 4) <-- Worker 1 PASS  tests/nightwatch/tests/e2e.test.js @env:chrome_58_Windows_10_Desktop|executor:sauce
[INFO] [Magellan] ============= Suite Complete =============
[INFO] [Magellan]      Status: PASSED
[INFO] [Magellan]     Runtime: 5s
[INFO] [Magellan] Total tests: 4
[INFO] [Magellan]  Successful: 4 / 4
[INFO] [Magellan] Checking for zombie processes...

Before the NPM Cache clean our infrastructure worked fine, so we assume that it might be possible that there is a module required by magellan which is unavailbe by now.

We tried also with older versions of all magellan modules but this problem persists.

Would be great if you can help us :)

[archlichking]

@DeltaLikeM My initial response for this is to create a base-test for your repo and inherit from https://github.com/TestArmada/nightwatch-extra/blob/master/src/base-test-class.js.

Example: https://github.com/TestArmada/boilerplate-nightwatch/blob/master/lib/example-base-test-class.js and https://github.com/TestArmada/boilerplate-nightwatch/blob/master/tests/demo-mobile-web.js#L1

But my second guess, which might not relate to the first response, is there is a js format issue in the test files. We've seen this fake pass before whose root cause can be highlighted by a lint.

[DamianLion]

Thank you very much for your answer

[vibskant007]

any updates on this?

[vibskant007]

I am also getting the same issue with magellan.. whats the solution?