reuse port range for new worker if possible

[archlichking]

this will minimize the need of opening new ports per worker

[michaeljacobdavis]

Currently

When delegating tests (and ports for those tests) to workers we start at BASE_PORT_START we increment by BASE_PORT_SPACING checking that those ports are available.

Once we hit the MAX_PORT we wrap around to BASE_PORT_START

Proposal

In addition to the current functionality, we can keep track of tests that were successfully allocated and released.

Once released, we can short circuit when get-ing a worker and reuse any previously successful port ranges that have been released.

Questions

• Would setting a smaller BASE_PORT_RANGE accomplish a similar thing?
• When reusing should we re-check the ports? or just assume they were open previously?

[archlichking]

Setting a smaller BASE_PORT_RANGE would accomplish a similar thing, but the port availability check will still move forward from current port to MAX_PORT. Always check forward is the fastest way to get a free port range as we can always assume the next port range is always available.

However, since there is a chance that zombie child process might happen from previous worker(s), we cannot have a hard limit of BASE_PORT_RANGE (which is ideally [port range per worker] * [worker amount]).

When re-checking the ports, we need to send http request to check them (or if there is a better way to handle this) as mentioned previously there might be zombie process still holding it.

One solution is to scan from BASE_PORT_START per port check. Usually when there is no zombie process, magellan releases ports by range per worker. So once the free port range is located, it can be assigned to next work directly. But when multiple workers are running, there is a chance this solution has more port check (more http requests to be sent) than the current solution.

[email2vimalraj]

Hi,

Did we test whether the magellan works on windows machine? I don't have logs to share as my machine got changed to Mac recently and unable to reproduce on Mac. But just few days before when I ran the nightwatch boilerplate, I saw connection refused error: 127.0.0.1:12000. I believe the 12000 is the default BASE_PORT_START.