Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch from running E2E tests against preview deployments to running them against local instance in the workflow (?) #1275

Open
olayway opened this issue Aug 14, 2024 · 0 comments

Comments

@olayway
Copy link
Member

olayway commented Aug 14, 2024

Before running E2E tests in the GitHub workflow, Playwright runs our custom global.setup.ts file. It's purpose is to sync the test site before running tests against it. So the steps are:

  1. wait for the branch preview deployment on Vercel to complete
  2. sync the site (necessary if there has been any changes made to it and our tests depend on those changes) - will update R2 staging bucket and the staging db
  3. run E2E tests against the preview deployment

There are two problems that can cause E2E tests to fail:

  • the site sync performed in step 2 will not use the branch version of the inngest sync workflow, but rather the staging version of it; this means if you've made some changes to the workflow itself and you want to run your tests against a site that has been synced with an updated workflow - it won't work
  • even if the above worked, there is a potential issue related to the site being already "in-sync" with the latest github changes (e.g. some else synced the site, manually or through the E2E workflow on their own PR); this means even if you run your version of the sync workflow, it would return immediately, because the hash of the tree stored in R2 and of the latest one pulled from GitHub would be the same.

It should be possible to solve the first problem with setting branch environments in Inngest.

@olayway olayway self-assigned this Aug 14, 2024
@olayway olayway removed their assignment Aug 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 💤 Someday
Development

No branches or pull requests

1 participant