trackslash
VAULT-88 P2

Add a UI test target with launch tests

0
Sub-issue of VAULT-83 P2 Test that lock, unlock, encryption and restore work end to end

Description

Part of VAULT-83.

Why

Vault has no UI tests. VaultApp.xcodeproj has only three targets: the app and its two extensions. Snapshot tests render views in isolation, so nothing checks that the real app launches, shows the vault and responds to taps. There's also some leftover scaffolding:

  • Vault/.swiftpm/.../VaultUITests.xcscheme points at a VaultUITests target that doesn't exist.
  • CI_iOS.xcscheme refers to a CI/ios_alltests.xctestplan and a CryptoDocumentExporterTests target that don't exist. It also has a wrong path to macOS_SupportedTests.xctestplan.

Plan

  • Add a VaultAppUITests XCUITest target to VaultApp.xcodeproj. Swift packages can't host UI tests.

  • Run it from make validate. Either:

    • add it to iOSAllTests, which means CI_iOS has to build the app; or
    • give it its own test plan and its own ios.buildForTesting / testWithoutBuilding check in local-check.config.ts.

    Choose whichever keeps validate reliable. The UI tests don't need the TSAN configuration.

  • Delete the stale VaultUITests.xcscheme, and fix or remove the broken references in CI_iOS.xcscheme.

  • Find elements by accessibility identifier, not by localised text. Add identifiers where they're missing.

  • Add a line to AGENTS.md on how to run only the UI tests.

Tests

Both tests launch with -screenshot-scene feed. That flag gives an in-memory demo vault, with App Lock off, that never touches the simulator's real vault.

  1. Launch smoke test:
    • The app launches to the feed, and the demo codes are visible.
    • Open a code's detail and go back.
    • Open Settings and go back.
    • Background the app (XCUIDevice.shared.press(.home)), bring it back, and it's still on the feed.
  2. Encrypted note:
    • Open the demo vault's encrypted note, "Recovery codes" (password "hello").
    • A wrong password shows the error and keeps the contents hidden.
    • The right password shows the contents.
    • Close the note and open it again: it's locked again.

Done when

  • Both tests pass in make validate.
  • The stale schemes are gone.
  • The whole UI test target adds well under a minute to validate.

Linked issues

1
Blocks VAULT-89 P2 UI test: launch locked, unlock, and lock again Done

GitHub

0

No branches or pull requests linked.

Comments

2
Bradley

PR: https://github.com/badbundle/vault-app/pull/676. It's validated, and make validate now builds and runs the UI tests too. It's waiting to be merged.

Notes:

  • Scheme: the UI tests have their own scheme (VaultAppUITests) and test plan, rather than being part of iOSAllTests. That plan runs everything twice, once under the thread sanitizer, and a package scheme can't cleanly point at a target in the app project. They share DerivedData with the unit tests, so only the app and the tests are rebuilt.
  • Timing: in validation the UI tests step took 47 s. The ticket asked for "well under a minute", so it's close rather than well under.
  • Code details: the launch test reaches a code's details through Settings → Tap a Code To → Show Details. Once a code's touch-and-hold menu is open, XCUITest never sees the app go idle, so the test can't use that menu.