trackslash
VAULT-83 P2

Test that lock, unlock, encryption and restore work end to end

0
All issues

Description

Why

The storage and crypto layer is well tested piece by piece. The slot file, the encrypted store, duress slots, the attempt counter and Argon2 all have suites that use real crypto on temporary directories. What's thin is everything that joins those pieces together:

  • The composition root has no tests. No test references VaultiOS/VaultRoot.swift, so nothing covers the observers it registers, launch recovery or the erase path it wires up.
  • Lock and unlock are mostly tested against a fake. Most of that behaviour runs against FakeAppLockPasswordService. EncryptedVaultPasswordServiceTests (11 tests) is the only suite that runs over the real services.
  • Formats are pinned field by field, with no whole-file fixtures. No fixture from a shipped build exists. A change made on both the write side and the read side passes every round trip, and then can't open the vaults and backups people already have.
  • Restore has one round-trip suite. BackupRoundTripTests has 4 tests, with a random key and an in-memory store. The repo has no backups from older builds.
  • There are no UI tests. VaultApp.xcodeproj has no UI test target.

Plan

Sub-issues, one PR each:

  1. VAULT-84: App Lock: the lock paths through the real services.
  2. VAULT-85: Unlock: real passwords, erasing and launch recovery.
  3. VAULT-86: Encryption: golden fixtures from shipped builds.
  4. VAULT-87: Restore: a corpus of old backups, and end-to-end restores.
  5. VAULT-88: UI tests: a UI test target and launch tests.
  6. VAULT-89: UI tests: launch locked, unlock, lock again.

VAULT-84 to VAULT-88 don't depend on each other. VAULT-89 needs VAULT-88.

Ground rules for all of them:

  • Go through the real services, real crypto and real files wherever that's practical. Keep the cheap Argon2 parameters the tests use today, unless a ticket says otherwise.
  • Where code needs a seam before it can be tested (for example the static lets in VaultRoot), add the smallest one that works. Don't restructure.
  • If a test finds a bug, fix it in the same PR when it's small. Otherwise file a follow-up.
  • Anything security-relevant follows VAULT-82's disclosure rule: neutral titles and no exploitable detail in public until the fix has shipped.
  • Keep make validate fast. One deliberately slow test, such as unlocking with the production Argon2 parameters, is fine.

Done when

  • Every sub-issue is done, and make validate runs the new tests, UI tests included.
  • The "Test strategy" section of docs/on-device-encryption.md matches what's actually tested.
  • VAULT-82's list of guarantees can point at these tests.

Related: VAULT-82 (security review), VAULT-26 (on-device encryption design), VAULT-61 (no passcode means "Passcode Required").

Sub-issues

6
VAULT-84 P2 Test the App Lock paths through the real services Done VAULT-85 P2 Test unlocking with real passwords, erasing and launch recovery Done VAULT-86 P2 Pin the encryption formats with golden fixtures Done VAULT-87 P2 Test restores end to end, against a corpus of old backups Done VAULT-88 P2 Add a UI test target with launch tests Done VAULT-89 P2 UI test: launch locked, unlock, and lock again Done

Linked issues

1
Relates to VAULT-82 P1 Security review: check the app upholds its security guarantees Done

GitHub

0

No branches or pull requests linked.

Comments

1
Bradley

All six sub-issues are done, and make validate now runs every new suite, including the UI test plan.

  • VAULT-84, #695: the app lock through the real services, and what a lock clears.
  • VAULT-85, #684: unlocking with real passwords, erasing, launch recovery, the lock screen's wait, and one test with the production Argon2 parameters.
  • VAULT-86, #679: golden fixtures (slot file, encrypted items, backups).
  • VAULT-87, #699: a corpus of old backups restored end to end, plus malformed and fuzzed input.
  • VAULT-88, #676: the UI test target and launch tests.
  • VAULT-89, #696: launching locked, unlocking and locking again.

These tests turned up and fixed four things:

  • the AutoFill sheet now locks when the device does (#689);
  • saved-backup padding now always lands in its window (#680);
  • the bounded decompressor refuses anything after the stream (#699);
  • the VAULT-85 tests found nothing to fix.

Where things stand:

  • Test strategy: the "Test strategy" section of docs/on-device-encryption.md matches what's tested (#684, #700).
  • Security model: docs/security-model.md (#700) points at these tests.