E2E Testing Offline PWA Workflows Using Playwright

by Alex Johnson 51 views

In this comprehensive guide, we'll delve into the crucial aspect of end-to-end (E2E) testing for Progressive Web Applications (PWAs) that function offline. Using Playwright, a powerful testing framework, we'll explore how to validate complete user workflows in both offline and online scenarios. This ensures your PWA delivers a seamless and reliable experience, even when network connectivity is intermittent or unavailable. Let's dive into the world of offline PWA testing and discover how to build robust and resilient web applications.

🎯 Goal: Ensuring Seamless Offline Functionality

The primary goal here is to implement robust end-to-end (E2E) tests using Playwright. These tests will validate complete user workflows in both offline and online scenarios. This validation is crucial for ensuring that our PWA functions as expected in production-like environments. By simulating real-world conditions, including network interruptions, we can confidently deliver a reliable user experience.

This involves a series of tests designed to mimic user interactions, from initial installation to complex data synchronization scenarios. The aim is to ensure that the PWA behaves predictably and effectively, even when the user is not connected to the internet. This is a cornerstone of PWA development, providing users with consistent access to functionality regardless of their network status. We will cover various aspects, such as service worker registration, offline data storage, and conflict resolution, to provide a holistic view of the application's offline capabilities. Effective E2E testing not only identifies potential issues but also builds confidence in the application's resilience.

📋 Test Scenarios: Covering All Bases

To thoroughly test our PWA's offline capabilities, we've outlined a series of test scenarios that cover a wide range of user interactions and potential edge cases. Each scenario focuses on a specific aspect of offline functionality, ensuring that we leave no stone unturned in our quest for a robust PWA experience.

1. Installation & First Load: Setting the Stage

This initial set of tests focuses on the first encounter a user has with the PWA. The goal is to ensure a smooth installation process and verify that the core offline functionalities are correctly set up. This includes:

  • [ ] Open app in browser: Simulating a user accessing the PWA for the first time.
  • [ ] Verify PWA installability prompt: Ensuring the browser correctly prompts the user to install the PWA.
  • [ ] Install app to home screen: Testing the installation process itself, mimicking a user adding the PWA to their device's home screen.
  • [ ] Launch from home screen: Verifying that the PWA launches correctly when opened from the home screen.
  • [ ] Verify service worker registration: A crucial step to confirm the service worker, which enables offline functionality, is properly registered.
  • [ ] Verify offline fallback cached: Ensuring essential assets are cached, allowing the PWA to function even without a network connection.

2. Offline Secret Creation Workflow: A Core Feature

This scenario dives into one of the core functionalities of SecPal: creating secrets while offline. It tests the entire workflow, from inputting data to handling synchronization when the network is restored:

  • [ ] Login to app: Starting with the authentication process to mimic a real user session.
  • [ ] Disconnect network: Simulating an offline scenario using Playwright's page.route('**/*', route => route.abort()).
  • [ ] Navigate to "New Secret": Testing the navigation within the app while offline.
  • [ ] Fill form (title, username, password): Inputting data into the form fields to create a new secret.
  • [ ] Submit form: Submitting the form while the app is offline.
  • [ ] Verify success message: "Saved offline, will sync when online": Confirming that the user receives appropriate feedback when saving data offline.
  • [ ] Verify secret appears in list with "Offline" badge: Ensuring the newly created secret is displayed in the list with an indication that it's pending synchronization.
  • [ ] Reconnect network: Simulating the restoration of network connectivity.
  • [ ] Wait for auto-sync: Allowing the PWA to automatically synchronize the offline data with the server.
  • [ ] Verify badge changes to "Synced": Confirming that the secret's status updates to indicate successful synchronization.

3. Offline Edit with Conflict Resolution: Handling Data Conflicts

This scenario tackles the complex issue of data conflicts that can arise when editing data offline and synchronizing it with server-side changes. It's a critical test for ensuring data integrity:

  • [ ] Open existing secret: Starting with an existing data entry to simulate a common user action.
  • [ ] Disconnect network: Again, simulating an offline environment.
  • [ ] Edit password field: Making changes to the data while offline.
  • [ ] Save changes → "Pending sync" badge: Confirming that the changes are saved locally and marked for synchronization.
  • [ ] Simulate server edit (via API mock or separate session): Introducing a conflicting change on the server-side to mimic a real-world scenario.
  • [ ] Reconnect network: Restoring network connectivity.
  • [ ] Conflict dialog appears: Verifying that the PWA correctly detects the data conflict and presents a resolution dialog.
  • [ ] Select "Keep Local" option: Choosing to prioritize the local changes over the server-side changes.
  • [ ] Verify local changes preserved: Ensuring that the user's local edits are retained.
  • [ ] Verify sync status: "Synced": Confirming that the data is eventually synchronized without losing the user's changes.

4. Large File Upload Offline: Testing Data Handling Capacity

This scenario tests the PWA's ability to handle large file uploads while offline, a crucial feature for many applications:

  • [ ] Open secret detail: Navigating to the section where file attachments are managed.
  • [ ] Disconnect network: Simulating offline conditions.
  • [ ] Upload 10MB PDF file: Uploading a sizable file to test the data handling capabilities.
  • [ ] Verify file queued in IndexedDB: Ensuring the file is correctly queued for upload in the local storage (IndexedDB).
  • [ ] Reconnect network: Restoring network connectivity.
  • [ ] Verify upload progress indicator: Monitoring the upload process to provide feedback to the user.
  • [ ] Verify file appears in attachments list after sync: Confirming that the file is successfully uploaded and displayed in the attachments list.
  • [ ] Download file, verify integrity (hash check): Downloading the uploaded file and verifying its integrity to ensure no data loss during the process.

5. Multi-Device Sync: Ensuring Data Consistency Across Devices

This scenario tests the PWA's ability to synchronize data across multiple devices, a key feature for users who access the application from various devices:

  • [ ] Device A: Create secret "Test Secret": Creating a new secret on one device.
  • [ ] Device B (offline): Create secret with same title: Creating another secret with the same title on a different device while offline. This is designed to create a conflict.
  • [ ] Device B goes online: Simulating Device B reconnecting to the network.
  • [ ] Conflict detected (duplicate titles): Verifying that the PWA correctly detects the conflict caused by the duplicate titles.
  • [ ] Device B resolves conflict (choose LWW or manual): Simulating the user resolving the conflict, either by choosing the last write wins (LWW) strategy or manually merging the data.
  • [ ] Both devices show correct final state: Ensuring that the final synchronized state is consistent across both devices.

6. Network Interruption Recovery: Handling Intermittent Connectivity

This scenario tests the PWA's resilience to network interruptions during synchronization. It ensures that data sync can resume smoothly after a disconnection:

  • [ ] Create 5 secrets offline: Creating multiple data entries to simulate a more complex sync operation.
  • [ ] Go online, start syncing: Initiating the synchronization process.
  • [ ] After 2 synced, disconnect network: Simulating a network interruption mid-sync.
  • [ ] Verify sync pauses: Confirming that the synchronization process pauses when the network is disconnected.
  • [ ] Reconnect network: Restoring network connectivity.
  • [ ] Verify sync resumes from operation 3: Ensuring that the sync process resumes from where it left off, rather than restarting from the beginning.
  • [ ] All 5 secrets synced successfully: Confirming that all data entries are eventually synchronized without errors.

7. Session Expiry During Offline: Maintaining Security

This scenario tests the PWA's handling of session expiry while the user is offline, ensuring that security is maintained even in disconnected states:

  • [ ] Login to app: Starting with the authentication process.
  • [ ] Disconnect network: Simulating an offline environment.
  • [ ] Wait for token to expire (simulate by clearing cookies): Mimicking session expiry by manipulating the authentication token.
  • [ ] Create secret offline: Creating a new secret while the session is expired.
  • [ ] Reconnect network: Restoring network connectivity.
  • [ ] Redirected to login: Verifying that the user is redirected to the login screen due to session expiry.
  • [ ] Login again: Simulating the user re-authenticating.
  • [ ] Secret sync completes after re-auth: Ensuring that the offline data is synchronized after the user logs in again.

8. Slow Network Handling: Optimizing for Poor Connectivity

This scenario tests the PWA's performance under slow network conditions, ensuring a smooth experience even with limited bandwidth:

  • [ ] Throttle network to 3G: Simulating a slow network connection using Playwright's network throttling capabilities (page.route('**/*', route => route.continue({ delay: 2000 }))).
  • [ ] Create secret with large attachments: Creating a data entry with large files to test the upload process under slow network conditions.
  • [ ] Verify upload progress indicator: Monitoring the upload progress to provide feedback to the user.
  • [ ] Verify timeout doesn't fail sync (retry logic): Ensuring that the synchronization process doesn't fail due to timeouts and that retry logic is in place.
  • [ ] Sync completes eventually: Confirming that the data is eventually synchronized, even with a slow network connection.

9. Background Sync After App Close: Leveraging Service Workers

This scenario tests the PWA's background synchronization capabilities using service workers, ensuring that data is synced even when the app is closed:

  • [ ] Create secret offline: Creating a new secret while offline.
  • [ ] Close browser tab (app in background): Simulating the user closing the app.
  • [ ] Reconnect network: Restoring network connectivity.
  • [ ] Service Worker triggers Background Sync: Verifying that the service worker triggers background synchronization.
  • [ ] Open app again: Re-opening the app.
  • [ ] Verify secret synced (badge shows "Synced"): Confirming that the data has been synchronized in the background and the UI reflects this status.

10. Cross-Browser Compatibility: Ensuring Consistent Performance

This final scenario emphasizes the importance of cross-browser compatibility, ensuring that the PWA functions consistently across different browsers:

  • [ ] Run all scenarios in:
    • [ ] Chrome/Edge (Chromium)
    • [ ] Firefox
    • [ ] Safari (WebKit)
  • [ ] Verify offline/online detection works: Ensuring that the PWA correctly detects network connectivity status in all browsers.
  • [ ] Verify IndexedDB works consistently: Confirming that the local storage (IndexedDB) functions consistently across browsers.
  • [ ] Verify service worker behavior: Ensuring that the service worker behaves as expected in different browsers.

🧪 Playwright Configuration: Setting Up the Testing Environment

Configuring Playwright correctly is essential for running effective E2E tests. Here’s how we set up our environment to simulate various scenarios, including offline conditions and network throttling.

Setup: Configuring the Playwright Environment

To begin, we need to configure Playwright using the playwright.config.ts file. This file defines the test directory, base URL, and other essential settings. Here’s a sample configuration:

// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './tests/e2e',
  use: {
    baseURL: 'http://localhost:5173',
    trace: 'on-first-retry',
    screenshot: 'only-on-failure',
  },
  webServer: {
    command: 'npm run dev',
    port: 5173,
    reuseExistingServer: !process.env.CI,
  },
  projects: [
    { name: 'chromium', use: devices['Desktop Chrome'] },
    { name: 'firefox', use: devices['Desktop Firefox'] },
    { name: 'webkit', use: devices['Desktop Safari'] },
    { name: 'mobile', use: devices['iPhone 13'] },
  ],
});

Key configurations include:

  • testDir: Specifies the directory where our test files are located.
  • baseURL: Sets the base URL for our application, making it easier to define test routes.
  • trace: Enables tracing, which helps in debugging by providing detailed logs and snapshots.
  • screenshot: Configures Playwright to capture screenshots only on test failures.
  • webServer: Defines how to start our development server, ensuring the application is running before tests begin.
  • projects: Configures different browser environments to run tests in, including Chromium, Firefox, WebKit, and a mobile device (iPhone 13).

Offline Simulation: Mimicking Disconnected Environments

Simulating offline conditions is crucial for testing PWA functionality. Playwright provides a simple way to disconnect the network using the page.route method:

// Disconnect network
await page.route('**/*', route => route.abort());

// Reconnect network
await page.unroute('**/*');
  • page.route('**/*', route => route.abort()): This line intercepts all network requests and aborts them, effectively simulating an offline state.
  • page.unroute('**/*'): This line removes the interception, allowing network requests to proceed normally, thus reconnecting the network.

Network Throttling: Simulating Slow Connections

Testing under slow network conditions helps ensure a smooth user experience even with limited bandwidth. Playwright allows us to emulate network conditions using the Chrome DevTools Protocol (CDP):

const client = await page.context().newCDPSession(page);
await client.send('Network.emulateNetworkConditions', {
  offline: false,
  downloadThroughput: 50 * 1024, // 50 KB/s (3G)
  uploadThroughput: 20 * 1024,   // 20 KB/s
  latency: 500,                   // 500ms
});
  • page.context().newCDPSession(page): Creates a new CDP session for the page.
  • client.send('Network.emulateNetworkConditions', ...): Sends a command to the CDP to emulate network conditions, including:
    • offline: Set to false to ensure the browser is online.
    • downloadThroughput: Sets the download speed in bytes per second (50 KB/s in this example).
    • uploadThroughput: Sets the upload speed in bytes per second (20 KB/s in this example).
    • latency: Sets the network latency in milliseconds (500ms in this example).

By using these configurations, we can accurately simulate various network conditions and ensure our PWA performs optimally in all scenarios. This includes handling intermittent connectivity, slow networks, and complete offline states.

📂 Test Structure: Organizing for Clarity

To maintain a clean and manageable test suite, we organize our tests into a structured directory. This structure makes it easier to locate specific tests and understand the scope of each test file.

tests/
└── e2e/
    ├── installation.spec.ts         # PWA installation tests
    ├── offline-creation.spec.ts     # Create secrets offline
    ├── offline-edit.spec.ts         # Edit and sync conflicts
    ├── file-upload.spec.ts          # Attachment handling
    ├── multi-device.spec.ts          # Cross-device sync
    ├── network-recovery.spec.ts     # Network interruption tests
    ├── auth-expiry.spec.ts          # Token expiration handling
    ├── slow-network.spec.ts         # Throttled network tests
    ├── background-sync.spec.ts      # Service Worker sync tests
    └── fixtures/
        ├── auth.ts                  # Authentication helpers
        ├── secrets.ts               # Secret creation helpers
        └── network.ts               # Network simulation helpers

Key components of this structure include:

  • tests/e2e/: The main directory for our E2E tests.
  • Individual test files (.spec.ts): Each file focuses on a specific scenario or feature, such as installation, offline creation, or network recovery.
  • fixtures/: A directory containing helper files that provide reusable functions and configurations. This includes:
    • auth.ts: Helpers for authentication-related tasks.
    • secrets.ts: Helpers for creating and managing secrets.
    • network.ts: Helpers for simulating network conditions.

This structured approach ensures that our test suite remains organized and easy to maintain, even as it grows in complexity. Well-organized tests are crucial for efficient debugging and continuous integration.

✅ Acceptance Criteria: Defining Success

To ensure that our E2E testing efforts are successful, we've established clear acceptance criteria. These criteria define the standards that must be met for the testing process to be considered complete and satisfactory.

  • [ ] All 10 E2E scenarios implemented: Every scenario listed in the Test Scenarios section must have corresponding test implementations.
  • [ ] Tests run in CI/CD (GitHub Actions): The tests must be integrated into our Continuous Integration and Continuous Deployment (CI/CD) pipeline using GitHub Actions. This ensures that tests are run automatically on every code change.
  • [ ] Tests pass on all target browsers (Chrome, Firefox, Safari): The tests must pass consistently on all browsers that we support, including Chrome, Firefox, and Safari. This ensures cross-browser compatibility.
  • [ ] Mobile device tests included: Tests must include scenarios that simulate mobile device usage, ensuring the PWA functions correctly on mobile devices.
  • [ ] Flakiness eliminated (retry logic if needed): The tests should be stable and not prone to flakiness. If flakiness is unavoidable, retry logic should be implemented to mitigate the issue.
  • [ ] Screenshots/videos captured on failure: On test failures, screenshots and videos should be captured to aid in debugging. This provides visual context that can help identify the root cause of the failure.
  • [ ] Test execution time <10 minutes: The entire test suite should execute in under 10 minutes. This ensures that the testing process doesn't become a bottleneck in the development cycle.
  • [ ] Documentation: "Running E2E Tests Locally": Clear documentation should be provided on how to run the E2E tests locally. This enables developers to easily run tests during development and debugging.

Meeting these acceptance criteria ensures that our E2E tests are comprehensive, reliable, and efficient. This, in turn, helps us deliver a high-quality PWA that meets the needs of our users.

🔗 Related: Context and Dependencies

To provide context and highlight dependencies, we've included references to related issues and pull requests. This helps in understanding the broader scope of this testing effort.

  • Part of: #196 (Phase 5: Offline Support & PWA)
  • Depends on: PR #236 (Phase 5 implementation)
  • Complements: #237 (Integration Tests)

📊 Estimated Effort: Time and Priority

To provide a sense of the scale and importance of this task, we've estimated the effort required and assigned a priority.

Size: L (5-7 days) Priority: High (validates production readiness)

Conclusion

In conclusion, end-to-end (E2E) testing of offline PWA workflows using Playwright is a critical step in ensuring the reliability and user experience of your application. By implementing comprehensive test scenarios, configuring Playwright effectively, and adhering to clear acceptance criteria, you can confidently deliver a PWA that functions seamlessly in both online and offline environments. This guide has provided a detailed roadmap for achieving this goal, covering everything from initial setup to advanced testing techniques. Embrace these practices to build robust and resilient PWAs that meet the demands of today's users. For more information on PWA testing and best practices, visit Google's Web Fundamentals.

Type: Testing Labels: testing, e2e, playwright, pwa, offline