Testing

Introduction

Only when writing this documentation and wanting to include my - until then - manual test cases in here, I recognized that there are apparently better and more efficient options for testing websites, than clicking buttons one-by-one 😅.

There are no unit tests for any of the javascript code pieces which comprise the Website. However, as the last piece of the development work, automated End-to-End (E2E) tests were created for all relevant parts of the functionality. The coverage can only be estimated, but is likely in the range of 80-90%.

The test are implemented using the excellent test library Playwright. Playwright was selected because of its superior support for testing PWA - it supports accessing caches, indexedDb and the service worker sw.js which is a fundamental requirement for the Website. The setup and configuration for Playwright (and the reporting backend Testomat) can be found here

The test are separated into tests for the legacy pages (also including the very basic functionality of the SIA) and a separate block for the SIA app itself - the folder structure is provided in Figure 7 . A number of helpers like data getters and setters and larger snippets of test were split off in separate folders.

---

📂 tests
├── 📂 helpers
│   ├── 📂 data-getters
│   ├── 📂 data-setters
│   ├── 📂 sia-actions
│   │   ├── 📂 guests
│   │   │   ├── 📄 guests-add.js
│   │   │   ├── 📄 guests-delete.js
│   │   │   ├── 📄 guests-helpers.js
│   │   │   ├── 📄 guests-register.js
│   │   │   ├── 📄 guests-signin.js
│   │   │   ├── 📄 guests-signout.js
│   │   │   ├── 📄 guests-update-amount.js
│   │   │   ├── 📄 guests-update-payment.js
│   │   │   └── 📄 index.js
│   │   ├── 📂 members
│   │   └── 📂 sales
│   ├── 📂 sia-scenarios
│   │   ├── 📂 actions
│   │   └── 📂 validations
│   └── 📂 test-snippets
├── 📂 pages
│   ├── 📂 aboutus
│   ├── 📂 contact
│   │   └── 📂 assets
│   ├── 📂 homepage
│   ├── 📂 info
│   ├── 📂 locations
│   ├── 📂 otherhashers
│   ├── 📂 runs
│   └── 📂 signinsheet
└── 📂 sign-in-app
    ├── 📂 guest
    │   ├── 📄 guestsoffline.spec.js
    │   └── 📄 guestsonline.spec.js
    ├── 📂 member
    ├── 📂 onsite-sales
    ├── 📂 reports
    └── 📂 scenarios

Figure 7: The folder structure of the Playwright tests

---

Testing legacy / static pages

The code for testing the "static" legacy pages is straightforward (although Playwright has it challenges) and will not be discussed here - its just click here -> click there -> assert something sequences and should be self-explanatory.

Testing the SIA app

The code for testing the SIA however needs some deeper discussion at this point as it is developed with a high-level of abstraction with the objective not to repeat code too much.

Fixtures

A fixture in testing is a reusable setup that prepares the necessary context or resources (like database records, test data, or configurations) before a test runs. It ensures tests start from a known state and can also handle cleanup afterward.

For tests of the SIA we run a global fixture (helpers/sia-fixture) which:

• deletes all records from the relevant collections before each test to ensure the same starting conditions for all tests
• deletes all records from the relevant collections after each test to avoid cloaking the database with test data.
• provides certain resources (tabContents) and functions (callToAction) to all tests
• provides a function to cleanup guests added for test purposes as we do not want to delete all guests (like above) but only the ones newly added for tests. This deletion must run after the related guest registrations / guest records in oder to maintain database integrity.

Helper functions and folder structure

There is a highly repetitive pattern in testing each action (create, update or delete a member/guest registration, member/guest sign-in, onsite sales etc.) within the SIA. For each action:

• with internet connection:

  • a request to the Strapi backend is send and needs to be monitored
  • for some a secondary - downstream - entry needs to be validated (registration in runrecords and guestrecords)
  • a respective entry to the local indexedDB must be made

• without internet connection:

  • an entry in the task queue in the indexedDb must be validated
  • after switching online again, the processing of the task queue must be monitored by validating the same backend requests as in the online case.

The following diagram depicts these steps in context:

---
config:
  look: handDrawn
  theme: base
  themeVariables:
        noteBkgColor: '#AD1457'
        primaryTextColor: '#AD1457'
        secondaryTextColor: '#FFFFFF'
        noteTextColor: '#FFFFFF'
        primaryColor: '#FCE4EC'
        lineColor: '#AD1457'
        background: '#FFFFFF'
---
flowchart TD
    A[Click Action Button] --> B[Persist Action in IndexedDB]
    B --> C{Online or Offline?}
    C -->|Offline| D[Write Task to Queue]
    D --> E[Write Data to Primary Backend Collection]
    C -->|Online| E
    E --> G[[Write Data to Secondary Backend Collection]]
    classDef dashed stroke-dasharray: 5 5;
    class G dashed;

To support this highly repetitive pattern without duplication, helper functions were created:

• performAction (which itself is wrapped as callToAction in the global fixture) in the file helpers/test-snippets/sia-actionrunner.js runs an action and is directly called by most tests
• performActionAndValidate in the file helpers/test-snippets/sia-entryvalidation.js which executes the respective action (it literally "clicks the button") and controls the subsequent validation of the backend communication and the communication with the indexedDB
• validateBackendResponse in the file helpers/test-snippets/sia-entryvalidation.js which handles the validation of the backend related create/update/delete calls including the validation of the secondary/downstream entry where required
• waitForStoreToUpdate in the file helpers/data-getters/indexedDb.js which similarly validates all interactions with the indexedDb

Each test controls the action to test and the related validations using a massive context object. As these objects are rather complex they have been

• split by business object (guests, members, sales) folders
• into action (member-register, member-signin, member-delete) files.
• each file contains the context objects for an online and an offline test, respectively.

This folder/file structure is created in helpers/sia-actions and a detailed example is displayed for guests in Figure 7.

The tests itself are implemented in the folder sign-in-app with a folder for each business object and a test file for online- and offline tests respectively.

Context Objects

The code snippet below shows an example of an actions context/configuration object. The main building blocks will be explained below.

MEMBER_REGISTER_ONLINE: {
  name: "Member: Register Online",
  prepare: async ({ page, userLoggedIn, testObject }) => {
    const option = await registerMember({ page, userLoggedIn, testObject });
    return option;
  },
  snackbarSelector: "snackbar-member",
  snackbarText: ({ testObject }) =>
    `Member registration successful for ${testObject.alias}`,
  networkMode: "online",
  primaryValidation: {
    method: "POST",
    pEndpoint: "/members-sign-ins",
    populate: "member",
  },
  secondaryValidation: ({ currentRunDate }) => ({
    sEndpoint: "/runrecords",
    runDate: currentRunDate,
  }),
  actionDb: ({ testObject }) => ({
    dbName: "klh3-data-store",
    storeName: "keyvaluepairs",
    key: "signInMembers",
    subValuePath: "memberSignIns",
    matchFn: (root) =>
      root.memberSignIns.some(
        (e) =>
          Number(e.member.id) === Number(testObject.id) && e.sign_in === false
      ),
  }),
  afterValidation: async ({ page, userLoggedIn, testObject }) => {
    await afterRegistration({ page, userLoggedIn, testObject });
  },
},

• name - defines the test name, which will be used in log messages
• prepare - defines all test steps which need to be executed before the respective action is triggered, e.g., navigating to the member tab -> click on the select box -> clear select box -> fill the members name -> select the option (name) which needs to be clicked to trigger the action which should be tested. This function *must return the page element which needs to be clicked to trigger the action. These steps (also for afterValidation see below) are often provided by helper functions (in folder sia-actions/{objectName}/{objectName}-helpers.js).
• snackbarSelector - as multiple snackbars were implemented (a major flaw in implementation 😶) we need to provide the testid of the respective selector for each action
• snackbarText - the text (template) which is displayed after the action was executed
• networkMode - online or offline depending on the test case
• primaryValidation - the endpoint/method which is used to trigger the respective backend activity; the populate entry is required for pulling the relationships where required. The test listens for these requests and checks if they are executed correctly
• secondaryValidation - (optional) if a secondary validation is required (the /members-sign-ins e.g., automatically creates/deletes a /runrecords entry). The data is pulled from the backend in addition to the primary validation and validated separately
• actionDb - defines the parameters of the indexedDb store where the local copy of the action will be persisted:
  • dbName: name of the database used for the action in indexedDb (all actions are persisted in klh3-datastore by default)
  • storeName: name of the store used for the action under investigation (usuallly keyvaluepairs)
  • key: name of the access group used for the respective action (as default named like the pinia store file, e.g., memberSignIns, guestSignIns)
  • subValuePath: some access groups hold more than one "table" (e.g. key memberSignIns has activeMembers and membersSignedIn) - the subValuePath specifies in which "table" the validation function is supposed to look.
  • matchFn: a function that is used for the validation (search) for a particular object (the element created by the object) - the function always validates key level (see above), if a subValuePath shall be used this needs to be amended (like root.**memberSignIns**.some() in the code snippet above)

  INFO

  This matchFn is also used for the record validation of the backend data

• queueDb: defines the parameters of the indexedDb store where the task queue for this group of actions (the business object like member, guest, onsiteSales) will be persisted
  • dbName: name of the database used for the action in indexedDb (all tasks of a task group - business object, like memberSignIns- will be persisted in a store named sync-queue-${businessObject}, e.g., sync-queue-memberSignIns`)
  • storeName: name of the store used for the action under investigation (usuallly keyvaluepairs)
  • key: all tasks will be persisted in an access group named tasks.
  • matchFn: a function that is used for the validation (search) for a particular task (the tasks created by the action)

TIP

The queueDb is optional and obviously only must be provided when testing offline actions

• afterValidation: all test steps which should be executed after the data validation. This includes usually assertions that certain elements are shown on the page or a status changed etc.

Running Tests

The setup and install process is described in the Section Server Setup.

Test should be run on the GitHub with development branch against staging environment for System -Integration-Testing and with the production branch on the staging environment for Release-to-Production Testing.

How to actually tun the tests is described in detail here

The complete deployment process including testing is described here