12 posts tagged with "testing"

Imagine a tightrope walker blaming their safety net for a fall — retrying, hoping the net won't catch them the second time, or worse, removing it altogether. Sounds absurd, right? Yet, in software testing, we often treat "flaky tests" with the same flawed logic.

Something I read recently in a chat when one of the so called e2e tests failed:

Was it just a "normal flaky test"?

The "normal" first step is to retry the failed test, hope that it pass, then go on - and probably cheer automation tools that support retrying failed tests for x times before declaring them as failed.

The "normal" second step is to disable this tests to get the build green.

When software people refer to tests as their "saftey net", then I start to wonder the situation, if they were tightrope walkers. Imagine a situation that you made mistakenly a step a side and fall down into the saftey net - would you act like in the situation I described above?

Retry again - hope that the net won't catch you the 2nd or 3rd time? Remove the net, because it was obvoiusly the net's fault you made the wrong step.

Okay, don't let us be to metaphorical and I don't want to elaborate about the reasons behind flaky tests. There was enough stuff written about that (a lot of good and a lot of bad stuff).

One good collection of reasons that is a bit aged but I can still relate to: https://testing.googleblog.com/2020/12/test-flakiness-one-of-main-challenges.html

So today I just like to remind you about some skills to learn to avoid "normal flaky tests":

Learn more about your application under test​

Understand application behavior and interactions: Deep dive into how your application behaves, especially in edge cases.

Handle asynchronous operations: Ensure you understand and properly handle asynchronous operations within the application.

Learn more about your test environment​

Manage test data consistently: Ensure your tests have reliable and consistent data to work with, avoiding fluctuations that cause flakiness.

Maintain environmental consistency: Ensure that your test environments are consistent and reflect the production environment as closely as possible.

Ensure network stability and handling: Understand how to manage and mitigate network instability that could affect your tests.

Implement robust error handling: Develop robust error handling to manage unexpected issues gracefully.

Learn more about your test automation framework​

Understand parallel execution: Learn how your test framework handles parallel execution and manage shared resources to avoid conflicts.

Manage timeouts and synchronization: Configure proper timeouts and synchronization mechanisms to handle asynchronous operations effectively.

Learn more about your tests​

Isolate tests: Ensure that tests are isolated and independent, avoiding interdependencies that could cause cascading failures.

Test data management: Develop strategies for consistent and reliable test data management to avoid flaky results.

Error handling in tests: Implement robust error handling within your tests to manage unexpected issues gracefully.

Learn how to fix it​

Finally, learn how to identify the root of your issues within the knowledge fields above, learn how to fix them - and then - just fix them!

Conclusion​

Implementing these strategies to avoid "normal flaky tests" requires effort and isn't free of challenges. However, the investment in learning about your application, test environment, test automation framework, and tests themselves pays off significantly. By addressing the root causes of flakiness, you can improve the reliability and robustness of your tests, ultimately enhancing the overall quality of your software.

Are your test scripts drowning in test data chaos? Say goodbye to clutter and hello to clarity with Serenity/JS's Actor Notepads! Learn how to inject user credentials seamlessly for cleaner, more maintainable tests. Ready to dive in?

Actors in Serenity/JS​

Serenity/JS is a testing framework that promotes collaboration, clarity, and maintainability in automated testing. At its core, Serenity/JS introduces the concept of actors, which represent the different users or personas interacting with a web application. Actors are equipped with capabilities and context, allowing them to perform actions and interact with the application under test.

Introducing Actor Notepads​

A key feature of Serenity/JS is the concept of actor notepads. Notepads serve as containers for actor-specific data, such as credentials, expected outcomes, and any other contextual information relevant to the test scenario. Instead of cluttering test scripts with data setup details, actor notepads encapsulate this information within the actor itself.

Example​

Let's assume a small test where an actor:

• Navigates to the login page of the application (https://the-internet.herokuapp.com/login).
• Enters the username retrieved from the actor's notepad into the username input field on the login page.
• Enters the password retrieved from the actor's notepad into the password input field on the login page.
• Clicks on the login button to submit the credentials.
• Waits until a flash message appears on the page, indicating a successful login, with the expected result retrieved from the actor's notepad.

First, we prepare two actors called Tom Smith, who is allowed to login and Jan Doe, who is not allowed to login. We put their login credentials and expected result into their notepads, using the ability to TakeNotes.

// To avoid typos and repetition when instantiating and retrieving actors, 
// we use string enums to store actor names:
enum ActorNames {
    Tom = 'Tom Smith, who is allowed to use our system,',
    Jane = 'Jane Doe, who is not allowed to use our system,'
}

// We are using a model for our actors notepad
type FormAuthenticationModel = {
    Username: string,
    Password: string,
    ExpectedResult: string
}

class AuthenticationActors implements Cast {
    constructor(private readonly browser: Browser) {
    }

    // Prepare method to configure actors with specific capabilities and data
    prepare(actor: Actor): Actor {
        // Initialize the actor with the capability to browse the web using Playwright
        let preparedActor = actor.whoCan(BrowseTheWebWithPlaywright.using(this.browser));

        // Switch statement to customize actor based on their name
        switch (actor.name) {
            // Configuration for actor named Tom
            case ActorNames.Tom: {
                // Configure actor to take notes using a notepad with specific authentication data
                preparedActor = preparedActor.whoCan(
                    TakeNotes.using(Notepad.with<FormAuthenticationModel>({
                        Username: 'tomsmith',
                        Password: 'SuperSecretPassword!',
                        ExpectedResult: 'You logged into a secure area!'
                    }))
                );
                break;
            }
            // Configuration for actor named Jane
            case ActorNames.Jane: {
                // Configure actor to take notes using a notepad with specific authentication data
                preparedActor = preparedActor.whoCan(
                    TakeNotes.using(Notepad.with<FormAuthenticationModel>({
                        Username: 'janedoe',
                        Password: 'WrongPassword!',
                        ExpectedResult: 'Your username is invalid!'
                    }))
                );
                break;
            }
        }
        // Return the prepared actor with customized capabilities and data
        return preparedActor;
    }
}

For our test we have to prepare the PageElements and a Task:

class FormAuthenticationPage {
    static UserNameInputField = PageElement.located(By.css('#username')).describedAs('field to input the username')
    static PasswordInputField = PageElement.located(By.css('#password')).describedAs('field to input the password')
    static LoginButton = PageElement.located(By.css('[type="submit"]')).describedAs('button to login')
    static FlashMessage = PageElement.located(By.css('#flash')).describedAs('result message')
}

const LoginToAppUsingActorsNotepad = () => 
    Task.where(`#actor logs in using username and password from their notepad`,
        Navigate.to('https://the-internet.herokuapp.com/login'),
        Enter.theValue(notes<FormAuthenticationModel>().get('Username')).into(FormAuthenticationPage.UserNameInputField),
        Enter.theValue(notes<FormAuthenticationModel>().get('Password')).into(FormAuthenticationPage.PasswordInputField),
        Click.on(FormAuthenticationPage.LoginButton),
        Wait.until(Text.of(FormAuthenticationPage.FlashMessage), includes(notes<FormAuthenticationModel>().get('ExpectedResult')))
    )

Finally, when using Serenity/JS with Playwright Test our spec looks like:

//make sure to import { describe, it, test } from '@serenity-js/playwright-test'

describe('form authentication notepad', () => {
    test.use({
        
        /*
         * Override the default cast of actors, 
         * so that each actor receives their own,
         * independent browser window.
         */
        actors: async ({ browser }, use) => {
            use(new AuthenticationActors(browser));
        },
    });
    describe('login page', () => {
        it('should log in with correct username and password', async ({ actorCalled }) => {
            await actorCalled(ActorNames.Tom).attemptsTo(
                LoginToAppUsingActorsNotepad(),
            )
        })

        it('should deny login with incorrect username and password', async ({ actorCalled }) => {
            await actorCalled(ActorNames.Jane).attemptsTo(
                LoginToAppUsingActorsNotepad(),
            )
        })

    })
})

To start right away with Serenity/JS using Playwright test I recommend to use a Serenity/JS template.

Advantages of Using Actor Notepads​

• Separation of Concerns: By delegating data setup to actor notepads, test scripts focus solely on orchestrating actor actions, leading to cleaner and more readable code.

• Encapsulation: Actor notepads encapsulate actor-specific data, promoting better organization and reducing the likelihood of data misuse or modification.

• Reusability: Actor notepads facilitate the reuse of actors across multiple tests without duplicating data setup logic, improving test maintainability and reducing code duplication.

• Scalability: As test suites grow, managing data setup through actor notepads proves more scalable and manageable than injecting data through parameters.

• Enhanced Reporting: Serenity/JS emphasizes descriptive and readable test reports. Actor notepads enable better reporting by associating test steps with relevant actor context, enhancing test comprehensibility.

• Flexibility for Extension: Actor notepads provide a flexible mechanism to extend actor capabilities or introduce additional data for testing purposes, without significantly modifying test script structure.

Conclusion​

In conclusion, leveraging actor notepads within Serenity/JS tranforms automated testing, offering a streamlined approach to managing test data. By encapsulating test data within actors, test scripts become clearer, more maintainable, and less prone to errors. With the separation of concerns afforded by actor notepads, the focus shifts from data setup to orchestrating meaningful test actions, resulting in enhanced readability and scalability.

Don't let visual bugs slip through! Learn how to quickly add WDIO visual regression testing to your Serenity/JS project.

I recently watched a video tutorial by Wim Selles here explaining the new WebdriverIO snapshot and visual snapshot features introduced in version 8.29.0.

Playwright also offers some support for snapshot and visual snapshot testing here.

I won't go into detail of visual regression testing since there are many resources available online.

While snapshot testing isn't currently part of Serenity/JS, this guide shows how to quickly integrate the feature into your Serenity/JS test specs using custom interactions.

For more on custom interactions, refer to the Serenity/JS documentation here.

Setting Up the Project​

1. Clone the serenity-js-mocha-webdriverio-template repository:

git clone git@github.com:serenity-js/serenity-js-mocha-webdriverio-template.git

2. Navigate to the cloned directory and run:

npm install
npm test

This ensures your project is set up correctly.

3. Install the visual testing support for WebdriverIO:

npm install @wdio/visual-service --save-dev

If you encounter errors, check the system requirements for visual testing with WDIO here.

4. Enable the visual service in your project's root directory file wdio.conf.ts:

// Test runner services
// Services take over a specific job you don't want to take care of. They enhance
// your test setup with almost no effort. Unlike plugins, they don't add new
// commands. Instead, they hook themselves up into the test process.
services: [ 'visual' ],

5. Update your project's root directory file tsconfig.json to include the necessary WDIO types:

{
    "compilerOptions": {
        "types": [
          ...
          "expect-webdriverio",
          "@wdio/visual-service"
        ],
    }
}

Creating Custom Interactions​

Now, let's create our custom Serenity/JS interactions:

import { Interaction, MetaQuestionAdapter } from '@serenity-js/core';
import { PageElement } from '@serenity-js/web';

export const CompareScreen = (snapshotIdentifier: string) =>     
    Interaction.where('#actor compares whole page screenshot', 
        async actor => {
            return expect(browser).toMatchFullPageSnapshot(snapshotIdentifier)
        })

export const CompareElement = (
    pageElement: MetaQuestionAdapter<PageElement<unknown>, PageElement<unknown>>,
    snapshotIdentifier: string) =>     
    Interaction.where('#actor compares element screenshot', 
        async actor => {
            const selector = await pageElement.answeredBy(actor)
            return expect(selector.nativeElement())
                .toMatchElementSnapshot(snapshotIdentifier)
        }
    )

Using the Interactions in Specs​

Here's how to use the interactions in your spec files:

 it(`offers examples to help you practice test automation`, async () => {
        ...
        // Once we know the system is up and running, Wendy can proceed with the web-based scenario.
        await actorCalled('Wendy').attemptsTo(
            TodoList.createListContaining([
                `Buy dog food`,
                `Feed the dog`,
                `Book a vet's appointment`,
            ]),
            CompareScreen('list_after_creation'),
            TodoList.markAsCompleted([
                `Buy dog food`,
                `Feed the dog`,
            ]),
            CompareScreen('list_after_completion'),
            Ensure.that(
                TodoList.outstandingItemsCount(),
                equals(1)
            ),
        )
    })

Conclusion​

n this guide, we've explored how to integrate visual regression testing into your Serenity/JS project using custom interactions. This approach allows you to leverage Serenity/JS's powerful testing capabilities while ensuring the visual consistency of your web application. Remember, visual testing is a valuable tool, but it's important to use it strategically and with clear expectations. By following the best practices, you can effectively incorporate visual regression testing into your Serenity/JS test suite.

Next Chapter: Don't reinvent the wheel! Discover how to seamlessly integrate BDD scenarios into your existing Serenity/JS and Playwright setup, unlocking the power of code reuse across component testing and user acceptance tests. Say goodbye to redundant work and hello to efficient testing.

In the last tutorial, I showed how to do component testing using Serenity/JS and Playwright. To be honest, it's a very simple component, making it difficult to create a real-world example. Let's just assume it's a coffee counter for controlling how many cups of coffee you drink.

Component testing ensures that your self-contained components work as expected. For user acceptance, there may be some additional things necessary to verify functionality.

The good news is: You can reuse what you've already prepared for the component tests and dive into the user scenarios.

In this article, I will demonstrate how we can integrate BDD scenarios into the project I prepared the last time. I will use Serenity/JS BDD features. These BDD scenarios will run in the same project like the component tests and will reuse Serenity/JS PageElements and Tasks that we already used in the component tests.

I cloned the last example and updated all packages to the latest versions to be up to date and created a new repository. If you want to follow all the steps from scratch I suggest going back and starting with the component testing tutorial and coming back once you've finished.

To start, you've to add some additional dev dependencies to your project:

npm install --save-dev @cucumber/cucumber 
npm install --save-dev @serenity-js/cucumber

This time I will not list the file content in detail but refer to the repository and some explanations.

Key to using BDD and Serenity/JS with cucumber is the features folder.

features
|- coffee-counter
|  |- coffee-counter.feature
|  \- readme.md
|
|- step-definitions
|  |- counter.steps.ts
|  \- parameter.steps.ts
|
|- support
|  |- Actors.ts
|  \- serenity.config.ts

To enable Serenity/JS, we need the support folder with the Actors.ts and the serenity.config.ts.

To run cucumber, add a cucumber.js configuration file to your project root. Among other things, we configure to use Serenity/JS Cucumber adapter in here.

Add two new scripts to the package.json

"scripts": {
    ...
    "test:cucumber:execute": "cucumber-js",
    "test:cucumber": "failsafe clean serenity-bdd:update test:cucumber:execute test:report"
}

There is still one configuration thing left: The create-next-app from the last tutorial created a tsconfig.json for us. Some compile options are not compatible with cucumber.js, and we have to include the features folder.

{
  "compilerOptions":
   {
     "module": "commonjs",
     "moduleResolution": "node",
     "esModuleInterop": true,
   },
   {
    "include": [
      "features/**/*.ts",
    ]
   }
}

To not interfere with the original Next.js configuration, I create a copy tsconfig.next.ts. I configured the Next.js itself to use this configuration in next.config.mjs instead of the tsconfig.ts file, just to make sure we still use the right config when building the Next.js app.

/** @type {import('next').NextConfig} */
const nextConfig = {
    typescript: { tsconfigPath: 'tsconfig.next.json'}
};

export default nextConfig;

Until now, we just did some configuration tweaks. Now we're ready to take a look back into our coffee counter scenario again. Now have a look into the counter steps definitions. Here is where the magic happens. We just reuse the same tasks we already provided for component testing:

import { Given, Then, When } from '@cucumber/cucumber';
import { Actor } from '@serenity-js/core';

import { CounterSerenity } from '../../src/components/CounterSerenity';

Then('{pronoun} should see that the "Increase" button is enabled', async (actor: Actor) =>
    actor.attemptsTo(
        CounterSerenity.EnsureIncreaseButtonIsEnabled()
    ))

You're nearly ready to run the test the first time. Make sure that you start your server with npm run dev on port 3000.

Then run npm run test:cucumber and find the BDD report in the target/site folder.

Did you know that many software defects arise from unclear requirements? What we're seeing today is AI attempting to assist in test creation, promising to increase code coverage. However, this approach often emphasizes testing our implementations rather than testing against our requirements, which I believe has its value but also carries certain risks that must be carefully considered.

Don't test your implementation​

To illustrate, let's conduct an experiment using our familiar chatbots. I'll present a simple function implementation suitable for our demonstration, intentionally flawed.

This is a TS function. Can you generate me a jest test spec for it?

function addTwoNumbers(a: number, b: number): number {
    return a / b
}

Here's, what ChatGPT 3.5 responded:

Sure, here's a Jest test spec for the addTwoNumbers function:

import { addTwoNumbers } from './yourFileName';

describe('addTwoNumbers', () => {
    it('should correctly add two numbers', () => {
        // Arrange
        const a = 5;
        const b = 3;
        
        // Act
        const result = addTwoNumbers(a, b);

        // Assert
        expect(result).toEqual(8);
    });

    it('should handle negative numbers correctly', () => {
        // Arrange
        const a = -10;
        const b = 7;

        // Act
        const result = addTwoNumbers(a, b);

        // Assert
        expect(result).toEqual(-3);
    });

    it('should handle zero correctly', () => {
        // Arrange
        const a = 0;
        const b = 5;

        // Act
        const result = addTwoNumbers(a, b);

        // Assert
        expect(result).toEqual(0);
    });

    // Add more test cases as needed
});

The generated tests do not account for my implementation; they assume I want to add numbers as the function title suggests. The test will fail. However, this allows me to challenge my code.

Here's, what Gemini 1.0 answered:

I'd be glad to provide a Jest test spec for the addTwoNumbers function, incorporating the valuable insights from the ratings and addressing the identified issues:

import { addTwoNumbers } from './path/to/your/function'; // Replace with the actual path

describe('addTwoNumbers', () => {
  // Test case 1: Basic addition with positive numbers
  it('should add two positive numbers correctly', () => {
    expect(addTwoNumbers(5, 3)).toBe(5 / 3); // Corrected the expected result
  });

  // Test case 2: Addition with negative numbers
  it('should handle addition with negative numbers', () => {
    expect(addTwoNumbers(-2, 4)).toBe(-2 / 4); // Corrected the expected result
  });

  // Test case 3: Division by zero (error handling)
  it('should throw an error when dividing by zero', () => {
    expect(() => addTwoNumbers(5, 0)).toThrowError('Division by zero'); // Handle division by zero appropriately
  });

  // Test case 4: Data type validation (optional)
  it('should only accept numbers as arguments (optional)', () => {
    expect(() => addTwoNumbers('five', 3)).toThrowError(
      'addTwoNumbers: Arguments must be numbers' // Customized error message
    );

    expect(() => addTwoNumbers(5, 'three')).toThrowError(
      'addTwoNumbers: Arguments must be numbers' // Customized error message
    );
  });
});

Gemini generated tests that pass. The test reads like an addition is tested, but effectively, it's the division.

To be fair, sometimes the bots detect inconsistencies between the function name and implementation.

However, we miss the point here:

What's unknown is if our customer was requiring an addition or a division.

Either the function implementation is incorrect and should be a + b, or there's a naming issue, and it should be named divideTwoNumbers().

So, what are the takeaways?

1. Testing should align with requirements and expected behavior

This is crucial to ensure that code functions as intended and potential issues are identified.

2. Importance of human judgment

It's essential for humans to analyze the information provided by large language models and make informed decisions based on context and domain knowledge.

Align your tests with your requirements​

While I don't want to leave you without a solution to this problem, it's worth noting that you should utilize AI to assist in testing.

However, it's important to focus on explaining the purpose of the test and the problem you want to solve, rather than just testing the mplementation itself.

While this principle is not new, it's crucial to remember, as it's something that can be overlooked when rushing into the use of AI.

Can you write me a jest test spec that test test a typescript function that takes two numbers and divide them. Please cover edge cases.

import { divideNumbers } from './yourDivideFunction'; // Import your divide function

describe('divideNumbers function', () => {
  it('should divide two positive numbers correctly', () => {
    expect(divideNumbers(10, 2)).toBe(5);
  });

  it('should divide a positive number by zero and return Infinity', () => {
    expect(divideNumbers(10, 0)).toBe(Infinity);
  });

  it('should divide a negative number by a positive number correctly', () => {
    expect(divideNumbers(-10, 2)).toBe(-5);
  });

  it('should divide zero by a positive number and return 0', () => {
    expect(divideNumbers(0, 10)).toBe(0);
  });

  it('should return NaN if divided by zero', () => {
    expect(divideNumbers(0, 0)).toBe(NaN);
  });

  it('should divide a large number by a small number correctly', () => {
    expect(divideNumbers(1000000, 0.000001)).toBe(1e+12);
  });

  it('should handle decimal precision properly', () => {
    expect(divideNumbers(1, 3)).toBeCloseTo(0.333, 3); // Rounded to 3 decimal places
  });

  // Add more test cases as needed for your specific requirements
});

With the right input, you can even get out BDD scenarios in Gherkin:

Feature: Division of Two Numbers
  As a user
  I want to divide two numbers
  So that I can perform mathematical operations accurately

  Scenario Outline: Divide two numbers
    Given I have two numbers <numerator> and <denominator>
    When I divide <numerator> by <denominator>
    Then the result should be <result>

    Examples:
      | numerator | denominator | result |
      | 6         | 3           | 2      |
      | 10        | 2           | 5      |
      | 0         | 5           | 0      |
      | 8         | 4           | 2      |
      | 15        | 5           | 3      |

  Scenario: Divide by zero
    Given I have a number
    When I try to divide the number by zero
    Then the operation should fail with an error message

  Scenario Outline: Divide by a number close to zero
    Given I have a number <numerator>
    And I have a very small positive number <denominator>
    When I divide <numerator> by <denominator>
    Then the result should be positive infinity

    Examples:
      | numerator | denominator |
      | 10        | 0.0001      |
      | 100       | 0.00001     |

Conclusion​

While AI-generated tests can be a great tool for initial test coverage or catching basic errors, relying solely on them can lead us down the wrong path if we're not careful. The true power lies in combining AI's capabilities with human expertise. By guiding AI with clear requirements and leveraging its strengths in areas like data analysis and automation, we can create a more efficient and effective testing process.

Continuing my series about Serenity/JS: Dive into the potent pairing of Serenity/JS and Playwright. Witness how this winning combination revolutionizes component testing. Ready to elevate your testing game?

Playwright introduces an experimental component testing approach. In this tutorial, we'll explore how to seamlessly integrate component testing into a React/Next.js application using Playwright in conjunction with Serenity/JS, a powerful testing framework.

Setting Up an Example Application​

Before diving into testing, let's establish a baseline React application. Following the recommended guidelines, we'll leverage Next.js as our framework of choice.

We'll initialize a Next.js application in non-interactive mode, incorporating TypeScript and Tailwind CSS.

npx create-next-app@latest next-sjs-ct-tutorial --ts --tailwind --eslint --app --src-dir --import-alias "@/*"
cd next-sjs-ct-tutorial
npm run dev[def]

Once completed, your newly created Next.js application will be accessible at http://localhost:3000/.

volta pin node@latest
git add .
git commit -m "Pinned Node.js version using Volta"

Now, let's enhance our basic page by creating a small application with the following user story:

Create a ./src/components/Counter.tsx file within your project to implement this functionality.

'use client'

import { useState } from "react"

export const Counter = () => {
    // Define state variables for count and minimum/maximum values
    const [count, setCount] = useState(0)
    const minCount = 0
    const maxCount = 10
    
    // Function to handle click on the "Increase" button
    function onIncreaseButtonClick(): void {
        // Check if count is less than maximum allowed count
        if (count < maxCount) {
            // Increment count by 1
            setCount(count + 1)
        }
    }

    // Function to handle click on the "Decrease" button
    function onDecreaseButtonClick(): void {
        // Check if count is greater than minimum allowed count
        if (count > minCount) {
            // Decrement count by 1
            setCount(count - 1)
        }
    }

    // Define button styles for normal, hover, and disabled states
    const buttonStyle = "bg-blue-500 text-white font-bold py-2 px-4 rounded"
    const hoverStyle = "hover:bg-blue-700"
    const disabledButtonStyle = "bg-gray-400 cursor-not-allowed"

    return (
        <>
          {/* Display the current count */}
          <div className="flex space-x-2">
            <div>Count:</div>
            <div
                data-testid="count">
                {count}
            </div>
          </div>
          {/* Render buttons for incrementing and decrementing the count */}
          <div className="flex space-x-4">
            {/* Increase button */}
            <button 
                className={`${buttonStyle} ${count >= maxCount ? disabledButtonStyle : hoverStyle}`}
                onClick={onIncreaseButtonClick}
                disabled={count === maxCount}
                data-testid="button-increase">
                Increase
            </button>
            {/* Decrease button */}
            <button 
                className={`${buttonStyle} ${count === minCount ? disabledButtonStyle : hoverStyle}`}
                onClick={onDecreaseButtonClick}
                disabled={count === minCount }
                data-testid="button-decrease">
                Decrease
            </button>
           </div>
        </>
    )
}

Update ./src/app/page.tsx to include the Counter component on the main page:

import { Counter } from "@/components/Counter";

export default function Home() {
  return (
    <main className="flex min-h-screen flex-col items-center p-24 space-y-4">
      <Counter />
    </main>
  );
}

With these changes, you can now utilize the increase/decrease counter functionality on your Next.js page.

git add .
git commit -m "Implemented Counter"

Integrating Playwright and Serenity/JS​

Now, let's integrate the necessary dependencies for testing, including Playwright and Serenity/JS, along with associated packages for assertions and reporting.

npm install --save-dev \
  @serenity-js/assertions \
  @serenity-js/console-reporter \
  @serenity-js/core \
  @serenity-js/playwright \
  @serenity-js/playwright-test \
  @serenity-js/serenity-bdd \
  @serenity-js/web \
  @playwright/experimental-ct-react \ 
  rimraf \
  npm-failsafe

Update the package.json file with scripts for cleaning, executing tests, and generating reports using Serenity/JS.

[...]

"scripts": { 

    [...]

    "clean": "rimraf dist target",
    "serenity-bdd:update": "serenity-bdd update",
    "test:ct": "failsafe clean serenity-bdd:update test:execute test:report",
    "test:execute": "playwright test -c playwright-ct.config.ts",
    "test:report": "serenity-bdd run --features ./src/components"
}

[

...]

Define the Playwright configuration file playwright-ct.config.ts, which includes settings for test directories, reporters, browser configurations, and other options for running the tests.

import { defineConfig, devices } from '@playwright/experimental-ct-react'
import { SerenityOptions } from '@serenity-js/playwright-test'

/**
 * See https://playwright.dev/docs/test-configuration.
 */
export default defineConfig<SerenityOptions>({
  testDir: './src/components',
  /* The base directory, relative to the config file, for snapshot files created with toMatchSnapshot and toHaveScreenshot. */
  snapshotDir: './__snapshots__',
  /* Maximum time one test can run for. */
  timeout: 10 * 1000,
  /* Run tests in files in parallel */
  fullyParallel: true,
  /* Fail the build on CI if you accidentally left test.only in the source code. */
  forbidOnly: !!process.env.CI,
  /* Retry on CI only */
  retries: process.env.CI ? 2 : 0,
  /* Opt out of parallel tests on CI. */
  workers: process.env.CI ? 1 : undefined,
  /* Reporter to use. See https://playwright.dev/docs/test-reporters */
  reporter: [
    [ 'line' ],
    [ 'html', { open: 'never' } ],
    [ '@serenity-js/playwright-test', {
      crew: [
        [ '@serenity-js/serenity-bdd', { specDirectory: './src/components' } ],
        '@serenity-js/console-reporter',
        [ '@serenity-js/core:ArtifactArchiver', { outputDirectory: './target/site/serenity' } ],
        // [ '@serenity-js/core:StreamReporter', { outputFile: './target/events.ndjson' }]
      ],
    } ],
  ],

  /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
  use: {
    /* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
    trace: 'on-first-retry',

    /* Port to use for Playwright component endpoint. */
    ctPort: 3100,

    /* Set headless: false to see the browser window */
    headless: true,

    crew: [
      [ '@serenity-js/web:Photographer', {
        strategy: 'TakePhotosOfInteractions'
      } ]
    ],
    defaultActorName: 'Tess',
  },

  /* Configure projects for major browsers */
  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },
    {
      name: 'firefox',
      use: { ...devices['Desktop Firefox'] },
    },
    {
      name: 'webkit',
      use: { ...devices['Desktop Safari'] },
    },
  ],
})

In this code block, tasks and page elements for Serenity/JS are defined to ensure the functionality of the counter component. Tasks include checking button states, verifying count values, and other assertions related to the counter component.

import { Ensure, contain, equals, not } from "@serenity-js/assertions";
import { Task } from "@serenity-js/core";
import { By, PageElement, isEnabled, Text, CssClasses } from "@serenity-js/web";

export class CounterSerenity {
    static CounterLabel = PageElement
        .located(By.css('[data-testid="count"]'))
        .describedAs('label that shows the current count')

    static IncreaseButton = PageElement
        .located(By.css('[data-testid="button-increase"]'))
        .describedAs('button to increase the count')

    static DecreaseButton = PageElement
        .located(By.css('[data-testid="button-decrease"]'))
        .describedAs('button to decrease the count')

    static EnsureIncreaseButtonIsEnabled = () =>
        Task.where('#actor ensures that increase button is enabled',
            Ensure.that(
                CssClasses.of(this.IncreaseButton),
                not(contain('cursor-not-allowed')))
        )

    static EnsureIncreaseButtonIsNotEnabled = () =>
        Task.where('#actor ensures that increase button is not enabled',
            Ensure.that(
                CssClasses.of(this.IncreaseButton),
                contain('cursor-not-allowed'))
        )

    static EnsureDecreaseButtonIsEnabled = () =>
        Task.where('#actor ensures that decrease button is enabled',
            Ensure.that(
                CssClasses.of(this.DecreaseButton),
                not(contain('cursor-not-allowed')))
        )

    static EnsureDecreaseButtonIsNotEnabled = () =>
        Ensure.that(
            CssClasses.of(this.DecreaseButton),
            (contain('cursor-not-allowed'))
        )

    static EnsureCount = (count: string) =>
        Task.where(`#actor ensures count label displays a value of ${count}`,
            Ensure.that(Text.of(this.CounterLabel), equals(count)))
}

This section contains the test specifications for the Counter component using Playwright and Serenity/JS. Tests cover scenarios such as initial state verificatio and interactions to reach upper count limits while ensuring button states and count values are correctly maintained.

import { test as componentTest } from '@playwright/experimental-ct-react'
import { useBase } from '@serenity-js/playwright-test'
import { Click, PageElement } from '@serenity-js/web'
import React from 'react'
import { Counter } from './Counter'
import { CounterSerenity } from './CounterSerenity'
import { Duration, Wait } from '@serenity-js/core'


// eslint-disable-next-line react-hooks/rules-of-hooks
const { it, describe } = useBase(componentTest)

describe('Counter', () => {

    it('ensures initial state', async ({ mount, actor }) => {

        const x = PageElement.from(await mount(
            <Counter />
        )).describedAs('counter component')

        await actor.attemptsTo(
            CounterSerenity.EnsureIncreaseButtonIsEnabled(),
            CounterSerenity.EnsureDecreaseButtonIsNotEnabled(),
            CounterSerenity.EnsureCount('0')
        )
    })

    it('ensures state after click to upper limit', async ({ mount, actor }) => {

        const x = PageElement.from(await mount(
            <Counter />
        )).describedAs('counter component')

        await actor.attemptsTo(
            CounterSerenity.EnsureIncreaseButtonIsEnabled(),
            CounterSerenity.EnsureDecreaseButtonIsNotEnabled(),
            CounterSerenity.EnsureCount('0'),
            Click.on(CounterSerenity.IncreaseButton),
            CounterSerenity.EnsureIncreaseButtonIsEnabled(),
            CounterSerenity.EnsureDecreaseButtonIsEnabled(),
            CounterSerenity.EnsureCount('1'),
            Click.on(CounterSerenity.IncreaseButton),
            Click.on(CounterSerenity.IncreaseButton),
            Click.on(CounterSerenity.IncreaseButton),
            Click.on(CounterSerenity.IncreaseButton),
            Click.on(CounterSerenity.IncreaseButton),
            CounterSerenity.EnsureIncreaseButtonIsEnabled(),
            CounterSerenity.EnsureDecreaseButtonIsEnabled(),
            CounterSerenity.EnsureCount('6'),
            Click.on(CounterSerenity.IncreaseButton),
            Click.on(CounterSerenity.IncreaseButton),
            Click.on(CounterSerenity.IncreaseButton),
            Click.on(CounterSerenity.IncreaseButton),
            CounterSerenity.EnsureCount('10'),
            CounterSerenity.EnsureIncreaseButtonIsNotEnabled(),
            CounterSerenity.EnsureDecreaseButtonIsEnabled(),
        )
    })
})

Finally, include a basic HTML template index.html for the testing page, and create an entry point (index.tsx) for rendering React components.

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Testing Page</title>
  </head>
  <body>
    <div id="root"></div>
    <script type="module" src="index.tsx"></script>
  </body>
</html>

import '../src/app/globals.css'

With everything set up, you can run the tests using npm run test:ct, and find the Serenity-BDD report under ./target/site/serenity.

Before committing the latest changes, update your .gitignore file:

# serenity-js
/target/site

# playwright
/playwright-report
/playwright/.cache

This concludes our tutorial on simplifying component testing with React/Next.js, Playwright, and Serenity/JS. Happy testing!

Ever feel like your software teams are speaking different languages when it comes to define concrete examples? But here's the spoiler: there's no magical tool to solve it — just the power of collaboration. Get ready for a paradigm shift towards cohesive teamwork!

Effective collaboration among Developers, Testers, and Product Owners (POs) in software development is crucial for success. However, one common hurdle in this collaboration is the struggle to find concrete examples to guide development and testing. Overcoming this challenge requires patience, training, and a team-driven approach , without imposing strict directives.

Image by martynaszulist

The Reasons Behind the Struggle

First and foremost it's important to understand the reasons behind this struggle. Here are some of them:

1. Unclear Requirements: Incomplete or vague requirements lead to confusion and differing interpretations, making it hard to articulate clear examples.

2. Communication Barriers: Technical differences, language nuances, or cultural diversity can hinder clear communication, causing misunderstandings in example definition.

3. Differing Perspectives: Each stakeholder brings unique viewpoints, sometimes leading to discordance and difficulty in agreeing on examples.

4. Assumptions and Implicit Knowledge: Unspoken assumptions can obscure clarity and perpetuate misunderstandings.

5. Complexity of Features: Some features are inherently complex, making it challenging to cover all scenarios with concise examples.

6. Time Constraints: Pressures to deliver quickly can limit thorough discussion and exploration of examples.

7. Inadequate Documentation: Poorly documented or outdated examples add to confusion and inconsistency.

Steps to Overcome the Struggle

1. Clarify Requirements Collaboratively: Hold regular refinement sessions where team members openly discuss and clarify requirements, fostering an environment where everyone's input is valued.

2. Use Techniques for Requirement Elicitation: Employ methods like user story mapping and behavior-driven development to uncover hidden requirements and ensure alignment with user needs.

3. Document and Maintain Examples: Establish a shared repository for documenting examples, ensuring accessibility and encouraging everyone to contribute and maintain it for consistency.

By taking these steps, teams can enhance collaboration, improve communication, and overcome the challenge of finding examples effectively in software development.

The Fear of Incomplete Examples

While collaboration is key, it's important to acknowledge the hesitancy some stakeholders may have in providing examples. They sometimes fear that the examples they give might not fully capture the complexities of the final solution.

They understand that an example is just that — an example — and worry that it may not encompass every aspect of the desired outcome. However, it's essential to emphasize to stakeholders that starting with simple, straightforward examples is a crucial first step.

These initial examples lay the foundation upon which more complex scenarios can be built in subsequent iterations. By understanding that examples are iterative and can evolve over time, stakeholdes can feel more confident in contributing to the collaborative process.

Conclusion

The ability to define concrete examples is a cornerstone of effective collaboration. By acknowledging the hurdles and embracing the power of teamwork, teams can transcend language barriers and navigate through the complexities of requirements with confidence. Through iterative refinement, stakeholders can contribute valuable insights, ensuring that examples evolve to accurately reflect the desired outcomes.

So, while there may not be a magical tool to solve the challenge of defining examples, the collective efforts of dedicated teams pave the way for innovation, efficiency, and ultimately, the delivery of exceptional software solutions.

When I chose Serenity/JS as a vendor-agnostic API, my intention was to isolate my test specification from the test runner. Today, I seamlessly transitioned from Protractor to Playwright in less than 10 minutes. The kicker? No need for me, any AI, or non-AI tool to lay a finger on selectors or test specs. All it took was a swift config update.

In this blog post, we'll explore a seamless transition from Protractor to Playwright in a Serenity/JS project, all achieved in a mere 10 minutes. Witness how the power of Serenity/JS makes this migration effortless, requiring no modifications to your selectors or Jasmine test specifications. Let's dive in and unlock a smoother, more efficient testing experience.

Prepare Example project​

(Example on this page was tested with Serenity/JS version 3.16.0)

Let's start by setting up a small example project based on the Protractor with Jasmine template provided by Serenity/JS, as this combination was the most common.

Clone the serenity-js-jasmine-protractor-template, install the NodeJS dependencies, and start a test run to ensure that the template is working.

git clone https://github.com/serenity-js/serenity-js-jasmine-protractor-template.git
cd serenity-js-jasmine-protractor-template
npm install
npm test

Migration​

Additional Dependencies​

Add additional dependencies to your project:

npm install @serenity-js/playwright
npm install playwright
npm install mocha
npm install @serenity-js/mocha

Install Playwright:

npx playwright install

Mocha Configuration​

Add a .mocharc.yml file to the root directory of you project:

check-leaks: false
color: true
diff: true
full-trace: true
reporter: '@serenity-js/mocha'
require:
  - 'ts-node/register'
  - 'src/serenity.config.ts'
timeout: 30000
v8-stack-trace-limit: 100
spec:
  - './spec/**/*.spec.ts'

Serenity/JS Actors​

Create a src folder in your project root and create an Actors.ts file:

import { Actor, Cast, TakeNotes } from '@serenity-js/core';
import { BrowseTheWebWithPlaywright, PlaywrightOptions } from '@serenity-js/playwright';
import { CallAnApi } from '@serenity-js/rest';
import * as playwright from 'playwright-core';

export class Actors implements Cast {
    constructor(
        private readonly page: playwright.Page,
        private readonly options: PlaywrightOptions,
    ) {
    }

    prepare(actor: Actor): Actor {
        return actor.whoCan(
            BrowseTheWebWithPlaywright.usingPage(this.page, this.options),
            TakeNotes.usingAnEmptyNotepad(),
            CallAnApi.at(this.options.baseURL)
        );
    }
}

Serenity/JS Configuration​

Create a serenity.config.ts file in the same src folder.

import { configure, Duration, NoOpDiffFormatter } from '@serenity-js/core';
import { BrowserContextOptions } from 'playwright';
import * as playwright from 'playwright-core';

import { Actors } from './Actors';

let browser: playwright.Browser;
let page: playwright.Page;

export const mochaHooks = {
    async beforeAll(): Promise<void> {
        browser = await playwright.chromium.launch({
            headless: false
        });

        const contextOptions : BrowserContextOptions = { baseURL: 'https://serenity-js.org' }
        const context = await browser.newContext(contextOptions);
        page = await context.newPage();

        configure({
            actors: new Actors(page, {
                defaultNavigationTimeout:   Duration.ofSeconds(2).inMilliseconds(),
                defaultTimeout:             Duration.ofMilliseconds(750).inMilliseconds(),
            }),
            diffFormatter: new NoOpDiffFormatter(),
            crew: [
                '@serenity-js/console-reporter',
                [ '@serenity-js/serenity-bdd',          { specDirectory: 'spec' }], 
                [ '@serenity-js/web:Photographer',      { strategy: 'TakePhotosOfInteractions'    } ],
                [ '@serenity-js/core:ArtifactArchiver', { outputDirectory: 'target/site/serenity' } ],
            ]
        });
    },

    async afterAll(): Promise<void> {
        if (browser) {
            await browser.close()
        }
    }
}

In our example Actors class, we initiate a Browser object, create a new Context with a new Page and give the Actor the Ability to interact with this page.

Angular Sychronisation in Non-Angular Project​

In the serenity-js_website.spec.ts example file, remove the UseAngularSynchronisation, as the page we're interacting with is not even an Angular page.

Uncomment the following line in protractor.conf.ts. so we will be able to use Protractor and Playwright side by side.

    onPrepare: function() {
         ...
         browser.waitForAngularEnabled(false);
    },

Additional npm Scripts​

In the script section of your package.json and two new scripts:

...
  "scripts": {
    ...
    "test:execute:playwright": "mocha --config .mocharc.yml",
    "test:playwright": "failsafe clean test:execute:playwright test:report",
    ... 
  }

First Testrun Using Playwright​

Congratulations! You're done. Just run npm run test:playwright. You can still run the tests with Protractor by running npm test. If you're satisfied with Playwright, you can remove all dependencies from Protractor and Jasmine from your project now.

Conclusion​

In just a brief span, we've navigated the transition from Protractor to Playwright with Serenity/JS. The beauty lies not just in the simplicity of the process but in the retained integrity of your existing codebase. As you embark on your testing endeavors with this upgraded setup, feel empowered by the speed, efficiency, and flexibility now at your fingertips. The synergy of Serenity/JS's familiarity, Playwright's power, and Serenity/JS's adaptability ensures a testing ecosystem that not only meets but exceeds your expectations. Happy testing!

Unravel the Complexity of HTML Table Automation with Serenity/JS and Playwright! Tired of wrestling with hierarchical selectors? Serenity/JS introduces the Page Element Query Language (PEQL), making HTML table automation a breeze. Dive into hands-on examples, from pinpointing rows by index to dynamically locating them based on content. Empower your web automation journey with Serenity/JS – where simplicity meets precision!

Automating HTML Tables with Serenity/JS​

If you're dealing with web automation and HTML tables, Serenity/JS provides a powerful and expressive way to interact with page elements.

Traditional HTML selectors, such as those based on CSS or XPath, are often hierarchical. However, tables are conceptually two-dimensional, with rows and columns, making it less straightforward to express certain queries using these selectors.

In this article, we'll explore the usage of the Page Element Query Language (PEQL) in Serenity/JS for automating HTML tables, focusing on

• scenarios where you know the index of the row you want to access and
• situations where you need to locate a row based on its content.

Knowing the Index of the Row​

Consider a scenario where you have a simple table, like this one from w3schools.com, and you know the index of the row you want to access.

<table>
  <tr>
    <th>Company</th>
    <th>Contact</th>
    <th>Country</th>
  </tr>
  <tr>
    <td>Alfreds Futterkiste</td>
    <td>Maria Anders</td>
    <td>Germany</td>
  </tr>
  <tr>
    <td>Centro comercial Moctezuma</td>
    <td>Francisco Chang</td>
    <td>Mexico</td>
  </tr>
  <tr>
    <td>Ernst Handel</td>
    <td>Roland Mendel</td>
    <td>Austria</td>
  </tr>
  <tr>
    <td>Island Trading</td>
    <td>Helen Bennett</td>
    <td>UK</td>
  </tr>
  <tr>
    <td>Laughing Bacchus Winecellars</td>
    <td>Yoshi Tannamuri</td>
    <td>Canada</td>
  </tr>
  <tr>
    <td>Magazzini Alimentari Riuniti</td>
    <td>Giovanni Rovelli</td>
    <td>Italy</td>
  </tr>
</table>

The following example demonstrates how to use Serenity/JS to interact with the table:

import { Ensure, equals } from '@serenity-js/assertions';
import { describe, it } from '@serenity-js/playwright-test';
import { By, Navigate, PageElements, Text } from '@serenity-js/web';

const CustomersTable = () => PageElement.located(By.css('#customers'))
    .describedAs('customer table')

const TableRows = () => PageElements.located(By.css('tr'))
    .describedAs('all table rows').of(CustomersTable());

const TableColumns = () => PageElements.located(By.css('td'))
    .describedAs('all table columns')
    .of(CustomersTable())

const TableColumnsInRow = (index: number) => 
    TableColumns().of(TableRows().nth(index));

const CompanyInRow = (index: number) => TableColumnsInRow(index).nth(0)
    .describedAs(`company column in row ${index}`);

const ContactInRow = (index: number) => TableColumnsInRow(index).nth(1)
    .describedAs(`contact column in row ${index}`);

const CountryInRow = (index: number) => TableColumnsInRow(index).nth(2)
    .describedAs(`country column in row ${index}`);

describe('table selector', () => (
    it('ensures text of columns in a row', async ({actor}) => (
        actor.attemptsTo(
            Navigate.to('https://www.w3schools.com/html/html_tables.asp'),
            Ensure.that(Text.of(CompanyInRow(1)), equals('Alfreds Futterkiste')),
            Ensure.that(Text.of(ContactInRow(1)), equals('Maria Anders')),
            Ensure.that(Text.of(CountryInRow(1)), equals('Germany'))
        ))
    )
));