4 posts tagged with "BDD"

Next Chapter: Don't reinvent the wheel! Discover how to seamlessly integrate BDD scenarios into your existing Serenity/JS and Playwright setup, unlocking the power of code reuse across component testing and user acceptance tests. Say goodbye to redundant work and hello to efficient testing.

In the last tutorial, I showed how to do component testing using Serenity/JS and Playwright. To be honest, it's a very simple component, making it difficult to create a real-world example. Let's just assume it's a coffee counter for controlling how many cups of coffee you drink.

Component testing ensures that your self-contained components work as expected. For user acceptance, there may be some additional things necessary to verify functionality.

The good news is: You can reuse what you've already prepared for the component tests and dive into the user scenarios.

In this article, I will demonstrate how we can integrate BDD scenarios into the project I prepared the last time. I will use Serenity/JS BDD features. These BDD scenarios will run in the same project like the component tests and will reuse Serenity/JS PageElements and Tasks that we already used in the component tests.

I cloned the last example and updated all packages to the latest versions to be up to date and created a new repository. If you want to follow all the steps from scratch I suggest going back and starting with the component testing tutorial and coming back once you've finished.

To start, you've to add some additional dev dependencies to your project:

npm install --save-dev @cucumber/cucumber 
npm install --save-dev @serenity-js/cucumber

This time I will not list the file content in detail but refer to the repository and some explanations.

Key to using BDD and Serenity/JS with cucumber is the features folder.

features
|- coffee-counter
|  |- coffee-counter.feature
|  \- readme.md
|
|- step-definitions
|  |- counter.steps.ts
|  \- parameter.steps.ts
|
|- support
|  |- Actors.ts
|  \- serenity.config.ts

To enable Serenity/JS, we need the support folder with the Actors.ts and the serenity.config.ts.

To run cucumber, add a cucumber.js configuration file to your project root. Among other things, we configure to use Serenity/JS Cucumber adapter in here.

Add two new scripts to the package.json

"scripts": {
    ...
    "test:cucumber:execute": "cucumber-js",
    "test:cucumber": "failsafe clean serenity-bdd:update test:cucumber:execute test:report"
}

There is still one configuration thing left: The create-next-app from the last tutorial created a tsconfig.json for us. Some compile options are not compatible with cucumber.js, and we have to include the features folder.

{
  "compilerOptions":
   {
     "module": "commonjs",
     "moduleResolution": "node",
     "esModuleInterop": true,
   },
   {
    "include": [
      "features/**/*.ts",
    ]
   }
}

To not interfere with the original Next.js configuration, I create a copy tsconfig.next.ts. I configured the Next.js itself to use this configuration in next.config.mjs instead of the tsconfig.ts file, just to make sure we still use the right config when building the Next.js app.

/** @type {import('next').NextConfig} */
const nextConfig = {
    typescript: { tsconfigPath: 'tsconfig.next.json'}
};

export default nextConfig;

Until now, we just did some configuration tweaks. Now we're ready to take a look back into our coffee counter scenario again. Now have a look into the counter steps definitions. Here is where the magic happens. We just reuse the same tasks we already provided for component testing:

import { Given, Then, When } from '@cucumber/cucumber';
import { Actor } from '@serenity-js/core';

import { CounterSerenity } from '../../src/components/CounterSerenity';

Then('{pronoun} should see that the "Increase" button is enabled', async (actor: Actor) =>
    actor.attemptsTo(
        CounterSerenity.EnsureIncreaseButtonIsEnabled()
    ))

You're nearly ready to run the test the first time. Make sure that you start your server with npm run dev on port 3000.

Then run npm run test:cucumber and find the BDD report in the target/site folder.

Did you know that many software defects arise from unclear requirements? What we're seeing today is AI attempting to assist in test creation, promising to increase code coverage. However, this approach often emphasizes testing our implementations rather than testing against our requirements, which I believe has its value but also carries certain risks that must be carefully considered.

Don't test your implementation​

To illustrate, let's conduct an experiment using our familiar chatbots. I'll present a simple function implementation suitable for our demonstration, intentionally flawed.

This is a TS function. Can you generate me a jest test spec for it?

function addTwoNumbers(a: number, b: number): number {
    return a / b
}

Here's, what ChatGPT 3.5 responded:

Sure, here's a Jest test spec for the addTwoNumbers function:

import { addTwoNumbers } from './yourFileName';

describe('addTwoNumbers', () => {
    it('should correctly add two numbers', () => {
        // Arrange
        const a = 5;
        const b = 3;
        
        // Act
        const result = addTwoNumbers(a, b);

        // Assert
        expect(result).toEqual(8);
    });

    it('should handle negative numbers correctly', () => {
        // Arrange
        const a = -10;
        const b = 7;

        // Act
        const result = addTwoNumbers(a, b);

        // Assert
        expect(result).toEqual(-3);
    });

    it('should handle zero correctly', () => {
        // Arrange
        const a = 0;
        const b = 5;

        // Act
        const result = addTwoNumbers(a, b);

        // Assert
        expect(result).toEqual(0);
    });

    // Add more test cases as needed
});

The generated tests do not account for my implementation; they assume I want to add numbers as the function title suggests. The test will fail. However, this allows me to challenge my code.

Here's, what Gemini 1.0 answered:

I'd be glad to provide a Jest test spec for the addTwoNumbers function, incorporating the valuable insights from the ratings and addressing the identified issues:

import { addTwoNumbers } from './path/to/your/function'; // Replace with the actual path

describe('addTwoNumbers', () => {
  // Test case 1: Basic addition with positive numbers
  it('should add two positive numbers correctly', () => {
    expect(addTwoNumbers(5, 3)).toBe(5 / 3); // Corrected the expected result
  });

  // Test case 2: Addition with negative numbers
  it('should handle addition with negative numbers', () => {
    expect(addTwoNumbers(-2, 4)).toBe(-2 / 4); // Corrected the expected result
  });

  // Test case 3: Division by zero (error handling)
  it('should throw an error when dividing by zero', () => {
    expect(() => addTwoNumbers(5, 0)).toThrowError('Division by zero'); // Handle division by zero appropriately
  });

  // Test case 4: Data type validation (optional)
  it('should only accept numbers as arguments (optional)', () => {
    expect(() => addTwoNumbers('five', 3)).toThrowError(
      'addTwoNumbers: Arguments must be numbers' // Customized error message
    );

    expect(() => addTwoNumbers(5, 'three')).toThrowError(
      'addTwoNumbers: Arguments must be numbers' // Customized error message
    );
  });
});

Gemini generated tests that pass. The test reads like an addition is tested, but effectively, it's the division.

To be fair, sometimes the bots detect inconsistencies between the function name and implementation.

However, we miss the point here:

What's unknown is if our customer was requiring an addition or a division.

Either the function implementation is incorrect and should be a + b, or there's a naming issue, and it should be named divideTwoNumbers().

So, what are the takeaways?

1. Testing should align with requirements and expected behavior

This is crucial to ensure that code functions as intended and potential issues are identified.

2. Importance of human judgment

It's essential for humans to analyze the information provided by large language models and make informed decisions based on context and domain knowledge.

Align your tests with your requirements​

While I don't want to leave you without a solution to this problem, it's worth noting that you should utilize AI to assist in testing.

However, it's important to focus on explaining the purpose of the test and the problem you want to solve, rather than just testing the mplementation itself.

While this principle is not new, it's crucial to remember, as it's something that can be overlooked when rushing into the use of AI.

Can you write me a jest test spec that test test a typescript function that takes two numbers and divide them. Please cover edge cases.

import { divideNumbers } from './yourDivideFunction'; // Import your divide function

describe('divideNumbers function', () => {
  it('should divide two positive numbers correctly', () => {
    expect(divideNumbers(10, 2)).toBe(5);
  });

  it('should divide a positive number by zero and return Infinity', () => {
    expect(divideNumbers(10, 0)).toBe(Infinity);
  });

  it('should divide a negative number by a positive number correctly', () => {
    expect(divideNumbers(-10, 2)).toBe(-5);
  });

  it('should divide zero by a positive number and return 0', () => {
    expect(divideNumbers(0, 10)).toBe(0);
  });

  it('should return NaN if divided by zero', () => {
    expect(divideNumbers(0, 0)).toBe(NaN);
  });

  it('should divide a large number by a small number correctly', () => {
    expect(divideNumbers(1000000, 0.000001)).toBe(1e+12);
  });

  it('should handle decimal precision properly', () => {
    expect(divideNumbers(1, 3)).toBeCloseTo(0.333, 3); // Rounded to 3 decimal places
  });

  // Add more test cases as needed for your specific requirements
});

With the right input, you can even get out BDD scenarios in Gherkin:

Feature: Division of Two Numbers
  As a user
  I want to divide two numbers
  So that I can perform mathematical operations accurately

  Scenario Outline: Divide two numbers
    Given I have two numbers <numerator> and <denominator>
    When I divide <numerator> by <denominator>
    Then the result should be <result>

    Examples:
      | numerator | denominator | result |
      | 6         | 3           | 2      |
      | 10        | 2           | 5      |
      | 0         | 5           | 0      |
      | 8         | 4           | 2      |
      | 15        | 5           | 3      |

  Scenario: Divide by zero
    Given I have a number
    When I try to divide the number by zero
    Then the operation should fail with an error message

  Scenario Outline: Divide by a number close to zero
    Given I have a number <numerator>
    And I have a very small positive number <denominator>
    When I divide <numerator> by <denominator>
    Then the result should be positive infinity

    Examples:
      | numerator | denominator |
      | 10        | 0.0001      |
      | 100       | 0.00001     |

Conclusion​

While AI-generated tests can be a great tool for initial test coverage or catching basic errors, relying solely on them can lead us down the wrong path if we're not careful. The true power lies in combining AI's capabilities with human expertise. By guiding AI with clear requirements and leveraging its strengths in areas like data analysis and automation, we can create a more efficient and effective testing process.

Continuing my series about Serenity/JS: Dive into the potent pairing of Serenity/JS and Playwright. Witness how this winning combination revolutionizes component testing. Ready to elevate your testing game?

Playwright introduces an experimental component testing approach. In this tutorial, we'll explore how to seamlessly integrate component testing into a React/Next.js application using Playwright in conjunction with Serenity/JS, a powerful testing framework.

Setting Up an Example Application​

Before diving into testing, let's establish a baseline React application. Following the recommended guidelines, we'll leverage Next.js as our framework of choice.

We'll initialize a Next.js application in non-interactive mode, incorporating TypeScript and Tailwind CSS.

npx create-next-app@latest next-sjs-ct-tutorial --ts --tailwind --eslint --app --src-dir --import-alias "@/*"
cd next-sjs-ct-tutorial
npm run dev[def]

Once completed, your newly created Next.js application will be accessible at http://localhost:3000/.

volta pin node@latest
git add .
git commit -m "Pinned Node.js version using Volta"

Now, let's enhance our basic page by creating a small application with the following user story:

Create a ./src/components/Counter.tsx file within your project to implement this functionality.

'use client'

import { useState } from "react"

export const Counter = () => {
    // Define state variables for count and minimum/maximum values
    const [count, setCount] = useState(0)
    const minCount = 0
    const maxCount = 10
    
    // Function to handle click on the "Increase" button
    function onIncreaseButtonClick(): void {
        // Check if count is less than maximum allowed count
        if (count < maxCount) {
            // Increment count by 1
            setCount(count + 1)
        }
    }

    // Function to handle click on the "Decrease" button
    function onDecreaseButtonClick(): void {
        // Check if count is greater than minimum allowed count
        if (count > minCount) {
            // Decrement count by 1
            setCount(count - 1)
        }
    }

    // Define button styles for normal, hover, and disabled states
    const buttonStyle = "bg-blue-500 text-white font-bold py-2 px-4 rounded"
    const hoverStyle = "hover:bg-blue-700"
    const disabledButtonStyle = "bg-gray-400 cursor-not-allowed"

    return (
        <>
          {/* Display the current count */}
          <div className="flex space-x-2">
            <div>Count:</div>
            <div
                data-testid="count">
                {count}
            </div>
          </div>
          {/* Render buttons for incrementing and decrementing the count */}
          <div className="flex space-x-4">
            {/* Increase button */}
            <button 
                className={`${buttonStyle} ${count >= maxCount ? disabledButtonStyle : hoverStyle}`}
                onClick={onIncreaseButtonClick}
                disabled={count === maxCount}
                data-testid="button-increase">
                Increase
            </button>
            {/* Decrease button */}
            <button 
                className={`${buttonStyle} ${count === minCount ? disabledButtonStyle : hoverStyle}`}
                onClick={onDecreaseButtonClick}
                disabled={count === minCount }
                data-testid="button-decrease">
                Decrease
            </button>
           </div>
        </>
    )
}

Update ./src/app/page.tsx to include the Counter component on the main page:

import { Counter } from "@/components/Counter";

export default function Home() {
  return (
    <main className="flex min-h-screen flex-col items-center p-24 space-y-4">
      <Counter />
    </main>
  );
}

With these changes, you can now utilize the increase/decrease counter functionality on your Next.js page.

git add .
git commit -m "Implemented Counter"

Integrating Playwright and Serenity/JS​

Now, let's integrate the necessary dependencies for testing, including Playwright and Serenity/JS, along with associated packages for assertions and reporting.

npm install --save-dev \
  @serenity-js/assertions \
  @serenity-js/console-reporter \
  @serenity-js/core \
  @serenity-js/playwright \
  @serenity-js/playwright-test \
  @serenity-js/serenity-bdd \
  @serenity-js/web \
  @playwright/experimental-ct-react \ 
  rimraf \
  npm-failsafe

Update the package.json file with scripts for cleaning, executing tests, and generating reports using Serenity/JS.

[...]

"scripts": { 

    [...]

    "clean": "rimraf dist target",
    "serenity-bdd:update": "serenity-bdd update",
    "test:ct": "failsafe clean serenity-bdd:update test:execute test:report",
    "test:execute": "playwright test -c playwright-ct.config.ts",
    "test:report": "serenity-bdd run --features ./src/components"
}

[

...]

Define the Playwright configuration file playwright-ct.config.ts, which includes settings for test directories, reporters, browser configurations, and other options for running the tests.

import { defineConfig, devices } from '@playwright/experimental-ct-react'
import { SerenityOptions } from '@serenity-js/playwright-test'

/**
 * See https://playwright.dev/docs/test-configuration.
 */
export default defineConfig<SerenityOptions>({
  testDir: './src/components',
  /* The base directory, relative to the config file, for snapshot files created with toMatchSnapshot and toHaveScreenshot. */
  snapshotDir: './__snapshots__',
  /* Maximum time one test can run for. */
  timeout: 10 * 1000,
  /* Run tests in files in parallel */
  fullyParallel: true,
  /* Fail the build on CI if you accidentally left test.only in the source code. */
  forbidOnly: !!process.env.CI,
  /* Retry on CI only */
  retries: process.env.CI ? 2 : 0,
  /* Opt out of parallel tests on CI. */
  workers: process.env.CI ? 1 : undefined,
  /* Reporter to use. See https://playwright.dev/docs/test-reporters */
  reporter: [
    [ 'line' ],
    [ 'html', { open: 'never' } ],
    [ '@serenity-js/playwright-test', {
      crew: [
        [ '@serenity-js/serenity-bdd', { specDirectory: './src/components' } ],
        '@serenity-js/console-reporter',
        [ '@serenity-js/core:ArtifactArchiver', { outputDirectory: './target/site/serenity' } ],
        // [ '@serenity-js/core:StreamReporter', { outputFile: './target/events.ndjson' }]
      ],
    } ],
  ],

  /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
  use: {
    /* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
    trace: 'on-first-retry',

    /* Port to use for Playwright component endpoint. */
    ctPort: 3100,

    /* Set headless: false to see the browser window */
    headless: true,

    crew: [
      [ '@serenity-js/web:Photographer', {
        strategy: 'TakePhotosOfInteractions'
      } ]
    ],
    defaultActorName: 'Tess',
  },

  /* Configure projects for major browsers */
  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },
    {
      name: 'firefox',
      use: { ...devices['Desktop Firefox'] },
    },
    {
      name: 'webkit',
      use: { ...devices['Desktop Safari'] },
    },
  ],
})

In this code block, tasks and page elements for Serenity/JS are defined to ensure the functionality of the counter component. Tasks include checking button states, verifying count values, and other assertions related to the counter component.

import { Ensure, contain, equals, not } from "@serenity-js/assertions";
import { Task } from "@serenity-js/core";
import { By, PageElement, isEnabled, Text, CssClasses } from "@serenity-js/web";

export class CounterSerenity {
    static CounterLabel = PageElement
        .located(By.css('[data-testid="count"]'))
        .describedAs('label that shows the current count')

    static IncreaseButton = PageElement
        .located(By.css('[data-testid="button-increase"]'))
        .describedAs('button to increase the count')

    static DecreaseButton = PageElement
        .located(By.css('[data-testid="button-decrease"]'))
        .describedAs('button to decrease the count')

    static EnsureIncreaseButtonIsEnabled = () =>
        Task.where('#actor ensures that increase button is enabled',
            Ensure.that(
                CssClasses.of(this.IncreaseButton),
                not(contain('cursor-not-allowed')))
        )

    static EnsureIncreaseButtonIsNotEnabled = () =>
        Task.where('#actor ensures that increase button is not enabled',
            Ensure.that(
                CssClasses.of(this.IncreaseButton),
                contain('cursor-not-allowed'))
        )

    static EnsureDecreaseButtonIsEnabled = () =>
        Task.where('#actor ensures that decrease button is enabled',
            Ensure.that(
                CssClasses.of(this.DecreaseButton),
                not(contain('cursor-not-allowed')))
        )

    static EnsureDecreaseButtonIsNotEnabled = () =>
        Ensure.that(
            CssClasses.of(this.DecreaseButton),
            (contain('cursor-not-allowed'))
        )

    static EnsureCount = (count: string) =>
        Task.where(`#actor ensures count label displays a value of ${count}`,
            Ensure.that(Text.of(this.CounterLabel), equals(count)))
}

This section contains the test specifications for the Counter component using Playwright and Serenity/JS. Tests cover scenarios such as initial state verificatio and interactions to reach upper count limits while ensuring button states and count values are correctly maintained.

import { test as componentTest } from '@playwright/experimental-ct-react'
import { useBase } from '@serenity-js/playwright-test'
import { Click, PageElement } from '@serenity-js/web'
import React from 'react'
import { Counter } from './Counter'
import { CounterSerenity } from './CounterSerenity'
import { Duration, Wait } from '@serenity-js/core'


// eslint-disable-next-line react-hooks/rules-of-hooks
const { it, describe } = useBase(componentTest)

describe('Counter', () => {

    it('ensures initial state', async ({ mount, actor }) => {

        const x = PageElement.from(await mount(
            <Counter />
        )).describedAs('counter component')

        await actor.attemptsTo(
            CounterSerenity.EnsureIncreaseButtonIsEnabled(),
            CounterSerenity.EnsureDecreaseButtonIsNotEnabled(),
            CounterSerenity.EnsureCount('0')
        )
    })

    it('ensures state after click to upper limit', async ({ mount, actor }) => {

        const x = PageElement.from(await mount(
            <Counter />
        )).describedAs('counter component')

        await actor.attemptsTo(
            CounterSerenity.EnsureIncreaseButtonIsEnabled(),
            CounterSerenity.EnsureDecreaseButtonIsNotEnabled(),
            CounterSerenity.EnsureCount('0'),
            Click.on(CounterSerenity.IncreaseButton),
            CounterSerenity.EnsureIncreaseButtonIsEnabled(),
            CounterSerenity.EnsureDecreaseButtonIsEnabled(),
            CounterSerenity.EnsureCount('1'),
            Click.on(CounterSerenity.IncreaseButton),
            Click.on(CounterSerenity.IncreaseButton),
            Click.on(CounterSerenity.IncreaseButton),
            Click.on(CounterSerenity.IncreaseButton),
            Click.on(CounterSerenity.IncreaseButton),
            CounterSerenity.EnsureIncreaseButtonIsEnabled(),
            CounterSerenity.EnsureDecreaseButtonIsEnabled(),
            CounterSerenity.EnsureCount('6'),
            Click.on(CounterSerenity.IncreaseButton),
            Click.on(CounterSerenity.IncreaseButton),
            Click.on(CounterSerenity.IncreaseButton),
            Click.on(CounterSerenity.IncreaseButton),
            CounterSerenity.EnsureCount('10'),
            CounterSerenity.EnsureIncreaseButtonIsNotEnabled(),
            CounterSerenity.EnsureDecreaseButtonIsEnabled(),
        )
    })
})

Finally, include a basic HTML template index.html for the testing page, and create an entry point (index.tsx) for rendering React components.

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Testing Page</title>
  </head>
  <body>
    <div id="root"></div>
    <script type="module" src="index.tsx"></script>
  </body>
</html>

import '../src/app/globals.css'

With everything set up, you can run the tests using npm run test:ct, and find the Serenity-BDD report under ./target/site/serenity.

Before committing the latest changes, update your .gitignore file:

# serenity-js
/target/site

# playwright
/playwright-report
/playwright/.cache

This concludes our tutorial on simplifying component testing with React/Next.js, Playwright, and Serenity/JS. Happy testing!

Ever feel like your software teams are speaking different languages when it comes to define concrete examples? But here's the spoiler: there's no magical tool to solve it — just the power of collaboration. Get ready for a paradigm shift towards cohesive teamwork!

Effective collaboration among Developers, Testers, and Product Owners (POs) in software development is crucial for success. However, one common hurdle in this collaboration is the struggle to find concrete examples to guide development and testing. Overcoming this challenge requires patience, training, and a team-driven approach , without imposing strict directives.

Image by martynaszulist

The Reasons Behind the Struggle

First and foremost it's important to understand the reasons behind this struggle. Here are some of them:

1. Unclear Requirements: Incomplete or vague requirements lead to confusion and differing interpretations, making it hard to articulate clear examples.

2. Communication Barriers: Technical differences, language nuances, or cultural diversity can hinder clear communication, causing misunderstandings in example definition.

3. Differing Perspectives: Each stakeholder brings unique viewpoints, sometimes leading to discordance and difficulty in agreeing on examples.

4. Assumptions and Implicit Knowledge: Unspoken assumptions can obscure clarity and perpetuate misunderstandings.

5. Complexity of Features: Some features are inherently complex, making it challenging to cover all scenarios with concise examples.

6. Time Constraints: Pressures to deliver quickly can limit thorough discussion and exploration of examples.

7. Inadequate Documentation: Poorly documented or outdated examples add to confusion and inconsistency.

Steps to Overcome the Struggle

1. Clarify Requirements Collaboratively: Hold regular refinement sessions where team members openly discuss and clarify requirements, fostering an environment where everyone's input is valued.

2. Use Techniques for Requirement Elicitation: Employ methods like user story mapping and behavior-driven development to uncover hidden requirements and ensure alignment with user needs.

3. Document and Maintain Examples: Establish a shared repository for documenting examples, ensuring accessibility and encouraging everyone to contribute and maintain it for consistency.

By taking these steps, teams can enhance collaboration, improve communication, and overcome the challenge of finding examples effectively in software development.

The Fear of Incomplete Examples

While collaboration is key, it's important to acknowledge the hesitancy some stakeholders may have in providing examples. They sometimes fear that the examples they give might not fully capture the complexities of the final solution.

They understand that an example is just that — an example — and worry that it may not encompass every aspect of the desired outcome. However, it's essential to emphasize to stakeholders that starting with simple, straightforward examples is a crucial first step.

These initial examples lay the foundation upon which more complex scenarios can be built in subsequent iterations. By understanding that examples are iterative and can evolve over time, stakeholdes can feel more confident in contributing to the collaborative process.

Conclusion

The ability to define concrete examples is a cornerstone of effective collaboration. By acknowledging the hurdles and embracing the power of teamwork, teams can transcend language barriers and navigate through the complexities of requirements with confidence. Through iterative refinement, stakeholders can contribute valuable insights, ensuring that examples evolve to accurately reflect the desired outcomes.

So, while there may not be a magical tool to solve the challenge of defining examples, the collective efforts of dedicated teams pave the way for innovation, efficiency, and ultimately, the delivery of exceptional software solutions.