Blog | kodziak.com - Przemysław Paczoski

Arrays and objects destructuring in JavaScript

Mon, 05 Oct 2020 15:00:00 GMT

Assume, that we have an object:

const obj = {
  a: 'a',
  b: 'b',
};

And we want to get b property. We have a simple way:

obj.b; // 'b'
// or
obj['b']; // 'b'

but, we can use a JavaScript ES6 thing called destructuring - but what's this?

Destructuring assignment

Most used JavaScript data structures are Array and Object.

Array let us store data in ordered collection, syntax looks like:

const student = ['John', 'Doe'];

Object allow us to create an entity, where we can store data by keys, for example:

const student = {
  firstName: 'John',
  secondName: 'Doe',
};

And when we'll need to use Array or Object, in most common cases, we'll need all data or only some of them.

The New ES6 feature gives us syntax sugar which allows us to in an easy way get only fields that we'll need. It works really well when we're facing big objects, or arrays and need only a few fields.

Object destructuring

Let's get back to our example student object.

const student = {
  firstName: 'John',
  secondName: 'Doe',
};

A classic way of getting one of the field value is

student.firstName; // John
// or
student['firstName']; // John

The new syntax allows us to do it in my opinion - a cleaner way.

const { firstName } = student;
firstName; // John

The same thing we can do with secondName

const { firstName, secondName } = student;
firstName; // John
secondName; // Doe

One what we need to remember is that we need to know what keys objects store.

Also, it isn't a problem to change the key name on the fly.

const { first: firstName, second: secondName } = student;
first; // John
second; // Doe

But what if we will have a more complex structure? Maybe like this?

const student = {
  firstName: 'John',
  secondName: 'Doe',
  parents: {
    father: {
      firstName: 'Joe',
      secondName: 'Doe',
    },
    // ...
  },
};

To get father field, we can as in old way - do it in dot or bracket notation or new way

const { parents: { father: { firstName } } = student;
firstName // Joe

Summary

Getting variables using destructuring makes our life a bit easier. It looks nicer than the normal way and makes code reading easier.

For now, I'm using them more than the old way, and what do you think about it?

Automated testing setup for smart contracts: a practical guide

Wed, 06 Nov 2024 16:00:00 GMT

Today, we're diving into the world of automated testing for smart contracts. If you're building in Web3, you know how crucial it is to have rock-solid code. Let's explore how to set up a testing environment that'll make your smart contracts bulletproof! 🛡️

Why Automated Testing for Smart Contracts?

Before we jump into the how-to, let's quickly chat about why this is so important:

Security: Smart contracts often handle millions of dollars. One tiny bug could be catastrophic.
Immutability: Once deployed, you can't just push a quick fix. Testing thoroughly before deployment is key.
Complexity: DeFi protocols can get pretty wild. Automated tests help manage that complexity.

Alright, let's get our hands dirty! 🛠️

Setting Up Your Testing Environment

First things first, we need to choose our tools. For this guide, we'll use:

Hardhat: A development environment for Ethereum
Chai: An assertion library
Ethers.js: To interact with our contracts

Let's set it up:

npm init -y
npm install --save-dev hardhat chai @nomiclabs/hardhat-ethers ethers
npx hardhat

Choose "Create a basic sample project" when prompted.

Writing Your First Test

Let's say we have a simple Token contract. Here's how we might test it:

const { expect } = require("chai");

describe("Token contract", function() {
  it("Deployment should assign the total supply of tokens to the owner", async function() {
    const [owner] = await ethers.getSigners();

    const Token = await ethers.getContractFactory("Token");

    const hardhatToken = await Token.deploy();

    const ownerBalance = await hardhatToken.balanceOf(owner.address);
    expect(await hardhatToken.totalSupply()).to.equal(ownerBalance);
  });
});

This test checks if the total supply is assigned to the contract owner upon deployment. Simple, but powerful! 💪

Automating Your Tests

Now, let's make sure these tests run automatically. We can use GitHub Actions for this. Create a file .github/workflows/test.yml:

name: Smart Contract Tests

on: [push]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Use Node.js
        uses: actions/setup-node@v2
        with:
          node-version: '14.x'
      - run: npm ci
      - run: npx hardhat test

Now, every time you push to your repo, your tests will run automatically. No more forgetting to test before that big deploy! 😅

Best Practices for Smart Contract Testing

Test Edge Cases: Don't just test the happy path. What happens when things go wrong?
Gas Optimization: Include tests for gas usage to avoid costly surprises.
Fuzz Testing: Use tools like Echidna to generate random inputs and find unexpected bugs.
Simulation: For complex DeFi protocols, simulate various market conditions.

Wrapping Up

Setting up automated testing for your smart contracts might seem like a hassle at first, but trust me, it's worth it. It'll save you from headaches (and possibly heart attacks) down the line.

Remember, in the world of Web3, code is law. Make sure your law is air-tight! ⚖️

Happy testing, and may your contracts always compile on the first try! 🚀

Want to level up your Web3 QA game even more? Check out web3qa.xyz for advanced testing strategies and tools specifically designed for blockchain projects. Don't let bugs be the weak link in your Web3 revolution! 💪🔗

Creating Test Data for DeFi: Beyond Simple ERC20 Mocks

Thu, 21 Nov 2024 12:00:00 GMT

Ever felt like your test data is as bland as unseasoned tofu? You're not alone. A whopping 80% of DeFi projects I've audited rely solely on basic ERC20 mocks for testing. But in the wild world of decentralized finance, that's like bringing a knife to a gunfight. Let's level up your test game! 🚀

The Problem with Simple Mocks

Picture this: You've built an amazing DEX, tested it with your trusty ERC20 mock, and everything's green. You deploy... and boom! 💥 Real-world tokens with transfer fees, rebasing mechanisms, or non-standard implementations start flowing in, and suddenly your perfectly tested DEX is about as stable as a house of cards in a hurricane.

// ❌ Don't do this:
function testSwap() {
  const mockToken = await MockERC20.deploy();
  await dex.swap(mockToken.address, amount);
  // Assert some basic stuff
}

// ✅ Do this instead:
function testSwapWithComplexToken() {
  const complexToken = await DeployComplexToken();
  await dex.swap(complexToken.address, amount);
  // Assert considering token's unique behavior
}

In the first example, we're living in La La Land. In the second, we're preparing for the DeFi jungle. Let's dive deeper into creating test data that actually reflects the chaotic beauty of the blockchain.

🛠 Crafting Realistic Test Tokens

1. Fee-on-Transfer Tokens

Some tokens take a cut on every transfer. Surprise! 🎉

contract FeeToken is ERC20 {
    uint256 public constant FEE = 100; // 1%

    function transfer(address recipient, uint256 amount) public override returns (bool) {
        uint256 fee = amount * FEE / 10000;
        uint256 actualAmount = amount - fee;
        
        _transfer(_msgSender(), recipient, actualAmount);
        _transfer(_msgSender(), address(this), fee);
        
        return true;
    }
}

When testing with this bad boy, make sure your contracts can handle receiving less than they expected. It's like ordering a large pizza and getting a medium - disappointing, but you gotta deal with it.

2. Rebasing Tokens

These tokens adjust everyone's balance periodically. It's like your bank account on steroids (or a crash diet).

contract RebasingToken is ERC20 {
    uint256 public constant REBASE_INTERVAL = 1 hours;
    uint256 public lastRebaseTime;

    function rebase() public {
        if (block.timestamp >= lastRebaseTime + REBASE_INTERVAL) {
            uint256 supply = totalSupply();
            _mint(address(this), supply / 100); // 1% inflation
            lastRebaseTime = block.timestamp;
        }
    }
}

Testing tip: Make sure to simulate time passages and check how your contracts handle sudden balance changes. It's like testing your app's resilience to users who can't make up their minds.

3. Blacklistable Tokens

Some tokens come with a built-in bouncer. No entry for you! 🚫

contract BlacklistableToken is ERC20 {
    mapping(address => bool) public blacklist;
    
    function transfer(address recipient, uint256 amount) public override returns (bool) {
        require(!blacklist[msg.sender] && !blacklist[recipient], "Blacklisted!");
        return super.transfer(recipient, amount);
    }
}

When testing, try to swap or provide liquidity with blacklisted addresses. Your contract should handle rejection gracefully, like a seasoned bouncer at an exclusive club.

🎭 Simulating Real-World Scenarios

Now that we have our exotic token menagerie, let's put them through their paces:

Liquidity Crunches: Simulate a sudden drain of liquidity. How does your AMM handle it?
Flash Loan Attacks: Use your test tokens in a flash loan scenario. Can your protocol withstand the pressure?
Gas Price Spikes: Mimic Ethereum congestion by manipulating gas prices in your tests. Does your contract prioritize transactions correctly?

Here's a quick example of simulating a liquidity crunch:

async function testLiquidityCrunch() {
  const complexToken = await DeployComplexToken();
  const pair = await factory.createPair(weth.address, complexToken.address);
  
  // Provide initial liquidity
  await addLiquidity(weth, complexToken, ethers.utils.parseEther("100"), ethers.utils.parseEther("10000"));
  
  // Simulate large withdrawal
  await pair.connect(whale).burn(await pair.balanceOf(whale.address));
  
  // Now try to swap
  await expect(
    router.swapExactTokensForTokens(
      ethers.utils.parseEther("1"),
      0,
      [weth.address, complexToken.address],
      user.address,
      (await ethers.provider.getBlock('latest')).timestamp + 1000
    )
  ).to.be.revertedWith("Insufficient Liquidity");
}

This test ensures your DEX doesn't implode when the liquidity suddenly decides to go on vacation.

🧠 Why This Matters

Creating diverse and realistic test data isn't just about being a perfectionist (though that helps in DeFi). It's about:

Anticipating the Unexpected: DeFi is where traditional finance meets the Wild West. Prepare for anything.
Building Trust: Robust testing translates to reliable protocols. Users trust what doesn't break.
Saving Time (and ETH): Catching bugs in testnet is way cheaper than explaining to users why their funds are now part of a very expensive lesson.

🚀 Leveling Up Your Testing Game

Ready to take your DeFi testing to the next level? Here are your marching orders:

Audit your test suite. How many different token types are you really testing against?
Create a "token zoo" - a collection of various token implementations to test against.
Simulate extreme market conditions. What happens to your protocol during a market crash?
Consider property-based testing to uncover edge cases you haven't even thought of yet.

Remember, in DeFi, we're not just building financial applications; we're architecting the future of finance. Every test you write is a brick in the foundation of that future.

Hungry for more DeFi testing wisdom? Want to make your protocols as robust as a cockroach in a nuclear winter? Swing by web3qa.xyz for more advanced testing strategies, war stories from the trenches, and a community of battle-hardened DeFi testers. Let's make DeFi unbreakable, one test at a time! 💪🚀

First steps with end-to-end testing using jest + playwright (puppeteer)

Thu, 03 Dec 2020 17:00:00 GMT

This post is a continuation of the “First steps with test automation” article.

Recently I wrote a post about my first steps with test automation. I tried to answer questions related to the topic and guide you where and what to search for (if you're interested - click here!). Now, I want to expand on it with technical steps you need to take and show you how easily you can create a basic test with a real world example (my blog 🥳).

We will create a base repository with Playwright and JestJS and write a basic test which will:

Load the blog page,
Check if the blog title contains the author’s name,
Check if the posts list isn't empty,
Click on the first post title and check if it loaded correctly.

To achieve it, I think you need a little more knowledge, so I split the article into three topics (+bonus):

What do I need to start with automation?
Project setup with Playwright and JestJS
Let's write an automated test
Bonus: Test recorder!

Before starting the game, here are some topics which you should be familiar with:

Basic knowledge of JavaScript,
You know how to use CLI and NPM/Yarn,
You're familiar with HTML/CSS.

What do I need to start with automatization?

First of all, you need a library/tool which allows you to control a browser using code. Today, I'll use NodeJS library called Playwright (which is very similar to Puppeteer – here you can find a nice article about the differences: https://blog.logrocket.com/playwright-vs-puppeteer/) to automate our actions with an API. You need a tool like this to write tests that will imitate the behaviour of end user (behave like our clients).

Basically, doing a test manually, you're using a mouse to click and scroll or a keyboard to fill in the forms. In automated tests, you're using a library API to do the same things but the browser is controlled by automation software, not a user.

Next, we need assertions. Playwright provides their own ones, but honestly – I've never used them. I like JestJS, which, besides that, is a test runner, which gives you more opportunities to control the whole test base. With an easy way to do parallelisation or configs for dev/production envs, writing tests is only a pleasure 🥰.

In summary, you need:

A library to control a browser → we'll use Playwright,
A test runner → we'll use JestJS

Project setup with Playwright and JestJS

So we will use:

And if you decided to stick with puppeteer:

Note: When using puppeteer, you need to change selectors because the API is slighty different. In our basic test, it is enough to remove css= part and change waitForLoadState ⇒ waitForNavigation. After those two changes, everything should work.

Before we start writing our first automated test, we need to set up the repository:

Let's initialize it first:

yarn init or npm install
git init

Then install the required dependencies:

yarn install jest jest-playwright-preset playwright

npm install jest jest-playwright-preset playwright

And create two config files,

// jest.config.js
module.exports = {
  preset: 'jest-playwright-preset',
};

// jest-playwright.config.js
module.exports = {
  // Describe which browsers we want to run
  browsers: ['chromium', 'firefox', 'webkit'],
  launchOptions: {
    // If we want to run browsers in headless mode or not,
    headless: false,
    // If we want to have opened devtools from start
    devtools: false,
  },
};

Also, we need to an execute script to our package.json:

"scripts": {
    "test": "jest"
  },

Now, you can yarn test or npm run test which will run all files matching *.test.js names – but we don't have any tests yet 🙈

One more thing that I added is tests/ directory where we will store our test.

Let's write an automated test

First we need to know what we want to test 🙈

I created a basic flow relating to my blog, which you're reading now:

Check if the blog title contains the author’s name,
Check if the posts list isn't empty,
Click on the first post title and check if it loaded correctly.

In the directory tests/ I added a file example.test.js. And to make it easier for you, I coded the above scenario, adding comments line by line, describing what’s happening.

let blogPosts = [];

describe('Blog - kodziak.com/blog/', () => {
  // Will trigger methods before tests
  beforeAll(async () => {
    // Load blog page
    await page.goto('https://kodziak.com/blog/');
  });

  test('title should be "Blog | Przemysław Paczoski"', async () => {
    // Get website title
    const title = await page.title();

    // Compare title of current page
    expect(title).toBe('Blog | Przemysław Paczoski');
  });

  test('should display list of blog posts', async () => {
    // Get all blog posts as an array of objects
    blogPosts = await page.$$eval('css=.post', elems =>
      elems.map(el => {
        return {
          title: el.querySelector('.post-title').textContent.trim(),
          description: el.querySelector('.post-description').textContent.trim(),
          href: el.href,
        };
      })
    );

    // Check if list length is greater than 0
    expect(blogPosts.length).toBeGreaterThan(0);
  });

  test('click on blog post should redirect to article', async () => {
    // Go to first blog post, there we're waiting to resolve all promises from array
    await Promise.all([
      // Click oo .post-title css class element
      page.click('css=.post-title'),
      // Wait for networkidle event, promise resolves after event
      page.waitForLoadState('networkidle'),
    ]);

    // Get title, content and href of current article
    const [articleTitle, articleContent, articleHref] = await Promise.all([
      // Get article title, we use here array destructuring
      page.$eval('css=h2', el => el.textContent.trim()),
      // Get article content
      page.$eval('css=.content', el => el.textContent),
      // Get article href
      page.url(),
    ]);

    // Destructuring first element from an blog posts array. First we use array destructuring and then object destructuring
    const [{ title, description, href }] = blogPosts;

    // Compare title from posts list with current article's title
    expect(title).toBe(articleTitle);
    // Check if the current article content includes description from posts list, compare output with boolean true
    expect(articleContent.includes(description)).toBe(true);
    // Compare href from posts list with current article's href
    expect(href).toBe(articleHref);
  });
});

After we finish coding our test, let's run it using the command yarn test.

Playwright will open three browsers which we described earlier: chromium, firefox, webkit.

And if everything goes well, we should see the following log:

And that's it! We ran our tests positively in three different browsers.

Bonus: Test recorder!

Lastly, a good way to start with test automation could be to use test recorders. Nice one, created with playwright is: https://github.com/microsoft/playwright-cli which allows you to record your browser interactions and automatically generate a fully working code – an automated test with Playwright.

Conclusion

In this article, I wanted to show you how easy it is to automate a basic test scenario using tools such as Playwright and JestJS. I hope that will be a good start for you to explore this part of software engineering.

After reading it, you should have learned:

What Playwright (Puppeteer) and JestJS are used for,
How to set up a repository using those technologies,
How to write a basic test.

Fully working, implemented code from this article can be found in my repository: https://github.com/Kodziak/playwright-first-steps

Have a nice day!

First steps with UI test automation

Thu, 26 Nov 2020 09:00:00 GMT

Starting your test automation journey isn't always as smooth as it should be. You may have a lot of questions like:

Do I have to be a manual tester before?
What language should I start with?
What framework/library should I pick?
What to automate first?
Where to search for information?

and probably many more. I'll try to answer all of them based on my experience and the steps I took when I started with automated functional tests.

So my case was:

experience with manual tests,
no programming background,
the team wanted to start with test automation,
the business was convinced that test automation would be useful.

And I’m going to write this article from that perspective.

Do I have to be a manual tester before?

The short answer is: it would be helpful.

I started my adventure as an intern manual tester and after quite a short time I moved to junior quality assurance position. The whole job was to perform and execute manual tests. I was doing it only for about half a year and then decided to go with automation. I knew that some regression cases could be easily automated and give me more time for other tests. And in my opinion it's a good place to start with automation. It will save you a lot of time for testing new features and probably shorten your time to market.

When you start with manual tests first, you’ll most certainly learn how to write a good test case which is a very important skill. I mean, doing it manually you’re checking specific elements and behaviours of the application so with this knowledge it would be much easier to write good automated tests. If you skip the part of being a manual tester, you will have to catch up – so another topic for beginners:)

That's why, in my opinion, it is important and could be very helpful to start as a manual tester, learn how to test, and then start with automation testing.

What programming language should I pick?

Like everything, it depends. When starting with test automation it’s good to have buddies in the team who could help you. It's possible if you're writing tests in the same language as they're using. I started like that and in my opinion this is the best way.

So, you're working in a scrum team, you're writing front-end applications and you have to write UI tests without a programming background. Having those prerequisites, I would go with a language that my teammates know. Especially in the beginning, they could help you to set up the architecture and write the first test. From a developer’s perspective, learning how another framework/library works is kinda easy. And problems we could face may vary – a problem with framework and a variable not being passed, bad syntax or wrong if statement. Having a bunch of people that could help you in those situations is priceless.

Another nice thing is that the whole team could write test scripts. We are all working to make the application as good as we can, so why couldn’t a developer update a test script after it was affected by his/her changes to UI. And having gone through a given test scenario, he/she could write a whole test. It would speed up our work and let us exchange knowledge. It's very beneficial when you're not the only person who can maintain the test base.

We all want to have a bug free product (and we all know it's impossible).

What framework/library?

Of course it depends on the programming language you have chosen. Here are the most popular solutions and combinations which I know (together with links to documentation):

Java ⇒ Selenium,
C# ⇒ Selenium,
JavaScript/TypeScript ⇒ Jest + Puppeteer, Cypress, TestCafe, Selenium, Nightwatch, Protractor
Python ⇒ Robot Framework, Selenium.

You can notice that Selenium gives you the opportunity to write in almost any language you want. And overall it's a nice solution with another big plus: most of the problems you’ll face with it have already been solved on StackOverflow, which is very helpful in the beginning.

What to automate first?

I've already mentioned manual testing and regression. As we all know, regression is kinda repeatable. Many times, we're doing the same cases all over again which is quite boring. Also, it requires quite some time: sometimes a few hours, sometimes days. So if we write all the cases as automated we could save a lot of time for feature tests and reduce time to market.

Regression test cases aren't very dynamic either. I mean, we probably won't be changing half of the tests by deployment. They don't need much maintenance time.

Reduced time to market is also an important thing related to automated regression testing. Doing it manually could take, for example, two days, and when we switch to automation, we could make it, for example, one hour. It's very beneficial for business, which could be an argument for starting the automation. And one hour is still a pretty long time, in my opinion. Using parallelization or other techniques we could shorten it down to a few minutes. But it depends – of course.

So I would start with regression and probably with login test cases. It’s repeatable, well-known and not so hard.

Where to search for information?

I would stick to the official documentation, which is linked above, but you can also use:

There are two aggregates of links and general websites which could be helpful. Honestly, most of the time I'm using the first two ones because typing any problem into Google will probably return a solution from StackOverflow.

Summary

I hope I answered the given questions, but let's sum it up:

Do I have to be manual tester before? → in my opinion, it’s very helpful but not necessary,
What language should I start with? → pick the language that your team knows,
What framework/library should I pick? → related to the chosen language, the list above,
What to automate first? → regression tests and login test case are kinda nice to start with,
Where to search for information? → documentation + Google and StackOverflow for problems.

It's just my opinion based on my own experience. If I had to start with automation again, I'd do the same, with those prerequisites. But of course – you have to adjust these steps to your business, type of work, and the situation you’re in.

Have a nice day!

Gas optimization through testing: how we reduced costs by 40%

Wed, 13 Nov 2024 12:00:00 GMT

Ever felt that punch in the gut when you deploy your smart contract, only to watch gas costs skyrocket? 😱 You're not alone. In fact, a whopping 80% of Web3 projects struggle with high gas fees, often due to unoptimized code.

But here's the kicker - we managed to slash our gas costs by 40% through smart testing. Yep, you read that right. Let's dive into how we did it, shall we?

The Problem: Gas Guzzling Contracts 🚗💨

First things first, let's look at a typical gas-hungry function:

// ❌ Gas guzzler alert!
function distributeBonuses(address[] memory employees) public {
    for (uint i = 0; i < employees.length; i++) {
        uint bonus = calculateBonus(employees[i]);
        payable(employees[i]).transfer(bonus);
    }
}

This function looks innocent enough, but it's a silent killer for your gas budget. Why? It's doing separate transfers for each employee, and that's expensive on Ethereum.

The Solution: Batch Those Transfers! 🚀

Here's how we optimized it:

// ✅ Gas saver extraordinaire
function distributeBonuses(address[] memory employees, uint[] memory bonuses) public {
    require(employees.length == bonuses.length, "Arrays must be same length");
    
    for (uint i = 0; i < employees.length; i++) {
        payable(employees[i]).transfer(bonuses[i]);
    }
}

What's the difference? We're now passing in pre-calculated bonuses, eliminating the need for separate calculateBonus calls for each employee. This might seem small, but in a large organization, it can save a ton of gas.

The Testing Strategy 🧪

Now, how did we catch this? Enter: Gas Optimization Tests. We set up a test suite that specifically targets gas usage. Here's a simplified version:

import { expect } from "chai";
import { ethers } from "hardhat";

describe("BonusDistributor", function () {
  it("should optimize gas for bonus distribution", async function () {
    const BonusDistributor = await ethers.getContractFactory("BonusDistributor");
    const bonusDistributor = await BonusDistributor.deploy();

    const employees = [addr1.address, addr2.address, addr3.address];
    const bonuses = [100, 200, 300];

    const tx = await bonusDistributor.distributeBonuses(employees, bonuses);
    const receipt = await tx.wait();

    expect(receipt.gasUsed).to.be.below(100000); // Set your gas limit here
  });
});

This test ensures that our distributeBonuses function stays under a specific gas limit. If it ever creeps up, we'll catch it before deployment.

The Results: 40% Gas Savings 💰

By implementing this and similar optimizations across our codebase, we saw a dramatic 40% reduction in gas costs. Here's a quick before and after:

Scenario	Before	After	Savings
10 employees	500,000 gas	300,000 gas	40%
100 employees	5,000,000 gas	3,000,000 gas	40%

Key Takeaways 🗝️

Always batch operations when possible
Implement gas-specific tests in your CI/CD pipeline
Profile your contracts regularly to catch gas hogs
Consider off-chain calculations to reduce on-chain complexity

What's Next? 🚀

Gas optimization is an ongoing process. As your project evolves, keep testing and refining. And remember, every little optimization counts!

Struggling with high gas costs in your Web3 project? Don't sweat it! At web3qa.xyz, we specialize in helping teams like yours slash gas costs and optimize smart contracts. Let's chat about how we can supercharge your project's efficiency!

Check out web3qa.xyz to learn more about our Web3 testing and optimization services. Your future self (and your users' wallets) will thank you! 💪

How much does web3 testing really cost? Breaking down the numbers

Thu, 31 Oct 2024 12:00:00 GMT

Today, we're gonna crunch some numbers and get real about the costs of testing in the Web3 world. Buckle up, 'cause this might surprise you (and your CFO)!

The True Cost of Web3 Testing: It's Not Just About the Tools 🧰

When we talk about Web3 testing costs, it's easy to focus on the obvious stuff - like test automation tools or fancy blockchain simulators. But let's be real, the actual expenses go way beyond that. Here's what you need to consider:

Smart Contract Audits: This ain't your grandma's code review. Expect to shell out anywhere from $5,000 to $50,000+ depending on complexity.
Specialized Talent: Web3 devs and QA engineers don't come cheap. Annual salaries can easily hit $150,000+.
Gas Fees: Yep, even testing costs gas. For complex DApps, this can add up to hundreds or even thousands of dollars per month.
Infrastructure: Running nodes, managing testnets - it all adds up. Budget at least a few hundred bucks monthly.
Security Tools: Blockchain-specific security scanners can cost $500-$2000 per month.

Hidden Costs That'll Sneak Up On Ya 🕵️‍♂️

Here's where it gets tricky. These sneaky expenses often slip under the radar:

Learning Curve: Your team needs time to get up to speed. Factor in at least a month of reduced productivity.
Blockchain-Specific Bugs: These can be nasty and time-consuming. Budget extra time for debugging.
Testnet Instability: When testnets go down, so does your productivity. Always have a Plan B.

Show Me The Code: A Real-World Example 💻

Let's break down a typical test scenario for a DeFi project:

describe('Liquidity Pool', () => {
  it('should allow deposits and calculate shares correctly', async () => {
    const depositAmount = ethers.utils.parseEther('10');
    await token.approve(pool.address, depositAmount);
    await pool.deposit(depositAmount);

    const userShares = await pool.balanceOf(user.address);
    expect(userShares).to.equal(depositAmount);
  });
});

Looks simple, right? But here's what it actually costs to run:

Gas fees for approvals and deposits
Time to set up mock environments
Potential issues with testnet stability

All this for just one test! Now multiply that by hundreds of test cases, and you'll see why proper budgeting is crucial.

Strategies to Keep Costs in Check 💰

Don't panic! Here are some pro tips to keep your testing budget from exploding:

Use Local Blockchain Simulators: Tools like Ganache can slash your gas fees during development.
Implement Proper CI/CD: Catch issues early to save on costly late-stage fixes.
Prioritize Test Cases: Focus on critical paths first. Not everything needs 100% coverage.
Invest in Training: A skilled team is more efficient and catches issues faster.
Consider Managed Services: Sometimes, outsourcing specific testing tasks can be more cost-effective.

The Bottom Line: What Should You Budget? 🤔

For a mid-sized Web3 project, here's a rough annual breakdown:

Smart Contract Audits: $30,000
QA Team (2 engineers): $300,000
Tools and Infrastructure: $20,000
Misc. Expenses (gas, security, etc.): $50,000

Total: Around $400,000 per year

Sounds like a lot? Well, consider the alternative - a single major bug could cost you millions in lost funds or reputation damage. Suddenly, that testing budget doesn't seem so bad, eh?

Wrapping Up: The Real Value of Web3 Testing

At the end of the day, proper testing isn't just a cost - it's an investment in your project's future. Sure, the numbers might make your accountant sweat, but the peace of mind and user trust you gain? Priceless.

Remember, in the wild west of Web3, it's not just about launching fast - it's about launching right. So budget wisely, test thoroughly, and watch your project thrive! 🚀

Curious about optimizing your Web3 testing strategy without breaking the bank? Check out web3qa.xyz for expert insights and cost-effective solutions tailored for the blockchain world. Let's make your Web3 project bulletproof - without needing a money printer! 💪

How to make your UI automation tests resilient to changes?

Sun, 20 Dec 2020 14:00:00 GMT

Using CSS selectors to get elements in end-to-end tests is the most common way (besides xpath), but it can also be tricky and hard to maintain. One day everything could work fine and another a CSS class could be renamed. Styles could change without touching the business flow and then we could have a problem - we'll need to maintain the tests. Of course, the developers could do those changes for us, but they don't know our scenarios, our tests etc. It would be time-consuming for them, because they don't always know if their changes could affect our tests. And for these problems, I think I might have a solution.

But first let's talk about CSS a bit.

What are the common problems with CSS selectors?

Imagine that one day, you had to write a very ugly selector. The element you wanted to use was nested and the only solution available was to base it on CSS classes and HTML structure. And then, a developer made a little refactor that removed some useless HTML tags, which were of course, used in your selector. But he didn't change the business flow. And guess what will happen? Yes, you're right! The test will fail. Your selector will be broken and will need maintenance.

But let's take a step back and think about what are CSS classes designed for? For sure, not to use them in end-to-end tests. Why? Because they're designed to style elements. Everyday there might be changes in the application that don't affect the business flow - which is the most important part in end-to-end tests - but are related with class names changes. That hurts when we need to maintain a test that didn't involve any user flow changes.

Two examples, one simple solution - find better a mechanism to get elements.

What do I recommend? Read below!

Use `data-test-id` for help!

For sure, they're controversial but there are two main advantages of using them.

First one is that they're bulletproof. What? I mean, when we use this attribute with an element, no matter if the developer changes the HTML structure or CSS classes, we still get what we want. Unless they remove the element or change the business flow, our test should work fine.

And second, as you may notice, the developer will see that attribute while working with the code. It could help because they can notify us about possible changes or just maintain the tests if needed - using data-test-id makes it easier find occurrences in tests. And if we have more people working on end-to-end tests we can build more reliable solutions. That's what we all want to achieve - always working product:)

More about working closely with developers below:)

But should I use them everywhere?

Nope. Only with the elements used in tests. We don't want a mess in the code where almost every element has this attribute and only a few of them is used in tests. We want to build trust towards the tests and somehow show the devs that their changes could break our work.

Of course, we'll add some code, but with babel or similar solution we'll get rid of data-test-id on production version. Also, you can take a look at facebook where these attributes are used;).

A little bit of extra work can bring us many benefits.

What are the benefits of using them?

For sure, we can build communication between us and developers. How? Talk. If they see this kind of attribute with an element they want to refactor, they have to maintain the test (in the best case) or just notify us that something could affect it. It's beneficial to know when something can break our tests before running them on testing environment. It'll build trust to the tests. They won't fail because of changes which don't affect the business flow. It will help all of us.

The only thing we need to do is give the developers an opportunity and reason to maintain the tests.

Conclusion

Code structure can always change and it shouldn't affect our tests unless the business flow changes. With selectors based on CSS it could. That's why we decided to go with data-test-id which seems like a reliable solution.

They have some shortcomings but overall I think there is more pros. If you use them you'll work closely with devs which could result in other interesting solutions:) Just remember to add them wisely - only in the elements you're using in tests:)

Have a nice day! 🥳

Importance of CI/CD and parallelisation in test automation

Sun, 31 Jan 2021 17:00:00 GMT

Nowadays, products change very often. We're create new versions of applications on daily basis. That means we need to know fast if everything works. To achieve this, we could run tests manually before every release or...?

Let's automate this process.

How can CI/CD help with test automation?

Let's be honest. If we tell the developers "you need to run the tests manually", they probably won't do it too often. Why? Because they are time-consuming. The execution usually takes more than an hour. Maybe it will work a few times, but I'm pretty sure there will always be some kind of excuse. But that's life.

Then we're coming to the problem that we don't want to merge untested code, ever. We want to run unit, integration, and end-to-end tests (of course, if we have all of them:)). In a convenient way of course. Without any extra work from the user.

The worst we can do, is to make something dependant on us. Continuous processes require knowledge from the developers how to maintain the tests and that's very close to writing them. When we all work on one product, we're all responsible for its quality. And to achieve that, we should work closely, together. Who said that only testers could write E2E tests? Anyone from the development team can do it.

Last but not least, in continuous processes we can block the deployment when a test suite fails. Then the author needs to fix the feature or maintain the tests, if changes affect them. Of course, deterministic tests are needed here, which we all know - is hard to reach.

Loooong time execution...

Yeah... The problem we all know. Many times I've heard that tests takes one or two hours to finish.

We should avoid that.

Try to stick with 10-20 minutes to execute the core ones. To reach that, we can use parallelisation.

We can distinguish two types:

run tests by groups,
run multiple tests at a time.

Tests by groups

First, we should split our tests by groups/criticality. Recently I've written an article about that: What is smoke, sanity, regression testing? How to CI/CD with them. I've split the tests by types. And with that, we could run the groups sequentially.

For example, yours pipeline could looks like this:

An approach like that, makes sure that the most important user flows are always gonna work. Then, on production environment we could check these less important ones. That's fine, until we're aware that something could be broken.

You'll secure our time-to-market, while still checking everything you're supposed to.

Multiple tests at a time

Secondly, use parallel nodes. Basically, most of the tools are configured to run one test and then another. With a strong machine and properly configured tests you can run a few of them at a time.

One of the out of the box tools which solves this problem, and makes the execution faster is Knapsack Pro. A very good tool to get the best execution time you can imagine.

Spread knowledge about tests

But what if a test suite fails? What happens then?

There are two solutions:

you can always monitor what happened and try to find a solution,
you can make it easier for people to get the information about what happened,

And I really love the second option.

I think everyone who works on software should be able to write, maintain, and fix the tests. You're working together to deliver the best product quality you can. That means you should avoid the stereotypical division between testers and developers. You're working together, as a development team.

You can also implement automatically generated reports. Showing people, business, and other stakeholders what the tests cover, builds trust. And the trust in tests is what you need.

Conclusion

You should deliver the best product quality you can in reasonable time. It's good when you can do it fast and with fine user experience (our internal users). People won't run tests manually, so you should automate it. That's why you could use continuous processes.

Combining it with parallelisation, you'll secure the time-to-market.

Spreading knowledge about tests makes more people feel responsible for them. If something fails - someone from the team will need to fix or maintain it. The more people work on tests, the better these tests are.

Integration Testing in Web3: Connecting Smart Contracts with Frontend

Sat, 26 Oct 2024 12:00:00 GMT

Ever deployed a smart contract only to find out it doesn't play nice with your frontend? Frustrating, right? You're not alone - a whopping 65% of Web3 projects face integration issues between smart contracts and frontends.

Let's dive into how we can tackle this beast head-on! 🐉

The Challenge: Bridging Two Worlds

Imagine you're building a decentralized exchange (DEX). You've got your ERC20 token contract and your exchange contract all set up on the blockchain. Now comes the tricky part - connecting these bad boys to your slick React frontend. How do you ensure everything works seamlessly? 🤔

Step 1: Set Up Your Testing Environment

First things first, we need a solid testing environment. Here's what you'll need:

Hardhat or Truffle for local blockchain simulation
Ethers.js or Web3.js for interacting with smart contracts
Jest or Mocha for running tests
React Testing Library for frontend testing

Let's set up a basic test file:

import { ethers } from "hardhat";
import { expect } from "chai";
import { render, fireEvent, waitFor } from "@testing-library/react";
import { Exchange } from "../components/Exchange";

describe("DEX Integration", () => {
  let tokenContract;
  let exchangeContract;

  beforeEach(async () => {
    // Deploy contracts
    const Token = await ethers.getContractFactory("Token");
    tokenContract = await Token.deploy();

    const Exchange = await ethers.getContractFactory("Exchange");
    exchangeContract = await Exchange.deploy(tokenContract.address);

    // More setup...
  });

  it("should allow token swap", async () => {
    // Test implementation
  });
});

Step 2: Mock Blockchain Interactions

One common pitfall is trying to test with real blockchain interactions. That's a no-no for integration tests. Instead, we'll mock these calls:

// ❌ Don't do this:
const balance = await web3.eth.getBalance(userAddress);

// ✅ Do this instead:
jest.mock("web3");
web3.eth.getBalance.mockResolvedValue("1000000000000000000");

Always explain your mocks and why you're using them. In this case, we're simulating a user with 1 ETH balance.

Step 3: Test Contract-Frontend Interaction

Now for the juicy part - testing how our frontend interacts with the smart contracts. Let's test a token swap:

it("should allow token swap", async () => {
  const { getByText, getByLabelText } = render(<Exchange 
    tokenContract={tokenContract}
    exchangeContract={exchangeContract}
  />);

  // Simulate user input
  fireEvent.change(getByLabelText("Amount"), { target: { value: "10" } });
  fireEvent.click(getByText("Swap"));

  // Wait for the swap to complete
  await waitFor(() => {
    expect(getByText("Swap Successful!")).toBeInTheDocument();
  });

  // Verify contract state
  const userBalance = await tokenContract.balanceOf(userAddress);
  expect(userBalance).to.equal(ethers.utils.parseEther("10"));
});

This test simulates a user swapping 10 tokens and verifies both the UI update and the contract state change.

Common Pitfalls and How to Avoid Them

Async Hell 😈 Web3 is async by nature. Use async/await and handle promises properly.
Gas Estimation ⛽ Mock gas estimations in your tests to avoid unpredictable behavior.
Event Listening 👂 Don't forget to test event emissions from your contracts and how your frontend handles them.

Here's a quick example of testing event emission:

it("should emit Transfer event", async () => {
  await expect(tokenContract.transfer(recipient, amount))
    .to.emit(tokenContract, "Transfer")
    .withArgs(sender, recipient, amount);
});

Best Practices for Web3 Integration Testing

Use a Local Blockchain 🏠 Tools like Hardhat make it easy to simulate blockchain environments.
Snapshot Testing 📸 Great for catching unexpected changes in your UI.
Fuzz Testing 🐝 Try random inputs to discover edge cases in your smart contracts.
CI/CD Integration 🔄 Automate your tests to run on every push or pull request.

Wrapping Up

Integration testing in Web3 is a unique challenge, but with the right approach, you can ensure your dApps are rock-solid. Remember, it's not just about testing your smart contracts or your frontend in isolation - it's about how they dance together! 💃🕺

Got more questions about Web3 testing? Struggling with a specific integration issue? Don't sweat it! Head over to web3qa.xyz where we're building a community of Web3 QA enthusiasts. Let's level up your testing game together! 🚀

P.S. Drop a comment below if you want me to dive deeper into any specific aspect of Web3 integration testing. Your feedback shapes the content! 🙌

Load testing types: load, stress, soak, spike. Examples in k6 load testing tool

Fri, 12 Aug 2022 10:00:00 GMT

Load testing is a well-known technique of software testing to make sure, that system we’re working on, behaves in a way, we’d like to. During day-to-day tasks, I started thinking if testing the product only from business perspective is enough. Well… No, so I started some research about it.

I wanted to clarify for myself, what types give me what, as an outcome. What tool should I use, or environment? Do I need to set up something, or just use production? (yeah, jokes aside).

I’ve tested wrk, jmeter, and k6. At the end, the last one looks as the easiest and most configurable one for me. I’ve created a couple of sample scripts, to show you how to use that tool.

In the next article, I’ll explain a bit more about environment, and flow how you can set up your load testing.

Types of load tests

We encounter four types of load tests:

Load
Stress
Soak
Spike

Each of them is slightly different, and gives us possibility to check other scenarios/perform quite tests.

Load testing

Being sure, that the system is meeting the performance goals is one of the things, you should pay attention. This kind of test, is the one, which can provide you this answer.

This type is concerned with assessing the current performance in terms of concurrent users or requests per second.

Example code implementation

import { sleep } from 'k6';
import http from 'k6/http';

export const options = {
  stages: [
    { duration: '5m', target: 100 },
    { duration: '10m', target: 100 },
    { duration: '5m', target: 0 },
  ],
  thresholds: {
    http_req_duration: ['p(99)<250'],
  },
};

export default function () {
  http.get('https://httpbin.test.loadimpact.com/');
  sleep(1); // Make sure, that each user cooldown between requests is 1s
}

Stress testing

The purpose of this type of load test, is to verify the stability and reliability of the system, under extreme condition’s.

Running this type of load test, will:

determine behavior of your system under extreme condition’s,
determine maximum capacity in terms of users throughput,
determine the breaking point of your system,
tell you, if your system will recover without manual intervention after the stress test is over.

Example code implementation

import { sleep } from 'k6';
import http from 'k6/http';

export const options = {
  stages: [
    { duration: '2m', target: 100 },
    { duration: '5m', target: 100 },
    { duration: '2m', target: 200 },
    { duration: '5m', target: 200 },
    { duration: '2m', target: 300 },
    { duration: '5m', target: 300 },
    { duration: '2m', target: 400 },
    { duration: '5m', target: 400 },
    { duration: '10m', target: 0 },
  ],
};

export default function () {
  http.get('https://httpbin.test.loadimpact.com/');
  sleep(1);
}

Soak testing

The purpose of this type of load test, is to validate the system under long-time overload.

Running this type of load test, will:

verify that your system doesn’t suffer from bugs or memory leaks,
make sure, that your database works under high overload,
make sure, that your logs works under high overload,
make sure, that external services you depend on, doesn’t stop under high overload.

To run them:

determine the maximum number of users your system can handle,
get the 80% of that value,
run the test in three stages: ramp, stay and cooldown.

Example code implementation

import { sleep } from 'k6';
import http from 'k6/http';

export const options = {
  stages: [
    { duration: '5m', target: 400 },
    { duration: '3h55m', target: 400 },
    { duration: '5m', target: 0 },
  ],
  thresholds: {
    http_req_duration: ['p(99)<250'],
  },
};

export default function () {
  http.get('https://httpbin.test.loadimpact.com/');
  sleep(1); // Make sure, that each user cooldown between requests is 1s
}

Spike testing

The purpose of this type of load test, is to validate the system under extreme increments and decrements in traffic over a very short time. The main purpose is to evaluate the behaviour of the system to determine recovery time.

Running this type of load test, will:

determine how your system will perform under a sudden spikes of traffic,
determine if your system will recover.

Example code implementation

import { sleep } from 'k6';
import http from 'k6/http';

export const options = {
  stages: [
    { duration: '10s', target: 100 },
    { duration: '1m', target: 100 },
    { duration: '10s', target: 1400 },
    { duration: '3m', target: 1400 },
    { duration: '10s', target: 100 },
    { duration: '3m', target: 100 },
    { duration: '10s', target: 0 },
  ],
  thresholds: {
    http_req_duration: ['p(99)<250'],
  },
};

export default function () {
  http.get('https://httpbin.test.loadimpact.com/');
  sleep(1); // Make sure, that each user cooldown between requests is 1s
}

Summary

Load testing is a powerful technique to make sure, that your system behaves as expected in various conditions. My personal feeling is that every project should consider them at some point. Getting more, and more clients can make flood the system, and produce terrible feedback, if it can’t handle them.

k6 seems to be great tool, to set up them fast, clearly with low learning curve.

Sources

Separate or project repository to store tests? Which, why, and when?

Mon, 13 Dec 2021 20:00:00 GMT

While starting to work with end-to-end tests, one of the first questions is where to store them. Alongside source code or in separate repository? Two solutions, both slightly different. Each of the approaches has their pros and cons – but which to choose?

In this short article, I've tried to describe them both. They're different, and from your perspective you've to choose which one to use, which isn't simple at all. I believe that this could help you a bit.

Project repository

Store tests alongside source code keeps everything in the same place. You've both app-related and tests-related stuff like tasks, workflows etc. in the same spot. It causes, that the developers could execute everything much easier than searching some commands in another repository. It closes the gap between the developers and test engineers (SDET's). There aren't any barriers.

When the tests fail, it would fail on the programmer branch, so it'd be his job to maintain them. That's the mental shift when things are together. You wouldn't be responsible for bringing back the tests to love after every failure. Or find the person which going to fix them or the development is blocked.

The point is, that storing everything in the same repository, there aren't any other “strange” place where some code checks the developer's job and makes it easier to maintain.

What's more, when the newest feature change the application flow and breaks tests, you don't have problem with different versions of tests and app. They're synced because of the same repository. No concern with merges and only one thing which left would be to maintain them on the same branch as the feature is and push a complete working solution. The time when tests aren't compatible with the newest application version shouldn't exist.

Example structure

<application-repository>/<root>/app
<application-repository>/<root>/tests

Separate repository

When the test code is in “another” project, and it usually breaks — the developers probably won't care about it. It's going to be your job to maintain the tests and unstuck the blocker.

Unfortunately, when the things are separated, it would reduce the collaboration between team members. If your organization have clear distinction between the developers and test engineers, a separate repository will widen that gap. One wouldn't work on each other solutions. That's really not a situation you should go. The most beneficial would be working together on both, the application code and tests, as a team.

Imagine a situation. From the developer's perspective. You've created a feature which breaks some of them and want's to merge it, fast. Obvious is, the tests would fail, so you've to set up to another repository, maintain the tests and prepare request, or just merge the feature without properly working tests. It would create a gap because it's impossible to merge both, new versions of the code at the same time.

In that scenario, each of the repositories going to have workflow according to its needs – which is good. It will separate the contexts, so the person which going to start working on one of the projects will have only things related to that. The application has tasks related to build and running the application, and in tests will contain jobs to set up the application under tests and execute them. But it has downsides too. It's just annoying to run the whole flow on local machines. You'll need to execute commands in different places, which bring up the complexity.

Of course, I didn't say that only the test's repository has to contain tests – nah. The application should have their own. The developers should write unit tests alongside the source code and execute them in pipelines.

Separating the repositories keeps more clean environment. Repositories contain only things it's concerned about. It's make easier to navigate on the daily basis. But there's a downside to this approach. You'll have more moving parts to your workflow, which makes things more complicated along the way. For example, with new developers, setup everything it's going to be a little more difficult in front of a situation, where everything is in one place.

Example structure

<application-repository>/
<tests-repository>/

Conclusion

Project repository

Increase ownership through the team,
Closes the gap between the developers and test engineers,
No problem with merging, you'll always have synced versions between tests and application,
Everything app and tests related going to be in the same place.

Separate repository

Decrease ownership through the team,
Widen the gap between the developers and test engineers,
Could be a situation where tests aren't working on newest application version,
Cleaner environment, only things concerned about are in the repository.

Reading the article and points above, it seems like a single repository is a straight winner. But it's just how it works for me, most of the time. It doesn't mean, that it's the best solution for your organization. It always depends – on your project architecture, company structure, how many people you have and how you're working as a test engineers.

The best would be to talk to your teammates and decide as a team which could work better for you. Work together on that topic. I assumed that you have in-house test engineers. If not, then, probably you should choose a separate repository for the tests.

Have a nice day! 🥳

Set up special Github readme! Examples included!

Thu, 17 Sep 2020 15:00:00 GMT

Some time ago I hit an article, that Github added a special type of repository, the ones you see above (we'll make it better), where we can write something about us. I think, that's a great idea to show, what exactly I'm doing, and maybe... it could give me some movement to my initiatives?

But there's one problem. I don't like writing about me, I just... don't. I can't do this right (but I'm writing a blog, huh).

The good one is that Github gave us a basic template, where we just can edit some entries and voila, we'll have something similar as I have.

If you're curious how to add this to your Github profile - read the whole article.

How to create your own?

You need to create a repository, but it's a special type. It has to be named as your GitHub account. So just go to https://github.com/new, same as you'll have to create a normal repository, and when you'll type the name of your account (in my case it's kodziak), you'll be greeted with a message telling you about this special repository.

Highly recommend setting up this repository as a private to avoid showing a messy/broken page in time you'll be working on it.

Also, you can initialize base README.md file, which will automatically appear in your profile.

And voila - you have your own page.

Let's make it awesome

Doing research, I found some nice (nicer than mine, huh) showing pages. So I thought, why not make me more interesting?

I solicited my page into categories. As you may know, I'm also a podcaster (I hope, you know if not, shame on you - I'm kidding), so I'll go with categories:

Who am I?
My core actions (like blog, podcast)

And at the footer - some of my social media, where you can connect and write to me (feel free, I'm always answering!).

Assumptions I go with

I like emojis, so I'll go with like every header for a specific category that will contain one. I think it's the way we can add some colors and fun to this text.

But conversely to emojis, I like simplicity, so I thought without adding so many colors, images, gifs, etc, because it just not my style. All the time, I'm trying to keep it simple, to show this nice guy/woman who will come to my profile, as many pieces of information as he/she can get in just a few seconds. Not distracting him/her by jumping ducks.

Another rule I think you should stick is to make it short. No one wants to read long, boring texts (like this one). I'm a fan of short tips and tricks, wherein seconds I can get what I wanted - so I'm tried to keep my page in this style.

Conclusion

Make it fun, simple, and short. But make it. I could help you.

I hope, this article gave you knowledge on how to set up this and inspiration how can you improve (setup?) your page. If yes - feel free to write to me with a link, I'll put it there!

Feel free to write to me with any topic 😇

Have a nice day!

In case you need inspiration, there are some interesting ones:

Six things you should consider while designing a test architecture

Tue, 23 Feb 2021 10:00:00 GMT

The beginning of automated tests in a project is easy and difficult at the same time. You can start smoothly, using base architecture, simple tests, etc - and it's gonna work! But... With growing architecture, solutions you'll take could be very hard to develop in the future.

That's why, you should consider a few things, before the start. I want to show you the problems and solutions do they solve.

Page Object Model

POM - Page Object Model is a design pattern that is commonly used in test automation for avoiding code duplication and improve test maintenance.

This model contains classes that represent web pages. You break down each page into a corresponded page class. This class contains locators (selectors) to web page elements and methods that perform actions on them.

This fills the most common actions you'll be doing in test automation - find the element, fill the value, click the button. For small scripts, you can store selectors in the tests, but with growing suites, you'll face many duplications - that's bad, you don't want them. If one of the used selectors gonna change, you'll need to update every place, you've used it.

And... POM is the solution. You'll create a separate class that all of the tests gonna use. Then, if any of the selectors change, you'll only have to update one place. It's faster for the person which updates it.

But... There is another model of creating an architecture for automation tests. It's called App Actions. I've never tested it, so I can't recommend it, but here's a nice article about it - click!

Logging

You should use a logging system to give the user (developer, or tester) enough information about what's happened in the tests while executing in a relatively short time. It's most possible, that you'll run your tests through a remote machine. If they fail, you need to know fast what's happened. Troubleshoot or reproduce failed tests with a stack trace and the latest assertion message only will be very hard.

That's why you should design a logging system.

A well-designed strategy will show the user step by step, what's happened in the test. In the project that I've worked on, we created a three-step strategy:

begin,
info,
end,

To do this quickly, we've used decorators (TypeScript only).

For example (code base on puppeteer):

@logger
function getText(selector) {
	return page.$eval(selector, (el) => el.textContent);
}

Result with:

[BEGIN] - getText from: #email
[INFO] - getText result: sampleemail@gmail.com
[END] - getText from: #email

The assumption was simple. We want to know when the method starts, what's the result and when ends. We've used a method name and parameters as a recognizer. That comes out with well-structured logs, which helps us to know exactly what happened in test execution.

Also, you can use these logs in your reporting system, which is described below.

Another good idea to mention is to write custom, detailed assertion messages. For example,

Expect: 5, Received: 8

It's saying not much. Without the context and knowledge of what the test is checking, it's hard to assume what's happened. You'll need to check it manually. But...

We expect 5 elements on the listing, got 8.

It's easy to read and clearly saying what's happened. From the message, you can easily assume where's the problem.

Also, there are two more areas, where detailed messages could be helpful:

timeout exceptions - ex. internet connections problem,
architecture exceptions - ex. DB connection, auth connection problem.

I believe, that having a proper logging system could make your work easier and speed up the troubleshooting process.

Test Data

Most common action in your tests gonna be to fill the value within the data. But... You need to have, or generate it. It's one of the most time-consuming actions, but necessary.

Let's split the data into two types:

static data - the one you store in database/repository/somewhere,
dynamic data - the one you generate using tools like a faker, etc.

Using class patterns you can design a system that handles the data for you. Connections to a database or dynamically creating sensitive data (ex. from faker.js).

Parallel Executions

Parallelization is an automated process that automatically launches your actions simultaneously. The goal is to save execution time by distributing tests across available resources.

Consider a situation. You have a big database. Execution time takes 50 minutes to complete. You have 50 tests, where one approximately takes 1 minute. Tests are isolated (if not, you need to work on them). Using 5 parallel executions, you can reduce the total testing time to 10 minutes.

Fast, reliable tests are key to a stable product.

One, the interesting out-of-the-box solution is Knapsack Pro. It helps run your tests in a parallel efficient way.

Deployment Pipeline

A deployment pipeline is a process, where you take the code from version control and make it available for the users in an automated way. The team, which works on a project, needs an efficient way to build, test, and deploy their work to the production environment. In old fashion way, this was a manual process, where there were few deployments per week, now it's a few per day/hour. A set of automated tools doing it for us.

Imagine a problem. You have 50 tests and execution takes 50 minutes (without parallel executions). To test the fresh version of the code, the developers need to build an application locally and run the tests against it. And okay, the tests found an issue, so the user fixes the code and does it again. Hours and nervousness growing. I'm pretty sure, that after a few rounds they're not gonna use the tests. You don't want this.

The solution is to use an automated deployment pipeline. It's designed especially for cases like this.

Combining it with parallel executions, you will create an efficient, fast strategy, which gonna defend your production with the tests. Even better, if failed tests block the deployment. You'll never push broken code to the real users. 🥳

Reports

Reporting is about documenting the process of test execution. They should combine a summary for the management and stakeholders. Also, should be detailed to give the developers feedback. Especially, when something fails.

Consider a failed execution. Tests are running on a remote machine, and somehow, you need to know what exactly happened. Why they failed and what's broken. Access to plain logs could be painful in searching this one failed test. A well-created report should cover that.

Each report could be different, but in my opinion, there are a few must-have metrics:

machine/environment name,
total number of running tests,
duration,
list of all test cases with logs (steps from logging system) in toggle,
test result (for every test case).

Have I mentioned failed test executions and it's crucial to know what happened? 🥳 Constantly looking at a deployment pipeline dashboard or searching in logs isn't that efficient. A well-written report, mixed with the slack notification can speed up this process much.

Conclusion

I've described six topics, which are really nice to consider in every mature test automation project. They solve problems, which you'll encounter on daily basis. By doing some coding and research, you'll easier your work and make the architecture expandable. Using a few hacks, you'll speed up your troubleshooting process and with automatically generated reports - you'll be as transparent as possible with your tests.

I believe, that this knowledge will help you at the beginning of your journey.

What's also good to consider:

base architecture - pattern to re-use base methods, create browsers,
video recordings - record whole test session,

Have a nice day! 🥳

Testing NFT Smart Contracts: Common Edge Cases You're Missing

Tue, 26 Nov 2024 12:00:00 GMT

If you're building in this space, you know how critical it is to get your contracts right. But here's the kicker - a whopping 80% of NFT projects miss crucial edge cases in their testing. Yikes! 😱

Let's fix that, shall we? We'll explore some common pitfalls and how to avoid them. Ready to level up your NFT contract game? Let's go!

1. The Sneaky Approve and Transfer 🕵️‍♂️

One of the most overlooked scenarios is when a user approves a transfer, but the transfer doesn't happen immediately. Here's what can go wrong:

// ❌ Don't do this:
function testApproveAndTransfer() {
  // Approve
  // Assert approval
  // Transfer
  // Assert transfer
}

// ✅ Do this instead:
function testApproveAndTransferWithDelay() {
  // Approve
  // Assert approval
  // Simulate time passing
  // Transfer
  // Assert transfer
}

Why does this matter? In real-world scenarios, there might be a delay between approval and transfer. By simulating this delay, you can catch potential vulnerabilities that only surface when transactions don't occur back-to-back.

2. The Greedy Minter 🤑

Ever thought about what happens if someone tries to mint more NFTs than allowed? Many don't, but you should!

// ❌ Incomplete test:
function testMint() {
  // Mint allowed amount
  // Assert success
}

// ✅ Comprehensive test:
function testMintWithEdgeCases() {
  // Mint allowed amount
  // Assert success
  // Try to mint one more
  // Assert failure with correct error message
  // Try to mint max uint256
  // Assert failure with correct error message
}

This test covers not just the happy path, but also attempts to break the system. It's crucial to ensure your contract handles these edge cases gracefully.

3. The Royalty Roundabout 💰

Royalties are a big deal in the NFT world, but testing them can be tricky. Here's an often-missed scenario:

// ❌ Basic royalty test:
function testRoyalties() {
  // Set royalties
  // Sell NFT
  // Check royalty payment
}

// ✅ Comprehensive royalty test:
function testRoyaltiesWithEdgeCases() {
  // Set royalties
  // Sell NFT
  // Check royalty payment
  // Change royalty recipient
  // Sell NFT again
  // Check new recipient gets payment
  // Try to set royalties > 100%
  // Assert failure
}

This expanded test covers changing royalty recipients and attempts to set invalid royalty percentages - crucial for maintaining fairness and preventing exploits.

4. The Metadata Mayhem 🎭

Metadata is the soul of an NFT, but it's often undertested. Let's change that:

// ❌ Simple metadata test:
function testMetadata() {
  // Set tokenURI
  // Assert tokenURI
}

// ✅ Robust metadata test:
function testMetadataEdgeCases() {
  // Set valid tokenURI
  // Assert tokenURI
  // Set empty tokenURI
  // Assert failure or default behavior
  // Set extremely long tokenURI
  // Check gas costs and potential DOS vectors
}

This comprehensive test ensures your metadata handling is bulletproof, even in extreme scenarios.

5. The Ownership Odyssey 🚀

Ownership transfers are common, but are you testing all scenarios?

// ❌ Basic ownership test:
function testOwnership() {
  // Transfer ownership
  // Assert new owner
}

// ✅ Thorough ownership test:
function testOwnershipScenarios() {
  // Transfer ownership
  // Assert new owner
  // Try to transfer ownership as old owner
  // Assert failure
  // Transfer to zero address
  // Assert failure
  // Renounce ownership
  // Assert owner is zero address
  // Try to transfer after renouncing
  // Assert failure
}

This test suite covers various ownership scenarios, including edge cases that could lead to locked contracts or loss of control.

Wrapping Up 🎁

Testing NFT smart contracts is no walk in the park, but catching these edge cases can save you from major headaches (and potential losses) down the line. Remember, in the world of blockchain, there are no take-backsies!

By implementing these more comprehensive tests, you're not just ticking boxes - you're building a fortress around your NFTs. And in this wild west of Web3, that's exactly what you need.

Feeling overwhelmed? Don't sweat it! This stuff is complex, and it's okay to need a hand. If you want to dive deeper into Web3 testing strategies or need expert help in securing your NFT projects, head over to web3qa.xyz. We're here to help you build rock-solid NFT contracts that stand the test of time (and hackers).

Stay curious, keep testing, and let's make the NFT space safer for everyone! 🛡️💎

Testing smart contract upgrades: a complete testing strategy

Sun, 10 Nov 2024 12:00:00 GMT

Ready to level up your smart contract upgrade testing game? You're in the right place!

Did you know that over 60% of smart contract vulnerabilities are introduced during upgrades? 😱 That's a scary number, but don't worry - I've got your back. Let's dive into a complete testing strategy that'll help you sleep better at night (and keep your users' funds safe).

Why is testing upgrades so crucial? 🤔

Imagine this: You've just pushed an upgrade to your DeFi protocol, and suddenly... 💥 BOOM! Users can't withdraw their funds, or worse, someone's exploiting a new vulnerability. Not cool, right?

That's why we need a rock-solid testing strategy for upgrades. Let's break it down step by step.

1. Baseline Testing 📊

Before you even think about upgrading, make sure your current contract is thoroughly tested. This means:

Unit tests for each function
Integration tests for complex interactions
Fuzz testing to catch edge cases

Here's a quick example of a baseline test:

// ✅ Do this:
describe("TokenVault", function() {
  it("should allow deposits", async function() {
    const [owner, user] = await ethers.getSigners();
    const TokenVault = await ethers.getContractFactory("TokenVault");
    const vault = await TokenVault.deploy();
    await vault.deposit({ value: ethers.utils.parseEther("1") });
    expect(await vault.balanceOf(owner.address)).to.equal(ethers.utils.parseEther("1"));
  });
});

Always explain your tests and what they're checking. This helps your team understand the coverage and importance of each test.

2. Upgrade Simulation 🔄

Before deploying to mainnet, simulate the upgrade process in a test environment. This includes:

Deploying the new contract
Migrating state (if necessary)
Verifying that all functions work as expected post-upgrade

Here's how you might test an upgrade:

// ✅ Do this:
it("should maintain user balances after upgrade", async function() {
  const TokenVaultV1 = await ethers.getContractFactory("TokenVaultV1");
  const vaultV1 = await upgrades.deployProxy(TokenVaultV1);
  await vaultV1.deposit({ value: ethers.utils.parseEther("1") });

  const TokenVaultV2 = await ethers.getContractFactory("TokenVaultV2");
  const vaultV2 = await upgrades.upgradeProxy(vaultV1.address, TokenVaultV2);

  expect(await vaultV2.balanceOf(owner.address)).to.equal(ethers.utils.parseEther("1"));
});

This test ensures that user balances are preserved during the upgrade process. Always check critical state variables and functionality!

3. Backwards Compatibility Checks 🔙

Upgrades shouldn't break existing functionality. Test that:

Old function calls still work
Events are emitted correctly
External contract interactions remain intact

4. New Feature Testing 🆕

For any new features introduced in the upgrade:

Write comprehensive unit tests
Perform integration tests with existing functions
Fuzz test new inputs

5. Gas Optimization Verification ⛽

Upgrades can sometimes introduce unexpected gas costs. Always check that:

Function gas costs haven't increased significantly
New functions are gas-efficient

Here's a simple gas test:

// ✅ Do this:
it("should not increase gas costs significantly", async function() {
  const tx = await vaultV2.deposit({ value: ethers.utils.parseEther("1") });
  const receipt = await tx.wait();
  expect(receipt.gasUsed).to.be.below(100000); // Adjust based on your expectations
});

6. Security Audits 🔒

Don't skip this step! After your internal testing:

Conduct a thorough security audit
Use automated tools like Slither or Mythril
Consider a manual audit by security experts

7. Testnet Deployment 🌐

Before hitting mainnet:

Deploy to testnet
Run all tests in a live environment
Encourage community testing and bug bounties

8. Upgrade Rehearsal 🎭

Practice makes perfect:

Simulate the entire upgrade process
Time each step
Prepare a rollback plan

9. Post-Upgrade Verification ✅

After the upgrade:

Run all tests again on the upgraded contract
Monitor for any unusual activity
Be ready to respond to user reports

Remember, testing is not a one-time thing. It's an ongoing process that continues even after the upgrade is live.

Wrapping Up 🎁

Testing smart contract upgrades is no joke. It requires a systematic approach, attention to detail, and a healthy dose of paranoia. But with this strategy, you're well-equipped to handle upgrades like a pro.

Got questions about implementing this strategy? Need help setting up your testing pipeline? I'm here to help! Check out web3qa.xyz for more resources on Web3 testing and quality assurance. Let's make the blockchain a safer place, one test at a time! 💪

Happy testing, and may your upgrades always be smooth! 🚀

Web2 vs Web3 Testing: Key Differences That Will Surprise You

Sat, 26 Oct 2024 12:00:00 GMT

Hey there, fellow testers and Web3 enthusiasts! 👋 Today, we're diving deep into the world of Web2 vs Web3 testing. Buckle up, 'cause it's gonna be a wild ride! 🎢

The Basics: Web2 vs Web3

Before we jump into the testing differences, let's quickly recap what Web2 and Web3 actually are:

Web2: The internet we know and love (or sometimes hate 😅). Centralized, controlled by big tech companies.
Web3: The new kid on the block. Decentralized, powered by blockchain, and giving power back to the users.

Key Differences in Testing Approaches

1. Environment Setup

Web2: Pretty straightforward. You set up your local environment, maybe a staging server, and you're good to go.

Web3: Oh boy, here's where things get spicy! 🌶️ You need to set up a local blockchain, deal with test networks (testnets), and sometimes even fork the mainnet. It's like cooking a gourmet meal instead of microwaving a TV dinner.

Example:

// Web3 local environment setup using Hardhat
require('@nomiclabs/hardhat-waffle');

module.exports = {
  solidity: '0.8.0',
  networks: {
    hardhat: {
      forking: {
        url: 'https://mainnet.infura.io/v3/YOUR-PROJECT-ID',
      },
    },
  },
};

2. State Management

Web2: State is typically managed on the server or in the browser. Resetting state is as easy as clearing a database or local storage.

Web3: State lives on the blockchain, my friend! Resetting it means redeploying smart contracts or resetting your local blockchain. It's like trying to erase your embarrassing high school photos – not impossible, but definitely more complicated. 😅

3. Handling Asynchronous Operations

Web2: Async operations are usually related to API calls or database queries.

Web3: Welcome to the world of blockchain confirmations! Every transaction needs to be mined and confirmed. It's like waiting for your crush to text back – sometimes quick, sometimes painfully slow.

// Web3 transaction wait
const tx = await contract.someFunction();
await tx.wait(2); // Wait for 2 block confirmations

4. Security Testing

Web2: Focus on things like XSS, CSRF, SQL injection.

Web3: Enter the realm of smart contract vulnerabilities! Reentrancy attacks, front-running, and gas optimization become your new best friends (or worst enemies). It's like playing chess, but every piece is trying to steal your ETH.

5. Performance Testing

Web2: Load testing, stress testing, you know the drill.

Web3: Gas optimization becomes crucial. You're not just testing how many users your app can handle, but how efficiently it uses blockchain resources. It's like trying to fit your entire wardrobe into a tiny suitcase for a weekend trip.

6. User Authentication

Web2: Username/password, OAuth, maybe some 2FA if you're fancy.

Web3: Welcome to the world of wallets and signatures! Users sign transactions with their private keys. It's like having a unique, cryptographic fingerprint for every action.

// Web3 authentication example
const message = 'Login to MyAwesomeWeb3App';
const signature = await signer.signMessage(message);
// Verify signature on the backend

Common Challenges and Solutions

Deterministic Testing:
- Challenge: Blockchain's inherent randomness makes repeatable tests tricky.
- Solution: Use controlled environments and seed your RNG for deterministic results.
Long-running Tests:
- Challenge: Waiting for blockchain confirmations can make tests slow.
- Solution: Use time-travel features in local blockchains to speed things up.
Gas Costs:
- Challenge: Tests can get expensive on public testnets.
- Solution: Use local blockchains or forked mainnet for most tests.
Complex State:
- Challenge: Blockchain state can be complex and interdependent.
- Solution: Use snapshots to quickly reset to a known state between tests.

Conclusion

Testing in Web3 is like Web2 testing on steroids. 💪 It's more complex, more challenging, but also more exciting! As the Web3 space evolves, so do our testing methodologies. Stay curious, keep learning, and most importantly, have fun with it!

Ready to level up your Web3 testing game? Check out web3qa.xyz for more in-depth guides, tools, and best practices. Let's make the decentralized world a better-tested place, one smart contract at a time! 🚀

What is screenshot, UI, API, and performance testing?

Mon, 28 Dec 2020 14:00:00 GMT

Building a product, we face the moment that we need more types of tests. Screenshot, UI, API, or Performance? There are many of them and every one is designed for other purposes.

Every type is different, and can help us with other things. I think, most of them should occur in mature projects, but for sure - you should know all of them.

So... What is...?

Screenshot testing

What is screenshot testing?

Screenshot testing is a technique used by end-to-end testing which captures a screenshot from a given URL and then compares the result with an expected image.

The most common usage of this type of test is to check if the user interface (UI) isn't broken and the page has been rendered properly.

Why you should do them?

Coming from old internet explorer to newest version of chrome. Using mobile and desktop in various OS. It's hard to check everywhere if our application renders properly. Not to mention a problem, where the developer will break something accidentally. A simple change could destroy our design.

How you can do them?

You can capture bulk screenshots through different desktop and mobile devices running in the various OS. All of them, you can compare with an expected image (snapshot). If the test will pass, you can deploy a new version of the application, otherwise, with the designed flow, you can ask a manager or designer to accept or reject changes. A simple diagram, I've prepared below.

Deployment diagram

But there is one thing you need to remember. You need to work closely with designers and developers. It's their job to accept changes in UI or revert them to the developer if something's broken.

Popular tools

UI testing

What is UI testing?

UI testing is a technique used by end-to-end testing whose main aspect is to check how the application handles user actions performed via input devices - mouse, keyboard, and others. Another one is to check if displayed elements on the page are interacting correctly.

Why you should do them?

Imagine that by an accident, your application has a faulty UI element, which is core for your business. Your users won't be able to get what they want, book an appointment, or buy a book. You will lose money and for sure clients, because with no trust in the application - you can't build the value.

By that, UI testing is fundamental. Tests will behave like a real user. Will interact with your application and check the output. What's better, they will do more than check the presentation layer. A complex test needs the business and data layer also working.

How you can do them?

There are various tools created for control browsers. Some of them, you will find at the end of this caption. These tools will give you the possibility to create a test, which behaviors should be designed as a real user. Running them every every deploy, you'll always know, if somethings break. Also, you can set up these tests in the deployment pipeline.

But... Not all that's pretty. These tests are slow. By slow, I mean that everyone needs to load a page, interact with it, probably reload, etc. This takes time. Many tests, takes much time to create an output, so it's important to run only the core one. More about it, you'll find in my latest article - click!

For sure, you shouldn't give up on them. We always need to provide a complete working application.

Deployment diagram

Popular tools

API testing

What is API testing?

The API testing is a set of actions, which includes sending calls to API to get output and validate the response against defined input parameters. They determine if the API returns the correct response or react correctly to failures or unexpected behavior. Instead of using standard user inputs, we use the software.

They're different than UI testing because don't concentrate on the presentation layer and the feel of the application. The main point is to check the business layer.

Why you should do them?

When you modify code in the backend, automated UI tests can be time-consuming and repetitive. With API testing we skip UI, which makes tests much faster. That's why we will know faster if our changes break business behavior.

How you can do them?

These tests can run similarly to UI ones. The main aspect is that you need a freshly build API, then you can run tests and base on the result deploy the new version of the application, or reject changes.

There are many various tools to perform API tests. Some of them, I've put below. Designing and writing API tests is crucial for business work. Especially, when your architecture is built on micro-services. You can check every one separately.

Deployment diagram

Popular tools

Performance testing

What is Performance testing?

Performance testing is a non-functional testing technique that determines the speed, scalability, stability, and responsiveness of an application under a particular workload. The goal of this type of test is to evaluate application output, speed, data transfer velocity, network bandwidth usage, and response times.

Why you should do them?

You need to know if application speed meets expectations on production. Performance is a key determinant of adoption, success, and productivity.

Running them, you will identify performance-related bottlenecks, which affect the user and created a poor experience - where you lost money. Bottlenecks are a single point or component within a system's overall function, that holds back overall performance.

You can also use them to compare two or more systems and identify the one that performs better.

How you can do them?

This is specified by organization and application. It depends on what you need and what the business considers most important.

You can use some of the tools, which I've listed below.

Deployment diagram

Researching this post I've found some interesting articles on how to build a pipeline with performance tests. I'm not sure if it's working well, but - why not to get a try?:)

Here a few links:

Popular tools

Conclusion

The listed types of tests has pros and cons. All of them are useful, and many you should have in your projects. Of course, everything depends on needs.

We've discussed:

Screenshot testing,
UI testing,
API testing,
Performance testing.

I hope, that this article clarifies to you a bit which types could be useful where and for what we're doing them.

Have a nice day! 🥳

What is smoke, sanity, regression testing? How to CI/CD with them.

Sat, 12 Dec 2020 17:00:00 GMT

With growing test architecture, we are facing various problems. Execution time, flakiness, duplicate test cases etc. Tests are becoming hard to manage and we may start to feel like writing and maintaining them is meaningless.

While lurking on reddit I came across a discussion about test types. Then it hit me - why can't we categorize them and instead of one long run, perform them group by group? We won't lose any coverage, and we'll reduce time to market. 🥳

Types of end-to-end tests

Sticking to E2E tests only, after reading the reddit discussion and thinking about the topic, I've decided classify them into:

smoke testing,
sanity testing,
regression testing.

You probably already know the last group very well (if not → click here).

Now let's dig into each of the types mentioned before.

Smoke testing

The main characteristics of these tests are:

They're performed to cover critical functionalities of the application (Priority1 scenarios),
The objective is to verify "stability" of the application to continue with further testing,
They're a subset of regression tests,
They're like a general health check up,
Failing these tests results in instant rejection.

Simply speaking, smoke testing should cover the most critical parts, the ones that let us confirm that the applications is working and delivers business value.

For example, this kind of scenario could be in this category:

[Netflix.com] User can log in, choose a tv series/movie and start watching.

Sanity testing

Next group is sanity tests, whose main aspects are:

They're performed to check more detailed functionalities of the application (Priority2 scenarios),
The objective is to verify "rationality" of the application to continue with further testing,
They're a subset of regression tests,
They're like a detailed health check-up,

In simple words, sanity tests should cover more specialized user actions.

Here's an example of a scenario that falls into this category:

[Netflix.com] User can search for movies using search bar.

As you can imagine, sanity tests may have some test cases in common with smoke tests but also there are certain differences between these categories.

Regression testing

Last but not least - regression tests:

They're performed to check detailed functionalities of the application (Priority1 + Priority2 + Priority3 scenarios)
The objective is to verify every existing feature of the application,
Regression test cases should be well documented,
They're like a comprehensive health check-up,

Simply speaking, regression tests should cover all the aspects of application - the important and less important ones. Executing regression tests should give us an overview that everything works properly.

For example, a scenario that could be in this category is:

[Netflix.com] User can open "more information" about a tv series/movie and choose genre.

Amount of tests by type

Regression tests > Sanity tests > Smoke tests.

And here's a visual representation of how the sets intersect.

Arrange a deployment process based on test types

I've worked with one test database and deployment process for a while, and I figured that it's not perfect and not bad, but as it grows, it needs some changes.

Depending on the type of tests, we can choose the most important ones and those that are less important. As we all know, the most important ones are those that deliver the business value, so we have to check if those functionalities work as expected with every deploy. Based on these criteria we can design our deployment process.

Working with an application based on CI/CD carries potential challenges such as designing a deployment process with automation testing. Because we don't want it to be long, we can categorize the tests, and run each group at a different time.

As you can see, only smoke tests are run with each application deploy. This way we'll always know that the most important functionalities work.

Conclusion

As the number of your tests grows, they become harder to maintain. Running all of them after every deployment consumes too much time. If you organize them into categories you will reduce time to market without losing any necessary coverage.

How you categorize your tests depends on you. They don't necessarily need to be divided into smoke, sanity and regression categories. You can do it differently, for example, by criticality level.

Automated testing in deployment process is a standard practice in modern software development, used by the best teams and companies. CI/CD should therefore depend on the results of our tests, which will ensure that the critical functionalities always work.

Have a nice day! 🥳

Who is a Quality Assurance Engineer and what should he know?

Sun, 17 Jan 2021 15:00:00 GMT

Have you ever think about steps in the testing journey? Manual tester, QA, QA Engineer, Test Developer, SDET, TestOps? There are many ways you can develop in. But today, I want to stick with one - QA Engineer.

What does exactly this person do? What are the responsibilities and requirements? What should he know to work efficiently on daily basis?

I've summed up areas and described basing on daily tasks.

What does a QA Engineer do?

QA Engineer means Quality Assurance Engineer is a specialist whose activities are aimed to improve the software development process and product quality using engineering knowledge. Manual tests aren't optimal in most cases. QA engineers automate the most common cases, which saves us time and money.

Responsibilities:

write, design, and execute automated tests,
maximize test coverage,
determine the priority for test scenarios,
create execution plans for these scenarios,
write documentation for automated processes,
build automation frameworks,
set up continuous integration

And to reach the QA Engineer job, you need to know various things. I've categorized and described them.

Methodologies

You'll be obligated to write, design, and execute automated tests. It means you have to know how to do it. Writing automation tests isn't something very different than you're doing manually. I mean, to write this one, you have to perform these steps manually. So you have to take exactly the same steps as a user, and write this as a script. Experience with manual testing will let you write these scripts better.

From time to time, you'll also decide what's important, and what's not. What should be tested next, and how to maximize test coverage. Experience and knowledge about methodologies will help you with these decisions.

The priority of scenarios? Another important topic. Creating complex test solutions, most sure you'll face a problem with them. What should we automate first and why? What should we run first? These are hard questions, but important. We should always focus on more important parts of the application. Because we don't want to show the user some unexpected behaviors, bugs, or unavailability of the website. They'll lose trust in our product, which we can't afford.

Programming language

Writing tests, creating test frameworks and architecture is the same as with applications. To do this, you need to know a programming language.

Some of the wanna-be test developers, decide to learn frameworks first. Without the proper programming knowledge, they're trying to write tests. Probably this gonna work, but this solution won't be maintainable so much. While growing, they will encourage more and more problems, because basics were created wrong. Fix these mistakes lately, will cost much more time and afford than proper planning.

But language knowledge isn't in my opinion enough. You should also be familiar with the concepts of programming. KISS, YAGNI, DRY, SOLID - right now, they could sound for you like a buzz word, but believe me - you'll use them daily. Knowing them will give you the opportunity of creating more maintainable and better solutions.

Sometimes you'll also need to pass framework methods. Instead of using a page.click() you'll need to perform language native one. Sometimes, you'll need an API call that your test framework doesn't wrap up. You'll face many cases, where knowledge of the language is important.

Languages:

JavaScript/TypeScript,
Java,
C#,
Python

Test tools

To write a test we're using test frameworks. These are tools set, out of the box, which gives you an easy way to control the browser. It means, writes a user like a scenario end's with a few methods. We don't have to reinvent a wheel, write from scratch integration with browser, etc.

Tooling is important, but concepts are more. While you know one test framework, you can easily switch to another. Most of them are very similar, it's like with a programming language, while you know the concepts and knows how to program, a language is literally a tool.

Example test frameworks:

Playwright,
Cypress,
Selenium

Processes

One of the bigger challenges is to set up pipelines. Nowadays, we're using Continuous Integration practice. Design and build a whole process is a very valuable skill.

You need to understand how deployment works, where you can run tests, how to do this?

To reach this, be familiar with topics like:

Docker,
CI/CD - Gitlab, Jenkins,
Cloud - AWS, Azure

There's a role called TestOps which is a connection between operations and testing, but I think mature QA Engineers should know the basics about processes, to be able to set up them from scratch.

Conclusion

QA Engineer role requires knowledge of many different areas. All of them affect the final solution, which with this knowledge will be better.

Knowledge of these areas also allows you to develop in various roles. Dive into processes? You can go into the TestOps position. Development? Then pick SDET.

Have a nice day! 🥳

Why do most test engineers fail interviews?

Fri, 20 Nov 2020 08:00:00 GMT

Recently I had the pleasure of interviewing candidates for my QA team. Sometimes, they were hard. You may ask why so I'll try to explain. I hope that in the future we can teach and prepare wanna-be test engineers to pass every interview they want to:)

Our roadmap for this article:

Basics of the language
Compatibility with the team
Problem-solving
Conclusion

Basics of the language

In my opinion, knowing the basics of the language you want to use to write automated tests is quite important to write them well.

Many wanna-be test engineers (and not only beginners) know how to write a test, but when it comes to creating a custom solution - it could be hard without the basics of the language. In my daily work I'm facing various problems. Sometimes I'm just writing tests, which is kinda repetitive and easy, but there’s been many situations when we had to create some custom solution to simplify our work (like our custom runner, based on jest or TestKeeper which is an application to handle tests by micro-services). In this situation, I'm pretty sure that they could write this, but the quality of the code would probably be awful and hard to maintain. It's not the point:)

And to give you an example. One of my questions was very simple: what’s the difference between let, const, and var? It's a very basic question often asked in interviews. And you know what? Many of the candidates couldn't answer it. I was kinda shocked because it's like the first question from the Top Ten JS interview questions.

Compatibility with the team

Another important aspect of the recruitment process is your compatibility with the team. I mean, if your approach and personality doesn't fit the team – you won't be processed to the next stages. And there’s nothing wrong with that. It's not your fault but you have to be aware of this.

I'm mentioning this because for us, leaders, it's a very important topic when building the right team.

But... I can give you some tips, what skills a good tester should have:

proactivity,
problem-solving,
business point of view,
flexibility,
quick learner,
collaboration and social skills,
communication skills,

Problem-solving

As you may have guessed, the quality that I particularly value is the problem-solving attitude. And it’s essential not only during the interview but also on a daily basis. But what does it mean?

In my opinion, a good tester shouldn't be afraid of confronting problems. Sometimes people come to us and ask difficult questions – it's just daily work. We should try to face them and if at that moment we don't know the answer – ask around and try to find it. Probably this asking part will be connected with doing something so treat it as a challenge.

In my recent interviews I met various candidates. Some of them, when given a task, tried to solve it, even if they haven’t encountered a similar problem before. The answer from another group was "I don't know", and it was wrong. I mean, when you don't have experience in a given topic, just say it and try to answer what you feel it could be. I'm not saying it's the best approach every time to every question but I really like it when someone tries to give an answer with their own words, just thinking about how it could work. We could at that moment recognize how each of the candidates handled challenges and problems.

Conclusion

Going through interviews is hard and most of the time – stressful. It's not a piece of cake but proper preparation before the talk should help you get the job. In my opinion, as I've written above, you have to go with a proper attitude and thorough knowledge of the language basics. But of course it's not everything, these are just the fundamentals you should have. The rest depends on the position you’re applying for, your seniority and experience.

I hope this article gives you an idea of some basic problems which are easy to tackle and could boost your confidence in interviews.

Useful articles to help you build your future career:

Have a nice day!

Why it's so hard to get rid of bad habits?

Sat, 24 Oct 2020 10:00:00 GMT

Some time ago I read a book - The Power of Habits, written by Charles Duhigg. I wanted to learn, how to manage my habits and the most - how to get rid of the bad ones. The author of this book tries to prove, that everything which we're doing can change. Just by changing our doing path. This can change our whole life.

What is a habit?

Habit is the thing you're doing without thinking. For example, if you're daily eating dinner at 1 p.m. - it's a habit. In coding - if you, for example: have own style of brackets,

if () {

}

if ()
{

}

You aren't thinking when writing brackets in your code. We call this easily as a habit. Bad, good - it's your pick.

Types of habits

There are good habits, neutral ones and also... bad ones.

Of course, we want to get rid of the last ones. But we can't. Why?

Bad habits are challenging. They're difficult to overcome, especially, when we have them for the whole life.

How to get rid of them?

We should follow two steps:

Recognition. You have to look at them and ask yourself: How long I'm doing this shit?
Realize. You have to realize, that what you're doing is wrong. You have to realize, that the process to change this habit won't be comfortable. It's gonna be hard.

Take the example of trying to eat more healthly.

Starting from being fatty, if you develop a habit of doing workout 3-4 times a week and eating healthier, in some months you can change your styling about one hundred percent. It sounds great, yea? But it's hard to develop. We aren't prepared to change our habits (like eating trashy food) in one second. Saying: "I won't eat that food from now" and trying to do this - probably you will fail. That's about what Charles tries to explain - we have to do this by small steps. We want to reach the goal fast, but we can't. We will fail. But if we will start with small steps, as I mentioned earlier, we could probably win.

For an example of eating healthy. Start with eating one of five meals healthier. After some time, add another one, and another and another. In the end, you will change your whole eating habit and probably, you won't want to get back into trashy one, because your style change and you will look better.

We can do this way with every habit. I've used this for reading books. A few years ago I didn't read. Zero books in a whole year - for now, me, it sounds ridiculous, such a waste of time. I could read many brilliant books and have much more knowledge than now, which probably could give me more money or feel better with myself.

But you still need to remember. It's going to take time. It's going to take commitment, and it's going to take perseverance.

It's a slow process.

Applications

Apps can help out us here. Many of them allow us to pick a habit which we want to develop and every day if we mark as done, the line will look good (line of progress).

I'm using https://www.thehabithub.com/ and totally recommend it. It's simple, easy to use application decicated to habits. Try it:)

Similar apps you can find in https://www.reddit.com/r/Habits/, where are many fantastic people which trying to change their lives, by improving habits.

Also, you can try Habitica, which I didn't even try, but many people recommended it to me.

Why you should have financial cushion?

Thu, 03 Sep 2020 15:00:00 GMT

Yesterday I didn't know how to manage an income. I thought it's very complex to build a financial cushion, manage two accounts, invest money or just simply not live beyond the state.

I've made some simple steps to save money for my future. And if we want to talk about saving, I have one rule.

Don't be stingy.

If I want to buy a new laptop? Why not - it's my cash, I earn it. But not so fast, this expense shouldn't be as fast as I wrote this sentence. I've rule, that if I think about it for about month (or less, but still not like to think -> buy in ten seconds), this says to me that I need it. It's even better if this expense is helpful in earning money.

Why I need financial cushion?

Financial cushion guarantees you, that if you have smaller or bigger expenses and troubles at that time, you won't have to take an loan to finance them. Pillows give you an opportunity to move on this hard time without any major perturbations. But imagine, that at this moment you don't have savings - it's a straight line to snare bank and living on credit.

That's why financial cushion is like armor to help you go through financial problems easy way.

Ok, but how big it should be?

That's a tricky question. My personal savings need to cover my normal life (without a job) for about half a year. But I'm single, living in a big city without any large monthly expenses. In a fast way, I can go back to my family's house and cut my costs by about 50%. You can have a harder situation.

I think fit for everyone divisions should be:

3-6 months -> if you are single and you're not a source for family,
6-12 months -> if you have a family and kids,
12+ months -> if you have a family and running a company.

How to do this right?

I've got a simple system to manage my incomes. We need two accounts:

the primary account is where your income goes. You pay bills from this account, have a credit card connect with it and you cover all expenses from this account,
the secondary account is only for savings. You store here your cash and this account doesn't have a connected credit card, so you can't easily pay with this money. I also prefer to limit this account only to one transfer a month (of course, two and more are possible, but you have to pay for them).

But I strongly recommend splitting your financial cushion from typical savings for a target. Probably you need some kind of investment account, but be careful, because if you need this money, you could be able to use them in a short time.

But always remember.

Pay yourself first

If you want to build a financial cushion fast, you need to prepare a plan and stick to it. In my opinion, if you won't pay yourself first a specified amount of money, sometimes you will forget to do it, or just pay a smaller amount for any reason. It's dangerous. The best if you set up an automatic payment a few days after income goes to your primary account.

I hope you will get something from this post. We all need to be prepared to worst times than we expect they're gonna be - it's important.

Have a nice day.

Blog | kodziak.com - Przemysław Paczoski

Arrays and objects destructuring in JavaScript

Destructuring assignment

Object destructuring

Summary

Automated testing setup for smart contracts: a practical guide

Why Automated Testing for Smart Contracts?

Setting Up Your Testing Environment

Writing Your First Test

Automating Your Tests

Best Practices for Smart Contract Testing

Wrapping Up

Creating Test Data for DeFi: Beyond Simple ERC20 Mocks

The Problem with Simple Mocks

🛠 Crafting Realistic Test Tokens

1. Fee-on-Transfer Tokens

2. Rebasing Tokens

3. Blacklistable Tokens

🎭 Simulating Real-World Scenarios

🧠 Why This Matters

🚀 Leveling Up Your Testing Game

First steps with end-to-end testing using jest + playwright (puppeteer)

What do I need to start with automatization?

Project setup with Playwright and JestJS

Let's write an automated test

Bonus: Test recorder!

Conclusion

First steps with UI test automation

Do I have to be a manual tester before?

What programming language should I pick?

What framework/library?

What to automate first?

Where to search for information?

Summary

Gas optimization through testing: how we reduced costs by 40%

The Problem: Gas Guzzling Contracts 🚗💨

The Solution: Batch Those Transfers! 🚀

The Testing Strategy 🧪

The Results: 40% Gas Savings 💰

Key Takeaways 🗝️

What's Next? 🚀

How much does web3 testing really cost? Breaking down the numbers

The True Cost of Web3 Testing: It's Not Just About the Tools 🧰

Hidden Costs That'll Sneak Up On Ya 🕵️‍♂️

Show Me The Code: A Real-World Example 💻

Strategies to Keep Costs in Check 💰

The Bottom Line: What Should You Budget? 🤔

Wrapping Up: The Real Value of Web3 Testing

How to make your UI automation tests resilient to changes?

What are the common problems with CSS selectors?

Use data-test-id for help!

But should I use them everywhere?

What are the benefits of using them?

Conclusion

Importance of CI/CD and parallelisation in test automation

How can CI/CD help with test automation?

Loooong time execution...

Tests by groups

Multiple tests at a time

Spread knowledge about tests

Conclusion

Integration Testing in Web3: Connecting Smart Contracts with Frontend

The Challenge: Bridging Two Worlds

Step 1: Set Up Your Testing Environment

Step 2: Mock Blockchain Interactions

Step 3: Test Contract-Frontend Interaction

Common Pitfalls and How to Avoid Them

Best Practices for Web3 Integration Testing

Wrapping Up

Load testing types: load, stress, soak, spike. Examples in k6 load testing tool

Types of load tests

Load testing

Stress testing

Soak testing

Spike testing

Summary

Separate or project repository to store tests? Which, why, and when?

Project repository

Example structure

Separate repository

Use `data-test-id` for help!