Smoke Testing Your SaaS: A Practical Guide for Founders

Learn how to implement effective smoke testing for your SaaS application. This guide covers essential test scenarios, implementation strategies, and best practices to quickly verify core functionality.

Looking to build a robust Next.js SaaS application? Smoke testing is essential for quickly verifying your core functionality works after each deployment. This guide will show you how to implement smoke tests in your Next.js SaaS boilerplate or starter kit.

What is Smoke Testing?

Smoke testing verifies your application's critical features work after deployment. It's like a quick health check for your SaaS - run a few key tests to ensure nothing's broken before doing deeper testing.

For SaaS applications, smoke tests typically:

  • Run quickly (usually under 5 minutes)
  • Cover only the most critical user paths
  • Verify core functionality works
  • Run after every deployment
  • Serve as an early warning system

Essential Smoke Test Scenarios for SaaS

1. Authentication Flow

test('basic auth flow', async ({ page }) => {
  const auth = new AuthPageObject(page);
  
  // Test sign up
  await auth.signUp({
    email: 'smoke@example.com',
    password: 'smoketest123'
  });
  await expect(page).toHaveURL('/dashboard');
  
  // Test logout
  await auth.logout();
  await expect(page).toHaveURL('/login');
  
  // Test sign in
  await auth.signIn({
    email: 'smoke@example.com',
    password: 'smoketest123'
  });
  await expect(page).toHaveURL('/dashboard');
});

2. Payment Integration

test('subscription flow', async ({ page }) => {
  const billing = new BillingPageObject(page);
  
  // Verify pricing page loads
  await billing.visitPricingPage();
  await expect(page.locator('[data-test="pricing-plans"]')).toBeVisible();
  
  // Test basic subscription flow with test card
  await billing.selectPlan('starter');
  await billing.fillPaymentDetails({
    card: '4242424242424242',
    exp: '12/25',
    cvc: '123'
  });
  
  await expect(page.locator('[data-test="subscription-success"]')).toBeVisible();
});

3. Core Feature Verification

test('core features', async ({ page }) => {
  const dashboard = new DashboardPageObject(page);
  
  // Create a new project
  await dashboard.createProject('Smoke Test Project');
  await expect(page.locator('[data-test="project-title"]'))
    .toHaveText('Smoke Test Project');
  
  // Verify basic CRUD operations
  await dashboard.addItem({ name: 'Test Item' });
  await expect(dashboard.getItemsList()).toContainText('Test Item');
  
  await dashboard.updateItem('Test Item', { name: 'Updated Item' });
  await expect(dashboard.getItemsList()).toContainText('Updated Item');
  
  await dashboard.deleteItem('Updated Item');
  await expect(dashboard.getItemsList()).not.toContainText('Updated Item');
});

Implementing Smoke Tests in Your CI Pipeline

GitHub Actions Configuration

name: Smoke Tests
on:
  deployment_status:
jobs:
  smoke:
    if: github.event.deployment_status.state == 'success'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
      
      - name: Install dependencies
        run: npm install
      
      - name: Install Playwright browsers
        run: npx playwright install --with-deps chromium
      
      - name: Run smoke tests
        run: npm run playwright test --grep "smoke"
        env:
          BASE_URL: ${{ github.event.deployment_status.target_url }}
          CI: true

Best Practices for Smoke Testing

1. Keep Tests Focused and Fast

Your smoke test suite should run in under 5 minutes. Focus on mission-critical paths:

  • User authentication
  • Payment processing
  • Core feature accessibility
  • Basic CRUD operations
  • API health checks

2. Use Environment-Specific Test Data

// config/smoke.config.ts
export const smokeConfig = {
  testUser: {
    email: process.env.SMOKE_TEST_EMAIL || 'smoke@example.com',
    password: process.env.SMOKE_TEST_PASSWORD || 'smoketest123'
  },
  testCard: {
    number: '4242424242424242',
    exp: '12/25',
    cvc: '123'
  }
};

3. Implement Health Checks

test('api health check', async ({ request }) => {
  const response = await request.get('/api/health');
  expect(response.ok()).toBeTruthy();
  
  const json = await response.json();
  expect(json.status).toBe('healthy');
  expect(json.dependencies.database).toBe('connected');
  expect(json.dependencies.cache).toBe('connected');
});

4. Monitor and Alert

test.afterEach(async ({ }, testInfo) => {
  if (testInfo.status !== 'passed') {
    const { WebClient } = require('@slack/web-api');
    const slack = new WebClient(process.env.SLACK_TOKEN);
    
    try {
      await slack.chat.postMessage({
        channel: process.env.SLACK_CHANNEL,
        text: `🚨 Smoke test failed: ${testInfo.title}`,
        blocks: [
          {
            type: "section",
            text: {
              type: "mrkdwn",
              text: `*Smoke Test Failed*\n*Test:* ${testInfo.title}\n*Error:* ${testInfo.error}`
            }
          }
        ]
      });
    } catch (error) {
      console.error('Failed to send Slack notification:', error);
    }
  }
});

Simple Monitoring Integration

Here's a straightforward way to track your smoke test results:

// monitoring/results.ts
interface TestResult {
  name: string;
  duration: number;
  status: 'passed' | 'failed';
  timestamp: number;
}
export async function reportTestResults(results: TestResult[]) {
  try {
    // Send to your error tracking service (e.g. Sentry, LogRocket)
    const failedTests = results.filter(test => test.status === 'failed');
    if (failedTests.length > 0) {
      await notifyTeam(failedTests);
    }
    // Log to your existing monitoring solution
    console.log(JSON.stringify({
      timestamp: Date.now(),
      tests: results,
      summary: {
        total: results.length,
        passed: results.filter(t => t.status === 'passed').length,
        failed: failedTests.length,
        avgDuration: results.reduce((acc, t) => acc + t.duration, 0) / results.length
      }
    }));
  } catch (error) {
    console.error('Failed to report results:', error);
    throw error;
  }
}

Testing Environments: Where to Run Your Tests

To explain visually the flow of smoke testing, let's visualize it in a diagram:

Smoke Tests in Different Environments

  1. Staging Environment (Recommended Primary)
    • Run full smoke test suite after every deployment
    • Safe environment to test payment integrations
    • Catches issues before they reach production
    • Should mirror production as closely as possible
  2. Production Environment (Essential but Limited)
    • Run minimal, non-destructive smoke tests
    • Focus on read-only operations
    • Avoid test data creation
    • Never test payment flows
    • Example production-safe tests:
      test('production health checks @prod', async ({ request }) => {
        // Check public API endpoints
        const healthCheck = await request.get('/api/health');
        expect(healthCheck.ok()).toBeTruthy();
        
        // Verify critical pages load
        const marketing = await request.get('/');
        expect(marketing.ok()).toBeTruthy();
        
        // Check authentication endpoint availability
        const auth = await request.get('/api/auth/csrf');
        expect(auth.ok()).toBeTruthy();
      });

E2E Tests in Production?

Production E2E testing is controversial but can be valuable when done right:

Benefits:

  • Catches real-world issues
  • Verifies actual user experiences
  • Monitors true system performance
  • Identifies third-party integration issues

Risks:

  • Potential data corruption
  • False positives from real user interactions
  • Risk of triggering real transactions
  • Privacy concerns with user data

Safe Production E2E Strategy:

// Configure separate test modes
const config = {
  production: {
    // Read-only operations only
    allowedOperations: ['GET', 'HEAD'],
    // Skip sensitive flows
    skipTests: ['payment', 'email', 'delete'],
    // Use dedicated test accounts
    testAccounts: {
      viewer: 'prod-test-viewer@company.com',
      // No admin test accounts in prod
    }
  },
  staging: {
    // All operations allowed
    allowedOperations: ['GET', 'POST', 'PUT', 'DELETE'],
    skipTests: [],
    testAccounts: {
      admin: 'staging-admin@company.com',
      viewer: 'staging-viewer@company.com'
    }
  }
};
// Example production-safe test
test('critical user journey @prod', async ({ page }) => {
  const env = process.env.TEST_ENV || 'staging';
  
  // Skip if test isn't production-safe
  test.skip(
    env === 'production' && 
    !config[env].allowedOperations.includes('POST'),
    'Test requires write operations'
  );
  // Use environment-specific configuration
  const account = config[env].testAccounts.viewer;
  await page.login(account);
  
  // Verify critical path
  await expect(page.locator('dashboard')).toBeVisible();
  await expect(page.locator('user-menu')).toContainText(account);
});

Alternative: Health Checks

Instead of running full E2E tests in production, consider simple health checks:

// monitoring/health.ts
export async function checkHealth() {
  const criticalPaths = [
    { path: '/api/health', name: 'API Health' },
    { path: '/api/auth/session', name: 'Auth Service' },
    { path: '/pricing', name: 'Pricing Page' }
  ];
  for (const { path, name } of criticalPaths) {
    try {
      const start = Date.now();
      const response = await fetch(path);
      const duration = Date.now() - start;
      if (!response.ok) {
        throw new Error(`${name} returned ${response.status}`);
      }
      console.log(`✅ ${name}: ${duration}ms`);
    } catch (error) {
      console.error(`❌ ${name} failed:`, error);
      await notifyTeam({
        service: name,
        error: error.message,
        timestamp: new Date().toISOString()
      });
    }
  }
}

This approach provides:

  • Continuous uptime monitoring
  • Performance tracking
  • Early warning system
  • No risk of data corruption
  • Real-world latency metrics
  1. Staging Environment:
    • Full smoke test suite
    • Complete E2E test coverage
    • Payment integration testing
    • Data modification tests
  2. Production Environment:
    • Minimal, read-only smoke tests
    • Synthetic monitoring
    • Uptime checks
    • Performance monitoring
    • Limited, safe E2E tests

This balanced approach gives you confidence in your deployments while protecting your production environment.

Common Pitfalls to Avoid

  1. Too Many Tests
    • Keep your smoke test suite minimal
    • Focus on business-critical paths
    • Move detailed tests to your full E2E suite
  2. Unreliable Test Data
    • Use dedicated test accounts
    • Reset test data before each run
    • Never rely on production data
  3. Missing Error Handling
    • Implement proper error reporting
    • Set up alerting for failures
    • Track test metrics over time

Smoke Testing vs. E2E Testing

While our previous article covered comprehensive E2E testing, smoke testing serves a different purpose:

Smoke Tests:

  • Scope: Critical paths only
  • Duration: Less than 5 minutes
  • Frequency: Every deployment
  • Coverage: 10-20% of functionality
  • Purpose: Quick verification

E2E Tests:

  • Scope: Comprehensive
  • Duration: 15-60 minutes
  • Frequency: Daily/Weekly
  • Coverage: 70-90% of functionality
  • Purpose: Deep validation

Conclusion

Smoke testing is your first line of defense against major issues in your SaaS application. By implementing a focused set of smoke tests, you can:

  • Catch critical issues quickly
  • Reduce deployment risks
  • Maintain continuous delivery
  • Build confidence in your deployment process

Remember, the goal of smoke testing isn't to catch every possible issue - it's to quickly verify that your application's core functionality works as expected. Combined with comprehensive E2E testing, smoke tests form a crucial part of your quality assurance strategy.

Start with the essential scenarios outlined in this guide, and gradually expand your smoke test suite based on your application's specific needs. The time invested in setting up proper smoke tests will pay off in reduced downtime, faster issue detection, and more confident deployments.

About Makerkit

Looking for a production-ready SaaS starter kit with built-in testing infrastructure? Check out Makerkit, a Next.js SaaS starter kit that comes with pre-configured smoke tests and E2E testing setup.