⚙️ Automating Test Documentation — Generate a Live Playwright Test Plan in Excel

🎯 The Problem

As automation suites grow, it becomes difficult to answer a very basic question:

“What tests do we actually have, and what areas do they cover?”

Playwright provides great runtime reports, but it lacks a static overview — a way to visualize all tests, titles, and tags without running them.

That’s what inspired me to build a Static Test Info Writer — a small utility that scans all Playwright test files and generates a clean Excel report. No test execution. Just a single command → instant visibility.

🧩 The Concept

Playwright has a little-known command:

npx playwright test --list --reporter=json
        

It lists every test, its hierarchy, metadata, and annotations — all without running them. Using this as a base, we can build a discovery + export pipeline that:

1. Collects test metadata statically.
2. Formats it neatly.
3. Exports it to Excel with filters and styling.

🧱 Architecture Overview

🧭 Collector Uses Playwright CLI to gather test metadata recursively.

📘 Exporter Converts structured test info into an Excel sheet using ExcelJS.

⚡ Runner Script A one-liner to generate a live test plan anytime.

🧩 1️⃣ Test Collector (test-collector.ts)

This module scans the project, invokes Playwright’s --list mode, and collects metadata like titles, tags, annotations, and file paths.

import config from 'playwright.config.js';
import fs from 'fs';
import path from 'path';
import { spawnSync } from 'child_process';

export interface DiscoveredTest {
  title: string;
  file: string;
  line?: number;
  column?: number;
  tags?: string[] | undefined;
  annotations?: string[] | undefined;
  describeBlock?: string | undefined; // immediate parent describe block
  describePath?: string[] | undefined; // full path of describe blocks
}

export function collectTests(testFolder?: string) {
  let testDir = config.testDir;
  console.log('Test Directory:', testDir);
  testDir = testDir ?? './tests';
  const files = fs
    .readdirSync(path.join(testDir), { recursive: true })
    .map(file => (typeof file === 'string' ? file : file.toString()))
    .filter(file => file.endsWith('.spec.ts') || file.endsWith('.test.ts'));
  console.log('Test Files:', files);
  const discovered = collectTestInfo(testFolder);
  console.log('Discovered Tests:', JSON.stringify(discovered, null, 2));
  return discovered;
}

export function collectTestInfo(testFolder?: string): DiscoveredTest[] {
  console.log('Collecting test information...');
  const isWin = process.platform === 'win32';
  const bin = isWin
    ? path.join(process.cwd(), 'node_modules', '.bin', 'playwright.cmd')
    : path.join(process.cwd(), 'node_modules', '.bin', 'playwright');

  let proc;
  if (isWin) {
    proc = spawnSync(
      'cmd.exe',
      ['/c', bin, 'test', testFolder ? `${testFolder}` : '', '--project=chromium', '--list', '--reporter=json'],
      {
        cwd: process.cwd(),
        encoding: 'utf8',
        env: { ...process.env, PW_TEST_HTML_REPORT_OPEN: 'never' },
        maxBuffer: 1024 * 1024 * 10,
      },
    );
  } else {
    proc = spawnSync(
      bin,
      ['test', testFolder ? `${testFolder}` : '', '--project=chromium', '--list', '--reporter=json'],
      {
        cwd: process.cwd(),
        encoding: 'utf8',
        env: { ...process.env, PW_TEST_HTML_REPORT_OPEN: 'never' },
      },
    );
  }
  if (proc.error) throw proc.error;
  if (proc.status !== 0) {
    throw new Error(`Playwright discovery failed:\n${proc.stdout}\n${proc.stderr}`);
  }
  const content = proc.stdout;
  let jsonContent = content;
  const startIndex = content.indexOf('{');
  const lastIndex = content.lastIndexOf('}');
  if (startIndex !== -1 && lastIndex !== -1 && lastIndex > startIndex) {
    jsonContent = content.substring(startIndex, lastIndex + 1);
  }
  const parsed = JSON.parse(jsonContent) as Record<string, any>;
  const out: DiscoveredTest[] = [];
  const seen = new Set<string>();
  const visit = (node: any, parents: string[] = []) => {
    if (!node) return;
    if (node.suites) {
      for (const s of node.suites) {
        const name = s.title || '';
        visit(s, name ? [...parents, name] : parents);
      }
    }
    if (node.specs) {
      for (const spec of node.specs) {
        console.log('Processing spec:', JSON.stringify(spec, null, 2));
        const testTitle = spec.title;
        const f = spec.file || node.file;
        const loc = spec.location || {};
        // Extract tags (string or array)
        let tags: string[] = [];
        let annotations: string[] = [];
        if (spec.tags) {
          if (Array.isArray(spec.tags)) {
            tags = spec.tags.map((tag: any) => String(tag));
          } else if (typeof spec.tags === 'string') {
            tags = [spec.tags];
          }
        }
        // Extract annotation descriptions only
        if (spec.tests) {
          const testAnns = spec.tests[0]?.annotations;
          if (testAnns && Array.isArray(testAnns)) {
            annotations = testAnns
              .filter((a: any) => typeof a.description === 'string')
              .map((a: any) => `${a.type} : ${a.description}`);
          }
        }
        // Create a unique key for deduplication
        const key = `${testTitle}|${f}|${loc.line}|${loc.column}`;
        if (!seen.has(key)) {
          out.push({
            title: testTitle,
            file: f,
            line: loc.line,
            column: loc.column,
            tags: tags.length ? tags : undefined,
            annotations: annotations.length ? annotations : undefined,
            describeBlock: parents.length ? parents[parents.length - 1] : undefined, // immediate parent describe
            describePath: parents.length ? [...parents] : undefined, // full describe path
          });
          seen.add(key);
        }
      }
    }
  };
  visit(parsed);
  return out;
}
        

🧾 2️⃣ Excel Exporter (export-tests-to-excel.ts)

This script converts the collected test metadata into a clean, styled Excel report — with frozen headers, filters, and zebra striping for easy readability.

import ExcelJS from 'exceljs';
import type { DiscoveredTest } from './test-collector';

export async function exportTestsToExcel(tests: DiscoveredTest[], outputPath: string) {
  const workbook = new ExcelJS.Workbook();
  const worksheet = workbook.addWorksheet('Tests', {
    views: [{ state: 'frozen', ySplit: 1 }], // Freeze header row
  });

  // Define columns
  worksheet.columns = [
    { header: 'Test Name', key: 'title', width: 40 },
    { header: 'Describe Block', key: 'describeBlock', width: 30 },
    { header: 'Describe Path', key: 'describePath', width: 40 },
    { header: 'Test Tags', key: 'tags', width: 30 },
    { header: 'Test Annotations', key: 'annotations', width: 50 },
    { header: 'Test Location', key: 'location', width: 30 },
  ];

  // Style the header row
  const headerRow = worksheet.getRow(1);
  headerRow.font = { bold: true, color: { argb: 'FFFFFF' } };
  headerRow.fill = {
    type: 'pattern',
    pattern: 'solid',
    fgColor: { argb: '2F75B5' }, // Professional blue color
  };
  headerRow.alignment = { vertical: 'middle', horizontal: 'center' };

  // Add table with filter
  worksheet.autoFilter = {
    from: { row: 1, column: 1 },
    to: { row: 1, column: 6 },
  };

  // Add border and zebra striping
  worksheet.views = [{ state: 'frozen', ySplit: 1 }];
  tests.forEach(test => {
    worksheet.addRow({
      title: test.title,
      describeBlock: test.describeBlock || '',
      describePath: test.describePath ? test.describePath.join(' → ') : '',
      tags: test.tags ? test.tags.join(', ') : '',
      annotations: test.annotations ? test.annotations.map((a, i) => `${i + 1}. ${a}`).join('\n') : '',
      location: `${test.file}${test.line ? ':' + test.line : ''}${test.column ? ',' + test.column : ''}`,
    });
  });

  const lastRow = worksheet.rowCount;
  worksheet.eachRow({ includeEmpty: false }, (row, rowNumber) => {
    row.eachCell({ includeEmpty: false }, cell => {
      cell.border = {
        top: { style: 'thin' },
        left: { style: 'thin' },
        bottom: { style: 'thin' },
        right: { style: 'thin' },
      };
      if (rowNumber > 1) {
        cell.alignment = { vertical: 'top', wrapText: true };
        row.fill = {
          type: 'pattern',
          pattern: 'solid',
          fgColor: { argb: rowNumber % 2 === 0 ? 'F5F5F5' : 'FFFFFF' },
        };
      }
    });
  });

  await workbook.xlsx.writeFile(outputPath);
  console.log(`Excel report generated at ${outputPath}`);
}
        

▶️ Running It

Add a small runner script like scripts/write-test-index.ts:

import { collectTests } from '../src/test-collector';
import { exportTestsToExcel } from '../src/export-tests-to-excel';

async function main() {
  const tests = collectTests();
  await exportTestsToExcel(tests, 'results/test-index.xlsx');
}

main();
        

Then add a shortcut in your package.json:

{
  "scripts": {
    "test:discover": "tsx scripts/write-test-index.ts"
  }
}
        

Run it anytime:

pnpm test:discover
        

✅ Output → results/test-index.xlsx

✅ Optional Next Step: You could easily extend this script later to:

• Auto-sync Excel results into your test management system (e.g., Azure Test Plans or Xray) or
• Add tag-based dashboards and charts.

For now, this static discovery alone gives your project instant visibility — and that’s a big step toward true TestOps maturity.

🧠 Key Takeaways

• Zero Execution → Lists every test statically.
• Real-Time Visibility → Always up-to-date with your repo.
• Excel Friendly → Easy to share with non-technical teams.
• CI Ready → Run in pipelines to keep test inventory current.

🧩 Closing Thoughts

This approach transforms your Playwright suite into a self-documenting test ecosystem — one that keeps your entire team informed without a single test run.

The generated Excel file isn’t just a report — it’s a data source that can power dashboards, integrate with test management tools, or feed into metric systems for coverage tracking and analytics.

By automating test documentation, you bridge the gap between development and quality visibility — ensuring that your tests don’t just validate code, but also communicate impact.