feat: integration test batching with timing-based auto-balancing

[nick-y-snyk]

Summary

Implements 4-way test splitting for integration tests across all OS platforms using hashicorp-forge/go-test-split-action with JUnit XML timing data for intelligent load balancing.

Changes

Makefile

• Added gotestsum tool installation target
• Follows existing pattern for tool management via .bin/ directory

GitHub Actions Workflow

• Matrix expansion: Added split-index: [0, 1, 2, 3] dimension (12 total jobs: 3 OS × 4 batches)
• Timing download: Downloads JUnit XML from main branch using dawidd6/action-download-artifact@v6
• Test splitting: Uses hashicorp-forge/go-test-split-action@v1 for intelligent distribution
• Individual uploads: Each batch uploads node-summary.xml to unique artifact
• Combine job: New combine-test-summaries job merges 4 splits per OS (main branch only)
• Platform-specific setup: Preserved Xvfb, clipboard tools, and Pact configuration

Implementation Pattern

Follows hashicorp-forge/go-test-split-action recommended setup:

1. Download merged timing data from main branch
2. Split tests using historical timing (or even file-count on first run)
3. Upload individual split results (1-day retention)
4. Merge all splits into canonical timing file (90-day retention, main only)

Testing Notes

First run behavior:

• No timing data exists yet on main
• Tests will split evenly by file count (8 files across 4 batches)
• After first successful run on main, timing data will be available for subsequent runs

Workaround for this PR:

• Temporarily commented out branch: main to test without main baseline
• Will download timing from current PR runs instead
• Uncomment before merging to production

Expected Results

• ✅ All 12 jobs should complete (3 OS × 4 batches)
• ✅ All 21 integration/smoke tests should execute across batches
• ✅ Individual JUnit XML files uploaded per split
• ✅ Combined timing file uploaded (if on main branch)
• ✅ Validation steps confirm tests ran and env vars work

Acceptance Criteria Coverage

From tech-spec:

• AC1-AC7: Functional requirements (batching, timing, artifacts, platform setup) ✓
• AC8-AC10: Error handling (continue-on-error, if: always()) ✓
• AC13-AC15: Integration (env vars, tool caching, git config) ✓
• AC11-AC12: Performance improvements (requires empirical validation after this run)

/cc @snyk/hammerhead

[snyk-io]

✅ Snyk checks have passed. No issues have been found so far.

Status	Scanner	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues
✅	Licenses	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

[gemini-code-assist]

Summary of Changes

Hello @nick-y-snyk, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the integration test infrastructure by implementing a robust batching and load-balancing mechanism. It aims to improve test efficiency and feedback speed by distributing integration tests across multiple parallel jobs, intelligently balancing the workload based on historical timing data. This change also integrates gotestsum for better test reporting and management.

Highlights

• Integration Test Batching: Introduces a system for splitting integration tests into batches across different OS platforms, leveraging hashicorp-forge/go-test-split-action.
• Timing-Based Auto-Balancing: Implements intelligent load balancing for test batches using JUnit XML timing data, aiming to optimize test execution time.
• gotestsum Tool Integration: Adds gotestsum to the project's Makefile for streamlined installation and management, aligning with existing tool management practices.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Ignored Files

• Ignored by pattern: .github/workflows/** (1)
  • .github/workflows/build.yaml

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces test batching for integration tests using gotestsum to improve CI performance. However, a significant security concern is the use of the latest version tag for gotestsum, which can lead to non-deterministic builds and increases the risk of supply chain attacks by automatically pulling potentially compromised upstream packages. To ensure robust dependency management, build stability, and reproducibility, it's crucial to pin gotestsum to a specific version, aligning with the established practice for other tools in the Makefile.

[gemini-code-assist]

The GOTESTSUM_V variable is set to latest, which causes the go install command to pull the most recent version of the tool every time the tools target is run. This introduces non-determinism into the build process and poses a supply chain risk. If the upstream gotestsum package is compromised, the malicious version will be automatically incorporated into the build environment. To ensure build stability and reproducibility, it is recommended to pin the version to a specific, verified tag, consistent with the other tools defined in this Makefile (e.g., OVERRIDE_GOCI_LINT_V, GOLICENSES_V).

GOTESTSUM_V := v1.12.0