Use retry orchestrator handshake in dotnet test - #56165
Open
Evangelink wants to merge 2 commits into
Open
Conversation
Consume the testfx retry orchestrator capability before the first test-host attempt so retry UX no longer depends on parsing --retry-failed-tests from the command line. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Azure Pipelines: Successfully started running 1 pipeline(s). 2 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The PR description states a command-line fallback for older platforms remains, but the implementation removes --retry-failed-tests detection without an equivalent fallback path, risking a regression in (try 1) rendering for older MTP versions.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review tier: Lite
Findings: 1
New issues introduced by this change (2)
| Severity | Finding |
|---|---|
 |
src/Cli/dotnet/Commands/Test/MTP/MicrosoftTestingPlatformTestCommand.cs — The PR description says a command-line fallback for older Microsoft.Testing.Platform versions is… |
 |
src/Cli/dotnet/Commands/Test/MTP/Terminal/TerminalTestReporter.cs — This comment still refers to "fail[ing] to parse out the parameter" enabling retry on re-run, but… |
What changed in this PR
This PR updates dotnet test’s terminal retry UX to rely on the Microsoft.Testing.Platform handshake (via a new OrchestratorFeature property on a TestHostOrchestrator host type) so retry rendering can be enabled before the first test-host attempt starts, rather than depending on command-line argument inspection.
Changes:
- Adds/updates protocol constants to mirror
OrchestratorFeatureandTestHostOrchestrator, and updates the vendored-source tracking note accordingly. - Enables retry rendering when a
TestHostOrchestratorhandshake reportsOrchestratorFeature=RetryOrchestrator, and makes retry enabling a monotonic (thread-safe) transition in the terminal reporter. - Adds focused tests covering retry-orchestrator handshake behavior and “enable retry before first assembly run” rendering.
| File | Description |
|---|---|
| test/dotnet.Tests/CommandTests/Test/TestApplicationHandlerTests.cs | Adds handshake tests for retry orchestrator and unknown orchestrator behavior. |
| test/dotnet.Tests/CommandTests/Test/TerminalTestReporterTests.cs | Adds coverage for enabling retry before the first assembly run to render (try 1). |
| src/Cli/dotnet/Commands/Test/MTP/TestApplicationHandler.cs | Enables retry rendering based on orchestrator handshake feature detection. |
| src/Cli/dotnet/Commands/Test/MTP/Terminal/TerminalTestReporter.cs | Makes retry enabling monotonic and thread-safe; uses helper to enable retry when try count indicates retries. |
| src/Cli/dotnet/Commands/Test/MTP/MicrosoftTestingPlatformTestCommand.cs | Stops enabling retry rendering via --retry-failed-tests argument inspection at startup (now starts with isRetry: false). |
| src/Cli/dotnet/Commands/Test/CliConstants.cs | Mirrors new handshake property and host-type constants for the wire contract. |
| eng/vendored-files.json | Updates vendored baseline and notes for the mirrored wire constants. |
Keep attempt 1 retry rendering for older Microsoft.Testing.Platform versions that do not send the orchestrator handshake, and clarify the instance-based fallback. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
dotnet testcurrently enables retry-specific terminal output by inspecting the forwarded--retry-failed-testsargument. Newer Microsoft.Testing.Platform versions now identify the retry orchestrator through the handshake before launching the first test host, so the SDK can use the protocol signal instead of depending primarily on raw command-line parsing.This change:
OrchestratorFeatureproperty andTestHostOrchestratorhost type from the shared protocol constants;RetryOrchestratorhandshakes and enables retry rendering before attempt 1 starts;The protocol handshake is authoritative when available. The legacy fallback remains intentionally because older MTP packages can support
--retry-failed-testswithout advertising the orchestrator capability.