Skip to content

fix(workflows): validate non-string step types - #4111

Merged
mnriem merged 1 commit into
github:mainfrom
NgoQuocViet2001:fix/cli-edge-case-audit
Aug 14, 2026
Merged

fix(workflows): validate non-string step types#4111
mnriem merged 1 commit into
github:mainfrom
NgoQuocViet2001:fix/cli-edge-case-audit

Conversation

@NgoQuocViet2001

Copy link
Copy Markdown
Contributor

Description

A workflow step whose YAML type is a sequence or mapping currently reaches the step-registry membership check and raises a raw TypeError because the value is unhashable. This prevents workflow run, validation, and installation paths from reporting their normal actionable configuration errors.

This change validates that every step type is a string before registry lookup. Unknown string types keep the existing invalid type behavior, while list/mapping values now produce a typed validation error instead of a traceback.

Testing

  • Tested locally with .venv\Scripts\specify.exe --help
  • Ran existing tests with uv sync && uv run pytest
  • Tested with a sample workflow

Validation performed:

  • .venv\Scripts\python -m pytest tests/test_workflows.py -k "invalid_step_type or non_string_step_type_reports_error" -q — 3 passed
  • .venv\Scripts\python -m pytest tests/test_workflows.py -q -k "not symlink and not cross_project_registry_root" — 886 passed, 7 skipped, 36 deselected
  • Full tests/test_workflows.py — 902 passed, 7 skipped; 20 environment-only failures because this Windows session lacks symlink privilege (WinError 1314)
  • uvx ruff@0.15.0 check src tests — passed
  • .venv\Scripts\python -m compileall -q src — passed
  • CLI repro with type: [shell] — exits 1 with 'type' must be a string, got list and no traceback

The repository-wide suite was also attempted, but the local command runner timed out before completion; CI should provide the complete platform matrix.

AI Disclosure

  • I did not use AI assistance for this contribution
  • I did use AI assistance (describe below)

OpenAI Codex (GPT-5, autonomous) independently identified the bug, authored the code and regression tests, ran the validations listed above, and prepared this pull request on behalf of @NgoQuocViet2001. The operator requested autonomous repository improvements and approved this candidate; the operator did not manually author or line-by-line review the change.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Validates workflow step types before registry lookup, preventing unhashable YAML values from causing raw TypeError exceptions.

Changes:

  • Rejects non-string step types with an actionable validation error.
  • Adds regression tests for sequence and mapping values.
Show a summary per file
File Description
src/specify_cli/workflows/engine.py Adds step-type validation.
tests/test_workflows.py Tests non-string step types.

Review details

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

  • Files reviewed: 2/2 changed files
  • Comments generated: 0
  • Review effort level: Balanced

@mnriem mnriem left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address test & lint errors

@NgoQuocViet2001

NgoQuocViet2001 commented Aug 14, 2026

Copy link
Copy Markdown
Contributor Author

Ruff is green. Five of the six pytest matrix jobs pass. The only failure is pytest (windows-latest, 3.14), which timed out in an unrelated test outside this PR’s two-file workflow-validation diff. The same suite passes on Windows/Python 3.13 and on Ubuntu/macOS for Python 3.13 and 3.14, so no PR change is needed. HEAD remains 471a18ff5d71e5e4a11f7165e08eadf270c18909.

Return an actionable validation error when a workflow step type is a YAML list or mapping instead of raising during registry membership checks.

Assisted-by: OpenAI Codex (model: GPT-5, autonomous)
@NgoQuocViet2001
NgoQuocViet2001 force-pushed the fix/cli-edge-case-audit branch from 471a18f to c03cb4c Compare August 14, 2026 07:24
@NgoQuocViet2001

Copy link
Copy Markdown
Contributor Author

@mnriem I rebased the PR onto the latest main and reran the relevant validation locally:

  • tests/extensions/test_extension_agent_context.py: 28 passed, 12 skipped
  • Workflow regression tests: 3 passed
  • Ruff: passed

GitHub is currently holding four workflows for maintainer approval. Could you approve them and take another look once the matrix completes?

Current HEAD: c03cb4cd07ee7b90b3a79436c46367a51aa2ba9d

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 2/2 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

@mnriem
mnriem self-requested a review August 14, 2026 14:18
@mnriem
mnriem merged commit d6e09a1 into github:main Aug 14, 2026
14 checks passed
@mnriem

mnriem commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Thank you!

@NgoQuocViet2001

Copy link
Copy Markdown
Contributor Author

@mnriem Thanks for the review and merge!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants