sedum run exits non-zero, on any CI system. Exits are
0 for a pass (a flagged pass included), 1 for a failed test, 2 for a
flagged pass under --strict, and 3 when the run could not produce a
verdict you can trust. The reporters below add detail on top of that gate.
--reporter takes a comma-separated list: --reporter junit,markdown. When
no terminal reporter is selected, stdout prints markdown <path> and then
junit <path>, so a later step can read both paths. Each run gets its own
<run-id> directory, so a glob such as .sedum/reports/*/junit.xml finds
the file.
How a run maps to JUnit
Each test file is one<testsuite> with one <testcase>, named by the test’s
description (or its file when there is none). The testcase’s classname and
the suite’s file are the test file. A suite named sedum run comes first
and carries run totals as properties: tests passed and failed, flagged steps,
and counts for each flag.
So the XML never contradicts the exit code, and no testcase carries more
than one status. A failure lists each step that needs attention: its
sentence,
file:line:col, scores against the decision lines, page, browser
error and call log, judged page text, frame, and a rerun command.
Many JUnit viewers, including GitLab, ignore properties, so a flagged pass
shows as green by default. Read the flag counts from report.md or
result.json, or run with --strict to make flags fail the job.
If writing junit.xml fails, the run exits 3 with reporter_output_error,
all rendered reports are removed, and result.json records the error. If the
reporter directory itself breaks, the error is output_error when json is
selected; the canonical result.json survives either way. When a terminal
reporter fails, junit.xml is rewritten to describe the errored run.
Evidence attachments
A failed or flagged step’s frame is attached as[[ATTACHMENT|<path>]] in the testcase output. GitLab shows the first one
per test and the Jenkins JUnit Attachments plugin shows all of them. Upload
the run directory with the XML so the paths resolve.
The path is relative to the CI checkout: Sedum uses the first of
CI_PROJECT_DIR (when GITLAB_CI=true), WORKSPACE (when JENKINS_URL is
set) or GITHUB_WORKSPACE (when GITHUB_ACTIONS=true) that contains the run,
and otherwise the project root. Each variable counts only inside its own CI,
so a stray WORKSPACE elsewhere cannot put local directory names in the
report. If the run directory’s path contains a character that would break an
attachment line (such as [, | or a backslash), no frames are attached. In a monorepo with
sedum.config.yaml in apps/web, paths start with
apps/web/.sedum/runs/. Only the path is derived from these variables; their
values are never written to the report. Frames can contain private page
pixels; --sensitive-origin and --no-evidence omit them.
Text from test files and pages can never form an attachment marker: every
[[ in it is written as [ [.
Install Sedum in the project
The examples below runnpx sedum, which uses the version installed in your
project. Add it as a dev dependency first:
npx sedum would download an unrelated npm package that happens
to be named sedum. To run Sedum without installing it, use
npx sedum-cli instead.
GitHub Actions
GitHub renders Markdown job summaries natively, soreport.md is the
simplest summary. Keep the steps running when the test step fails.
TYPESAFE_BASE_URL and, if needed, TYPESAFE_DEFAULT_MODEL in the step’s
environment, and use that provider’s key for TYPESAFE_API_KEY. See
provider configuration.
To show the JUnit file as a test summary instead, add a JUnit action such as
mikepenz/action-junit-report with report_paths: .sedum/reports/*/junit.xml.
Its annotate_only: true mode needs no checks: write permission.
Automatic Sedum Cloud history
Once configured, the normalsedum run command uploads its final JSON report
automatically. There is no additional CI step or upload command:
SEDUM_CLOUD_URL to your instance’s HTTPS base URL, SEDUM_PROJECT_ID to the
project UUID, and SEDUM_UPLOAD_TOKEN to that project’s revocable sdu_…
credential. A read-only credential cannot upload. Locally, put the same settings
in the project-root .env or export them; process values override .env.
Never commit a credential or make it available to untrusted fork pull requests.
HTTP is accepted only for loopback development instances.
Branch and commit context are captured before the tests start. GitHub Actions
and GitLab CI supply the source branch and CI link; otherwise Sedum uses the
current Git branch. The commit is the checked-out HEAD (including a CI merge
commit), falling back to the CI commit variable if Git is unavailable. Explicit
SEDUM_BRANCH and SEDUM_COMMIT_SHA override detection. For other CI systems or
detached checkouts without a known source branch, set both explicitly; the SHA
must be the full lowercase 40- or 64-character commit hash. Sedum never guesses
that an unknown branch is main.
Unicode branch names are preserved. The optional CI link is included only when
it is a valid HTTPS URL without embedded credentials; an invalid link is omitted
without preventing the report upload.
Passed, failed, flagged, errored and interrupted runs upload when an authoritative
terminal report was saved. Upload problems only warn on stderr: they never
replace the test exit code or remove the local reports. Network failures, HTTP
408, 429 and 5xx responses retry up to three total attempts, with one five-second
delivery budget covering requests, response bodies and backoff. Other HTTP
errors do not retry. A revoked token, configuration error or conflicting run ID
therefore cannot turn a passing suite red. This is best-effort delivery, not a
persistent delivery queue; keep the CI report artifacts for recovery.
A successful upload prints the Cloud run UUID and the local run ID:
sedum runs show.
Warnings name the likely cause without printing secret values:
Cloud does not reveal whether a well-formed token or the project ID is wrong; an
HTTP 401 means one of the two, so check that the credential was issued for that
project.
Without Cloud settings, runs remain local-only. Use
sedum run --no-cloud to
disable uploading for a particular invocation, even when configured. Upload
credentials are excluded from Sedum’s test variables and provider flow context.
An agent can keep monitoring local progress.json and investigating failures
while the suite runs. Uploading the terminal result.json adds the completed
run to Cloud history for cross-branch and weekly flakiness analysis. It does not
stream in-progress results or upload replay/evidence image files. Reports larger
than 8 MB remain local and produce a warning. Hard-killed processes that cannot
save a terminal report cannot upload one.
GitLab CI
paths entry uploads the frames the attachments name.
Jenkins
--reporter-dir .sedum/reports/$BUILD_NUMBER. Sedum never
deletes files it did not create. In a monorepo, call the junit step from
the workspace root with apps/web/.sedum/reports/*/junit.xml, not inside
dir('apps/web'), because attachment paths are relative to WORKSPACE.
A broken sedum.config.yaml still exits 3. Its junit.xml goes to
--reporter-dir or .sedum/reports, because a reporterDir set in the
broken file cannot be read.