# Set up Simulithic for this product — instructions for a coding agent > **Faster, if your agent supports MCP:** connect it to Simulithic's MCP server, `https://app.simulithic.com/mcp` > (Claude Code: `claude mcp add --transport http simulithic https://app.simulithic.com/mcp`). It signs in through the > browser, with no token or password to handle, and does every step below with typed tools and a live status > (`setup_status`). This guide is the same setup through the CLI, for agents that cannot connect to it. You are setting up Simulithic for the product in this repository. Simulithic sends simulated people through the product's real flows and, on every pull request, runs the same people on the PR build and on the current release; the verdict (what people could do before and cannot now, and what changed on screen) is posted on the PR. Work through the steps in order. After each step, run the verification line and only continue when it passes. Some steps need a human: an invite code, a password typed at a prompt, repository secrets, a stored sign-in. Stop and ask for those; do not guess them. **Before you start, get from the human:** - the invite code — `` - the email address the account should use - whether the product is a **web app** (give the production URL and, if there is one, how preview deployments are made — Vercel previews, a staging URL, or a local dev server) or a **native app** (Mac `.app`, iOS simulator `.app`, or Android `.apk`; and the CI step that builds it) - if the product has a login: a test account the simulated people may use (email + password of an account whose data may be touched; for a product that emails a one-time code instead of asking for a password, a test account set up so the same code works every time, and that code) --- ## 1. Install the CLI ```sh curl -fsSL https://app.simulithic.com/cli/install.sh | sh export PATH="$HOME/.simulithic/cli:$PATH" # add this line to the shell profile as well ``` Verify: `simulithic --help` prints the command list (it is one global help page; every subcommand's options are in it). The installer checks the package checksum silently and only speaks up on a mismatch; if it does, retry once and then stop. ## 2. Create the account In a terminal this prompts for a password. Without a terminal (you, an agent, or a script) export `SIMULITHIC_PASSWORD` for that one command instead; there is deliberately no `--password` flag, and the password must never land in the command line, the repo or a log: ```sh SIMULITHIC_PASSWORD="" simulithic signup --email --code ``` It creates the account, signs in, saves a token locally and creates a first workspace named after the email's domain (you will create the product's own in step 3; the first one can be ignored, the CLI cannot rename or delete it). If the human already has an account, use `simulithic login --email ` instead. If `simulithic whoami` already shows someone else's account on this machine, run every command of this setup with `SIMULITHIC_HOME=` so their saved token is not overwritten. CI does not depend on it: it uses the `SIMULITHIC_TOKEN` secret from step 7. Verify: `simulithic whoami` shows the email and at least one workspace id (`ws_…`). ## 3. Pick the workspace One workspace per product. `whoami` lists them; if none fits, create one: ```sh simulithic workspace create "" --surface web --use # or --surface desktop | mobile ``` `--use` makes it the default for every later command (`simulithic use ws_…` does the same later). Note the workspace id. Verify: `simulithic whoami` shows the workspace, and `simulithic flows show ` (web) or `simulithic flows show app:` (native: `CFBundleIdentifier` in the app's Info.plist or build script) answers — an empty list is fine at this point. ## 4. If the product needs a login: store the test sign-in Simulated people sign in with a stored account when a flow needs it. This is done in the Studio, not the CLI: open `https://app.simulithic.com/studio/settings?project=`, section **Credentials**, enter the login URL, the email and **either** the password **or** the one-time code of the test account (a product that emails a code instead of asking for a password: a test account set up so the same code works every time). Ask the human to do this step; never store a real customer's or admin's personal account, and use one without CAPTCHA or two-factor. Verify: the Credentials section shows the email as stored. The first run (step 6) proves the sign-in: a run that ends in `Could not sign in with the stored credentials: …` means the account, not the product, is wrong. ## 5. Map the product: the flows the people will run **Web app** — the explorer walks the product and writes its journeys into the workspace: ```sh simulithic map https:// --upload # a local dev server works too: simulithic map http://localhost:3000 --upload --max-pages 60 ``` **Native app** — upload the build, a Mac worker explores it (about 10 minutes) and writes the journeys: ```sh simulithic map --app path/to/MyApp.app --branch main # .apk for Android, iOS simulator .app for iOS ``` Verify: `simulithic flows show ` (web) or `simulithic flows show app:` (app) lists 4–8 journeys, each with a goal and a success text. Read them. Each journey should describe something a user actually comes to do, and its success text must be something the product itself displays after that thing is done (a row label, a heading, a button that only appears afterwards) — never text the user typed, and never text that is already on screen at the start. ### Make the journeys assert state, not just screens A journey that creates, edits, pins, deletes or saves anything must check the result **after a reload** (web) or **after the app is quit and reopened** (native), otherwise a change that looks saved but is not will pass. The explorer's journeys only check the screen right after the action. Keep them, and add one relaunch-verified journey for each kind of change the product makes (create, edit, pin, delete…) with `simulithic flows add journeys.json` (a journey with the same name replaces the explorer's). Two examples, a create and a pin: ```json [ { "name": "Create a titled note", "goal": "Press New note and type exactly {{unique}} into the Title field. The note you just created must carry that title: its Title field must read {{unique}} and its row in the list must show it. If the title turns up on a different note, or the field stays empty, say so.", "successText": "{{unique}}", "successIn": "Title", "preserve": true, "verify": "reload" }, { "name": "Pin a note", "goal": "Select the 'Welcome to Notebook' note and pin it with the toolbar's Pin note button. The proof is the toolbar reading Unpin note while that note is selected. Do not quit the app. If the button still reads Pin note afterwards, or a different note turned out pinned, say so.", "successText": "Unpin note", "verify": "reload" } ] ``` When the asserted thing is not a field but a state (pinned, deleted, archived), make the success text something the product shows only in that state (a button that flips its label, a row in a section that lists only such items) and leave `successIn` out. Rules that matter, learned the hard way: - `{{unique}}` becomes a value only this person knows (`qa-x7k2q9`), so the check proves *their* change exists. - `successIn` names the field or element that must hold the text (an input's label, or a CSS selector such as `#team` on the web), so the text being somewhere else on screen does not count. - `verify: "reload"` judges success only after the reload/relaunch. Do **not** give such a journey a `successPath`: a URL match alone counts as success. - `preserve: true` fails the journey if anything that was on screen at the start is gone at the end (creating one thing must not alter another). Use it only when the journey starts and ends on the same screen. - Never write "quit and reopen the app" in a goal — the people will do exactly that themselves and the run breaks. The harness relaunches; write "do not quit the app". - In native apps, a text field's value is read 60 characters deep: assert on short fields, or on text near the start. ## 6. Run one simulation by hand **Web app** — `qa` targets a URL (production, staging, or a local dev server through a tunnel): ```sh simulithic qa --url https:// --goal "" simulithic qa --url https:// --all-flows # every journey, one person per flow per device simulithic qa --port 3000 --goal "…" # a local dev server, through a tunnel ``` **Native app** — the same, against the build you mapped (`map` printed its id as `build:bld_…`; a `.app`/`.apk` path is uploaded first): ```sh simulithic qa --app build: --goal "" simulithic qa --app build/MyApp.app --all-flows # every journey, two people each ``` Every run is a **verification run** unless asked otherwise: a tester carries each journey out end to end, giving up means stuck, and "What went wrong" holds product defects only. The CLI prints the per-flow table and "What went wrong" itself (and writes them to the `--out` file); the Studio link it prints adds the recordings and every step with the person's reasoning, for a human. Verify from the printed table: the run finishes, every journey succeeds (or nearly), and nothing in "What went wrong" is a setup problem (sign-in failed, page not found, a journey's success text that the product never shows). Low and medium items that describe the product as it is (a share panel that only copies, say) will appear on every run and do not block a check (`--fail-on high`). Fix journeys with `flows add` / `flows drop` until the baseline is clean. Only then put the check on PRs: a journey that fails on the release today can never show a regression. Sizing: `--users N` sends N people **plus one cautious tester** per flow per device, so `--users 1` is 2 sessions per flow and `--users 2` is 3. The footer's seconds are worker time summed over both sides, not wall time. ## 7. Put the check on every pull request First mint a token for CI and store it as the repository secret `SIMULITHIC_TOKEN` — the workflow's very first run needs it, so this comes before the workflow is pushed. The command asks for the password in a terminal; without one, export `SIMULITHIC_PASSWORD` for this command as in step 2. The token goes to stdout once (the notice goes to stderr), so it pipes straight into the secret: ```sh simulithic token --name github-ci | gh secret set SIMULITHIC_TOKEN -R / ``` Then add **one** of the two workflows below as `.github/workflows/simulithic.yml`, replacing the placeholders. ### Web app with preview deployments (Vercel) GitHub fires `deployment_status` workflows from the default branch only, so this file must be on it. The check runs the same people on the preview URL and on production and posts one PR comment, updated in place. ```yaml name: Simulithic on: deployment_status: workflow_dispatch: inputs: preview_url: { description: "Preview deployment URL", required: true } jobs: simulate: if: github.event_name == 'workflow_dispatch' || (github.event.deployment_status.state == 'success' && github.event.deployment.environment == 'Preview') runs-on: ubuntu-latest permissions: { pull-requests: write, contents: read } env: PREVIEW_URL: ${{ github.event.inputs.preview_url || github.event.deployment_status.target_url || github.event.deployment_status.environment_url }} DEPLOY_SHA: ${{ github.event.deployment.sha || github.sha }} SIMULITHIC_TOKEN: ${{ secrets.SIMULITHIC_TOKEN }} steps: - uses: actions/checkout@v4 - name: Install the Simulithic CLI run: curl -fsSL https://app.simulithic.com/cli/install.sh | sh - name: Simulithic check id: sim continue-on-error: true run: | code=0 ~/.simulithic/cli/simulithic ci --url "$PREVIEW_URL" --base "https://" \ --project --users 2 --fail-on high --timeout 40 --sha "$DEPLOY_SHA" --out comment.md || code=$? echo "exit=$code" >> "$GITHUB_OUTPUT" - name: Comment on the PR (updated in place) uses: actions/github-script@v7 with: script: | const fs = require("fs"); const marker = ""; const body = marker + "\n" + (fs.existsSync("comment.md") ? fs.readFileSync("comment.md", "utf8") : "Simulithic check could not run (see the workflow log)."); const { owner, repo } = context.repo; const open = (await github.rest.pulls.list({ owner, repo, state: "open", per_page: 100 })).data; const prs = open.filter((p) => p.head.sha === process.env.DEPLOY_SHA); for (const pr of prs) { const { data: comments } = await github.rest.issues.listComments({ owner, repo, issue_number: pr.number, per_page: 100 }); const mine = comments.find((c) => c.body && c.body.includes(marker)); if (mine) await github.rest.issues.updateComment({ owner, repo, comment_id: mine.id, body }); else await github.rest.issues.createComment({ owner, repo, issue_number: pr.number, body }); } - name: Fail the job on a regression if: steps.sim.outputs.exit == '2' run: exit 2 ``` If Vercel deployment protection is on, add the repository secret `VERCEL_AUTOMATION_BYPASS_SECRET` (the bypass secret itself, one line) and export `SIMULITHIC_API_HEADERS="x-vercel-protection-bypass: $VERCEL_AUTOMATION_BYPASS_SECRET"` before the check step. With a fixed staging URL instead of previews, trigger on `pull_request` and pass that URL as `--url`. ### Native app (Mac, iOS simulator, Android) Every push to the default branch uploads a base build; every PR is checked against the newest base build. Replace the build step with the one this repo already uses (see the build table in Simulithic's build-upload notes: Electron, Tauri, xcodebuild, Gradle, Flutter all leave a file on the runner). ```yaml name: Simulithic on: pull_request: push: { branches: [main] } jobs: simulate: runs-on: macos-latest # ubuntu-latest for an Android .apk timeout-minutes: 45 permissions: { pull-requests: write, contents: read } env: SIMULITHIC_TOKEN: ${{ secrets.SIMULITHIC_TOKEN }} SIMULITHIC_PROJECT: steps: - uses: actions/checkout@v4 - name: Build the app run: # leaves e.g. build/MyApp.app - name: Install the Simulithic CLI run: curl -fsSL https://app.simulithic.com/cli/install.sh | sh - name: Upload the main build (the base for later checks) if: github.event_name == 'push' run: ~/.simulithic/cli/simulithic upload build/MyApp.app --project "$SIMULITHIC_PROJECT" - name: Check this pull request against main if: github.event_name == 'pull_request' run: | ~/.simulithic/cli/simulithic ci --app build/MyApp.app --base-branch "${{ github.base_ref }}" \ --project "$SIMULITHIC_PROJECT" --users 2 --fail-on high \ --repo "$GITHUB_REPOSITORY" --sha "${{ github.event.pull_request.head.sha }}" --pr "${{ github.event.pull_request.number }}" \ --out comment.md || echo "simulithic exit=$?" - name: Comment on the PR (updated in place) if: github.event_name == 'pull_request' uses: actions/github-script@v7 with: script: | const fs = require("fs"); const marker = ""; const body = marker + "\n" + (fs.existsSync("comment.md") ? fs.readFileSync("comment.md", "utf8") : "Simulithic check could not run (see the workflow log)."); const { owner, repo } = context.repo; const issue_number = context.issue.number; const { data: comments } = await github.rest.issues.listComments({ owner, repo, issue_number, per_page: 100 }); const mine = comments.find((c) => c.body && c.body.includes(marker)); if (mine) await github.rest.issues.updateComment({ owner, repo, comment_id: mine.id, body }); else await github.rest.issues.createComment({ owner, repo, issue_number, body }); ``` The first check needs a base build to exist: push the workflow to the default branch first (that uploads one), then open a PR. (A PR that does not change the binary, a docs-only one, is compared with itself and simply reads "same".) Verify: open a small PR. Within about 10 minutes (web) or 20 minutes (native) the PR has a Simulithic comment with a table of flows, base vs PR, and links to the recordings. 🟢 means the same people did the same things on both; 🔴 lists what people could do on base and cannot on the PR, plus defects seen only on the PR. The job exits 2 on a regression. ## 8. Optional, recommended - **Capture real visitors** (web): the Behavior SDK is the one way real-visitor data comes in. Add the one-line `` tag (Studio → Setup shows it filled in for the workspace) to the product's root layout or HTML, on every page. It records behaviour, never content or identity, and after a few hundred sessions the simulated people are modelled on the real ones and runs cover the devices they use. - **Monitors**: Studio → Simulate → schedule, to run the flows against production on a timer and get told when one stops working. - **Alerts**: Studio → Settings → Integrations for Slack (Add to Slack, or a webhook) and email. ## If something does not work - `not signed in` — run `simulithic login`, or check `SIMULITHIC_TOKEN` in CI. - `pass --project` — run `simulithic use ` or add `--project`. - A journey is blocked on base and on the PR alike — the journey is wrong, not the product: rewrite it (`flows add`) or drop it (`flows drop`). - `no sign-in stored for this workspace` — step 4. - `--base-branch main` refuses (`no build … from branch main has been uploaded yet`) — that branch has no build of this app yet: push it with the upload step (step 7) or run `simulithic upload … --project …` from a checkout of it. (A PR whose binary is identical to main's is compared with itself and reads "same".) - `qa` says `give me a target: --port 3000 or --url …` — pass `--url`, `--port`, or for a native build `--app ` (step 6). - The map's progress line sits at "running" for the whole ten minutes — it is not hung; `simulithic qa:status ` shows the run. - Anything else: team@simulithic.com, with the Studio link of the run.