Skip to main content

TestPlanIt 1.2: Failure Patterns, Alerts, Risk Scores, and Automation Rules

· 7 min read
TestPlanIt Team
TestPlanIt Team
Contributor

Today we're releasing TestPlanIt v1.2.0.

Automated testing's second act is the hard one. The suite goes red, and someone still has to work out whether that's one problem or forty, whether anyone is already on it, and whether the run will even get started. 1.2 is aimed squarely at that: it turns a wall of red into a short list of real problems, watches the metrics you care about so you hear about trouble before you go looking, and lets a merged pull request start the testing on its own. And when you can't run everything, it now scores every case on how much it deserves a run.

Highlights​

  • Failure Patterns — Every automated failure is grouped by what actually went wrong, so a run with two hundred red rows comes down to a handful of problems. AI gives each one a root cause from a catalogue you control, and linking the issue that explains it marks the pattern as a known issue right in the run, so the next person doesn't rediscover it.
  • Quarantine — A test failing for a reason you already know about keeps running, but stops counting against your pass rate until it passes consistently again. Your green stays meaningful while the fix is in flight.
  • Alerts — Pick a metric, set a threshold, choose who hears about it: a test that keeps failing, a case that has started to flake, a run below your pass bar or stalled, a milestone falling behind. Delivered in the app, by email, in the digest, or to Slack by webhook.
  • Risk Scores — Every test case carries a score from 0 to 100 for how much it deserves a run right now, blending failure history, flakiness, staleness, the criticality of the requirements it covers, and its open defects. Sort or filter the repository by it, pick a run's cases by it, weigh it beside Impact Analysis, and click any score to see exactly which signals produced it. Scoring is deterministic and never calls an AI model.
  • Automation rules on webhooks — A merged pull request or a new ticket can import the issue, build a run from just the tests that change affects, and send it off for automated execution, with nobody opening TestPlanIt.

Also in 1.2​

  • Test case dependencies — Hold a case in a run until its prerequisites pass, then tell the next tester it's their turn.
  • Staged test cases — Write cases across several sittings in private, then add them to the repository together.
  • ClickUp — Connect ClickUp as fully as Jira: create, link, and sync tasks, custom fields included.
  • Notification settings per type — Silence milestone reminders and still hear about review requests and mentions.
  • Steps side by side — Put each expected result beside its step instead of beneath it, and see far more of a test case at once.
  • Failure rate trend report — Answers one question: is your failure rate getting better or worse? Reports with a date can now be grouped by day, week, month, quarter, or year.
  • Counts on every search filter — See how many results each filter option returns before you click, and stop narrowing a search into nothing.
  • Audit log retention — Hold audit history to a window you choose instead of letting it grow forever.
  • More ways to slice a report — New Result Source, Issue, and Root Cause dimensions, metrics over when cases were created, an Exclude mode on every value filter so "everything except the sandboxes" keeps working as new values appear, and calendar-aligned week and month ranges.
  • Revert a review decision — Undo an approval or rejection made in error, within a time limit each project can tighten.
  • Restore your own deletes — Deleted a test case, folder, run, session, or milestone by mistake? A new Trash section on your profile lists what you deleted, by name, and restores it along with everything that went with it — no administrator required. Administrators choose how many days that window stays open, or turn it off.
  • Several SAML providers at once — Run more than one SAML identity provider side by side, each with its own configuration, its own toggle, and its own button on the sign-in page.
  • A default template per project — Each project chooses which template new test cases start from, and you pick it when you create the project.
  • Smaller wins — Runs offer to close themselves when the last result lands, clear formatting and a grouped toolbar in the rich-text editor, per-row detail that opens in a row of its own, collapsible panels on test cases, dragging cases whatever the sort, imports that name the fields you missed, milestone completion over the API, and review reminders measured in days instead of hours.

Upgrading​

Back up your database before upgrading. Migrations apply automatically on container start, or with pnpm db:deploy on a source install. See the upgrade notes.

  • Two new role permissions. Failure Patterns needs no action: every role that could already add or edit test runs has it. Alerts starts ungranted, so give it to the roles that should create alerts. Administrators always have both. See Roles.
  • Failure Patterns starts with your next results. To fold in results you already have, run Analyze existing results once from the project's settings. Root cause suggestions need an AI model connected; grouping and quarantine don't.
  • Risk scoring is on by default. Cases are rescored shortly after a run receives results, plus a nightly sweep per project; only changed scores are written, and recomputes stay out of the audit log. Turn it off per project under Project Settings → Risk Scoring — existing scores are kept but stop updating, and the repository's Risk Score filter goes away. The Risk Score column is off until you add it from the column picker.
  • Self-service restore is on, with a 30-day window. Every user can restore their own deletes from the Trash section of their profile for 30 days. Change the window, or set it to 0 to turn self-service restore off, from Admin → Trash.
  • One API relation was renamed. On the test case model, the relation to a run's cases is now testRunCases rather than testRuns, which matches the TestRunCases records it actually returns. If you query the model API directly — include or select on repositoryCases — update those calls. Nothing changes in the UI, in CSV imports, or for the testRuns relation on issues, milestones, tags, and projects, which are different relations and keep their names.

Thank You​

To everyone who filed an issue, opened a pull request, translated a locale, or told a colleague about TestPlanIt: this release is yours too.