runtimeanalyz.ing

Runtime Analysis · Ruby

Your agent guesses.
Your runtime knows.

Ruby

Runtime Analysis records the calls your objects make while your tests or your app run. Your coding agent and your editor query the recording for the classes that arrive at a call site, what a method returns and who calls it.

    $ curl -fsSL https://www.runtimeanalyz.ing/install.sh | sh  # the ra binary, and a walk through sign-in and the recorder gem$ ra record  # runs your tests once and changes no file in your project$ ra verify_mock spec  # the stubs that differ from your real code
  

The recorder, the editor's type hints and five tools, verify_mock and what_calls among them, are free with a free account, made the first time you sign in. The other tools are Pro, with a trial. Recording and every tool run on your laptop, from your test suite alone if you have no production recording, and your recordings stay on your machine.

The problem

In Ruby, what a method returns and who calls it are decided at runtime.

To change Ruby code, your agent needs to know four things: what a method returns, which collaborators an object needs, which of a class's methods each caller uses, and which boundaries your tests cross with real objects on both sides. The source contains none of the four. A type signature records what someone declared, coverage records which lines ran, and grep finds text. One recording of your test suite contains all four.

What runtime_analysis is

It records each call, builds a graph, and answers questions from the graph.

The recording covers the class of every object in your application* and every message it receives. The answers come back in your agent, your editor, the command line and CI.

01 · record

The gem runs inside your test suite and records every message sent to every object in your application*: the receiver's class, the method called and the call site, in order. It writes class names and method names to files on your machine. You can also run it in your app in development, staging or production to record real traffic.

02 · graph

The graph is built on your machine from the recorded files. The methods your code calls on a value are treated as that value's type, so a method's return type is the set of methods called on its return value in the recorded runs. The hosted graph, once it opens and you opt in, will combine every deploy's runs into one graph.

03 · ask

Your agent asks over MCP and your editor over LSP. The command line and CI run the same tools. The tools read the graph and do not run your application.

Questions it answers

One recording answers questions about any method that ran.

Grep for .call matches every service object, proc and middleware in your app and its gems. For one call site, the recording lists the receivers that arrived there, with their classes and how often. Each tool asks one question of the same recording:

Asking the hosted graph anything else, in Cypher, is planned. See all the tools →

Examples

Three of those questions, with their answers.

Each example takes one question from the list above and shows the answer in the tool where you would read it.

Example 1 · a spec, written first

One spec, written first by an agent alone and by an agent with the recording.

✗ agent alone · green, but wrong

◆ enrollment_spec.rb

5

6

7

8

9

10

11

12

13

it "unlocks the next book" do
  # agent: failing spec first, stubbing the curriculum
  curriculum = instance_double(Curriculum)
  allow(curriculum).to receive(:unlocked_for).and_return([:geometry])
  enrollment = Enrollment.new(student: Student.new)
  allow(enrollment).to receive(:curriculum).and_return(curriculum)

  expect(enrollment.unlock_next).to eq([:geometry])
end
⛔ runtime_analysis error

stub Curriculum#unlocked_for is a false positive

your stub returns [:geometry] Array<Symbol>
the real one returns an Array whose elements receive #unlocked?, #title
from 9 spec examples and 42 app runs
a Symbol was never seen answering any of them

⛔ 1Rubyruntime_analysis · verify_mockgreen, but wrong
claudeenrollment · rspec

> Enrollment needs #unlock_next. Failing spec first, then make it pass

● I'll stub the curriculum and write the failing spec.

● Bash(bundle exec rspec spec/models/enrollment_spec.rb)

⎿ 1 example, 1 failure · undefined method 'unlock_next'

● Edit(app/models/enrollment.rb)

⎿ 1 example, 0 failures

● Red, then green. Done.

✓ agent + runtime analysis · fixed before it ran

◆ enrollment_spec.rb

5

6

7

8

9

10

11

12

13

it "unlocks the next book" do
  # agent → observed_shape(Curriculum#unlocked_for): element → #unlocked? #title · Book
  book = instance_double(Book, unlocked?: true, title: "Geometry")
  curriculum = instance_double(Curriculum, unlocked_for: [book])
  enrollment = Enrollment.new(student: Student.new)
  allow(enrollment).to receive(:curriculum).and_return(curriculum)

  expect(enrollment.unlock_next).to eq([book])
end
✓ 0 problemsRubyruntime_analysis · verify_mock1 stub · 0 false positives
claudeenrollment · rspec

● runtime_analysis · verify_mock(spec: "spec/models/enrollment_spec.rb") [MCP]

⎿ 1 stub · 1 false positive · the real one returns an Array whose elements receive #unlocked?, #title

● My stub returns Symbols and the real method returns Books. Fetching the recorded return.

● runtime_analysis · observed_shape(method: "Curriculum#unlocked_for") [MCP] Pro

⎿ element → #unlocked? #title · Book · from 9 spec examples and 42 app runs

● Rebuilding the spec with a double that matches.

● Bash(bundle exec rspec spec/models/enrollment_spec.rb)

⎿ 1 example, 0 failures

● The spec passes with a double that matches the real return.

The second time, the agent asked Runtime Analysis what Curriculum#unlocked_for returns. That return was recorded while Curriculum's own spec ran, and the agent built a double that matches it. The spec's recording is enough for this, and recorded app runs add to it. The agent wrote no code ahead of its test. verify_mock is free, and its finding includes the fix: return an Array of Book. observed_shape, the second call, is a Pro tool.

Example 2 · boundaries only doubles cross

Every unit spec passes, and no example runs the two classes together.

A stub with the wrong return is a fault in one double. A whole boundary between two classes can go untested the same way, whether you test first or after. Checkout's spec stubs the gateway and the gateway's spec tests the gateway alone, so every line is covered and no example runs the real Billing::Checkout calling the real Billing::Gateway. Coverage counts lines, so its report shows nothing missing.

Runtime Analysis sorts the recorded calls into the layers of your testing pyramid and lists the boundaries your specs cross through a double and in no other way. If you record your app too, the list is ranked by traffic. It then drafts a test that reaches a method, with setup taken from a recorded run.

jobs · 2 requests · 6 unit · 56 only crossed via doubles Billing::Checkout → Billing::Gateway stubbed in 5 specs · 0 real · 58 app runs Account.find stubbed once · never recorded never crossed for real

The same finding is reported in your agent, your editor and the command line.

MCP · your agent
runtime-analysis MCP
testing_pyramid(stubbed_only: true)
→ structuredContent: {
  summary: "3 boundaries only crossed via doubles",
  blocks: [ { type: "table",
    columns: [boundary, stubbed, real, app runs],
    rows: [
      ["Billing::Checkout#complete → Billing::Gateway#charge",
        5, 0, 58],
      ["(no recorded caller) → Billing::Gateway.new",
        2, 0, 0],
      ["(no recorded caller) → Account.find",
        1, 0, 0] ] }, … ] }
LSP · your editor
◆ billing/gateway.rb

3

4

5

▶ reached by 3 examples · 58 app runs
def charge(amount)
  Receipt.new(id: "rc_…", amount: amount, paid: !amount.negative?)
end
runtime_analysis · code lensgateway_spec.rb:4 · :10 · :16
CLI · your terminal
testing_pyramid
$ ra testing_pyramid --stubbed-only

testing_pyramid
· 3 boundaries only crossed via doubles
  Billing::Checkout#complete → Billing::Gateway#charge  5  0  58
  (no recorded caller) → Billing::Gateway.new           2  0   0
  (no recorded caller) → Account.find                   1  0   0
  models     37
  billing     9
  requests    6
  finance     4
  services    4
  jobs        2
  reporting   2

ra scaffold then drafts a test whose setup comes from a recorded run. Your agent gets the same draft over MCP. An editor quick-fix that inserts it is planned.

runtime_analysis · scaffold
$ ra scaffold Invoice#total
# drafted by ra scaffold from the recorded setup in spec/models/invoice_spec.rb:6
RSpec.describe Invoice do
  it "total" do
    invoice = build(:invoice)
    line_item = LineItem.new(sku: "…")
    invoice.add_line(item: line_item, quantity: 1)
    expect(invoice.total).to be_a(Money)
  end
end

Example 3 · interface use by caller

The recording shows which part of an interface each caller uses.

A type has a fat interface when each caller uses a different, small part of it, and the recording measures that from the calls each caller made. split_interface reports it as a hint and does not fail a build, because partial use of an interface is normal. The example app's Curriculum has four callers, and each uses its own set of methods.

runtime_analysis · split_interface
$ ra split_interface Curriculum

Curriculum
· 4 roles · 0 callers span more than one
  for Report             final_books, weeks_for                                       Report
  for Scheduler          first_books, next_after                                      Scheduler
  for Billing::Checkout  for, unlock_paid_books                                       Billing::Checkout
  for Enrollment         unlocked_for, level_for, books_at, archived?, prerequisites  Enrollment
never called in any recorded run: name
· extract 4 role interfaces   ISP

The same answer comes back over MCP. An editor hint with a quick-fix that extracts the roles is planned.

The same graph answers other design questions from SOLID, the Gang-of-Four patterns, and package cohesion and coupling. Each answer names a method or file to move.

duck typing types

Types read from recorded runs

Sorbet and RBS have you declare each interface. Runtime Analysis reads the interfaces your code already depends on from recorded runs.

See more →
design playbook SOLID · packages

Design findings, each with a move

Feature envy, dependency cycles, unstable dependencies and packages used in parts, each reported with the move to make and the recorded calls that support it.

See more →

Where you ask

In your agent, your editor, your terminal and CI.

One recording of Curriculum#unlocked_for, queried by your agent, shown in your editor, printed on the command line and checked in CI. The free tools and the editor's hover work in the same four places. observed_shape, the code lens and ra check shown here are Pro.

MCP · your agent
runtime-analysis MCP
observed_shape(
  method: "Curriculum#unlocked_for"
)
→ structuredContent: {
  title: "Curriculum#unlocked_for",
  summary: "observed return",
  blocks: [
    { type: "table", rows: [
      ["return", "(no app messages)", Array],
      ["element", "#unlocked? #title", Book] ] },
    { type: "text", lines: [
      "· from 9 spec examples and 42 app runs"] } ] }
LSP · your editor
◆ enrollment.rb

15

16

17

▶ reached by 9 examples · 42 app runs
def unlocked_for(student)
  eligible(student)
end
⬡ runtime_analysis hover

Curriculum#unlocked_for · 3 callers

returns Array (628)
elements Book (1506) → #unlocked? #title
arg 1: Student (628) → #completed?

runtime_analysis · lsphover · code lens
CLI · your terminal
observed_shape
$ ra observed_shape \
    Curriculum#unlocked_for

Curriculum#unlocked_for
· observed return
  return   → (no app messages)  Array
  element  → #unlocked? #title  Book
· from 9 spec examples
  and 42 app runs
CI · github actions
runtime-analysis / check
▸ ra check --since origin/main

check --since origin/main
· 13 findings introduced · 185 methods changed
✗ stub Curriculum#unlocked_for is a false positive
  spec/models/enrollment_spec.rb:9
  your stub returns [:geometry] Array<Symbol>
  the real one returns an Array whose elements receive #unlocked?, #title
  → return an Array of Book, as the real method did
  … 12 more

Error: Process completed with exit code 1

Your agent gets the same tools over MCP, so it can check a stub in the edit that introduced it. A Claude Code hook that runs verify_mock on every spec the agent writes is planned.

claude enrollment · rspec

> Enrollment needs #unlock_next. Test first, then make it pass

● I'll stub the curriculum collaborator and write the failing spec.

● Write(spec/models/enrollment_spec.rb)

⎿ +12 lines

● runtime_analysis · verify_mock(spec: "spec/models/enrollment_spec.rb") [MCP]

MCP result · verify_mock

✗ stub Curriculum#unlocked_for is a false positive
your stub returns [:geometry] Array<Symbol>
the real one returns an Array whose elements receive #unlocked?, #title
from 9 spec examples and 42 app runs
a Symbol was never seen answering any of them

● My double returns Symbols and the real method returns Books. Fetching the recorded return before I run the spec.

● runtime_analysis · observed_shape(method: "Curriculum#unlocked_for") [MCP] Pro

⎿ return → (no app messages) Array · element → #unlocked? #title Book

● Rebuilding the spec with a double that matches.

● Write(spec/models/enrollment_spec.rb)

● Bash(bundle exec rspec …)

⎿ 1 example, 1 failure · undefined method 'unlock_next'

● Red: unlock_next is undefined. Now the implementation.

● Edit(app/models/enrollment.rb)

● Bash(bundle exec rspec spec/models/enrollment_spec.rb)

⎿ 1 example, 0 failures

● Green, with a double that matches the real return.

> run ra verify_mock spec over the whole suite

Each MCP tool is also a command. Run ra verify_mock on your suite, or ra check on a diff in CI, and a stub with the wrong return fails the build.

runtime-analysis · verify_mock
$ ra verify_mock spec/models/enrollment_spec.rb

verify_mock spec/models/enrollment_spec.rb
· 1 stub · 1 false positive · 0 unverified
✗ stub Curriculum#unlocked_for is a false positive
  spec/models/enrollment_spec.rb:9
  your stub returns [:geometry] Array<Symbol>
  the real one returns an Array whose elements receive #unlocked?, #title
  from 9 spec examples and 42 app runs
  a Symbol was never seen answering any of them
  → return an Array of Book, as the real method did

$ ra find_implementers '#unlocked?, #title'

real types responding to {#unlocked?, #title}
· 1 class
  Book  4  630  outside traced code (a factory or seeds)
1 false positive · reported with the fix

What you can record

Record a test-first suite, a mutation-tested suite, or a running app.

What the recording gives you differs in each case.

test first tdd

A test-first suite

The collaborators you stub already exist, and each runs as a real object in its own spec, so your suite's recording is enough. You need no production recording and write no code ahead of its test.

See more →
test hard mutant · no mocks

A mutation-tested suite

Mutant leaves your doubles as they are. Runtime Analysis checks them, lists the most-called untested methods, and gives recorded setups for building real collaborators.

See more →
no tests legacy

An app with no tests

Code with no specs is recorded when your app runs. The tools report its callers and the classes it takes and returns, and ra scaffold drafts a first test from a recorded run.

See more →

Test first

In a test-first suite, each collaborator is recorded in its own spec.

The class you are about to write does not exist yet. Its collaborators do, and the tests from earlier cycles already ran them as real objects. When you mock Gateway in Checkout's spec, Gateway's own spec still runs the real class, so each double has a recording to be checked against.

red

The stub you just wrote is checked against what the real collaborator did in its own spec.

green

Run the specs that reach what you changed and skip the rest.

refactor

See which methods of the interface each caller uses, and which methods are called by their own spec and nothing else.

Test hard

Mutant tests the code under test and leaves your doubles unchecked.

mutant mutates the code under test and leaves the double you stubbed for a collaborator as it is. instance_double checks that the method exists with the right arity. Runtime Analysis checks what the stub returns against what the real collaborator returned in recorded runs.

verify_mock

CLI · LSP · MCP · CI free

Stubs that disagree with the real object. Compares what each stub in a spec returns with what the real method returned in the recording, and reports the stubs that return a class the real method did not.

See more →

unit_test_candidates

CLI · MCP Pro

Which methods to test first. Every recorded method ranked by how many distinct callers it has and how many calls away the nearest spec is, to choose which untested code gets a test first.

See more →

how_to_reach

MCP · CLI Pro

The setup a method needs. The collaborators and the call path that put an object into the state a method needs, taken from a recorded run. Use it to write setup for tests that use real objects in place of doubles.

See more →

split_interface

CLI · LSP · MCP Pro

The role interfaces in a class. Groups a class's public methods by which callers used them in the recording, and lists each group as a role interface to extract.

See more →

dead_api

LSP · CLI Pro

Find dead public API. The public methods that no caller reached in any recorded run, including those whose only caller is their own spec. Shown as a hint in your editor.

See more →

If you never mock, verify_mock has nothing to check, and the fan-in ranking, setup recipes and interface hints still apply.

No tests

With no tests, record the running app.

Coverage and mutation testing report on code that a test runs. Runtime Analysis records whatever runs, so with the recorder on in development, staging or production, code with no specs is recorded too. A path that runs more often has more recorded calls behind each answer.

For a module with no specs you get its callers, the classes each method takes and returns, and the exception classes it raised. what_a_change_touches lists the callers a signature change would break. ra scaffold drafts a first spec from a recorded setup.

runtime-analysis · observe
$ ra observe app/billing/billing/proration.rb

observe app/billing/billing/proration.rb
· 4 recorded methods
Billing::Proration.for  from 15 spec examples and 106 app runs
  callers   Invoice#total · Billing::Checkout#complete · Api::QuotesController#show · spec/billing/proration_spec.rb
  returns   Billing::Proration  → #amount #adjust
Billing::Proration#adjust  from 10 spec examples and 46 app runs
  callers   Invoice#total · spec/billing/proration_spec.rb
  returns   Money  → #cents #currency #to_ledger_s #to_s
  … 2 more

# when a method raised, observe lists the exception class and how often
$ ra scaffold Invoice#total   # drafts a spec from a recorded setup

Under load the recorder drops records and counts them, and it does not block your app. Production support is still being hardened: see the questions page.

Plans

A free recorder, paid tools on your laptop, and a hosted graph to come.

On every plan, recording runs on your machine and the recordings stay there. A Pro licence is checked on your machine. The hosted graph will receive what you choose to send it, and Enterprise runs that graph inside your own network. What leaves your machine lists every field.

Free

$0

The recorder, five tools and editor type hints, with a free account.

  • The recorder, for your test suite and your running app, on your machine.
  • verify_mock: the stubs whose return differs from what the real collaborator returned.
  • what_calls: the callers of a method, from recorded runs.
  • what_it_receives: the classes each argument was given.
  • who_actually_answered: which classes received the message at a call site.
  • where_nil_comes_from: the methods that returned nil, and what followed.
  • Your editor's type hints and hover.
  • Nothing is sent to us.
Install it →

Pro paid, with a trial

$19 a month $19 a developer × 1

Every other tool, run on your own recording.

  • Call stacks, hot loops, dead code, which specs reach a line, and the rest of the capabilities.
  • Feature envy, cycles and coupling, with one refactor report that ranks them.
  • Available from the command line, your coding agent (MCP) and your editor (LSP).
  • Runs on your laptop and in CI. The licence is a key checked on your machine.
  • A trial lasts 14 days and needs no card. A person sends each key, so it does not arrive instantly.
Ask for a trial →

Team

$88 a month $49 base + $39 a developer × 1

Everything in Pro, and one hosted graph shared by the team.

  • Sync up to 5 applications into one hosted graph, from CI and from the servers you record.
  • An application counts once, on any number of servers and environments.
  • Coupling, feature envy and cycles measured across every deploy. Pro measures the runs recorded on one laptop.
  • The hosted graph stores tokens, and your class and method names stay on your machines. What an upload contains.
  • Recording a production server is at an early stage, so start with CI as the source.
  • The hosted graph is built and is not open yet. Sign up and we will tell you when it opens.
Hear when it opens →

Enterprise

Enterprise is for needs beyond the card plans: more than 5 synced applications, SSO and audit, the graph on-prem or in your own cloud, billing by invoice and purchase order, and a security or legal review. Each is arranged with you directly, so none is self-serve. There is no SOC 2 report or DPA yet. A team of any size can buy Team by card.

Talk to us →

For early customers, the code FOUNDER takes 50% off for the life of your subscription, up to $1,000 a month.

A seat is one person on any of their machines, and CI needs no seat. Checkout is not open yet, and these amounts may change before it opens. The FOUNDER code applies when checkout opens. Team's base fee pays for the hosted graph. VAT, EU reverse charge and US sales tax are calculated at checkout.

Early access

Ask for a trial, or tell us what you're waiting for.

The free tools need no sign-up: install them. Leave an email for a Pro trial key, for word when the hosted graph opens, or to talk about Enterprise.

a person reads these and replies · no other email

early access

A validation demo is in progress: an agent measured with and without the recording, by false positives caught and mutants killed. It is not released yet.

limits

What gets recorded and how far to trust an answer are on the questions page.