Runtime Analysis · Ruby
Your agent guesses.
Your runtime knows.
Runtime Analysis records the calls your objects make while your tests or your app run. Your coding agent and your editor query the recording for the classes that arrive at a call site, what a method returns and who calls it.
$ curl -fsSL https://www.runtimeanalyz.ing/install.sh | sh # the ra binary, and a walk through sign-in and the recorder gem$ ra record # runs your tests once and changes no file in your project$ ra verify_mock spec # the stubs that differ from your real code
The recorder, the editor's type hints and five tools, verify_mock and what_calls among them, are free with a free account, made the first time you sign in. The other tools are Pro, with a trial. Recording and every tool run on your laptop, from your test suite alone if you have no production recording, and your recordings stay on your machine.
The problem
In Ruby, what a method returns and who calls it are decided at runtime.
To change Ruby code, your agent needs to know four things: what a method returns, which collaborators an object needs, which of a class's methods each caller uses, and which boundaries your tests cross with real objects on both sides. The source contains none of the four. A type signature records what someone declared, coverage records which lines ran, and grep finds text. One recording of your test suite contains all four.
What runtime_analysis is
It records each call, builds a graph, and answers questions from the graph.
The recording covers the class of every object in your application* and every message it receives. The answers come back in your agent, your editor, the command line and CI.
The gem runs inside your test suite and records every message sent to every object in your application*: the receiver's class, the method called and the call site, in order. It writes class names and method names to files on your machine. You can also run it in your app in development, staging or production to record real traffic.
The graph is built on your machine from the recorded files. The methods your code calls on a value are treated as that value's type, so a method's return type is the set of methods called on its return value in the recorded runs. The hosted graph, once it opens and you opt in, will combine every deploy's runs into one graph.
Your agent asks over MCP and your editor over LSP. The command line and CI run the same tools. The tools read the graph and do not run your application.
Questions it answers
One recording answers questions about any method that ran.
Grep for .call matches every service object, proc and middleware in your app and
its gems. For one call site, the recording lists the receivers that arrived there, with their
classes and how often. Each tool asks one question of the same recording:
Testing
- Does this stub match the real thing?verify_mock
- Which boundaries do my tests only cross through a double?testing_pyramid
- Which specs reach the method I just changed?specs_that_reach
- How do I build a real Invoice for this test?how_to_reach
- How far does this unit test reach?how_far_it_reached
- Which untested methods are called most?unit_test_candidates
Understanding
- What does this method return?observed_shape
- What does it get passed?what_it_receives
- Who calls it, and how often?what_calls
- How did execution get here?how_it_got_here
- Which classes answer this message?who_actually_answered
- Where does this nil come from?where_nil_comes_from
Changing
- What will changing this signature break?what_a_change_touches
- Is anything still calling this?dead_api
- Did this PR break a contract?check_diff
Performance
- Where does the time go?where_time_goes
- Is there an N+1 in this request?whats_looping
Design
- Which roles is this class playing?split_interface
- Does this method belong on another object?feature_envy
- Which file belongs in another folder?relocate_file
- Which dependency cycle should I cut, and where?break_cycle
Asking the hosted graph anything else, in Cypher, is planned. See all the tools →
Examples
Three of those questions, with their answers.
Each example takes one question from the list above and shows the answer in the tool where you would read it.
Example 1 · a spec, written first
One spec, written first by an agent alone and by an agent with the recording.
✗ agent alone · green, but wrong
5
6
7
8
9
10
11
12
13
it "unlocks the next book" do # agent: failing spec first, stubbing the curriculum curriculum = instance_double(Curriculum) allow(curriculum).to receive(:unlocked_for).and_return([:geometry]) enrollment = Enrollment.new(student: Student.new) allow(enrollment).to receive(:curriculum).and_return(curriculum) expect(enrollment.unlock_next).to eq([:geometry]) end
stub Curriculum#unlocked_for is a false positive
your stub returns [:geometry] Array<Symbol>
the real one returns an Array whose
elements receive #unlocked?, #title
from 9 spec examples and 42 app runs
a
Symbol was never seen answering any of them
> Enrollment needs #unlock_next. Failing spec first, then make it pass
● I'll stub the curriculum and write the failing spec.
● Bash(bundle exec rspec spec/models/enrollment_spec.rb)
⎿ 1 example, 1 failure · undefined method 'unlock_next'
● Edit(app/models/enrollment.rb)
⎿ 1 example, 0 failures
● Red, then green. Done.
✓ agent + runtime analysis · fixed before it ran
5
6
7
8
9
10
11
12
13
it "unlocks the next book" do # agent → observed_shape(Curriculum#unlocked_for): element → #unlocked? #title · Book book = instance_double(Book, unlocked?: true, title: "Geometry") curriculum = instance_double(Curriculum, unlocked_for: [book]) enrollment = Enrollment.new(student: Student.new) allow(enrollment).to receive(:curriculum).and_return(curriculum) expect(enrollment.unlock_next).to eq([book]) end
● runtime_analysis · verify_mock(spec: "spec/models/enrollment_spec.rb") [MCP]
⎿ 1 stub · 1 false positive · the real one returns an Array whose elements receive #unlocked?, #title
● My stub returns Symbols and the real method returns Books. Fetching the recorded return.
● runtime_analysis · observed_shape(method: "Curriculum#unlocked_for") [MCP] Pro
⎿ element → #unlocked? #title · Book · from 9 spec examples and 42 app runs
● Rebuilding the spec with a double that matches.
● Bash(bundle exec rspec spec/models/enrollment_spec.rb)
⎿ 1 example, 0 failures
● The spec passes with a double that matches the real return.
The second time, the agent asked Runtime Analysis what
Curriculum#unlocked_for returns. That return was recorded while
Curriculum's own spec ran, and the agent built a double that matches it. The
spec's recording is enough for this, and recorded app runs add to it. The agent wrote no code
ahead of its test. verify_mock is free, and its finding includes the fix: return
an Array of Book. observed_shape, the second call, is a Pro tool.
Example 2 · boundaries only doubles cross
Every unit spec passes, and no example runs the two classes together.
A stub with the wrong return is a fault in one double. A whole boundary between two classes
can go untested the same way, whether you test first or after. Checkout's spec
stubs the gateway and the gateway's spec tests the gateway alone, so every line is covered and
no example runs the real Billing::Checkout calling the real
Billing::Gateway. Coverage counts lines, so its report shows nothing missing.
Runtime Analysis sorts the recorded calls into the layers of your testing pyramid and lists the boundaries your specs cross through a double and in no other way. If you record your app too, the list is ranked by traffic. It then drafts a test that reaches a method, with setup taken from a recorded run.
The same finding is reported in your agent, your editor and the command line.
testing_pyramid(stubbed_only: true) → structuredContent: { summary: "3 boundaries only crossed via doubles", blocks: [ { type: "table", columns: [boundary, stubbed, real, app runs], rows: [ ["Billing::Checkout#complete → Billing::Gateway#charge", 5, 0, 58], ["(no recorded caller) → Billing::Gateway.new", 2, 0, 0], ["(no recorded caller) → Account.find", 1, 0, 0] ] }, … ] }
3
4
5
▶ reached by 3 examples · 58 app runs def charge(amount) Receipt.new(id: "rc_…", amount: amount, paid: !amount.negative?) end
$ ra testing_pyramid --stubbed-only testing_pyramid · 3 boundaries only crossed via doubles Billing::Checkout#complete → Billing::Gateway#charge 5 0 58 (no recorded caller) → Billing::Gateway.new 2 0 0 (no recorded caller) → Account.find 1 0 0 models 37 billing 9 requests 6 finance 4 services 4 jobs 2 reporting 2
ra scaffold then drafts a test whose setup comes from a recorded run. Your agent
gets the same draft over MCP. An editor quick-fix that inserts it is planned.
$ ra scaffold Invoice#total # drafted by ra scaffold from the recorded setup in spec/models/invoice_spec.rb:6 RSpec.describe Invoice do it "total" do invoice = build(:invoice) line_item = LineItem.new(sku: "…") invoice.add_line(item: line_item, quantity: 1) expect(invoice.total).to be_a(Money) end end
Example 3 · interface use by caller
The recording shows which part of an interface each caller uses.
A type has a fat interface when each caller uses a different, small part of it, and the
recording measures that from the calls each caller made. split_interface reports
it as a hint and does not fail a build, because partial use of an interface is normal. The
example app's Curriculum has four callers, and each uses its own set of methods.
$ ra split_interface Curriculum Curriculum · 4 roles · 0 callers span more than one for Report final_books, weeks_for Report for Scheduler first_books, next_after Scheduler for Billing::Checkout for, unlock_paid_books Billing::Checkout for Enrollment unlocked_for, level_for, books_at, archived?, prerequisites Enrollment never called in any recorded run: name · extract 4 role interfaces ISP
The same answer comes back over MCP. An editor hint with a quick-fix that extracts the roles is planned.
The same graph answers other design questions from SOLID, the Gang-of-Four patterns, and package cohesion and coupling. Each answer names a method or file to move.
Types read from recorded runs
Sorbet and RBS have you declare each interface. Runtime Analysis reads the interfaces your code already depends on from recorded runs.
See more →Design findings, each with a move
Feature envy, dependency cycles, unstable dependencies and packages used in parts, each reported with the move to make and the recorded calls that support it.
See more →Where you ask
In your agent, your editor, your terminal and CI.
One recording of Curriculum#unlocked_for, queried by your agent, shown in your
editor, printed on the command line and checked in CI. The free tools and the editor's hover
work in the same four places. observed_shape, the code lens and
ra check shown here are Pro.
observed_shape( method: "Curriculum#unlocked_for" ) → structuredContent: { title: "Curriculum#unlocked_for", summary: "observed return", blocks: [ { type: "table", rows: [ ["return", "(no app messages)", Array], ["element", "#unlocked? #title", Book] ] }, { type: "text", lines: [ "· from 9 spec examples and 42 app runs"] } ] }
15
16
17
▶ reached by 9 examples · 42 app runs def unlocked_for(student) eligible(student) end
Curriculum#unlocked_for · 3 callers
returns Array (628)
elements Book (1506) → #unlocked? #title
arg 1: Student (628) → #completed?
$ ra observed_shape \ Curriculum#unlocked_for Curriculum#unlocked_for · observed return return → (no app messages) Array element → #unlocked? #title Book · from 9 spec examples and 42 app runs
▸ ra check --since origin/main check --since origin/main · 13 findings introduced · 185 methods changed ✗ stub Curriculum#unlocked_for is a false positive spec/models/enrollment_spec.rb:9 your stub returns [:geometry] Array<Symbol> the real one returns an Array whose elements receive #unlocked?, #title → return an Array of Book, as the real method did … 12 more Error: Process completed with exit code 1
Your agent gets the same tools over MCP, so it can check a stub in the edit that introduced it.
A Claude Code hook that runs verify_mock on every spec the agent writes is
planned.
> Enrollment needs #unlock_next. Test first, then make it pass
● I'll stub the curriculum collaborator and write the failing spec.
● Write(spec/models/enrollment_spec.rb)
⎿ +12 lines
● runtime_analysis · verify_mock(spec: "spec/models/enrollment_spec.rb") [MCP]
✗ stub Curriculum#unlocked_for is a false positive
your stub returns [:geometry] Array<Symbol>
the
real one returns an Array whose elements receive #unlocked?, #title
from 9 spec examples and 42 app runs
a Symbol was never seen answering any of them
● My double returns Symbols and the real method returns Books. Fetching the recorded return before I run the spec.
● runtime_analysis · observed_shape(method: "Curriculum#unlocked_for") [MCP] Pro
⎿ return → (no app messages) Array · element → #unlocked? #title Book
● Rebuilding the spec with a double that matches.
● Write(spec/models/enrollment_spec.rb)
● Bash(bundle exec rspec …)
⎿ 1 example, 1 failure · undefined method 'unlock_next'
● Red: unlock_next is undefined. Now the implementation.
● Edit(app/models/enrollment.rb)
● Bash(bundle exec rspec spec/models/enrollment_spec.rb)
⎿ 1 example, 0 failures
● Green, with a double that matches the real return.
Each MCP tool is also a command. Run ra verify_mock on your suite, or
ra check on a diff in CI, and a stub with the wrong return fails the build.
$ ra verify_mock spec/models/enrollment_spec.rb verify_mock spec/models/enrollment_spec.rb · 1 stub · 1 false positive · 0 unverified ✗ stub Curriculum#unlocked_for is a false positive spec/models/enrollment_spec.rb:9 your stub returns [:geometry] Array<Symbol> the real one returns an Array whose elements receive #unlocked?, #title from 9 spec examples and 42 app runs a Symbol was never seen answering any of them → return an Array of Book, as the real method did $ ra find_implementers '#unlocked?, #title' real types responding to {#unlocked?, #title} · 1 class Book 4 630 outside traced code (a factory or seeds)
What you can record
Record a test-first suite, a mutation-tested suite, or a running app.
What the recording gives you differs in each case.
A test-first suite
The collaborators you stub already exist, and each runs as a real object in its own spec, so your suite's recording is enough. You need no production recording and write no code ahead of its test.
See more →A mutation-tested suite
Mutant leaves your doubles as they are. Runtime Analysis checks them, lists the most-called untested methods, and gives recorded setups for building real collaborators.
See more →An app with no tests
Code with no specs is recorded when your app runs. The tools report its callers and the
classes it takes and returns, and ra scaffold drafts a first test from a
recorded run.
Test first
In a test-first suite, each collaborator is recorded in its own spec.
The class you are about to write does not exist yet. Its collaborators do, and the tests from
earlier cycles already ran them as real objects. When you mock Gateway in
Checkout's spec, Gateway's own spec still runs the real class, so
each double has a recording to be checked against.
The stub you just wrote is checked against what the real collaborator did in its own spec.
See which methods of the interface each caller uses, and which methods are called by their own spec and nothing else.
Test hard
Mutant tests the code under test and leaves your doubles unchecked.
mutant mutates the code under test and leaves the double you stubbed for a
collaborator as it is. instance_double checks that the method exists with the
right arity. Runtime Analysis checks what the stub returns against what the real collaborator
returned in recorded runs.
verify_mock
Stubs that disagree with the real object. Compares what each stub in a spec returns with what the real method returned in the recording, and reports the stubs that return a class the real method did not.
See more →unit_test_candidates
Which methods to test first. Every recorded method ranked by how many distinct callers it has and how many calls away the nearest spec is, to choose which untested code gets a test first.
See more →how_to_reach
The setup a method needs. The collaborators and the call path that put an object into the state a method needs, taken from a recorded run. Use it to write setup for tests that use real objects in place of doubles.
See more →split_interface
The role interfaces in a class. Groups a class's public methods by which callers used them in the recording, and lists each group as a role interface to extract.
See more →dead_api
Find dead public API. The public methods that no caller reached in any recorded run, including those whose only caller is their own spec. Shown as a hint in your editor.
See more →
If you never mock, verify_mock has nothing to check, and the fan-in ranking, setup
recipes and interface hints still apply.
No tests
With no tests, record the running app.
Coverage and mutation testing report on code that a test runs. Runtime Analysis records whatever runs, so with the recorder on in development, staging or production, code with no specs is recorded too. A path that runs more often has more recorded calls behind each answer.
For a module with no specs you get its callers, the classes each method takes and returns,
and the exception classes it raised. what_a_change_touches lists the callers a
signature change would break. ra scaffold drafts a first spec from a recorded
setup.
$ ra observe app/billing/billing/proration.rb observe app/billing/billing/proration.rb · 4 recorded methods Billing::Proration.for from 15 spec examples and 106 app runs callers Invoice#total · Billing::Checkout#complete · Api::QuotesController#show · spec/billing/proration_spec.rb returns Billing::Proration → #amount #adjust Billing::Proration#adjust from 10 spec examples and 46 app runs callers Invoice#total · spec/billing/proration_spec.rb returns Money → #cents #currency #to_ledger_s #to_s … 2 more # when a method raised, observe lists the exception class and how often $ ra scaffold Invoice#total # drafts a spec from a recorded setup
Under load the recorder drops records and counts them, and it does not block your app. Production support is still being hardened: see the questions page.
Plans
A free recorder, paid tools on your laptop, and a hosted graph to come.
On every plan, recording runs on your machine and the recordings stay there. A Pro licence is checked on your machine. The hosted graph will receive what you choose to send it, and Enterprise runs that graph inside your own network. What leaves your machine lists every field.
Free
$0
The recorder, five tools and editor type hints, with a free account.
- The recorder, for your test suite and your running app, on your machine.
verify_mock: the stubs whose return differs from what the real collaborator returned.what_calls: the callers of a method, from recorded runs.what_it_receives: the classes each argument was given.who_actually_answered: which classes received the message at a call site.where_nil_comes_from: the methods that returned nil, and what followed.- Your editor's type hints and hover.
- Nothing is sent to us.
Pro paid, with a trial
$19 a month $19 a developer × 1
Every other tool, run on your own recording.
- Call stacks, hot loops, dead code, which specs reach a line, and the rest of the capabilities.
- Feature envy, cycles and coupling, with one refactor report that ranks them.
- Available from the command line, your coding agent (MCP) and your editor (LSP).
- Runs on your laptop and in CI. The licence is a key checked on your machine.
- A trial lasts 14 days and needs no card. A person sends each key, so it does not arrive instantly.
Team
$88 a month $49 base + $39 a developer × 1
Everything in Pro, and one hosted graph shared by the team.
- Sync up to 5 applications into one hosted graph, from CI and from the servers you record.
- An application counts once, on any number of servers and environments.
- Coupling, feature envy and cycles measured across every deploy. Pro measures the runs recorded on one laptop.
- The hosted graph stores tokens, and your class and method names stay on your machines. What an upload contains.
- Recording a production server is at an early stage, so start with CI as the source.
- The hosted graph is built and is not open yet. Sign up and we will tell you when it opens.
Enterprise
Enterprise is for needs beyond the card plans: more than 5 synced applications, SSO and audit, the graph on-prem or in your own cloud, billing by invoice and purchase order, and a security or legal review. Each is arranged with you directly, so none is self-serve. There is no SOC 2 report or DPA yet. A team of any size can buy Team by card.
For early customers, the code FOUNDER takes 50% off for the life of your subscription, up to $1,000 a month.
A seat is one person on any of their machines, and CI needs no seat. Checkout is not open yet, and these amounts may change before it opens. The FOUNDER code applies when checkout opens. Team's base fee pays for the hosted graph. VAT, EU reverse charge and US sales tax are calculated at checkout.
Early access
Ask for a trial, or tell us what you're waiting for.
The free tools need no sign-up: install them. Leave an email for a Pro trial key, for word when the hosted graph opens, or to talk about Enterprise.
a person reads these and replies · no other email
A validation demo is in progress: an agent measured with and without the recording, by false positives caught and mutants killed. It is not released yet.
What gets recorded and how far to trust an answer are on the questions page.