Runtime Analysis is in early access. The recorder exists and runs against a real Rails app. The tools that answer questions from a recording, and the hosted graph, are still being built. Each answer below says which of those it is describing.
Running it in your process
What does it cost per call?
We don’t have a number from a real application yet, and we won’t quote one until we do.
What we have is a worst-case microbenchmark: a loop calling one-line methods that do almost nothing, so the wrapper is nearly the whole cost. On Ruby 3.4 that loop runs 64 times slower when recording call edges, and 117 times slower at the deepest tier. Those multiples are an upper bound. A method that builds a string, runs a query or renders a template does far more work per call, and the same fixed cost is a much smaller share of it.
Adaptive mode is the main lever. It records a path in full the first time it appears and samples it after that. On the same benchmark it writes about 60 times fewer rows and costs under a third of the deepest tier.
What does it do to memory and garbage collection?
With the native extension, the wrapper forwards your arguments without copying them. Each record is copied into a buffer the extension owns, outside Ruby’s heap, and a background thread encodes it and writes it to disk without holding Ruby’s global lock. Working out the call site and the deeper tiers’ argument classes still happens in Ruby and allocates; we haven’t measured the effect on GC in a real app. How the native sink works walks through the path.
Can it take my app down?
The rule the recorder is built to is that a fault in the tracer loses trace data and leaves your app running. Today that holds for these cases:
- The buffer fills up. The record is dropped and counted. The calling thread never waits.
- An internal check fails. That one recording stream is switched off and a fresh one is opened.
- Memory can’t be allocated. The record is dropped and counted.
- The disk write fails. The error goes to stderr and your app carries on.
- Your method raises. The exception reaches your code unchanged.
It does not yet hold everywhere. In development we have seen a Rails server freeze with the gem enabled and not without it, under concurrent requests during a code reload, in an app using ActionCable’s PostgreSQL adapter. The tracer was not part of the deadlock itself, but it triggered it. We are removing every lock the recording thread can wait on and moving class wrapping out of the code-loading window. Until that is done and verified, record your test suite and development, and treat production as something to try with care.
Does it work with threads, forking servers and fibers?
Each process and thread gets its own recording stream, so threads don’t contend for one file. After a fork the child opens its own streams. The call tree is tracked per fiber. Ractors are untested.
How does it hook into my code?
It prepends one module onto each class it records, and watches for new method definitions to do so.
It does not use TracePoint to capture calls. We have not yet tested it alongside APM agents or
other gems that prepend onto the same classes.
Methods defined before recording starts are not wrapped, so start it before your application code loads.
Can I turn it off, or sample?
RuntimeAnalysis.finish! stops recording in a running process. The wrappers stay in place and pass
straight through. Adaptive mode is the sampling option.
Which Rubies does it run on?
CRuby 3.2 and later; the test suite runs on 3.2, 3.3, 3.4 and 4.0. The C extension compiles at install. If it can’t, a pure-Ruby recorder takes over, which is slower.
What gets recorded
Every object and every message, really?
No. It records calls to methods defined in your own code: by default, files under app and lib.
You can add directories, and exclude directories or single files. Calls into gems, Rails and Ruby’s
core classes are not recorded, so a call to String#upcase or Array#map leaves no row.
What exactly is written to disk?
You choose a tier. Each adds to the one before.
- Tier 1: for each call, the caller’s and callee’s class name, method name, file path and line number, plus the file and line of the call site. Each call tree also gets an id, and a label you can set for where it started, such as a spec location or a job name.
- Tier 2: the class name of each argument, and the names and classes of keyword arguments.
- Tier 3: the object id of the receiver, each argument and the return value, and the return value’s class. An object id is an integer that says “the same object as that one” within a run.
- Tier 4: the class of an exception that passes through a call. No message and no backtrace.
At no tier does it record a value: no strings, no numbers, no attributes, no SQL, no output of
inspect.
Aren’t class and method names sensitive too?
Yes. They describe your domain model, and file paths describe your layout. That is why recordings are written to your own disk and nothing is sent anywhere unless you switch it on.
Where do the files go, and how big do they get?
They go to ./tmp/runtime_analysis by default, as JSON lines, one file per process and thread.
Every name is written once and referred to by number after that. You can rotate files by size, age
or number of call trees, and cap the total kept. With no rotation configured a file grows for as
long as the process runs, so set one for anything long-lived.
In a container the files go when the container does, unless you mount a volume. Collecting them from short-lived containers is part of the hosted work and isn’t built.
Trusting the answers
My tests cover part of the app. What about the rest?
A recording only knows what ran. A method no recorded run reached has no callers, no argument types and no return shape, and each tool’s page lists what that means for its answers. Recording your app as well as your suite widens what is seen.
What is missing today is a report of the gap itself: which methods and which call sites were never observed. That is designed and not built.
How do I know a recording still matches the code?
Today you don’t, beyond re-recording. The design is to fingerprint each method’s signature and body, keep the evidence for methods that haven’t changed, and mark the rest as unverified until they run again. Not built.
Is an observed type a real type?
It is what was seen: the classes that arrived and the messages that were sent to them. It is evidence from real runs, and a class that could arrive but never did is not in it.
How does it cope with metaprogramming?
Methods created with define_method are recorded like any other. A message handled by
method_missing is recorded as a call to method_missing, not under the name that was sent.
The hosted graph
The hosted graph is not built. This is what we intend.
What leaves my network?
Nothing, unless you opt in, and the choice is per project. Recordings hold the names and numbers listed above, never values.
Who can read it?
The design we are working towards encrypts the graph with a key held by your team, so that our servers store data they cannot read and queries run on your machine or in your CI. It is a design direction, not a shipped guarantee.
Is there a SOC 2 report, a DPA, an on-prem option?
No report and no DPA yet. Running the graph inside your own network is what the Enterprise tier is for.
How are different versions of my code kept apart?
By the method fingerprints described above: evidence attaches to a version of a method, so a deploy inherits what is still true and nothing more. Designed, not built.
Asking
Are the tools available now?
Not yet. The recorder is what exists. The tools on this site describe what the recording supports and how each will answer; they are being built for early access.
Do the tools need my source code or production access?
They read the recording. Several also read your source on your machine, to connect a value returned in one place to the calls made on it in another. They do not connect to your running app.
Does my code go to an AI provider?
Runtime Analysis makes no calls to a language model. If you use it from a coding agent, the agent you chose sees the answers it asks for, which contain class and method names.
What does a tool say when the recording has no answer?
It says so. A stub whose real method never ran is reported as unverified, not as passing.
The rest
What does this add to an APM, a type checker or a coverage report?
An APM tells you how long requests take. A type checker tells you what someone declared. A coverage report tells you which lines ran. None of them tells you which class actually arrived at a call site, which messages it was sent, or who called a method and with what.
Is it open source?
No. It is free with an account, and the licence is not final. The account tells us which tools you use and never what they ran on. Recordings are plain JSON lines on your disk, so they stay readable whatever happens to us.
Who else runs this in production?
Nobody yet. It runs against one real Rails application, ours, in development.