Migrating to RubyLLM 2.0: What We Deleted and What Came Back

Two gems for one job

# config/initializers/ruby_llm_openai_responses_patch.rb
# Monkey patch: Responses API requires a flat tool format, not a nested one
module RubyLLM::Providers::OpenAI::Tools
  module_function

  def tool_for_responses(tool)
    {
      name: tool.name,
      type: "function",
      description: tool.description,
      parameters: parameters_schema_for(tool)
    }
  end
end

class ApplicationTool < RubyLLM::Tool
  def to_openai
    RubyLLM::Providers::OpenAI::Tools.tool_for_responses(self)
  end
end

def render_prompt(namespace, key, context = {})
  template = @prompts.dig(namespace.to_sym, key.to_sym)["prompt"]
  binding_obj = Struct.new(*context.keys).new(*context.values).instance_eval { binding }
  ERB.new(template).result(binding_obj)
end

class Reports::ResultTool < ApplicationTool
  description "Return the structured report items."
  param :items, type: :array, desc: "The report items"

  def execute(items:)
    halt({ "items" => items })
  end
end

What 2.0 changed, and what it didn't

• OpenAI over the Responses API. 2.0 splits providers from wire protocols, and OpenAI uses Responses by default. This is what kept ruby-openai in our Gemfile. With it gone, one RubyLLM client covers OpenAI, Anthropic and Gemini, the Responses patch goes, and so does our hand-written tool loop.
• Provider tools with their own API. Web search and other vendor-hosted tools are declared with provider_tools :web_search next to your own tools, instead of a raw hash in with_params. The deep_merge patch goes.
• Fallbacks. fallbacks "model-b", "model-c", on: [...] retries a failed request on the next model. In 1.x there was nothing like it; our AIClient did it.
• Prompt rendering outside agents. RubyLLM.render_prompt renders any template under app/prompts with locals. In 1.x templates belonged to an agent: rendered through its class, from its own folder.
• Tool approval and cancellation. requires_approval parks a tool call and persists it; cancel on an acts_as_chat record is persisted too, so a stop in one request reaches a job in another.
• Usage per attempt. Every physical request to a vendor, failed retries and fallbacks included, gets its own usage entry and usage.ruby_llm event. 1.x only saw the successful response and counted tokens and cost per message.
• Responses as a reusable protocol. In 1.x an OpenAI-compatible vendor could already subclass the OpenAI provider, but only for Chat Completions. In 2.0 a vendor that speaks Responses can reuse that protocol too.

• Structured output across all three vendors.
• Agents and the app/prompts/<agent>/instructions.txt.erb convention. 2.0 added model blocks that read inputs, rescue_from, and agent-level fallbacks, provider_tools and tool_options.
• Tool choice (with_tools(..., choice:), forcing a specific tool), cost per message and per chat, instrumentation through ActiveSupport::Notifications (chat.ruby_llm, tool_call.ruby_llm, request.ruby_llm, embedding.ruby_llm).
• Provider registration with RubyLLM::Provider.register, and OpenAI-compatible providers built by subclassing the OpenAI one.

How we moved

Step 1: freeze what "working" means

Step 2: swap the transport

Step 3: prompts

Step 4: result tools become schemas

class Documents::ClassificationSchema < Schematist::Schema
  array :labels, description: "One label per category in the input" do
    object do
      integer :category_id, description: "ID of the category, copied verbatim from the input"
      string :relevance, enum: %w[high medium low none]
    end
  end
end

Step 5: callers become agents

class Documents::Classifier
  include LLMConfigurable

  llm_profile :classify, task: :documents, timeout: 5.minutes.to_i,
              tools: [ Documents::ClassificationResultTool ]

  def request_labels(client: AIClient)
    client.ask(messages: [ { role: "system", content: build_prompt } ], **classify_llm_params)
  end
end

class Documents::ClassifierAgent < ApplicationAgent
  runs_on ModelChains::CAREFUL
  instructions { prompt("instructions") }
  schema Documents::ClassificationSchema
  context RubyLLM.context { |config| config.request_timeout = 5.minutes.to_i }
end

def request_labels(agent: Documents::ClassifierAgent)
  prompt = RubyLLM.render_prompt("documents/classifier_agent/question", document:, categories:)
  agent.new.ask(prompt).parsed
end

module NoProviderRequests
  def post(url, _payload, **, &)
    raise AttemptedRequest, "a test tried to POST #{url}, give it a fake agent"
  end
end

RubyLLM::Transport::Connection.prepend(NoProviderRequests)

Step 6: check it against the old version

Agents, even small ones

class Reports::SummaryAgent < ApplicationAgent
  inputs :account
  runs_on ModelChains::FAST
  instructions { prompt("instructions") }
  tools { [ DocumentSearchTool.new(account) ] }
  temperature 0.3
end

classifier.request_labels(agent: FakeAgent.answering("labels" => []))

What we kept: the circuit breaker

module AI::CircuitBreaker
  THRESHOLD = 3
  COOL_OFF = 30.minutes.to_i
  # The vendor is down or turned us away. A 400 is about the request, not the model, and never trips a light.
  TRIPPED_BY = [
    *RubyLLM::Fallback::DEFAULT_ERRORS,
    RubyLLM::UnauthorizedError, RubyLLM::ForbiddenError, RubyLLM::PaymentRequiredError
  ].freeze

  # The chain without the models sitting out; all of it if every one is.
  def self.usable(chain)
    closed = chain.reject { |model| light(model).color == Stoplight::Color::RED }
    closed.presence || chain
  end

  # RubyLLM already hit the error; raise it once more inside the light so Stoplight counts it.
  def self.failed(model, error)
    light(model).run { raise error }
  rescue StandardError
    nil
  end

  def self.light(model)
    Stoplight("ai_models:#{model}", threshold: THRESHOLD, cool_off_time: COOL_OFF, tracked_errors: TRIPPED_BY)
  end
end

class ApplicationAgent < RubyLLM::Agent
  class_attribute :chain, default: []

  def self.runs_on(chain)
    self.chain = chain
    model chain.first
    fallbacks(*chain.drop(1), on: ModelChains::FALLBACK_ERRORS)
  end

  def initialize(**)
    super
    usable = AI::CircuitBreaker.usable(chain)
    chat.with_model(usable.first)
    chat.with_fallbacks(*usable.drop(1), on: ModelChains::FALLBACK_ERRORS)

    chat.after_fallback do |fallback|
      AI::CircuitBreaker.failed(fallback.from.id, fallback.error)
    end
  end
end

A custom provider, five days before the deadline

module RubyLLM::Providers
  class PerplexityAgent < Provider
    class Responses < Protocols::Responses
      PRESETS = %w[fast low medium high xhigh].freeze # the Agent API presets we accept, sent in place of a model

      def completion_url = "agent"

      private

      def render_payload(messages, model:, **)
        payload = super
        payload[:preset] = payload.delete(:model) if PRESETS.include?(model.id)
        payload
      end
    end

    protocol :responses, Responses

    def api_base = "https://api.perplexity.ai/v1"
    def headers = { "Authorization" => "Bearer #{@config.perplexity_api_key}" }
    def self.configuration_requirements = %i[perplexity_api_key]
  end
end

RubyLLM::Provider.register :perplexity_agent, RubyLLM::Providers::PerplexityAgent

RubyLLM.chat(model: "low", provider: :perplexity_agent, assume_model_exists: true)
       .with_provider_tools(:web_search)

What went away

• The whole custom transport: the provider adapters, the OpenAI wire protocol with its helpers, the retry manager, both monkey patches.
• AIClient, the profile DSL, the prompt store, and every result tool.
• ruby-openai, plus the webhook integration for deep research, a feature we removed rather than port, since RubyLLM 2.0 has no public API for background Responses.
• The tests that covered all of that, which went with the code.

If you're considering the same move