Translating Rich Documents with AI using Ruby_LLM

• A unified API across OpenAI, Gemini, and Anthropic.
• Stateful chats that turn segmented batches into a single conversation.
• Schema constraints that return parsed Ruby hashes instead of raw strings.
• Granular control over temperature and reasoning effort per request.
  
  

Why a Pipeline?

• Extract: Decomposes the document into translatable segments.
• Batch: Groups them under a character size budget.
• Translate: Sends each batch to the model under a schema constraint.
• Validate: Checks the answers against the source before persisting.
• Apply: Writes the translated text onto a fresh copy of the document.
  
  

Extracting Segments

# Gemfile
gem "ruby_llm"
gem "schematist"
gem "nokogiri"

Segment = Data.define(:id, :text)

class TextExtractor
  PROSE = "p, h1, h2, h3, h4, li, blockquote"

 # Skip empty editor artifacts and extract only leaf nodes to prevent double translation.
  def self.segments(html)
    nodes = Nokogiri::HTML5.fragment(html).css(PROSE).reject { |node| node.at_css(PROSE) }
    nodes.each_with_index.filter_map do |node, index|
      Segment.new(id: index.to_s, text: node.inner_html) unless node.text.strip.empty?
    end
  end
end

The Schema Contract

class LLMSchema < Schematist::Schema
    # ruby_llm unwraps this shape: `name` names the schema, `schema` is the document.
  def to_json_schema
    document = super
    { "name" => document["title"], "schema" => document.except("$schema", "title") }
  end
end

class TranslationSchema < LLMSchema
  array :translations, description: "Exactly one entry per source segment, same ids, any order." do
    object do
      string :id,   description: "The segment id, copied verbatim from the input."
      string :text, description: "The translation. If the input contained HTML, the same " \
                                 "markup with only human-readable text translated."
    end
  end
end

The Translator

class Translator
  # Matches the full model's output via strict prompting
  MODEL = "gpt-5.4-mini"

  def initialize(language:)
    @language = language
  end

  def call(segments)
    payload = JSON.generate(segments.map { |s| { id: s.id, text: s.text } })
    response = chat.ask("Translate these segments:\n#{payload}")
    raise "unparseable response: #{response.content.class}" unless response.content.is_a?(Hash)
    translations = response.content.fetch("translations")
    verify_ids!(segments, translations)
    translations.to_h { |t| [t.fetch("id"), t.fetch("text")] }
  end

  private

  def chat
    # Deterministic output (copying, not inventing). with_params, not with_temperature:
    # ruby_llm 1.16 normalizes with_temperature to 1.0 for the whole gpt-5 family
    # (fixed upstream in #719, unreleased); with_params merges after that normalization.
    @chat ||= RubyLLM.chat(model: MODEL)
                     .with_params(temperature: 0)
                     .with_thinking(effort: :none)
                     .with_instructions(system_prompt)
                     .with_schema(TranslationSchema)

  end

  def system_prompt
    <<~PROMPT
      You are translating the readable text of a document into #{@language}.
      Reproduce any HTML markup exactly: every tag and attribute must reappear unchanged;
      only human-readable text (including alt and title attribute values) is translated.
      Keep proper nouns, brand names and code identifiers in their original form.
      Return exactly one entry per source segment with the same id, copied verbatim.
    PROMPT
  end

  def verify_ids!(segments, translations)
    expected = segments.map(&:id)
    answered = translations.map { |t| t.fetch("id") }
    raise "ids answered more than once" if answered.tally.any? { |_, n| n > 1 }
    raise "ids do not match the batch" unless answered.sort == expected.sort
  end
end

The Validation Gate

class Validator
  class Invalid < StandardError; end

  # Compare presence only, as the model is expected to translate these values.
  PROSE_ATTRIBUTES = %w[alt title caption].freeze

  def self.validate!(segments, translated_by_id)
    segments.each do |segment|
      text = translated_by_id.fetch(segment.id)
      raise Invalid, "blank translation for #{segment.id}" if text.strip.empty?
      raise Invalid, "markup changed for #{segment.id}" unless tags(segment.text) == tags(text)
      raise Invalid, "attributes changed for #{segment.id}" unless attributes(segment.text) == attributes(text)
    end
  end

  def self.tags(html)
    Nokogiri::HTML5.fragment(html).css("*").map(&:name).tally
  end

  # Tag counts miss injected styles or scripts; validate every attribute explicitly.
  def self.attributes(html)
    Nokogiri::HTML5.fragment(html).css("*").flat_map do |element|
      element.attribute_nodes.map do |attribute|
        value = attribute.value unless PROSE_ATTRIBUTES.include?(attribute.name)
        [element.name, attribute.name, value]
      end
    end.tally
  end
end

1. Strengthen the prompt, never relax the validator: a legacy editor once split a single sentence across three <p> tags. The model merged them into a better paragraph, but the validator correctly rejected it (16 tags ≠ 18 tags). You will be tempted to tolerate "benign" tag-count drift. Don't. We added a prompt rule instead ("never merge or split paragraph boundaries"), dropping the failure rate to zero. A relaxed validator is a blind spot for dropped links; a prompt rule is a targeted fix.
2. Tolerances must be explicit: the PROSE_ATTRIBUTES exemption list is where deliberate tolerance lives. Values like alt and title are prose; the model should change them. Later, we added caption to support Action Text image galleries. Every tolerance is an explicit decision, not an accident.

class Applier
  def self.apply(html, translated_by_id)
    copy = Nokogiri::HTML5.fragment(html)
    copy.css(TextExtractor::PROSE)
        .reject { |node| node.at_css(TextExtractor::PROSE) }
        .each_with_index do |node, index|
          node.inner_html = translated_by_id.fetch(index.to_s) if translated_by_id.key?(index.to_s)
        end
    copy.to_html
  end
end

Batching: Turns of One Conversation

class Batcher
  MAX_BATCH_CHARS = 20_000

  def self.batches(segments)
    return [] if segments.empty?

    segments.each_with_object([[]]) do |segment, acc|
      current = acc.last
      if current.any? && current.sum { |s| s.text.length } + segment.text.length > MAX_BATCH_CHARS
        acc << [segment]
      else
        current << segment
      end
    end
  end
end

html = document.body_html
segments = TextExtractor.segments(html)
translator = Translator.new(language: "pt-BR")

translated = Batcher.batches(segments).reduce({}) do |acc, batch|
  acc.merge(translator.call(batch))
end

Validator.validate!(segments, translated)
translated_html = Applier.apply(html, translated)

# Input:  <p>Orion will carry the crew about <a href="/mission-profile"
#         title="Mission profile">4,000 miles</a> beyond the far side of the Moon.</p>
# Output: <p>Orion levará a tripulação a cerca de <a href="/mission-profile"
#         title="Perfil da missão">4.000 milhas</a> além do lado oculto da Lua.</p>

Lessons from Production

Trade-offs: When (and When Not) to Use It

• Atomicity: A failure at any stage leaves the source untouched and leaves no partial copies behind.
• Deterministic Guards: Markup fidelity and alphabet checks don't depend on the model having a good day.
• Testability: Extract, batch, validate, and apply are pure functions you can unit-test without API calls.
• Portability: The provider abstraction means the pipeline runs on OpenAI, Gemini, or Anthropic with a single model swap.

Conclusion

Additional Resources

• 📁 ruby_llm gem on GitHub
• 📁 schematist on GitHub
• 📄 RubyLLM documentation — structured output