How to fetch a Turbo Stream?

Dog, who sometimes responds to the name Ben, going after a ball labelled Turbo Stream

Hotwire is nice. It feels like a breath of fresh air when compared to front-end frameworks like React or Angular. More importantly, it is a genuine improvement to the experience of writing JavaScript for Rails apps. Turbo Frames and Streams extract out common JavaScript operations like adding, removing and replacing HTML, and let you achieve basic interactivity using Ruby (that generates or triggers JavaScript). And, if you need to actually write JavaScript, Stimulus provides an uncomplicated way to organize it.

However, any new tool or framework comes with its own learning curve, and Hotwire is no different. As a developer who values being up to date with new tools, you don’t mind spending the time to learn Hotwire (especially because of the potential quality of life improvements it brings), but it can be frustrating when something that was easy to do the “old” way is now hard or downright inscrutable. This can be especially true when you’re at the “edges” of Hotwire’s default use case.

Consider a form submission and a server that responds with a turbo stream. In the default use case, Turbo intercepts form submissions, sends the data to the server and then automatically processes the turbo stream response (let’s assume your server is responding with a turbo stream), which contains a bunch of <turbo-stream> elements. Turbo adds these <turbo-stream> elements to the document, and Turbo stream “actions” like add or replace are triggered for each element.

Now, what if instead of letting Turbo doing its thing, you wanted to intercept the form submission so that you could make your own request to the server and then run some custom JavaScript based on the response? Would it “just work” to make a request to your server using fetch?

everything looks OK in my logs and browser’s network tab, but somehow the content is not replaced…

i don’t know what the rules are around turbo-stream and replace…

Why fetch a Turbo Stream?

Even though Ruby on Rails has strong support for native HTML forms, it is sometimes necessary or desirable to use the fetch API (or something like axios) to send user information either to your app server or a third party.

Consider an autobiographical example. Imagine the app you’re working on sells things and you’re using Stripe as the payment processor. Stripe offers a JavaScript library which you can leverage to submit payments. This JS library contains a confirmPayment method which collects the user’s credit card information and submits it directly to the Stripe servers. This is a good thing, because it absolves you of the need to handle sensitive credit card information (being PCI compliant is a headache as far as I understand it).

Let’s say your app also allows promotional codes to be used, and that promo codes in your app live outside Stripe, because Stripe does not support promo codes for your type of product. What this means in practice is that before calling Stripe’s confirmPayment, you have to send a request to your server with the promo code that the user wants to apply. This request can respond in one of two ways - either the promo code is valid, or it isn’t. If it is valid, then you can go ahead and call confirmPayment. If it isn’t valid, then you need to render validation errors.

Life before Turbo Streams

A relatively straightforward way to orchestrate such a flow would be to write some custom JavaScript in a Stimulus controller:

export default class extends Controller {
  // called when the user fills in credit card info and promo code
  async submitPayment() {
    const promoCodeApplied = await applyPromoCode()
    if (promoCodeApplied) {
      this.stripe.confirmPayment(...)
    } // else user should see a promo code validation error
  }

  async applyPromoCode(){
    const response = await fetch("my-app.com/apply_promo_code", {
      method: "PATCH",
      body: JSON.stringify({ promo_code: this.promoCodeTarget.value }),
      headers: ...
    });

    return response.status === 204
  }
}

While this would work fine in the situations where the user-entered promo code applies succesfully, you run into an issue if the promo code does not apply successfully. If your server is responding to this fetch with JavaScript (in a .js.erb file), then there is nothing actually executing that JavaScript when the response comes back to the user’s browser. To make that happen, you’ll need to do something like this:

async applyPromoCode() {
  const response = await fetch("my-app.com/apply_promo_code", {
    method: "PATCH",
    body: JSON.stringify({ promo_code: this.promoCodeTarget.value }),
    headers: ...
  });

  if (response.status === 204) {
    return true
  } else {
    window.eval(await response.text()) // assuming the response is JS that needs to be executed
    return false
  }
}

This is the “old” way.

fetching Turbo Streams

To render validation errors, the JavaScript in your .js.erb file is likely doing something uncomplicated like replacing the form on the page, with a form that is generated using the invalid promo code (assuming that the form, like a typical Rails form, will show validation errors in that case). You could, in theory, get rid of at least some of this JavaScript (both in the Stimulus controller and in your .js.erb file) by using Turbo Streams, which provides a replace action for this purpose.

Your Rails controller could just respond like so:

def apply_promo_code
  ...
  unless promo_code_applied
    @error = true
    render status: :unprocessable_entity
  end
end

And, in your turbo stream view, you’d do something like:

-# apply_promo_code.turbo_stream.html
= turbo_stream.replace("promo-code", partial: "promo_code_form", locals: { promo_code: @promo_code, error: @error })

The JS in the Stimulus controller can no longer use the window.eval trick, because JS is no longer coming back from the server. If you removed the window.eval and didn’t replace it with something else, would the user see a validation error if their promo code wasn’t valid?

As I alluded to earlier, a validation error won’t be rendered on the page. For a Turbo Stream action to occur, the <turbo-stream> element needs to be added to the DOM. It is at this point that Turbo event listeners in the browser kick in and perform the desired operation (replace, add etc.). In essence, you need a way to programatically invoke the turbo stream. There are a few ways to solve this problem, all minor variations of each other.

First, you can just add the response text to the document:

async applyPromoCode() {
  const response = await fetch("my-app.com/apply_promo_code", {
    method: "PATCH",
    body: JSON.stringify({ promo_code: this.promoCodeTarget.value }),
    headers: { Accept: "text/vnd.turbo-stream.html" }
  });

  if (response.status === 204) {
    return true
  } else {
    const tsElement = document.createElement("template");
    const text = await response.text()
    tsElement.innerHTML = text;
    document.documentElement.appendChild(tsElement.content.firstChild);
    return false
  }
}

This is nice and conceptually simple, but it is tedious. I wrote the above code, but I’m sure I won’t remember how to do it if you ask me a week from now.

Fortunately for us, Turbo provides a method that does pretty much the same thing, which saves us a few lines of code and leads us to the second way to solve the problem. Turbo.renderStreamMessage can be used to add all the <turbo-stream> elements to the DOM and “invoke” them.

async applyPromoCode() {
  const response = await fetch(...)

  if (response.status === 204) {
    return true
  } else {
    const text = await response.text()
    Turbo.renderStreamMessage(text)
  }
}

Finally, if you wanted to be even more concise, you could use the request helpers provided by the @rails/request.js library. Turbo Stream requests made by this library automatically call Turbo.renderStreamMessage with the response contents.

import { patch } from "@rails/request.js"

...

async applyPromoCode() {
  const response = await patch("my-app.com/apply_promo_code", {
    body: JSON.stringify({ promo_code: this.promoCodeTarget.value }),
    responseKind: "turbo-stream",
  })

  return response.status === 204
}

Conclusion

In this article, we saw how we can programatically invoke Turbo Streams. Conceptually, it turns out that just adding a <turbo-stream> element to the DOM triggers the action associated with the turbo stream, whether it’s an add, replace or what have you. As long as the <turbo-stream> element is formatted correctly and contains the data neccessary for the action it wants to perform, Turbo observers will do the rest of the work for you. That’s pretty nifty.

Though we only talked about Turbo Streams in the context of responses to fetch requests, it doesn’t actually matter where the turbo stream is coming from. You can use this technique for websocket connections as well.

What’s more, you can also define custom actions above and beyond the default ones that Turbo provides. If you haven’t read it already, check out my article on breaking out of Turbo Frames for an example of how to create a custom Turbo Stream action. As expected, adding <turbo-stream> elements with custom actions will make Turbo automatically run the custom action for you. If you want to learn more about what a Turbo Stream element is and how adding it to the DOM causes JavaScript to run, check out my article on the anatomy of a Turbo Stream.

If you got value out of this article and want to hear more from me, consider subscribing to my email newsletter to get notified whenever I publish a new article.

Want to be notified when I publish a new article?