Using Turbo and Rails to develop a browser extension

This article will cover how to develop a browser extension in Chrome while employing Rails to do almost all of the heavy lifting. At Clinch Talent, we try to minimise Javascript and maximise Ruby and HTML. Through careful usage of Hotwire, we now have less than 2% javascript in our code base and this number continues to drop over time.

Below, we will demonstrate how to

Minimise handwritten client side Javascript and maximise server side Rails code
Test the browser extension with Capybara
Prepare the browser extension for shipping

While this article uses Rails as the server side framework, the same technique should in theory work with any framework that supports Turbo.

Iteration #1: Hello browser extension

Make a new Rails application

rails new browser_extension
cd browser_extension
mkdir client

The following files are needed to make the basic browser extension

mkdir -p client/html
mkdir -p client/js
mkdir -p client/css
touch client/manifest.json
touch client/html/popup.html
touch client/js/popup.js
touch client/css/popup.css
wget https://cdn.jsdelivr.net/npm/@hotwired/turbo@7.2.0/dist/turbo.es2017-esm.js -O client/js/turbo.js

// manifest.json
{
  "name": "Browser Extension",
  "description": "A Turbo powered browser extension",
  "version": "0.1",
  "manifest_version": 3,
  "host_permissions": ["http://localhost:3000/*", "http://127.0.0.1/*"],
  "action": {
    "default_popup": "html/popup.html"
  }
}

Note the host permissions have been set to the url of the Rails development server, and the url of the test server. If this is causing problems, just set the host_permissions to ["http://*/*"] and deal with it later

//popup.js
import './turbo.js'
//Any other client side javascript here

<!-- popup.html -->
<html>
  <head>
    <script src="../js/popup.js" type="module"></script>
    <script src="../css/popup.css" type="module"></script>
  </head>
  <body>
    <turbo-frame id="content" src="http://localhost:3000/candidates">
      Hello from client
    </turbo-frame> 
  </body>
</html>

/* css/popup.css */
body {
  width: 20rem;
  height: 20rem;
}

input {
  margin-bottom: 1rem;
}

Open the browser extension

Visit chrome://extensions
Enable developer mode
Click Load unpacked and select the location of your browser extension

A preview of iteration #1

To refresh changes, click the reload button on the Chrome extensions page.

Observations:

Opening the Chrome Web inspector is annoying as it will disappear when the popup closes
If you close and open the popup, the popup resets
We have no tests

Iteration #2: Hello Turbo

Controllers, routes, and views

Scaffold up a Rails application and make a controller with an index action. Since Clinch Talent is in the business of Recruitment, we’re going to use Candidate as the resource.

rails g controller candidates index
rake db:create
rails s

Add the routes

# routes.rb
resources :candidates

Set the layout

We don’t need a layout file served from the server, because the browser extension client code will act as the layout

# app/controllers/application_controller.rb
class ApplicationController
  layout false # just send back turbo frames
end

Define the controller action and view

# app/controllers/candidates_controller.rb
class CandidatesController < ApplicationController
  def index
  end
end

<!-- index.html.erb -->
<%= turbo_frame_tag 'content' do %>
  Hello from Rails
<% end %>

What just happened?

A preview of iteration #2

On load of the browser extension, the Turbo library running in the browser located a <turbo-frame id="content"> tag, and examined the src attribute. The src attribute points to a Rails server url, and Turbo called fetch on this url. The Rails server responded with html containing a <turbo-frame> with the matching id content and Turbo replaced the contents of the client with the server turbo-frame html.

A quicker feedback loop

Observe the updates to your browser extension by opening the popup. One click of the mouse is enough to get a tight feedback loop. There is now no need to reload the browser extension from the Chrome extensions page. Experiment by changing the message inside the turbo-frame and then reopening the popup window.

Iteration #3: Show some candidate data

Set up the models

Create an ActiveRecord model called Candidate which has a name and email address. Add some sample records.

rails g model candidate name:string email:string
rake db:migrate
rails runner 'Candidate.create!(name: "Bill", email: "bill@example.com") ; Candidate.create!(name: "Tom", email: "tom@example.com")'

Fetch the candidates

 def index
   @candidates = Candidate.order(created_at: :desc)
 end

Render the candidates inside the turbo frame

# candidates#index.html.erb
<%= turbo_frame_tag 'content' do %>
  <table>
    <tbody>
    <% @candidates.each do |candidate| %>
      <tr>
        <td><%= candidate.name %></td>
        <td><%= candidate.email %></td>
      </tr>
    <% end %>
    </tbody>
  </table>
<% end %>

A preview of iteration #3

Iteration #4: Submit some data

Add new controller actions

# candidates_controller.rb
class CandidatesController < ApplicationRecord
  def index
    @candidates = Candidate.order(created_at: :desc)
  end

  def new
    @candidate = Candidate.new
  end

  def create
    @candidate = Candidate.create!(params.require(:candidate).permit(:name, :email))
    redirect_to candidates_path
  end
end

Create new view

Make sure to use the absolute url for any links as the browser extension client code will not intrinsically know about the location of the server side code.

<!-- candidates#new.html.erb -->

<turbo-frame id="content">
  <%= form_with(model: @candidate, url: candidates_url) do |f| %> 
    <div>
      <%= f.label :name %>
      <%= f.text_field :name %>
    </div>
    <div>    
      <%= f.label :email %>
      <%= f.email :email %>
    </div>
    <div>    
      <%= f.submit 'Save' %>
    </div>
  <% end %>
</turbo-frame>

Add a link to the new candidate form

<!-- candidates#index.html.erb -->

<%= turbo_frame_tag 'content' do %>
  <p><%= link_to "Add candidate", new_candidate_url %></p>
  <table>
    <tbody>  
     <% @candidates.each do |candidate| %>
     ....
<% end %>

A preview of iteration #4

Configure the turbo-root value in the browser client popup.html file

Warning: Without this, Turbo POST requests will not work

<!-- client/html/popup.html -->
<html>
  <head>
   <meta name="turbo-root" value="http://localhost:3000">

Disable the forgery protection origin check

# application.rb
module BrowserExtension
  class Application < Rails::Application
    #....
  config.action_controller.forgery_protection_origin_check = false
end

Recap

We have built a browser extension and added support for

viewing a table of candidates
viewing the candidate form
saving the candidate
redirecting urls

This has been achieved with minimal change to the popup.html, and no handcrafted javascript

Controller testing

Rails controller tests are a cheap and reliable way to test the browser extension. While they won’t test anything in the client itself, they will test the majority of the browser extension.

require "test_helper"

class CandidatesControllerTest < ActionDispatch::IntegrationTest
  test 'index' do
    get candidates_url
    assert_response :success
    assert_select 'turbo-frame#content' do
      assert_select "a[href='http://www.example.com/candidates/new']"
      assert_select 'table'
      assert_select 'tr', count: 2
      assert_select 'tr:nth-child(1) td:nth-child(1)', 'Bill'
      assert_select 'tr:nth-child(1) td:nth-child(2)', 'bill@example.com'
      assert_select 'tr:nth-child(2) td:nth-child(1)', 'Tom'
      assert_select 'tr:nth-child(2) td:nth-child(2)', 'tom@example.com'
    end
  end

  test 'new' do
    get new_candidate_url
    assert_response :success
    assert_select 'turbo-frame#content' do
      assert_select "form[action='http://www.example.com/candidates']" do
        assert_select "input[type='text'][name='candidate[name]']"
        assert_select "input[type='email'][name='candidate[email]']"
        assert_select "input[type='submit']"
      end
    end
  end

  test 'create' do
    post candidates_url, params: {candidate: {name: 'Bill', email: 'bill@example.com'}}
    assert_redirected_to candidates_url
  end
end

Browser testing

To test the browser extension client code, use Capybara.

Modify the Capybara set up to open the browser extension

Chrome will install the unpacked browser extension via the command line argument --load-extension=<path/to/browser/extension>. By adding this to the Capybara driven_by options, we can make the browser extension client code available to Chrome.

class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
  driven_by :selenium, using: :chrome, screen_size: [1400, 1400] do |driver_option|
    driver_option.add_argument("--load-extension=#{Rails.root.join('client')}")
  end
end

Create a capybara file

After the Capybara browser has been opened with the test extension, the chrome:://extensions page is visited and developer mode is enabled.

By visiting chrome-extensions://<extension-id>/html/popup.html, the browser extension client code is available to the test.

However, the client is still fetching all of its turbo frames from http://localhost:3000. To fix that replace the <meta name="turbo-root" and the content turbo frame src attribute with the url of the test. This test url changes with each test: http://#{Capybara.current_session.server.host}:#{Capybara.current_session.server.port}

class BrowserExtensionTest < ApplicationSystemTestCase
  test 'add a candidate' do
    visit 'chrome://extensions'
    enable_developer_mode
    visit "chrome-extension://#{find_chrome_extension_id}/html/popup.html"
    
    # now check the table
    assert_selector 'tr:nth-child(1)', text: /Bill/
    assert_selector 'tr:nth-child(2)', text: /Tom/
    
    # add a candidate
    click_link 'Add candidate'
    fill_in 'candidate_name', with: 'Tim Flimpson'
    fill_in 'candidate_email', with: 'tim@example.com'
    click_button 'Save'
    
    # check the save redirect worked
    assert_selector 'tr:nth-child(1)', text: /Bill/
    assert_selector 'tr:nth-child(2)', text: /Tom/
    assert_selector 'tr:nth-child(3)', text: /Tim Flimpson/
  end

private
  def enable_developer_mode
    execute_script "document.querySelector('extensions-manager').shadowRoot.querySelector('extensions-toolbar').shadowRoot.querySelector('#devMode').click()"
  end

  def find_chrome_extension_id
    execute_script("return document.querySelector('extensions-manager').shadowRoot.querySelector('#items-list').shadowRoot.querySelector('.items-container extensions-item').id")
  end

  def replace_base_url
    test_url = "http://#{Capybara.current_session.server.host}:#{Capybara.current_session.server.port}"
    execute_script "document.querySelector('meta[name=\"turbo-root\"]').value = '#{test_url}'"
    execute_script "const content = document.querySelector('#content'); content.src = content.src.replace('http://localhost:3000', '#{test_url}')"
  end
end

Shipping

Shipping the browser extension involves:

Replacing references to the development server url http://localhost:3000 with the production url. In this case it will be https://acme.example
Making a CRX file using the unpackaged folder and a certificate

Prepare a rake task that will

Copy the client files to a build folder
Replace references to http://localhost:3000 with https://acme.example

# lib/tasks/browser_extension.rake
#
# Note: this rake task uses the jq command line utility

desc "Build the browser extension"
namespace :browser_extension do
  localhost_url = 'http://localhost:3000'
  production_url = 'https://acme.example'

  task :build_production do
    `mkdir -p build/production`
    `cp -r client\/ build/production`
    `sed -i '' "s|#{localhost_url}|#{production_url}|g" build/production/html/popup.html`
    `cat client/manifest.json | jq '.host_permissions=(["#{production_url}"])' > build/production/manifest.json`
  end
end

Creating the CRX file manually

Visit chrome://extensions and click Pack extension.
Point the extension root directory at the build folder
Fill out the private key
Click pack extension

Conclusion

Using Turbo to develop the Clinch Talent browser extensions was an enormous time saver for us. We found improvements in:

Development pace. It is cheaper to develop a browser extension that sends turbo frames back and forth between the browser extension and the server, than to manipulate the dom with javascript, which is the method most encouraged.
Testing: By focusing on server side generated HTML, we can spend most of our testing effort in Rails controller tests, and minimise Capybara tests, which are expensive to run and more prone to random failures.
Updates: If most of the code is coming from the server side, then most updates can come without the involvement of the Chrome Web Store review process.