The nuts and bolts of our Ruby-based realtime charts solution

Scout's realtime charts have been a big hit. Once you start using them for major deploys or performance incidents, going back to ten terminal windows running "top" feels like the dark ages.

Realtime was a lot of fun to implement and it's been rock-solid so far. A big reason it was so much fun? We were able to implement all of it in Ruby (outside the Javascript used to handle websockets in the browser) and didn't need to deploy any infrastructure .

So, how did we go about it?

Overview

The Scout agent runs a process on your server(s), sending metrics to Pusher's realtime PaaS every second or so. Pusher publishes those metrics in realtime through websockets to any browser that’s connected and authenticated for your metrics.

So the essence of realtime:

• Scout-managed process on your server
• HTTP posts from your server to Pusher
• Websocket connections between Pusher and browser(s)

In Code

Diagrams are for project managers, right? Here's how it looks in simplified Ruby code:

The Scout Agent (Ruby)

Javascript on scoutapm.com

So far so good – but what about session control and security?

Realtime Session Control

You initiate a realtime session by clicking the realtime button on a Scout chart. When clicked, scoutapm.com will send a directive to the agent (which runs once a minute) telling it to spin up a a realtime streamer process.

Under normal operation, the Scout agent sends data directly to scoutapm.com. Only when viewing a realtime chart is data sent to Pusher.

Security

We need to be careful that only members of the originating Scout account have access to reatime data as it flows through the 3rd party service (Pusher). Here’s how we restrict access:

Why didn't we roll our own?

We don't stream metrics unless you initiate a realtime session – most of the time, Scout isn't using realtime. This means our realtime volume isn’t huge. We're able to use Pusher's startup plan ($49/mo), which is a trivial expense compared to managing our own realtime infrastructure.

We investigated two Ruby EventSource servers for handling web sockets: Cramp and Goliath. It was clear that just using Pusher was a much simpler path for our limited use case.

Also see

• The Scout agent's Scout::Streamer class (handles sending data to Pusher)
• Pusher
• Smoothie (for our realtime charts)

Takeaways

• Use Pusher. Rolling your own realtime infrastructure isn’t trivial. Using a PaaS like Pusher lets you start focusing on the experience faster.
• Easy to get started. You can use tools that you are already familiar with (Ruby + Javascript).
• Don’t be intimidated by realtime. Is there anything in your product that can be enhanced by realtime communication?