In my experience with web applications at non-Googley scale, most performance problems are occurring in places like SQL queries, rather than the raw speed of the web application stack itself.

Another really common mistake is tasks being performed inside the web request that should instead be offloaded to background job processing. If something is going to take more than milliseconds, it should be a job. If you see a PR where someone is bumping up the HTTP timeout in order to keep their slow processing request from being timed out by the webserver, they are doing something inline that should actually be a job.

Are there any good books or other resources on building out a system to do background processing? It seems that it's a common task, but there are different approaches.

The Ruby world at least seems to have settled on relatively simple Redis backed queues, where you directly request the job to be executed rather than use a more generic message broker.

For sure there are more complex scenarios, but if you're just getting started you'll be well served by just reading the docs for sidekiq.

Unless you're building something with very specific requirements just use Sidekiq [0]. It's easy to install and performs very well.

0 - https://github.com/mperham/sidekiq

Not a direct answer, but Rails now includes ActiveJob which abstracts over the various jobs are available and is nice and simple to set up.

> And when people whinge about performance... well, so what? Horizontal scaling, friends.

I hear this a lot, but in my experience, it's not a workable solution to this particular problem. 100% of the systems I've worked on in the conventional 'slow' languages (Ruby, Python, Elisp, declaration-less Common Lisp, etc) hit significant performance bottlenecks in typical single-thread use cases. These languages enable me to work quickly when making a prototype, but then I spend a long time optimizing it to run fast enough for a single user.

Don't get me wrong. The speed of development for these HLLs is great. It's much better to be able to move continuously from the first prototype to a production system. (I don't want to write a prototype in C, nor do I want to throw out a working HLL prototype and rewrite it all in C.) The benefit, though, is that I can get a prototype running quickly, and iterate quickly. It doesn't obviate all performance requirements just because there's an HTTP server involved.

Horizontal scaling is a great solution to "my web app suddenly got very popular, and it works great for 1000 occasional users but today I need the capacity to deal with 10,000 simultaneous users".

It really depends on the problem space, though. And for most reasonable applications, Rails/Sinatra scale just fine through well-understood horizontal methods. By the time you're getting to where they don't scale well, you're probably longing desperately for a rewrite anyway, because your core problem has shifted from "Does this solve an actual business case?" to "How do I make this support a heavy load?"

This split between micro and macro frameworks wasn't done in Ruby because of some intrinsic properties of the language. The same split was copied in many other languages, from JavaScript to Java. I think the reason it started in Ruby is because that language drew a lot of pioneers who said "the way things are right now isn't good enough, let's try something completely different" which mindset led to Rails in the first place, largely as a response to the Java web ecosystem, and then to Sinatra as a response to Rails. It's because this mindset is a sort of (admirable) perfectionism, seeking the holy grail of computer programming, and it's good that there are many people like that. A large number of these Ruby devs inevitably moved on from Ruby in this continual search, some seeking statically typed languages like Haskell or OCaml but most popularly Go, and some seeking dynamically typed languages like Clojure and Elixir but often settling on Node.js.

> This split between micro and macro frameworks wasn't done in Ruby because of some intrinsic properties of the language. The same split was copied in many other languages, from JavaScript to Java. I think the reason it started in Ruby

web.py predates Sinatra (the 2005 rewrite of Reddit was on web.py), Django is about as old as RoR, Zope dates back to the late 90s, TwistedWeb is quoted in the 2003 WSGI PEP (333).

Stating that Ruby started µfw and the split between micro- and macro-frameworks seems like an incomplete and severely blinkered reading of history.

I do lots of experiments in Sinatra and some of those use Sequel (when I need a database). I think Sinatra has some rough edges, and it surprised me how I needed to initialise a Sinatra app:

    module MyApp
      class Web < Sinatra::Base
        def initialize(app = nil, params = {})
          super(app)
          # setup stuff
        end
      end
    end

Also, I also find it sad that people never seem to know that they can use mount command in Rails where you can mount random rack-apps in your routes, even though most people probably do use it with some of their gems, here it is with Sidekiq:

    mount Sidekiq::Web => '/sidekiq'

Why then don't you use the classic style without any initialization?

In my opinion, Rails is only really great if you have a very tight correspondence between your routes, your models, their JSON, and the database tables. When you start separating those apart, that's when you start to suffer a lot with rails.

Why don't you have a tight correspondence, though? Is it something intrinsic in the problem space, or you just want to do it the way you think of it?

In a number of cases, I've found that the Rails Way was smarter than my original assumptions.

... because not everything needs it?

If you’re mostly writing CRUD apps, you’re correct. But the world is not all CRUD, even if you’re only looking at the web space.

So at some point, you have to ask yourself if the headache of working around something Rails doesn't do well is worth throwing Rails out completely over. Can it be tossed off to a microservice and routed away in nginx or whatever? Does it really not suit the Active Record pattern?

If you need a screwdriver, then stop complaining about how much your hammer sucks and get a screwdriver instead. Or ask yourself if a nail could work in the same situation.

Yeah, that was my point.

So what exactly are those correspondences that don't match? Because in a decade of web dev I have not really seen a satisfactory counterexample. Fundamentally everything lives in a request-response cycle and almost everything you load comes from a DB or as a file asset.

Curious why you think that, or why that problem is specific to Rails. I've worked on some massive rails apps, and never experienced that pain. Or, at least not any more pain than any other large app.

I’m curious why you wouldn’t agree. Rails makes a ton of assumptions about how tables, models, controllers, and routes all line up together. Overriding that or removing part of it can get painful.

I don’t think it’s a problem, nor do I think it’s specific to rails. All technologies work best for the cases they were designed for, that’s how design works. Rails was designed for CRUD apps, and it’s brilliant at that. The trick is knowning when you’re not making a CRUD app.

> Rails makes a ton of assumptions about how tables, models, controllers, and routes all line up together.

Rails assumes some very basic, predictable, and consistent naming conventions by default. Beyond that, it has virtually no assumptions about what data feeds into a particular route or controller.

For instance, if I have a controller named 'dashboard#index', at /dashboard, that controller doesn't care if I have a Dashboard model, or UserDashboard, or if I have a 'dashboards' table in my db, or a collection of 50 other random models that all combine to create a dashboard on the fly. If I have a 'people#update' route, it makes no assumptions that I have a Person or People model or db table. I can put whatever I want into that controller action with zero penalty.

Generally speaking I am going to align my controller, model, and route names, but that's more for readability and maintainability concerns, not for the benefit of the framework.

In an app, I can import a picture either through attaching an image or pasting a URL in a field on the UI. That URL field does not exist in the DB, but in order to benefit from ActionView's helpers, I had to declare the field on my model, even though the value in that field never goes to the model. The controller takes the field's value and does the right thing with it.

This is a very simple example of a common task that the separation between UI and backend fails.

Obviously I have no insight into your implementation, but I"m not sure why you would "have to" declare the field on your model. There are probably a lot of different ways handle this situation. For instance, you probably don't have to use the ActionView helpers. There's nothing preventing your controller and view from just passing non-model parameters back and forth.

Rails has never made the promise of giving you every a solution for every situation in the wild right out of the box, and I don't understand why people seem to think it does. No framework could ever do that. What it does is abstract away the common stuff that makes up probably 95% of most web apps, and gives you the ability to roll the rest on your own.

Sinatra is not an "approach to web apps" -- it's a single purpose tool that maps HTTP requests to Ruby. Like Flask or Web.py (Python) and Express (JS).

You are comparing apples to oranges.

And you're saying oranges aren't fruit because they aren't like apples.

It's actually a not-insignificant amount of code that creates an approach so cleverly transparent that you are able to interpret it as no approach at all.

"oranges" is a very narrow range of colours between 585 and 620 nm wavelength. I'm not surprised wikipedia has a page on that https://en.wikipedia.org/wiki/Shades_of_orange

Sinatra is the DSL/object model. Rack is the http request mapper.

No, that's Rack.

Let's not forget merb which Rails got merged into (or the other way round).