Improvement: better memory usage in dry-monitor


After testing a dry-web-roda app with the gem derailed_benchmarks I found that my app was using more memory than a bigger Rails app.

Actually, this is the first part of the output:

$ bundle exec derailed bundle:mem
TOP: 42.4297 MiB
  dry-web: 13.6484 MiB
    dry/web/container: 13.6484 MiB
      dry/monitor: 13.4023 MiB
        dry/monitor/sql/logger: 12.9414 MiB
          rouge: 12.9414 MiB

That means, that the app is using 42 MiB, and the dry-web component is the biggest one, that is using 13.6 MiB, because of rouge (12.9 MiB).

So, requiring only the needed components of rouge, by changing in lib/dry/monitor/sql/logger.rb the line 2:

require 'rouge'


require 'rouge/util'
require 'rouge/token'
require 'rouge/theme'
require 'rouge/themes/gruvbox'
require 'rouge/formatter'
require 'rouge/formatters/terminal256'
require 'rouge/lexer'
require 'rouge/regex_lexer'
require 'rouge/lexers/sql'

We can get an improvement. The mem test after that returns:

$ bundle exec derailed bundle:mem
TOP: 26.1211 MiB
  rom: 8.0352 MiB
  dry-web: 1.1211 MiB
    dry/web/container: 1.1211 MiB
      dry/monitor: 0.75 MiB
        dry/monitor/sql/logger: 0.5 MiB

So, the memory used by dry-monitor was reduced from 13.4 MiB to 0.75 Mib. And the impact on the memory usage of a small dry-web-roda app was bigger than that: from 42.4297 MiB to 26.1211 MiB.

I did not analyze how derailed_benchmarks make its maths. But it’s clear that rouge has a lot of cool components that are not used by dry-monitor. Then, Why do not get rid of it?

Please, let me know what you think about this. Thank you.


Oh wow, I did not expect that. I’m all for limiting requires to what you actually need. In fact, I’m a strong believer that all gems should be designed in a way that they always provide cherry-pickable components.

Would you mind sending a PR to dry-monitor? :slight_smile:


@solnic also, I checked my job project for object allocation and found interesting things:

$ bundle exec derailed bundle:objects

Measuring objects created by gems in groups [:default, "production"]
Total allocated: 91115563 bytes (800628 objects)
Total retained:  15093944 bytes (134684 objects)

allocated memory by gem
  10664845  rouge-2.2.1
   8722637  ruby-2.4.2/lib
   8059914  newrelic_rpm-
   6488996  mime-types-3.1
   4489137  dry-initializer-2.3.0

As you can see, rouge allocate more objects raise than other dependencies. And also, this dependency needs only for color output. Sometimes it can break production (when logger services doesn’t support the color format, for example).

That’s why I have a question: WDYT if we make rouge dependency as optional? I mean if a user wants to use rouge they can install the gem and that’s all. Also, a user gets control on color/not color logs for any environments.

Something like this (check logger file):

  require 'rouge'

  module Dry
    module Monitor

      module SQL
        class Logger
          # use rouge lexer and other stuff
rescue LoadError
  module Dry
    module Monitor

      module SQL
        class Logger
          # logger without color formatter


Yeah, it should be optional. I just wanted to have colors and put it there by default. Logger is configurable so it is very easy to add config setting for disabling colors.


Quick update, I reported an issue about it I also discovered that I actually made colorful output configurable already :sweat_smile: