Optimize memory when using Rails

We always think what is optimal is needed, memory is of course optimized, if less memory, then our application will run faster, have more resources to handle the work. Another case ..., so the more optimizations the better.

Now we have a question that optimize memory is necessary , good or not?

If you want to optimize, you should to be sure and can answer the following questions:

• Why do you really need to optimize ? Is it really a matter of slowing down the app, or is it just speculation? If it's just speculation, it's best not to do anything.
• Do you know the basic of problems ? If you do not see or find it because otherwise, you will only guess, but chances are your guess is wrong.
• Do you have any way to solve out that problems ? It can be solved by another method, you think or think broader. For the sake of avoiding complex optimizations, in order to solve memory problems, it is entirely possible to create a worker on the server that restarts any process Ruby that begins to use too much memory.

And when you've asked yourself and found that there's no other way, then that's when you need to do the optimization. But before optimization you need to set the metrics.

When optimizing, we need numbers to evaluate our optimal work, and now it's time to set up the metrics.

You need to measure memory usage after tens and hundreds of requests, a request is not enough because the memory allocation of Ruby is not really good. You can use script to measure the total memory usage of your Rails process after 30 requests.

Once you have measured the memory usage of your application, you first need to change the parameters garbage collectionbefore changing the application code.

You can change the frequency with which Ruby reclaims blank memory by resetting a series of environment variables. The variables below are their default values, and the lower bound is for Ruby 2.2.0

RUBY_GC_HEAP_FREE_SLOTS=4096              #           Must be > 0
RUBY_GC_HEAP_INIT_SLOTS=10000             #           Must be > 0
RUBY_GC_HEAP_GROWTH_FACTOR=1.8            #           Must be > 1.0
RUBY_GC_HEAP_GROWTH_MAX_SLOTS=0           # Disabled; Must be > 0
RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR=2.0   #           Must be > 0
RUBY_GC_MALLOC_LIMIT=16777216             # 16 MiB;   Must be > 0
RUBY_GC_MALLOC_LIMIT_MAX=33554432         # 32 MiB;   Must be > 0
RUBY_GC_MALLOC_LIMIT_GROWTH_FACTOR=1.4    #           Must be > 1.0
RUBY_GC_OLDMALLOC_LIMIT=16777216          # 16 MiB;   Must be > 0
RUBY_GC_OLDMALLOC_LIMIT_MAX=134217728     # 128 MiB;  Must be > 0
RUBY_GC_OLDMALLOC_LIMIT_GROWTH_FACTOR=1.2 #           Must be > 1.0

When we set the values smaller then Ruby will trigger the GC more often. But how should we change it?

2 values RUBY_GC_HEAP_GROWTH_FACTORand RUBY_GC_HEAP_GROWTH_MAX_SLOTSwhen dropped will trigger the GC run more often.

You can also lower Ruby's ability to allocate off-heapas much memory as Ruby runs the GC.

#with minor GC
RUBY_GC_MALLOC_LIMIT=4000100
RUBY_GC_MALLOC_LIMIT_MAX=16000100
RUBY_GC_MALLOC_LIMIT_GROWTH_FACTOR=1.1
#with full GC
RUBY_GC_OLDMALLOC_LIMIT=16000100
RUBY_GC_OLDMALLOC_LIMIT_MAX=16000100

However, decreasing the value to activate the GC runs much more often when it can slow down your application or increase memory usage, so before deciding to change, we need to double check the settings. Compile the data by re-measuring the parameters after the change.

If changing the GC does not reduce the memory usage of the application, you should change the code in the application. But to find the code that allocates memory is quite time consuming. But memory_profiler gem will show you where your code allocates memory or you can count where your object allocates with gem stackprof.

Here I will give some suggestions when changing code to optimize memory usage.

In Rails, the ActiveRecord object uses very greedy memory. If you only use about 30 ActiveRecord objects, it does not matter much, but if it's 300 or 3000, it's really a lot of memory. So the most useful is to use pluckto avoid using ActiveRecord objects.

# This uses WAY more memory (and time)...
User.select(:id).map(&:id)

# ...than this!
User.pluck(:id)

User.pluck(:id, :name, :email)
# [
#   [123, "Jonh",   "jonh@example.com"],
#   [124, "Rna",   "rina@example.com"],
#   [125, "Jacson", "jacson@example.com"]
# ]

Although it's used pluck to limit memory over usage, ActiveRecord but if your data set is too large, you should avoid loading data from all your records at the same time. We can browse through them all without keeping all the results in memory by using find_each:

User.find_each.lazy.map(&:some_calculation_in_ruby).reduce(:+)

find_each It helps to load records into a 1000-record set, then every 1000 records are processed so that it can be collected by the GC. lazy Make sure map not to create a large array.

If you need to preload association to avoid n + 1 queries, you can use:

User.includes(:posts).find_each.lazy
# Every user will have user.posts loaded before

One more thing, you should not ignore the minor modifications if it does not cause a lot of memory loss, as if you improve a lot of that, your application will dramatically reduce that memory. I have an example for how minor modifications affect

#this code will be created alots object string
1_000_000.times { "some string or object" }

#this code is better cos use only one object
my_invariant = "some string or object"
1_000_000.times { my_invariant }

#or you can use like this
1_000_000.times {"some string or object".freeze}