Enabling a Large Number of Image Processing on Rails Server! Understanding Memory Overflow Solutions Using Semaphores and Parallel Processing
The following is a tech blog I wrote while working at CyberOwl, Inc. The original version, written in Japanese, can be found here.
I was in charge of implementing the image hosting system at CyberOwl. During this process, it was discovered that when the Ruby on Rails server performed a large number of image edits simultaneously, it led to a memory overflow. To address this issue, semaphores were utilised. In addition to discussing this solution, this article compiles the necessary knowledge about processes, threads, semaphores, mutex, and how the Rails web server puma handles multiple requests.
Contents:
Program execution
Process and Thread
Process:
A process is an instance of a program that is being executed and is allocated independent resources. These resources include memory space. When multiple processes are created, the program within a process cannot access the memory or other resources of another process.
Thread:
A thread is a lightweight unit of execution that operates within a process. Threads within the same process run concurrently, and during this time, they share resources such as the memory space of that process.
Puma's Implementation of Processes and Threads
When creating a web server using puma, you can choose between Single mode, which uses one process, and Cluster mode, which utilises multiple processes.
Single Mode:
In Single mode, a thread pool is created within a single process. You can set both the upper and lower limits of thread count, and scale the number according to the traffic.
Cluster Mode:
In Cluster mode, multiple processes are used, and to enable this, Master and Worker processes are created.
Master Process (master process):
The Master process is the primary process from which Worker processes are forked. While the Master process does not handle requests directly, it forks Worker processes to handle them. After the Worker processes are created, the Master process has the role of monitoring them.
Worker Process (worker process):
These processes are forked from the Master process. In Ruby on Rails using puma.rb, there is one Master process and multiple Worker processes created. These Worker processes run the application and each has multiple threads.
Implementing a Rails Application in a Container Environment
The service we're implementing this time will be run on AWS's ECS (Elastic Container Service). In environments like ECS, it's possible to create multiple containers and perform load balancing, so there's no need to create more than two worker processes for a Rails application. Therefore, we use Single mode and handle requests with a thread pool.
Behaviour of Rails When Receiving Multiple Simultaneous Requests
In Puma's Single mode, when Rails receives multiple requests, it processes them using multiple threads. Puma maintains a thread pool, and each time a request is received, it identifies an available thread from the pool and assigns the processing to that thread. If there are other requests during the processing of one request, they are handled simultaneously using other available threads. Since threads have the necessary environment to execute application code, they can process multiple requests in parallel.
Exclusive Processing
Semaphore:
A semaphore is one method of exclusive processing. It limits how many programs can use a shared resource (such as memory) at the same time. When using a semaphore, the implementation is as follows:
This allows for setting a limit on the number of simultaneous resource accesses.
Mutex:
A mutex is another method of exclusive processing, similar to a semaphore. It's a technique used to ensure that when one task is using a particular shared resource, other tasks are prevented from accessing that resource. In other words, it behaves similarly to a semaphore that can only have the values 0 or 1.
Limiting the Number of Concurrent Image Processing Operations Using Semaphores
One example of a process that consumes memory is image editing. When a server handling image editing receives multiple requests simultaneously, it processes several images at the same time. This can consume a large amount of memory and potentially cause a memory overflow. Therefore, we used semaphores to set a limit on the number of image editing operations that can be executed concurrently.
Introducing Concurrent::Semaphore
When using a semaphore within the same process, Concurrent::Semaphore is utilised. For instance, if you want to limit the number of concurrent image processing operations to four, the semaphore count is set to four.
config/puma.rb
$semaphore = Concurrent::Semaphore.new(4)
app/controllers/images_controller.rb
def image_editing
$semaphore.acquire # Decrement semaphore by 1, wait here when semaphore count is 0
#------------------------------
#
#Image editing logic
#
#------------------------------
$semaphore.release # Increment semaphore by 1
end
Points to Note During Implementation
Mutexes and semaphores can be defined in Rails' application.rb, models, controllers, etc., but when it is necessary to restrict access from all threads, they must be defined before the server is set up (such as in puma.rb). This is because if a semaphore is defined in application.rb, models, controllers, etc., an instance of it is created for each request, making it unsuitable for defining a global variable to be used by all threads within a process.
Additionally, when using mutexes or semaphores, there is a risk of causing a deadlock, where the process or threads stop progressing because they are waiting to acquire resources occupied by each other. Therefore, when controlling two or more shared resources, attention must be paid to the order of requests.
Furthermore, if there is a need to use multiple processes, just limiting access to shared resources among threads using Concurrent::Semaphore may not be sufficient. In that case, to restrict access to shared resources from applications in all processes, it is necessary to introduce redis and create a semaphore there.