Friday, August 30, 2024

Why use SQLite?

SQLite is optimized for situations where the workload is predominantly reads with occasional writes, making it ideal for small to medium-sized applications. SQLite is very efficient and can operate comfortably within a few megabytes of RAM.

For highly concurrent environments where many users need to write to the database simultaneously, SQLite might not be the best choice, and a more robust client-server database like MySQL or PostgreSQL would be more appropriate. Recommended RAM for light use for MySQL is 1GB and for PostgreSQL 2GB, which is about 1000 times more than SQLite.

SQLite is an embedded database that runs in the same process as the application using it. This means it doesn’t require a separate server process to manage database connections, unlike MySQL, which operates as a server and listens for incoming connections on a specific port (e.g. 3306). SQLite databases are stored as files on the local filesystem. When an application wants to interact with an SQLite database, it directly accesses the database file without needing to communicate over a network. So with SQLite, if you run multiple apps, there won't be problems like port conflicts or the need for containers.

SQLite doesn't require authentication mechanisms like usernames and passwords. The database file is accessed directly without any user credentials.

If you want to use a cheap (5$/month) VPS with 512GB RAM, SQLite is your only option. It is not just a toy, it is used in popular apps like nomadlist, see Pieter Levels video. You can optimize SQLite even further.

Music: Mike Oldfield - Sentinel

Friday, August 23, 2024

Web server vs database server scaling

Every user interaction with a website, such as loading a page, clicking a button, or submitting a form, generates a request to the web server. These interactions are frequent and often involve static assets (like images, CSS, and JavaScript files), rendering HTML, and processing logic. As the number of users increases, the web server has to handle a rapidly growing number of these requests.

While many web requests may involve querying the database, they don't always result in database interactions. For example, pages with static content, content cached in memory, or content generated by the web server without requiring a database query won’t proportionally increase the database load. Additionally, many web requests might use the same data that can be cached on the web server, reducing the number of database queries.

Only dynamic content that requires data retrieval or storage will result in database queries. Since not all web interactions require new data from the database, the database load doesn't increase as rapidly as the web server load.

Initially, it's easier and more effective to scale web servers horizontally to handle increased traffic. The database server can handle a significant amount of load on its own due to its ability to manage data consistency, and because you can optimize performance through caching and other techniques. Once the traffic and data operations reach a certain threshold, you will need to consider scaling your database server as well.

Thursday, August 22, 2024

Adventures with PHP & Laravel

I am updating a demo chat app written with the Laravel framework using Reverb. I must say, it takes some getting used to, as a lot of things happen behind the scenes, using certain naming conventions and strings, and you are not able to follow it with ctrl+click from PhpStorm. Laravel uses Convention over Configuration, i.e. it expects you to follow certain naming conventions.

To run the app locally, you have to open 4 terminals and run:
  1. php artisan serve starts a built-in development server for your Laravel application. It's a convenient way to test and develop your application locally without the need for a full-fledged web server.
  2. php artisan queue:listen initiates a worker process to listen for and process jobs in your Laravel application's queue. It's crucial for background tasks, email sending, and other time-consuming operations.
  3. php artisan reverb:start initiates the Laravel Reverb WebSocket server, enabling real-time communication between your Laravel application and clients.
  4. npm run build optimizes the code for performance, minifies it, and bundles it into a single file or multiple files, making it suitable for deployment to a web server.
Passing a variable from controller to blade view:
Controller passing $users to view:
$users = User::where('id', '!=', Auth::id())->get();
return view('proceed', compact('users'));
Using $users in view:
<select id="userSelect" class="form-control">
  <option value="" disabled selected>Select user...</option>
  @foreach($users as $user)
    <option value="{{ $user->id }}">{{ $user->name }}</option>
  @endforeach
</select>

Naming conventions are very important. For example, if you want to use parameters in routes/channels.php, you have to use the following syntax:
Broadcast::channel('channelBetweenUsers.{user1_id}.{user2_id}', function ($user, $user1_id, $user2_id) {
    return (int) $user->id === (int) $user1_id || (int) $user->id === (int) $user2_id;    
});
The call order is broadcastOn(), new PrivateChannel("channelName") → routes/channels.php, Broadcast::channel("channelName"). If there is no matching channelName in routes/channels.php, the communication will fail.

The $user1_id, and $user2_id parameters are passed by broadcastOn() in new PrivateChannel call parameter. $user represents the authenticated user and is always available. The other parameters, $user1_id and $user2_id, are extracted from the channel name and passed to the callback function in Broadcast::channel(...). 

Laravel accessor methods allow you to define logic that will be executed when an attribute is accessed, enabling you to perform calculations, transformations, or other operations before returning the value. PhpStorm will mark them with "no usages".

Frontend (with React) rendering call order: HomeController.php → home.blade.php → app.blade.php → app.js → Main.jsx → ChatBox.jsx → Message.jsx, MessageInput.jsx

Middleware is a mechanism to intercept and modify HTTP requests before they reach the application's routes and controllers. It provides a flexible way to implement cross-cutting concerns like authentication, authorization, rate limiting, logging.

The __() function is a Laravel helper function used for translating text. It looks up the given key (e.g. {{ __('Logout') }}) in the localization files and returns the appropriate translation based on the current locale, e.g. 'Çıkış'.

When you want to execute a job sending and receiving messages asynchronously, you can use queues. A queue driver acts as a bridge between your Laravel application and the underlying queue system. It handles the communication and management of tasks or jobs that are placed on the queue. A queue worker is a process that continuously monitors the queue for new jobs. Once a job is available, it processes the job and removes it from the queue. The simplest way is to use the database as a queue driver. However, database operations, especially writes, can be relatively slow. Using a database as a queue driver could introduce noticeable delays in message delivery, especially as the number of users and messages increases. But until you pass 100 users messaging each other in the same time frame (concurrent), a database can handle it.

For higher concurrent messaging, you can use Redis as your queue driver, which is an in-memory data store that excels at handling high-throughput, low-latency tasks. But it adds to setup complexity because you have install the Redis server and configure it (hostname, port, authentication). You still need to issue php artisan queue:listen command to connect the Redis queue to your app.

While RabbitMQ can also be used for chat applications, it might introduce unnecessary complexity and overhead, especially for smaller-scale chat apps. Redis' focus on pub/sub and in-memory storage makes it a more suitable choice for this particular use case.

Messaging steps:
  1. Client1 sends a message to Client2 via POST request → web server → router → controller
  2. Controller adds message to queue via SendMessage::dispatch($message)
  3. Queue driver stores the job.
  4. A queue worker gets the job from the stored job queue (via polling for database or polling/push notification for Redis).
  5. Worker processes job via SendMessage::handle(), obtaining the WebSocket channel via GotMessage::dispatch() → constructor → broadcastOn()
  6. The worker broadcasts the message to all connected clients via WebSocket channel. The worker maintains a persistent WebSocket connection to the Reverb server, allowing it to broadcast messages efficiently, avoid unnecessary routing through the controller.
  7. Client 2 receives the message via WebSocket.
Traits vs Interface: Traits provide reusable methods and properties, the class using the trait does not have to call or implement it. Interfaces define a contract for method signatures that must be implemented. Traits help overcome PHP's single inheritance limitation by allowing code reuse across unrelated classes.

Queues vs Threads: Queues handle background tasks that do not need immediate execution (e.g., sending notifications or processing data). Threads are more suitable for scenarios requiring simultaneous execution of tasks that are tightly coupled or need to share state. PHP does not have built-in support for multithreading, it can utilize extensions such as parallel for multithreaded capabilities.

PHP constructor property promotion:
class MyClass {
    public function __construct(public int $val) {
        // No need to manually assign $this->val = $val;
    }
}
$obj = new MyClass(5);
echo $obj->val; // Output: 5
Queue workers vs defer(): defer() does not guarantee that messages will be processed in order or without conflicts, especially if multiple users are sending messages simultaneously. Queue workers offer better reliability for handling failed jobs, retries, and monitoring. While defer() might seem like a quick solution, utilizing queue workers is the recommended approach for handling message sending and receiving in a Laravel Reverb chat application to ensure optimal performance and responsiveness.

php artisan down: Puts the application into maintenance mode, all incoming HTTP requests will receive a 503 "Service Unavailable" response, and a maintenance mode page will be displayed. This is useful when you need to perform maintenance or updates on your Laravel application and you want to prevent users from accessing the site during that time. To take the application out of maintenance mode, you can use the php artisan up command.

Sunday, August 4, 2024

Process models in web apps and mobile/desktop apps

Web applications use a different process model than mobile/desktop applications. Mobile/desktop apps have a process running until the user exits the app, they are typically stateful, meaning the application maintains its state as long as it is running. In contrast, web apps are generally stateless, meaning each request is independent, and the server does not maintain the state between requests. The server processes a request, sends a response, and then terminates the process handling that request. Multiple users accessing a web app will create multiple independent processes on the server. Web apps are designed to handle many short-lived processes. A web process is typically terminated after 300s (set in web server config) if it has not already sent a response.

REST (Representational State Transfer) is designed to be stateless. This means that each HTTP request from a client to a server must contain all the information needed to understand and process the request. The web server handles each request separately, delegating it to PHP, which terminates the process after the response is sent back to the client. This approach ensures scalability and efficiency, as the server can handle many requests from different users simultaneously. Scalability is achieved through load balancing and distributing requests across multiple servers or processes.

  1. Request-Response Cycle: Each time a user interacts with your web app (e.g., by clicking on a product), the browser sends an HTTP request to the server.
  2. Single Process Execution: When the server receives a request, it starts a new process or thread to handle that specific request. This process runs your PHP code, generates the necessary response (usually HTML), and sends it back to the browser.
  3. End of Process: Once the response is sent back to the client, the PHP process or thread is terminated by the web server. The server does not maintain any long-running processes for individual users between requests.
  4. No Inter-process Communication: Each request is handled independently, and PHP does not keep any information about previous requests in memory. Any necessary state information (like user sessions) is typically managed using session variables, cookies, or other storage mechanisms like databases or caching systems.
  5. Session Management: To maintain state (like user login status, shopping cart contents, etc.), PHP uses session management. When a user clicks on a different product, the session data is used to retain the necessary information across multiple requests. When a session is started in PHP (using session_start()), PHP generates a unique session ID for that session. This session ID is then stored in a cookie on the client's browser. The actual session data (such as user information, preferences, etc.) is stored on the server, typically in files or a database. The session data is associated with the session ID. When the client makes subsequent requests to the server, the session ID stored in the cookie is sent along with the request. PHP uses this session ID to retrieve the corresponding session data from the server.

Note that PHP-FPM (PHP FastCGI Process Manager) uses FastCGI. Unlike CGI, which creates a new process for each request, FastCGI keeps processes alive and reuses them for multiple requests. This avoids the overhead of process creation and teardown. PHP-FPM runs a master process responsible for managing the worker processes. These processes are persistent and handle multiple requests over their lifetime. Each incoming request is assigned to an available worker process. After completing a request, the worker process becomes idle and waits for the next request.

Note about cPanel NPROC (nb. of processes): The NPROC limit in cPanel refers to the maximum number of processes that a single user can have running simultaneously. This includes all types of processes, not just web server processes (e.g., background scripts, cron jobs, etc.). While this can indirectly impact the number of concurrent connections your web server can handle, it doesn't directly translate to the number of concurrent connections. Your server might use a threaded model (like Apache's worker MPM or Nginx), where each process can handle multiple connections through threads. The NPROC limit then becomes less directly tied to the number of concurrent connections.

Music: Legendary Pakistani Singer goes Metal [Sanson Ki Mala Pe]