When it comes to web server software, we hear several names. J2EE (based on Java), Ruby on Rails (based on Ruby), .NET, LAMP (based on PHP) and Django (based on Python) are some of the most popular names on the list. However, in the last 5-6 years, another name has been growing in popularity rapidly. That’s NodeJS.
If you are new to web technology, the concept of why NodeJS is a gamechanger might be lost on you. And if you are from the world of Java or .NET, it will take some time to think of solutions in the NodeJS way. That is why for this article, I got hold of NodeJS ‘herself’ for an interview. Since a lot of my work for my clients are taken care of by NodeJS, I know ‘her’ personally and ‘she’ was happy to sit down with me for a discussion on why ‘she’ is set to take on a lion’s share of the web.
Me: Hello NodeJS, it is so nice to have you here for this interview. I was hoping to do this piece for a long time.
NodeJS: Hey, Hari. Thank you so much for having me here. It will be nice to help the world understand the work that I and my friends are doing. It has been such as long journey since I started working at my first job at Google Chrome browser!
Me: Hmm, yes, Chrome indeed. We will come back to that in a while. But first tell me exactly who you are and what your role is in the world of web.
- Validates whatever the user enters in web pages that need data entry and provides error messages where necessary. Login pages are the most common example.
- Fetches data from a web server and shows that data in the currently visible web page. E.g. When you type a search term in Google Search, it automatically fetches suggestions and shows them to you in a dropdown.
Me: Okay, that explains the learning curve side of the equation. A single language for both sides helps the browser team learn server skills quickly and vice-versa. But how you been able to stand out as a performer? You work so fast, yet require so little memory.
NodeJS: Hmm, let me correct you. I do not necessarily work faster than other web server solutions like J2EE or LAMP. I am just more responsive and hence appear fast. I told you before, I am able to prioritise what to do NOW, vis-a-vis what to defer for later. This helps me get things done in a more streamlined way with less resources on hand and that appears faster.
Me: That’s geek speak. Please explain.
NodeJS: Okay, let us imagine a sandwich outlet with a self-service counter. Let’s say 50 customers are waiting for their turn to order a sandwich each and eat it. Now how can you serve 50 people in a single sandwich shop? Let us see how Java, .NET, PHP or Django do it. When a customer approaches the counter, he/she is asked to place his/her order. Now here’s the deal. Once the order is placed, the customer is made to stand by the counter and wait, while his/her order is baking away in the oven. As long as this customer’s order is not ready, he/she just stands there and the rest of the customers still wait in the queue. So how do you serve those waiting customers while one customer’s order is cooking away. Well, the above four platforms believe in becoming more responsive by adding more counters and hiring more people to take orders. If you want to serve 25 customers at a time, well, construct 25 counters and hire 25 people to man the counters. However, the basic principle is the same. Once an order is placed and the sandwich is in the grill, the counter is occupied and another customer has to wait until the counter is free. It is only when a customer’s food is ready and served, that the counter person is ready to take the order from another waiting customer. This can lead to dramatic delays during peak hours when even 25 counters may seem like too little. This is especially frustrating for customers who are just looking for plain jam sandwich without toasting the bread. The other caveat is that adding more counters will require more space and resources.
Let’s analyse this a bit. You have to note that the true bottleneck is actually grilling the sandwich. Other customers shouldn’t have to wait to place their orders just because one customer’s sandwich is in the oven. Why can’t we change the model a little bit. We take orders in rapid succession, give each customer a token number. When an order is ready, we call out the number. This way customers will see an immediate response from the store. Their order is taken almost as soon as they enter or at least within 5 minutes. Customers with really big orders such as a beef grill sandwich will have to wait for the grill to do the job, but customers who ordered jam sandwich can get their food within no time. This only seems fair.
This is what makes us different. PHP, Java and the like create more ‘counters’ (techies call them threads) to take in more requests during peak time, but that needs a lot of memory on the server. But each thread can only serve one web request at a time and cannot serve any other request. My team and I do differently. We take in as many requests as we get and then ask those requests to wait until we call them back with their responses. You have to realise that bottlenecks in computing are generally in the form of reading from hard disks or getting data from across a network. These operations are like grilling and can keep a request waiting. If we accept all requests as soon as they are made, then we can immediately satisfy those requests which are expecting shorter responses.
E.g. Request 1 asks for a video, request 2 asks for an image and request 3 for text data. In the traditional model, the thread serving request 1 is blocked for sometime while it waits for the entire video to load from the disk. If there were only a single thread, then request 3 would have to wait for way too long even though it needs only text. So the server would have to create 3 threads to serve each request in a timely manner and take up more memory on the server. It is such as waste to see thread 1 hogging memory lazily while simply waiting for something to load from a hard disk. That thread can be put to much better use. This is what we do here. We have only one thread, the equivalent of one counter. The job of that thread is to take in as many requests as it can in rapid succession and ask
them to wait until their responses are ready. Short responses like text can be done immediately, while the longer ones like video will take more time. But the request thread is still accepting each request and ready for requests 4, 5, 6, 7… 20.. 100 and staying super responsive. That is why we appear to run fast. We optimise the waiting time by taking in more requests and doing all the smaller jobs, finishing them off and checking them off our list. And yes, we need only one counter, so we need much less memory.
Me: That was quite insightful. But tell me, when you have only one counter, isn’t there a chance that there is a single point of failure?
NodeJS: That is a very good point. Yes, indeed our system is a single point of failure. We do enlist other systems to make sure that our counter is repaired speedily. But remember that we are only suitable for a certain type of application and we don’t claim to be a silver bullet. As I told you, I am good at manning the counter and prioritising what is to be done immediately and then pushing the heavy lifting to someone in the back office. Remember the sandwich shop? The counter person simply takes orders rapidly, but does not make the sandwich himself / herself. If that were so, he / she cannot handle orders so fast. The chefs behind the scenes in the kitchen are the ones who make the sandwich and notify the counter person when it is ready. The counter person in turn notifies the customer. But take the case of an immigration office or the airport check-in. At these places, the person at the counter does all the work him/herself, constantly talking to the passenger, be it the creation of the boarding pass or the checking of documents or the entry stamping. The counter person does not push the request to be executed by someone else at a back office and then move onto the next passenger. In such situations, you cannot perform with a single counter and would need as many counters as necessary.
It’s the same with web requests. There are certain types of web requests where the response needs to be computed on the fly, like for instance, a Google search request. It is only when a person types in a search term does Google begin to compute the results. There is no way of having results computed and stored beforehand since the user can practically type anything. Google performs a rather expensive computation on the fly: matching keywords, finding matching pages, going through page rankings, sorting the pages and building a response. We are not a good fit for such a web request.
Me: Despite this serious flaw, your list of adopters keeps increasing by the hour!
NodeJS: Well, 90-95% of the web requests fit our model. When the request comes in, most of the data reside on slow media like hard disks and other networks. So we can push the hard work to the hard disk or the Internet and immediately take in other requests. Only 5-10% of requests on the Internet ever need expensive computation. Take Google’s own example. Google Search is the only application that needs computation on the fly. Look at their other apps. GMail, Google Photos, Google Drive, Youtube, for instance. All of these are simply storing or retrieving some kind of data (Emails, photos, documents and videos) stored somewhere on Google’s hard disks. The amount of computation and calculation is minimal. Ditto for Facebook. They store a humongous list of status updates in their cloud. When you request your Timeline, all they do is retrieve status updates from all your friends. They are not computing anything much expensive on the fly. Twitter: similar thing. Instagram: just a collection of photos. Pinterest: the same. Same story for Trello and Asana. The top websites on the Internet are based on retrieving data from relatively slow media, during which the web server is only waiting. This situation is tailor-made for our “one customer-facing counter with a busy backoffice” kind of work. We are always keeping more customers happy by taking in all the orders and servicing the smaller orders rapidly and getting them out of the way.
Me: That is indeed a strong case for your success. But haven’t others ever tried building the type of system that you have?
NodeJS: Of course they did. We are hardly the first system that works on this model of rapid intake and staying responsive. Many did this before we did. The most popular one that comes to mind is the Twisted framework built for Python language programmers. Only, they were way too ahead of their time, when the world was not yet ready for this model. Companies at that time were big corporations and for them, building more counters was a matter of buying more servers, more powerful machines and more memory and processing power. They were cash-rich and it was boom time. Secondly, Python was not a popular programming language during that time and it was too much of a learning curve for the tech teams to switch.
Me: NodeJS, it has been a pleasure talking to you and learning about how you operate and what you bring to the world of web. Keep up the great work and wishing you all the best.
NodeJS: Thank you so much Hari, and I hope you have a wonderful experience working with us on your future projects. All the best!