You have probably seen ways in which the physical world represents digital data in print. The one you are most familiar with is perhaps bar codes. You see them on products at a shopping mall. The checkout counter scans them and fetches the information about the product from a database. Even books have their ISBN codes printed in a bar code.
Over the last decade, another technology named QR or Quick Response code has gained popularity. While traditional bar codes are 1-dimensional and can represent at best between 16 – 20 characters of data, QR falls under a group of data encoding technology called the ‘data-matrix code’ or colloquially, ‘2-dimensional bar codes’. QR codes were pioneered by the Japanese company, Denso wave.
QR codes are capable of holding 7089 characters of data. You can see QR codes all over the Internet, on packaging, on information booklets and even on branding boards at shops. They usually serve as a way to encode URLs so that the user can get further information about a product or an organisation. But URLs are not the only thing possible inside a QR code. Let’s see how they work. Continue reading “How QR codes work”
As one of the latest catchwords in the world of software, DevOps is a term that has been growing in popularity over the last decade. Is it yet another meaningless jargon which will fade away when the fashion wears off (Blu Ray discs, are you listening?!). Or is it something really useful, practical, helpful and here to stay for generations to come?
The world of software and information technology throws up jargon at an explosive rate today. It is hard to keep up with them and even harder to understand what they mean. One such term is SaaS or Software as a Service. Let me explain what it means and how it is important to you.
What is SaaS?
SaaS refers to a software that MANDATORILY has the following properties.
The core of the service runs over the Internet on a server, while the user works with a bare-bones tool on his/her device.
Usually nothing is installed on the user’s device and a browser is used to connect to the service. Even if something needs to be installed, it is just a small app with a user interface and a set of functions that connect to the Internet to exchange data with the service’s server. No service functionality resides inside the app.
No storage happens on the user’s device. All of the user’s data is stored on the server securely.
The user needs to connect to the Internet and login to an account in order to use the service and access his/her data.
Due to the service being on the Internet, a user can access his/her data from any device from any location, be it on a smartphone from a car seat, on the home desktop computer or on a laptop from a resort.
As a result, changes made from one device are almost immediately available on another device.
Functionality can be extended on a daily basis because all the functional code is on the server. The user doesn’t have to install a new version on his/her device. In case of a web application via a browser, even the UI can be updated frequently without the user having to install anything.
This is contrast to a traditional desktop software application, where a huge software application with UI and all the functionality is installed on the device’s storage, all the files are stored locally on the device’s storage device and the user needs to remember to copy the files to a portable storage unit such as a thumb drive in order to transfer them from one device to another. Changes also need to be synced among devices every time. Changes to functionality are released as a new version of the application and they too need to be installed on the server.
Apart from the above features which are mandatory, Saas MAY provide some of the following features
Limited offline access to use the service when Internet is down.
The ability to download files to local storage.
The ability to connect one SaaS to another in order to use the features / content of one in another. E.g. Pixlr can use your Google Drive as storage.
Ability to share your files with other accounts as read-only or read/write.
Examples of Saas and equivalent desktop applications
The biggest failure of SaaS is when going for long periods of time without Internet. While some SaaSes allow you to edit your content offline, your content will not be synced to the server. So if another person makes changes to the same files from another device, there will be conflicts.
An insecurely set up SaaS can cause a risk where contents can be snooped and even downloaded by unauthorised persons.
All the data is in the hands of the company providing the SaaS and not in your control. A change of policy or an infrastructure collapse can lead to loss of your data.
What you need to build your own SaaS
If you are a company who wants to build your own SaaS, the following should be your technical know-how. There are further requirements such as strong policies and legal agreements which are beyond the scope of this post.
Setting up your own server either in physical form or on a cloud service such as Linode, Rackspace, Amazon Web Services or Google Cloud Compute.
Complete knowledge of HTTP protocol. The app on the device and the server communicate using HTTP.
Securing your HTTP communications using HTTPS.
Programming on the server side using one of NodeJS / Python / Java / Ruby / PHP / .NET.
Using server side file system and databases for data storage.
Using OAuth to validate users who log in and to reject users who are either no signed in or are using invalid credentials.
Using OAuth to allow other apps and users to access a given user’s data.
If you are making platform-specific apps, then here are your requirements.
Java / Kotlin programming for Android
Swift programming for iOS
.NET programming for Windows
Cocoa application programming for MacOS.
QT / GNOME using C++ or Python for Linux.
Usage of Continuous Integration and Continuous Delivery to roll out new versions.
System administration for the following requirements.
Monitoring the server to make sure that things run smoothly.
Frequent backing up so that data can be recovered.
Being the trendsetter of the last decade, SaaS is now commonplace since 2010 or so, when rich web applications, Android and iOS took the world by storm. There was a need to access apps and their data from anywhere to which SaaS provides the solution. And that need and the success of SaaS is not going away anytime soon.
Software programs are like food and clothing. They come in all shapes and sizes. There are billions of programs. But all of them can be classified in a few categories. This post informs you about the different categories of software programs, what can be achieved with each, where they fall short and how you can decide which one you should build, or assemble a team for.
The topic seems very basic, but I feel that companies today are churning out too much code and too many variants of the same program for their own good. As a company, it is worth pausing and asking yourself what you really want to achieve for the users.
In part 1 of this series, we saw what VoIP is and how it is synonymous to traditional phone calls. We also listed some advantages of VoIP over traditional phone networks, that it is possible to set up your own VoIP network for use within your company. Finally, we saw that VoIP calls can be automated. One day your devices, such as home protection or a stock market ticker, will make an audio or video call to you via VoIP if something interesting occurs.
During the 1990s, there were only two ways to transfer voice communication from one place to another. The first type was radio communication, using gadgets like walkie-talkies. The second was telephone networks. At the turn of the century, the improved speed of the Internet was begging for a way to transfer voice in real time from one desktop computer to another. Enter VoIP or Voice over Internet Protocol.
What is VoIP?
VoIP is a way to transfer sound, mainly voice, from one device to another over the Internet in as much real-time as possible, similar to the way radio or phone does. But instead of a radio frequency or a telephone exchange, the medium over which your voice is carried from your device to the recipient’s device is the Internet. Please note that this is not the same as downloading a song in MP3 form and listening to it later. This is like a phone conversation where the voice is streamed from one end to another live.
In a telephone, a connection request is indicated to the recipient by ringing his/her device. A connection is established when the receiver picks up. VoIP too has the concept of connection request and establishment. Just like both the phones have a phone number for unique identity, VoIP devices have VoIP endpoints. These usually look like email addresses. Users connect to each other using endpoints. The act of dialling, ringing, picking up and having a conversation over a phone is known as a phone call. In VoIP, the entire cycle is called a VoIP session. Like calls can be recorded in modern telephony systems, VoIP sessions can be recorded too. And just like conference calls in telephones, VoIP sessions can happen with more than two participants.
In a VoIP call, the two parties use a software app or a dedicated device that connects to a VoIP service provider’s server over Internet. This is similar to the way that your landline telephone is plugged into a wall unit using an RJ11 cable and your mobile phone contains a SIM card that connects to the nearest tower belonging to your provider. The software connection to a VoIP server is established just like your web browser connects to the server of a website or your email software (e.g. Outlook, Thunderbird) connects to an email server. Once connected, the software app can request the server that it wants to talk to another person connected to the service. The server will then send a notification to the call recipient, who can choose to receive the call. The server then manages the path and the streaming of voice between the sender and receiver apps.
Advantages of VoIP
A phone communication requires dedicated hardware. You must use a telephone with a subscribed landline connection or a mobile phone with a SIM card whose service has been enabled.
The cost of a phone call depends on the distance between the participants. There are local, national and international calls all with different tariffs.
Phone networks can only be created and maintained by licensed service providers. You cannot start your own landline or GSM networks. It is possible to set up a local telephone network, also known as an intercom, as long as all the phones are within the same building / locality. But if your office is spread around the country or the globe, you cannot simply start a landline or a mobile network of your own. It would be illegal to. You need to purchase a service from BSNL, Vodafone, Rogers, AT & T, Movistar, etc. depending on where you are.
The phone numbers you receive from phone companies cannot be fenced such that only those from your company can call you. Phone numbers are globally accessible and anyone with your number can call you, whether they work in your organisation or not.
In contrast, VoIP can be used from a laptop, desktop computer and mobile phones with Internet enabled. Even a mobile phone which has no SIM card, but has access to Internet over WiFi is ready for VoIP. All you need is a software app that can connect to the VoIP network of your choice.
Since all the calls are over the Internet, you only pay for your Internet package, whether you speak to your spouse in the next room or to your friend on the other side of the globe.
Also, it is possible to set up your own VoIP network for communication between the multiple offices owned by your company across the world. Based on your access control, the rest of the Internet may be allowed to dial into your VoIP network.
Proprietary VoIP networks
Just like any software solution, VoIP comes in proprietary and open solutions. In proprietary solutions, you need to download the software or purchase the VoIP hardware that is released by the company providing the solution. The dedicated software or hardware automatically connects to the VoIP servers of the solution-providing company, without letting you know where they connect to or without allowing you to set any parameters for the connection. Usually, proprietary solutions allow people to talk using only their software and only to those people who have joined their network and who also use their software. Cross-platform VoIP calls did work for certain companies, but the solutions didn’t last.
Skype (now owned by Microsoft) was one of the first VoIP solutions. They broke ground at the start of the millenium, making VoIP a household name. Other solutions followed over the next decade: Google Talk, Yahoo voice chat, etc. Other than Skype, most other solutions were simply voice add-ons to their chat applications. With the success of Android and iPhone, Viber and FaceTime became some of the earliest VoIP apps available for smart phones. The success of Viber prompted WhatsApp to add audio and video to their otherwise text-only application. Google scrapped their erstwhile Google Talk solution and rebuilt a new solution called Hangouts. Hangouts remains one of the very few VoIP offerings that works purely from a browser and does not need an app. Google have not settled for their robust Hangout app and instead confused the users of their Play Store with another app named Duo. Other companies such as Zoom started their own VoIP solutions mainly with the intention of business conference calls where multiple participants can hold meetings by signing into a pre-determined ‘meeting room’.
All of the above solutions are proprietary and incompatible with each other. Skype cannot talk to WhatsApp, whose users cannot join a Hangout, whose users cannot converse with a user over FaceTime. This dreaded situation is called ‘vendor lock-in’. A user has to either confine him/herself to talk to users who also use their solution. Or they have to download / purchase multiple solutions to include everyone. Sometimes, even that’s not possible. For instance, even if an Android user is ready to pay an arm and a leg, Apple simply will not make an Android FaceTime app.
How will it be if we can use an open VoIP protocol that lets us set up our own VoIP server, and one that can chat with other open VoIP servers. Instead of being tied in to specific software apps from vendors, users can have a choice of apps ranging from basic to advanced, which can be programmed to connect to any VoIP server of their choice. Why not two VoIP servers? One for family and one for work. Can we build an ecosystem of VoIP apps and providers, where a user can simply change from app to app until he / she finds one that is right for him / her and then can simply switch from one VoIP provider within that favourite app to another based on the time of the day, one for casual purposes and one for work?
Yes, we can. That is promise of two protocols named SIP (Session Initiation Protocol) and H.323, which are open protocols. We will learn more details about those protocols in part 2 of the series. In this post, I’ll simply summarise that any SIP-compliant app can connect to any SIP-compliant VoIP provider. The providers of one network may set up their servers such that the contacts and sessions of every user be accessible to other providers.
Radio communications assisted the world during the World Wars. Telephones changed the face of voice communication between the 1950s and 1990s. But VoIP has democratised the way people speak to each other, making voice and video calls available to anyone with a device that has Internet, front-facing camera and a microphone, at a very reasonable price.
With VoIP making its way into the Internet of Things, it may even be possible in the future to speak to your home’s virtual assistant to get things done while you are away. Your home’s security camera may automatically dial you with a VoIP call if it detects something fishy. It remains to be seen how much more advanced this technology will turn out to be.
In the last post, we saw how version control system such as Git helps three poets, i.e. Nancy, Tommy and yourself write a poem and maintain several versions of it in a secure database, so that you can go back to an older version and start over again if you wanted to. We saw how only the changes across each version is saved. We also saw how the poets commit their own changes to a local repository, push them to a central repository and pull others’s changes into their own repositories.
…. thus goes a famous nursery rhyme from our childhood. How is it relevant to version control? In this post, you are going to imagine yourself as an author of the afore-mentioned nursery rhyme, working with a few more colleagues. Using that example, we will see how version control software works.
One of the biggest technological advances in this decade is the usage of machines hosted by server giants like Amazon and Google for our businesses, in the form of AWS and Google Cloud Compute. Not only do these companies offer machines, but they also offer specific services such as databases, service to send SMS, online development tools and backup services. These services are collectively referred to as PaaS or Platform-as-a-Service.
Over the last two years, another new concept has rapidly caught, mainly thanks to Amazon’s Lambda. We call this FaaS or Function-as-a-Service, where instead of running an entire software program or a website throughout the day, we simply run a single function, such as sorting a list of million names or converting the format of 50 videos from MPEG to AVI, etc on a remote machine which stays on for only the duration of the time that our function runs and then shuts down. By not keeping machines running all day, maintenance and operational costs go down significantly. This particular way of running machines for a specific short-term purpose and then shutting them down is now termed as ‘serverless’ architecture. Continue reading “Serverless architecture”
In the last article Introduction to clean architecture: Part 1, we saw how clean architecture is a set of principles for designing software such that the purpose of a software program is clear on seeing its code and the details about what tools or libraries are used to make it are buried deeper, out of sight of the person who views it. This is in line with real world products such as buildings and kitchen tools where a user knows what they are seeing rather than how they are made.
In this article, we will see how a very simple program is designed using clean architecture. I am going to present only the blueprint of a program. I won’t use any programming language, staying true to one of the principles of clean architecture, i.e. it doesn’t matter which programming language is used.
The simple program
In our program, we will allow our system to receive a greeting ‘Hi’ from the user while greeting him/her back with a ‘Hello’. That’s all we need to study how to produce a good program blueprint with clean architecture.
In our system, we have a single user who greets our system. Let’s call him/her the greeter. Let’s just use the word ‘system’ to describe our greeting application. We have just one case in our system which we can call, ‘Greet and be greeted back’. Here’s how it will look.
The greeter greets our system.
On receiving the greeting ‘Hi’ (and only ‘Hi’), our system responds with ‘Hello’, which the greeter receives.
Any greeting other than ‘Hi’ will be ignored and the system will simply not respond.
This simple use case has two aspects.
It comprehensively covers every step in the use case covering all inputs and outputs. It distinctly says that only a greeting of ‘Hi’ will be responded to and that other greetings will be ignored without response. No error messages, etc.
The use case also has obvious omissions. The word ‘greet’ is a vague verb which doesn’t say how it’s done. Does the greeter speak to the system and the system speak back. Does the greeter type at a keyboard or use text and instant messaging? Does the system respond on the screen, shoot back an instant message or send an email? As far as a use case is concerned, those are implementation details, the decisions for which can be deferred for much later. In fact, input and ouput systems should be plug-and-play, where one system can be swapped for another without any effect on the program’s core working, which is to be greeted and to greet back.
The EBI system
Once the requirements are clear, we start with the use cases. The use case is the core of the system we are designing and it is converted into a system of parts known as the EBI or Entity-Boundary-Interactor. There are 5 components within the EBI framework. Every use case in the system is converted to an EBI using these five parts.
Interactor (I): The interactor is the object which receives inputs from user, gets work done by entities and returns the output to the user. The interactor sets things in motion like an orchestra director to make the execution of a use case possible. There is exactly one interactor per use case in the system.
Entities (E): The entities contain the data, the validation rules and logic that turns one form of input into another. After receiving input from the user, the interactor uses different entities in the system to achieve the output that is to be sent to the user. Remember that the interactor itself must NEVER directly contain the logic that transforms input into output. In our use case, our interactor uses the services of an entity called GreetingLookup. This entity contains a table of which greeting from the user should be responded to with which greeting. Our lookup table only contains one entry right now, i.e. a greeting of ‘Hi’ should be responded to with ‘Hello’.
Usually, in a system that has been meant to make things easy, automated or online based on a real world system, entities closely resemble the name, properties and functionality of their real world equivalents. E.g. in an accounting system, you’ll have entities like account, balance sheet, ledger, debit and credit. In a shopping system, you’ll have shopping cart, wallet, payment, items and catalogues of items.
Boundaries (B): Many of the specifications in a use case are vague. The use case assumes that it receives input in a certain format regardless of the method of input. Similarly it sends out output in a predetermined format assuming that the system responsible for showing it to the user will format it properly. Sometimes, an interactor or some of the entities will need to use external services to get some things done. The access to such services are in the form of a boundary known as a gateway.
E.g., in our use case, our inputs and outputs may come from several forms such as typed or spoken inputs. The lookup table may seek the services of a database. Databases are an implementation detail that lie outside the scope of the use case and EBI. Why? Because, we may even use something simpler such as an Excel sheet or a CSV file to create a lookup table. Using a database is an implementation choice rather than a necessity.
Request and response model: While not abbreviated in EBI, request and response models are important parts of the system. A request model specifies the form of data that should be sent across the boundaries when requests and responses are sent. In our case, the greeting from the user to the system and vice-versa should be sent in the form of plain English text. This means that if our system works on voice-based inputs and outputs, the voices must be converted to plain English text and back.
With our EBI system complete to take care of the use case, we must realise that ultimately the system will be used by humans and that different people have different preferences for communication. One person may want to speak to the system, while another prefers instant messaging. One person may want to receive the response as an email message, while another may prefer the system to display it on a big flat LCD with decoration.
A controller is an object which takes the input in the form the user gives and converts it into the form required by the request model. If a user speaks to the system, then the controller’s job is to convert the voice to plain English text before passing it on to the interactor.
On the other side is a presenter that receives plain text from the interactor and converts it into a form that can be used by the UI of the system, e.g. a large banner with formatting, a spoken voice output, etc.
Being able to test individual components is a big strength of the clean architecture system. Here are the ways in which the system can be tested.
Use case: Since the use case in the form of EBI is seperated from the user interface, we can test the use case without having to input data manually through keyboards. Testing can be automated by using a tool that can inject data in the form of the request model, i.e. plain text. Likewise the response from the use case can be easily tested since it is plain text. Also individual entities and the interactor can be seperately tested.
Gateway: The gateways such as databases or API calls can be individually tested without having to go through the entire UI and use case. One can tools that use mock data to see if the inputs to and outputs from databases and services on the Internet work correctly.
Controllers and presenters: Without involving the UI and the use case, one can test if controllers are able to convert input data to request model correctly or if presenters are able to convert response model to output data.
Freedom to swap and change components
Interactors: Changes to the interactors are often received well by the entire system. Interactors are usually algorithms and pieces of code that bind the other components together, usually a sequence of steps on what to do. Changes to the steps does not change any functionality in the other components of the system.
Entities: Entities are components that contain a lot of data and rules relevant to the system. Changes to entities will usually lead to corresponding changes in the interactor to comply with the new rules.
Boundaries: Boundaries are agreements between the interactor and external components like controllers, presenters and gateways. A change to the boundary will inevitably change some code in the external components, so that the boundary can be complied.
UI: With a solid use case in place, you can experiment with various forms of UI to see which one is most popular with your users. You can experiment with text, email, chat, voice, banner, etc. The use case and the gateway do not change. However, some changes to the UI can cause a part of the controller and the presenter to change, since these two are directly related to how the UI works.
Controller and presenter: It is rare for the controller or presenter to change in their own rights. A change to the controller or presenter usually means that the UI or the boundary has also changed.
Clean architecture seperates systems such that the functionality is at the core of the system, while everything like user interface, storage and web can be kept at the periphery, where one component can be swapped for another. Hopefully, our example has given you a good idea about how to approach any system with clean architecture in mind.