6 Things To Consider When Building Large-Scale Web Applications

By Bhavin Prajapati

A bit about myself…

I’ve been in the technology industry for over 8 years now and everyday I learn cool new things that challenge me. One thing I’ve noticed over the years is that as the languages, tools and frameworks improve & evolve over time, we as software engineers need to adapt to these changes. The way to build web applications 4 years ago was different than how it is today.

However, there are always certain aspects that are common and reluctant to have drastic changes even if the languages, tools and frameworks have become obsolete. I’ll describe some important things to consider when starting out to build a web application so that there’s a solid foundation for growth.

1. Architecture

The most important thing when building a web application is of-course the architecture, this can be broken up into 5 subcategories: languages, API, frameworks, integrations and topology. There are a plethora of items in this list which can be explored but rather than discussing that, we can just say that it’s the main aspect to spend time exploring when building a web application.

One thing I’ve learned is that it does NOT matter too much what stack you end up using, it’s how well you know how to use what you choose. Everyone has a strong suit and depending on what you like working with, you can become good at it. There are of-course languages/frameworks/software suitable for specific situations and as a developer gains experience, they learn to figure out what belongs where.

“Choose your technology!”

author

It’s much like choosing a character in Mortal Kombat, in the end they can all accomplish the same things but are you able to work with what you choose…“do you know all the characters’ moves?”

More recently, I’ve been working heavily with NodeJS (back-end) and React/Redux (front-end) and there are of-course pros and cons to using this stack like any other. Node (8 yo) is also relatively new compared to languages like C# (18 yo), Ruby (23 yo), PHP (23 yo) or Python (28 yo) which have been around for years. You should NOT pick just because it’s a new language, it’s actually counter intuitive because the older languages have stood the test of time. They’re most likely more reliable, have a bigger community and better support for their modules.

Some of the languages and frameworks TribalScale works with includes…NodeJS, Ruby on Rails, .NET Core, Python, React/Redux, GraphQL, RabbitMQ, AWS, CloudFoundry and Git/Gitflow

2. TDD (Test-Driven Development)

The choice of whether or not to use TDD when implementing your project is actually a tough one, it’s not a black and white type scenario like most people think it is. People who are very opinionated will tell you right away that TDD is the way to go and if you don’t think otherwise, then you are a noob! I on the other hand do not see it this way, there’s really a lot of grey areas when it comes to whether or not to use TDD. More often than not, it makes sense to apply it but situations where you know nothing can possibly go wrong is where I find testing unnecessary. For example: If you are rendering a simple React component given a set of props without much business logic in the render method, does it really make sense to test that it renders correctly?

Where TDD has shown a lot of value to me is when new developers join a project and begin refactoring the code, the tests really help to ensure that they are not breaking another part of an application. This results in an architecture that is modular, easy to understand as well as refactor and finally does not produce spaghetti code.

“Slow and steady does win the race!”

Working with TDD is much like being the tortoise shown above, you get a slow start because you have to do the ground work of integrating a testing and coverage framework and implement the initial tests which requires overhead. However, the progression is steady and continuous. You won’t end up like the confused hare in the middle of the project.

Some of the testings frameworks TribalScale works with includes…Jest, Mocha, Chai, Istanbul, RSpec, xUnit.net and Moq

3. CI (Continuous Integration)

Having a CI & CD (Continuous Integration & Continuous Delivery) system in place is a huge bonus for any project that requires builds to be delivered on a timely basis. It allows developers to focus on building out features instead of doing monotonous tasks such as testing, building and deploying regularly, which leave’s room for human error. I’ve done projects with and without CI & CD in place and let me emphasize that it’s such a crucial piece that you will save a ton of time from the overhead of tedious tasks and you will get way fewer headaches as well.

“It’s the assembly line of the software industry!”

author

It’s like a conveyor belt with 3 steps: testing, building and deploying. Continuous integration and continuous delivery has done for the software industry what the assembly line had done for the automotive / manufacturing industry back in the 1910s. Automation in this area has made it possible to deliver results incrementally which makes clients feel confident in what they are getting.

Some of the CI & CD TribalScale works with includes…Concourse, Buildkite, Jenkins and CircleCI

4. Persistance

Another very important aspect that could be a sub-category to architecture is persistence but there are many things that are worth mentioning so it has it’s own section. The patterns related to persistence are the same across all languages/platforms so here are a list of things that you should know be familiar with as it relates to data storage.

ORMs (Object Relational Mapping): These are libraries that allow developers to work with databases without having to worry about writing queries and creates an abstraction layer above the type of storage. Eg.

Seeders: When starting out implementing, a developer may need test data to work with so these are needed to initialize a database with dummy data.

Migrations: These are scripts needed to build the database from scratch by using the models and their relationships. Any changes to the database architecture are reapplied from the beginning by all the migration scripts to rebuild the database from the beginning.

“It’s not so great when you have to pick up the cards, learn to shuffle!”

author

The above image summarizes how it all works quite well. At first it seems hard but with practice, it becomes second-hand. It only takes a few tries before you start getting the hang of it. Implementing applications with these patterns is very common and quite difficult to do without unless you’re brave and want to write all your queries just for fun. I consider this one of the more difficult areas to work with but regardless, these patterns will make your life much easier when applying them in practice.

Some of the database technologies TribalScale works with includes…Entity Framework, Active Record, Sequelize, Hibernate, MySQL, Redis, MongoDB, PostgresSQL

5. Security

Since the internet isn’t quite secure yet, we must implement our applications as if everyone is out to hack you. One thing is that regardless of how many layers of security you have, it is never fully secure so you as a developer must decide what is enough. There are of course a lot of best practices and standard ways of implementing security which have many resources online. Here are some of best practices that I’ve worked with…

Password Hashing/Salting: This is a very important security concern because you do not want people who have access to the databases to get the passwords so they must be hashes and salted when being stored.

JWT (JSON Web Token): This is standard that is used to allow secure communication between the client and the server. The server gives the client a JWT Token when signing in.

Cookies/Session Expiry: The JWT Token is often stored in the cookies with an expiry date, every subsequent request that is made to the server sends the cookie so that it can be verified that an authenticated user is making the request.

“It can sometimes result in headaches if not implemented correctly!”

author

This can be difficult aspect to deal with sometimes due to corner cases but there are enough popular frameworks and best practices now that it has made simple to implement.

Some of the security related stuff TribalScale works with includes…OAuth, Auth0, JWT, BCrypt

6. Performance

Based on my experience, it’s not often that developers have to deal with performance but I was fortunately enough to have worked on websites which had problems which required performance improvement. It is entirely dependent on the project where performance can be improved, some general things include…

Page Load Time: There are many ways to improve the load time such as lazy-loading sections or infinite scrolling. Using a CDN (Content Delivery Network) to significantly reduce load time of assets and javascript files.

Caching: If you are working on a public facing web application, chances are you will have to implement some type of caching solution so that you don’t have to re-render the page every time.

Server-side Rendering: This can be useful for high traffic public facing sites with content from a CMS, a cache can be used to store rendered pages.

Client-side Rendering: This can be useful for single page web applications where there is a lot of dynamic content and a complex front-end UX/UI.

Database Queries: If there are database queries with long response times which could be bottle necks to the application, it makes sense to improve them by indexing tables, refactoring code to reduce iteration and optimizing the query itself.

“Having a fast car doesn’t make you a better driver!”

author

Even if you’ve bought the best in class car, in the end it’s the driver that takes it across the finish line. Similarly even if you’re using the newest technology stack, it’s the developer that will have to architect and deliver the final result.

So put the pedal to the metal and enjoy the ride, you’ve got a long road ahead.

PS: Please checkout my StarterKit on Github as a starting point for building web applications using NodeJS, React, Redux and Authentication.

https://github.com/bhavin-prajapati/react-redux-node-auth-starter

author

About the author

Bhavin Prajapati is a Full-Stack Agile Software Engineer at TribalScale with over 8 years of experience working in the technology industry architecting and implementing solutions for web, mobile, OTT video and voice platforms. He has a strong passion for architecture and performance oriented design. He’s a web technology expert who is always looking to connect with like-minded individuals. In his spare time he likes to drink beer, watch movies, listen to tunes, travel, cook and code some more. Feel free to reach out for a coffee!

Join our fast growing Tribe and connect with us on Twitter, LinkedIn & Facebook! Learn more about us on our website.

Visit Us on Medium

You might also be interested in…