Search

How to Create MongoDB Collections

In MongoDB, data is stored in the form of documents. These documents are stored inside collections and collections are placed inside database. A database in MongoDB can have several collections. Collections in MongoDB can be created implicitly while we are importing the data or explicitly by specifying the ‘createCollection’ command. Usually the implicit creation is the most common method but for instances where we need to create a capped collection or collection with schema validation rules explicit method is used. In this article, we will explore both the implicit and explicit methods to create acollection.  Implicit Creation In this method, we do not specify the create collection command. The collection is created automatically when data is imported from the application. This is the most common method used for creating collection in MongoDB. We can insert the data from the mongo shell or directly from the application. Example 1: Create a collection named implicit through the mongo shell.In the above example we have inserted one row inside the implicit collection which was not present before the document insertion. As the row is inserted we can see a new collection is created. This collection can now be used for future data insertion and is persistent. Example 2: Create a collection named application through python application. Steps:  First,We need to establish connection to themongodb server.  conn = MongoClient(‘host’, port-number)  Here, host and port will be of the server hosting mongodb.  Switch to the database where we want to import data.  db = conn.databasename  Create a collection object  Collection = db.application  Insert data  Collection.insert_one({“name” : “implicit”}) This will create a collection named application and insert data inside it.Now we need to verify if the data inserted is present inside the collection.  We can see that the application collection is created inside the implicitdb database and the rows we inserted are present.Explicit Creation The explicit method to create collection is mainly used in case there is need to create special collections like capped collections or collections with document validation rules.  Syntax: db.createCollections(name,options) Below is the description of the arguments passed in the createCollection() method.ParameterTypeDescriptionnameStringName that we want to give to collection.optionsdocumentThis is optional, depending on the type of collection we want to create we can specify different options.The ‘name’ parameter is of type string while the ‘option’ parameter is document. Let’s explore the ‘option’ parameter document to see some of major options while creating collection explicitly.FieldDescriptioncappedSpecify value ‘true’ if want to create capped collectionsizeMaximum size in bytes for capped collectionmaxMaximum number of documents allowed in capped collection.StorageEngineTo specify storage engine options. Valid only for WiredTigerwriteConcernThe write concern that will be used for the collection for write operations.ValidatorAllows to specify validation rules for collectionExample: Creating a collection named ‘explicit’ explicitlyWe can see ‘explicit’ collection is created using the create eCollection command. Capped Collection Capped collections are fixed size collections that start to overwrite the oldest log entries when the size reaches the maximum limit. The oplog that is used for MongoDB replication are capped collections.  Example: Create a collection named ‘cc’ which is capped collection.In above example we have create capped collection ‘cc’ explicitly using the createCollection command. While creating the capped collection we need to specify the collection size and maximum number of documents it can have. Creating Collection with Document validation Documents with validation in MongoDB compare each insert and updates against the validation rules set for the collection. These rules are specified using validator option in the createCollection command. The row is inserted or updated if the validation is passed and rejected if the validation fails. Example: create a collection named ‘validate’ with validation rules We will create a collection named ‘validate’ with validation rules and try to insert data that does not meet the validation requirement.We can see document insertion failed as it did not pass the validation of schema defined by the validator. Access Needed to Create Collections To read and write data inside MongoDB we should have proper authentication/authorization. Authentication defines who can access the system and Authorization controls what activity can be performed by whom. MongoDB uses RBAC [Role Based Access Control] where access is provided to set of roles and then these roles are assigned to users.  Below is the access matrix for createCollection.User with built-in role readWrite can also create collection inside the database. Locking Behaviour of createCollection() command From MongoDB version 4.2 onwards db.createCollection() obtains an exclusive lock on the specified collection or view for the period of the operation. This lock is being held typically for a very short time but during this operation all other operations must wait for the lock to be released. If we are creating view additional lock will be placed on collection system.views and will block all other view creation until this command completes. In summaryIn this blog, we have seen how collection can be created in MongoDB both implicitly and explicitly. The implicit way to create collection inside the MongoDB database is the most common method, widely used by developers.  The explicit method is needed when special kinds of collections need to be created such as capped collection or if we want to place validation rules for our schema. We have also pored over the access matrix which helps developers to be provided with adequate access so that they can create collections implicitly. Finally, we saw the behaviour of locks when the collection is created. To advance your career in the much sought after MongoDB, explore the MongoDB developer course. You will learn critical aspects of MongoDB and use it to perform real-time analytics, data management, and other activities with a unique blended learning experience.  In the MongoDB administrator course, you will learn about architecture and installation, CRUD operations, schema design, data modelling and how replication works. You will also learn to use various GUI tools required to perform DBA tasks.Find easy solutions to Database management. Explore our MongoDB online workshops here.

How to Create MongoDB Collections

8K
How to Create MongoDB Collections

In MongoDB, data is stored in the form of documents. These documents are stored inside collections and collections are placed inside database. A database in MongoDB can have several collections. 

Collections in MongoDB can be created implicitly while we are importing the data or explicitly by specifying the ‘createCollection’ command. Usually the implicit creation is the most common method but for instances where we need to create a capped collection or collection with schema validation rules explicit method is used. 

In this article, we will explore both the implicit and explicit methods to create acollection.  

Implicit Creation 

In this method, we do not specify the create collection command. The collection is created automatically when data is imported from the application. This is the most common method used for creating collection in MongoDB. We can insert the data from the mongo shell or directly from the application. 

Example 1: Create a collection named implicit through the mongo shell.

In the abovehow to Create a collection named implicit through the mongo shell example we have inserted one row inside the implicit collection which was not present before the document insertion. As the row is inserted we can see a new collection is created. This collection can now be used for future data insertion and is persistent. 

Example 2: Create a collection named application through python application. 

Steps:  

First,We need to establish connection to themongodb server. 
conn = MongoClient(‘host’, port-number) 
Here, host and port will be of the server hosting mongodb. 
Switch to the database where we want to import data. 
db = conn.databasename 
Create a collection object 
Collection = db.application 
Insert data 
Collection.insert_one({“name” : “implicit”}) 

This will create a collection named application and insert data inside it.

how to Create a collection named application through python application

Now we need to verify if the data inserted is present inside the collection.  

how to Create a collection named application through python application

We can see that the application collection is created inside the implicitdb database and the rows we inserted are present.

Explicit Creation 

The explicit method to create collection is mainly used in case there is need to create special collections like capped collections or collections with document validation rules.  

Syntax

db.createCollections(name,options) 

Below is the description of the arguments passed in the createCollection() method.

ParameterTypeDescription
nameStringName that we want to give to collection.
optionsdocumentThis is optional, depending on the type of collection we want to create we can specify different options.

The ‘name’ parameter is of type string while the ‘option’ parameter is document. Let’s explore the ‘option’ parameter document to see some of major options while creating collection explicitly.

FieldDescription
cappedSpecify value ‘true’ if want to create capped collection
sizeMaximum size in bytes for capped collection
maxMaximum number of documents allowed in capped collection.
StorageEngineTo specify storage engine options. Valid only for WiredTiger
writeConcernThe write concern that will be used for the collection for write operations.
ValidatorAllows to specify validation rules for collection

Example: Creating a collection named ‘explicit’ explicitly

Creating a collection named ‘explicit’ explicitly

We can see ‘explicit’ collection is created using the create eCollection command. 

Capped Collection 

Capped collections are fixed size collections that start to overwrite the oldest log entries when the size reaches the maximum limit. The oplog that is used for MongoDB replication are capped collections.  

Example: Create a collection named ‘cc’ which is capped collection.

Create a collection named ‘cc’ which is capped collection

In above example we have create capped collection ‘cc’ explicitly using the createCollection command. While creating the capped collection we need to specify the collection size and maximum number of documents it can have. 

Creating Collection with Document validation 

Documents with validation in MongoDB compare each insert and updates against the validation rules set for the collection. These rules are specified using validator option in the createCollection command. The row is inserted or updated if the validation is passed and rejected if the validation fails. 

Example: create a collection named ‘validate’ with validation rules 

We will create a collection named ‘validate’ with validation rules and try to insert data that does not meet the validation requirement.

create a collection named ‘validate’ with validation rules

We can see document insertion failed as it did not pass the validation of schema defined by the validator. 

Access Needed to Create Collections 

To read and write data inside MongoDB we should have proper authentication/authorization. Authentication defines who can access the system and Authorization controls what activity can be performed by whom. MongoDB uses RBAC [Role Based Access Control] where access is provided to set of roles and then these roles are assigned to users.  

Below is the access matrix for createCollection.

Access matrix for createCollection

User with built-in role readWrite can also create collection inside the database. 

Locking Behaviour of createCollection() command 

From MongoDB version 4.2 onwards db.createCollection() obtains an exclusive lock on the specified collection or view for the period of the operation. This lock is being held typically for a very short time but during this operation all other operations must wait for the lock to be released. If we are creating view additional lock will be placed on collection system.views and will block all other view creation until this command completes. 

In summary

In this blog, we have seen how collection can be created in MongoDB both implicitly and explicitly. The implicit way to create collection inside the MongoDB database is the most common method, widely used by developers.  

The explicit method is needed when special kinds of collections need to be created such as capped collection or if we want to place validation rules for our schema. We have also pored over the access matrix which helps developers to be provided with adequate access so that they can create collections implicitly. Finally, we saw the behaviour of locks when the collection is created. 

To advance your career in the much sought after MongoDB, explore the MongoDB developer course. You will learn critical aspects of MongoDB and use it to perform real-time analytics, data management, and other activities with a unique blended learning experience.  

In the MongoDB administrator course, you will learn about architecture and installation, CRUD operations, schema design, data modelling and how replication works. You will also learn to use various GUI tools required to perform DBA tasks.


Find easy solutions to Database management. Explore our MongoDB online workshops here.

KnowledgeHut

KnowledgeHut

Author

KnowledgeHut is an outcome-focused global ed-tech company. We help organizations and professionals unlock excellence through skills development. We offer training solutions under the people and process, data science, full-stack development, cybersecurity, future technologies and digital transformation verticals.
Website : https://www.knowledgehut.com

Join the Discussion

Your email address will not be published. Required fields are marked *

Suggested Blogs

How to Install Node.JS on Ubuntu

Node.js is a general-purpose programming JavaScript platform that allows users to quickly build network applications. Node.js makes development more consistent and integrated by using JavaScript on both the front and backend, and allows you to write JavaScript on the server-side.JavaScript, as you know, is a browser-based language. The browser's engine will take and compile JavaScript code into commands. The creator of Node.js took the engine of Chrome and set it to work on a server. The language can be interpreted in an environment.In this article, we will read about Node.js installation using three methods:Installing the Stable VersionInstall Using a PPAInstall Using NVMPrerequisitesHardware Requirements:RAM: 4 GBStorage: 256 GB of Hard Disk SpaceSoftware Requirements:Web Browser: Any browser such as Google Chrome, Mozilla Firefox, Microsoft Edge.Operating System: An Ubuntu 18.04 server installed with a non-root sudo user and firewall.Installation Procedure1. Installing the Stable Version for UbuntuIn its default repositories, Ubuntu 18.04 contains a version of Node.js that provides a consistent experience on a number of systems. The version of the repositories is 8.10.0 when it is written. This is not the latest version but should be stable and sufficient for quick language experiments.Step 1: You can use the apt package manager to obtain this version. Refresh your index for your local package with:$ sudo apt updateStep 2: Now, install Node.js from the repository:$ sudo apt install nodejsStep 3: This is all you need to do to get set up with Node.js if the package in the repositories fits your needs. In most cases, the package manager Node.js will also install npm. This can be done by:$ sudo apt install npmThis allows the installation of Node.js modules and packages.The executable from Ubuntu repositories is known as nodejs instead of node because of conflict with another package. Take this into account when you run the software.Step 4: To check which Node.js version you installed, try this:$ nodejs -vYou can decide if you want to work with different versions, package archives, and version managers, after you have established which version of Node.js you have installed from Ubuntu repositories. The following elements will be discussed with more flexible and robust installation methods.2. Install Using a PPAStep 1: First install curl on Ubuntu:$ sudo apt install curlStep 2: To access its contents, install the PPA first. Use curl to retrieve the installation script in your favourite version from your home directory so that your favourite version (if different) replaces 10.x:$ cd ~ $ curl -sL https://deb.nodesource.com/setup_10.x -o nodesource_setup.shStep 3: nano (or your preferred text editor) can be used to inspect the contents of this script:$ nano nodesource_setup.shStep 4: Now, run the script under sudo:$ sudo bash nodesource_setup.shStep 5: You add the PPA to the settings and automatically update your local package cache. You can install the Node.js package in the same way that you did above after running the Nodesource setup script:$ sudo apt install nodejsStep 6: To check the version of Node.js, use:$ nodejs -vThe nodejs package includes the nodejs binary and npm so that you don't have to install npm individually.Step 7: Npm uses a setup file to keep track of updates in your home directory. The first time you run npm, it will be created. To check that npm has been installed and to create a settings file, execute this command:$ npm -vStep 8: You need to install the build-essential package to allow certain npm packages (for instance those that require compilation code from source) to work.$ sudo apt install build-essentialYou now have the tools to work with npm packages, which require source code compilation.3. Install Using NVMAn alternative is to use a tool called nvm which stands for "Node.js Version Manager" when installing Node.js with apt.  You can access the latest versions of Node.js and manage previous releases by controlling your environment with nvm. However, the Node.js versions that you handle are different from the apt versions.Step 1: You can use curl to download the nvm installer from the GitHub project page. Note that the version number may differ from the above:$ curl -sL https://raw.githubusercontent.com/creationix/nvm/v0.35.3/install.sh -o install_nvm.shStep 2: Use nano to inspect the installation script:$ nano install_nvm.shStep 3: Now, run the script with bash:$ bash install_nvm.shIt installs the software in your home ~/.nvm subdirectory. The lines needed to use the file will also be added into your ~/.profile.Step 4: You will either need to log out and log in back in to get access to NVM functionality or to source the ~/.profile file so you know the changes in the current session:$ source ~/.profileStep 5: You can install isolated versions of Node.js with nvm installed. For information on the available Node.js versions, type:$ nvm ls-remoteStep 6: As you can see, at the time of writing this blog, the current version of LTS was v12.18.3. You can install this with:$ nvm install 12.18.3Step 7: Normally, nvm will use the most recently installed version. You can instruct nvm to use the newly downloaded version by typing:$ nvm use 12.18.3Step 8: The executable is called node when you install Node.js with nvm. The current version of the shell can be seen by:$ node -vStep 9: You can see what is installed in your system if you have multiple Node.js versions:$ nvm lsStep 10: If you wish to default to any of the versions, use:$ nvm alias default 12.18.3Step 11: When a new session spawns, this version is automatically selected. You can also mention it through the alias as follows:$ nvm use defaultEach Node.js version will track and have npm available to manage its own packages.Step 12: In the directory of Node.js project's /node modules you can also have npm install packages. Install the express module using the following syntax:$ npm install expressStep 13: If you want the module to be installed globally by other projects with the same node.js version, you can add the flag -g:$ npm install -g expressThe package will install in:~/.nvm/versions/node/12.18.3/lib/node_modules/expressStep 14: If you install the module globally, you can execute commands from the command line, but the package must be linked to your local area in order to access this from a program:$ npm link expressStep 15: You can learn more about the nvm by using:$ nvm helpCreate Demo Web ServerStep 1: If your node is to be tested. Set up js. Let's build a "Hello World!" web server. You can use the following command to create a file app.js and open it in a text editor.Step 2: Set up a JavaScript (js) file to test the node. Let's build a "Hello World!" web server. You can use the following command to create a file app.js and open it in a text editor.$ gedit app.jsStep 3: Now, add the following content in the text editor and save it:var express = require('express'); var app = express();app.get('/', function (req, res) {   res.send('Hello World!'); });app.listen(3000, function () {   console.log('Example app listening on port 3000!'); });Step 4: Start the node application using the following command:$ node app.jsYou will see the output as an example app running in port 3000!Step 5: On port 3000 the webserver was launched. Here is a web browser access http://127.0.0.1:3000/ URL. Now you need to set up your app's front-end server.You're done, that's it. You have created your first Node application successfully. Don't stop here, continue to explore the beautiful Node.js world, as it has more to offer.How to Uninstall Node.js?Depending on the version you want to target, you can uninstall Node.js with apt or nvm. You have to work with the apt utility at system level in order to remove the distro-stable version.1. Using apt:$ sudo apt remove nodejsThis command will remove the package and keep the setup files. This can be useful for you if the package is to be reinstalled later.2. If the configuration files for future use are not to be saved, then run the following:$ sudo apt purge nodejsThis deactivates the package and deletes its associated configuration files.3. You can then delete any unused packages, which have been installed automatically with the deleted package:$ sudo apt autoremoveUninstall using NVM1. To uninstall a Node.js version that you have enabled with nvm, first determine whether the version you want to remove is the current active version:$ nvm current2. If the current active version is the version you want to remove, you must first deactivate NVM in order to allow your changes:$ nvm deactivateYou can uninstall the current version by using the command above to uninstall all the associated Node.js files with the exception of cached files which can be reinstalled.Learn more about the core concepts of Node with REPL, Cluster, Routing, Express with Node.js Certification Course.Conclusion:On your Ubuntu 18.04 server, there are a few ways to run Node.js. You can decide which of the above-mentioned methods are best suited to your needs. The easiest way to use the packaged version in Ubuntu's repository is by adding flexibility using nvm.
3435
How to Install Node.JS on Ubuntu

Node.js is a general-purpose programming JavaScrip... Read More

How to Drop a MongoDB Database?

MongoDB is the most popular NoSQL database among both enterprise and startups, and the fact that it is highly scalable makes it perfectly suited for current web-apps, which need to scale once the user base increases. MongoDB is different from traditional relational databases because it uses json like objects to store data, instead of tables in relational databases.In this post, we will learn to drop a MongoDB database, using MongoDB Drop Database command. Dropping a database means dropping all the collections (tables in MongoDB) in it, along with the indexes. If you don’t want to drop the entire database, you can drop individual collections.PrerequisitesWe are using Windows 10 in this tutorial. Please make sure you have downloaded the MongoDB Community Server and installed it. It is a very easy setup and you will find a lot of good articles on the internet to guide you through this process. Do make sure that you have added it in the Environment variable in your PC. We have also created an example database, which contains a collection named products.OverviewDropping a database is quite simple and can be done with three methods from the command line and also by any GUI (Graphical User Interface) tool, which is used to connect with the running MongoDB instance.To delete a MongoDB database through the command line, we mainly use db.dropDatabase(). We will first understand how to do this.Drop Database definitionAs mentioned earlier, we drop a database in MongoDB, or delete MongoDB database using the db.dropDatabase() command.db.dropDatabase()This removes the current database, deleting all collections inside it, along with the indexes.writeConcern is the optional parameter, which is a document expressing the write concern to use, if greater than majority. Write concern can include the following fields:{ w: , j: , wtimeout: }w option is used to request acknowledgement, that the write operation has passed to a specified number of mongod instances.j option is used to request acknowledgement, that the write operation had been written to the disk-journalwtimeout option is used to specify the time limit, to prevent the write operations from blocking indefinitely.Using the w option, the following are available:w: 1 – Requests acknowledgement that the write concern has passed to the single mongod instance. It is also the default option.w: 0 – Requests no acknowledgement of the write operation.w: “majority” – It requests acknowledgement that the write operation has passed to the majority of primary and secondary servers.Now, we will look into the j option values.j: true – It requests acknowledgement that the different mongod instances, mentioned in w: , have been written to the on-disk journal.The wtimeout option specifies the time limit in milliseconds, for the write concern. It causes write operations to return with an error after the specified limit.dropDatabaseWe can also use the dropDatabase command directly to delete MongoDB, current database. The command has the following form.{ dropDatabase: 1, writeConcern: , comment: }This command optional fields are-writeConcern is the optional parameter, which is in the form of a document expressing the write concern to use if it is greater than majority. We have just looked into it in detail earlier.comment is a user provided comment, which is attached to this command. Once set, this will appear in the logs.Dropping with dropDatabase CommandWe can drop the database through dropDatabase command. I am connected to my local mongoDB instance from the windows terminal, by giving the mongo command.> mongoAfter that to show all the databases in my MongoDB database, we will use the command show dbs command. It will show all our databases. Out of it I created the example database only.> show dbs admin    0.000GB config   0.000GB example  0.000GB local    0.000GBNow, to drop the example database, first we have to go inside it by using the use example command.> use example switched to db exampleNow, we will use the dropDatabase command from within a db.runCommand(). Now, the runCommand runs the command in the context of the current database. This means it will run the dropDatabase command in the current database, which is seen in this example:> db.runCommand( { dropDatabase: 1 } ) { "dropped" : "example", "ok" : 1 }As discussed, the earlier dropDatabase command accepts an optional writeConcern argument, and also a comment field. The syntax then is written as shown below.{ dropDatabase: 1, writeConcern: , comment: }Dropping with dropDatabase() MethodWe can drop the database through db.dropDatabase() method also. It also removes the current database, deleting all associated files.Once again connect to mongoDb database by giving the command mongo. After that we can check all the databases with show dbs command.For db.dropDatabase() method also, we need to go inside the database that we are dropping, by using use command.Now, we will use the db.dropDatabase() command to drop the database, which is example in our case.> db.dropDatabase() { "dropped" : "example", "ok" : 1 } The db.dropDatabase() method also accepts an optional writeConcern argument. The syntax then becomes: { w: , j: , wtimeout: }Dropping with Unix Shell and Eval commandThis method only works in a Unix environment like Ubuntu, Mac or WSL2 for Windows. Here, the mongo command with the database name can be used to connect to a specific database. For example, to connect to the example database, we can use the below command.$ mongo exampleNow, we can drop our example database with one line command, where we use the eval command followed by the JavaScript code that we wish MongoDB to execute. As seen earlier to drop the database, we use the db.dropDatabase() command. Now, we will wrap this method in printjson function, to get the output of the command.$ mongo example --eval "printjson(db.dropDatabase())" MongoDB shell version: 3.0.9 connecting to: example { "dropped" : "example", "ok" : 1 }Dropping with MongoDB CompassNow we can drop a database or delete MongoDB database very easily using a Graphical tool like MongoDB compass. Infact this tool was auto-downloaded when I had installed MongoDB on my Windows machine.So, first open the MongoDB Compass and you will see the option to connect to a database. We want to connect to our local database, in which the hostname is localhost and port is 27017. These are the default values for all MongoDB connections, unless you changed it during installation.Don’t change anything and click on the Connect button.We are now connected to the MongoDB running on our localhost and we can see all the databases including example database. Next, hover over example database, which will show a delete button. Click on the delete button.We will be shown a pop-up where we have to write the name of our database, which is example in our case. Then click on the DROP DATABASE button, to MongoDB remove database.After that it will take us back to the earlier screen and we can see the example database is dropped.ConclusionIn this article, we have learned about the different MongoDB delete database commands. We have also learned about the write concern optional parameter, which can be passed to both db.dropDatabase() method and dropDatabase command. This parameter is mainly used in large database systems, which have several servers. We hope you found this article useful! Keep learning.
1678
How to Drop a MongoDB Database?

MongoDB is the most popular NoSQL database among b... Read More

The 2021 Learning Path To Becoming a Full Stack Web Developer

Full stack developer roles are among the hottest careers in the tech space now. These talented folks can develop a whole product from scratch. A full stack developer is a combination of Front-end developer and Backend developer. These two in themselves are full time jobs and most people make careers out of one of them. So, we will start with Front-end roadmap and then go to Back-end roadmap. A person interested in becoming a Full-stack developer needs to have proficiency in both the front end and back-end tools, just like I started as a Front-end developer and later on become a Full stack developer by mastering JavaScript backend technologies and databases.The demand for Full Stack Web DeveloperThe demand for Full stack developers is the highest in early-stage startups, where they want to create a Minimum Viable Product at the earliest to showcase to the investors. It is also a nice skill to have in addition to frontend technologies or backend technologies alone, since an employer prefers people with both skills.There are a lot of technologies to learn to be a Full-Stack developer. We will discuss about them in the coming sections.   List of technologies to master to become a Full-Stack developer A full-stack developer is actually a combination of Frontend developer and Backend developer. We need to master both, and both have different Roadmaps. Let’s start with the basics. The frontend is the web-site which we see and it is primarily made with HTML and CSS.  JavaScript was also used earlier but nowadays, it is created with JavaScript frameworks like ReactJS, Angular or Vue. All these frameworks require one to learn the basics of HTML, CSS, & JavaScript. So, we need to learn the basics followed by at least one framework.In the backend we have a lot of technologies and databases also. So, we need to choose one backend framework from Java (Spring Framework), JavaScript (NodeJS) etc and then also learn databases. Databases are divided into two categories, which is NoSQL(MongoDB) and SQL(PostgreSQL, MySQL, Oracle) databases. So, you need to choose one of the databases.We are also required to know about DevOps, which is a practice of harmonizing development and operations whereby the entire pipeline from development, testing, deployment, continuous integration and feedback is automated. The knowledge of either AWS or Azure based cloud ecosystem is required, and also CI/CD like Jenkins and containerizing & orchestrating applications using Docker and Kubernetes.1. Frontend RoadmapLearn the BasicsPlease refer to the attached figure for Front-end roadmap, as we will be referring to this throughout this article. We have to start our journey by learning HTML, CSS and JavaScript which is the base for a web-app or website. HTML has changed a bit over the years, with the introduction of HTML 5 and semantics tags, so make sure to update yourself. JavaScript which was released in 1995, didn’t change much during the next 20 years. But once more and more developers started using it, the ECMA committee decided to add some very nice features and enhance the language, and renamed it ES6 in 2015. After that they regularly added new features to the language and have just released ES2020 in June 2020, which has many additional features. So, learn the basic JavaScript first and then upgrade to ES6 and newer versions. CSS is what makes a website or web-app beautiful, and is often considered the hardest part by a developer. Earlier, CSS was very confusing and had a steep learning curve, because of the use of floats to create a layout. Developers usually used to work with CSS frameworks like bootstrap to design a site. But things have changed a lot with the invention of CSS Grid and Flexbox. Some of the best resources to learn the basics are - html.specdeveloper.mozilla.HTMLStyle CSSdeveloper.mozilla.CSSdeveloper.mozilla.JavaScriptGetting Deeper Now, just learning JavaScript and some basic CSS will not make you a good Front-end developer as you have to take a deep dive into JavaScript. We will discuss CSS later, after learning the essentials of JavaScript.JavaScript EssentialsThere are many things associated with JavaScript which we need to learn before moving forward.The Terminal The first thing to learn is to work in a terminal, and master some of the basic commands. If you are on a Mac, it’s already based on Linux and runs most Linux commands. If you are working on Windows then you must install git bash, which will give you a Linux environment to work with. In JavaScript frameworks, we need to run a lot of commands from the terminal, like if we want to install a third-party dependency by npm.  The basics of Linux can be learnt from their official site.Version ControlNext, learning version control is very important because we should always keep our code in some remote repository like Github. The industry works on Git, which is version control software. It is completely command-based and is used heavily everywhere. Learn the basic commands which will be useful even for an individual developer. Later on, when working with teams, more advanced knowledge of the git command is required.Through the git commands, we store our code in repositories. The most popular ones are Github and Bit Bucket, so we need to learn how to store and link them.The basics of git can be learnt from this awesome tutorial.freecodecamp.orgTask Runners Task runners are applications which are used to automate tasks required in projects. These tasks include minification of JavaScript and CSS files, CSS preprocessing like from SASS to CSS, image optimization and Unit testing. The three popular task runners are npm scripts, gulp and grunt. The npm script is nothing but the package.json file which comes with React projects or is created in a Node.js project using npm init. Gulp and Grunt are much bigger applications and also have a plugin ecosystem that is suited for large JavaScript projects. The basics for these two technologies can be learnt from here. Module Loader and Bundler Both module loaders and bundlers are required for large JavaScript applications. Knowledge of both is required, if the project you are working is a big Vanilla JavaScript project. When a large JavaScript application consists of hundreds of files, the module loader takes care of the dependency and makes sure all the modules are loaded when the application is executed. Examples are RequireJS and SystemJS.Module bundlers also do the same thing, building it at the time of application build rather than at the runtime. Popular examples are Webpack and Rollup. Testing Testing nowadays is very important in any type of project. There are two types of testing; one is known as Unit testing and other as end-to-end testing. For unit testing we write test cases and the most popular tool nowadays is Jest. End-to-end testing is automated testing, which emulates the whole app. Suppose, an app has a login screen and then it shows posts. The testing tool will run the web-app to check whether all the functionalities are done correctly. The two most popular options today are Puppeteer and Cypress. The tutorials to refer for these topics are - Libraries and FrameworkThey are the most important part of the JavaScript ecosystem nowadays. It all started with the release of AngularJS in 2010. Before that period most enterprise apps were made in Java and were desktop apps. But AngularJS changed everything, because it made it easy to manage big projects with JavaScript and helped to create complex web-apps.1. React It is the most popular JavaScript library today and is used by both enterprises and startups that have a huge ecosystem. It is not a complete framework like Angular and we have to install third party dependencies for most things. But if you want to learn a framework that will get you a job, then that framework would be ReactJS, and its demand is not going away for the next 5 years. The component approach and its easy learning curve have made React more popular than other frameworks. A good starting tutorial for React isState Management In React state management can sometimes become complex, when we need to share data between components. We generally take help of external packages in it with the most popular being Redux. But we also have other state management libraries like XState and Recoil. Server-side rendering With performance becoming important nowadays, Server-Side Rendering speeds up the React projects even faster. In SSR projects, the React code is rendered on the server and the client browser directly receives the HTML, CSS, JS bundle. The only framework to do it is NextJS. Static Site Generators Lot of sites don’t need to be updated frequently and it is the place where the only Static Site Generator for ReactJS, which is GatsbyJS shines. With the help of GatsbyJS we can create extremely fast static sites and it gets into Wordpress domain a lot with it. GatsbyJS also has a huge ecosystem of plugins, which enhances its functionalities. React Testing Unit testing is a very important part of ReactJS projects, especially the ones which are very large. Unit testing ensures that we have lower bugs in Production build. The two popular libraries are – Enzyme and Jest. 2. Angular It is a complete framework and unlike React requires very few external dependencies. Everything is built within Angular and we don’t have to go outside for more features. Since it was among the earliest frameworks, older projects are in Angular and it is still widely used in enterprises. A good tutorial to learn Angular is below. 3. Vue Vue is another very popular JavaScript library, which has the best features of both ReactJS and Angular and has become very popular in recent years. It is widely used in both enterprise and startups. A good tutorial to start with Vue is below. 4. NuxtJS It is used for Server-Side Rendering in Vue projects and is similar to the NextJS framework used in ReactJS for SSR.  5. Svelte It is the newest of all frameworks/libraries and has become quite popular, but still not used much in enterprises and startups. It is different from React, Vue and Angular and converts the app at build time rather than at run time as in the other three. Good tutorials to start with Svelte are below. CSS Deep DiveA lot has changed in CSS after it included CSS Grid and Flexbox; it has become much easier for developers to work with. CSS Essentials It is now mandatory for frontend developers to learn CSS Grid and Flexbox, because through it we can develop beautiful layouts with ease. More companies are moving away from CSS Frameworks and have started working with CSS Grid and Flexbox, which are now supported by all browsers. Good tutorials to learn Flexbox and CSS Grid are below. Preprocessors CSS preprocessors are used to add special functionalities in CSS, which it lacks. An example is Sass, which adds special features like variables and nested rules in CSS and is widely used in the industry for larger projects. The other popular one is PostCSS, in which we can use custom plugin and tools in CSS. CSS Frameworks Frameworks were very popular from the early days of CSS, when it was very complicated because of floats. Bootstrap This is the most popular and oldest CSS framework; easy to learn and also has a wide variety of elements, templates and interfaces. Bulma It is another CSS framework, which is very popular and much easier to use than bootstrap. Tailwind CSS This is a fairly new CSS framework and is quite popular nowadays. It follows a different approach than the other frameworks and contains easier classes. Styled Components (React) This is a CSS in JS library and is for React only. It is used to create components out of every style and is very popular in the React world.  CI/CDThe Continuous Integration/ Continuous deployment is mainly used by DevOps. But a frontend engineer should know its basics. It is used to build, test and deploy applications automatically.Github Actions  It is a freely available CI/CD pipeline, which directly integrates to your github based project and can be used in a variety of languages. Deployment It is again a task which mainly falls into the domain of Backend engineers and DevOps, but a frontend engineer should know some basic and simple tools. Static Deployment These products are mainly used to deploy static sites, which consists of HTML, CSS and JavaScript only. Two very popular services are Amazon S3 and Surge.sh Node Application Deployment The projects containing node code cannot be deployed using static deployment. Even if the project is a simple ReactJS project, it also uses node for processing. These applications require services which run the Node code and deploy it. The three most popular services are Vercel, Firebase and Netlify. 2. Backend Roadmap (Including Storage, Services & Deployment)Understanding the BackendBackend is the part of the website that provides the functionality, allowing people to browse their favorite site, purchase a product and log into their account, for instance. All data related to a user or a product or anything else are generally stored in databases or CMS (Content Management System) and when a user visits any website, they are retrieved from there and shown. One of the responsibilities of a backend engineer involves writing APIs, which actually interact with the database and get the data. They are also involved in writing schemas of database and creating the structure of databases. Backend EssentialsFor a backend engineer, working in a Linux environment is an essential skill. A lot of the configurations are done on the terminal. So, he or she should be very good with Linux commands.Also, they should know both commands and the use of any git powered platforms like Github or bitbucket.Languages and FrameworksAll of the popular languages have some framework, which has been used for backend development. These frameworks are generally used to create API endpoints, which are used to fetch or store data in the database. For example, when we scroll articles on Facebook, these articles are fetched from a database and we use the GET method to fetch them. Similarly, when we write an article and hit submit, it uses POST method.Now, different frameworks implement this GET, POST and other APIs also referred to as RESTful APIs in their own way.Java Java is by far the oldest and the most used language for backend development. It is also used for a variety of other tasks like Android development, but it shines in the backend because of its multithreading abilities. So, enterprise grade web-apps and web-apps with a lot of traffic prefer Java, because it handles loads better. The most popular frameworks for backend development in Java are Spring Framework and Hibernate. Some good beginner's tutorials are - JavaScript It is a very popular choice for backend development, because on the frontend side JavaScript is the only choice. So, a lot of frontend engineers can take this choice to become Full-stack developers. Node.js It allows developers to use JavaScript to write server-side code, through which they can write APIs. Actually, the API part can be done by numerous frameworks of Node.js out of which Express is widely used. The other popular framework is Fastify. Some good beginner's tutorials are - Python Python is one of the most popular languages among developers and has been used in a variety of fields. The two most popular frameworks for Python are Flask and Django. Some good beginner tutorials are - C# It is a very popular programming language which was developed by Microsoft and it has the power of C++. Its popularity increased once the .NET framework was released for backend development. As Microsoft is very popular in enterprises, the .NET framework is generally preferred in enterprises. A good tutorial to learn .NET is - Go Go language which is also referred to as Golang, has gained popularity in recent years. It is used a lot in Backend programming and the two popular frameworks are Gin and Beego. DatabaseFor a Backend engineer, after making APIs with framework based on language, it's time to learn about Databases. Databases are used to store most of the things which we see in a web-app, from user login credentials to user posts and everything else. In the earlier days we only used to have one type of Database and that was Relational databases, which use tables to store data. Now we have two other categories also, one being NoSQL databases and the other In-memory databases. 1. Relational databases Relational databases allow you to create, update and delete data stored in a table format. This type of database mostly uses SQL language to access the data, hence is also known as an SQL database. MySQL It is one of the oldest databases and was released in 1995. It is an open-source database and was very popular in the 2000s with the rise of LAMP (Linux, Apache, MySQL, PHP) stack. It is still widely in use, but there are other popular Relational databases. A good tutorial to learn MySQL is - PostgreSQL PostgreSQL, which is also known as Postgres is also an old open-source Relational database, which was released in 1996. But it gained popularity recently, as it goes very well with modern stacks containing NodeJS and other backend technologies. A good tutorial to learn PostgreSQL is - Oracle is the most popular and oldest relational database. It was released in 1979 and still remains the number one preference for enterprise customers. All the big banks and other organizations, run on Oracle databases. So, the knowledge of Oracle is a must in many companies for an Engineer. A good tutorial to learn Oracle is - MS-SQL MS-SQL is also known as Microsoft SQL and is yet another commercial Relational database. It has got different editions, used by different audiences. It is also heavily used by enterprise users and powers a whole lot of big systems around the world. A good tutorial to learn MS-SQL is - 2. NoSQL databases NoSQL databases are also called non-SQL databases. The NoSQL databases mainly store data as key-value pairs, but some of them also use a SQL-like structure. These databases have become hugely popular in the 21st century, with the rise of large web-apps which have a lot of concurrent users. These databases can take huge loads, even millions of data connections, required by web-apps like Facebook, Amazon and others. Beside this, it is very easy to horizontally scale  a NoSQL database by adding more clusters, which is a problem in Relational Databases. MongoDB It is the most popular NoSQL database, used by almost every modern app. It is a free to use database, but the hosting is charged if we host on popular cloud services like MongoDB atlas. Its knowledge is a must for backend engineers, who work on the modern stack. MongoDB uses json like documents to store data. A good tutorial to learn MongoDB is - It is a proprietary database service provided by Amazon. It is quite similar to MongoDB and uses key-value pairs to store data. It is also a part of the popular AWS services. A good tutorial to learn DynamoDB is-Cassandra is an open-source and free to use NoSQL database . It takes a different approach when compared to other NoSQL databases, because we use commands like SQL, which are known as CQL (Cassandra Query Language). A good tutorial to learn Cassandra is - 3. In-memory databases The in-memory database is a database, which keeps all of the data in the RAM. This means it is the fastest among all databases.  The most popular and widely used in-memory database is Redis. Redis Redis (Remote Dictionary Server) is an in-memory database, which stores data in RAM in a json like key-value format. It keeps the data persistent by updating everything in the transaction log, because when systems are shut down their RAM is wiped clean. A good tutorial to learn Redis - StorageStoring the data is an important part of any application. Although this is mainly DevOps territory, every backend developer should know the basics for the same. We need to store the database data and also the backend code. Beside this the frontend code must also be stored somewhere. Nowadays everything is stored in the cloud, which is preferred by individuals, startups and enterprises. The two most popular cloud-based storages are – Amazon S3 Azure Blob Storage Good beginner's tutorials for both areServices and APIsThese are theoretical concepts and are implemented by various services, but a backend engineer should know them and how to use them. Restful APIs This is by far the most popular way to get data from a database. It was made more popular, with the rise of web-apps. We do GET, PUT, POST and DELETE operations to read, update, create or delete data from databases. We have earlier discussed different languages and frameworks, which have their own implementations for these operations. Microservices Architecture In microservice architecture, we divide a large and complex project into small, independent services. Each of these is responsible for a specific task and communicates with other services through simple APIs. Each service is built by a small team from the beginning, and separated by boundaries which make it easier to scale up the development effort if needed. GraphQL It is the hottest new kid in the block, which is an alternative to the Restful APIs. The problem with Restful APIs is that if you want some data stored in database, you need to get the whole data sent by the endpoint. On the other hand, with GraphQL, you get a query type language which can return only the part of the data which you require.  DevOps & DeploymentA backend engineer requires a fair bit of DevOps knowledge. So, we will next deep dive into the methodologies in DevOps. 1. Containerization & Orchestration Containers are a method of building, packaging and deploying software. They are similar to but not the same thing as virtual machines (VMs). One of the primary differences is that containers are isolated or abstracted away from the underlying operating system and infrastructure that they run on. In the simplest terms, a container includes both an application’s code and everything that code needs to run properly. Container orchestration is the automatic process of managing the work of individual containers for applications based on microservice architecture. The popular Containerization and Orchestration tools are – Kubernetes Docker Good beginner's tutorials for both are -2. DevOps DevOps is a set of practices that combine software development (Dev) and IT operations (Ops). It aims to shorten the systems development life cycle and provide continuous delivery with high software quality. The two most popular DevOps services are AWS and Azure. Both of them are cloud based and are market leaders. Both of these platforms contain a wide variety of similar services. AWS It consists of over 200 products and services for storage, database, analytics, deployment, serverless function and many more. AWS is the market leader as of now with 33% of market share. The AWS certifications are also one of the most in-demand certifications and a must for frontend engineers as well as Backend engineers. Azure Microsoft Azure is second in terms of market share of cloud-based platforms, with 18% of the market. It also consists of SaaS (Software as a Service), PaaS (Platform as a Service) and IaaS (Infrastructure as a Service) like AWS. 3. PaaS (Platform as a Service) There are several smaller players, which provide Platform as a Service and are much easier to use than services like AWS and Azure. With these services you can directly deploy your React or other web-apps, by just hosting them on GitHub and pushing the code. These services are preferred a lot by freelancers, hobbyists and small companies as they don’t require investment in learning complicated services like AWS and Azure. The three most popular PaaS services are Digital Ocean Heroku Netlify 4. Serverless Serverless computing is an execution model where the cloud provider (AWS, Azure, or Google Cloud) is responsible for executing a piece of code by dynamically allocating resources and only charging for the number of resources used to run the code. The code is typically run inside stateless containers that can be triggered by a variety of events including http requests, database events, queuing services, monitoring alerts, file uploads, scheduled events (cron jobs), etc. The code that is sent to the cloud provider for execution is usually in the form of a function. AWS Lambda It is an event-driven, serverless platform which is part of AWS. The various languages supported by AWS Lambda are Node.js, Python, Java, Go, Ruby and .NET. AWS Lambda was designed for use cases such as updates to DynamoDB tables, responding to a website click etc. After that it will “spin down” the database service, to save resources. Azure Functions They are quite similar to AWS Lambda, but are for Microsoft Azure. Azure functions have a browser-based interface to write code to respond to events generated by http requests etc. The service accepts programming languages like C#, F#, Node.js, Python, PHP and Java. Serverless Framework It is an open-source web-framework written using Node.js. The popular services like AWS Lambda, Azure functions and Google cloud functions are based on it. CI/CD A backend developer should know the popular CI/CD (Continuous Integration/Continuous deployment) tools. These tools help to automate the whole process of building, testing and deployment of applications. Github Actions  It is a freely available CI/CD pipeline, which directly integrates to your GitHub based project and can be used in variety of languages. Jenkins Jenkins is the most popular CI/CD automation tool, which helps in building, testing and deployment of applications. Jenkins was written in Java and over the years has been built to support over 1400 plugins, which extend its functionalities. Circle CI Circle CI is also a CI/CD automation tool, which is cloud based and so it is different from Jenkins. It is much easier to use than Jenkins, but has a smaller community and lower user base. SecuritySecurity is an important aspect of any application. Most applications containing user personal data, like email etc, are often targeted by hackers. OWASP The Open Web Application Security Project (or OWASP), is a non-profit organization dedicated to web application security. They have free material available on their website, making it possible for anyone to improve their web application security. Protecting Services & databases against threats Hackers target databases of popular web-apps on a regular basis to get sensitive information about their customers. This data is then sold to the highest bidder on the dark-net. When such public breaches are reported, then it's a reputation loss for the enterprise also. So, a lot of emphasis should be given to Authentication, Access, Backups, and Encryption while setting up a database. The databases should also be monitored for any suspicious activities. Besides this the API routes also need to be protected, so that the hacker cannot manipulate them. Career roles Most of the companies hire Frontend developers, Backend developers and DevOps engineers separately. This is because most of the enterprise projects are huge, in which roles and responsibilities are distributed. But there is a huge demand for Full Stack developers in the startup sector in US and India. These companies need specialists who can get the product out as soon as possible with agile and small teams. Top companies hiringAlmost every company on the planet is hiring web-developers or outsourcing the development work. Since the past decade, the demand for developers has risen exponentially. The top technology companies which hire full stack developers are Facebook, Amazon, Apple, Netflix, Google, Uber, Flipkart, Microsoft and more.  The sites of each of these companies are web-apps (excluding Apple and Microsoft), with complex frontend and backend systems. The frontend generally consists of React or Angular and the backend is a combination of various technologies. The DevOps part is also quite important in these web-apps as they handle millions of concurrent connections at once.Salaries  The salary of a beginner Frontend developer in India starts from Rs. 300,000($ 3980) per year in service-based companies to Rs. 12,00,000($ 15,971) per year in the top tech companies mentioned above. The salary of a Beginner Full-Stack developer in India starts at Rs. 4,50,000 ($ 5989) per year in service companies to Rs. 12,00,000($ 15,971) per year in top tech companies. The salary for an entry level Frontend developer in USA is $ 59,213 per year and for an entry level Full stack developer is $ 61,042 per year.Below are some sources for salaries. Top regions where there is demand There are plenty of remote and freelancing opportunities in web-development across the world. The two countries with most developers and top tech companies are USA and India. Silicon Valley, which is the San Francisco Bay Area, in Northern California, USA is the hub of technology companies.  The top city in India to start a developer job is the Silicon Valley of India – Bengaluru. The number of jobs is more than all the other cities combined and it also has a very good startup ecosystem. Almost all the big technology companies mentioned earlier and top Indian service companies are located in the city. After Bengaluru, the city where the greatest number of technology jobs are based is Hyderabad, followed by Chennai and then Pune. Entry PointsThe demand for web-developers is high and anyone with a passion for creating apps can become a web-developer. An Engineering degree is not mandatory to land a job as a web developer.  The most in-demand skill today and for the next 5 years is React and its ecosystem. So, if you know HTML, CSS, JavaScript and React, it is impossible to not get a job. Career Pathway  Most people start as an intern Front-end developer or Intern Full-Stack developer and in many cases Intern Backend developer. Many companies directly hire junior Frontend/Backend/Full-stack developers.  After that, the next step is the role of Senior Frontend/Backend/Full-stack developers. Many Frontend and Backend developers become full stack developers at this level, by learning additional technologies. Senior resources in Frontend/Backend/Full-stack can then go on to assume Team Lead roles. These people manage small teams in addition to being individual contributors.  After this a professional can become a Project manager, whose main responsibility is managing the team. Another role is that of Technical Project Manager, who manages the team and also has hands-on knowledge in Technology. The last role at this level is that of a Software Architect, who handles and designs big projects and has to look at every aspect of the technology to create the enterprise app. Generally Full-stack developers are preferred in this role, as they need to know all technologies. The highest career milestone is CTO or Chief Technology Officer, who handles all the technology teams and makes all technology decisions in a Technology company. Job SpecializationThere are some Full stack development specializations which I see nowadays in the industry. Full stack developers who work with React in the Frontend and Java in the Backend are in great demand. Similarly, developers who work with Angular in the Frontend and .NET in the backend are in great demand.How KnowledgeHut can helpAll these free resources are a great place to start your Frontend or Full-Stack journey. Beside these there are many other free resources on the internet, but they may not be organized and may not have a structured approach.  This is where KnowledgeHut can make a difference and serve as a one stop shop alternative with its comprehensive Instructor-led live classes. The courses are taught by Industry experts and are perfect for aspirants who wish to become Frontend or FullStack developers.Links for some of the popular courses by KnowledgeHut are appended below-CSS3JavaScriptReactJSNodeJSDevopsConclusion This completes our article on the Full stack developer journey by combining both the Frontend and backend roadmap. There are many people who become backend developers first by working on languages like Java and then go on to learn React to become full stack developers.  Again, many developers learn front-end development first with frameworks like React, and then become full stack developers by learning Node.JS. This path is easier for developers because both React and Node.JS use the same language which is JavaScript.We hope you have found this blog useful, and can now take the right path to become a full stack developer. Good luck on your learning journey!
9557
The 2021 Learning Path To Becoming a Full Stack We...

Full stack developer roles are among the hottest c... Read More