Search

What are the Data-types in MongoDB

In this article, you will learn about one of the leading NoSQL databases - MongoDB. You will understand the basics of MongoDB and how data is stored in a NoSQL database, but most of the article will focus on the data types that are supported by MongoDB.MongoDB is a cross-platform, document-oriented. NoSQL database. MongoDB is known for its high scalability, amazing availability and higher performance compared to a similar SQL database like MySQL.In any NoSQL database, data is stored as a set of key-value pairs. Here is an example."name":  "Knowledgehut"When we store related key-value pairs together in a set of key-value pairs, the set is known as a document. Here is an example of a document that contains data about an employee.{ "employee_name":  "John Doe", "employee_skills":  "UI Design", "employee_salary":  40000, "employee_status":  true, }Introduction to Data TypesIn the document above, you can see that we have stored multiple values for an employee. This is very similar to how we stored data within a row in a typical RDBMS. Similar documents are stored together within a collection. You can think of collections as the NoSQL equivalent of an RDBMS table with some key differences that we are not going to discuss in this article.In the above document, you can see that we have 4 different key-value pairs. The values can be of different types, for example, in this case the employee_name and employee_skills have the values of String type, employee_salary is of the number type and the employee_status is of the boolean type.  Having these (and more) data types in MongoDB allows us to store the data in a more efficient format and also perform highly efficient and robust queries on the stored data.Using the correct data type for storing the data fields in a document is crucial to the success of the database system. Here are some of the most used data types available in MongoDB.StringIntegerBooleanDoubleDateMix/Max keysArraysTimestampObjectNullSymbolRegular ExpressionsWe will have a look at all of these with examples but before that, let’s have a look at JSON and BSON to understand how MongoDB stores the data.JSON and BSONJSON stands for JavaScript Object Notation. It is a very common format used by APIs and web services to return the data to the client. This format is widely used because of its simplicity and ease of parsing. Most modern programming languages do not need an additional application layer to parse JSON data.JSON objects are simple associative containers, where the data is stored as a set of key-value pairs. In this case, a key is mapped to a value (which can be a number, string, function, or even another object).MongoDB also stores the data as JSON documents but the JSON data is binary encoded. This results in BSON. BSON simply stands for Binary JSON. BSON’s binary structure encodes type and length information, which allows it to be parsed much more quickly and therefore delivers better performance.In a nutshell, MongoDB stores data in BSON format both internally, and over the network, but that doesn’t mean you can’t think of MongoDB as a JSON database. Anything you can represent in JSON can be natively stored in MongoDB, and retrieved just as easily in JSON.Different MongoDB data typesLet’s have a look at each data type offered by MongoDB with examples and understand the best use-cases for them.String - This is one of the simplest and most used data types. String type is used to simply represent text. The strings in BSON are UTF-8 which allow us to represent most international characters in BSON strings without any problems. Here are some examples of string values in a document.{ "employee_name":  "John Doe", "employee_skills":  "UI Design", "employee_salary":  40000, "employee_status":  true, }The above document has two keys that have the value of String type. Namely, employee_name and employee_skills have the String values. These are the simplest values and are used to represent a bunch of characters.Integer - The integer data type is used to store numeric values. It can store 32-bit or 64-bit integers which depends on the server. Here is an example of an integer stored in a document.{ "employee_name":  "John Doe", "employee_skills":  "UI Design", "employee_salary":  40000, "employee_status":  true, }The key employee_salary stores a numeric value and therefore it is of the type integer.Double - The double datatype is used to store numeric values with 8 bytes (64-bit IEEE 754 floating point) floating-point. Here is an example of a document that contains a double value in the field employee_score.{ "employee_name":  "John Doe", "employee_skills":  "UI Design", "employee_score":  97.67, "employee_status":  true, }Boolean - The boolean datatype is used to store boolean (true or false) values. In the below example, you can see that the field employee_status stores the value true, hence this field is of the type boolean.{ "employee_name":  "John Doe", "employee_skills":  "UI Design", "employee_score":  97.67, "employee_status":  true, }Booleans use less storage than an integer or string and avoid any unexpected side effects of comparison.Arrays - Arrays are used to store multiple values of the same type under a single key. Here is an example of an array stored within a document.{ "employee_name":  "John Doe", "employee_skills":  ["UI Design", "Graphic Design", "2D Animation"], "employee_score":  97.67, "employee_status":  true, }In the above example, the employee_skills field contains an array of type String where each value within the array is a String.Here is another example where instead of an array of a simple type (String), documents are embedded within the array.{ "item_code": "1234-ABCD", "item_price": 49.99, "item_stock": [{ "warehouse": "Warehouse A", "qty": 1200 }, { "warehouse": "Warehouse B", "qty": 900 }], }In the above document, the field item_stock contains an array of embedded documents.Date - The date datatype is used to store date and time in Unix-time format. Unix timestamps can be easily converted to and from the JavaScript Date object. Date is a 64-bit integer that represents the number of milliseconds since the Unix epoch (Jan 1, 1970). This results in a representable date range of about 290 million years into the past and future.Here is an example of how a date is stored in a document.{ "student_name": "Bob Stan", "student_dob": ISODate("2006-02-10T10:50:42.389Z"), "student_marks": 78.98 }In the above example, the stored date can be easily converted to a readable format using JavaScript's new Date("2006-02-10T10:50:42.389Z") function. It will return the following output.Fri Feb 10 2006 16:20:42 GMT+0530 (India Standard Time)Internally, Date objects are stored as a signed 64-bit integer representing the number of milliseconds since the Unix epoch (Jan 1, 1970).Min/Max keys - Min and Max keys are both internal data types. It is used to compare a value against the lowest and highest BSON elements.Object - This data type is used to store embedded documents within a document. Let’s look at an example to understand it better.{ "item_code": "1234-ABCD", "item_price": 49.99, "item_dimensions": { "item_height": 1200, "item_width": 100, "item_depth": 900, }, "item_availability": true, }In the above example, the item_dimensions field is an embedded document as it contains its own set of key-value pairs. This field therefore is of the type Object.Timestamp - The timestamp type is a special type for internal MongoDB use and is not associated with the regular Date type. This internal timestamp type is a 64-bit value where the most significant 32 bits are seconds since the Unix epoch and the least significant 32 bits are an incrementing ordinal for operations within a given second.Here is how the timestamp value looks like in the document when it is queried.{ "item_code": "1234-ABCD", "item_price": 49.99, "item_created": Timestamp(1412180887, 1), "item_availability": true, }The timestamp data type is generally used to keep track of document creation/editing/updation times. The new Timestamp() function is used during the insertion and the server automatically adds the timestamp to the field.Null - The null datatype is used to store null or non-existent values. Here is how a field in a document with a null value would look like when queried.{ "item_code": "1234-ABCD", "item_price": 49.99, "item_color": null, "item_availability": true, }This is similar to the following document as well where the field is completely absent.{ "item_code": "1234-ABCD", "item_price": 49.99, "item_availability": true, }ObjectID - This datatype is used to store a document’s unique ID. No two documents in a collection can have the same ObjectIDs. It is a 12-byte value that contains the timestamp, a random value and an incrementing counter value as well, all combined together to generate a unique ID.Here is an example.{ "_id": "5349b4ddd2781d08c09890f3", "item_code": "1234-ABCD", "item_price": 49.99, "item_availability": true, }The _id field is automatically added for every document if you do not specify a field explicitly with the ObjectID type.Binary - This datatype is used to store binary data in a field. This data type corresponds to the Blob type in a Relational DBMS. There is, however, a limit of 16MB per document in MongoDB, so if the binary data plus other fields have a total size less than 16MB, then binary data can be embedded within the document using the Binary data type.Here is an example.{ "_id": "5349b4ddd2781d08c09890f3", "item_code": "1234-ABCD", "item_price": 49.99, "item_availability": true, "item_picture":BinData(1, "wekud3298eyx2398ey293..."), }BinData here is the base64 representation of the binary content.Undefined - This datatype is used to store the undefined value in a field. Note that MongoDB differentiates between null and undefined but the shell casts both to null. This behavior can, however, be changed.{ "item_code": "1234-ABCD", "item_price": 49.99, "item_color": undefined, "item_availability": true, }Undefined is now deprecated in MongoDB 4.4.Regular Expression - This datatype is used to store Regular Expressions or RegExs in a field. These can be used for pattern matching across different languages. Here is an example.{ "item_code": "1234-ABCD", "item_price": 49.99, "item_color": undefined, "item_prefix": /%_Y675%, }JavaScript with Scope - It is possible to store a live function in MongoDB within a field. The functions with closure can also be stored. They will bind to the scope of the MongoDB session when they're executed.In BSON, there are two different types defined for functions without closures, JavaScript and another one for functions with closures, JavaScript with Scope. JavaScript with Scope is now deprecated in MongoDB 4.4.So these are all the key and most prominent datatypes in MongoDB. BSON supports more datatypes than JSON. Some older and less often used datatypes are removed from the MongoDB support shelf and the range or support for newer types is improved with time. This is an evergreen process.

What are the Data-types in MongoDB

6K
What are the Data-types in MongoDB

In this article, you will learn about one of the leading NoSQL databases - MongoDB. You will understand the basics of MongoDB and how data is stored in a NoSQL database, but most of the article will focus on the data types that are supported by MongoDB.
MongoDB is a cross-platform, document-oriented. NoSQL database. MongoDB is known for its high scalability, amazing availability and higher performance compared to a similar SQL database like MySQL.

In any NoSQL database, data is stored as a set of key-value pairs. Here is an example.

"name":  "Knowledgehut"

When we store related key-value pairs together in a set of key-value pairs, the set is known as a document. Here is an example of a document that contains data about an employee.

{
"employee_name":  "John Doe",
"employee_skills":  "UI Design",
"employee_salary":  40000,
"employee_status":  true,
}

Introduction to Data Types

In the document above, you can see that we have stored multiple values for an employee. This is very similar to how we stored data within a row in a typical RDBMS. Similar documents are stored together within a collection. You can think of collections as the NoSQL equivalent of an RDBMS table with some key differences that we are not going to discuss in this article.

In the above document, you can see that we have 4 different key-value pairs. The values can be of different types, for example, in this case the employee_name and employee_skills have the values of String type, employee_salary is of the number type and the employee_status is of the boolean type.  

Having these (and more) data types in MongoDB allows us to store the data in a more efficient format and also perform highly efficient and robust queries on the stored data.

Using the correct data type for storing the data fields in a document is crucial to the success of the database system. Here are some of the most used data types available in MongoDB.

  • String
  • Integer
  • Boolean
  • Double
  • Date
  • Mix/Max keys
  • Arrays
  • Timestamp
  • Object
  • Null
  • Symbol
  • Regular Expressions

We will have a look at all of these with examples but before that, let’s have a look at JSON and BSON to understand how MongoDB stores the data.

JSON and BSON

JSON stands for JavaScript Object Notation. It is a very common format used by APIs and web services to return the data to the client. This format is widely used because of its simplicity and ease of parsing. Most modern programming languages do not need an additional application layer to parse JSON data.

JSON objects are simple associative containers, where the data is stored as a set of key-value pairs. In this case, a key is mapped to a value (which can be a number, string, function, or even another object).

MongoDB also stores the data as JSON documents but the JSON data is binary encoded. This results in BSON. BSON simply stands for Binary JSON. BSON’s binary structure encodes type and length information, which allows it to be parsed much more quickly and therefore delivers better performance.

In a nutshell, MongoDB stores data in BSON format both internally, and over the network, but that doesn’t mean you can’t think of MongoDB as a JSON database. Anything you can represent in JSON can be natively stored in MongoDB, and retrieved just as easily in JSON.

Different MongoDB data types

Let’s have a look at each data type offered by MongoDB with examples and understand the best use-cases for them.

  • String - This is one of the simplest and most used data types. String type is used to simply represent text. The strings in BSON are UTF-8 which allow us to represent most international characters in BSON strings without any problems. Here are some examples of string values in a document.
{
"employee_name":  "John Doe",
"employee_skills":  "UI Design",
"employee_salary":  40000,
"employee_status":  true,
}

The above document has two keys that have the value of String type. Namely, employee_name and employee_skills have the String values. These are the simplest values and are used to represent a bunch of characters.

  • Integer - The integer data type is used to store numeric values. It can store 32-bit or 64-bit integers which depends on the server. Here is an example of an integer stored in a document.
{
"employee_name":  "John Doe",
"employee_skills":  "UI Design",
"employee_salary":  40000,
"employee_status":  true,
}

The key employee_salary stores a numeric value and therefore it is of the type integer.

  • Double - The double datatype is used to store numeric values with 8 bytes (64-bit IEEE 754 floating point) floating-point. Here is an example of a document that contains a double value in the field employee_score.
{
"employee_name":  "John Doe",
"employee_skills":  "UI Design",
"employee_score":  97.67,
"employee_status":  true,
}
  • Boolean - The boolean datatype is used to store boolean (true or false) values. In the below example, you can see that the field employee_status stores the value true, hence this field is of the type boolean.
{
"employee_name":  "John Doe",
"employee_skills":  "UI Design",
"employee_score":  97.67,
"employee_status":  true,
}

Booleans use less storage than an integer or string and avoid any unexpected side effects of comparison.

  • Arrays - Arrays are used to store multiple values of the same type under a single key. Here is an example of an array stored within a document.
{
"employee_name":  "John Doe",
"employee_skills":  ["UI Design", "Graphic Design", "2D Animation"],
"employee_score":  97.67,
"employee_status":  true,
}

In the above example, the employee_skills field contains an array of type String where each value within the array is a String.

Here is another example where instead of an array of a simple type (String), documents are embedded within the array.

{
"item_code": "1234-ABCD",
"item_price": 49.99,
"item_stock": [{
"warehouse": "Warehouse A",
"qty": 1200
}, {
"warehouse": "Warehouse B",
"qty": 900
}],
}

In the above document, the field item_stock contains an array of embedded documents.

  • Date - The date datatype is used to store date and time in Unix-time format. Unix timestamps can be easily converted to and from the JavaScript Date object. Date is a 64-bit integer that represents the number of milliseconds since the Unix epoch (Jan 1, 1970). This results in a representable date range of about 290 million years into the past and future.

Here is an example of how a date is stored in a document.

{
"student_name": "Bob Stan",
"student_dob": ISODate("2006-02-10T10:50:42.389Z"),
"student_marks": 78.98
}

In the above example, the stored date can be easily converted to a readable format using JavaScript's new Date("2006-02-10T10:50:42.389Z") function. It will return the following output.

Fri Feb 10 2006 16:20:42 GMT+0530 (India Standard Time)

Internally, Date objects are stored as a signed 64-bit integer representing the number of milliseconds since the Unix epoch (Jan 1, 1970).

  • Min/Max keys - Min and Max keys are both internal data types. It is used to compare a value against the lowest and highest BSON elements.
  • Object - This data type is used to store embedded documents within a document. Let’s look at an example to understand it better.
{
"item_code": "1234-ABCD",
"item_price": 49.99,
"item_dimensions": {
"item_height": 1200,
"item_width": 100,
"item_depth": 900,
},
"item_availability": true,
}

In the above example, the item_dimensions field is an embedded document as it contains its own set of key-value pairs. This field therefore is of the type Object.

  • Timestamp - The timestamp type is a special type for internal MongoDB use and is not associated with the regular Date type. This internal timestamp type is a 64-bit value where the most significant 32 bits are seconds since the Unix epoch and the least significant 32 bits are an incrementing ordinal for operations within a given second.

Here is how the timestamp value looks like in the document when it is queried.

{
"item_code": "1234-ABCD",
"item_price": 49.99,
"item_created": Timestamp(1412180887, 1),
"item_availability": true,
}

The timestamp data type is generally used to keep track of document creation/editing/updation times. The new Timestamp() function is used during the insertion and the server automatically adds the timestamp to the field.

  • Null - The null datatype is used to store null or non-existent values. Here is how a field in a document with a null value would look like when queried.
{
"item_code": "1234-ABCD",
"item_price": 49.99,
"item_color": null,
"item_availability": true,
}

This is similar to the following document as well where the field is completely absent.

{
"item_code": "1234-ABCD",
"item_price": 49.99,
"item_availability": true,
}
  • ObjectID - This datatype is used to store a document’s unique ID. No two documents in a collection can have the same ObjectIDs. It is a 12-byte value that contains the timestamp, a random value and an incrementing counter value as well, all combined together to generate a unique ID.

Here is an example.

{
"_id": "5349b4ddd2781d08c09890f3",
"item_code": "1234-ABCD",
"item_price": 49.99,
"item_availability": true,
}

The _id field is automatically added for every document if you do not specify a field explicitly with the ObjectID type.

  • Binary - This datatype is used to store binary data in a field. This data type corresponds to the Blob type in a Relational DBMS. There is, however, a limit of 16MB per document in MongoDB, so if the binary data plus other fields have a total size less than 16MB, then binary data can be embedded within the document using the Binary data type.

Here is an example.

{
"_id": "5349b4ddd2781d08c09890f3",
"item_code": "1234-ABCD",
"item_price": 49.99,
"item_availability": true,
"item_picture":BinData(1, "wekud3298eyx2398ey293..."),
}

BinData here is the base64 representation of the binary content.

  • Undefined - This datatype is used to store the undefined value in a field. Note that MongoDB differentiates between null and undefined but the shell casts both to null. This behavior can, however, be changed.
{
"item_code": "1234-ABCD",
"item_price": 49.99,
"item_color": undefined,
"item_availability": true,
}

Undefined is now deprecated in MongoDB 4.4.

  • Regular Expression - This datatype is used to store Regular Expressions or RegExs in a field. These can be used for pattern matching across different languages. Here is an example.
{
"item_code": "1234-ABCD",
"item_price": 49.99,
"item_color": undefined,
"item_prefix": /%_Y675%,
}
  • JavaScript with Scope - It is possible to store a live function in MongoDB within a field. The functions with closure can also be stored. They will bind to the scope of the MongoDB session when they're executed.

In BSON, there are two different types defined for functions without closures, JavaScript and another one for functions with closures, JavaScript with Scope. JavaScript with Scope is now deprecated in MongoDB 4.4.

So these are all the key and most prominent datatypes in MongoDB. BSON supports more datatypes than JSON. Some older and less often used datatypes are removed from the MongoDB support shelf and the range or support for newer types is improved with time. This is an evergreen process.

KnowledgeHut

KnowledgeHut

Author

KnowledgeHut is an outcome-focused global ed-tech company. We help organizations and professionals unlock excellence through skills development. We offer training solutions under the people and process, data science, full-stack development, cybersecurity, future technologies and digital transformation verticals.
Website : https://www.knowledgehut.com

Join the Discussion

Your email address will not be published. Required fields are marked *

Suggested Blogs

How to Install Angular on Windows

In this article, you will learn how you can install Angular CLI on your Windows machine and use it to create real-life Angular Applications/Projects.As you know, Angular is popularly used to create SPAs (Single page applications). Angular is an in-demand, open-source, and modern web application development framework by Google, as compared to all other Libraries and Frameworks like ReactJs, VueJs, MithrilJs, etc.The reasons behind this are many. Angular provides you full access to use all its sub-packages like routing, Async calls, Security, etc.Introduction — What  you will learnIn this blog, we will cover the installation of Angular on Windows. By end of the article, you will understand how to -Install Node/npm on your machine.Use and install Angular CLI globally.Run Angular CLI commands.Create an initial workspace for the application.Run the Angular application in Browser.We assume you are already aware of the basics of web development including HTML, CSS, and JavaScript. (if so, please proceed otherwise we strongly recommand to learn those first before jumping into Angular).In order to continue, you must have — Windows MachineGood Internet ConnectivityWindows user account with all/admin installation privileges where you can install any packages freely.PS: While working with Angular, knowledge of TypeScript is good to have, but is not mandatory. A brief note about Node.jsThe very first thing required to install is Node.js. (If already installed, skip this part and jump to the next part of this blog.)Node.js is again an open-source and cross-platform JavaScript run-time environment that allows to execute JavaScript code outside of a browser (on the Server-side). Node.js is responsible to manage and install all npm dependencies for your Angular application. It provides the required libraries to run the Angular project. Node.js acts as a run-time environment for any Angular application which uses servers over localhost. For more information on Node.js refer to official docs.Angular requires the latest/stable version of Node.js in order to continue. Go to the official website in order to install Node.js on your machine locally.You can install any version, the current one or latest stable version, as per your need. Just click on the button and it will start downloading the package on your machine. Once downloaded click on the icons downloaded and follow the steps, till the installation is completed. To check the version installed of Node.js you can use the following command in a terminal/console window. —node -vornode — — versionNeed for NPM for Angular development As Node.js serves as Run time environment for the application, similarly NPM(node package manager) is used as a dependency management tool for any JavaScript application. NPM will allow us to install the libraries required for any Angular application; for example jQuery, angular-router, angular-http, bootstrap, and many more. You can explore a number of packages available in the npm store here.Note: Once you have installed Node.js it will automatically install NPM on your machine, and you can check the version installed using the below command. npm -vAngular CLI — What it is and complete Installation guideEarlier in the initial days of Angular, developers used to create whole architecture, webpack files, build process, etc for any project on their own from scratch which was quite a time-consuming and lengthy process. To make it easier for the developer, Angular team come up with the easy-to-use tool named Angular CLI. As the name suggests CLI (command line interface) provides a user (developer) friendly interface where you can run commands to make your development life easier and faster. Angular CLI comes with a number of commands available to use from creating a new project, to creating components, creating a routing configuration file, services, and many more.Installation Process of Angular CLITo install the Angular CLI on your machine, open the terminal window and run the following command: npm install -g @angular/cli where -g denotes that CLI is being installed globally to your machine, which means you can create or run any command of CLI anywhere on your machine. Once you run the above command CLI will be installed on your machine, and you can verify the version installed using the following command: ng version To know more about Angular CLI commands, you can refer to this blog which contains a number of commands to use. Creating a Project using Angular CLINow, let’s create our first ever Angular project using Angular CLI. Open your terminal window and type the command below on your machine.ng new hello-world Here ng is our CLI prefix, new denotes that we are creating a new project and hello-world is our project name. You can choose any name you want.After running this command you will find the full architecture of the project in the directory where you run this command. The project folder will be something like below in the image -Angular Project architecture The first file to render on running this application will be index.html which present in the src folder. src folder contains Source files for the root-level application project. assets folder contains all the static assets like images, fonts, etc. node_modules This folder is created while you run npm installby package manager (npm) and it contains all the project dependencies or any third party modules required for the project.  e2e folder contains all the source code related to Test Cases.You can customise it as per your requirements. README.md file is being used as documentation for the app. Package.json configures npm dependencies which are available for the project in the workspace along with their versions specified.  For more details about components/Service and source code you can explore this link. Run the application  one with the installation process of Angular application via CLI, it’s time to run the application locally. Angular CLI comes with complete tool-chain/commands for the development of front-end applications on your machine.Run the following command on the terminal (Navigate to the project directory if you are not in that directory).ng serve or ng serve --open The --open (or just -o) option automatically opens your browser to http://localhost:4200/. ng servecommand will serve your application on localhost server which you can check by navigating to your browser with the below URL http://localhost:4200/. You can customise the port as per your requirement. Accessing Angular Web Interfaceng serve command may take a few seconds to run your application, and once completed, you should see a web page similar to the following. And we are done!Now you can make changes in the default component’s template which is app.component.html .Also, moving forward, you can generate a number of components/service/routing files using CLI commands as per your requirements and keep learning.ConclusionIn this blog you have learnt - What is Node/NPM and how you can install and use it,  What is Angular CLI and how you can use it after installing it on your Windows machine, How to create a new Angular project from scratch using CLI commands and run it. Now that you have basic hands-on knowledge of getting started with an Angular application, you can start exploring Angular further. If you have any questions/doubts related to this blog, please let us know in the comment box below...we would be more than happy to help you out!  
9556
How to Install Angular on Windows

In this article, you will learn how you can instal... Read More

What is NPM in Node JS?

If you have ever worked with Node or any JavaScript framework, then you have already worked with NPM directly or indirectly. Let’s get into the details to understand more about NPM, and learn how to install the latest versions and manage different versions using NVM. In later articles we can discuss mastering basic tools like adding, updating or removing a package. We can also touch base on some advanced features like cache, audits, scripting and more. What is NPM ? To build an application with JavaScript or any other frameworks like Angular, React, Express or Node.js, we rely on a Package Manager to install its dependencies. This package manager is called as NPM. Note: Dependencies are the packages that are used in our projects and part of the package.json. NPM comes with Node.js and is pre-installed with Node. Package.json is the file containing the details of your project such as who created it, and what is the version of node and packages that your project is dependent on. A package is basically a set of files combined together to serve a specific interest of function. If we visit then we can search for different packages based on the functionality we need. For example, if you search for ‘date format’ to support multiple locales, we get 1171 packages (as on the date this article was written) with the topmost package being ‘moment’.  In short, if you are thinking of a functionality to build, then there is a high chance that there is a package already available in NPM for the same. As of January 2021, the current count is 1,493,231 packages.  NPM is used for building lightweight projects that can be easily shared across multiple development teams without dependencies being shared. It allows free use of resources and installs the dependencies only when needed. Install npm Node/npm can be installed on Mac, Windows or Linux as well. Let’s go through the steps involved in installing it on Mac. The approach should be similar for the other Operating systems. For Linux, refer to this. Navigate to and you will see the download section. Under the download, there is the other downloads link which will display different operating systems. Download the one specific to your Operating System. Double click on the node-v**.**.*.pkg to install the node. It is a simple wizard with straightforward steps. npm is installed with Node.js If you are wondering why we are installing node instead of NPM, this is because Node.js installs NPM along with it. This is evident from the below snapshot. How to check that you have node and npm installed The most common way to check if node or npm is installed is by looking at the terminal. Open terminal and Type in `node -v` and you should see the same version display in the installation wizard. The same applies to npm; i.e. You can type in ‘npm -v’ in terminal. On Versions - A note on versions, npm versions, Node versions and Long-Term Support When we are downloading the node.js from, we have two options; i.e. LTS and Current, both pointing to different versions, and currently on the higher version. LTS stands for ‘long time support’ and Current is the version that has the latest features, and offers support for 6 months. After 6 months, odd-numbered releases will become unsupported and even numbered ones will be given LTS status with support for 30 months. So, for all production applications, one should rely on Active LTS or Maintenance LTS releases. Current can be used for any trainings or by source contributors i.e. library authors.  Use a Node.js version manager Imagine you are working on an Enterprise application for an organization which uses a specific version of Node LTS. Also imagine that there is another app (it could be your pet project) that you are working on, for which you prefer to work on the latest version. But how can we have two different versions of Node in the same machine?  To achieve the same, we have NVM (Node.js version manager). Here is the official site for Mac and Linux users. Windows users can visit this link. Follow the installation steps to install the NVM on your machine.  To verify if NVM is installed correctly, open the terminal and type in ‘nvm --version.’ Type in ‘nvm list’ to display all the node.js versions that are installed on your machine. For now, you should be seeing only one version. Say you want to install an outdated version of Node.js, say 12. Type in ‘nvm install 12’ in terminal and it should install the 12 version of Node.js for you.  Now, type in ‘nvm list’ to see both the versions of node that are available for use. To switch to a specific version of node, type in ‘nvm use 12’. To check if it is the active one, type in ‘node -v’. Now you are good to go ahead with your project for that specific version of node.js. ConclusionNPM is one of the world's largest software registries. The Source contributors or developers are from across the world, and use npm to share or use packages. Many organizations/firms use npm for private development as well. NPM has 3 components i.e. Website, Command Line Interface and Registry.  We used the website to identify the package for ‘date format’ above. We learnt to set up private packages as well alongside public. The Command Line Interface is run from the terminal and is used by most of the developers, and the Registry is the public database of the JavaScript software.  
5636
What is NPM in Node JS?

If you have ever worked with Node or any JavaSc... Read More

Node.Js - Net Module

Node.js has a ‘net’ module which provides an asynchronous network API for creating stream-based TCP/IPC servers and clients. It can be accessed using: const net = require('net');  To create a TCP/IPC based server, we use the createServer method. var server = net.createServer(); The ‘server' object is of type net.Server. Let’s explore a few properties, events and methods on this class. First and foremost, the method needed is ‘listen’ which starts the server for listening to connections in async, firing the ‘listening’ event. server.listen(9000, () => {    console.log('opened server on port: ', 9000);  }); To find out on which address a server is running, we can use the address() method on the net.Server instance. If we need to log the port on which the server is running, then we can get this info as well without hardcoding. server.listen(9000, () => {    console.log('opened server on %j', server.address().port);  }); The first parameter of listen is the port in which the server starts listening, and a callback which gets called once it has started listening. A few of the common errors raised are:  ERR_SERVER_ALREADY_LISTEN – server is already listening and hasn’t been closed. EADDRINUSE – another server is already listening on the given port/handle/path. Whenever an error happens, an ‘error’ event is raised. We can hook to it and capture the errors accordingly. server.on('error', (e) => {    if (e.code === 'EADDRINUSE') {      console.log('Address in use, retrying...');      setTimeout(() => {        server.close();        server.listen(PORT, HOST);      }, 1000);    }  }); Whenever a client connects to this server then a 'connection' event is raised and in the callback we can get hold of the client object for communicating data. server.on("connection", (socket) => {    console.log("new client connection is made");  }); The second parameter is actually a callback which has the reference to the connection object, and the client object is of type ‘net.Socket’. To get the details like address and port, we can rely on remoteAddress, and remotePort properties respectively.   server.on("connection", (socket) => {    console.log("Client connection details - ", socket.remoteAddress + ":" + socket.remotePort);  }); Let’s assume that we are developing an application server like bot which needs to take inputs from clients and respond to the client. We can get hold of the client object and send messages to it from the server. As soon as the client is connected, we can send a sample return message on successful connection. server.on("connection", (socket) => {    console.log("Client connection details - ", socket.remoteAddress + ":" + socket.remotePort);    socket.write('SERVER: Hello! Connection successfully made.');  }); Now if there is any data being sent by client, we can capture that data on the server by subscribing to ‘data’ event on the client socket object.  socket.on('data', (data) => {    console.log(data.toString());// since data is streamed in bytes, toString is used.  }); Some of the most commonly used events on ‘net.Socket’ are data, error and close. As the names suggest, data is for listening to any data sent, error when there is an error happens and close event is raised when a connection is closed which happens once. Here is an example in server.js file: const net = require('net');  var server = net.createServer();  server.on("connection", (socket) => {    console.log("new client connection is made", socket.remoteAddress + ":" + socket.remotePort);    socket.on("data", (data) => {      console.log(data.toString());    });    socket.once("close", () => {      console.log("client connection closed.");    });    socket.on("error", (err) => {      console.log("client connection got errored out.")    });    socket.write('SERVER: Hello! Connection successfully made.');  });  server.on('error', (e) => {    if (e.code === 'EADDRINUSE') {      console.log('Address in use, retrying...');      setTimeout(() => {        server.close();        server.listen(PORT, HOST);      }, 1000);    }    else {      console.log("Server failed.")    }  });  server.listen(9000, () => {    console.log('opened server on %j', server.address().port);  }); ‘net’ module also has another class type net.BlockList. This helps in controlling or disabling the inbound or outbound traffic based on rules from any specific IP addresses, IP ranges, or IP subnets. Here is an example snippet from the documentation: const blockList = new net.BlockList();  blockList.addAddress('123.123.123.123');  blockList.addRange('10.0.0.1', '10.0.0.10');  blockList.addSubnet('8592:757c:efae:4e45::', 64, 'ipv6');  console.log(blockList.check('123.123.123.123')); // Prints: true  console.log(blockList.check('10.0.0.3')); // Prints: true  console.log(blockList.check('222.111.111.222')); // Prints: false  // IPv6 notation for IPv4 addresses works:  console.log(blockList.check('::ffff:7b7b:7b7b', 'ipv6')); // Prints: true  console.log(blockList.check('::ffff:123.123.123.123', 'ipv6')); // Prints: true Now that we have the server up and running, we can build a client to connect to the server and start sending bi-directional data. This client could be another node.js application, java/c# application working with TCP sockets, asp.net MVC application talking to node.js TCP server or any other client application. But that client application should have TCP based communication mechanism support. Since we are talking about ‘net’ module, let’s build the client application as well using net module. Moreover, it supports TCP based communication as well. 'net’ module has a factory function called ‘createConnection’ which immediately creates a socket and establishes a connection with the server running on the specified port.  Let's create another client.js file and create a connection. const net = require('net');  const client = net.createConnection({ port: 9000 }, () => {    console.log('CLIENT: I connected to the server.');  }); The first parameter contains the details of the server. Since we are running the server locally, providing the port number would suffice for us as the host default address is localhost for TCP connections. The second parameter is the callback called once the connection is made successfully with the server. The returned value is of type net.Socket which we have learnt about earlier. Let’s hook to ‘data’ event and console log the information sent by the server. client.on('data', (data) => {    console.log(data.toString());    client.end();  }); Here we are not persisting the TCP connection and ending it once we receive a message from the server. We can subscribe to close event and handle any clean up needed. client.on('end', () => {    console.log('CLIENT: I disconnected from the server.');  }) The output on the client terminal has to be:  CLIENT: I connected to the server.  SERVER: Hello! This is server speaking.  CLIENT: I disconnected from the server. Output on server terminal will be: new client connection is made ::ffff:127.0.0.1:51680  CLIENT: Hello this is client!  client connection closed. In case we want to continue the client instance till the server is alive, we can comment out the ‘client.end()’ call. Any message in the terminal can be processed and sent to the server. For reading the text from terminal we use the readline module. Here is a complete example: const net = require('net');  const readline = require('readline');  const rl = readline.createInterface({    input: process.stdin,    output: process.stdout  });  const client = net.createConnection({ port: 9000 }, () => {  console.log('CLIENT: I connected to the server.');    client.write('CLIENT: Hello this is client!');  });  client.on('data', (data) => {    console.log(data.toString());    //client.end();  });  client.on('end', () => {    console.log('CLIENT: I disconnected from the server.');  })  rl.on('line', (input) => {    client.write(`CLIENT: ${input}`);  }); Both client and server now can communicate. When we type any text in client terminal, that is communicated to the server, and the server can respond back to the client via terminal.  ConclusionWebsockets help in creating a full-duplex connection for sending messages from client to server and vice-versa. Some of the real-time use cases that you may be familiar with are chat apps, IoT devices and so on. The Node.js net module helps you to create a server application easily, which can communicate to any type of client application like a web browser, mobile app, IoT device, Node.js client, or anything that knows TCP where the messaging need is bi-directional with streams. ‘net’ module can be used to communicate among child processes within a node.js server as well. 
5630
Node.Js - Net Module

Node.js has a ‘net’ module which provide... Read More