Search

Back Up, Restore, and Migrate a MongoDB Database

Popular among both enterprises and startups, MongoDB is a database that is perfectly suited for web-apps that need to scale once the user base increases. MongoDB is different from traditional relational databases because it uses json like objects to store data, instead of tables in relational databases. In this post, we will learn to backup and restore a MongoDB database. In all software products there is an import and export feature, which in database terms, deals with human-readable format. On the other hand, the backup and restore operations use MongoDB specific data, which preserve the MongoDB attributes.  So, when migrating the database, we should prefer backup and restore over import and export. But we should also keep in mind that our source and target systems are compatible, which means that both should be Windows or both should be a Linux based system like Ubuntu/Mac. Prerequisites We are using Windows 10 in this tutorial. Please make sure you have downloaded the MongoDB Community Server and installed in it. It is a very easy setup and you will find lot of good articles on the internet detailing this out. Please ensure that you have added it in the Environment variable in your PC. Backup Considerations In a production environment, backups act as a snapshot of the database at a certain point. Large and complex databases do fail or can be hacked. If that happens, we can use the last backup file to restore the database to the point, before it failed. These are some of the factors which should be taken into consideration when doing a recovery.  1. Recovery Point Objective We should know the objective of the recovery point, which means how much data we are willing to lose during a backup and restoration. A continuous backup is preferred for critical data like bank information and backups should be taken several times during the day. On the other hand, if the data doesn’t change frequently, then the backup can be taken every 6 months.  2. Recovery Time ObjectiveThis tells how quickly the restoration can be done. During restoration the application will be down for some time; and this downtime should be minimized, or else customer relationships will be lost.  3. Database and Snapshot IsolationThis refers to the distance between the primary database server and the backup server. If they are close enough i.e., in the same building, then the recovery time reduces. However, in the event of a physical event such as a fire, there is a likelihood of it been destroyed with the primary database.   4. Restoration Process We should always test our backups in test servers to see if they will work, in case a restoration is required.  5. Available Storage Backup of database generally takes a lot of space and in most cases, it will never be required. So, we should try to minimize the space taken on the disk, by archiving the database into a zip file.  6. Complexity of DeploymentThe backup strategy should be easy to set and should be automated, so that we don’t have to remember to take the backup after regular intervals. Understanding the Basics The first thing that we should know is that MongoDB uses json and bson(binary json) formats for storing data. So, people coming from a JavaScript background can relate to objects for json, which have a key-value pair. Also, json is the preferred format in which we receive or send data to an API endpoint. You can check the json data of a MongoDB database in any tool or online editors. Even the famous Windows application Notepad++ has a json viewer. An example of a json document looks like below. As we can see from the above example, json is very convenient to work with, especially for developers.  But it doesn’t support all the data types available in bson. So, for backup and restoring, we should use binary bson. The second thing to keep in mind is that MongoDB automatically creates databases and collection names if they don’t exist during restore operations. Third, since MongoDB is a document-based database, in many user cases we store large amounts of data in one collection, such as the whole post of an article. MongoDB is also used extensively in large databases and big data. So, reading and inserting the data can consume a lot of CPU, memory and disk space. We should always run the backups during the non-peak hours like night. As already mentioned earlier, we can use import and export functions for backup and restoration of MongoDB databases, but we should use commands like mongodump and mongorestore to backup and restore respectively. MongoDB backup We will first cover backing up the MongoDB database. For this we use the mongodump command.  First open the Windows command prompt and go to the location in which MongoDB is installed. If you have chosen the default setting, while installing MongoDB though the pop-up it will be installed in a location like C:\Program Files\MongoDB\Server\4.4\bin The version number only will change if you are reading this blog in the future. Also, note that it’s better to run the command prompt in the Admin mode. So, once we open the command prompt, we need to change the directory to MongoDB bin folder by giving the below command. cd C:\Program Files\MongoDB\Server\4.4\binNow, enter mongod and press enter. It will show some json text.Now, we can backup to any location. For this post I am backing up on my Desktop in a Backup folder, which I have created through the command line.Now, we have to run mongodump command, but it should be also present in our MongoDB bin folder. If it is not present, we need to download it from and install it. After this, copy the entire exe files from the download to the MongoDB bin folder. MongoDB Backup with no option Now, run the mongodump command from the bin directory. Here, we are not giving any argument so the backup of the whole database will be taken in the same bin directory.MongoDB Backup to an output directory Now, run the mongodump command from the bin directory. Here, the argument –out specifies the directory in which the data backup will be maintained. In our case we are giving the Backup folder in the  Desktop, which we have created earlier. mongodump --out C:\Users\pc\Desktop\Backup Now, go to the desktop and you can find the backup that has been created in our Backup folder.  MongoDB Backup a specific database Now, we can also backup only a database in mongodump using the –db option. I have an example database, so to backup only that I will use the below command. mongodump --db example --out C:\Users\pc\Desktop\Backup As, you can see in the below output only the example database was backed up. MongoDB Backup a specific collection Now, if we want to only backup a specific collection, we need to use the –collection option and give the collection name. Also, note that the database name is mandatory in this case, as mongodb needs to know about the database to search for the collection. I have a products collection within the example database, so to backup only that I will use the below command. mongodump --db example --out C:\Users\pc\Desktop\Backup –collection products As, you can see in the below output only the products collection from example database was backed up. MongoDB Backup from remote MongoDB instances We can get the backup from remote mongodb instances also. I have a lot of MongoDB databases for my personal projects on MongoDB atlas, which is the free to use Cloud database for MongoDB. To get a backup of remote databases, we have to use the connection string with –uri parameter. I used the below command. mongodump --uri "mongodb+srv://xxxx:xxxxxxxxxxx@cluster0.suvl2.mongodb.net/xxxxxDB?retryWrites=true&w=majority" --out C:\Users\pc\Desktop\Backup You can see in the below output the backup of the remote instance. MongoDB Backup procedures We should try to make the backup procedure as automated as possible. One of the best ways is to use a cron job, so that it can run every day. As, discussed earlier it is best to run the backup in the night when the database has the least load.  Setting up a cron job is easier on a Linux or a Mac because the Windows equivalent of it is not good. Alternatively, you can do install mongodb in WSL2 for Windows which supports Ubuntu.  Suppose, on a Linux host which has a mongoDB instance running, you want to run the backup at 04:04 am daily. For this in the terminal, open the cron editor by running the below command in the terminal. sudo crontab –e Now, in the cron editor, you need to add a command like below for our case. 4 4 * * * mongodump --out /var/backups/mongobackups/`date +"%m-%d-%y"`Restoring and migrating a MongoDB database When we restore the MongoDB database from a backup, we will be able to take the exact copy of the MongoDB information, including the indexes. We restore MongoDB by using the command mongorestore, which works only with the binary backup produced by mongodump. Now, we have taken the backup of example database earlier and it is in our Backup folder. We will use the below command to restore it. In the arguments we will specify the name of the database first with –db option. After that with –drop, we make sure that the example database is first dropped. And in the final argument, we specify the path of our backup. mongorestore --db example --drop C:\Users\pc\Desktop\Backup\example Now, if we check in terminal, we have our example database restored properly. Conclusion In this article, we have learned about MongoDB backup and restore. We have learned the different options for the backups, and why and when backups are required. Keep learning! 

Back Up, Restore, and Migrate a MongoDB Database

6K
Back Up, Restore, and Migrate a MongoDB Database

Popular among both enterprises and startups, MongoDB is a database that is perfectly suited for web-apps that need to scale once the user base increases. MongoDB is different from traditional relational databases because it uses json like objects to store data, instead of tables in relational databases. 

In this post, we will learn to backup and restore a MongoDB database. In all software products there is an import and export feature, which in database terms, deals with human-readable format. On the other hand, the backup and restore operations use MongoDB specific data, which preserve the MongoDB attributes.  

So, when migrating the database, we should prefer backup and restore over import and export. But we should also keep in mind that our source and target systems are compatible, which means that both should be Windows or both should be a Linux based system like Ubuntu/Mac. 

Prerequisites 

We are using Windows 10 in this tutorial. Please make sure you have downloaded the MongoDB Community Server and installed in it. It is a very easy setup and you will find lot of good articles on the internet detailing this outPlease ensure that you have added it in the Environment variable in your PC. 

Backup Considerations 

In a production environment, backups act as a snapshot of the database at a certain point. Large and complex databases do fail or can be hacked. If that happens, we can use the last backup file to restore the database to the point, before it failed. These are some of the factors which should be taken into consideration when doing a recovery. 

 1. Recovery Point Objective 

We should know the objective of the recovery point, which means how much data we are willing to lose during a backup and restoration. A continuous backup is preferred for critical data like bank information and backups should be taken several times during the day. On the other hand, if the data doesn’t change frequently, then the backup can be taken every 6 months. 

 2. Recovery Time Objective

This tells how quickly the restoration can be done. During restoration the application will be down for some time; and this downtime should be minimized, or else customer relationships will be lost. 

 3. Database and Snapshot Isolation

This refers to the distance between the primary database server and the backup server. If they are close enough i.e., in the same building, then the recovery time reduces. However, in the event of a physical event such as a fire, there is a likelihood of it been destroyed with the primary database.  

 4. Restoration Process 

We should always test our backups in test servers to see if they will work, in case a restoration is required. 

 5. Available Storage 

Backup of database generally takes a lot of space and in most cases, it will never be required. So, we should try to minimize the space taken on the disk, by archiving the database into a zip file. 

 6. Complexity of Deployment

The backup strategy should be easy to set and should be automated, so that we don’t have to remember to take the backup after regular intervals. 

Understanding the Basics 

The first thing that we should know is that MongoDB uses json and bson(binary json) formats for storing data. So, people coming from a JavaScript background can relate to objects for json, which have a key-value pair. Also, json is the preferred format in which we receive or send data to an API endpoint. 

You can check the json data of a MongoDB database in any tool or online editors. Even the famous Windows application Notepad++ has a json viewer. 

An example of a json document looks like below. Back Up, Restore, and Migrate a MongoDB Database

As we can see from the above example, json is very convenient to work with, especially for developers.  But it doesn’t support all the data types available in bson. So, for backup and restoring, we should use binary bson. 

The second thing to keep in mind is that MongoDB automatically creates databases and collection names if they don’t exist during restore operations. 

Third, since MongoDB is a document-based database, in many user cases we store large amounts of data in one collection, such as the whole post of an article. MongoDB is also used extensively in large databases and big data. So, reading and inserting the data can consume a lot of CPU, memory and disk space. We should always run the backups during the non-peak hours like night. 

As already mentioned earlier, we can use import and export functions for backup and restoration of MongoDB databases, but we should use commands like mongodump and mongorestore to backup and restore respectively. 

MongoDB backup 

We will first cover backing up the MongoDB database. For this we use the mongodump command.  

First open the Windows command prompt and go to the location in which MongoDB is installed. If you have chosen the default setting, while installing MongoDB though the pop-up it will be installed in a location like 

C:\Program Files\MongoDB\Server\4.4\bin 

The version number only will change if you are reading this blog in the future. Also, note that it’s better to run the command prompt in the Admin mode. So, once we open the command prompt, we need to change the directory to MongoDB bin folder by giving the below command. 

cd C:\Program Files\MongoDB\Server\4.4\bin

Back Up, Restore, and Migrate a MongoDB Database

Now, enter mongod and press enter. It will show some json text.

Back Up, Restore, and Migrate a MongoDB Database

Now, we can backup to any location. For this post I am backing up on my Desktop in a Backup folder, which I have created through the command line.Back Up, Restore, and Migrate a MongoDB Database

Now, we have to run mongodump command, but it should be also present in our MongoDB bin folder. If it is not present, we need to download it from and install it. Back Up, Restore, and Migrate a MongoDB DataBack Up, Restore, and Migrate a MongoDB Databasebase


After this, copy the entire exe files from the download to the MongoDB bin folder. 

Back Up, Restore, and Migrate a MongoDB Database

MongoDB Backup with no option 

Now, run the mongodump command from the bin directory. Here, we are not giving any argument so the backup of the whole database will be taken in the same bin directory.Back Up, Restore, and Migrate a MongoDB Database

MongoDB Backup to an output directory 

Now, run the mongodump command from the bin directory. Here, the argument –out specifies the directory in which the data backup will be maintained. In our case we are giving the Backup folder in the  Desktop, which we have created earlier. 

mongodump --out C:\Users\pc\Desktop\Backup 

Back Up, Restore, and Migrate a MongoDB Database

Now, go to the desktop and you can find the backup that has been created in our Backup folder.  Back Up, Restore, and Migrate a MongoDB Database

MongoDB Backup a specific database 

Now, we can also backup only a database in mongodump using the –db option. I have an example database, so to backup only that I will use the below command. 

mongodump --db example --out C:\Users\pc\Desktop\Backup 

As, you can see in the below output only the example database was backed up. Back Up, Restore, and Migrate a MongoDB Database

MongoDB Backup a specific collection 

Now, if we want to only backup a specific collection, we need to use the –collection option and give the collection name. Also, note that the database name is mandatory in this case, as mongodb needs to know about the database to search for the collection. I have a products collection within the example database, so to backup only that I will use the below command. 

mongodump --db example --out C:\Users\pc\Desktop\Backup –collection products 

As, you can see in the below output only the products collection from example database was backed up. Back Up, Restore, and Migrate a MongoDB Database

MongoDB Backup from remote MongoDB instances 

We can get the backup from remote mongodb instances also. I have a lot of MongoDB databases for my personal projects on MongoDB atlas, which is the free to use Cloud database for MongoDB. To get a backup of remote databases, we have to use the connection string with –uri parameter. I used the below command. 

mongodump --uri "mongodb+srv://xxxx:xxxxxxxxxxx@cluster0.suvl2.mongodb.net/xxxxxDB?retryWrites=true&w=majority" --out C:\Users\pc\Desktop\Backup 

You can see in the below output the backup of the remote instance. Back Up, Restore, and Migrate a MongoDB Database

MongoDB Backup procedures 

We should try to make the backup procedure as automated as possible. One of the best ways is to use a cron job, so that it can run every day. As, discussed earlier it is best to run the backup in the night when the database has the least load.  

Setting up a cron job is easier on a Linux or a Mac because the Windows equivalent of it is not good. Alternatively, you can do install mongodb in WSL2 for Windows which supports Ubuntu 

Suppose, on a Linux host which has a mongoDB instance running, you want to run the backup at 04:04 am daily. For this in the terminal, open the cron editor by running the below command in the terminal. 

sudo crontab –e 

Now, in the cron editor, you need to add command like below for our case. 

4 * * * mongodump --out /var/backups/mongobackups/`date +"%m-%d-%y"`

Restoring and migrating a MongoDB database 

When we restore the MongoDB database from a backup, we will be able to take the exact copy of the MongoDB information, including the indexes. We restore MongoDB by using the command mongorestore, which works only with the binary backup produced by mongodump. 

Now, we have taken the backup of example database earlier and it is in our Backup folder. We will use the below command to restore it. In the arguments we will specify the name of the database first with –db option. After that with –drop, we make sure that the example database is first dropped. And in the final argument, we specify the path of our backup. 

mongorestore --db example --drop C:\Users\pc\Desktop\Backup\example 

Now, if we check in terminal, we have our example database restored properly. Back Up, Restore, and Migrate a MongoDB Database

Conclusion 

In this article, we have learned about MongoDB backup and restore. We have learned the different options for the backups, and why and when backups are required. Keep learning! 

Nabendu

Nabendu Biswas

Author

Nabendu Biswas is a Full Stack JavaScript Developer, who has been working in the IT industry for the past 16 years and has worked for world’s top development firms, and Investment banks. He is a passionate tech blogger. He is also a tech youtuber and loves to teach people JavaScript. He is also an Apress author with three Gatsby books published. 

Join the Discussion

Your email address will not be published. Required fields are marked *

Suggested Blogs

How to Work With Forms In JavaScript

Forms also referred as web forms are a very important part of front end web application development for sake of interaction with users. Most commonly, forms are used to collect the data from users or provide a provision for user to control the user interface. Forms are great potential assets if correctly used in building an interactive web application. We would be touch basing some of the essential aspects of them like HTML structure, styling form controls, events, data validation and submitting data to server.Understanding forms in detail needs expertise in other areas than just HTML like styling form controls (CSS), scripting to validate or create custom controls (JavaScript).We would be referring or using libraries like Jquery (for document traversal, manipulation etc) and parsley (form validation library) to build better forms.A typical form’s HTML is made of HTML elements called as form controls like single or multiline text fields, dropdowns, checkboxes, button etc mostly created using element with specific type being set on Type attribute. These form controls can be programmed to add some validations to support specific values based on constraints set on them. These controls can be enriched to support accessibility for enabling the interaction for less privileged users.Let’s create a simple html page to build a form.           Learning Forms       All forms have to start with element which is container having the form fields user would interact with. All attributes of element are optional but for programming forms to capture data we need at least ‘action’ and ‘method’ attributes.action – is basically the URL where the form fields data would be sent to.method – corresponds to the HTTP method to submit the form data. Possible HTTP method names which can be set as values are post and get. And another value dialog is set when form is imbedded inside a .Note: Both formaction and formmethod can be overridden by button, input with type submit elements which we will learn as we go forward.Refer to this link to know more about form attributes.Let’s add a form element to our body with action (“”) and method(“get”). This implies that form will send a GET request to the current URL. If it is post then it would be a POST request to the URL in action. Add few fields to form say name, email and a submit button using with type being specified as text, email and submit respectively.Note: The tag is an empty element, meaning that it doesn't need a closing tag. Value attribute can be populated to set the default value.          Enter your name:                    Enter your email:                       Save and open the html in chrome or your preferred browser. Clicking on ‘Click me!’ should send a http get call with empty name and email.Note: We can use instead of with type as submit. The difference is that button can contain HTML content allowing to create a complex button whereas input allows only plain text.Let’s understand the Sending of form data.If we observer all the form fields again, we have added an attribute called ‘name’. This property is important to inform that which data is associated with which form field i.e. name/value pairs. Try adding some data to our fields rendering in html (say myName and first.last@email.com) and click submit button. You should see the data being sent as query parameters in the browser URL.?name=myName&email=first.last@email.com.Change the Form method value to POST instead of GET and send the submitted data by clicking the ‘Click me!’ button. You should be seeing Form Data being sent but the browser URL will not get update.name: myName email: first.last@email.comAll this while, we have our action method being set as empty. Replace this with another URL on server side say ‘/captureFormData’. Now on clicking submit button the data should be received by the script at ‘/captureFormData’ with key/value items contained in the HTTP request object.Note that each server-side language like Node.js, C# etc have their own way of handling the submitted form data. And this blog would not cover those topics and it is beyond the scope.Let’s refine our basic form structure with help of other HTML elements like , , etc. Though we used few of them in basic example. Let’s go little deep on them.Note: Nesting of form inside another form is unacceptable as it might result in unpredictable behavior. is a convenient way of grouping for sake of styling and semantic purpose. This control can be associated with so that some assistive technologies can read this legend and associate it with the controls inside the . Let’s understand this will an example:         Interested programming language                             JavaScript                                     CSharp                                     Java               When reading the above form by any screen readers, it will read as “Interested programming language JavaScript” for the first radio, “Interested programming language CSharp” and “Interested programming language Java” for second and third radio.Imagine if you have a long form with multiple fields. It would help to improve the usability if we can categorize/section them with the help of . It would even help to improve the accessibility of forms.Talking about accessibility, with the associated correctly with the via its for attribute (which contains the element's id attribute), a screenreader will read out something like "name, edit text" for below one.Enter your name: Another advantage of having label associated with input of type text, radio etc is they are clickable too.  If you click on a label then the associated input control will get the focus. If the input control is of type checkbox or radio, clicking on label will select the check box and radio. This will be useful as clickable area of checkbox or radio is small and having label gives provision to select it easily.Note: We can always associate multiple labels to a single input control but it is not a good idea as it will impact the accessibility and assistive technologies. along with can be used to separate the functionality in a form and group the same purpose elements like radio buttons.Here is an example of the same.               Contact information                   Title                                                                             Mr                                                                                                 Mrs                                                                               Name:                                                           E-mail:                                                           Password:                                                 Additional information                               Social type:                                 LinkedIn             Twitter             Instagram                                                 Phone number:                                                           Submit                   Every time you like to create an HTML form you need to start using element and  nesting all the content controls inside it. Most of the assistive technologies and browser plugins can help to discover elements and implement special hooks to make them easier to use.We have already some of the form elements like , , , , , and . Other common input types are button, checkbox, file, hidden, image, password, radio, reset, submit, and text.Input types.Attributes of Input.Few attributes on element help in validating the data like required, max, maxlength, min, minlength, multiple, pattern, step etc based on their respective type.Also other attributes on of type submit/image like formaction, formmethod, formnovalidate, formenctype etc helps in overriding the form level methods.ValidationBefore submitting the data to the server, it is important to perform some client side validation to avoid unwanted round trips. Client-side validation is needed but it is not a replacement to the server side validation. Advantage of having client side validation is to capture the invalid data and fix it immediately.Some of the important and popular checks which are most commonly used on client areField requiredSpecific data formatEnter valid email addressPassword and more…Let’s build a form with the above validation checks.                         Do you have experience in programming ?*                     Yes           No                             How many years of experience you have ?                                     What's your programming language?*                           TypeScript           Java           CSharp           Ruby           Go           Swift                             What's your company e-mail address?                             Cover letter                             Submit       Say, if we enter an value which is more than 40 in experience field. We should see an inbuilt error as shown below:All these validations and notifications are coming out of the box. Thanks to inbuilt functionality in control. Let’s see how we can perform validation of forms using JavaScript and take control of look and feel of error message.Most browsers support constraint validation API by providing few validation properties on HTML elements like , , , etc.validationMessage: we can customize this message if the control value failed validation otherwise it will return an empty string. It is dependent on other constraint i.e. willValidate and isValid.willValidate: If element is validated then it will be true otherwise false.validity: is the validity state of the element and it is dependent on other properties likepatternMatch for specified pattern attribute,tooLong and tooShort are for string fields based on maxLength and minLengthrangeOverflow and rangeUnderflow for numeric fields based on max and min attributestypeMatch for fields which are based on email or url.valid if all the validation constraints are metvalueMissing if the field is set as required.Along with properties, we do also have methods to perform validation like checkValidity() which returns true or false and setCustomValidity(message) is to set the message if the element is considered invalid. Also if the element is invalid then checkValidity will raise an event called invalid Event.Let’s create a simple form and customize the validation message.       Please enter an email address:             Submit     Add a script tag and customize the message as shown below:     const email = document.getElementById("mail");     email.addEventListener("input", function (event) {       if (email.validity.typeMismatch) {         email.setCustomValidity("I am expecting an e-mail address!");       } else {         email.setCustomValidity("");       }     });   Here we are listening to the input event on email field and checking if the validity on the control is valid or not and based on that we are setting the custom message.Here are we relying on inbuilt validation method. Let’s disable the validation at form level by with the help of ‘novalidate’ and take control over validation. This would mean the browser will not perform auto check on validation before sending the data. But still we have access to constraint validation API to perform validation ourself.Refine the above form to add few addition validation like required and minLength etc.               Please enter an email address:                             Submit     Let’s update the script to handle the validation     const form  = document.getElementsByTagName('form')[0];     const email = document.getElementById('mail');     const emailError = document.querySelector('#mail + span.error');     email.addEventListener('input', function (event) {       // Each time the user types something, we check if the form fields are valid.       if (email.validity.valid) {         // In case there is an error message visible, if the field is valid, we remove the error message.         emailError.textContent = ''; // Reset the content of the message         emailError.className = 'error'; // Reset the visual state of the message       } else {         // If there is still an error, show the correct error         showError();       }     });     form.addEventListener('submit', function (event) {       // if the email field is valid, we let the form submit       if(!email.validity.valid) {         // If it isn't, we display an appropriate error message         showError();         // Then we prevent the form from being sent by cancelling the event         event.preventDefault();       }     });     function showError() {       if(email.validity.valueMissing) {         // If the field is empty display the following error message.         emailError.textContent = 'You need to enter an e-mail address.';       } else if(email.validity.typeMismatch) {         // If the field doesn't contain an email address display the following error message.         emailError.textContent = 'Invalid value is entered, expected an e-mail address.';       } else if(email.validity.tooShort) {         // If the data is too short display the following error message.         emailError.textContent = `Email should be at least ${ email.minLength } characters; you entered ${ email.value.length }.`;       }       // Set the styling appropriately       emailError.className = 'error active';     } Reload the HTML and try entering an invalid email address, the corresponding error message should be displayed.Note: In the current scope of this blog, we are not working on styling.Is it possible to validate forms without built in APIs ? Let’s see with the same example.We would consider the same form again but have lot of functionality in                           Please enter an email address:                                             Submit           const form  = document.getElementsByTagName('form')[0];     const email = document.getElementById('mail');     let error = email.nextElementSibling;     const emailRegExp = /^[a-zA-Z0-9.!#$%&'*+/=?^_`{|}~-]+@[a-zA-Z0-9-]+(?:\.[a-zA-Z0-9-]+)*$/;     function addEvent(element, event, callback) {       let previousEventCallBack = element["on"+event];       element["on"+event] = function (e) {         const output = callback(e);         // A callback that returns `false` stops the callback chain and interrupts the execution of the event callback.         if (output === false) return false;         if (typeof previousEventCallBack === 'function') {           output = previousEventCallBack(e);           if(output === false) return false;         }       }     };     // Now we can rebuild our validation constraint. Because we do not rely on CSS pseudo-class, we have to explicitly set the valid/invalid class on our email field     addEvent(window, "load", function () {       // Here, we test if the field is empty (remember, the field is not required)       // If it is not, we check if its content is a well-formed e-mail address.       const test = email.value.length === 0 || emailRegExp.test(email.value);       email.className = test ? "valid" : "invalid";     });     // This defines what happens when the user types in the fiel     addEvent(email, "input", function () {       const test = email.value.length === 0 || emailRegExp.test(email.value);       if (test) {         email.className = "valid";         error.textContent = "";         error.className = "error";       } else {         email.className = "invalid";       }     });     // This defines what happens when the user tries to submit the data     addEvent(form, "submit", function () {       const test = email.value.length === 0 || emailRegExp.test(email.value);       if (!test) {         email.className = "invalid";         error.textContent = "Expecting an e-mail";         error.className = "error active";         return false;       } else {         email.className = "valid";         error.textContent = "";         error.className = "error";       }     });   On refreshing the page, the output with invalid email address should be displayed as shown below.In real time applications, we can rely on existing libraries like Parsley along with JQuery which would ease our life by taking away lot of complexity.Overview of Parsley:Parsley is a front-end javascript validation library which helps to give proper feedback to user on submission of form. As mentioned earlier, it is not a replacement of server side validation. Parsley library helps us to define our own validation.Parsley uses a DOM API namely ‘data-parsley-’ prefix on the existing properties. For example if we want to add this on a property say ‘sample’ then we would add as [data-parsley-sample=’value’]. This will allow us to configure pretty much everything without any configuration or custom function.There is no specific installation process but adding the corresponding script tags will enable the validation. Parsley is relied on Jquery so it has to be included as well.             ...                 $('#form').parsley();     Assumption is that we have downloaded the Jquery and Parsley minified librarie and added it to our working directory. Otherwise we can refer to CDN location as shown below.   Adding attribute ‘data-parsley-validate’ to each form will allow us to validate. And “$(‘#form’).parsley()” will manually bind Parsley to your forms.Let’s understand further by configuring the attributes via JavaScript. For which, lets add two input fields inside the form element.                 Also let’s update the content to perform some pre-defined validation based on attributes.       var instance = $('#first').parsley();       console.log(instance.isValid()); // maxlength is 42, so field is valid       $('#first').attr('data-parsley-maxlength', 4);       console.log(instance.isValid()); // No longer valid, as maxlength is 4       // You can access and override options in javascript too:       instance.options.maxlength++;       console.log(instance.isValid()); // Back to being valid, as maxlength is 5       // Alternatively, the options can be specified as:       var otherInstance = $('#second').parsley({         maxlength: 10       });       console.log(otherInstance.options);     In the console.log, we should see thistrue false true {maxlength: 10}Options are inherited from the global level to form level and further to field. So if we set the options at global level then the same can be observed at field level.   Parsley.options.maxlength = 42; // maxlength of 42 is declared at global level var formInstance = $('form').parsley(); var field = $('input').parsley(); console.log(field.options.maxlength); // Shows that maxlength is 42 inherited from global Parsley.options.maxlength = 30; console.log(field.options.maxlength); // Shows that maxlength is automatically 30 formInstance.options.maxlength++; console.log(field.options.maxlength); // Shows that maxlength is automatically 31We can also add our own custom validations. Let understand this with an example.                     window.Parsley.addValidator('multipleOf', {         requirementType: 'integer',         validateNumber: function(value, requirement) {           return 0 === value % requirement;         },         messages: {           en: 'This value should be a multiple of %s',         }       });     Here we are adding a new attribute namely ‘data-parsley-multiple-of’ which takes only numeric values which are multiples of 3.In window.Parsley, we added a new validator with name ‘multiple-of’ with an object containing few important properties like ‘requirementType’, ‘validateNumber’ and ‘messages’ to be shown. This properties helps the library to check if the input value is valid or not.Similar to validateNumber, other properties are also there for different types like validateString, validateDate and validateMultiple.Also for requirementType, we have different options like string, number, date, regexp, boolean etc.Messages by default has English format, to support multiple locales we need to add the specific localization and also add specific locale.Events: Parsley triggers events that allows ParsleyUI to work and for performance reasons they don’t rely on JQuery events but the usage is similar to JQuery i.e. parsley events will also bubble up like JQuery events. For example, if a field is validated then the event ‘field:validate’ will be triggred on the field instance then on to form instance and finally to the window.Parsley.$('#some-input').parsley().on('field:success', function() {         // In here, `this` is the parlsey instance of #some-input       });       window.Parsley.on('field:error', function() {         // This global callback will be called for any field that fails validation.         console.log('Validation failed for: ', this.$element);       });Many times, we need some validation based on the response from server. Parsley provides an attributes i.e. data-parsley-remote and data-parsley-remote-validator to perform the same.Let’s consider this HTMLLet’s add the async validator on the window.Parsley object.window.Parsley.addAsyncValidator('customValidator', function (xhr) {           console.log(this.$element); // jQuery Object[ input[name="q"] ]           return 404 === xhr.status;         }, 'customURL');Parsley is a very useful and powerful JavaScript form frontend validation library.Note: For developers building react based web applications, they can rely on FORMIK which is most popular library for building forms in React and React Native.ConclusionForms are important in HTML and it was needed and still needed now. is an html tag that allow us to perform HTTP methods like GET/POST operation without writing any code in JavaScript. Form defines an boundary to identify all set of the form field elements to be submitted to the server. For example, if we perform an enter key or clicking on submit button , the agent triggers form submission data based on each form field value to the server based on the action URL on the form.Before HTML5, all the elements are expected to be part of the to send the data to server. In HTML5, they maintained the backward compatibility and also enhanced the capabilities who may want to use AJAX and don’t want to rely on default behaviours i.e. they have enabled designers who expect more flexibility in having their form elements outside the form and still maintain the connections with the form. 
1681
How to Work With Forms In JavaScript

Forms also referred as web forms are a very import... Read More

Introduction to Web APIs in JavaScript

Before we talk about APIs in JavaScript and how to use them, we need to know what exactly we mean by APIs. API stands for Application Programming Interface and is a concept that is not limited or specific to JavaScript, but is used in almost all web application languages. Being a web developer, it is expected that you know about API, so let’s try to understand the concept first. As a concept, API has been there since more than 50 years, but in recent years it has become very popular and has outlined a different method for how we create our applications. Now imagine you are creating a web application where users can sign up and talk about their recent travels, write stories, post pictures and share them. As an add-on they should be able to share the same across multiple platforms like Facebook, Instagram etc. How do we do that? We don’t have access to the Instagram application or their data. We don’t know how they are managing their data and whether data from our application will fit into their databases.  Let’s try to imagine how Instagram would allow add-on without APIs. Perhaps we might drop an email to Instagram asking them to allow us access to their databases where we can write content. They may have received millions of other such requests from people who also need to use Instagram add-ons. Instagram will go through our requests and the million others and probably would use a lucky draw to decide who gets access to their databases. This is obviously not a reasonable solution and without any reasonable solution all applications in the world will just become isolated; which means they can’t communicate and can’t support each other’s features. Luckily our imagination is just that!Fortunately, we are living in a world with APIs around us which allow us to easily communicate with other apps. I can create an application where I can use Google’s search, Facebook’s feed, Instagram’s post, twitter’s tweets all in one app. This is all possible using API. As the name suggests, it allows us to open an interface which can then be used by other apps to communicate using the interface. How does an API work?The purpose of API is to communicate between apps with each other as intermediary, where both apps might have been built with different tools and technologies. It can be achieved by using api standards like REST, SOAP which outline protocols or sets of rules which the client (who needs to call) and server (who serves client’s requests) need to follow. These standards are key to communication. These standards outline what a request should look like and the format in which client should expect response from server. Every api exposes its endpoints (also called entrypoints), which allows its client to use it to access features of apis. For example, in JavaScript, to work with DOM, it exposes Document as an entrypoint in order to work with multiple methods. It allows its client to use it in order to manipulate DOM. In JavaScript however we use containers, which are nothing but HTML controls which can call APIs and response can be rendered within that control. JavaScript: An Introduction JavaScript is one of the most widely used programming languages, even though it is mainly used on browsers to render HTML and CSS. It can also run on our servers to handle client requests, connect with databases and do almost everything any other programming language does on the server. This makes it a significant tool that needs to be learned. Here, we are going to discuss about APIs which are provided by JavaScript to work with almost every aspect of a web application. APIs in JavaScript JavaScript, being a very widely used programming language has a very large set of APIs available, which make a JavaScript developer’s life a lot easier. JavaScript is also a multi-purpose language which means it can not only work on browsers, but also on servers or phones. To support different platforms, it has a different set of APIs to work with. It also provides a rich set of controls and other features like storage on client which are all easily availed by JavaScript.  Browser APIs in JavaScript Browsers use JavaScript to display web pages, handle user interactions, send requests to servers and to receive responses. To make all this possible, JavaScript provides browser APIs which can be used to perform browser related functions:  Working with DOM  DOM stands for Document Object Model, which is a structure that holds all HTML controls on a web page. It not only holds but also allows us to add remove controls from it. You might have worked with document API which is used to get, push or even remove elements from DOM, all of this is coming from JavaScript DOM API. Example: document.getElementById(“header”).text(“This is a header”); Request data from server JavaScript is widely used to optimize web pages by updating only a part of a webpage instead of loading the entire page. for example, when you like a post on Facebook, it doesn’t load the entire page again, it only updates that particular post. Similarly, when we tweet on Twitter it doesn’t reload the entire page to show new tweets. This is how JavaScript makes partial calls to the server using popular fetch, AJAX apis. Example: const response= await fetch(url); // when response is success which is status 200 if (response.ok) {     // receive response from server.   let json = await response.json(); } else {   // if there are errors returned by server, show the error   alert("Error occured" + response.status); }Store data at client Even though we have our databases working smoothly on servers processing terabytes of data every day, you might still want to store a chunk of data on the client’s browsers to optimize the application performances. How do we do that? Again JavaScript will show us the way, by providing APIs like localStorage, sessionStorage which we can use to store data on client’s browser and remove it whenever we want. The difference between localStorage and sessionStorgae is that data which we store in localStorage persists even after closing the browser or tab,  whereas data stored in sessionStorage gets cleared as soon as the tab or browser is closed. Now both of these have its own advantages depending on your requirement.sessionStorage.setItem('key', value); sessionStorage.getItem('key')   localStorage.setItem('key', value); sessionStorage.getItem('key';)Work with Graphics There are lot of popular applications where you might have seen  2D or 3D graphics created using user’s interaction allowing users to create such drawings, and I am not talking about just CSS (Cascaded Style Sheets). Yes, CSS has its role to play, but to allow users to create 2D/3D drawings on the web we need more than just CSS. JavaScript provides canvas api which can be used to work with pixels on web page to draw 2D/ 3D images which will be within html tag.Third Party APIs.  Integration with other applications, the structures of which we don’t have much information about, can be difficult. But JavaScript allows us to integrate with many popular third-party apps like Twitter, YouTube, Facebook. For example, you have your company’s website where you also want to display recent tweets that your company’s Twitter account has tweeted. How do we do that? JavaScript’s Twitter API is the answer. This is not only for Twitter, but a lot of other popular apps can be integrated into your own web page using JavaScript’s third party APIs. Web Audio API  Audio API in JavaScript provides a smooth implementation of audio controls over a webpage. This includes multiple play controls on audio, and audio effects on the web. Web Audio API has multiple interfaces which include operations related to visualization effects, audio destinations, merging audio channels and audio processing. Example      Play   Geolocation APIThere are many web apps which work best with information of user’s locations, for example, if you are looking for restaurants nearby, you will get better results if Google knows your location. It is obviously up to the user’s consent if they want to share their location or not due to privacy concerns, but if they do want to share, geolocation API provides location of users to web applications. Example: var address = document.getElementById("address"); function accessUserLocation() {   if (navigator.geolocation) { navigator.geolocation.getCurrentPosition(displayLocation); } } function displayLocation(location) { address.innerHTML = "Latitude: " + location.coords.latitude +   "Longitude: " + location.coords.longitude;   }History API  History API is another useful although rarely used API, used if you want to allow user to traverse back and forth on a web application. History API runs with window objects using three different methods available-- go, back and forward. Example: // Visit page visited previously  window.history.back();  // Traverse to forward on web page.  window.history.forward();  // Move forward by 1 page  window.history.go(1); Web Workers API  Workers can be created using constructors in JavaScript, which can run a JavaScript file under worker thread context, which is different from the context of windows in JavaScript. There are a few exceptions on what kind of code we can run under worker thread, for example we can’t manipulate DOM using worker thread. With this a question arises-- if worker thread is different form main window thread, then how do they communicate? Because even though it is running in a different thread, the worker can still execute many window methods. To make this possible, the worker and main thread communicate using postMessage() method; to send data whenever onmessage event is called and handled. Example: const worker = new Worker('input.js'); Here input file which is a JavaScript file will contain the code which you need the worker thread to execute. Most Popular JavaScript APIs Browser APIs – We have discussed them in detail and since JavaScript leads how web applications are rendered on browser, the popularity of browser APIs is obvious. Canvas APIs – This is another favorite API of many developers who work in the graphics domain and need interactive applications for their client to easily work with drawings and graphics on web pages. SocialMedia APIs – This is very common nowadays. Any web application that targets an audience, needs social media integrations in their app, such as Facebook, Twitter, Instagram APIs, where they can showcase recent activities on their web page. Storage APIs – Every site I browse now a days pops up a panel asking me to accept cookies settings. Why? Because it helps the applications work well with user’s interaction and responses to the app, and the storage is not limited to cookies. sessionStorage and localStorage are widely used in JavaScript applications to store user’s info on client’s browsers. We just can’t imagine a life without storage APIs in JavaScript! JavaScript tools and relationship between themJavaScript Libraries There are JavaScript libraries like jQuery and React which implement JavaScript with their own new syntax. For example, the way we access our DOM using vanilla JavaScript and the way jQuery does it is different, but in the end, both use the same API which we have discussed above. Another example is when using React, where it extends JavaScript with JSX to embed JavaScript with HTML in React components.JavaScript FrameworksJavaScript Frameworks like Angular and Vue make most of the JavaScript APIs as they are built on top of JavaScript. The difference between the frameworks is in the structure, data flow and different patterns which they use to make it easier to work with JavaScript and implement its APIs.  For example, Angular uses Dependency Injection to inject services into the component’s constructors, making it a sophisticated form of implementing inversion control which would be not this smooth with vanilla JavaScript. Frameworks like Angular try to prevent users from manipulating DOM directly as it is an expensive operation. However, it doesn’t mean it doesn’t update DOM at all. In the end, Angular uses DOM manipulation internally in quite an effective way, using the same APIs we have discussed above.Conclusion An API can be compared to a phone with unlimited talk time that is given to an application, allowing it to communicate with all other applications in the world. Why do we need this? Because one application doesn’t need to and can’t do everything. They need each other’s help. APIs can be for external as well internal use. External APIs allows an app to communicate with other apps, and internal or built-in APIs allow developers to work effectively with the tools. JavaScript being the most popular language in the world is aware of this and hence presents plenty of interfaces for us as developers to use in our applications. Be it working with storage, creating controls, integrating with Twitter, presenting media, or drawing graphics, JavaScript will not let you down.  
7320
Introduction to Web APIs in JavaScript

Before we talk about APIs in JavaScript and how to... Read More

How to Install React Native on Windows

React Native is a renowned JavaScript mobile application framework that allows building mobile applications on Android and iOS platforms. It offers superb mobile development capabilities and concentrates on creating applications for multiple stages using a similar codebase.  Originally developed by Facebook for its internal app development, React Native was open sourced in March 2015 for iOS Mobile apps, and by September of the same year a version for Android dev was also released. Today, Native React has been vastly improved and powers popular worldwide mobile applications like Instagram, Facebook, Skype, and more.  Are you ready to get started with React Native? Let's understand how to download, install and set up React Native in Windows 10. 1. PrerequisitesBefore we get started, here are a few system requirements to download, install, and set up React-Native on your Windows 10.Hardware requirementsTo download and successfully install React-Native in your computer, you need to consider the minimum hardware specifications required to support the app and run it smoothly. RAM 8 GB CPU: Intel ® Core ™ i7‐4870 HQ CPU @ 2.50 GHz 256 GB ROM Software requirementsTo efficiently install and set up React-Native on your Windows device, you will need to install the following:  Android Studio  Android SDK  Install JDK Node Js  NPM 3.5.2 React native cli 2. Installation procedureTo successfully install and set up React-Native to your device, you need to follow the steps discussed below: Step-1: Install ChocolateyThe first process that you need to do is to install Chocolatey, a well‐known package manager for Windows. Installation of Chocolatey requires administrator access to the computer’s command prompt to run.  Go to and choose the option – get started. In the Chocolatey install section, choose Individual as shown:Now go to Windows Power Shell, and enter the following command.Get-ExecutionPolicySelect the A (Yes to all) option and then press Enter.Copy and paste the command mentioned below:Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))  To check whether you have installed it correctly, open the command prompt and type; choco -version If you managed a correct installation, this command returns the Chocolatey version you have installed; like shown in the screen below:Step-2: Install Node Js Using Chocolatey we will now install Node.js, and JDK8 as follows. i. Installing Node Js Installing Node.js is crucial as it is a JavaScript runtime environment and React Native uses it build the JavaScript code. To install Node.js, you need to open the command prompt as the administrator and put in the Chocolatey command prompt below: Choco install -y nodejs.install Installation may take some time. Once done, you will see a message stating that Chocolatey is installed as shown below:  To confirm that installation has been successful, we will execute the below command at the command prompt, as administrator: node --version If the installation was successful, the version number is displayed. After installing Node.js, the Node Package Manager NPM automatically installs. Now check the installation of NPM in your system by entering the following command.   npm --versionThe displayed message below verifies that the installation of NPM is successful.Step:4 Installing JDK8 (Java Development Kit)JDK8 is crucial in the improvement of android applications by React-Native. To install the JDK8, go to the Windows PowerShell and use the following command: choco install -y nodejs.install openjdk8 To know whether it installed successfully, open the command prompt again and enter the command: java -version The message below is displayed if the JDK8 is successfully installed hence showing the Java version as: openjdk version “1.8.0_222” The installed JDK8 also has an installed Java compiler. To confirm whether the Java compiler has been installed, open the command prompt and enter the below command: javac -version Step-5: Install Android StudioTo improve mobile applications with React Native, you need to install Android Studio. To download the installation file, go to. By default, android studio automatically installs the latest Android SDK. However, building a React Native app with native code demands Android 11.0 (R) SDK in particular. The installation display message looks like below: After downloading the file, perform the Android Studio installation process. To continue, click on the Next button to see the Choose Components screen.  Tick the box containing Android Virtual Device and click on the Next button.  You then need to choose the location in your computer where you would like the Android Studio to get installed.  You can even leave it to install at the default location to save time, and  then click the Next button to go to the next screen:  You will come to the Choose Start Menu Folder. Click the Install button to install the program.When the bar reaches the end, click on the Next button to complete the process.On the next screen, check Start Android Studio and click on the Finish button.   Customize Android Studio: Click the Next button to go to the Install Type screen: Choose the Custom type of setup and then click the Next button.On this screen, you can select the theme of your choice and then click on the Next button to go to the SDK Component Setup window.On the SDK Component Setup display, check on the Performance Intel HAXM option plus Android Virtual Device option and then click on the Next button.On the Emulator Settings window, do not make changes to anything. Leave it the way it is and click on the Next button.   Verify settings.  The dialogue box below completes the whole process by clicking on the Install button.Customize Android SDK When you select Configure, the SDK Manager as shown on the screen above is displayed. This allows you to choose the Android SDK settings. Click on the Show Package Details located at the bottom right of the screen. From the list, tick the following: Android SDK Platform 28 Intel x86 Atom System Image Google APIs Intel x86 Atom System Image Google APIs Intel x86 Atom_64 System Image Click the OK button to install them, and you are done setting up the Android SDK. React Native needs environment variables to be customized to create applications using the native codes. Next, we will customize the Android Studio Environment variable. 3. Setting the path to Environmental VariableRight‐click on This PC and click Properties, a screen displays as below. Click on the advanced system settings.Once you are on the System Properties window, select the Advanced tab, and then the Environment Variables button at the bottom. Then press Enter.  After selecting the environment variable, go to the Variable name, type ANDROID_HOME and your variable value Android Studio SDK path. After that, customize the Android Studio platform‐tools path by selecting the Path variable on User variables for your name list to move to the edit dialog.  Input platform tools folder path in the SDK path, like C:\Users\[user name]\AppData\Local\Android\Sdk\to the end of the list and press Enter key. Open the command prompt and input the command below: adb When the environmental variable customization is successful, you get the below message: Android Debug Bridge version 1.0.41  Version 29.0.1-5644136  Installed as /Users/Username/Library/Android/sdk/platform-tools/adb Install React Native CLI To install React-Native CLI, open the command prompt as an administrator and enter the following command: npm install -g react-native-cli 4. Creating a New ApplicationLet us create a new React Native project using react native CLI. reactapp is the first project we are creating on React Native. react-native init reactapp Running the Application Once you have created your first React application, it's time to run the application. Open the application in any IDE of your choice. Here we are using Visual Studio Code as shown: If you want to run your project on an Android device, open Android Studio and create a virtual device. When your virtual device opens, use the following command in your Windows Command Prompt:  Once done, you will see the application opened on your Android device as shown:  5. Uninstalling React-NativeReact native application is a global package and not the same as other frameworks. Let us understand how to uninstall React Native from your system using Control Panel. Here is a step-by-step guide on how to uninstall React Native: To uninstall React Native from your system, use the command: npm uninstall -g react-native-cliConclusion In this article, we have seen the systematic procedure of downloading and setting up React-Native on Windows 10 devices. Besides, we have also discussed about React-Native, its origin, the installation process, and the follow up setup procedures. In a nutshell, you have learned how to install and set up the following software packages:   Installing Chocolatey Installing Node and JDK8 Installing Android Studio Customizing the Android SDK Installing React Native CLI Creating a new App Running the application You have also learnt how to create the React Native app and use it on an Android device. Lastly, we saw the uninstallation process too. Hurray! You have come to the end of the tutorial. It’s time to get started with React Native and build your projects. Keep Coding!  
5358
How to Install React Native on Windows

React Native is a renowned JavaScript mobile app... Read More