Search

How to Create a Collection in MongoDB?

In recent years, MongoDB has retained its top spot among NoSQL databases. The term NoSQL means non-relational. Mongo is an open-source document-oriented, NoSQL Database that allows users to query data without having to know SQL. You can read more about MongoDB here.MongoDB stores data in the form of collections. In this blog we will learn how to create a collection in MongoDB.PrerequisitesTo follow this blog, you should be having the latest version of MongoDB installed in your machine. You can download MongoDB for your operating system from this link.Let’s start by understanding the term collection.What is a collection in MongoDB?We all know that MongoDB stores data in the form of documents. All the similar documents are stored in a collection. It is same as a Table in a SQL database or any of the Relational Databases.An example of a collection is shown below:Source: mongodb websiteEach object in MongoDB is called a document. All the objects put together create a collection.Creating a Collection in MongoDBHow to create a collection in MongoDB?There are a couple of methods to create a collection in MongoDB:The createCollection() MethodCreating the Collection in MongoDB on the flyLet’s have a look at each of the methods one by one.To start with we need a mongo server in our local machine. To do that, open the terminal for Mac and Linux, and PowerShell or command prompt for Windows. Run the following command.Command: mongodThis fires up the MongoDB server.  To run the MongoDB commands, we need to start a MongoDB shell. For this, open a new window in terminal or PowerShell or command prompt and run the following. We shall refer to this window as mongo shell in the rest of this article.Command: mongoThis creates a shell where we can run MongoDB commands.Let's create a database and use it so that we can test our examples. For doing the same, run the following command in the mongo shell.Command: use myDBThis creates a new database with the name myDB and switches to that database, so that we can work with it.The createCollection() Method:Using createCollection method we can create a collection which does not have any documents or an empty collection. The syntax of createCollection() command is as follows: Syntax: db.createCollection(name, options)The createCollection method takes 2 parameters, the first parameter is the name of the collection which is a string and the other is an options object which is used to configure the collection. The options object is optional.   To create the collection without passing the options using createCollection method, run the following command in the mongo shell. Command: db.createCollection("testCollection")This creates a collection with the name testCollection inside myDB database.To see the collection, run the following command inside the mongo shell.Command: show collectionsThis should show all the collections we have inside the database.We can add additional configurations to the collection. For example, we can add validation or create a capped collection using the second parameter which is the options object.Configuration options for creating a collection in MongoDBThe basic syntax for configuring an object in createCollection method is as shown below.Syntax:db.createCollection( <name>,    {      capped: <boolean>,      autoIndexId: <boolean>,      size: <number>,      max: <number>,      storageEngine: <document>,      validator: <document>,      validationLevel: <string>,      validationAction: <string>,      indexOptionDefaults: <document>,      viewOn: <string>,              // Added in MongoDB 3.4      pipeline: <pipeline>,          // Added in MongoDB 3.4      collation: <document>,         // Added in MongoDB 3.4      writeConcern: <document>    } ) Let’s look at all the options in detail.FieldTypeDescriptioncappedboolean(Optional). To create a capped collection, specify true. If you specify true, you must also set a maximum size in the size field.autoIndexIdboolean(Optional). Specify false to disable the automatic creation of an index on the _id field.sizenumberOptional. Specify a maximum size in bytes for a capped collection. Once a capped collection reaches its maximum size, MongoDB removes the older documents to make space for the new documents. The size field is required for capped collections and ignored for other collections.maxnumberOptional. The maximum number of documents allowed in the capped collection. The size limit takes precedence over this limit. If a capped collection reaches the size limit before it reaches the maximum number of documents, MongoDB removes old documents. If you prefer to use the max limit, ensure that the size limit, which is required for a capped collection, is sufficient to contain the maximum number of documents.storageEnginedocumentOptional. Available for the WiredTiger storage engine only.Allows users to specify configuration to the storage engine on a per-collection basis when creating a collection.validatordocumentOptional. Allows users to specify validation rules or expressions for the collection.validationLevelstringOptional. Determines how strictly MongoDB applies the validation rules to existing documents during an update.validationActionstringOptional. Determines whether to create an error on invalid documents or just warn about the violations and allow invalid documents to be inserted.indexOptionDefaultsdocumentOptional. Allows users to specify a default configuration for indexes when creating a collection.viewOnstringThe name of the source collection or view from which to create the view. The name is not the full namespace of the collection or view; i.e. does not include the database name and implies the same database as the view to create. You must create views in the same database as the source collection.pipelinearrayAn array that consists of the aggregation pipeline stage(s).creates the view by applying the specified pipeline to the viewOn collection or view.collationdocumentSpecifies the default collation for the collection.Collation allows users to specify language-specific rules for string comparison, such as rules for lettercase and accent marks.writeConcerndocumentOptional. A document that expresses the write concern for the operation. Omit to use the default write concern.To know more about the options go to this link.Example of Create Collection in MongoDBAn example for creating a collection with the options before inserting documents is shown below. Run the below command in the mongo shell.Command: db.createCollection("anotherCollection", { capped : true, autoIndexID : true, size : 6142800, max : 10000 } )This creates a capped collection.What is a capped collection?A fixed-sized collection that automatically overwrites its oldest entries when it reaches its maximum size. The MongoDB oplog that is used in replication is a capped collection.See more about capped collection and oplog over here.Create a Collection with Document ValidationMongoDB has the capability to perform schema validation during updates and insertions. In other words, we can validate each document before updating or inserting the new documents into the collection.To specify the validation rules for a collection we need to use db.createCollection() with the validator option.MongoDB supports JSON Schema validation. To specify JSON Schema validation, use the $jsonSchema operator in your validator expression. This is the recommended way to perform validation in MongoDB.What is $jsonSchema?The $jsonSchema operator matches documents that satisfy the specified JSON Schema. It has the following syntax.Syntax: { $jsonSchema: <JSON Schema object> }The example for json Schema object is given below.  Example:  {   $jsonSchema: {      required: [ "name", "year", "skills", "address" ],      properties: {         name: {            bsonType: "string",            description: "must be a string and is required"         },         address: {            bsonType: "object",            required: [ "zipcode" ],            properties: {                "street": { bsonType: "string" },                "zipcode": { bsonType: "string" }            }         }      }   } }To create a collection with validation rules, run the below command in the mongo shell.Command:db.createCollection("employees", {    validator: {       $jsonSchema: {          bsonType: "object",          required: [ "name", "year", "skills", "address" ],          properties: {             name: {                bsonType: "string",                description: "must be a string and is required"             },             year: {                bsonType: "int",                minimum: 2017,                maximum: 2021,                description: "must be an integer in [ 2017, 2021] and is required"             },             skills: {                enum: [ "JavaScript", "React", "Mongodb", null ],                description: "can only be one of the enum values and is required"             },             salary: {                bsonType: [ "double" ],                description: "must be a double if the field exists"             },             address: {                bsonType: "object",                required: [ "city" ],                properties: {                   street: {                      bsonType: "string",                      description: "must be a string if the field exists"                   },                   city: {                      bsonType: "string",                      description: "must be a string and is required"                   }                }             }          }       }    } })This creates a collection with validation.Now if you run show collections command, employees collection should show up.Now, let’s look at the second method that is “Creating the Collection in MongoDB on the fly”Creating the Collection in MongoDB on the flyOne of the best things about MongoDB is that you need not create a collection before you insert a document in it. We can insert a document in the collection and MongoDB creates a collection on the fly. Use the below syntax to create a collection on the fly.Syntax: db.collection_name.insert({key:value, key:value…})Now let’s create a collection on the fly. To achieve that, run the following command in the mongo shell.Command:db.students.insert({ name: "Sai",   age: 18,   class: 10 })This creates a collection with the name students in the database. To confirm, you can run show collections command and check. This should show all the collections which have students collection, as shown in the following image.To check whether the document is successfully inserted, run the below command in the mongoshell to check.Syntax: db.collection_name.find()Command: db.students.find()This should show all the documents inside the collection.ConclusionIn this blog you have seen how to create a collection in MongoDB using different methods, along with examples.  MongoDB is a rapidly growing technology nowadays as it is flexible, fast and for many more reasons. Many companies are using MongoDB as their go-to database of choice. Learning MongoDB is recommended by many of the web developers as it boosts the probability of getting a job as well.

How to Create a Collection in MongoDB?

3K
How to Create a Collection in MongoDB?

In recent years, MongoDB has retained its top spot among NoSQL databases. The term NoSQL means non-relational. Mongo is an open-source document-oriented, NoSQL Database that allows users to query data without having to know SQL. You can read more about MongoDB here.

MongoDB stores data in the form of collections. In this blog we will learn how to create a collection in MongoDB.

Prerequisites

To follow this blog, you should be having the latest version of MongoDB installed in your machine. You can download MongoDB for your operating system from this link.

Let’s start by understanding the term collection.

What is a collection in MongoDB?

We all know that MongoDB stores data in the form of documents. All the similar documents are stored in a collection. It is same as a Table in a SQL database or any of the Relational Databases.

An example of a collection is shown below:

Source: mongodb website

Each object in MongoDB is called a document. All the objects put together create a collection.

Creating a Collection in MongoDB

How to create a collection in MongoDB?

There are a couple of methods to create a collection in MongoDB:

  1. The createCollection() Method
  2. Creating the Collection in MongoDB on the fly

Let’s have a look at each of the methods one by one.

To start with we need a mongo server in our local machine. To do that, open the terminal for Mac and Linux, and PowerShell or command prompt for Windows. Run the following command.

Command: mongodThis fires up the MongoDB server.  

To run the MongoDB commands, we need to start a MongoDB shell. For this, open a new window in terminal or PowerShell or command prompt and run the following. We shall refer to this window as mongo shell in the rest of this article.

Command: mongo

This creates a shell where we can run MongoDB commands.

Let's create a database and use it so that we can test our examples. For doing the same, run the following command in the mongo shell.

Command: use myDBThis creates a new database with the name myDB and switches to that database, so that we can work with it.

The createCollection() Method:

Using createCollection method we can create a collection which does not have any documents or an empty collection. 

The syntax of createCollection() command is as follows: 

Syntax: db.createCollection(name, options)

The createCollection method takes 2 parameters, the first parameter is the name of the collection which is a string and the other is an options object which is used to configure the collection. The options object is optional.   

To create the collection without passing the options using createCollection method, run the following command in the mongo shell. 

Command: db.createCollection("testCollection")This creates a collection with the name testCollection inside myDB database.

To see the collection, run the following command inside the mongo shell.

Command: show collectionsThis should show all the collections we have inside the database.

We can add additional configurations to the collection. For example, we can add validation or create a capped collection using the second parameter which is the options object.

Configuration options for creating a collection in MongoDB

The basic syntax for configuring an object in createCollection method is as shown below.

Syntax:

db.createCollection( <name>, 
   { 
     capped: <boolean>, 
     autoIndexId: <boolean>, 
     size: <number>, 
     max: <number>, 
     storageEngine: <document>, 
     validator: <document>, 
     validationLevel: <string>, 
     validationAction: <string>, 
     indexOptionDefaults: <document>, 
     viewOn: <string>,              // Added in MongoDB 3.4 
     pipeline: <pipeline>,          // Added in MongoDB 3.4 
     collation: <document>,         // Added in MongoDB 3.4 
     writeConcern: <document> 
   } 
) 

Let’s look at all the options in detail.

FieldTypeDescription
cappedboolean(Optional). To create a capped collection, specify true. If you specify true, you must also set a maximum size in the size field.
autoIndexIdboolean(Optional). Specify false to disable the automatic creation of an index on the _id field.
sizenumberOptional. Specify a maximum size in bytes for a capped collection. Once a capped collection reaches its maximum size, MongoDB removes the older documents to make space for the new documents. The size field is required for capped collections and ignored for other collections.
maxnumberOptional. The maximum number of documents allowed in the capped collection. The size limit takes precedence over this limit. If a capped collection reaches the size limit before it reaches the maximum number of documents, MongoDB removes old documents. If you prefer to use the max limit, ensure that the size limit, which is required for a capped collection, is sufficient to contain the maximum number of documents.
storageEnginedocumentOptional. Available for the WiredTiger storage engine only.

Allows users to specify configuration to the storage engine on a per-collection basis when creating a collection.
validatordocumentOptional. Allows users to specify validation rules or expressions for the collection.
validationLevelstringOptional. Determines how strictly MongoDB applies the validation rules to existing documents during an update.
validationActionstringOptional. Determines whether to create an error on invalid documents or just warn about the violations and allow invalid documents to be inserted.
indexOptionDefaultsdocumentOptional. Allows users to specify a default configuration for indexes when creating a collection.
viewOnstringThe name of the source collection or view from which to create the view. The name is not the full namespace of the collection or view; i.e. does not include the database name and implies the same database as the view to create. You must create views in the same database as the source collection.
pipelinearrayAn array that consists of the aggregation pipeline stage(s).

creates the view by applying the specified pipeline to the viewOn collection or view.
collationdocumentSpecifies the default collation for the collection.

Collation allows users to specify language-specific rules for string comparison, such as rules for lettercase and accent marks.
writeConcerndocumentOptional. A document that expresses the write concern for the operation. Omit to use the default write concern.

To know more about the options go to this link.

Example of Create Collection in MongoDB

An example for creating a collection with the options before inserting documents is shown below. Run the below command in the mongo shell.

Command: db.createCollection("anotherCollection", { capped : true, autoIndexID : true, size : 6142800, max : 10000 } )

This creates a capped collection.

What is a capped collection?

A fixed-sized collection that automatically overwrites its oldest entries when it reaches its maximum size. The MongoDB oplog that is used in replication is a capped collection.

See more about capped collection and oplog over here.

Create a Collection with Document Validation

MongoDB has the capability to perform schema validation during updates and insertions. In other words, we can validate each document before updating or inserting the new documents into the collection.

To specify the validation rules for a collection we need to use db.createCollection() with the validator option.

MongoDB supports JSON Schema validation. To specify JSON Schema validation, use the $jsonSchema operator in your validator expression. This is the recommended way to perform validation in MongoDB.

What is $jsonSchema?

The $jsonSchema operator matches documents that satisfy the specified JSON Schema. It has the following syntax.

Syntax: { $jsonSchema: <JSON Schema object> }
The example for json Schema object is given below.  
Example:  

{
  $jsonSchema: {
     required: [ "name", "year", "skills", "address" ],
     properties: {
        name: {
           bsonType: "string",
           description: "must be a string and is required"
        },
        address: {
           bsonType: "object",
           required: [ "zipcode" ],
           properties: {
               "street": { bsonType: "string" },
               "zipcode": { bsonType: "string" }
           }
        }
     }
  }
}

To create a collection with validation rules, run the below command in the mongo shell.

Command:

db.createCollection("employees", { 
   validator: { 
      $jsonSchema: { 
         bsonType: "object", 
         required: [ "name", "year", "skills", "address" ], 
         properties: { 
            name: { 
               bsonType: "string", 
               description: "must be a string and is required" 
            }, 
            year: { 
               bsonType: "int", 
               minimum: 2017, 
               maximum: 2021, 
               description: "must be an integer in [ 2017, 2021] and is required" 
            }, 
            skills: { 
               enum: [ "JavaScript", "React", "Mongodb", null ], 
               description: "can only be one of the enum values and is required" 
            }, 
            salary: { 
               bsonType: [ "double" ], 
               description: "must be a double if the field exists" 
            }, 
            address: { 
               bsonType: "object", 
               required: [ "city" ], 
               properties: { 
                  street: { 
                     bsonType: "string", 
                     description: "must be a string if the field exists" 
                  }, 
                  city: { 
                     bsonType: "string", 
                     description: "must be a string and is required" 
                  } 
               } 
            } 
         } 
      } 
   } 
})

This creates a collection with validation.

Now if you run show collections command, employees collection should show up.

Now, let’s look at the second method that is “Creating the Collection in MongoDB on the fly”

Creating the Collection in MongoDB on the fly

One of the best things about MongoDB is that you need not create a collection before you insert a document in it. We can insert a document in the collection and MongoDB creates a collection on the fly. Use the below syntax to create a collection on the fly.

Syntax: db.collection_name.insert({key:value, key:value…})

Now let’s create a collection on the fly. To achieve that, run the following command in the mongo shell.

Command:

db.students.insert({
name: "Sai",
  age: 18,
  class: 10
})

This creates a collection with the name students in the database. To confirm, you can run show collections command and check. This should show all the collections which have students collection, as shown in the following image.

To check whether the document is successfully inserted, run the below command in the mongoshell to check.

Syntax: db.collection_name.find()

Command: db.students.find()

This should show all the documents inside the collection.

Conclusion

In this blog you have seen how to create a collection in MongoDB using different methods, along with examples.  

MongoDB is a rapidly growing technology nowadays as it is flexible, fast and for many more reasons. Many companies are using MongoDB as their go-to database of choice. Learning MongoDB is recommended by many of the web developers as it boosts the probability of getting a job as well.

KnowledgeHut

KnowledgeHut

Author

KnowledgeHut is an outcome-focused global ed-tech company. We help organizations and professionals unlock excellence through skills development. We offer training solutions under the people and process, data science, full-stack development, cybersecurity, future technologies and digital transformation verticals.
Website : https://www.knowledgehut.com

Join the Discussion

Your email address will not be published. Required fields are marked *

Suggested Blogs

Back Up, Restore, and Migrate a MongoDB Database

Popular among both enterprises and startups, MongoDB is a database that is perfectly suited for web-apps that need to scale once the user base increases. MongoDB is different from traditional relational databases because it uses json like objects to store data, instead of tables in relational databases. In this post, we will learn to backup and restore a MongoDB database. In all software products there is an import and export feature, which in database terms, deals with human-readable format. On the other hand, the backup and restore operations use MongoDB specific data, which preserve the MongoDB attributes.  So, when migrating the database, we should prefer backup and restore over import and export. But we should also keep in mind that our source and target systems are compatible, which means that both should be Windows or both should be a Linux based system like Ubuntu/Mac. Prerequisites We are using Windows 10 in this tutorial. Please make sure you have downloaded the MongoDB Community Server and installed in it. It is a very easy setup and you will find lot of good articles on the internet detailing this out. Please ensure that you have added it in the Environment variable in your PC. Backup Considerations In a production environment, backups act as a snapshot of the database at a certain point. Large and complex databases do fail or can be hacked. If that happens, we can use the last backup file to restore the database to the point, before it failed. These are some of the factors which should be taken into consideration when doing a recovery.  1. Recovery Point Objective We should know the objective of the recovery point, which means how much data we are willing to lose during a backup and restoration. A continuous backup is preferred for critical data like bank information and backups should be taken several times during the day. On the other hand, if the data doesn’t change frequently, then the backup can be taken every 6 months.  2. Recovery Time ObjectiveThis tells how quickly the restoration can be done. During restoration the application will be down for some time; and this downtime should be minimized, or else customer relationships will be lost.  3. Database and Snapshot IsolationThis refers to the distance between the primary database server and the backup server. If they are close enough i.e., in the same building, then the recovery time reduces. However, in the event of a physical event such as a fire, there is a likelihood of it been destroyed with the primary database.   4. Restoration Process We should always test our backups in test servers to see if they will work, in case a restoration is required.  5. Available Storage Backup of database generally takes a lot of space and in most cases, it will never be required. So, we should try to minimize the space taken on the disk, by archiving the database into a zip file.  6. Complexity of DeploymentThe backup strategy should be easy to set and should be automated, so that we don’t have to remember to take the backup after regular intervals. Understanding the Basics The first thing that we should know is that MongoDB uses json and bson(binary json) formats for storing data. So, people coming from a JavaScript background can relate to objects for json, which have a key-value pair. Also, json is the preferred format in which we receive or send data to an API endpoint. You can check the json data of a MongoDB database in any tool or online editors. Even the famous Windows application Notepad++ has a json viewer. An example of a json document looks like below. As we can see from the above example, json is very convenient to work with, especially for developers.  But it doesn’t support all the data types available in bson. So, for backup and restoring, we should use binary bson. The second thing to keep in mind is that MongoDB automatically creates databases and collection names if they don’t exist during restore operations. Third, since MongoDB is a document-based database, in many user cases we store large amounts of data in one collection, such as the whole post of an article. MongoDB is also used extensively in large databases and big data. So, reading and inserting the data can consume a lot of CPU, memory and disk space. We should always run the backups during the non-peak hours like night. As already mentioned earlier, we can use import and export functions for backup and restoration of MongoDB databases, but we should use commands like mongodump and mongorestore to backup and restore respectively. MongoDB backup We will first cover backing up the MongoDB database. For this we use the mongodump command.  First open the Windows command prompt and go to the location in which MongoDB is installed. If you have chosen the default setting, while installing MongoDB though the pop-up it will be installed in a location like C:\Program Files\MongoDB\Server\4.4\bin The version number only will change if you are reading this blog in the future. Also, note that it’s better to run the command prompt in the Admin mode. So, once we open the command prompt, we need to change the directory to MongoDB bin folder by giving the below command. cd C:\Program Files\MongoDB\Server\4.4\binNow, enter mongod and press enter. It will show some json text.Now, we can backup to any location. For this post I am backing up on my Desktop in a Backup folder, which I have created through the command line.Now, we have to run mongodump command, but it should be also present in our MongoDB bin folder. If it is not present, we need to download it from and install it. After this, copy the entire exe files from the download to the MongoDB bin folder. MongoDB Backup with no option Now, run the mongodump command from the bin directory. Here, we are not giving any argument so the backup of the whole database will be taken in the same bin directory.MongoDB Backup to an output directory Now, run the mongodump command from the bin directory. Here, the argument –out specifies the directory in which the data backup will be maintained. In our case we are giving the Backup folder in the  Desktop, which we have created earlier. mongodump --out C:\Users\pc\Desktop\Backup Now, go to the desktop and you can find the backup that has been created in our Backup folder.  MongoDB Backup a specific database Now, we can also backup only a database in mongodump using the –db option. I have an example database, so to backup only that I will use the below command. mongodump --db example --out C:\Users\pc\Desktop\Backup As, you can see in the below output only the example database was backed up. MongoDB Backup a specific collection Now, if we want to only backup a specific collection, we need to use the –collection option and give the collection name. Also, note that the database name is mandatory in this case, as mongodb needs to know about the database to search for the collection. I have a products collection within the example database, so to backup only that I will use the below command. mongodump --db example --out C:\Users\pc\Desktop\Backup –collection products As, you can see in the below output only the products collection from example database was backed up. MongoDB Backup from remote MongoDB instances We can get the backup from remote mongodb instances also. I have a lot of MongoDB databases for my personal projects on MongoDB atlas, which is the free to use Cloud database for MongoDB. To get a backup of remote databases, we have to use the connection string with –uri parameter. I used the below command. mongodump --uri "mongodb+srv://xxxx:xxxxxxxxxxx@cluster0.suvl2.mongodb.net/xxxxxDB?retryWrites=true&w=majority" --out C:\Users\pc\Desktop\Backup You can see in the below output the backup of the remote instance. MongoDB Backup procedures We should try to make the backup procedure as automated as possible. One of the best ways is to use a cron job, so that it can run every day. As, discussed earlier it is best to run the backup in the night when the database has the least load.  Setting up a cron job is easier on a Linux or a Mac because the Windows equivalent of it is not good. Alternatively, you can do install mongodb in WSL2 for Windows which supports Ubuntu.  Suppose, on a Linux host which has a mongoDB instance running, you want to run the backup at 04:04 am daily. For this in the terminal, open the cron editor by running the below command in the terminal. sudo crontab –e Now, in the cron editor, you need to add a command like below for our case. 4 4 * * * mongodump --out /var/backups/mongobackups/`date +"%m-%d-%y"`Restoring and migrating a MongoDB database When we restore the MongoDB database from a backup, we will be able to take the exact copy of the MongoDB information, including the indexes. We restore MongoDB by using the command mongorestore, which works only with the binary backup produced by mongodump. Now, we have taken the backup of example database earlier and it is in our Backup folder. We will use the below command to restore it. In the arguments we will specify the name of the database first with –db option. After that with –drop, we make sure that the example database is first dropped. And in the final argument, we specify the path of our backup. mongorestore --db example --drop C:\Users\pc\Desktop\Backup\example Now, if we check in terminal, we have our example database restored properly. Conclusion In this article, we have learned about MongoDB backup and restore. We have learned the different options for the backups, and why and when backups are required. Keep learning! 
5639
Back Up, Restore, and Migrate a MongoDB Database

Popular among both enterprises and startups, Mong... Read More

How to Work With Forms In JavaScript

Forms also referred as web forms are a very important part of front end web application development for sake of interaction with users. Most commonly, forms are used to collect the data from users or provide a provision for user to control the user interface. Forms are great potential assets if correctly used in building an interactive web application. We would be touch basing some of the essential aspects of them like HTML structure, styling form controls, events, data validation and submitting data to server.Understanding forms in detail needs expertise in other areas than just HTML like styling form controls (CSS), scripting to validate or create custom controls (JavaScript).We would be referring or using libraries like Jquery (for document traversal, manipulation etc) and parsley (form validation library) to build better forms.A typical form’s HTML is made of HTML elements called as form controls like single or multiline text fields, dropdowns, checkboxes, button etc mostly created using element with specific type being set on Type attribute. These form controls can be programmed to add some validations to support specific values based on constraints set on them. These controls can be enriched to support accessibility for enabling the interaction for less privileged users.Let’s create a simple html page to build a form.           Learning Forms       All forms have to start with element which is container having the form fields user would interact with. All attributes of element are optional but for programming forms to capture data we need at least ‘action’ and ‘method’ attributes.action – is basically the URL where the form fields data would be sent to.method – corresponds to the HTTP method to submit the form data. Possible HTTP method names which can be set as values are post and get. And another value dialog is set when form is imbedded inside a .Note: Both formaction and formmethod can be overridden by button, input with type submit elements which we will learn as we go forward.Refer to this link to know more about form attributes.Let’s add a form element to our body with action (“”) and method(“get”). This implies that form will send a GET request to the current URL. If it is post then it would be a POST request to the URL in action. Add few fields to form say name, email and a submit button using with type being specified as text, email and submit respectively.Note: The tag is an empty element, meaning that it doesn't need a closing tag. Value attribute can be populated to set the default value.          Enter your name:                    Enter your email:                       Save and open the html in chrome or your preferred browser. Clicking on ‘Click me!’ should send a http get call with empty name and email.Note: We can use instead of with type as submit. The difference is that button can contain HTML content allowing to create a complex button whereas input allows only plain text.Let’s understand the Sending of form data.If we observer all the form fields again, we have added an attribute called ‘name’. This property is important to inform that which data is associated with which form field i.e. name/value pairs. Try adding some data to our fields rendering in html (say myName and first.last@email.com) and click submit button. You should see the data being sent as query parameters in the browser URL.?name=myName&email=first.last@email.com.Change the Form method value to POST instead of GET and send the submitted data by clicking the ‘Click me!’ button. You should be seeing Form Data being sent but the browser URL will not get update.name: myName email: first.last@email.comAll this while, we have our action method being set as empty. Replace this with another URL on server side say ‘/captureFormData’. Now on clicking submit button the data should be received by the script at ‘/captureFormData’ with key/value items contained in the HTTP request object.Note that each server-side language like Node.js, C# etc have their own way of handling the submitted form data. And this blog would not cover those topics and it is beyond the scope.Let’s refine our basic form structure with help of other HTML elements like , , etc. Though we used few of them in basic example. Let’s go little deep on them.Note: Nesting of form inside another form is unacceptable as it might result in unpredictable behavior. is a convenient way of grouping for sake of styling and semantic purpose. This control can be associated with so that some assistive technologies can read this legend and associate it with the controls inside the . Let’s understand this will an example:         Interested programming language                             JavaScript                                     CSharp                                     Java               When reading the above form by any screen readers, it will read as “Interested programming language JavaScript” for the first radio, “Interested programming language CSharp” and “Interested programming language Java” for second and third radio.Imagine if you have a long form with multiple fields. It would help to improve the usability if we can categorize/section them with the help of . It would even help to improve the accessibility of forms.Talking about accessibility, with the associated correctly with the via its for attribute (which contains the element's id attribute), a screenreader will read out something like "name, edit text" for below one.Enter your name: Another advantage of having label associated with input of type text, radio etc is they are clickable too.  If you click on a label then the associated input control will get the focus. If the input control is of type checkbox or radio, clicking on label will select the check box and radio. This will be useful as clickable area of checkbox or radio is small and having label gives provision to select it easily.Note: We can always associate multiple labels to a single input control but it is not a good idea as it will impact the accessibility and assistive technologies. along with can be used to separate the functionality in a form and group the same purpose elements like radio buttons.Here is an example of the same.               Contact information                   Title                                                                             Mr                                                                                                 Mrs                                                                               Name:                                                           E-mail:                                                           Password:                                                 Additional information                               Social type:                                 LinkedIn             Twitter             Instagram                                                 Phone number:                                                           Submit                   Every time you like to create an HTML form you need to start using element and  nesting all the content controls inside it. Most of the assistive technologies and browser plugins can help to discover elements and implement special hooks to make them easier to use.We have already some of the form elements like , , , , , and . Other common input types are button, checkbox, file, hidden, image, password, radio, reset, submit, and text.Input types.Attributes of Input.Few attributes on element help in validating the data like required, max, maxlength, min, minlength, multiple, pattern, step etc based on their respective type.Also other attributes on of type submit/image like formaction, formmethod, formnovalidate, formenctype etc helps in overriding the form level methods.ValidationBefore submitting the data to the server, it is important to perform some client side validation to avoid unwanted round trips. Client-side validation is needed but it is not a replacement to the server side validation. Advantage of having client side validation is to capture the invalid data and fix it immediately.Some of the important and popular checks which are most commonly used on client areField requiredSpecific data formatEnter valid email addressPassword and more…Let’s build a form with the above validation checks.                         Do you have experience in programming ?*                     Yes           No                             How many years of experience you have ?                                     What's your programming language?*                           TypeScript           Java           CSharp           Ruby           Go           Swift                             What's your company e-mail address?                             Cover letter                             Submit       Say, if we enter an value which is more than 40 in experience field. We should see an inbuilt error as shown below:All these validations and notifications are coming out of the box. Thanks to inbuilt functionality in control. Let’s see how we can perform validation of forms using JavaScript and take control of look and feel of error message.Most browsers support constraint validation API by providing few validation properties on HTML elements like , , , etc.validationMessage: we can customize this message if the control value failed validation otherwise it will return an empty string. It is dependent on other constraint i.e. willValidate and isValid.willValidate: If element is validated then it will be true otherwise false.validity: is the validity state of the element and it is dependent on other properties likepatternMatch for specified pattern attribute,tooLong and tooShort are for string fields based on maxLength and minLengthrangeOverflow and rangeUnderflow for numeric fields based on max and min attributestypeMatch for fields which are based on email or url.valid if all the validation constraints are metvalueMissing if the field is set as required.Along with properties, we do also have methods to perform validation like checkValidity() which returns true or false and setCustomValidity(message) is to set the message if the element is considered invalid. Also if the element is invalid then checkValidity will raise an event called invalid Event.Let’s create a simple form and customize the validation message.       Please enter an email address:             Submit     Add a script tag and customize the message as shown below:     const email = document.getElementById("mail");     email.addEventListener("input", function (event) {       if (email.validity.typeMismatch) {         email.setCustomValidity("I am expecting an e-mail address!");       } else {         email.setCustomValidity("");       }     });   Here we are listening to the input event on email field and checking if the validity on the control is valid or not and based on that we are setting the custom message.Here are we relying on inbuilt validation method. Let’s disable the validation at form level by with the help of ‘novalidate’ and take control over validation. This would mean the browser will not perform auto check on validation before sending the data. But still we have access to constraint validation API to perform validation ourself.Refine the above form to add few addition validation like required and minLength etc.               Please enter an email address:                             Submit     Let’s update the script to handle the validation     const form  = document.getElementsByTagName('form')[0];     const email = document.getElementById('mail');     const emailError = document.querySelector('#mail + span.error');     email.addEventListener('input', function (event) {       // Each time the user types something, we check if the form fields are valid.       if (email.validity.valid) {         // In case there is an error message visible, if the field is valid, we remove the error message.         emailError.textContent = ''; // Reset the content of the message         emailError.className = 'error'; // Reset the visual state of the message       } else {         // If there is still an error, show the correct error         showError();       }     });     form.addEventListener('submit', function (event) {       // if the email field is valid, we let the form submit       if(!email.validity.valid) {         // If it isn't, we display an appropriate error message         showError();         // Then we prevent the form from being sent by cancelling the event         event.preventDefault();       }     });     function showError() {       if(email.validity.valueMissing) {         // If the field is empty display the following error message.         emailError.textContent = 'You need to enter an e-mail address.';       } else if(email.validity.typeMismatch) {         // If the field doesn't contain an email address display the following error message.         emailError.textContent = 'Invalid value is entered, expected an e-mail address.';       } else if(email.validity.tooShort) {         // If the data is too short display the following error message.         emailError.textContent = `Email should be at least ${ email.minLength } characters; you entered ${ email.value.length }.`;       }       // Set the styling appropriately       emailError.className = 'error active';     } Reload the HTML and try entering an invalid email address, the corresponding error message should be displayed.Note: In the current scope of this blog, we are not working on styling.Is it possible to validate forms without built in APIs ? Let’s see with the same example.We would consider the same form again but have lot of functionality in                           Please enter an email address:                                             Submit           const form  = document.getElementsByTagName('form')[0];     const email = document.getElementById('mail');     let error = email.nextElementSibling;     const emailRegExp = /^[a-zA-Z0-9.!#$%&'*+/=?^_`{|}~-]+@[a-zA-Z0-9-]+(?:\.[a-zA-Z0-9-]+)*$/;     function addEvent(element, event, callback) {       let previousEventCallBack = element["on"+event];       element["on"+event] = function (e) {         const output = callback(e);         // A callback that returns `false` stops the callback chain and interrupts the execution of the event callback.         if (output === false) return false;         if (typeof previousEventCallBack === 'function') {           output = previousEventCallBack(e);           if(output === false) return false;         }       }     };     // Now we can rebuild our validation constraint. Because we do not rely on CSS pseudo-class, we have to explicitly set the valid/invalid class on our email field     addEvent(window, "load", function () {       // Here, we test if the field is empty (remember, the field is not required)       // If it is not, we check if its content is a well-formed e-mail address.       const test = email.value.length === 0 || emailRegExp.test(email.value);       email.className = test ? "valid" : "invalid";     });     // This defines what happens when the user types in the fiel     addEvent(email, "input", function () {       const test = email.value.length === 0 || emailRegExp.test(email.value);       if (test) {         email.className = "valid";         error.textContent = "";         error.className = "error";       } else {         email.className = "invalid";       }     });     // This defines what happens when the user tries to submit the data     addEvent(form, "submit", function () {       const test = email.value.length === 0 || emailRegExp.test(email.value);       if (!test) {         email.className = "invalid";         error.textContent = "Expecting an e-mail";         error.className = "error active";         return false;       } else {         email.className = "valid";         error.textContent = "";         error.className = "error";       }     });   On refreshing the page, the output with invalid email address should be displayed as shown below.In real time applications, we can rely on existing libraries like Parsley along with JQuery which would ease our life by taking away lot of complexity.Overview of Parsley:Parsley is a front-end javascript validation library which helps to give proper feedback to user on submission of form. As mentioned earlier, it is not a replacement of server side validation. Parsley library helps us to define our own validation.Parsley uses a DOM API namely ‘data-parsley-’ prefix on the existing properties. For example if we want to add this on a property say ‘sample’ then we would add as [data-parsley-sample=’value’]. This will allow us to configure pretty much everything without any configuration or custom function.There is no specific installation process but adding the corresponding script tags will enable the validation. Parsley is relied on Jquery so it has to be included as well.             ...                 $('#form').parsley();     Assumption is that we have downloaded the Jquery and Parsley minified librarie and added it to our working directory. Otherwise we can refer to CDN location as shown below.   Adding attribute ‘data-parsley-validate’ to each form will allow us to validate. And “$(‘#form’).parsley()” will manually bind Parsley to your forms.Let’s understand further by configuring the attributes via JavaScript. For which, lets add two input fields inside the form element.                 Also let’s update the content to perform some pre-defined validation based on attributes.       var instance = $('#first').parsley();       console.log(instance.isValid()); // maxlength is 42, so field is valid       $('#first').attr('data-parsley-maxlength', 4);       console.log(instance.isValid()); // No longer valid, as maxlength is 4       // You can access and override options in javascript too:       instance.options.maxlength++;       console.log(instance.isValid()); // Back to being valid, as maxlength is 5       // Alternatively, the options can be specified as:       var otherInstance = $('#second').parsley({         maxlength: 10       });       console.log(otherInstance.options);     In the console.log, we should see thistrue false true {maxlength: 10}Options are inherited from the global level to form level and further to field. So if we set the options at global level then the same can be observed at field level.   Parsley.options.maxlength = 42; // maxlength of 42 is declared at global level var formInstance = $('form').parsley(); var field = $('input').parsley(); console.log(field.options.maxlength); // Shows that maxlength is 42 inherited from global Parsley.options.maxlength = 30; console.log(field.options.maxlength); // Shows that maxlength is automatically 30 formInstance.options.maxlength++; console.log(field.options.maxlength); // Shows that maxlength is automatically 31We can also add our own custom validations. Let understand this with an example.                     window.Parsley.addValidator('multipleOf', {         requirementType: 'integer',         validateNumber: function(value, requirement) {           return 0 === value % requirement;         },         messages: {           en: 'This value should be a multiple of %s',         }       });     Here we are adding a new attribute namely ‘data-parsley-multiple-of’ which takes only numeric values which are multiples of 3.In window.Parsley, we added a new validator with name ‘multiple-of’ with an object containing few important properties like ‘requirementType’, ‘validateNumber’ and ‘messages’ to be shown. This properties helps the library to check if the input value is valid or not.Similar to validateNumber, other properties are also there for different types like validateString, validateDate and validateMultiple.Also for requirementType, we have different options like string, number, date, regexp, boolean etc.Messages by default has English format, to support multiple locales we need to add the specific localization and also add specific locale.Events: Parsley triggers events that allows ParsleyUI to work and for performance reasons they don’t rely on JQuery events but the usage is similar to JQuery i.e. parsley events will also bubble up like JQuery events. For example, if a field is validated then the event ‘field:validate’ will be triggred on the field instance then on to form instance and finally to the window.Parsley.$('#some-input').parsley().on('field:success', function() {         // In here, `this` is the parlsey instance of #some-input       });       window.Parsley.on('field:error', function() {         // This global callback will be called for any field that fails validation.         console.log('Validation failed for: ', this.$element);       });Many times, we need some validation based on the response from server. Parsley provides an attributes i.e. data-parsley-remote and data-parsley-remote-validator to perform the same.Let’s consider this HTMLLet’s add the async validator on the window.Parsley object.window.Parsley.addAsyncValidator('customValidator', function (xhr) {           console.log(this.$element); // jQuery Object[ input[name="q"] ]           return 404 === xhr.status;         }, 'customURL');Parsley is a very useful and powerful JavaScript form frontend validation library.Note: For developers building react based web applications, they can rely on FORMIK which is most popular library for building forms in React and React Native.ConclusionForms are important in HTML and it was needed and still needed now. is an html tag that allow us to perform HTTP methods like GET/POST operation without writing any code in JavaScript. Form defines an boundary to identify all set of the form field elements to be submitted to the server. For example, if we perform an enter key or clicking on submit button , the agent triggers form submission data based on each form field value to the server based on the action URL on the form.Before HTML5, all the elements are expected to be part of the to send the data to server. In HTML5, they maintained the backward compatibility and also enhanced the capabilities who may want to use AJAX and don’t want to rely on default behaviours i.e. they have enabled designers who expect more flexibility in having their form elements outside the form and still maintain the connections with the form. 
1681
How to Work With Forms In JavaScript

Forms also referred as web forms are a very import... Read More

Introduction to Web APIs in JavaScript

Before we talk about APIs in JavaScript and how to use them, we need to know what exactly we mean by APIs. API stands for Application Programming Interface and is a concept that is not limited or specific to JavaScript, but is used in almost all web application languages. Being a web developer, it is expected that you know about API, so let’s try to understand the concept first. As a concept, API has been there since more than 50 years, but in recent years it has become very popular and has outlined a different method for how we create our applications. Now imagine you are creating a web application where users can sign up and talk about their recent travels, write stories, post pictures and share them. As an add-on they should be able to share the same across multiple platforms like Facebook, Instagram etc. How do we do that? We don’t have access to the Instagram application or their data. We don’t know how they are managing their data and whether data from our application will fit into their databases.  Let’s try to imagine how Instagram would allow add-on without APIs. Perhaps we might drop an email to Instagram asking them to allow us access to their databases where we can write content. They may have received millions of other such requests from people who also need to use Instagram add-ons. Instagram will go through our requests and the million others and probably would use a lucky draw to decide who gets access to their databases. This is obviously not a reasonable solution and without any reasonable solution all applications in the world will just become isolated; which means they can’t communicate and can’t support each other’s features. Luckily our imagination is just that!Fortunately, we are living in a world with APIs around us which allow us to easily communicate with other apps. I can create an application where I can use Google’s search, Facebook’s feed, Instagram’s post, twitter’s tweets all in one app. This is all possible using API. As the name suggests, it allows us to open an interface which can then be used by other apps to communicate using the interface. How does an API work?The purpose of API is to communicate between apps with each other as intermediary, where both apps might have been built with different tools and technologies. It can be achieved by using api standards like REST, SOAP which outline protocols or sets of rules which the client (who needs to call) and server (who serves client’s requests) need to follow. These standards are key to communication. These standards outline what a request should look like and the format in which client should expect response from server. Every api exposes its endpoints (also called entrypoints), which allows its client to use it to access features of apis. For example, in JavaScript, to work with DOM, it exposes Document as an entrypoint in order to work with multiple methods. It allows its client to use it in order to manipulate DOM. In JavaScript however we use containers, which are nothing but HTML controls which can call APIs and response can be rendered within that control. JavaScript: An Introduction JavaScript is one of the most widely used programming languages, even though it is mainly used on browsers to render HTML and CSS. It can also run on our servers to handle client requests, connect with databases and do almost everything any other programming language does on the server. This makes it a significant tool that needs to be learned. Here, we are going to discuss about APIs which are provided by JavaScript to work with almost every aspect of a web application. APIs in JavaScript JavaScript, being a very widely used programming language has a very large set of APIs available, which make a JavaScript developer’s life a lot easier. JavaScript is also a multi-purpose language which means it can not only work on browsers, but also on servers or phones. To support different platforms, it has a different set of APIs to work with. It also provides a rich set of controls and other features like storage on client which are all easily availed by JavaScript.  Browser APIs in JavaScript Browsers use JavaScript to display web pages, handle user interactions, send requests to servers and to receive responses. To make all this possible, JavaScript provides browser APIs which can be used to perform browser related functions:  Working with DOM  DOM stands for Document Object Model, which is a structure that holds all HTML controls on a web page. It not only holds but also allows us to add remove controls from it. You might have worked with document API which is used to get, push or even remove elements from DOM, all of this is coming from JavaScript DOM API. Example: document.getElementById(“header”).text(“This is a header”); Request data from server JavaScript is widely used to optimize web pages by updating only a part of a webpage instead of loading the entire page. for example, when you like a post on Facebook, it doesn’t load the entire page again, it only updates that particular post. Similarly, when we tweet on Twitter it doesn’t reload the entire page to show new tweets. This is how JavaScript makes partial calls to the server using popular fetch, AJAX apis. Example: const response= await fetch(url); // when response is success which is status 200 if (response.ok) {     // receive response from server.   let json = await response.json(); } else {   // if there are errors returned by server, show the error   alert("Error occured" + response.status); }Store data at client Even though we have our databases working smoothly on servers processing terabytes of data every day, you might still want to store a chunk of data on the client’s browsers to optimize the application performances. How do we do that? Again JavaScript will show us the way, by providing APIs like localStorage, sessionStorage which we can use to store data on client’s browser and remove it whenever we want. The difference between localStorage and sessionStorgae is that data which we store in localStorage persists even after closing the browser or tab,  whereas data stored in sessionStorage gets cleared as soon as the tab or browser is closed. Now both of these have its own advantages depending on your requirement.sessionStorage.setItem('key', value); sessionStorage.getItem('key')   localStorage.setItem('key', value); sessionStorage.getItem('key';)Work with Graphics There are lot of popular applications where you might have seen  2D or 3D graphics created using user’s interaction allowing users to create such drawings, and I am not talking about just CSS (Cascaded Style Sheets). Yes, CSS has its role to play, but to allow users to create 2D/3D drawings on the web we need more than just CSS. JavaScript provides canvas api which can be used to work with pixels on web page to draw 2D/ 3D images which will be within html tag.Third Party APIs.  Integration with other applications, the structures of which we don’t have much information about, can be difficult. But JavaScript allows us to integrate with many popular third-party apps like Twitter, YouTube, Facebook. For example, you have your company’s website where you also want to display recent tweets that your company’s Twitter account has tweeted. How do we do that? JavaScript’s Twitter API is the answer. This is not only for Twitter, but a lot of other popular apps can be integrated into your own web page using JavaScript’s third party APIs. Web Audio API  Audio API in JavaScript provides a smooth implementation of audio controls over a webpage. This includes multiple play controls on audio, and audio effects on the web. Web Audio API has multiple interfaces which include operations related to visualization effects, audio destinations, merging audio channels and audio processing. Example      Play   Geolocation APIThere are many web apps which work best with information of user’s locations, for example, if you are looking for restaurants nearby, you will get better results if Google knows your location. It is obviously up to the user’s consent if they want to share their location or not due to privacy concerns, but if they do want to share, geolocation API provides location of users to web applications. Example: var address = document.getElementById("address"); function accessUserLocation() {   if (navigator.geolocation) { navigator.geolocation.getCurrentPosition(displayLocation); } } function displayLocation(location) { address.innerHTML = "Latitude: " + location.coords.latitude +   "Longitude: " + location.coords.longitude;   }History API  History API is another useful although rarely used API, used if you want to allow user to traverse back and forth on a web application. History API runs with window objects using three different methods available-- go, back and forward. Example: // Visit page visited previously  window.history.back();  // Traverse to forward on web page.  window.history.forward();  // Move forward by 1 page  window.history.go(1); Web Workers API  Workers can be created using constructors in JavaScript, which can run a JavaScript file under worker thread context, which is different from the context of windows in JavaScript. There are a few exceptions on what kind of code we can run under worker thread, for example we can’t manipulate DOM using worker thread. With this a question arises-- if worker thread is different form main window thread, then how do they communicate? Because even though it is running in a different thread, the worker can still execute many window methods. To make this possible, the worker and main thread communicate using postMessage() method; to send data whenever onmessage event is called and handled. Example: const worker = new Worker('input.js'); Here input file which is a JavaScript file will contain the code which you need the worker thread to execute. Most Popular JavaScript APIs Browser APIs – We have discussed them in detail and since JavaScript leads how web applications are rendered on browser, the popularity of browser APIs is obvious. Canvas APIs – This is another favorite API of many developers who work in the graphics domain and need interactive applications for their client to easily work with drawings and graphics on web pages. SocialMedia APIs – This is very common nowadays. Any web application that targets an audience, needs social media integrations in their app, such as Facebook, Twitter, Instagram APIs, where they can showcase recent activities on their web page. Storage APIs – Every site I browse now a days pops up a panel asking me to accept cookies settings. Why? Because it helps the applications work well with user’s interaction and responses to the app, and the storage is not limited to cookies. sessionStorage and localStorage are widely used in JavaScript applications to store user’s info on client’s browsers. We just can’t imagine a life without storage APIs in JavaScript! JavaScript tools and relationship between themJavaScript Libraries There are JavaScript libraries like jQuery and React which implement JavaScript with their own new syntax. For example, the way we access our DOM using vanilla JavaScript and the way jQuery does it is different, but in the end, both use the same API which we have discussed above. Another example is when using React, where it extends JavaScript with JSX to embed JavaScript with HTML in React components.JavaScript FrameworksJavaScript Frameworks like Angular and Vue make most of the JavaScript APIs as they are built on top of JavaScript. The difference between the frameworks is in the structure, data flow and different patterns which they use to make it easier to work with JavaScript and implement its APIs.  For example, Angular uses Dependency Injection to inject services into the component’s constructors, making it a sophisticated form of implementing inversion control which would be not this smooth with vanilla JavaScript. Frameworks like Angular try to prevent users from manipulating DOM directly as it is an expensive operation. However, it doesn’t mean it doesn’t update DOM at all. In the end, Angular uses DOM manipulation internally in quite an effective way, using the same APIs we have discussed above.Conclusion An API can be compared to a phone with unlimited talk time that is given to an application, allowing it to communicate with all other applications in the world. Why do we need this? Because one application doesn’t need to and can’t do everything. They need each other’s help. APIs can be for external as well internal use. External APIs allows an app to communicate with other apps, and internal or built-in APIs allow developers to work effectively with the tools. JavaScript being the most popular language in the world is aware of this and hence presents plenty of interfaces for us as developers to use in our applications. Be it working with storage, creating controls, integrating with Twitter, presenting media, or drawing graphics, JavaScript will not let you down.  
7321
Introduction to Web APIs in JavaScript

Before we talk about APIs in JavaScript and how to... Read More