## How to Create MongoDB Database in Node.js

MongoDB is an open-source platform written in C++. It is a document database designed for ease of development and scaling. It is a cross-platform, document-oriented and non-structured database. This article is a quick guide on how to create MongoDB database in Node.js. We will understand MongoDB compatibility and language compatibility and exploreMongo shell. We will see how to import data into the database, how to use MongoDB driver for node.js and the Node.js based CRUD operations. We will conclude with building a simple app using node express app with MongoDB to store and serve content. How MongoDB is different from relational databases Relational databases are SQL enabled and are organised in tables with each entry containing a specific set of columns with a specific data type.  It is seen that sometimes relational databases become very complex for simple schemas, for e.g., where an employee who has more than one phone number, email IDs, addresses, teams, etc. and all the related data is stored in different tables. To retrieve complete information for an employee, multiple tables needs to be joined and queried.  One of the disadvantages with relational databases is that to update or change schema, we would need to modify multiple tables. As the requirement changes, more updates might be needed, further complicating the design. This makes it necessary for a dedicated DB admin. Mongo is a NoSql DB and is object-based containing an entire entity in document record. Unlike relational database, NoSqldb can be queried faster and easy to get complete details of a specific entity or document.  Another feature of Mongo is if there is no document matching to query than no error is returned. In relational database, a table contains the same set of fields whereas in Mongo DB, schema is dynamic and each one need not conform to the same schema.Mongo is developer friendly and JSON representation makes it easy to parse. MongoDB stores documents in collections. Collections are analogous to tables in relational databases. In MongoDB, databases hold collections of documents. In addition to collections, MongoDB supports: Read-only Views (Starting in MongoDB 3.4) On-Demand Materialized Views (Starting in MongoDB 4.2). MongoDB data structure Documents – JSON Objects Store data as BSON (Binary JSON) [For the BSON spec, see bsonspec.org] The value of a field can be any of the BSON data types, including other documents, arrays, and arrays of documents. For example, the following document contains values of varying types var mydoc = {_id: ObjectId("3423jksdhfi83p9fj90cd"),  name: { firstName: "Sachin", lastName: "Tendulkar" },  birth: new Date('Apr 24, 1973'),  awards:[ “Bharat Ratna”, “Padma Vibhushan”, “Padma Shri”]  followers :NumberLong(30000000)  } _id holds an ObjectId,  name holds an embedded document that contains firstName and lastName, birth hold values of Date types awards holds an array of strings followers holds a value of NumberLong type. Note: Mongo supports nesting up to 100 levels. Learn more about bson-types at https://docs.MongoDB.com/manual/reference/bson-types/ Setup MongoDB environmentFormacOS platform, download the tgz file and extract the file. Easiest way to get the items in your path is to copy them to usr/local/bin directory. Follow the below steps to get it doneNow type ‘mongod’ in command prompt which is mongo daemon and  you can see it has started. Invoke another command prompt and type ‘mongo’ to start off in the shell. To make sure everything is working let’s insert a dummy data and observe that it inserted one document. db.users.insert({“name”: “Sachin Tendulkar”})Database is all setup. For Windows platform, download the msi and run the installation to complete the installation. Once the installation is done, you can find where it has been installed. Two things important in bin folder are mongo and MongoD. Copy the path and add to the path in environment variables.  Once this has been done, we should be able to run both the Mongo Server and Mongo Shell from command prompt.  Invoke the command prompt and do ‘md \data’ and then create data DB by ‘md \data\db’. This is the file that mongo is going to look for to use to store information about database.  Now type ‘mongoD’ in command prompt which is mongo daemon and  you can see it has started. Invoke another command prompt and type ‘mongo’ to start off in the shell. To make sure everything is working let’sinsert a dummy data and observe that it inserted one document. db.users.insert({“name”: “Sachin Tendulkar”}) Database is all setup. The recommended way to get started using the Node.js driver is by using NPM (Node Package Manager) to install the dependency in your project. After you’ve created your project with npminit, you can install the MongoDB driver and its dependencies with the command: npm install MongoDB --save This will download the MongoDB driver and add a dependency entry in your package.json file. We will learn more of this as we go forward. Compatibility MongoDB compatibility Note: The driver does not support older versions of MongoDBNode.js DriverMongoDB 4.2MongoDB 4.0MongoDB 3.6MongoDB 3.4MongoDB 3.2MongoDB 3.0MongoDB 2.6>= 3.X✓✓✓✓✓✓✓>= 3.2.1✓✓✓✓✓✓>= 3.1✓✓✓✓✓✓>= 3.0✓✓✓✓✓>=2.2.12✓✓✓✓Language compatibilityNode.jsDriverNode.js v0.8.XNode.js v0.10.XNode.js v0.12.XNode.js v4.X.XNode.js v6.X.XNode.js v8.X.XNode.js v10.x.xNode.js v12.x.x>=3.X.X✓✓✓✓✓2.2.X – 2.0.X✓✓✓✓✓✓>=1.4.18✓✓✓1.4.x✓✓Few salient points When designing the schema for your database, a quick understanding of Embedded vs References is helpful. You can refer Coderwall for more information on this.  Three of the other features that mongo provides: Indexing, Sharding and replicationcan make Mongo the right choice for your application. Indexing- supports ad hoc queries 64 indices per collection Single Field Compound (multiple Fields) Unique Sharding  - Scalability Partitions data onto different machines Scale application into smaller systems Autosharding supported by Mongo Challenging to set upReplicationReliability (Primary and n number of secondary server) Maximizes uptime Replica sets Automatic failoverExplore Mongo Shell Note:All the examples and snaps are taken in macOS. Invoke terminal and type ‘mongod’ which will display the listening port 27017Sinncewe are running locally, we need not secure and optimize it Invoke another terminal (command + T) and type ‘mongo’You should see that we are now connected to MongoDB://127.0.0.1:27017 By default, we are connected to test database  Type ‘db’ to check it in terminal and observe ‘test’ Let us use a different database say ‘MyNewMongoDB’ Type ‘use MyNewMongoDB’ Output: Switched to dbMyNewMongoDB To view all available dbs, type ‘show dbs’.Note: We will not see the new db ‘MyNewMongoDB’. Instead the db is created when the first document is created. Mongo stores data in databases which are separated into collections of documents. To Create a new collection, we need to insert a document and the collection will be created as a side effect. Type ‘db.cricketers.insert({“name”:”Sachin”}) Now check ‘show dbs’ to view our new db Type ‘show collections’ to check that our collection is created.As you can see, mongo doesn’t require any schema or setup work to start working with data. The commands we used are very similar to what we have in SQL but mongo shell uses a JavaScript interpreter. So, we can interact with database using JavaScript expressions making it easy to perform various tasks. Print(‘test’) will print test var arr = [“one”, “two”, “three”] type arrwhich will print the results. Let us take this option to explore further i.e. we will insert 1000 records to db using a for loop.for(i=0;i<1000;i++){ db.numbers.insert({"number":i}) } Though it says there was only one insert, it’s counting the number of operations and not individual documents. This is evident from the command ‘db.numbers.count()’ Now let’s try to find a single document from the database and understand the query better. Type ‘db.numbers.find({"number": 2})’ and this should display the one record. Let’s ask the mongo to explain it for us by adding the explain command.db.numbers.find({“number”:”2”}).explain() Explain did mentioned about the query planner but didn’t give much information about the execution. So, let’s pass execution status to get that information. db.numbers.find({“number”:”2”}).explain(“executionStats”)Now we can see that it ran through 999 records and took 0 milliseconds. May be if we increase the count to 1 million we can find higher time in executionTime. Since we don’t want to run through all the records, let’s create a simple index to see how the outcome changes. db.numbers.createIndex({number: 2}) Here it is similar to find but the number is a special variable and it is not a string. Now, let us try execution status again and see the result.As we can see, only one document is examined and imagine if we want to search in a collection of 10 million or more records. It makes a huge difference. Import data into the database: Though inserting data into mongo using shell or language driver is handy. Most of the time we have the data in some form which we need to import to our database. And Mongoimport command can import the data in json, csv or tsv formats and no custom code needed.In our case, we are using random data generated using. Sample view of the data downloaded:‘mongoimport –help’ will display the options available. Namespace options display the db and the collection to which the data has to be imported. Input options is next section in which we will use the file option. Json format is default one here. However, the file being imported is an array and not an object so we need to dash dash json array flag as well. Invoke the terminal and make sure we are in the file directory where the json file resides or downloaded.As we can see, we have imported 7 documents successfully.  Now let us check if mongo has the data what we wanted Switch to mongo client and type ‘use MyNewMongoDB’ Type ‘show collections’ to view the collections now having persons. We can count records using commanddb.persons.count() or find using db.persons.find({“isActive”: true}). Mongo shell CRUD operations Now that we have the data in place, let’s explore the standard CRUD operations in database.If client is not running, start it by typing mongo in terminal and switch to newly created database ‘MyNewMongoDB’ [READ] In the available data,let’s find out the number of persons having eyecolor blue. GET: db.persons.find({“eyecolor”: “blue”}).count() This should result in count of 2 records. [CREATE] Now let’s go and add one more person with eyecolor blue. (INSERT) Insert is going to be little complex and might span across lines based on the data we insert.  db.persons.insert({ "_id": "5eee4d58c952ec90ad43dfe8", "index": 7, "guid": "fbc2510e-678f-4843-b612-96b899ee5e9b", "isActive": true, "balance": "$3,996.47", "picture": "http://placehold.it/32x32", "age": 18, "eyeColor": "blue", "name": "SteuPoluard", "gender": "female", "company": "EXTRAWEAR"}) Note: In the above case, it is spanning multiple lines and Mongo is not going to execute/process until the last parenthesis isclosed and also _id is unique. Output should display one new record is inserted. WriteResult({ "nInserted" : 1 }) If we try to find the people with eye color blue, we should get 3 records.UPDATE] Now let’s go and update the inserted record with an email Id. Updates can happen to replace the entire document or a sub-set of the data. We wanted to add the email id without losing the entire object.For this we need to use the set functionality.db.persons.update({"name":"SteuPoluard"}, ... {$set: {"email":"Steu.Poluard@extrawear.com"}}) Output: WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 }) Now if we find the record to check if email is been added.Now if we want to update the tags then we need to use a different operator called $addToSet. db.persons.update({"name":"Sara Pollard"}, ... { ...$addToSet: {"tags":"newtag"}  ... })Note: There is a large set of update operators are available, refer to mongo site to explore this functionality in depth. [DELETE] Now lets finally delete the record added with name – ‘SteuPoluard’ db.persons.remove({"name":"SteuPoluard"}) To verify, let’s see if find command finds it. db.persons.find({"name":"SteuPoluard"}) No result. So, the record got successfully deleted. Say if we want to delete the collection itself. db.persons.drop() Check show collections to find the persons collection is no more.Using MongoDBdriver for node.js Now let’s write some code in node.js to interact with the database. The drivers for all of the languages work similarly so once we have it working for node. Other language drivers should be pretty much same. The official MongoDB Node.js driver provides both callback-based and Promise-based interaction with MongoDB, allowing applications to take full advantage of the new features in ES6. The 2.x series of the driver is powered by a brand-new core driver and BSON library. To understand more about the changes made from drive 1.x to 2.x, refer to thisblog. First step is to make sure our Mongo server is up and running with db ‘MyNewMongoDB’ and the imported data collection i.e. person already in place. Note: In case if the collection is dropped, please follow the above steps to create and import the dataSteps:Create an index.js file  Setup the mongo client using MongoDB package. var MongoClient = require('MongoDB').MongoClient; Url for the database uses the MongoDB protocol and connects to the specified server. var url = 'MongoDB://localhost:27017/MyNewMongoDB'; Finally, we want to connect to the MongoClient and in the callback we log the connection is made successfullyvarMongoClient = require('MongoDB').MongoClient;  varurl = 'MongoDB://localhost:27017/MyNewMongoDB';  MongoClient.connect(url, function(err, db) {  console.log("Connected successfully to server");  db.close();  }) Invoke a terminal and navigate to the index.js file folder. Type node index.js and observe that ‘Connected successfully to server’ Note: If you are facing any error, check if MongoDB server is running and npm install is performed. Now, let’s go ahead and try to access all the documents or specific document and print them in console. Since we have already made the connection. Let’s find the record in the callback of the connection. varMongoClient = require('MongoDB').MongoClient;  // Server is running locally on the default port and db is name is 'MyNewMongoDB'  varurl = 'MongoDB://localhost:27017/MyNewMongoDB';  varfindDocuments = function(db, callback) {  // collection name is 'persons' which is imported  varcollection = db.collection('persons');  // looking for a document with field 'name' and value 'Sara Pollard'  collection.find({"name":"Sara Pollard"}).toArray(function(err, docs) {  console.log(docs);  callback();      })  }  MongoClient.connect(url, function(err, db) {  console.log("Connected successfully to server");  // On obtaining the connection, find the record and close the db connection.  findDocuments(db, function () {  db.close();      })  }) Type node index.js to check if the record is obtained successfully and is logged.As we can see, the same interfaces covered in the mongoShell work here as well for all CRUD operations. For sake of review, here is the list Create --> insert. Read --> find or findOne. Update --> update. Delete --> delete. Node.js based CRUD operations Let ussetup up a simple application using Node.js and MongoDB. We would understand how to setup the driver and perform simple CRUD operations. Follow the below steps to setup project mkdirlearnMongoDB cd learnMongoDB npminit [you would be prompted for inputs related to package.json] npm install MongoDB–save [--save is to update the package.json] Assuming that the mongo db server is still running. If not, start as mentioned earlier. Open the directory learnMongoDB in your favourite IDE. (I am using Visual Studio code). Add a file app.js with the below snippet of code. // import the MongoDB and get the client object.  constMongoClient = require('MongoDB').MongoClient;  constassert = require('assert');  // Connection URL - can be obtained in the MongoDB server  consturl = 'MongoDB://localhost:27017';  // Database Name   constdbName = 'myproject';  // Create a new MongoClient  constclient = newMongoClient(url);  // Use connect method to connect to the Server  client.connect(function(err) {  assert.equal(null, err);  console.log("Connected successfully to server");  constdb = client.db(dbName);  client.close();  }); Observe that we are not providing the db name in the URL. Instead we are getting the db instance on successfully connecting to the server. This way, we can get connected to different databases in the same client instances. Note: We are using assert module to check if connection is successful or not by validating the error object. Will be used for other CRUD operations as well. Run ‘node app.js’ and should observe the log mentioning connected successfully to server. Insertion: Now we have the db object, let’s get hold of the collection and insert some objects into it. Create a function ‘insertDocuments’ to perform insertion into the collection.Call this insertDocuments in the callback of successful connection. // insertDocuments take the db instance  constinsertDocuments = function(db, callback) {  // Get the documents collection  constcollection = db.collection('documents');  // Insert some documents  collection.insertMany([        {a :1}, {a :2}, {a :3}      ], function(err, result) {  assert.equal(err, null); // assertion to check if any error is there.  assert.equal(3, result.result.n);  assert.equal(3, result.ops.length);  console.log("Inserted 3 documents into the collection");  callback(result); // return the inserted documents      });  }  // Use connect method to connect to the Server  client.connect(function(err) {  assert.equal(null, err);  console.log("Connected successfully to server");  constdb = client.db(dbName);  //insert the documents and in the callback, close the client.  insertDocuments(db, function(result) {  console.log(result);  client.close();    })  }); In the callback of the insertDocuments, we get the result object which is as shown below: { result: { ok: 1, n: 3 }, ops: [ { a: 1, _id: 5eeefd2a16f61641f58a9418 },  { a: 2, _id: 5eeefd2a16f61641f58a9419 },  { a: 3, _id: 5eeefd2a16f61641f58a941a } ],  insertedCount: 3,  insertedIds:  { '0': 5eeefd2a16f61641f58a9418,  '1': 5eeefd2a16f61641f58a9419,  '2': 5eeefd2a16f61641f58a941a } } As you can see, result object gives an overview of the action. Ops is the objects added to the collection. And other fields are self-explanatory. Note: You might have noticed that a new field ‘_id’ has been added which is generated by MongoDB for unique identification. If we are not happy with the unique id, we can as well supply like we did during importing of the data. Find: Let’s check if documents added can be retrieved using find. Collection object has a method called ‘find’ which take a parameter i.e. search query to identify the objects. If it is empty then all documents are returned.Invoke the findDouments in the callback of insertion for validating the records inserted. We can use a query filter to get specific item as well. // findDocuments from the db instance  constfindDocuments = function(db, callback) {  // Get the documents collection  constcollection = db.collection('documents');  // Finds all documents. If required, parameterize the query filter to get specific items.  collection.find({}).toArray(function(err, docs) {  assert.equal(err, null);  console.log("Found the following records");  console.log(docs)  callback(docs);      });    }  // Use connect method to connect to the Server  client.connect(function(err) {  assert.equal(null, err);  console.log("Connected successfully to server");  constdb = client.db(dbName);  //insert the documents and in the callback, close the client.  insertDocuments(db, function(result) {  findDocuments(db, function() {  client.close();      })        })  }); For example, if we want to find a specific set of documents based on pre-defined query filter like ‘{‘a’:3}’ : // Find specific document with a query filter  constfindDocuments = function(db, callback) {  // Get the documents collection  const collection = db.collection('documents');  // Find some documents  collection.find({'a': 3}).toArray(function(err, docs) {  assert.equal(err, null);  console.log("Found the following record(s)");      console.log(docs);  callback(docs);    });  } Updation: Now go andupdate a specific document. There are various methods available for updation like update, updateMany, updateOne etc. Use updateOne method to update the first one which matches the query. Signature: updateOne(filter, update, options, callback) Let’s send the filter as {a : 2 } i.e. document whose field a is equal to 2, update as $set: {b:1} to avoid replacement of entire object i.e. adding a new field b to document with value set as 1. Ignore options which is an optional and have a callback to capture post actions. constupdateDocument = function(db, callback) { // Get the documents collection constcollection = db.collection('documents'); // Update document where a is 2, set b equal to 1 collection.updateOne({ a :2 } , {$set: { b :1 } }, function(err, result) {  assert.equal(err, null);  assert.equal(1, result.result.n);  console.log("Updated the document with the field a equal to 2");  callback(result);      });      }    // Use connect method to connect to the Server  client.connect(function(err) {  assert.equal(null, err);  console.log("Connected successfully to server");  constdb = client.db(dbName);  //insert the documents and in the callback, close the client.  insertDocuments(db, function(result) {  findDocuments(db, function() {  updateDocument(db, function() {  client.close();          })      })       })  }); Deletion: Let’s use deleteOne method to find and delete the first item with field a and value 3. constremoveDocument = function(db, callback) {  // Get the documents collection  constcollection = db.collection('documents');  // Delete document where a is 3  collection.deleteOne({ a :3 }, function(err, result) {  assert.equal(err, null);  assert.equal(1, result.result.n);  console.log("Removed the document with the field a equal to 3");  callback(result);      });        }  // Use connect method to connect to the Server  client.connect(function(err) {  assert.equal(null, err);  console.log("Connected successfully to server");  constdb = client.db(dbName);  //insert the documents and in the callback, close the client.  insertDocuments(db, function(result) {  findDocuments(db, function() {  updateDocument(db, function() {  removeDocument(db, function(){  client.close();              })          })      })    })  }); Note: All the methods on the collection returns promise object if callback is not passed. So to avoid callback hell as we see above, we can rely on promise and program the code better. Node express app with MongoDB to store and serve content Now that we have fair idea of how to work with MongoDB using mongoshell and node.js driver. Let us build a simple app using node express app with MongoDB to store and serve content. Note: Since this topic is more about MongoDB rather than express, we will keep the express app a basic version. Let us go and comment out all the code in the app.js file from the previous example. We should be left with this: // import the MongoDB and get the client object.  constMongoClient = require('MongoDB').MongoClient;  const { equal } = require('assert');  // Connection URL - can be obtained in the MongoDB server  consturl = 'MongoDB://localhost:27017';  // Database Name   constdbName = 'myproject'; Invoke a terminal and navigate to the directory where the app.js file is located. Let us install express with the below command. npm install express –save Import the express and body-parser as shown below: constexpress = require('express');  constbodyParser = require('body-parser');  create an app instance and use bodyParser.json().  constapp = express();  app.use(bodyParser.json());  Let the express app be listening on port 3000 with a default route.  app.get('/', (req, res) =>res.send('Express app with MongoDB!'));  app.listen(3000, () =>console.log("App listening at http://localhost:3000")); Consolidated code: constexpress = require('express');  constbodyParser = require('body-parser');  // import the MongoDB and get the client object.  constMongoClient = require('MongoDB').MongoClient;  const { equal } = require('assert');  // Connection URL - can be obtained in the MongoDB server  consturl = 'MongoDB://localhost:27017';  // Database Name   constdbName = 'myproject';  constapp = express();  app.use(bodyParser.json());  app.get('/', (req, res) =>res.send('Express app with MongoDB!'));  app.listen(3000, () =>console.log("App listening at http://localhost:3000")); In the terminal, run ‘node app.js’ and should see the console log that app is listening in port 3000. To check the default route is working correctly. Invoke the postman and perform get operation.Now let us create a new route to retrieve the document based on field a and parameterized value. app.get('/api/documents/:name', (req,res) => {  }) :name is the parameter being passed. So let’s retrieve it from request parameters. And since we know it is numeric and not string. Let’s parse it to integer. constname = parseInt(req.params.name); Now lets connect to the MongoClient as earlier. This time we will not be using callback approach rather rely on the promise being returned. constclient = awaitMongoClient.connect(url, {useNewUrlParser:true});Since returned object is a promise and in order for the promise to be resolved or rejected. We need to use await keyword before proceeding further. Note: Promise object has two properties i.e. state and result. Promise state property initially has ‘pending’ and post execution will be either ‘fulfilled’(resolve) or ‘rejected’(reject). Similarly, result will be ‘undefined’ initially and then changes to ‘value’[resolve(value)] or ‘error’[reject(error)]. Let us get the db ‘myproject’ and find the first document based on the query filter from collection using findOne method. constdb = client.db(dbName);  constdocument = awaitdb.collection('documents').findOne({"a":name}); Once we get the document, lets return the status as 200 with the obtained document and close the client connection. res.status(200).json(document);  client.close(); As we are using await keyword, we need to add async keyword to the callback. And also lets add a try-catch block to return an error status in case of connection failure as internal server error. app.get('/api/documents/:name',async (req,res) => {  try {     constname = parseInt(req.params.name);  constclient = awaitMongoClient.connect(url, {useNewUrlParser:true});  constdb = client.db(dbName);  constdocument = awaitdb.collection('documents').findOne({"a":name});  res.status(200).json(document);  client.close();            } catch(error) {  res.status(500).json({message:"Error connecting to db", error});      }  }) Now we are good to go and restart our express app. Press ctrl + c and run node app.js to see app is listening at. Make sure that MongoDB Server is up and running in the background. Switch to postman and perform a GET operation. This should return the first record in the collection as shown below. {"_id":"5eeefd2a16f61641f58a9419","a":2,"b":1} Same GET request can be performed for other documents by changing the name as 2Similar to get operations, we can configure the routes to perform other CRUD operations on MongoDB to store and serve the content.  Conclusion In this guide, we have gone overMongoDB compatibility and language compatibility and explored the Mongo shell. We have seen how to import data into the database, how to use MongoDB driver and the Node.js based CRUD operations. We have also looked at how to build a simple app using node express app with MongoDB to store and serve content.Armed with this knowledge, you are ready to take a deeper dive into MongoDB and NodeJS. Here’s wishing you the best! Real coding requires real exercises. Practice live coding with our programming  experts on live workshops now.

# How to Create MongoDB Database in Node.js

7K

MongoDB is an open-source platform written in C++. It is a document database designed for ease of development and scaling. It is a cross-platform, document-oriented and non-structured database.

This article is a quick guide on how to create MongoDB database in Node.js. We will understand MongoDB compatibility and language compatibility and exploreMongo shell. We will see how to import data into the database, how to use MongoDB driver for node.js and the Node.js based CRUD operations. We will conclude with building a simple app using node express app with MongoDB to store and serve content.

## How MongoDB is different from relational databases

Relational databases are SQL enabled and are organised in tables with each entry containing a specific set of columns with a specific data type.

It is seen that sometimes relational databases become very complex for simple schemas, for e.g., where an employee who has more than one phone number, email IDs, addresses, teams, etc. and all the related data is stored in different tables. To retrieve complete information for an employee, multiple tables needs to be joined and queried.

One of the disadvantages with relational databases is that to update or change schema, we would need to modify multiple tables. As the requirement changes, more updates might be needed, further complicating the design. This makes it necessary for a dedicated DB admin.

Mongo is a NoSql DB and is object-based containing an entire entity in document record. Unlike relational database, NoSqldb can be queried faster and easy to get complete details of a specific entity or document.

Another feature of Mongo is if there is no document matching to query than no error is returned. In relational database, a table contains the same set of fields whereas in Mongo DB, schema is dynamic and each one need not conform to the same schema.Mongo is developer friendly and JSON representation makes it easy to parse.

MongoDB stores documents in collections. Collections are analogous to tables in relational databases. In MongoDB, databases hold collections of documents.

In addition to collections, MongoDB supports:

• Read-only Views (Starting in MongoDB 3.4)
• On-Demand Materialized Views (Starting in MongoDB 4.2).

MongoDB data structure

• Documents – JSON Objects
• Store data as BSON (Binary JSON) [For the BSON spec, see bsonspec.org]

The value of a field can be any of the BSON data types, including other documents, arrays, and arrays of documents. For example, the following document contains values of varying types

var mydoc = {_id: ObjectId("3423jksdhfi83p9fj90cd"),
name: { firstName: "Sachin", lastName: "Tendulkar" },
birth: new Date('Apr 24, 1973'),
followers :NumberLong(30000000)
} 
• _id holds an ObjectId,
• name holds an embedded document that contains firstName and lastName,
• birth hold values of Date types
• awards holds an array of strings
• followers holds a value of NumberLong type.

Note: Mongo supports nesting up to 100 levels.

Setup MongoDB environment

FormacOS platform, download the tgz file and extract the file. Easiest way to get the items in your path is to copy them to usr/local/bin directory. Follow the below steps to get it done

Now type ‘mongod’ in command prompt which is mongo daemon and  you can see it has started. Invoke another command prompt and type ‘mongo’ to start off in the shell. To make sure everything is working let’s insert a dummy data and observe that it inserted one document.

db.users.insert({“name”: “Sachin Tendulkar”})

## Database is all setup.

For Windows platform, download the msi and run the installation to complete the installation. Once the installation is done, you can find where it has been installed. Two things important in bin folder are mongo and MongoD. Copy the path and add to the path in environment variables.

Once this has been done, we should be able to run both the Mongo Server and Mongo Shell from command prompt.

Invoke the command prompt and do ‘md \data’ and then create data DB by ‘md \data\db’. This is the file that mongo is going to look for to use to store information about database.

Now type ‘mongoD’ in command prompt which is mongo daemon and  you can see it has started. Invoke another command prompt and type ‘mongo’ to start off in the shell. To make sure everything is working let’sinsert a dummy data and observe that it inserted one document.

db.users.insert({“name”: “Sachin Tendulkar”})

Database is all setup.

The recommended way to get started using the Node.js driver is by using NPM (Node Package Manager) to install the dependency in your project.

After you’ve created your project with npminit, you can install the MongoDB driver and its dependencies with the command:

npm install MongoDB --save

## Compatibility

### MongoDB compatibility

Note: The driver does not support older versions of MongoDB

Node.js DriverMongoDB 4.2MongoDB 4.0MongoDB 3.6MongoDB 3.4MongoDB 3.2MongoDB 3.0MongoDB 2.6
>= 3.X
>= 3.2.1
>= 3.1
>= 3.0

>=2.2.12

### Language compatibility

Node.js
Driver
Node.js v0.8.XNode.js v0.10.XNode.js v0.12.XNode.js v4.X.XNode.js v6.X.XNode.js v8.X.XNode.js v10.x.xNode.js v12.x.x
>=3.X.X

2.2.X – 2.0.X

>=1.4.18

1.4.x

## Few salient points

When designing the schema for your database, a quick understanding of Embedded vs References is helpful. You can refer Coderwall for more information on this.

Three of the other features that mongo provides: Indexing, Sharding and replicationcan make Mongo the right choice for your application.

• 64 indices per collection
• Single Field
• Compound (multiple Fields)
• Unique

Sharding  - Scalability

• Partitions data onto different machines
• Scale application into smaller systems
• Autosharding supported by Mongo
• Challenging to set up

Replication

• Reliability (Primary and n number of secondary server)
• Maximizes uptime
• Replica sets
• Automatic failover

## Explore Mongo Shell

• Note:All the examples and snaps are taken in macOS.
• Invoke terminal and type ‘mongod’ which will display the listening port 27017

• Sinncewe are running locally, we need not secure and optimize it
• Invoke another terminal (command + T) and type ‘mongo’

You should see that we are now connected to MongoDB://127.0.0.1:27017

By default, we are connected to test database

• Type ‘db’ to check it in terminal and observe ‘test’

Let us use a different database say ‘MyNewMongoDB’

• Type ‘use MyNewMongoDB’
• Output: Switched to dbMyNewMongoDB

To view all available dbs, type ‘show dbs’.

Note: We will not see the new db ‘MyNewMongoDB’. Instead the db is created when the first document is created.

Mongo stores data in databases which are separated into collections of documents. To Create a new collection, we need to insert a document and the collection will be created as a side effect.

• Type ‘db.cricketers.insert({“name”:”Sachin”})
• Now check ‘show dbs’ to view our new db
• Type ‘show collections’ to check that our collection is created.

As you can see, mongo doesn’t require any schema or setup work to start working with data. The commands we used are very similar to what we have in SQL but mongo shell uses a JavaScript interpreter. So, we can interact with database using JavaScript expressions making it easy to perform various tasks.

• Print(‘test’) will print test
• var arr = [“one”, “two”, “three”]
• type arrwhich will print the results.

Let us take this option to explore further i.e. we will insert 1000 records to db using a for loop.

for(i=0;i<1000;i++){ db.numbers.insert({"number":i}) }

Though it says there was only one insert, it’s counting the number of operations and not individual documents. This is evident from the command ‘db.numbers.count()’

Now let’s try to find a single document from the database and understand the query better.

• Type ‘db.numbers.find({"number": 2})’ and this should display the one record.
• Let’s ask the mongo to explain it for us by adding the explain command.
• db.numbers.find({“number”:”2”}).explain()

• Explain did mentioned about the query planner but didn’t give much information about the execution. So, let’s pass execution status to get that information.
• db.numbers.find({“number”:”2”}).explain(“executionStats”)

• Now we can see that it ran through 999 records and took 0 milliseconds. May be if we increase the count to 1 million we can find higher time in executionTime.
• Since we don’t want to run through all the records, let’s create a simple index to see how the outcome changes.
• db.numbers.createIndex({number: 2})
• Here it is similar to find but the number is a special variable and it is not a string.

Now, let us try execution status again and see the result.

As we can see, only one document is examined and imagine if we want to search in a collection of 10 million or more records. It makes a huge difference.

## Import data into the database:

Though inserting data into mongo using shell or language driver is handy. Most of the time we have the data in some form which we need to import to our database.

And Mongoimport command can import the data in json, csv or tsv formats and no custom code needed.In our case, we are using random data generated using

‘mongoimport –help’ will display the options available.

• Namespace options display the db and the collection to which the data has to be imported.

• Input options is next section in which we will use the file option. Json format is default one here. However, the file being imported is an array and not an object so we need to dash dash json array flag as well.

Invoke the terminal and make sure we are in the file directory where the json file resides or downloaded.

As we can see, we have imported 7 documents successfully.

Now let us check if mongo has the data what we wanted

• Switch to mongo client and type ‘use MyNewMongoDB’
• Type ‘show collections’ to view the collections now having persons.
• We can count records using commanddb.persons.count() or find using db.persons.find({“isActive”: true}).

## Mongo shell CRUD operations

Now that we have the data in place, let’s explore the standard CRUD operations in database.If client is not running, start it by typing mongo in terminal and switch to newly created database ‘MyNewMongoDB’

• [READ] In the available data,let’s find out the number of persons having eyecolor blue.
• GET: db.persons.find({“eyecolor”: “blue”}).count()
• This should result in count of 2 records.
• [CREATE] Now let’s go and add one more person with eyecolor blue. (INSERT)
• Insert is going to be little complex and might span across lines based on the data we insert.
db.persons.insert({ "_id": "5eee4d58c952ec90ad43dfe8", "index": 7, "guid": "fbc2510e-678f-4843-b612-96b899ee5e9b", "isActive": true, "balance": "$3,996.47", "picture": "http://placehold.it/32x32", "age": 18, "eyeColor": "blue", "name": "SteuPoluard", "gender": "female", "company": "EXTRAWEAR"})  Note: In the above case, it is spanning multiple lines and Mongo is not going to execute/process until the last parenthesis isclosed and also _id is unique. • Output should display one new record is inserted. WriteResult({ "nInserted" : 1 })  • If we try to find the people with eye color blue, we should get 3 records. • UPDATE] Now let’s go and update the inserted record with an email Id. • Updates can happen to replace the entire document or a sub-set of the data. We wanted to add the email id without losing the entire object.For this we need to use the set functionality. db.persons.update({"name":"SteuPoluard"}, ... {$set: {"email":"Steu.Poluard@extrawear.com"}}) 

Output:

WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
• Now if we find the record to check if email is been added.

• Now if we want to update the tags then we need to use a different operator called $addToSet. db.persons.update({"name":"Sara Pollard"}, ... { ...$addToSet: {"tags":"newtag"}
... })

Note: There is a large set of update operators are available, refer to mongo site to explore this functionality in depth.

• [DELETE] Now lets finally delete the record added with name – ‘SteuPoluard’

db.persons.remove({"name":"SteuPoluard"})
• To verify, let’s see if find command finds it.

db.persons.find({"name":"SteuPoluard"})

No result. So, the record got successfully deleted.

• Say if we want to delete the collection itself.

db.persons.drop()
• Check show collections to find the persons collection is no more.

## Using MongoDBdriver for node.js

Now let’s write some code in node.js to interact with the database. The drivers for all of the languages work similarly so once we have it working for node. Other language drivers should be pretty much same.

The official MongoDB Node.js driver provides both callback-based and Promise-based interaction with MongoDB, allowing applications to take full advantage of the new features in ES6. The 2.x series of the driver is powered by a brand-new core driver and BSON library. To understand more about the changes made from drive 1.x to 2.x, refer to thisblog

First step is to make sure our Mongo server is up and running with db ‘MyNewMongoDB’ and the imported data collection i.e. person already in place.

Note: In case if the collection is dropped, please follow the above steps to create and import the data

Steps:

• Create an index.js file
• Setup the mongo client using MongoDB package.
var MongoClient = require('MongoDB').MongoClient;
• Url for the database uses the MongoDB protocol and connects to the specified server.

var url = 'MongoDB://localhost:27017/MyNewMongoDB';
• Finally, we want to connect to the MongoClient and in the callback we log the connection is made successfully

varMongoClient = require('MongoDB').MongoClient;
varurl = 'MongoDB://localhost:27017/MyNewMongoDB';
MongoClient.connect(url, function(err, db) {
console.log("Connected successfully to server");
db.close();
}) 

Invoke a terminal and navigate to the index.js file folder.

Type node index.js and observe that ‘Connected successfully to server’

Note: If you are facing any error, check if MongoDB server is running and npm install is performed.

Now, let’s go ahead and try to access all the documents or specific document and print them in console. Since we have already made the connection. Let’s find the record in the callback of the connection.

varMongoClient = require('MongoDB').MongoClient;
// Server is running locally on the default port and db is name is 'MyNewMongoDB'
varurl = 'MongoDB://localhost:27017/MyNewMongoDB';
varfindDocuments = function(db, callback) {
// collection name is 'persons' which is imported
varcollection = db.collection('persons');
// looking for a document with field 'name' and value 'Sara Pollard'
collection.find({"name":"Sara Pollard"}).toArray(function(err, docs) {
console.log(docs);
callback();
})
}
MongoClient.connect(url, function(err, db) {
console.log("Connected successfully to server");
// On obtaining the connection, find the record and close the db connection.
findDocuments(db, function () {
db.close();
})
}) 

Type node index.js to check if the record is obtained successfully and is logged.

As we can see, the same interfaces covered in the mongoShell work here as well for all CRUD operations. For sake of review, here is the list

• Create --> insert.
• Read --> find or findOne.
• Update --> update.
• Delete --> delete.

## Node.js based CRUD operations

Let ussetup up a simple application using Node.js and MongoDB. We would understand how to setup the driver and perform simple CRUD operations.

Follow the below steps to setup project

• mkdirlearnMongoDB
• cd learnMongoDB
• npminit [you would be prompted for inputs related to package.json]
• npm install MongoDB–save [--save is to update the package.json]

Assuming that the mongo db server is still running. If not, start as mentioned earlier.

Open the directory learnMongoDB in your favourite IDE. (I am using Visual Studio code). Add a file app.js with the below snippet of code.

// import the MongoDB and get the client object.
constMongoClient = require('MongoDB').MongoClient;
constassert = require('assert');
// Connection URL - can be obtained in the MongoDB server
consturl = 'MongoDB://localhost:27017';
// Database Name
constdbName = 'myproject';
// Create a new MongoClient
constclient = newMongoClient(url);
// Use connect method to connect to the Server
client.connect(function(err) {
assert.equal(null, err);
console.log("Connected successfully to server");
constdb = client.db(dbName);
client.close();
}); 

Observe that we are not providing the db name in the URL. Instead we are getting the db instance on successfully connecting to the server. This way, we can get connected to different databases in the same client instances.

Note: We are using assert module to check if connection is successful or not by validating the error object. Will be used for other CRUD operations as well.

Run ‘node app.js’ and should observe the log mentioning connected successfully to server.

Insertion: Now we have the db object, let’s get hold of the collection and insert some objects into it.

Create a function ‘insertDocuments’ to perform insertion into the collection.Call this insertDocuments in the callback of successful connection.

// insertDocuments take the db instance
constinsertDocuments = function(db, callback) {
// Get the documents collection
constcollection = db.collection('documents');
// Insert some documents
collection.insertMany([
{a :1}, {a :2}, {a :3}
], function(err, result) {
assert.equal(err, null); // assertion to check if any error is there.
assert.equal(3, result.result.n);
assert.equal(3, result.ops.length);
console.log("Inserted 3 documents into the collection");
callback(result); // return the inserted documents
});
}
// Use connect method to connect to the Server
client.connect(function(err) {
assert.equal(null, err);
console.log("Connected successfully to server");
constdb = client.db(dbName);
//insert the documents and in the callback, close the client.
insertDocuments(db, function(result) {
console.log(result);
client.close();
})
}); 

In the callback of the insertDocuments, we get the result object which is as shown below:

{ result: { ok: 1, n: 3 },

ops:

[ { a: 1, _id: 5eeefd2a16f61641f58a9418 },
{ a: 2, _id: 5eeefd2a16f61641f58a9419 },
{ a: 3, _id: 5eeefd2a16f61641f58a941a } ],
insertedCount: 3,
insertedIds:
{ '0': 5eeefd2a16f61641f58a9418,
'1': 5eeefd2a16f61641f58a9419,
'2': 5eeefd2a16f61641f58a941a } } 

As you can see, result object gives an overview of the action. Ops is the objects added to the collection. And other fields are self-explanatory.

Note: You might have noticed that a new field ‘_id’ has been added which is generated by MongoDB for unique identification. If we are not happy with the unique id, we can as well supply like we did during importing of the data.

Find: Let’s check if documents added can be retrieved using find.

Collection object has a method called ‘find’ which take a parameter i.e. search query to identify the objects. If it is empty then all documents are returned.Invoke the findDouments in the callback of insertion for validating the records inserted. We can use a query filter to get specific item as well.

// findDocuments from the db instance
constfindDocuments = function(db, callback) {
// Get the documents collection
constcollection = db.collection('documents');
// Finds all documents. If required, parameterize the query filter to get specific items.
collection.find({}).toArray(function(err, docs) {
assert.equal(err, null);
console.log("Found the following records");
console.log(docs)
callback(docs);
});
}
// Use connect method to connect to the Server
client.connect(function(err) {
assert.equal(null, err);
console.log("Connected successfully to server");
constdb = client.db(dbName);
//insert the documents and in the callback, close the client.
insertDocuments(db, function(result) {
findDocuments(db, function() {
client.close();
})
})
}); 

For example, if we want to find a specific set of documents based on pre-defined query filter like ‘{‘a’:3}’ :

// Find specific document with a query filter
constfindDocuments = function(db, callback) {
// Get the documents collection
const collection = db.collection('documents');
// Find some documents
collection.find({'a': 3}).toArray(function(err, docs) {
assert.equal(err, null);
console.log("Found the following record(s)");
console.log(docs);
callback(docs);
});
} 

Updation: Now go andupdate a specific document. There are various methods available for updation like update, updateMany, updateOne etc. Use updateOne method to update the first one which matches the query.

Signature: updateOne(filter, update, options, callback)

Let’s send the filter as {a : 2 } i.e. document whose field a is equal to 2, update as $set: {b:1} to avoid replacement of entire object i.e. adding a new field b to document with value set as 1. Ignore options which is an optional and have a callback to capture post actions. constupdateDocument = function(db, callback) { // Get the documents collection constcollection = db.collection('documents'); // Update document where a is 2, set b equal to 1 collection.updateOne({ a :2 } , {$set: { b :1 } }, function(err, result) {
assert.equal(err, null);
assert.equal(1, result.result.n);
console.log("Updated the document with the field a equal to 2");
callback(result);
});
}
// Use connect method to connect to the Server
client.connect(function(err) {
assert.equal(null, err);
console.log("Connected successfully to server");
constdb = client.db(dbName);
//insert the documents and in the callback, close the client.
insertDocuments(db, function(result) {
findDocuments(db, function() {
updateDocument(db, function() {
client.close();
})
})
})
}); 

Deletion: Let’s use deleteOne method to find and delete the first item with field a and value 3.

constremoveDocument = function(db, callback) {
// Get the documents collection
constcollection = db.collection('documents');
// Delete document where a is 3
collection.deleteOne({ a :3 }, function(err, result) {
assert.equal(err, null);
assert.equal(1, result.result.n);
console.log("Removed the document with the field a equal to 3");
callback(result);
});
}
// Use connect method to connect to the Server
client.connect(function(err) {
assert.equal(null, err);
console.log("Connected successfully to server");
constdb = client.db(dbName);
//insert the documents and in the callback, close the client.
insertDocuments(db, function(result) {
findDocuments(db, function() {
updateDocument(db, function() {
removeDocument(db, function(){
client.close();
})
})
})
})
}); 

Note: All the methods on the collection returns promise object if callback is not passed. So to avoid callback hell as we see above, we can rely on promise and program the code better.

## Node express app with MongoDB to store and serve content

Now that we have fair idea of how to work with MongoDB using mongoshell and node.js driver. Let us build a simple app using node express app with MongoDB to store and serve content.

Note: Since this topic is more about MongoDB rather than express, we will keep the express app a basic version.

Let us go and comment out all the code in the app.js file from the previous example. We should be left with this:

// import the MongoDB and get the client object.
constMongoClient = require('MongoDB').MongoClient;
const { equal } = require('assert');
// Connection URL - can be obtained in the MongoDB server
consturl = 'MongoDB://localhost:27017';
// Database Name
constdbName = 'myproject'; 

Invoke a terminal and navigate to the directory where the app.js file is located.

Let us install express with the below command.

• npm install express –save

Import the express and body-parser as shown below:

constexpress = require('express');
constbodyParser = require('body-parser');
create an app instance and use bodyParser.json().
constapp = express();
app.use(bodyParser.json());
Let the express app be listening on port 3000 with a default route.
app.get('/', (req, res) =>res.send('Express app with MongoDB!'));
app.listen(3000, () =>console.log("App listening at http://localhost:3000")); 

Consolidated code:

constexpress = require('express');
constbodyParser = require('body-parser');
// import the MongoDB and get the client object.
constMongoClient = require('MongoDB').MongoClient;
const { equal } = require('assert');
// Connection URL - can be obtained in the MongoDB server
consturl = 'MongoDB://localhost:27017';
// Database Name
constdbName = 'myproject';
constapp = express();
app.use(bodyParser.json());
app.get('/', (req, res) =>res.send('Express app with MongoDB!'));
app.listen(3000, () =>console.log("App listening at http://localhost:3000")); 

In the terminal, run ‘node app.js’ and should see the console log that app is listening in port 3000. To check the default route is working correctly. Invoke the postman and perform get operation.

Now let us create a new route to retrieve the document based on field a and parameterized value.

app.get('/api/documents/:name', (req,res) => {
}) 

:name is the parameter being passed. So let’s retrieve it from request parameters. And since we know it is numeric and not string. Let’s parse it to integer.

constname = parseInt(req.params.name);

Now lets connect to the MongoClient as earlier. This time we will not be using callback approach rather rely on the promise being returned.

constclient = awaitMongoClient.connect(url, {useNewUrlParser:true});

Since returned object is a promise and in order for the promise to be resolved or rejected. We need to use await keyword before proceeding further.

Note: Promise object has two properties i.e. state and result. Promise state property initially has ‘pending’ and post execution will be either ‘fulfilled’(resolve) or ‘rejected’(reject). Similarly, result will be ‘undefined’ initially and then changes to ‘value’[resolve(value)] or ‘error’[reject(error)].

Let us get the db ‘myproject’ and find the first document based on the query filter from collection using findOne method.

constdb = client.db(dbName);
constdocument = awaitdb.collection('documents').findOne({"a":name}); 

Once we get the document, lets return the status as 200 with the obtained document and close the client connection.

res.status(200).json(document);
client.close(); 

As we are using await keyword, we need to add async keyword to the callback. And also lets add a try-catch block to return an error status in case of connection failure as internal server error.

app.get('/api/documents/:name',async (req,res) => {
try {
constname = parseInt(req.params.name);
constclient = awaitMongoClient.connect(url, {useNewUrlParser:true});
constdb = client.db(dbName);
constdocument = awaitdb.collection('documents').findOne({"a":name});
res.status(200).json(document);
client.close();
} catch(error) {
res.status(500).json({message:"Error connecting to db", error});
}
}) 

Now we are good to go and restart our express app. Press ctrl + c and run node app.js to see app is listening at. Make sure that MongoDB Server is up and running in the background.

Switch to postman and perform a GET operation. This should return the first record in the collection as shown below.

{"_id":"5eeefd2a16f61641f58a9419","a":2,"b":1}

Same GET request can be performed for other documents by changing the name as 2

Similar to get operations, we can configure the routes to perform other CRUD operations on MongoDB to store and serve the content.

Conclusion

In this guide, we have gone overMongoDB compatibility and language compatibility and explored the Mongo shell. We have seen how to import data into the database, how to use MongoDB driver and the Node.js based CRUD operations. We have also looked at how to build a simple app using node express app with MongoDB to store and serve content.

Armed with this knowledge, you are ready to take a deeper dive into MongoDB and NodeJS. Here’s wishing you the best!

Real coding requires real exercises. Practice live coding with our  experts on live workshops now.

### Sumanth Reddy

Author

Full stack, UI Architect having 14+ Years of experience in web, desktop and mobile application development with strong Javascript/.Net programming skills .

Strong experience in microsoft tech stack and Certified in OCI associate .

Go-to-guy on integration of applications, building a mobile app by zeroing on tech stack. Majorily experienced on engineering based IIOT products and Marekting Automation products deployed on premise and on-cloud.

## Back Up, Restore, and Migrate a MongoDB Database

Popular among both enterprises and startups, MongoDB is a database that is perfectly suited for web-apps that need to scale once the user base increases. MongoDB is different from traditional relational databases because it uses json like objects to store data, instead of tables in relational databases. In this post, we will learn to backup and restore a MongoDB database. In all software products there is an import and export feature, which in database terms, deals with human-readable format. On the other hand, the backup and restore operations use MongoDB specific data, which preserve the MongoDB attributes.  So, when migrating the database, we should prefer backup and restore over import and export. But we should also keep in mind that our source and target systems are compatible, which means that both should be Windows or both should be a Linux based system like Ubuntu/Mac. Prerequisites We are using Windows 10 in this tutorial. Please make sure you have downloaded the MongoDB Community Server and installed in it. It is a very easy setup and you will find lot of good articles on the internet detailing this out. Please ensure that you have added it in the Environment variable in your PC. Backup Considerations In a production environment, backups act as a snapshot of the database at a certain point. Large and complex databases do fail or can be hacked. If that happens, we can use the last backup file to restore the database to the point, before it failed. These are some of the factors which should be taken into consideration when doing a recovery.  1. Recovery Point Objective We should know the objective of the recovery point, which means how much data we are willing to lose during a backup and restoration. A continuous backup is preferred for critical data like bank information and backups should be taken several times during the day. On the other hand, if the data doesn’t change frequently, then the backup can be taken every 6 months.  2. Recovery Time ObjectiveThis tells how quickly the restoration can be done. During restoration the application will be down for some time; and this downtime should be minimized, or else customer relationships will be lost.  3. Database and Snapshot IsolationThis refers to the distance between the primary database server and the backup server. If they are close enough i.e., in the same building, then the recovery time reduces. However, in the event of a physical event such as a fire, there is a likelihood of it been destroyed with the primary database.   4. Restoration Process We should always test our backups in test servers to see if they will work, in case a restoration is required.  5. Available Storage Backup of database generally takes a lot of space and in most cases, it will never be required. So, we should try to minimize the space taken on the disk, by archiving the database into a zip file.  6. Complexity of DeploymentThe backup strategy should be easy to set and should be automated, so that we don’t have to remember to take the backup after regular intervals. Understanding the Basics The first thing that we should know is that MongoDB uses json and bson(binary json) formats for storing data. So, people coming from a JavaScript background can relate to objects for json, which have a key-value pair. Also, json is the preferred format in which we receive or send data to an API endpoint. You can check the json data of a MongoDB database in any tool or online editors. Even the famous Windows application Notepad++ has a json viewer. An example of a json document looks like below. As we can see from the above example, json is very convenient to work with, especially for developers.  But it doesn’t support all the data types available in bson. So, for backup and restoring, we should use binary bson. The second thing to keep in mind is that MongoDB automatically creates databases and collection names if they don’t exist during restore operations. Third, since MongoDB is a document-based database, in many user cases we store large amounts of data in one collection, such as the whole post of an article. MongoDB is also used extensively in large databases and big data. So, reading and inserting the data can consume a lot of CPU, memory and disk space. We should always run the backups during the non-peak hours like night. As already mentioned earlier, we can use import and export functions for backup and restoration of MongoDB databases, but we should use commands like mongodump and mongorestore to backup and restore respectively. MongoDB backup We will first cover backing up the MongoDB database. For this we use the mongodump command.  First open the Windows command prompt and go to the location in which MongoDB is installed. If you have chosen the default setting, while installing MongoDB though the pop-up it will be installed in a location like C:\Program Files\MongoDB\Server\4.4\bin The version number only will change if you are reading this blog in the future. Also, note that it’s better to run the command prompt in the Admin mode. So, once we open the command prompt, we need to change the directory to MongoDB bin folder by giving the below command. cd C:\Program Files\MongoDB\Server\4.4\binNow, enter mongod and press enter. It will show some json text.Now, we can backup to any location. For this post I am backing up on my Desktop in a Backup folder, which I have created through the command line.Now, we have to run mongodump command, but it should be also present in our MongoDB bin folder. If it is not present, we need to download it from and install it. After this, copy the entire exe files from the download to the MongoDB bin folder. MongoDB Backup with no option Now, run the mongodump command from the bin directory. Here, we are not giving any argument so the backup of the whole database will be taken in the same bin directory.MongoDB Backup to an output directory Now, run the mongodump command from the bin directory. Here, the argument –out specifies the directory in which the data backup will be maintained. In our case we are giving the Backup folder in the  Desktop, which we have created earlier. mongodump --out C:\Users\pc\Desktop\Backup Now, go to the desktop and you can find the backup that has been created in our Backup folder.  MongoDB Backup a specific database Now, we can also backup only a database in mongodump using the –db option. I have an example database, so to backup only that I will use the below command. mongodump --db example --out C:\Users\pc\Desktop\Backup As, you can see in the below output only the example database was backed up. MongoDB Backup a specific collection Now, if we want to only backup a specific collection, we need to use the –collection option and give the collection name. Also, note that the database name is mandatory in this case, as mongodb needs to know about the database to search for the collection. I have a products collection within the example database, so to backup only that I will use the below command. mongodump --db example --out C:\Users\pc\Desktop\Backup –collection products As, you can see in the below output only the products collection from example database was backed up. MongoDB Backup from remote MongoDB instances We can get the backup from remote mongodb instances also. I have a lot of MongoDB databases for my personal projects on MongoDB atlas, which is the free to use Cloud database for MongoDB. To get a backup of remote databases, we have to use the connection string with –uri parameter. I used the below command. mongodump --uri "mongodb+srv://xxxx:xxxxxxxxxxx@cluster0.suvl2.mongodb.net/xxxxxDB?retryWrites=true&w=majority" --out C:\Users\pc\Desktop\Backup You can see in the below output the backup of the remote instance. MongoDB Backup procedures We should try to make the backup procedure as automated as possible. One of the best ways is to use a cron job, so that it can run every day. As, discussed earlier it is best to run the backup in the night when the database has the least load.  Setting up a cron job is easier on a Linux or a Mac because the Windows equivalent of it is not good. Alternatively, you can do install mongodb in WSL2 for Windows which supports Ubuntu.  Suppose, on a Linux host which has a mongoDB instance running, you want to run the backup at 04:04 am daily. For this in the terminal, open the cron editor by running the below command in the terminal. sudo crontab –e Now, in the cron editor, you need to add a command like below for our case. 4 4 * * * mongodump --out /var/backups/mongobackups/date +"%m-%d-%y"Restoring and migrating a MongoDB database When we restore the MongoDB database from a backup, we will be able to take the exact copy of the MongoDB information, including the indexes. We restore MongoDB by using the command mongorestore, which works only with the binary backup produced by mongodump. Now, we have taken the backup of example database earlier and it is in our Backup folder. We will use the below command to restore it. In the arguments we will specify the name of the database first with –db option. After that with –drop, we make sure that the example database is first dropped. And in the final argument, we specify the path of our backup. mongorestore --db example --drop C:\Users\pc\Desktop\Backup\example Now, if we check in terminal, we have our example database restored properly. Conclusion In this article, we have learned about MongoDB backup and restore. We have learned the different options for the backups, and why and when backups are required. Keep learning!
5639
Back Up, Restore, and Migrate a MongoDB Database

Popular among both enterprises and startups, Mong... Read More

## How to Work With Forms In JavaScript

Forms also referred as web forms are a very important part of front end web application development for sake of interaction with users. Most commonly, forms are used to collect the data from users or provide a provision for user to control the user interface. Forms are great potential assets if correctly used in building an interactive web application. We would be touch basing some of the essential aspects of them like HTML structure, styling form controls, events, data validation and submitting data to server.Understanding forms in detail needs expertise in other areas than just HTML like styling form controls (CSS), scripting to validate or create custom controls (JavaScript).We would be referring or using libraries like Jquery (for document traversal, manipulation etc) and parsley (form validation library) to build better forms.A typical form’s HTML is made of HTML elements called as form controls like single or multiline text fields, dropdowns, checkboxes, button etc mostly created using element with specific type being set on Type attribute. These form controls can be programmed to add some validations to support specific values based on constraints set on them. These controls can be enriched to support accessibility for enabling the interaction for less privileged users.Let’s create a simple html page to build a form.           Learning Forms       All forms have to start with element which is container having the form fields user would interact with. All attributes of element are optional but for programming forms to capture data we need at least ‘action’ and ‘method’ attributes.action – is basically the URL where the form fields data would be sent to.method – corresponds to the HTTP method to submit the form data. Possible HTTP method names which can be set as values are post and get. And another value dialog is set when form is imbedded inside a .Note: Both formaction and formmethod can be overridden by button, input with type submit elements which we will learn as we go forward.Refer to this link to know more about form attributes.Let’s add a form element to our body with action (“”) and method(“get”). This implies that form will send a GET request to the current URL. If it is post then it would be a POST request to the URL in action. Add few fields to form say name, email and a submit button using with type being specified as text, email and submit respectively.Note: The tag is an empty element, meaning that it doesn't need a closing tag. Value attribute can be populated to set the default value.          Enter your name:                    Enter your email:                       Save and open the html in chrome or your preferred browser. Clicking on ‘Click me!’ should send a http get call with empty name and email.Note: We can use instead of with type as submit. The difference is that button can contain HTML content allowing to create a complex button whereas input allows only plain text.Let’s understand the Sending of form data.If we observer all the form fields again, we have added an attribute called ‘name’. This property is important to inform that which data is associated with which form field i.e. name/value pairs. Try adding some data to our fields rendering in html (say myName and first.last@email.com) and click submit button. You should see the data being sent as query parameters in the browser URL.?name=myName&email=first.last@email.com.Change the Form method value to POST instead of GET and send the submitted data by clicking the ‘Click me!’ button. You should be seeing Form Data being sent but the browser URL will not get update.name: myName email: first.last@email.comAll this while, we have our action method being set as empty. Replace this with another URL on server side say ‘/captureFormData’. Now on clicking submit button the data should be received by the script at ‘/captureFormData’ with key/value items contained in the HTTP request object.Note that each server-side language like Node.js, C# etc have their own way of handling the submitted form data. And this blog would not cover those topics and it is beyond the scope.Let’s refine our basic form structure with help of other HTML elements like , , etc. Though we used few of them in basic example. Let’s go little deep on them.Note: Nesting of form inside another form is unacceptable as it might result in unpredictable behavior. is a convenient way of grouping for sake of styling and semantic purpose. This control can be associated with so that some assistive technologies can read this legend and associate it with the controls inside the . Let’s understand this will an example:         Interested programming language                             JavaScript                                     CSharp                                     Java               When reading the above form by any screen readers, it will read as “Interested programming language JavaScript” for the first radio, “Interested programming language CSharp” and “Interested programming language Java” for second and third radio.Imagine if you have a long form with multiple fields. It would help to improve the usability if we can categorize/section them with the help of . It would even help to improve the accessibility of forms.Talking about accessibility, with the associated correctly with the via its for attribute (which contains the element's id attribute), a screenreader will read out something like "name, edit text" for below one.Enter your name: Another advantage of having label associated with input of type text, radio etc is they are clickable too.  If you click on a label then the associated input control will get the focus. If the input control is of type checkbox or radio, clicking on label will select the check box and radio. This will be useful as clickable area of checkbox or radio is small and having label gives provision to select it easily.Note: We can always associate multiple labels to a single input control but it is not a good idea as it will impact the accessibility and assistive technologies. along with can be used to separate the functionality in a form and group the same purpose elements like radio buttons.Here is an example of the same.               Contact information                   Title                                                                             Mr                                                                                                 Mrs                                                                               Name:                                                           E-mail:                                                           Password:                                                 Additional information                               Social type:                                 LinkedIn             Twitter             Instagram                                                 Phone number:                                                           Submit                   Every time you like to create an HTML form you need to start using element and  nesting all the content controls inside it. Most of the assistive technologies and browser plugins can help to discover elements and implement special hooks to make them easier to use.We have already some of the form elements like , , , , , and . Other common input types are button, checkbox, file, hidden, image, password, radio, reset, submit, and text.Input types.Attributes of Input.Few attributes on element help in validating the data like required, max, maxlength, min, minlength, multiple, pattern, step etc based on their respective type.Also other attributes on of type submit/image like formaction, formmethod, formnovalidate, formenctype etc helps in overriding the form level methods.ValidationBefore submitting the data to the server, it is important to perform some client side validation to avoid unwanted round trips. Client-side validation is needed but it is not a replacement to the server side validation. Advantage of having client side validation is to capture the invalid data and fix it immediately.Some of the important and popular checks which are most commonly used on client areField requiredSpecific data formatEnter valid email addressPassword and more…Let’s build a form with the above validation checks.                         Do you have experience in programming ?*                     Yes           No                             How many years of experience you have ?                                     What's your programming language?*                           TypeScript           Java           CSharp           Ruby           Go           Swift                             What's your company e-mail address?                             Cover letter                             Submit       Say, if we enter an value which is more than 40 in experience field. We should see an inbuilt error as shown below:All these validations and notifications are coming out of the box. Thanks to inbuilt functionality in control. Let’s see how we can perform validation of forms using JavaScript and take control of look and feel of error message.Most browsers support constraint validation API by providing few validation properties on HTML elements like , , , etc.validationMessage: we can customize this message if the control value failed validation otherwise it will return an empty string. It is dependent on other constraint i.e. willValidate and isValid.willValidate: If element is validated then it will be true otherwise false.validity: is the validity state of the element and it is dependent on other properties likepatternMatch for specified pattern attribute,tooLong and tooShort are for string fields based on maxLength and minLengthrangeOverflow and rangeUnderflow for numeric fields based on max and min attributestypeMatch for fields which are based on email or url.valid if all the validation constraints are metvalueMissing if the field is set as required.Along with properties, we do also have methods to perform validation like checkValidity() which returns true or false and setCustomValidity(message) is to set the message if the element is considered invalid. Also if the element is invalid then checkValidity will raise an event called invalid Event.Let’s create a simple form and customize the validation message.       Please enter an email address:             Submit     Add a script tag and customize the message as shown below:     const email = document.getElementById("mail");     email.addEventListener("input", function (event) {       if (email.validity.typeMismatch) {         email.setCustomValidity("I am expecting an e-mail address!");       } else {         email.setCustomValidity("");       }     });   Here we are listening to the input event on email field and checking if the validity on the control is valid or not and based on that we are setting the custom message.Here are we relying on inbuilt validation method. Let’s disable the validation at form level by with the help of ‘novalidate’ and take control over validation. This would mean the browser will not perform auto check on validation before sending the data. But still we have access to constraint validation API to perform validation ourself.Refine the above form to add few addition validation like required and minLength etc.               Please enter an email address:                             Submit     Let’s update the script to handle the validation     const form  = document.getElementsByTagName('form')[0];     const email = document.getElementById('mail');     const emailError = document.querySelector('#mail + span.error');     email.addEventListener('input', function (event) {       // Each time the user types something, we check if the form fields are valid.       if (email.validity.valid) {         // In case there is an error message visible, if the field is valid, we remove the error message.         emailError.textContent = ''; // Reset the content of the message         emailError.className = 'error'; // Reset the visual state of the message       } else {         // If there is still an error, show the correct error         showError();       }     });     form.addEventListener('submit', function (event) {       // if the email field is valid, we let the form submit       if(!email.validity.valid) {         // If it isn't, we display an appropriate error message         showError();         // Then we prevent the form from being sent by cancelling the event         event.preventDefault();       }     });     function showError() {       if(email.validity.valueMissing) {         // If the field is empty display the following error message.         emailError.textContent = 'You need to enter an e-mail address.';       } else if(email.validity.typeMismatch) {         // If the field doesn't contain an email address display the following error message.         emailError.textContent = 'Invalid value is entered, expected an e-mail address.';       } else if(email.validity.tooShort) {         // If the data is too short display the following error message.         emailError.textContent = Email should be at least ${ email.minLength } characters; you entered${ email.value.length }.;       }       // Set the styling appropriately       emailError.className = 'error active';     } Reload the HTML and try entering an invalid email address, the corresponding error message should be displayed.Note: In the current scope of this blog, we are not working on styling.Is it possible to validate forms without built in APIs ? Let’s see with the same example.We would consider the same form again but have lot of functionality in                           Please enter an email address:                                             Submit           const form  = document.getElementsByTagName('form')[0];     const email = document.getElementById('mail');     let error = email.nextElementSibling;     const emailRegExp = /^[a-zA-Z0-9.!#$%&'*+/=?^_{|}~-]+@[a-zA-Z0-9-]+(?:\.[a-zA-Z0-9-]+)*$/;     function addEvent(element, event, callback) {       let previousEventCallBack = element["on"+event];       element["on"+event] = function (e) {         const output = callback(e);         // A callback that returns false stops the callback chain and interrupts the execution of the event callback.         if (output === false) return false;         if (typeof previousEventCallBack === 'function') {           output = previousEventCallBack(e);           if(output === false) return false;         }       }     };     // Now we can rebuild our validation constraint. Because we do not rely on CSS pseudo-class, we have to explicitly set the valid/invalid class on our email field     addEvent(window, "load", function () {       // Here, we test if the field is empty (remember, the field is not required)       // If it is not, we check if its content is a well-formed e-mail address.       const test = email.value.length === 0 || emailRegExp.test(email.value);       email.className = test ? "valid" : "invalid";     });     // This defines what happens when the user types in the fiel     addEvent(email, "input", function () {       const test = email.value.length === 0 || emailRegExp.test(email.value);       if (test) {         email.className = "valid";         error.textContent = "";         error.className = "error";       } else {         email.className = "invalid";       }     });     // This defines what happens when the user tries to submit the data     addEvent(form, "submit", function () {       const test = email.value.length === 0 || emailRegExp.test(email.value);       if (!test) {         email.className = "invalid";         error.textContent = "Expecting an e-mail";         error.className = "error active";         return false;       } else {         email.className = "valid";         error.textContent = "";         error.className = "error";       }     });   On refreshing the page, the output with invalid email address should be displayed as shown below.In real time applications, we can rely on existing libraries like Parsley along with JQuery which would ease our life by taking away lot of complexity.Overview of Parsley:Parsley is a front-end javascript validation library which helps to give proper feedback to user on submission of form. As mentioned earlier, it is not a replacement of server side validation. Parsley library helps us to define our own validation.Parsley uses a DOM API namely ‘data-parsley-’ prefix on the existing properties. For example if we want to add this on a property say ‘sample’ then we would add as [data-parsley-sample=’value’]. This will allow us to configure pretty much everything without any configuration or custom function.There is no specific installation process but adding the corresponding script tags will enable the validation. Parsley is relied on Jquery so it has to be included as well.             ...                 $('#form').parsley(); Assumption is that we have downloaded the Jquery and Parsley minified librarie and added it to our working directory. Otherwise we can refer to CDN location as shown below. Adding attribute ‘data-parsley-validate’ to each form will allow us to validate. And “$(‘#form’).parsley()” will manually bind Parsley to your forms.Let’s understand further by configuring the attributes via JavaScript. For which, lets add two input fields inside the form element.                 Also let’s update the content to perform some pre-defined validation based on attributes.       var instance = $('#first').parsley(); console.log(instance.isValid()); // maxlength is 42, so field is valid$('#first').attr('data-parsley-maxlength', 4);       console.log(instance.isValid()); // No longer valid, as maxlength is 4       // You can access and override options in javascript too:       instance.options.maxlength++;       console.log(instance.isValid()); // Back to being valid, as maxlength is 5       // Alternatively, the options can be specified as:       var otherInstance = $('#second').parsley({ maxlength: 10 }); console.log(otherInstance.options); In the console.log, we should see thistrue false true {maxlength: 10}Options are inherited from the global level to form level and further to field. So if we set the options at global level then the same can be observed at field level. Parsley.options.maxlength = 42; // maxlength of 42 is declared at global level var formInstance =$('form').parsley(); var field = $('input').parsley(); console.log(field.options.maxlength); // Shows that maxlength is 42 inherited from global Parsley.options.maxlength = 30; console.log(field.options.maxlength); // Shows that maxlength is automatically 30 formInstance.options.maxlength++; console.log(field.options.maxlength); // Shows that maxlength is automatically 31We can also add our own custom validations. Let understand this with an example. window.Parsley.addValidator('multipleOf', { requirementType: 'integer', validateNumber: function(value, requirement) { return 0 === value % requirement; }, messages: { en: 'This value should be a multiple of %s', } }); Here we are adding a new attribute namely ‘data-parsley-multiple-of’ which takes only numeric values which are multiples of 3.In window.Parsley, we added a new validator with name ‘multiple-of’ with an object containing few important properties like ‘requirementType’, ‘validateNumber’ and ‘messages’ to be shown. This properties helps the library to check if the input value is valid or not.Similar to validateNumber, other properties are also there for different types like validateString, validateDate and validateMultiple.Also for requirementType, we have different options like string, number, date, regexp, boolean etc.Messages by default has English format, to support multiple locales we need to add the specific localization and also add specific locale.Events: Parsley triggers events that allows ParsleyUI to work and for performance reasons they don’t rely on JQuery events but the usage is similar to JQuery i.e. parsley events will also bubble up like JQuery events. For example, if a field is validated then the event ‘field:validate’ will be triggred on the field instance then on to form instance and finally to the window.Parsley.$('#some-input').parsley().on('field:success', function() {         // In here, this` is the parlsey instance of #some-input       });       window.Parsley.on('field:error', function() {         // This global callback will be called for any field that fails validation.         console.log('Validation failed for: ', this.$element); });Many times, we need some validation based on the response from server. Parsley provides an attributes i.e. data-parsley-remote and data-parsley-remote-validator to perform the same.Let’s consider this HTMLLet’s add the async validator on the window.Parsley object.window.Parsley.addAsyncValidator('customValidator', function (xhr) { console.log(this.$element); // jQuery Object[ input[name="q"] ]           return 404 === xhr.status;         }, 'customURL');Parsley is a very useful and powerful JavaScript form frontend validation library.Note: For developers building react based web applications, they can rely on FORMIK which is most popular library for building forms in React and React Native.ConclusionForms are important in HTML and it was needed and still needed now. is an html tag that allow us to perform HTTP methods like GET/POST operation without writing any code in JavaScript. Form defines an boundary to identify all set of the form field elements to be submitted to the server. For example, if we perform an enter key or clicking on submit button , the agent triggers form submission data based on each form field value to the server based on the action URL on the form.Before HTML5, all the elements are expected to be part of the to send the data to server. In HTML5, they maintained the backward compatibility and also enhanced the capabilities who may want to use AJAX and don’t want to rely on default behaviours i.e. they have enabled designers who expect more flexibility in having their form elements outside the form and still maintain the connections with the form.
1680
How to Work With Forms In JavaScript

Forms also referred as web forms are a very import... Read More