Search

Top Devops Tools You Must Know

In the last decade for most of the enterprises, the term DevOps has transformed from just a buzzword to a way of working. The concept of DevOps originated in 2008 following a discussion on agile infrastructure by Patrick Debois and Andrew Clay Shafer. The idea started to gain momentum in 2009 after the first DevOpsDays in Belgium. What initially began as a practice to bring more efficiency in software infrastructure management, is now evolved into a continuous feedback model which has redefined every aspect of software development from requirement engineering to deployment. With this change, evolved new frameworks, practices and tools rooted in the core values of lean and agile. This paper discusses in detail the various tools that evolved during the DevOps movement. Readers would get a comprehensive understanding of what and where to apply these tools in their day to day DevOps journey.1. What is DevOps?DevOps is a culture where active collaboration between development, operations, and business teams are achieved. It’s not all about tools and DevOps in an organisation is to create value to end customer respecting human all team members. Tools are only aids to build this culture. DevOps increases organizations capability to deliver high-quality products or services at a swift pace. It automates all processes starting from build to deployment phase of an application. There are many tools available in the market to help us achieve this.2. What are the DevOps tools?DevOps tools are categorized into following categories:a. Collaboration Tools :DevOps teams rely on regular feedback and constant communication. Hence traditional email communication mechanism becomes less effective. Thus DevOps teams rely on more integrated collaboration suites that help in continuous communication and feedback loops. Some of these new generation collaboration tools include Slack, Teams, CA Flowdock etc.1. SlackSlack is a messaging tool for the teams providing a common place for all communications. We can set different channels for different kinds of work. Voice and video call options are also available with Slack. Atlassian and Slack have created a partnership and will be discontinuing other collaboration tools like Hipchat and Stride and will provide migration to Slack.Availability: Free version with limited features are available for users.For more details click here  2. CA FlowdockCA Flowdock is yet another collaboration tool from CA Technologies. It brings all conversations, chats, work items, etc to one place making it easier to prioritize work and solve problems.Availability: CA Flowdock is free for up to 5 member teams and free for non-profit organizations and student projects.Learn more about CA Flowdock here.     3. TeamsTeams is a unified communication platform by Microsoft. Teams combine workplace chats, video meetings, file storage, and application integration. The service also integrates with the company's existing Office 365 productivity suite and features extensions to integrate with non-Microsoft products and features.Availability: Teams is free for a small number of users.Learn more about Teams here.SL NoTool NameProsConsAvailability1.SlackIntuitiveSaaS productGood integration with other toolsThe video conferencing feature is not as great as its competitorsFreemium2.CA FlowdockEasy to configureIntegration with tools beyond CA tools is to be improvedFreemium for small users3.TeamOne stop shop -  Integrates file sharing, messaging, meetings, and other tools.Still early and could be a little buggyFreemium for small usersb. Top Application Life Management and Issue Tracking ToolsALM and planning tools help team members to plan their iterations by constantly getting feedback from the customers and prioritizing them. This helps to achieve visualisation of the works in hand, share plans, and track the progress. These tools make sure that all the team members are heard and addressed. Customer feedback is taken seriously and increases the responsiveness within the team. The tools enable teams to identify and track dependencies. It helps teams to plan their releases and sprints in a systematic way. Issue tracking tools enable features like auto triaging and assignment. Some of the tools are:1. JIRAJIRA is an issue tracking and project management tool from Atlassian. It could be used by small or large companies. Kanban and scrum boards which are simple and flexible are available with JIRA. It’s not free software.Availability: The pricing varies with the number of users.Learn more about JIRA here.  2. Mantis Bug TrackerMantis BT is an open source web-based issue tracker. It’s simple to use dashboard, helps to assign issues to developers and keep track of the issue progress. It is empowered with a built-in time tracking mechanism that helps the user to analyse the time spent by a developer on an issue.Availability: Paid version is available.For more details click here.3. TrelloTrello is a free project collaboration tool. It helps to manage projects with it’s simple and easy to work for boards. All tasks are defined as individual cards. These cards can be moved around helping the teams to visualise the work in progress.Click here for more information.4. CollabNet VersionOneCollabNet VersionOne is agile management  It helps in collaboration between teams at all levels to have a unified vision for software delivery.For more details about CollabNet VersionOne click here.  5. RallyRally is formerly known as CA Agile Central. It provides a platform to plan, track, prioritize work collaboratively. Thus improving visibility.6. OpsGenieOpsGenie is an incident management tool that helps to determine who should respond to events. It’s from Atlassian. It also helps in defining collaboration methods like video conferences etc. It’s free for small teams up to 5 users.Availability: Paid version is available which varies with the number of users and add on features.Learn more about OpsGenie here.7. Pivotal TrackerPivotal Tracker is an agile project management tool. Pivotal tracker helps to create public and private projects. Private projects are accessible only to the collaborators and it's the default setting. Public projects are available via URL in read-only mode. Edit permissions are given only to an invitee to the project. Open source software development process makes use of public projects.Availability: Pivotal Tracker for two projects,2GB of file storage, and a total of three collaborators. An upgrade from this could be only in the paid version.For more details click here.8. Azure BoardAzure Board is a tracking tool from Microsoft Azure. It helps to track and plan your projects via kanban boards, team dashboards etc. It supports all agile methodologies. Built-in analytics provide information about project progress and status.Availability: Azure Board is free for up to 5 users and unlimited stakeholders.Click here to know more about Azure Board.  9. TasktopTasktop is a stream management tool to integrate and synchronize development and operations tools together. It helps in tracking tasks across different task tracking systems.Availability: Tasktop is not a free tool but paid.Learn more about Tasktop here.10. KanboardKanboard is an open-source project used for project management. It is known for its super easy installations, great visualisation of the project tasks and drag and drops feature for project management.Availability: Free version of Kanboard is available.Click here for more details about Kanboard.SL NoTool NameProsConsAvailability1JIRAWidely  usedEnterprise-gradeLearning curveComplex to configurePaid2Mantis Bug TrackerFree and good communityPaid Hosting option availableNeed experts to configureGood for defects and simple projectsFree & open source3.TrelloEasy to useEasy to configureNot ideal for large teams /programsFreemium4CollabNet VersionOneWidely  usedEnterprise-gradeRich featuresLearning curveLess intuitivePai5RallyEnterprise-gradeEasy to set upLess intuitive and complex to learnPaid6OpsGenieRich features for issue tracking and on-call managementFeatures are limited to issue trackingFreemium7Pivotal TrackerRich feature set for trackingIntuitive and easy to use  Integrability with other toolsFreemium8Azure BoardIntegrates well with Microsoft toolchainLacks richness in feature set  in comparison with other enterprise-grade tools in the same segmentPaid9TasktopGood for Value Stream ManagementIntegrability with other toolsPaid10KanboardSimple to useLimited feature setNot ideal for large teams/programsFree & open sourcec. Cloud /iaas/paas/serverless toolsCloud along with Infrastructure as service and platform as service produces a platform for developing, testing and deployment of applications. Using such features DevOps reduces the much latency overload in acquiring and accessing assets. All private and public clouds provide support to DevOps tooling and thus reducing the cost spent for on-premises systems.Some of the platforms are1. AWSAmazon Web Services (AWS) is a cloud services platform, offering to compute power, database storage, content delivery, and other cloud-related functionalities.Availability: AWS is an on-demand cloud computing platform where we are charged on as you go basis.Learn more about AWS here.2. AWS LambdaLambda is a serverless computing platform from Amazon Web Services (AWS). It is a service that manages the computing resources and runs code in response to events.Availability: We are charged only on the computing time.Learn more about AWS Lambda here.3. AzureMicrosoft Azure is an enterprise-grade cloud computing service that helps in managing applications through Microsoft-managed data centers. It provides software as a service (SaaS), platform as a service (PaaS) and infrastructure as a service (IaaS).Click here to learn more about Azure.4. Google Cloud PlatformGoogle Cloud Platform, offered by Google, is a suite of cloud computing services. Platform as service, Infrastructure as a service, and serverless computing are provided by GCP.Click here to learn more about Google Cloud Platform.5. IBM cloudIBM Cloud is a suite of cloud computing services from IBM. It also provides infrastructure as a service (IaaS) and platform as a service (PaaS).Availability: Lite version of IBM Cloud is free and allows one instance per plan.Click here to learn more about IBM Cloud.6. OpenStackOpenStack is a free and open-source software platform for cloud computing, mostly deployed as infrastructure-as-a-service, whereby all virtual servers and other resources are made available to customers. It’s written in python.Learn more about OpenStack here.7. Cloud FoundryCloud Foundry is an open source cloud platform that helps to develop cloud applications. It’s from Pivotal.Learn more about Cloud Foundry here.8. HerokuHeroku is a  platform as a service cloud environment. Thus help developers to work entirely on cloud.Availability: Free version is available with limited features.Learn more about Heroku here.9. OpenWhiskApache OpenWhisk is an open source, distributed Serverless platform. OpenWhisk manages the infrastructure, servers and scaling using Docker containers.Click here for more details about OpenWhisk.SL NoTool NameProsConsAvailability1.AWSEnterprise-ready providerComplex cost structurePay as you go2.AWS LambdaServerless computingReduced operational costsLimit on concurrent executions after which causing Denial Of ServiceCharged for computing time3.AzureIntegrates better with many Microsoft toolsServices provided still needs to be improvedPay for resources used4.Google Cloud PlatformScalableBetter load balancingServerless computingCurrently, GCP has fewer services and features compared to AWS or AzurePay as you go5.IBM CloudEasy setupConsistent performanceDifficulty in scalingFree Lite version6.OpenStackMassive scalabilityEasy implementationComplex configurationsFreemium7.Cloud FoundrySupports on-premises and multi-cloud deploymentGreat privacy and securityLess feature set compared to AWS or AzurePaid8.HerokuAdvanced Continuous Integration PlatformHighly scalableLess reconfigurabilityFreemium9.OpenWhiskOpen event provider systemServerless computingNot efficient for long running applicationsPaid as per computing timed. Top Source control managementSource control management as practice stores and tracks the application and infrastructure code. Even delivery pipelines for an application is nowadays stored in source code repositories. Some of the tools are GitHub, Bit Bucket, Subversion, Mercurial, Rational ClearCase.1. GitHubGitHub is a popular repository hosting service using Git. Git is a  free and open source system. It’s of the ease with which it performs branching and merge operations. It’s a distributed version control system that adds to its preferences.Click here for more details.  2. MercurialMercurial is a free distributed version control system. It is very easy to learn compared to Git but the branching feature of Git is more widely loved. Big and small projects could be handled in Mercurial.Click here for more details.3. BitbucketBitbucket is a repository hosting service from Atlassian. It could be used to store source code using  Mercurial or Git revision control systems. It’s free for teams with a maximum of 5 users. The paid version is available for bigger teams.Learn more about Bit Bucket here.4. Rational ClearCaseRational ClearCase is a source control management tool from IBM. It helps in the parallel development of software. Software artefacts whether it be source code or design documents etc could be managed by ClearCase. Enterprise version is available for ClearCase.Learn more about ClearCase here.5. SubversionSubversion is a version control system from Apache. It’s a free tool and open source.It helps to track down all the changes done to files and directories.Click here to learn more about Subversion.  6. JFrog ArtifactoryArtifactory is an artefact repository management tool from JFrog. It’s a paid tool. It primarily stores binary files which are typically the product of our build process.Click here to learn more about Artifactory.SL NoTool NameProsConsAvailability1.GitHubEasy to navigate user interfaceDifficult to learnFree2.MercurialCannot rewrite commit historySlower network operationsFree3.BitbucketSupports Git and MercurialDifficult integration with other toolsFreemium4Rational ClearCaseIntegrates with Microsoft Visual StudioNot suitable for projects with a big code baseDifficult to work withPaid5.SubversionEasy to learn even for non-technical usersSlower because of centralised version control systemFree6.ArtifactorySupports many languages and toolsEasy to useExpensivePaide. Top Package managersPackage managers build or package code with all metadatas like software’s name, purpose, version and all dependencies needed by the software to function correctly. It lessens the burden of manual installs especially in big enterprises where we need to install big software. Some of the tools available are1. MavenMaven is an open-source build automation tool from Apache used mainly for Java applications. Main features are it provides easy and uniform builds. It also keeps aside a parallel space for test code.Learn more details about Maven here.  2. GradleGradle is also an open source build tool from Apache. It is built on Groovy domain-specific language. It is more like a combination of Ant and Maven.Learn more about Gradle here . 3. MSBuildMicrosoft Build automation tool is a free and open source mainly for  C++ and .NET applications. Visual Studio makes use of MSBuild to build its applications.Learn more about MSBuild here.  SL NoTool NameProsConsAvailability1.MavenAll dependencies are downloaded automaticallyBetter suited for java projectsComplex to work withLarge learning curveFree2.GradleCan write build script ourselvesPoor integration with eclipseFree3.MSBuildGreat community supportMainly for .NET applications onlyFreef. Continuous IntegrationIn continuous integration, a code is checked into the source code repository whenever a developer finishes a requirement or user story. Continuous Integration tools enable teams to build software application automatically in a decided time. Thus reducing the time elapsed in a manual build. Some of the popular tools available are   1. GitLab CIGitLab CI is an integrated part of GitLab, GitLab offers a continuous integration service.Availability: A free version is available with limited features.Learn more about GitLab CI here.  2. SemaphoreSemaphore is the fastest hosted continuous integration and delivery solution as claimed by its developers.Availability: Open source projects can use Semaphore for free in its full capacity, free use for private projects is limited to 100 builds per month.Learn more about Semaphore here.3. Circle CICircle CI's continuous integration and delivery platform make it easy for teams of all sizes to rapidly build and release quality software at scale. It is built for Linux servers and automates build, test and deployment processes.Availability: Circle CI has a free version available for a single container.Click here for more details about Circle CI.4. JenkinsJenkins is an open-source continuous integration tool written in Java. Jenkins is a fork by the core developers of Hudson after a dispute with Oracle. Jenkins is the most widely used CI tool. Availability: Both free and enterprise versions are available.Click here for more details about Jenkins.5. HudsonHudson is a continuous integration tool written in Java that runs in a servlet container such as Apache Tomcat or GlassFish.Click here for more details about Hudson.   6. CruiseControlCruiseControl is an open source continuous integration tool and extensible framework for facilitating a continuous build process. Distributed under a BSD-style license.Learn more about CruiseControl here.  7. BambooBamboo is a continuous integration (CI) server produced by Atlassian. Bamboo ties automated builds, tests, and releases together in a single workflow.Availability: Licensed version is available at a starting price of $10.Learn more about Bamboo here.8. Team Foundation BuildTeam Foundation Build (TFB) is part of the Team Foundation system and provides the functionality of a public build lab. With TFB, build managers can synchronize sources and compile.Click here to know more about Team Foundation Build9. GumpApache Gump is an open-source continuous integration tool, designed with the overarching aim of ensuring that projects are compatible at both the API level and regarding.Learn more about Gump here.10. Travis CITravis CI is an open-source distributed continuous integration (CI) service used to build and test projects hosted on GitHub. Open source projects can freely avail Travis CI.Availability: Travis CI is free for first 100 builds but after which it is priced.Learn more about Travis CI here.11. TeamCityTeamCity is an open-source CI platform from Jet Brains. It’s known for easy user interface and support for Microsoft stack.Availability: Free version of TeamCity is available with the limited feature set.Click here to know more about TeamCity.  12. Puppet PipelinesPuppet Pipelines makes software delivery easy and unites silos of automation across Dev and Ops teams. It automates your application builds and deployments.The community edition of Puppet Pipelines is available free of cost for up to three users.Click here for more details about Puppet Pipelines.  SL NoTool NameProsConsAvailability1.GitLab CIEasy to configureSource control and continuous integration in one placeNeed GitLab integrationFreemium2.Semaphore CISimple and to the pointLess user base and community supportFreemium3.CircleCIEasy to useLess known in the community4.JenkinsUses plugin model to integrate with several DevOps toolsGreater community supportCumbersome groovy syntaxesFreemium5.HudsonJenkins forked from Hudson so has all basic features of JenkinsNo much development of new features taking placeLess community supportOpen source6.CruiseControlGoes well with .NET applicationsDifficult setupOpen source7.BambooA lot of tasks available as a built-in option and not as pluginsGoes well with Atlassian products like Bitbucket and JIRAOnly paid option availablePaid8.Team Foundation BuildWorks smoothly with .NET applicationsIntuitive easy to installInteroperability with other stacks is a challengePaid9.GumpIntegrates well with Apache tools like MavenLess plugin supportOpen source10.Travis CIEasy to set up and configureSupports most technological stacks using Node, Ruby, etcDoesn’t  support BitbucketFreemium11.TeamCityGreat user interfaceEasy to learnCommunity support is good but not greatFreemium12.Puppet PipelinesEasy setup and installationPlugin availabilityFreemiumg. Top Continuous Delivery and Deployment toolsContinuous deployment tools automate the delivery pipeline of application development, thus reducing the wastage of time caused by transfer between different teams like development and release teams. Few of the most popular deployment tools used by DevOps teams are1. ChefChef is a tool used to manage and develop infrastructure. It could be used for application deployment also. It is an open-source tool but with an enterprise version available. Chef uses a  domain-specific language based on ruby to define and configure infrastructure. Chef allows high flexibility and typically preferred by developers. It has a higher learning curve compared to other tools in this space. Chef is known to be the most preferred tool for large scale, complex enterprise systems.Availability: Chef is free for up to a limited number of nodes which is five nodes now after which it’s priced.Learn more about Chef here.2. PuppetPuppet is another configuration management tool to define infrastructure as code. Puppet is an enterprise-grade tool. Puppet uses a more declarative language and hence makes it easier to work with. It’s preferred by operations teams as it doesn’t require programming skills.Learn more about Puppet here.3. Octopus DeployOctopus Deploy is a release management server from XebiaLabs. It’s used mainly for .NET applications and windows services. It’s a paid deployment as a service.Click here for more about Octopus Deploy.4. SpinnakerSpinnaker is an open source free release platform that increases the number of good-quality releases. This platform helps in deployment across multi-cloud providers like AWS EC2, Google Kubernetes Engine etc.Learn more about Spinnaker here.  5. GoCDGoCD is a free and open source server that helps in continuous delivery. It helps in creating a continuous delivery pipeline in cloud environments like Docker, AWS etc.Learn more about GoCD here.  6. UrbanCode DeployUrbanCode Deploy or uDeploy is a tool used to automate application deployment from IBM. It’s a licensed version and available as hosted services also.Click here to know more about UrbanCode Deploy.  7. XebiaLabs XL DeployXL Deploy is a release automation tool for any environment. It is a licensed version by XebiaLabs.Click here to know more about XL Deploy.8. AnsibleAnsible is an open source configuration management tool and application deployment tool. In comparison with Chef, Ansible works with a decentralised agentless architecture and hence it’s easy to get started with Ansible.Availability: CLI based Ansible is free for no limit on nodes.Learn more about Ansible here.  9. SaltStackSaltStack is an open-source configuration management software written in Python. It enables teams to craft  "Infrastructure as Code".SaltStack in comparison with Ansible is quickly scalable but enforces teams to learn python.Learn more about SaltStack here.  SL NoTool NameProsConsAvailability1.ChefGreat documentation availableHard to learnNeed programming skillsFreemium2.PuppetProgramming skills are not a mustNot much suitable for applications where updates are frequentPaid3.Octopus DeployEasy configurationIntegrates smoothly with TeamCityA quick and flexible deployment pipelineLess community support especially for non-Microsoft applicationsPaid4.SpinnakerGreatly preferred for cloud-based deploymentsLess community supportOpen source5.GoCDBetter suitable for end-to-end Continuous delivery pipeline where great visualisation needed.Less cost efficientA Steep learning curve with a confusing user interfaceOpen source6.UrbanCode DeploySimple and easy to useSlower deploymentsPaid7.XL DeployA large number of plugins availableLesser visibility for the deployment processPaid8.AnsibleSimpler installationEasy to useGUI is not that greatNo support for windowsOpen source9.SaltStackQuickly scalableUnderdeveloped GUIOpen sourceh. Testing automationTesting automation tools are used in close proximity to continuous integration and deployment tools. It helps in performing repetitive tasks unable to perform by manual tests. The automated test gives a more clear picture of the health of the software product without any bias.1. Unit testingUnit testing tools help to test a single unit or component of the software. Thus detecting the errors earlier and fixing. And, Unit testing also helps in smooth integration.2. Integration testingIntegration testing tools help validate every integration that happens in the integration phase. Only successful build of the code move to the next stage.3. End-to-end testingIn end-to-end testing, the entire system or application is checked from start to finish. The tools generate the reports which can be used to verify whether the new change is causing  any unexpected behavior from the entire system4. Performance testingPerformance testing tools analyse the system in an expected workload. The tools measure the responsiveness of the system, scalability, and stability. Tools also provide details on where the system is failing and where the system needs improvement.5. Infrastructure testing and auditingInfrastructure testing plays a very important part as an error in the infrastructure code can even alter the production environment creating unseen repercussions. Ensuring the compliance of an organisation is an integral part of such tools keeping security in mind.i. Other Popular DevOps toolsSome of the popular tools used are1. SeleniumSelenium is a free and open source testing framework for web applications. It’s a suite of four tools Selenium WebDriver, Selenium RC or Remote   Selenium IDE And Selenium-Grid.Learn more about Selenium here.2. CucumberCucumber is an open source testing tool. It’s the best choice for behavior driven development popularly known as BDD as it tests business readable requirements. Free and enterprise version of Cucumber is available.Click here for more details about Cucumber.3. InSpecInSpec is a free and open source testing framework from Chef. InSpec tests infrastructure. It’s also a compliance framework.Click here to know more about InSpec.  4. KarmaKarma is a free test runner created for testing, applications made with Angular CLI. It’s from the AngularJS team.Learn more about Karma here.5. Jasmine  Jasmine is an open source testing framework. It is used mainly for JavaScript applications. It’s used for behavior driven development also.Click here to learn more about Jasmine.6. UFTUFT or Unified Functional Testing is a test automation tool for web, desktop, mobile   Micro Focus. There is a 60-day free trial version available but after which it’s not a free tool.Learn more about UFT here.  7. SoapUISoapUI is an open-source testing tool for web applications. It’s the market leader in API testing. It’s a licensed tool.Click here to know more about Soap UI.  8. JMeterJMeter is an open source load testing tool from Apache. It helps in analyzing the performance of services mainly for web applications. It’s a free test suite.Learn more about JMeter here. SL NoTool NameProsConsAvailability1.SeleniumWide range of languages supportedEasy integration with Jenkins, MavenDifficult to useNo support officiallyOpen source2.CucumberGreat documentationSupports Behavior-driven developmentSlow compared to other testing toolsOpen source3.InSpecHighly flexible and can be used cross any Infra As code framework.Need to know the scripting languageOpen source4.KarmaEasy debuggingLesser user baseOpen source5.JasmineDifficult to debugEasy to set up and useOpen source6.UFTEasily integrated with continuous integration DevOps toolsLess compatibility with different operating systems ExpensivePaid7.SoapUIUser-friendlyPlugin availability is lessOpen source8.JMeterEasy installation Great user-friendly interfaceHigher learning curve Doesn’t support javascriptOpen source9. DockerDocker is a platform for working with containers, from Docker, Inc.Docker is an open source and available as free and enterprise version. Containers help to develop applications and package it with its dependencies and libraries, thus ensuring the application runs in any environment. Docker containers are like virtual machines but share the same OS resources like file system etc.It has less overhead unlike VM 's. The building block of a container is an image which is the executable package including libraries, dependencies, environment variables etc needed to run the application. Running instance of an image is a container.Learn more about Docker here.10. KubernetesKubernetes is an open source production grade container orchestration tool. It helps in managing multiple containers in an application. Kubernetes is the market leader in this category. It is often compared with Docker Swarm which is the native clustering method for docker.  Click here to know more about Kubernetes.  11. OpenShiftOpenShift from Red Hat is a group of containerization software. OpenShift Container Platform is the major software in the group that provides a platform as a service built around Docker containers. These docker containers are managed for experimenting by an individual a free version is available for one project.Learn more about OpenShift here.  SL NoTool NameProsConsAvailability1.DockerContainers are lightweight compared to virtual machinesSecurity is a concernOpen source2.KubernetesHighly scalableWork better with CI/CD pipelinesLess user-friendlyOpen source3.OpenShiftGreat community supportOnly supports Red Hat Enterprise LinuxFreemiumj. Release orchestrationRelease orchestration tools are used to achieve automation of the application release process. Some of the popular tools are Xebialabs XL release, Plutora Release, AWS Codepipeline, CACD Director, OpenMake, Spinnaker, HashiCorp Vault, SonarQube, BlackDuck, Signal Sciences, Checkmarx SAST.k. ContainerologyContainerology tools help to run an application on the virtual environment as a package with all dependencies. It avoids the situation “it doesn’t work in my system”.Some of the tools are:l. Monitoring ToolsMonitoring tools help to pinpoint and track issues and verify the health of the system. This enables fast recovery of the system with minimum or no human interventions.Popular tools are:1. PrometheusPrometheus is an open source monitoring tool from SoundCloud.Its mainly used with systems using microservices as it has a multi-dimensional data collection feature. It uses a flexible query language PromQL.All Prometheus server is standalone and doesn’t depend on network storage helping us to understand the defect especially during outages. Collected data through the multi-dimensional collection feature may not be too detailed and having complete information. So it is not suitable for systems where 100% accuracy is required.Click here for more details about Prometheus.2. SplunkSplunk as a monitoring tool is used across application management, security, compliance, web analytics etc. Splunk tools listen and store data, index the same and correlate the captured real-time data in a searchable repository from which it can generate useful graphs, reports, alerts, and various other visualizations. One can create and configure relevant dashboards based on various visualizations/ graphs.  Learn more about Splunk here.3. NagiosNagios is an open source and free tool to monitor services, applications and infrastructure. It’s known for its auto-discovery feature. Its user interface is a bit difficult for beginnersLearn more about Nagios here.  4. ZabbixSimilar to Nagios, Zabbix is an enterprise open source monitoring solution. Compared to Nagios, Zabbix is user-friendly and is comparatively easy to configure. The main disadvantage is that it doesn’t support plugins.Learn more about Zabbix here.5. ZenossZenoss is a free, open-source tool used for services and network monitoring. It is written in Python language.Click here to know more about Zenoss.6. ELK Stack"ELK" is the acronym for three free open source projects: Elasticsearch, Logstash, and Kibana. Elasticsearch is a search and analytics engine based on Java. Logstash is a server‑side data processing platform with the ability to clean, transform data and send it to Elasticsearch.Kibana is a virtualization tool that helps to visualize data with charts and graphs in Elasticsearch.The Elastic Stack is the next evolution of the ELK Stack.Learn more about ELK Stack here.SL NoTool NameProsConsAvailability1.PrometheusEasy to useEasy integration with other DevOps toolsBad user interfaceOpen source2.NagiosNumber of plugins available in the marketDifficult configurations needed to make the system stableOpen source3.ZabbixEasy configuration based on a web-based user interfaceNon-availability of pluginsOpen source4.ZenossGreat community supportLimitation on the number of devices monitoredOpen source5.ELK StackEasy to installHighly customisableDifficult to configureOpen source6SplunkEasy to installUser-friendlyEasy to configure  simple graphsFor complex configurations, the learning curve is a bit steepPaidm. AnalyticsAnalytics tools give a clear picture of what is happening in the team, be it code development or team interaction, code coverage and efficiency etc. Some tools used are XebiaLabs XL Impact,New Relic,Dynatrace,Datadog,AppDynamics, ElasticSearch .Bonus Information3. Why is DevOps needed?DevOps helps to remove silos in organisations and enable the creation of cross-functional teams, thus reducing reliance on any one person or team during the delivery process. Frequent communication between teams improves the confidence and efficiency of the team members. Through automation, DevOps team increase their productivity making satisfied customers. According to State of DevOps report 2016 “Teams that practice DevOps deploy 30x more frequently, have 60x fewer failures, and recover 160x faster“. It also provides better work environments with increased trust, better management of issues reducing unplanned works.4. How to implement DevOps?The “DevOps Handbook” defines the “Three Ways: The principles of underpinning DevOps” as a way to implement DevOps in large enterprises. In this session, we will detail these three ways and three core pillars.The First Way: Systems ThinkingThe First Way emphasizes the need for global optimisation as opposed to local optimisation, hence the focus is on optimising all business value streams enabled by IT.The Second Way: Amplify feedback loopsThe Second Way is about discovering and injecting right feedback loops so that necessary corrections can be made before it’s too late.The Third Way: Culture of Experimentation and learningThe third way is all about creating the right culture that fosters two things, continual experimentation and learning from failures. It emphasises the understanding that repetition and practice make teams perfect.While the three ways focus on the key principles, we also have three pillars which are keys to any successful DevOps adoption.The three pillars of any DevOps adoption are,Culture and PeopleTools and TechnologyProcesses and practices4.1 Important DevOps practicesa. Continuous IntegrationContinuous integration is a software engineering practice where software development team members frequently merge and build their code changes. The key benefit is to detect and fix code merge conflicts and integration bugs in the early stages of software development. Hence reducing the cost to detect and fix the issues.b. Continuous DeliveryContinuous delivery is a software engineering practice in which changes are automatically built, tested, and made release ready to production. In order to get into a continuous delivery state, it is very crucial to define a test strategy. The main goal is to identify functional and non-functional defects at a much earlier stage thus reducing the cost to fix defects. It also enables teams to come up with working software as defined in the agile manifesto. Continuous delivery as a practice depends on continuous integration and test automation. Hence it is crucial that teams need to ensure that they practice continuous integration along with test automation religiously, to effectively practice continuous delivery.c. Continuous DeploymentContinuous deployment is a software engineering practice in which codes committed by the developers are automatically built, tested and deployed to production. Continuous deployment as a practice, require that teams have already adopted continuous integration and continuous delivery approach. The primary advantage of this practice is reducing time-to-market and early feedback from users.d. Continuous TestingContinuous Testing can be defined as a software testing practice that involves a process of testing early, testing often and test automation. The primary goal of Continuous Testing is to shift left the test phase as much as possible to identify defects and reduce the cost of fixing.e. MicroservicesMicroservices architecture helps to create an application as a set of small services independent of each other. Any language could be used to create microservices and typically an HTTP based API is used to interact between services. Microservices as a design approach helps to achieve fewer risk deployments and enables continuous delivery.f. Infrastructure as codeInfrastructure as a code is an engineering practice in which infrastructure is developed and managed through code. Thus creating a consistent, reproducible and versioned infrastructure. Since the infrastructure is implemented as the code it’s easy for the team members to update and change it. Infrastructure as a code no more considers scaling as a major problem.g. Policy as codePolicy as a code is a software engineering practice where compliance rules or policies of the organisation could be monitored and verified. Policy as code enables organizations to enforce the compliance rules more strictly and helps to bring the non-compliant resource into compliance mode. This practice gained importance during the DevSecOps movement.h. Continuous Monitoring and LoggingMonitoring and logging as a best practice to help organizations to analyse the products’ end user experience. This helps the software teams to get to know about the root cause of the defects and latencies in the software development process. More transparency into the actions performed by the team members causes increased responsibility among the teams causing increased performance.i. Communication and collaborationEffective communication and collaboration are one of the key values emphasised by DevOps.Devops tools in the field of communication and collaborations bring together collective responsibility for the products delivered.   5. How to choose the right DevOps tools?Today the DevOps market is overcrowded with tools across different stages of software development life cycle. As enterprises, it's extremely crucial to select the right tools in order to get maximum benefit. Saying so choosing the right tool is an extremely difficult and time-consuming process given the spectrum of tools available today. Hence enterprises should have a five-point strategy towards deciding the right tools. Five point strategy would include dimensions likeAbility to integrateScalabilitySecurityTechnical know howReliability5.1. Ability to integrateThe ability to integrate is extremely crucial and is one of the fundamental requirements while checking out tools. Certain tools integrate smoothly with a particular technology stack in comparison with others. Hence it is vital for the DevOps architect to compare different tools on the basis of integration ability and ensure that tool that is been selected seamlessly integrates with the team’s technology stack. Another aspect that needs to be considered is how a particular tool integrates with other tools that are selected in the ecosystem. For eg., you would want your continuous integration system to constantly talk to the reporting system and alert prediction system in a smooth way. Hence integration between tools also becomes a very important factor while choosing tools.5.2. ScalabilityScalability is the second most important factor in choosing the right tools. Based on the need for scalability an enterprise might choose an enterprise version over a community version. Scalability also is a key factor why certain companies go for SaaS-based products. SaaS-based products are easily scalable and hence without any overhead, it can be adopted across large enterprises.5.3. SecurityThese days a lot of enterprises are emphasising on the need for security in the DevOps tooling space. Hence enterprise versions by various tooling companies have taken special care towards addressing these security-related issues. Thus enterprise versions are comparatively more preferred in comparison with that of open source solutions. Saying so this doesn't mean that all open source DevOps tools have security vulnerabilities. Certain open source DevOps tools fair much better than available enterprise versions5.4. Technical know howThis people dimension is one of the factors that is typically overlooked by enterprises. Knowing the skill levels and capability of team members is a key towards choosing the right tool. Often the tools available in the market wouldn't work out of the box and would need a substantial level of customisation to smoothly integrate with existing systems and workflows. Also, certain tools require a certain specific skill set towards configuration and customisation. Typical eg. is Chef, which is chosen by developers who are comfortable in ruby language whereas Puppet is preferred by system admins as it does not require much of programming skills.5.5. ReliabilityLast but not the least, reliability is extremely crucial for any successful tool adoption. Most of the tools available in the market, both enterprise and open source needs to be checked using this quality wheel. Tools should be reliable even during large scale and complex operational conditions. ConclusionIn this paper, we discussed the what, why, and how of DevOps.We also deep dived into various tool categories and tools available across the spectrum in today's DevOps market. Tools are definitely the key ingredients in successful DevOps adoption but saying so a lot of companies only invest in tool part without focusing on cultural and people dimensions. In order for tools to bear fruit its vital that the people operating and analysing the tools/data understand and realise the true spirit of DevOps. To conclude, would like to resonate with the wise words “yes, we need all the tools that can help us, but just tools will not help us get there!”.

Top Devops Tools You Must Know

10K
Top Devops Tools You Must Know

In the last decade for most of the enterprises, the term DevOps has transformed from just a buzzword to a way of working. The concept of DevOps originated in 2008 following a discussion on agile infrastructure by Patrick Debois and Andrew Clay Shafer. The idea started to gain momentum in 2009 after the first DevOpsDays in Belgium. What initially began as a practice to bring more efficiency in software infrastructure management, is now evolved into a continuous feedback model which has redefined every aspect of software development from requirement engineering to deployment. With this change, evolved new frameworks, practices and tools rooted in the core values of lean and agile. This paper discusses in detail the various tools that evolved during the DevOps movement. Readers would get a comprehensive understanding of what and where to apply these tools in their day to day DevOps journey.

1. What is DevOps?

DevOps is a culture where active collaboration between development, operations, and business teams are achieved. It’s not all about tools and DevOps in an organisation is to create value to end customer respecting human all team members. Tools are only aids to build this culture. DevOps increases organizations capability to deliver high-quality products or services at a swift pace. It automates all processes starting from build to deployment phase of an application. There are many tools available in the market to help us achieve this.

2. What are the DevOps tools?

DevOps tools are categorized into following categories:

a. Collaboration Tools :

Collaboration Tools

DevOps teams rely on regular feedback and constant communication. Hence traditional email communication mechanism becomes less effective. Thus DevOps teams rely on more integrated collaboration suites that help in continuous communication and feedback loops. Some of these new generation collaboration tools include Slack, Teams, CA Flowdock etc.

1. Slack

Slack is a messaging tool for the teams providing a common place for all communications. We can set different channels for different kinds of work. Voice and video call options are also available with Slack. Atlassian and Slack have created a partnership and will be discontinuing other collaboration tools like Hipchat and Stride and will provide migration to Slack.

Availability: Free version with limited features are available for users.

For more details click here  

2. CA Flowdock

CA Flowdock is yet another collaboration tool from CA Technologies. It brings all conversations, chats, work items, etc to one place making it easier to prioritize work and solve problems.

Availability: CA Flowdock is free for up to 5 member teams and free for non-profit organizations and student projects.

Learn more about CA Flowdock here.     

3. Teams

Teams is a unified communication platform by Microsoft. Teams combine workplace chats, video meetings, file storage, and application integration. The service also integrates with the company's existing Office 365 productivity suite and features extensions to integrate with non-Microsoft products and features.

Availability: Teams is free for a small number of users.

Learn more about Teams here.

SL NoTool NameProsConsAvailability
1.Slack
  • Intuitive
  • SaaS product
  • Good integration with other tools
  • The video conferencing feature is not as great as its competitors
Freemium
2.CA Flowdock
  • Easy to configure
  • Integration with tools beyond CA tools is to be improved
Freemium for small users
3.Team
  • One stop shop -  Integrates file sharing, messaging, meetings, and other tools.
  • Still early and could be a little buggy
Freemium for small users

b. Top Application Life Management and Issue Tracking Tools

Application Life Management and Issue Tracking Tools

ALM and planning tools help team members to plan their iterations by constantly getting feedback from the customers and prioritizing them. This helps to achieve visualisation of the works in hand, share plans, and track the progress. These tools make sure that all the team members are heard and addressed. Customer feedback is taken seriously and increases the responsiveness within the team. The tools enable teams to identify and track dependencies. It helps teams to plan their releases and sprints in a systematic way. Issue tracking tools enable features like auto triaging and assignment. Some of the tools are:

1. JIRA

JIRA is an issue tracking and project management tool from Atlassian. It could be used by small or large companies. Kanban and scrum boards which are simple and flexible are available with JIRA. It’s not free software.

Availability: The pricing varies with the number of users.

Learn more about JIRA here.  

2. Mantis Bug Tracker

Mantis BT is an open source web-based issue tracker. It’s simple to use dashboard, helps to assign issues to developers and keep track of the issue progress. It is empowered with a built-in time tracking mechanism that helps the user to analyse the time spent by a developer on an issue.

Availability: Paid version is available.

For more details click here.

3. Trello

Trello is a free project collaboration tool. It helps to manage projects with it’s simple and easy to work for boards. All tasks are defined as individual cards. These cards can be moved around helping the teams to visualise the work in progress.

Click here for more information.

4. CollabNet VersionOne

CollabNet VersionOne is agile management  It helps in collaboration between teams at all levels to have a unified vision for software delivery.

For more details about CollabNet VersionOne click here.  

5. Rally

Rally is formerly known as CA Agile Central. It provides a platform to plan, track, prioritize work collaboratively. Thus improving visibility.

6. OpsGenie

OpsGenie is an incident management tool that helps to determine who should respond to events. It’s from Atlassian. It also helps in defining collaboration methods like video conferences etc. It’s free for small teams up to 5 users.

Availability: Paid version is available which varies with the number of users and add on features.

Learn more about OpsGenie here.

7. Pivotal Tracker

Pivotal Tracker is an agile project management tool. Pivotal tracker helps to create public and private projects. Private projects are accessible only to the collaborators and it's the default setting. Public projects are available via URL in read-only mode. Edit permissions are given only to an invitee to the project. Open source software development process makes use of public projects.

Availability: Pivotal Tracker for two projects,2GB of file storage, and a total of three collaborators. An upgrade from this could be only in the paid version.

For more details click here.

8. Azure Board

Azure Board is a tracking tool from Microsoft Azure. It helps to track and plan your projects via kanban boards, team dashboards etc. It supports all agile methodologies. Built-in analytics provide information about project progress and status.

Availability: Azure Board is free for up to 5 users and unlimited stakeholders.

Click here to know more about Azure Board.  

9. Tasktop

Tasktop is a stream management tool to integrate and synchronize development and operations tools together. It helps in tracking tasks across different task tracking systems.

Availability: Tasktop is not a free tool but paid.

Learn more about Tasktop here.

10. Kanboard

Kanboard is an open-source project used for project management. It is known for its super easy installations, great visualisation of the project tasks and drag and drops feature for project management.

Availability: Free version of Kanboard is available.

Click here for more details about Kanboard.

SL NoTool NameProsConsAvailability

1

JIRA

  • Widely  used
  • Enterprise-grade
  • Learning curve
  • Complex to configure
Paid

2


Mantis Bug Tracker

  • Free and good community
  • Paid Hosting option available
  • Need experts to configure
  • Good for defects and simple projects

Free & open source

3.


Trello

  • Easy to use
  • Easy to configure
  • Not ideal for large teams /programs

Freemium

4


CollabNet VersionOne

  • Widely  used
  • Enterprise-grade
  • Rich features
  • Learning curve
  • Less intuitive

Pai

5


Rally

  • Enterprise-grade
  • Easy to set up
  • Less intuitive and complex to learn

Paid

6


OpsGenie

  • Rich features for issue tracking and on-call management
  • Features are limited to issue tracking

Freemium

7


Pivotal Tracker

  • Rich feature set for tracking
  • Intuitive and easy to use  
  • Integrability with other tools

Freemium

8


Azure Board

  • Integrates well with Microsoft toolchain
  • Lacks richness in feature set  in comparison with other enterprise-grade tools in the same segment

Paid

9


Tasktop

  • Good for Value Stream Management
  • Integrability with other tools

Paid

10


Kanboard

  • Simple to use
  • Limited feature set
  • Not ideal for large teams/programs

Free & open source

c. Cloud /iaas/paas/serverless tools
Cloud /iaas/paas/serverless tools

Cloud along with Infrastructure as service and platform as service produces a platform for developing, testing and deployment of applications. Using such features DevOps reduces the much latency overload in acquiring and accessing assets. All private and public clouds provide support to DevOps tooling and thus reducing the cost spent for on-premises systems.

Some of the platforms are

1. AWS

Amazon Web Services (AWS) is a cloud services platform, offering to compute power, database storage, content delivery, and other cloud-related functionalities.

Availability: AWS is an on-demand cloud computing platform where we are charged on as you go basis.

Learn more about AWS here.

2. AWS Lambda

Lambda is a serverless computing platform from Amazon Web Services (AWS). It is a service that manages the computing resources and runs code in response to events.

Availability: We are charged only on the computing time.

Learn more about AWS Lambda here.

3. Azure

Microsoft Azure is an enterprise-grade cloud computing service that helps in managing applications through Microsoft-managed data centers. It provides software as a service (SaaS), platform as a service (PaaS) and infrastructure as a service (IaaS).

Click here to learn more about Azure.

4. Google Cloud Platform

Google Cloud Platform, offered by Google, is a suite of cloud computing services. Platform as service, Infrastructure as a service, and serverless computing are provided by GCP.

Click here to learn more about Google Cloud Platform.

5. IBM cloud

IBM Cloud is a suite of cloud computing services from IBM. It also provides infrastructure as a service (IaaS) and platform as a service (PaaS).

Availability: Lite version of IBM Cloud is free and allows one instance per plan.

Click here to learn more about IBM Cloud.

6. OpenStack

OpenStack is a free and open-source software platform for cloud computing, mostly deployed as infrastructure-as-a-service, whereby all virtual servers and other resources are made available to customers. It’s written in python.

Learn more about OpenStack here.

7. Cloud Foundry

Cloud Foundry is an open source cloud platform that helps to develop cloud applications. It’s from Pivotal.

Learn more about Cloud Foundry here.

8. Heroku

Heroku is a  platform as a service cloud environment. Thus help developers to work entirely on cloud.

Availability: Free version is available with limited features.

Learn more about Heroku here.

9. OpenWhisk

Apache OpenWhisk is an open source, distributed Serverless platform. OpenWhisk manages the infrastructure, servers and scaling using Docker containers.

Click here for more details about OpenWhisk.

SL NoTool NameProsConsAvailability
1.

AWS

  • Enterprise-ready provider
  • Complex cost structure

Pay as you go

2.

AWS Lambda

  • Serverless computing
  • Reduced operational costs
  • Limit on concurrent executions after which causing Denial Of Service

Charged for computing time

3.

Azure

  • Integrates better with many Microsoft tools
  • Services provided still needs to be improved

Pay for resources used

4.


Google Cloud Platform

  • Scalable
  • Better load balancing
  • Serverless computing
  • Currently, GCP has fewer services and features compared to AWS or Azure

Pay as you go

5.

IBM Cloud

  • Easy setup
  • Consistent performance
  • Difficulty in scaling

Free Lite version

6.


OpenStack

  • Massive scalability
  • Easy implementation
  • Complex configurations

Freemium

7.


Cloud Foundry

  • Supports on-premises and multi-cloud deployment
  • Great privacy and security
  • Less feature set compared to AWS or Azure

Paid

8.


Heroku

  • Advanced Continuous Integration Platform
  • Highly scalable
  • Less reconfigurability

Freemium

9.


OpenWhisk

  • Open event provider system
  • Serverless computing
  • Not efficient for long running applications

Paid as per computing time

d. Top Source control management
source control management

Source control management as practice stores and tracks the application and infrastructure code. Even delivery pipelines for an application is nowadays stored in source code repositories. Some of the tools are GitHub, Bit Bucket, Subversion, Mercurial, Rational ClearCase.

1. GitHub

GitHub is a popular repository hosting service using Git. Git is a  free and open source system. It’s of the ease with which it performs branching and merge operations. It’s a distributed version control system that adds to its preferences.

Click here for more details.  

2. Mercurial

Mercurial is a free distributed version control system. It is very easy to learn compared to Git but the branching feature of Git is more widely loved. Big and small projects could be handled in Mercurial.

Click here for more details.

3. Bitbucket

Bitbucket is a repository hosting service from Atlassian. It could be used to store source code using  Mercurial or Git revision control systems. It’s free for teams with a maximum of 5 users. The paid version is available for bigger teams.

Learn more about Bit Bucket here.

4. Rational ClearCase

Rational ClearCase is a source control management tool from IBM. It helps in the parallel development of software. Software artefacts whether it be source code or design documents etc could be managed by ClearCase. Enterprise version is available for ClearCase.

Learn more about ClearCase here.

5. Subversion

Subversion is a version control system from Apache. It’s a free tool and open source.

It helps to track down all the changes done to files and directories.

Click here to learn more about Subversion.  

6. JFrog Artifactory

Artifactory is an artefact repository management tool from JFrog. It’s a paid tool. It primarily stores binary files which are typically the product of our build process.

Click here to learn more about Artifactory.

SL NoTool NameProsConsAvailability

1.

GitHub

  • Easy to navigate user interface
  • Difficult to learn

Free

2.

Mercurial

  • Cannot rewrite commit history
  • Slower network operations

Free

3.

Bitbucket


  • Supports Git and Mercurial
  • Difficult integration with other tools

Freemium


4

Rational ClearCase


  • Integrates with Microsoft Visual Studio
  • Not suitable for projects with a big code base
  • Difficult to work with

Paid

5.

Subversion

  • Easy to learn even for non-technical users
  • Slower because of centralised version control system

Free

6.

Artifactory

  • Supports many languages and tools
  • Easy to use
  • Expensive

Paid

e. Top Package managers

package managers tools

Package managers build or package code with all metadatas like software’s name, purpose, version and all dependencies needed by the software to function correctly. It lessens the burden of manual installs especially in big enterprises where we need to install big software. Some of the tools available are

1. Maven

Maven is an open-source build automation tool from Apache used mainly for Java applications. Main features are it provides easy and uniform builds. It also keeps aside a parallel space for test code.

Learn more details about Maven here.  

2. Gradle

Gradle is also an open source build tool from Apache. It is built on Groovy domain-specific language. It is more like a combination of Ant and Maven.

Learn more about Gradle here

3. MSBuild

Microsoft Build automation tool is a free and open source mainly for  C++ and .NET applications. Visual Studio makes use of MSBuild to build its applications.

Learn more about MSBuild here.  

SL NoTool NameProsConsAvailability
1.Maven
  • All dependencies are downloaded automatically
  • Better suited for java projects
  • Complex to work with
  • Large learning curve

Free


2.


Gradle
  • Can write build script ourselves
  • Poor integration with eclipse

Free

3.


MSBuild

  • Great community support
  • Mainly for .NET applications only

Free

f. Continuous Integration

continuous integration tools

In continuous integration, a code is checked into the source code repository whenever a developer finishes a requirement or user story. Continuous Integration tools enable teams to build software application automatically in a decided time. Thus reducing the time elapsed in a manual build. Some of the popular tools available are   

1. GitLab CI

GitLab CI is an integrated part of GitLab, GitLab offers a continuous integration service.

Availability: A free version is available with limited features.

Learn more about GitLab CI here.  

2. Semaphore

Semaphore is the fastest hosted continuous integration and delivery solution as claimed by its developers.

Availability: Open source projects can use Semaphore for free in its full capacity, free use for private projects is limited to 100 builds per month.

Learn more about Semaphore here.

3. Circle CI

Circle CI's continuous integration and delivery platform make it easy for teams of all sizes to rapidly build and release quality software at scale. It is built for Linux servers and automates build, test and deployment processes.

Availability: Circle CI has a free version available for a single container.

Click here for more details about Circle CI.

4. Jenkins

Jenkins is an open-source continuous integration tool written in Java. Jenkins is a fork by the core developers of Hudson after a dispute with Oracle. Jenkins is the most widely used CI tool. 

Availability: Both free and enterprise versions are available.

Click here for more details about Jenkins.

5. Hudson

Hudson is a continuous integration tool written in Java that runs in a servlet container such as Apache Tomcat or GlassFish.

Click here for more details about Hudson.   

6. CruiseControl

CruiseControl is an open source continuous integration tool and extensible framework for facilitating a continuous build process. Distributed under a BSD-style license.

Learn more about CruiseControl here.  

7. Bamboo

Bamboo is a continuous integration (CI) server produced by Atlassian. Bamboo ties automated builds, tests, and releases together in a single workflow.

Availability: Licensed version is available at a starting price of $10.

Learn more about Bamboo here.

8. Team Foundation Build

Team Foundation Build (TFB) is part of the Team Foundation system and provides the functionality of a public build lab. With TFB, build managers can synchronize sources and compile.

Click here to know more about Team Foundation Build

9. Gump

Apache Gump is an open-source continuous integration tool, designed with the overarching aim of ensuring that projects are compatible at both the API level and regarding.

Learn more about Gump here.

10. Travis CI

Travis CI is an open-source distributed continuous integration (CI) service used to build and test projects hosted on GitHub. Open source projects can freely avail Travis CI.

Availability: Travis CI is free for first 100 builds but after which it is priced.

Learn more about Travis CI here.

11. TeamCity

TeamCity is an open-source CI platform from Jet Brains. It’s known for easy user interface and support for Microsoft stack.

Availability: Free version of TeamCity is available with the limited feature set.

Click here to know more about TeamCity.  

12. Puppet Pipelines

Puppet Pipelines makes software delivery easy and unites silos of automation across Dev and Ops teams. It automates your application builds and deployments.

The community edition of Puppet Pipelines is available free of cost for up to three users.

Click here for more details about Puppet Pipelines.  

SL NoTool NameProsConsAvailability
1.GitLab CI
  • Easy to configure
  • Source control and continuous integration in one place
  • Need GitLab integration

Freemium

2.


Semaphore CI


  • Simple and to the point
  • Less user base and community support

Freemium


3.CircleCI
  • Easy to use
  • Less known in the community

4.


Jenkins
  • Uses plugin model to integrate with several DevOps tools
  • Greater community support
  • Cumbersome groovy syntaxes

Freemium

5.Hudson
  • Jenkins forked from Hudson so has all basic features of Jenkins
  • No much development of new features taking place
  • Less community support

Open source


6.


CruiseControl

  • Goes well with .NET applications
  • Difficult setup

Open source

7.


Bamboo

  • A lot of tasks available as a built-in option and not as plugins
  • Goes well with Atlassian products like Bitbucket and JIRA
  • Only paid option available

Paid

8.Team Foundation Build
  • Works smoothly with .NET applications
  • Intuitive easy to install
  • Interoperability with other stacks is a challenge

Paid


9.Gump
  • Integrates well with Apache tools like Maven
  • Less plugin support

Open source


10.


Travis CI


  • Easy to set up and configure
  • Supports most technological stacks using Node, Ruby, etc
  • Doesn’t  support Bitbucket

Freemium


11.


TeamCity


  • Great user interface
  • Easy to learn
  • Community support is good but not great

Freemium

12.

Puppet Pipelines

  • Easy setup and installation
  • Plugin availability

Freemium

g. Top Continuous Delivery and Deployment tools

continuous delivery and deployment tools

Continuous deployment tools automate the delivery pipeline of application development, thus reducing the wastage of time caused by transfer between different teams like development and release teams. Few of the most popular deployment tools used by DevOps teams are

1. Chef

Chef is a tool used to manage and develop infrastructure. It could be used for application deployment also. It is an open-source tool but with an enterprise version available. Chef uses a  domain-specific language based on ruby to define and configure infrastructure. Chef allows high flexibility and typically preferred by developers. It has a higher learning curve compared to other tools in this space. Chef is known to be the most preferred tool for large scale, complex enterprise systems.

Availability: Chef is free for up to a limited number of nodes which is five nodes now after which it’s priced.

Learn more about Chef here.

2. Puppet

Puppet is another configuration management tool to define infrastructure as code. Puppet is an enterprise-grade tool. Puppet uses a more declarative language and hence makes it easier to work with. It’s preferred by operations teams as it doesn’t require programming skills.

Learn more about Puppet here.

3. Octopus Deploy

Octopus Deploy is a release management server from XebiaLabs. It’s used mainly for .NET applications and windows services. It’s a paid deployment as a service.

Click here for more about Octopus Deploy.

4. Spinnaker

Spinnaker is an open source free release platform that increases the number of good-quality releases. This platform helps in deployment across multi-cloud providers like AWS EC2, Google Kubernetes Engine etc.

Learn more about Spinnaker here.  

5. GoCD

GoCD is a free and open source server that helps in continuous delivery. It helps in creating a continuous delivery pipeline in cloud environments like Docker, AWS etc.

Learn more about GoCD here.  

6. UrbanCode Deploy

UrbanCode Deploy or uDeploy is a tool used to automate application deployment from IBM. It’s a licensed version and available as hosted services also.

Click here to know more about UrbanCode Deploy.  

7. XebiaLabs XL Deploy

XL Deploy is a release automation tool for any environment. It is a licensed version by XebiaLabs.

Click here to know more about XL Deploy.

8. Ansible

Ansible is an open source configuration management tool and application deployment tool. In comparison with Chef, Ansible works with a decentralised agentless architecture and hence it’s easy to get started with Ansible.

Availability: CLI based Ansible is free for no limit on nodes.

Learn more about Ansible here.  

9. SaltStack

SaltStack is an open-source configuration management software written in Python. It enables teams to craft  "Infrastructure as Code".SaltStack in comparison with Ansible is quickly scalable but enforces teams to learn python.

Learn more about SaltStack here.  

SL NoTool NameProsConsAvailability
1.Chef
  • Great documentation available
  • Hard to learn

  • Need programming skills

Freemium
2.Puppet
  • Programming skills are not a must
  • Not much suitable for applications where updates are frequent
Paid
3.Octopus Deploy
  • Easy configuration
  • Integrates smoothly with TeamCity
  • A quick and flexible deployment pipeline
  • Less community support especially for non-Microsoft applications
Paid
4.Spinnaker
  • Greatly preferred for cloud-based deployments
  • Less community support
Open source
5.

GoCD


  • Better suitable for end-to-end Continuous delivery pipeline where great visualisation needed.
  • Less cost efficient
  • A Steep learning curve with a confusing user interface
Open source
6.UrbanCode Deploy
  • Simple and easy to use
  • Slower deployments
Paid
7.XL Deploy
  • A large number of plugins available
  • Lesser visibility for the deployment process
Paid
8.Ansible
  • Simpler installation
  • Easy to use
  • GUI is not that great
  • No support for windows

Open source

9.SaltStack
  • Quickly scalable
  • Underdeveloped GUI

Open source

h. Testing automation

Testing automation tools are used in close proximity to continuous integration and deployment tools. It helps in performing repetitive tasks unable to perform by manual tests. The automated test gives a more clear picture of the health of the software product without any bias.

1. Unit testing

Unit testing tools help to test a single unit or component of the software. Thus detecting the errors earlier and fixing. And, Unit testing also helps in smooth integration.

2. Integration testing

Integration testing tools help validate every integration that happens in the integration phase. Only successful build of the code move to the next stage.

3. End-to-end testing

In end-to-end testing, the entire system or application is checked from start to finish. The tools generate the reports which can be used to verify whether the new change is causing  any unexpected behavior from the entire system

4. Performance testing

Performance testing tools analyse the system in an expected workload. The tools measure the responsiveness of the system, scalability, and stability. Tools also provide details on where the system is failing and where the system needs improvement.

5. Infrastructure testing and auditing

Infrastructure testing plays a very important part as an error in the infrastructure code can even alter the production environment creating unseen repercussions. Ensuring the compliance of an organisation is an integral part of such tools keeping security in mind.

i. Other Popular DevOps tools

Some of the popular tools used are

1. Selenium

Selenium is a free and open source testing framework for web applications. It’s a suite of four tools Selenium WebDriver, Selenium RC or Remote   Selenium IDE And Selenium-Grid.

Learn more about Selenium here.

2. Cucumber

Cucumber is an open source testing tool. It’s the best choice for behavior driven development popularly known as BDD as it tests business readable requirements. Free and enterprise version of Cucumber is available.

Click here for more details about Cucumber.

3. InSpec

InSpec is a free and open source testing framework from Chef. InSpec tests infrastructure. It’s also a compliance framework.

Click here to know more about InSpec.  

4. Karma

Karma is a free test runner created for testing, applications made with Angular CLI. It’s from the AngularJS team.

Learn more about Karma here.

5. Jasmine  

Jasmine is an open source testing framework. It is used mainly for JavaScript applications. It’s used for behavior driven development also.

Click here to learn more about Jasmine.

6. UFT

UFT or Unified Functional Testing is a test automation tool for web, desktop, mobile   Micro Focus. There is a 60-day free trial version available but after which it’s not a free tool.

Learn more about UFT here.  

7. SoapUI

SoapUI is an open-source testing tool for web applications. It’s the market leader in API testing. It’s a licensed tool.

Click here to know more about Soap UI.  

8. JMeter

JMeter is an open source load testing tool from Apache. It helps in analyzing the performance of services mainly for web applications. It’s a free test suite.

Learn more about JMeter here

SL NoTool NameProsConsAvailability
1.Selenium
  • Wide range of languages supported
  • Easy integration with Jenkins, Maven
  • Difficult to use
  • No support officially

Open source


2.Cucumber
  • Great documentation
  • Supports Behavior-driven development
  • Slow compared to other testing tools
Open source
3.

InSpec

  • Highly flexible and can be used cross any Infra As code framework.
  • Need to know the scripting language
Open source

4.


Karma

  • Easy debugging
  • Lesser user base
Open source
5.Jasmine
  • Difficult to debug
  • Easy to set up and use
Open source
6.UFT
  • Easily integrated with continuous integration DevOps tools
  • Less compatibility with different operating systems Expensive
Paid
7.SoapUI
  • User-friendly
  • Plugin availability is less
Open source
8.JMeter
  • Easy installation Great user-friendly interface
  • Higher learning curve Doesn’t support javascript
Open source

9. Docker

Docker is a platform for working with containers, from Docker, Inc.Docker is an open source and available as free and enterprise version. Containers help to develop applications and package it with its dependencies and libraries, thus ensuring the application runs in any environment. Docker containers are like virtual machines but share the same OS resources like file system etc.It has less overhead unlike VM 's. The building block of a container is an image which is the executable package including libraries, dependencies, environment variables etc needed to run the application. Running instance of an image is a container.

Learn more about Docker here.

10. Kubernetes

Kubernetes is an open source production grade container orchestration tool. It helps in managing multiple containers in an application. Kubernetes is the market leader in this category. It is often compared with Docker Swarm which is the native clustering method for docker.  

Click here to know more about Kubernetes.  

11. OpenShift

OpenShift from Red Hat is a group of containerization software. OpenShift Container Platform is the major software in the group that provides a platform as a service built around Docker containers. These docker containers are managed for experimenting by an individual a free version is available for one project.

Learn more about OpenShift here.  

SL NoTool NameProsConsAvailability
1.Docker
  • Containers are lightweight compared to virtual machines
  • Security is a concern
Open source
2.Kubernetes
  • Highly scalable
  • Work better with CI/CD pipelines
  • Less user-friendly
Open source
3.OpenShift
  • Great community support
  • Only supports Red Hat Enterprise Linux
Freemium

j. Release orchestration

Release orchestration tools are used to achieve automation of the application release process. Some of the popular tools are Xebialabs XL release, Plutora Release, AWS Codepipeline, CACD Director, OpenMake, Spinnaker, HashiCorp Vault, SonarQube, BlackDuck, Signal Sciences, Checkmarx SAST.

k. Containerology

Containerology tools help to run an application on the virtual environment as a package with all dependencies. It avoids the situation “it doesn’t work in my system”.

Some of the tools are:

l. Monitoring Tools

Monitoring tools help to pinpoint and track issues and verify the health of the system. This enables fast recovery of the system with minimum or no human interventions.

Popular tools are:

1. Prometheus

Prometheus is an open source monitoring tool from SoundCloud.Its mainly used with systems using microservices as it has a multi-dimensional data collection feature. It uses a flexible query language PromQL.All Prometheus server is standalone and doesn’t depend on network storage helping us to understand the defect especially during outages. Collected data through the multi-dimensional collection feature may not be too detailed and having complete information. So it is not suitable for systems where 100% accuracy is required.

Click here for more details about Prometheus.

2. Splunk

Splunk as a monitoring tool is used across application management, security, compliance, web analytics etc. Splunk tools listen and store data, index the same and correlate the captured real-time data in a searchable repository from which it can generate useful graphs, reports, alerts, and various other visualizations. One can create and configure relevant dashboards based on various visualizations/ graphs.  

Learn more about Splunk here.

3. Nagios

Nagios is an open source and free tool to monitor services, applications and infrastructure. It’s known for its auto-discovery feature. Its user interface is a bit difficult for beginners

Learn more about Nagios here.  

4. Zabbix

Similar to Nagios, Zabbix is an enterprise open source monitoring solution. Compared to Nagios, Zabbix is user-friendly and is comparatively easy to configure. The main disadvantage is that it doesn’t support plugins.

Learn more about Zabbix here.

5. Zenoss

Zenoss is a free, open-source tool used for services and network monitoring. It is written in Python language.

Click here to know more about Zenoss.

6. ELK Stack

"ELK" is the acronym for three free open source projects: Elasticsearch, Logstash, and Kibana. Elasticsearch is a search and analytics engine based on Java. Logstash is a server‑side data processing platform with the ability to clean, transform data and send it to Elasticsearch.

Kibana is a virtualization tool that helps to visualize data with charts and graphs in Elasticsearch.

The Elastic Stack is the next evolution of the ELK Stack.

Learn more about ELK Stack here.

SL NoTool NameProsConsAvailability
1.Prometheus
  • Easy to use

  • Easy integration with other DevOps tools

  • Bad user interface
Open source
2.Nagios
  • Number of plugins available in the market
  • Difficult configurations needed to make the system stable
Open source
3.Zabbix
  • Easy configuration based on a web-based user interface
  • Non-availability of plugins
Open source
4.Zenoss
  • Great community support
  • Limitation on the number of devices monitored
Open source
5.ELK Stack
  • Easy to install
  • Highly customisable
  • Difficult to configure
Open source
6Splunk
  • Easy to install
  • User-friendly
  • Easy to configure  simple graphs
  • For complex configurations, the learning curve is a bit steep

Paid


m. Analytics

Analytics tools give a clear picture of what is happening in the team, be it code development or team interaction, code coverage and efficiency etc. Some tools used are XebiaLabs XL Impact,New Relic,Dynatrace,Datadog,AppDynamics, ElasticSearch .

Bonus Information

3. Why is DevOps needed?

DevOps helps to remove silos in organisations and enable the creation of cross-functional teams, thus reducing reliance on any one person or team during the delivery process. Frequent communication between teams improves the confidence and efficiency of the team members. Through automation, DevOps team increase their productivity making satisfied customers. According to State of DevOps report 2016 “Teams that practice DevOps deploy 30x more frequently, have 60x fewer failures, and recover 160x faster“. It also provides better work environments with increased trust, better management of issues reducing unplanned works.

4. How to implement DevOps?

The “DevOps Handbook” defines the “Three Ways: The principles of underpinning DevOps” as a way to implement DevOps in large enterprises. In this session, we will detail these three ways and three core pillars.

The First Way: Systems Thinking

The First Way emphasizes the need for global optimisation as opposed to local optimisation, hence the focus is on optimising all business value streams enabled by IT.

The Second Way: Amplify feedback loops

The Second Way is about discovering and injecting right feedback loops so that necessary corrections can be made before it’s too late.

The Third Way: Culture of Experimentation and learning

The third way is all about creating the right culture that fosters two things, continual experimentation and learning from failures. It emphasises the understanding that repetition and practice make teams perfect.

While the three ways focus on the key principles, we also have three pillars which are keys to any successful DevOps adoption.

The three pillars of any DevOps adoption are,

  1. Culture and People
  2. Tools and Technology
  3. Processes and practices

4.1 Important DevOps practices

Important DevOps practices

a. Continuous Integration

Continuous integration is a software engineering practice where software development team members frequently merge and build their code changes. The key benefit is to detect and fix code merge conflicts and integration bugs in the early stages of software development. Hence reducing the cost to detect and fix the issues.

b. Continuous Delivery

Continuous delivery is a software engineering practice in which changes are automatically built, tested, and made release ready to production. In order to get into a continuous delivery state, it is very crucial to define a test strategy. The main goal is to identify functional and non-functional defects at a much earlier stage thus reducing the cost to fix defects. It also enables teams to come up with working software as defined in the agile manifesto. Continuous delivery as a practice depends on continuous integration and test automation. Hence it is crucial that teams need to ensure that they practice continuous integration along with test automation religiously, to effectively practice continuous delivery.

c. Continuous Deployment

Continuous deployment is a software engineering practice in which codes committed by the developers are automatically built, tested and deployed to production. Continuous deployment as a practice, require that teams have already adopted continuous integration and continuous delivery approach. The primary advantage of this practice is reducing time-to-market and early feedback from users.

d. Continuous Testing

Continuous Testing can be defined as a software testing practice that involves a process of testing early, testing often and test automation. The primary goal of Continuous Testing is to shift left the test phase as much as possible to identify defects and reduce the cost of fixing.

e. Microservices

Microservices architecture helps to create an application as a set of small services independent of each other. Any language could be used to create microservices and typically an HTTP based API is used to interact between services. Microservices as a design approach helps to achieve fewer risk deployments and enables continuous delivery.

f. Infrastructure as code

Infrastructure as a code is an engineering practice in which infrastructure is developed and managed through code. Thus creating a consistent, reproducible and versioned infrastructure. Since the infrastructure is implemented as the code it’s easy for the team members to update and change it. Infrastructure as a code no more considers scaling as a major problem.

g. Policy as code

Policy as a code is a software engineering practice where compliance rules or policies of the organisation could be monitored and verified. Policy as code enables organizations to enforce the compliance rules more strictly and helps to bring the non-compliant resource into compliance mode. This practice gained importance during the DevSecOps movement.

h. Continuous Monitoring and Logging

Monitoring and logging as a best practice to help organizations to analyse the products’ end user experience. This helps the software teams to get to know about the root cause of the defects and latencies in the software development process. More transparency into the actions performed by the team members causes increased responsibility among the teams causing increased performance.

i. Communication and collaboration

Effective communication and collaboration are one of the key values emphasised by DevOps.Devops tools in the field of communication and collaborations bring together collective responsibility for the products delivered.   

5. How to choose the right DevOps tools?

how to choose the right devops tools?

Today the DevOps market is overcrowded with tools across different stages of software development life cycle. As enterprises, it's extremely crucial to select the right tools in order to get maximum benefit. Saying so choosing the right tool is an extremely difficult and time-consuming process given the spectrum of tools available today. Hence enterprises should have a five-point strategy towards deciding the right tools. Five point strategy would include dimensions like

  1. Ability to integrate
  2. Scalability
  3. Security
  4. Technical know how
  5. Reliability

5.1. Ability to integrate

The ability to integrate is extremely crucial and is one of the fundamental requirements while checking out tools. Certain tools integrate smoothly with a particular technology stack in comparison with others. Hence it is vital for the DevOps architect to compare different tools on the basis of integration ability and ensure that tool that is been selected seamlessly integrates with the team’s technology stack. Another aspect that needs to be considered is how a particular tool integrates with other tools that are selected in the ecosystem. For eg., you would want your continuous integration system to constantly talk to the reporting system and alert prediction system in a smooth way. Hence integration between tools also becomes a very important factor while choosing tools.

5.2. Scalability

Scalability is the second most important factor in choosing the right tools. Based on the need for scalability an enterprise might choose an enterprise version over a community version. Scalability also is a key factor why certain companies go for SaaS-based products. SaaS-based products are easily scalable and hence without any overhead, it can be adopted across large enterprises.

5.3. Security

These days a lot of enterprises are emphasising on the need for security in the DevOps tooling space. Hence enterprise versions by various tooling companies have taken special care towards addressing these security-related issues. Thus enterprise versions are comparatively more preferred in comparison with that of open source solutions. Saying so this doesn't mean that all open source DevOps tools have security vulnerabilities. Certain open source DevOps tools fair much better than available enterprise versions

5.4. Technical know how

This people dimension is one of the factors that is typically overlooked by enterprises. Knowing the skill levels and capability of team members is a key towards choosing the right tool. Often the tools available in the market wouldn't work out of the box and would need a substantial level of customisation to smoothly integrate with existing systems and workflows. Also, certain tools require a certain specific skill set towards configuration and customisation. Typical eg. is Chef, which is chosen by developers who are comfortable in ruby language whereas Puppet is preferred by system admins as it does not require much of programming skills.

5.5. Reliability

Last but not the least, reliability is extremely crucial for any successful tool adoption. Most of the tools available in the market, both enterprise and open source needs to be checked using this quality wheel. Tools should be reliable even during large scale and complex operational conditions.

 Conclusion

In this paper, we discussed the what, why, and how of DevOps.We also deep dived into various tool categories and tools available across the spectrum in today's DevOps market. Tools are definitely the key ingredients in successful DevOps adoption but saying so a lot of companies only invest in tool part without focusing on cultural and people dimensions. In order for tools to bear fruit its vital that the people operating and analysing the tools/data understand and realise the true spirit of DevOps. To conclude, would like to resonate with the wise words “yes, we need all the tools that can help us, but just tools will not help us get there!”.

Anju

Anju M Dominic

Blog Author

Anju M Dominic is the Principal consultant and co-founder of QuadraLogics, a boutique consulting firm. She is also a professional corporate trainer and passionate conference speaker who specialises in DevOps practices and tools. In her tenure in the industry, she specialised in designing and developing applications in  Pharma and Telecom domains.

Join the Discussion

Your email address will not be published. Required fields are marked *

6 comments

Ranjith Tharayil 10 May 2019

good read

Dominic Paul 10 May 2019

Good blog

Christy 16 May 2019

Good one!!

Swathic 16 May 2019

Really a great read. Lot of valuable information.

Jose 16 May 2019

Really well written. Helped to get a good understanding of various DevOps tools

Brayden 21 Jun 2019

Some good tips. I think the important points are on covered in the topic

Suggested Blogs

How to Install Docker on Ubuntu

Docker is a platform that packages the application and all its dependencies in the container so that the application works seamlessly. The Container makes the application run its resource in an isolated process similar to the virtual machines, but it is more portable. For a detailed introduction to the different components of a Docker container, you can check out Introduction to Docker, Docker Containers & Docker Hub This tutorial covers the installation and use of Docker Community Edition (CE) on an Ubuntu 20.04 machine. Pre-requisitesAudienceThis tutorial is meant for those who are interested in learning Docker as a container service System Requirements Ubuntu 20.04 64-bit operating system. (If Linux OS is not in system, we can run docker using Virtual Box, PFB the steps) A user account with sudo privileges An account on Docker Hub to pull or push an image from Hub. Ubuntu Installation on Oracle Virtual Box If you want to use Ubuntu 20.04 without making any change to the Windows Operating system, you can proceed with the Oracle Virtual box.  Virtual Box is free and open-source virtualization software from Oracle. It enables you to install other operating systems in virtual machines. It is recommended that the system should have at least 4GB of RAM to get decent performances from the virtual operating system. Below are the steps for downloading Ubuntu 20.04 on Oracle Virtual box:Navigate to the website of Oracle Virtual Box, download the .exe file and get the latest stable version. 1. Once done with downloading the virtual box, we can navigate to and download the  Ubuntu disk image (.iso file) by clicking on the download option 2. Once the download has been completed for Ubuntu .iso file, open the virtual box and click on "New" present on top.  3. Enter the details of your virtual machine by giving any name, type as "Linux " and Version as Ubuntu (64 bit)  4. Choose the memory (RAM ) that needs to be allocated to the Virtual machine  and click on Next. (I have chosen 3000 MB) 5. After the RAM allocation ,Click on  Create a virtual disk now. This serves as the hard disk of the virtual Linux system. It is where the virtual system will store its files 6. Now, we want to select the Virtual Hard Disk.  7. We can choose either the “Dynamically allocated” or the “Fixed size” option for creating the virtual hard disk. 8. Finally, we have  to specify our Ubuntu OS's size. The recommended size is 10 GB, but it  can be increased if required.8. Finally, we have  to specify our Ubuntu OS's size. The recommended size is 10 GB, but it  can be increased if required.9. Ubuntu OS is ready to install in Virtual Box, but before starting the Virtual system, we need to a make few changes in settings. Click on storage under the setting.  10. Click on Empty under Controller IDE. Navigate to Attributes and browse the Optical Drive option. 11. Choose the .iso file from the location where it is downloaded. Once selected, click on OK and start the Virtual box by clicking on start present on the Top menu.12. Click ok and start the machine. 13. Proceed with "Install Ubuntu" 14. Under "Updates and other software" section, check "Normal installation", and the two options under “Other options” and continue.15. In Installation type, check Erase disk and install Ubuntu.16. Choose your current location and set up your profile. Click Continue.  17. It may take 10-15 minutes to complete the installation 18. Once the installation finishes, restart the virtual systemWe are done with pre-request, and can now proceed with using this Ubuntu. Docker Installation Process on Ubuntu  Method 1: Install Docker on Ubuntu Using Default Repositories One of the easiest ways is the installation of Docker from the standard Ubuntu 20.04 repositories, but It’s possible that the Ubuntu default repositories have not updated to the latest revision of Docker. It happens because in some cases Docker is not supporting that particular Ubuntu version. Therefore, there can be a scenario where  Ubuntu default repositories have not updated to the latest version. Log in to Virtual Box. Run “docker” as command to check if it is previously installed.To install Docker on Ubuntu box, first update the packages. It will ask for a password. Enter it and allow the system to complete the updates.sudo apt updateTo install Docker from Ubuntu default repositories, use the below command: sudo apt install docker.io To check the installed version, use the below: docker --version Since discussed above, it has installed the 19.03.8 version of docker whereas the latest version is 20.04  Method 2: Install Docker from Official Repository For installing docker on ubuntu 20.04 with the latest version, we’ll proceed with enabling the Docker repository, importing the repository GPG key, and finally installing the package. To install the docker on Ubuntu box, update your existing list of packages. It will ask for a password. Enter it and allow the system to complete the updates. sudo apt update  We need to install a few prerequisite packages to add HTTPS repository : sudo apt install apt-transport-https ca-certificates curl software-properties-common Import the repository’s GPG key using the following curl command: curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - Add the Docker APT repository to the system sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"Again, update the package database with the Docker packages sudo apt update Finally, install Docker using below command: sudo apt install docker-ce To check the installed version use below: docker --versionTo start, enable and check the status of docker, use below command: sudo systemctl  status docker  sudo systemctl  start  docker  sudo systemctl  enable  docker To check system wide information regarding docker installation, we use the command “docker info”. Information that is shown includes the kernel version, number of containers and unique images. The output will contain details as given below, depending upon the daemon running: Source:$ docker info  Client:   Context:    default   Debug Mode: true  Server:   Containers: 14    Running: 3    Paused: 1    Stopped: 10   Images: 52   Server Version: 1.13.0   Storage Driver: overlay2    Backing Filesystem: extfs    Supports d_type: true    Native Overlay Diff: false   Logging Driver: json-file   Cgroup Driver: cgroupfs   Plugins:    Volume: local    Network: bridge host macvlan null overlay   Swarm: active    NodeID: rdjq45w1op418waxlairloqbm    Is Manager: true    ClusterID: te8kdyw33n36fqiz74bfjeixd    Managers: 1    Nodes: 2    Orchestration:     Task History Retention Limit: 5    Raft:     Snapshot Interval: 10000     Number of Old Snapshots to Retain: 0     Heartbeat Tick: 1     Election Tick: 3    Dispatcher:     Heartbeat Period: 5 seconds    CA Configuration:     Expiry Duration: 3 months    Root Rotation In Progress: false    Node Address: 172.16.66.128 172.16.66.129    Manager Addresses:     172.16.66.128:2477   Runtimes: runc   Default Runtime: runc   Init Binary: docker-init   containerd version: 8517738ba4b82aff5662c97ca4627e7e4d03b531   runc version: ac031b5bf1cc92239461125f4c1ffb760522bbf2   init version: N/A (expected: v0.13.0)   Security Options:    apparmor    seccomp     Profile: default   Kernel Version: 4.4.0-31-generic   Operating System: Ubuntu 16.04.1 LTS   OSType: linux   Architecture: x86_64   CPUs: 2   Total Memory: 1.937 GiB   Name: ubuntu   ID: H52R:7ZR6:EIIA:76JG:ORIY:BVKF:GSFU:HNPG:B5MK:APSC:SZ3Q:N326   Docker Root Dir: /var/lib/docker   Debug Mode: true    File Descriptors: 30    Goroutines: 123    System Time: 2016-11-12T17:24:37.955404361-08:00    EventsListeners: 0   Http Proxy: http://test:test@proxy.example.com:8080   Https Proxy: https://test:test@proxy.example.com:8080   No Proxy: localhost,127.0.0.1,docker-registry.somecorporation.com   Registry: https://index.docker.io/v1/   WARNING: No swap limit support   Labels:    storage=ssd    staging=true   Experimental: false   Insecure Registries:    127.0.0.0/8   Registry Mirrors:     http://192.168.1.2/     http://registry-mirror.example.com:5000/   Live Restore Enabled: false Note: In case you get below error after running “docker info” command, one way is to add sudo in front and run the command, OR you can refer to the same error-resolving steps mentioned under Running Docker Images section. Running Docker Images and Verifying the process: To check whether you can access and download the images from Docker Hub, run the following command: sudo docker run hello-worldIn case of errors received after running the docker run command, you can correct it using the following steps, otherwise proceed with the next step of checking the image. ERROR: docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.35/containers/create: dial unix /var/run/docker.sock: connect: permission denied. See 'docker run --help'.   Create the docker group if it does not exist sudo groupadd docker Add your user to the docker group.   sudo usermod -aG docker $USER   Eg:- sudo usermod -aG docker kanav Run the following command or Logout and login again and run ( if that doesn't work you may need to reboot your machine first)  newgrp docker Check if docker can be run without root docker run hello-world If the problem still continues, try to reboot it and run the command. To check the image, use this command: sudo docker images Uninstall Procedure: Below are the common commands used to remove images and containers: sudo  apt-get  purge docker-ce docker-ce-cli containerd.io To completely uninstall Docker, use below: To identify what are the installed packages, this is the command: dpkg -l | grep -i dockersudo apt-get purge -y docker-engine docker docker.io docker-ce docker-ce-cli  sudo apt-get autoremove -y --purge docker-engine docker docker.io docker-ce   To remove images, containers, volumes, or user created configuration files, these commands can be used: sudo rm -rf /var/lib/docker /etc/docker sudo rm /etc/apparmor.d/docker sudo groupdel docker sudo rm -rf /var/run/docker.sock  Conclusion: If you found this Install Docker on Ubuntu blog relevant and useful, do check out the Docker-Training workshop from KnowledgeHut, where you can get equipped with all the basic and advanced concepts of Docker! 
5568
How to Install Docker on Ubuntu

Docker is a platform that packages the application... Read More

How to Install Kubernetes on Windows

Kubernetes is a container-based platform for managing cloud resources and developing scalable apps. It is widely regarded as the most common platform for automating, deploying, and scaling the entire cloud infrastructure. The platform runs on all major operating systems and is the most widely used open-source cloud tool.  Kubernetes can scale your entire infrastructure, monitor each service's health, act as a load balancer, and automate deployments, among other things. You can deploy your pods (docker containers) and services across the cloud by installing and configuring as many nodes (clusters) as you want.Let’s get started. We will guide you through the complete roadmap on how to install Kubernetes for Windows users. This tutorial will show you how to set up Kubernetes and deploy the official web GUI dashboard, which will allow you to manage and monitor everything. PrerequisitesFor installing Kubernetes in your system, here are a few prerequisites that need special attention. The hardware and software requirements are discussed below:Hardware requirementsMaster node with at least 2 GB memory. (Additional will be great)Worker node with 700 MB memory capacity.Your Mouse/Keyboard (monitor navigation)Software requirementsHype-VDocker DesktopUnique MAC addressUnique product UUID for every nodeEnsuring that there is a full range of connectivity between all the machines in the cluster is a must.Installation ProcedureStep 1: Install & Setup Hyper-VAs we all know, Windows has its virtualization software, known as Hyper-V, which is essentially VirtualBox on steroids. Hyper-V allows you to manage your virtual machines (VMs) using either the free Microsoft GUI tool or the command line. It's simple to enable Hyper-V, but first, make sure your PC meets the following requirements:Your operating system should be Windows 10 (Enterprise, Pro, or Education), withAt least 4GB of RAM and CPU Virtualization support, though you should double-check that it's turned on in your BIOS settings.You can disable or enable features like Hyper-V that may not be pre-installed when Windows is installed. Always keep in mind that some of the features require internet access to download additional Windows Update components.To enable Hyper-V on your machine, follow the steps below:1. Open the Control Panel.2. Select Programs from the left panel.3. Next, go to Programs and Features, then Turn Windows Features On or Off.4. Examine Hyper-V and the Hypervisor Platform for Windows.5. Select OK.Your system will now begin installing Hyper-V in the background; it may be necessary to reboot a few times until everything is properly configured. Don't hold your breath for a notification or anything! Verify that Hyper-V is installed successfully on your machine by running the following command as Administrator in PowerShell:Get-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-VOnce the state is shown as Enabled for above command in Power shell, we are good to go.Step 2: Download Docker for Windows and install it.Kubernetes is a container orchestration system built on top of Docker. It is essentially just a tool for communicating with Docker containers and managing everything at an enterprise level. Simply go to install Docker and click to Get Docker Desktop for Windows (stable).Windows users can use Docker Desktop.Docker Desktop for Windows is a version of Docker optimized for Windows 10. It's a native Windows application that makes developing, shipping, and running dockerized apps simple. Docker Desktop for Windows is the fastest and most reliable way to develop Docker apps on Windows, as it uses Windows-native Hyper-V virtualization and networking. Docker Desktop for Windows can run Docker containers on both Linux and Windows.Installation of Docker DesktopLet us take a look on the different steps involved in installing docker desktop.Double-click Docker for Windows Installer to run the installer.Docker starts automatically once the installation is complete. Docker is running and accessible from a terminal, as indicated by the whale in the notification area.Run Try out some Docker commands in a command-line terminal like PowerShell!  Run the Docker version to check the version.Run Docker run hello-world to verify that Docker can pull and run images.Boom!As long as the Docker Desktop for Windows app is running, Docker is accessible from any terminal. The Docker whale in the taskbar has a setting button that can be accessed from the UI.For a detailed step by step installation guide with screenshot, visit the blog - How to Install Docker on Windows, Mac, & Linux: A Step-By-Step GuideWARNING: FOLLOW THE INSTRUCTIONS BELOW! If Docker was successfully installed but you can't find its tray icon, you'll need to restart your computer. Check the official troubleshooting guide here if the issue persists. Step 3: Install Kubernetes on Windows 10Docker includes a graphical user interface (GUI) tool that allows you to change some settings or install and enable Kubernetes.To install Kubernetes, simply follow the on-screen instructions on the screen:1. Right-click the Docker tray icon and select Properties.2. Select "Settings" from the drop-down menu.3. Select "Kubernetes" from the left panel.4. Check Enable Kubernetes and click "Apply"Docker will install additional packages and dependencies during the installation process. It may take between 5 and 10 minutes to install, depending on your Internet speed and PC performance. Wait until the message 'Installation complete!' appears on the screen. The Docker app can be used after Kubernetes has been installed to ensure that everything is working properly. Both icons at the bottom left will turn green if both services (Docker and Kubernetes) are running successfully and without errors.Example.Step 4: Install Kubernetes DashboardThe official web-based UI for managing Kubernetes resources is Kubernetes Dashboard. It isn't set up by default. Kubernetes applications can be easily deployed using the cli tool kubectl, which allows you to interact with your cloud and manage your Pods, Nodes, and Clusters. You can easily create or update Kubernetes resources by passing the apply argument followed by your YAML configuration file.Use the following commands to deploy and enable the Kubernetes Dashboard.1. Get the yaml configuration file from here.2. Use this to deploy it. kubectl apply -f .\recommended.yaml3. Run the following command to see if it's up and running.:kubectl.exe get -f .\recommended.yaml.txtStep 5: Access the dashboardThe dashboard can be accessed with tokens in two ways: the first is by using the default token created during Kubernetes installation, and the second (more secure) method is by creating users, giving them permissions, and then receiving the generated token. We'll go with the first option for the sake of simplicity.1. Run the following command PowerShell (not cmd)((kubectl -n kube-system describe secret default | Select-String "token:") -split " +")[1]2. Copy the generated token3. Runkubectl proxy.4. Open the following link on your browser: http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/5. SelectToken & paste the generated token6. Sign InFinallyYou'll be able to see the dashboard and your cloud resources if everything is set up correctly. You can then do almost all of the "hard" work without having to deal with the CLI every time. You may occasionally get your hands dirty with the command line, but if you don't understand Docker and Kubernetes or don't have the time to manage your own cloud, it's better to stick with some PaaS providers that can be quite expensive.Kubernetes Uninstallation ProcessThe procedures for uninstalling cert-manager on Kubernetes are outlined below. Depending on which method you used to install cert-manager - static manifests or helm - you have two options.Warning: To uninstall cert-maneger, follow the same steps as you did to install it, but in reverse. Whether cert-manager was installed from static manifests or helm, deviating from the following process can result in issues and potentially broken states. To avoid this, make sure you follow the steps outlined below when uninstalling.Step 1: Before continuing, make sure that all user-created cert-manager resources have been deleted. You can check for any existing resources with the following command:$ kubectl get Issuers,ClusterIssuers,Certificates,CertificateRequests,Orders,Challenges --all-namespacesAfter you've deleted all of these resources, you can uninstall cert-manager by following the steps outlined in the installation guide.Step 2: Using regular manifests to uninstall.Uninstalling from a regular manifest installation is as simple as reversing the installation process and using the delete command.kubectl.2. Delete the installation manifests using a link to your currently running version vX.Y. Z like so:$ kubectl delete -f https://github.com/jetstack/cert-manager/releases/download/vX.Y.Z/cert-manager.yamlStep 3: Uninstalling with Helm.1. Uninstalling cert-manager from a Helm installation is as simple as reversing the installation process and using the delete command on both the server and the client. kubectl and helm.$ helm --namespace cert-manager delete cert-manager2. Next, delete the cert-manager namespace:$ kubectl delete namespace cert-manager3. Finally, delete the cert-manger CustomResourceDefinitions using the link to the version vX.Y.Z you installed:$ kubectl delete -f https://github.com/jetstack/cert-manager/releases/download/vX.Y.Z/cert-manager.crds.yamlThe namespace is in the process of being terminated.The namespace may become stuck in a terminating state if it is marked for deletion without first deleting the cert-manager installation. This is usually because the APIService resource is still present, but the webhook is no longer active and thus no longer reachable.4. To fix this, make sure you ran the above commands correctly, and if you're still having problems, run:$ kubectl delete apiservice v1beta1.ConclusionIn this tutorial, we have explained in detail how to install Kubernetes with Hyper-V. Also, we have tackled what requirements we need, both in terms of the software and hardware. We have explained how to install Hyper-V and Docker on Windows 10.   It is important to note that the fundamental difference between Kubernetes and Docker is that Kubernetes is meant to run across a cluster and Docker is meant to run through nodes.   Kubernetes is also more extensive than Docker Swarm and is meant to coordinate a cluster of nodes at scale in production in an efficient manner. Each software is crucial to having a smooth installation process.   We finally looked at how to install and uninstall Kubernetes.
1706
How to Install Kubernetes on Windows

Kubernetes is a container-based platform for manag... Read More

How To Install Jenkins on Ubuntu

Jenkins is a Java-built open-source Continuous Integration (CI) and CD platform. Basically, Jenkins builds software projects, checks and deploys them. This is one of the most practical programming tools you can master, and today we will show you how Jenkins is installed on Ubuntu 18.04. Use this powerful tool to activate your VPS server!Jenkins is loved by teams of all sizes, for different language projects like Java, Ruby, Dot Net, PHP etc. Jenkins is a platform that is autonomous, and can be used on Windows, Linux or any other operating system.  Prerequisites Hardware Requirements: RAM- 4 GB (Recommended) Storage- more than 50 GB of Hard Disk Space (Recommended)        Software Requirements: Java: Java Development Kit (JDK) or Java Runtime Environment (JRE).  Web Browser: Any browser such as Google Chrome, Mozilla Firefox, Microsoft Edge. Operating System: An Ubuntu 18.04 server installed with a non-root sudo user and firewall. For help in the planning of production capability of a Jenkins installation see Choosing the right hardware for Masters. Why Use Jenkins? You need to consider continuous integration (CI) and continuous delivery (CD) to understand Jenkins: Continuous integration – the practice of continuous production combined with the main industry.  Continuous delivery – the code is constantly delivered to an area after the code is ready for delivery. It could be for production or staging. The commodity is supplied to a consumer base that can provide QA or inspection by customers. Developers update the code regularly in the shared repository (such as GitHub or TFS). Improvements made in the source code are made at the end of the day, making it difficult to identify the errors. So, Jenkins is used here. Once a developer changes the repository, Jenkins will automatically enable the build and immediately warn you in the event of an error (Continuous Integration CI). Installation Procedure: Step 1: Install Java Skip to the next section if you have Java already installed on your system. To check, please run the following command in the terminal: java --version Jenkins needs Java for running, but it doesn't include certain distributions by default, and Java versions of Jenkins are incompatible. Multiple Java implementations are available to you. OpenJDK is currently the most popular one, which we will use in this guide. Being an open-source Java application, Jenkins requires the installation of OpenJDK 8 on your system. The apt repositories can directly access OpenJDK 8. The installation of OpenJDK from standard repositories is recommended. Open and enter the following in the terminal window: $ sudo apt update  $ sudo apt install openjdk-8-jdk The download and installation will be requested. Press the "Y" button and press the Enter button to finish the process. Java 8 will be installed on your system. We are ready to download Jenkins package now as we have our requirements ready! Step 2: Install Jenkins The default Ubuntu packages for Jenkins are always behind the current version of the project itself. You may use the project-maintained packages to install Jenkins to take advantage of the newest patches and features. 1. add the framework repository key: $ wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add  The device returns OK when the key is inserted. 2. Next, link the repository of Debian packages to the sources.list of the server: $ sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list' 3. When both are in place, upgrade to apt to use the new repository: $ sudo apt update 4. Install Jenkins: $ sudo apt install jenkins Now we're going to start the Jenkins server, as Jenkins and its dependencies are in place. Step 3: Start Jenkins 1. You can start Jenkins using systemctl: $ sudo systemctl start jenkins 2. As systemctl does not display performance, you can use the status command to check that Jenkins has successfully launched: $ sudo systemctl status jenkinsIf all went well, the start of the performance should demonstrate that the service is active and ready to boot: Output: jenkins.service - LSB: Start Jenkins at boot time     Loaded: loaded (/etc/init.d/jenkins; generated)     Active: active (exited) since Sat 2021-04-17 00:34:17 IST; 26s ago       Docs: man:systemd-sysv-generator(8)    Process: 17609 ExecStart=/etc/init.d/jenkins start (code=exited, status=0/SUCC As Jenkins is running, so adjust the firewall rules to complete our further setup of Jenkins from the web browser. Step 4: Opening the Firewall 1. Jenkins works by default on port 8080, so let's open the port with ufw: $ sudo ufw allow 8080  2. Check ufw’s status: $ sudo ufw status You will see that traffic from anywhere is permitted to port 8080. Output: Status: active  To                         Action      From  --                         ------      ----  8000                       ALLOW       Anywhere                    CUPS                       ALLOW       Anywhere                    27017                      ALLOW       Anywhere                    27017                      ALLOW       192.168.1.10                8080                       ALLOW       Anywhere                    8000 (v6)                  ALLOW       Anywhere (v6)               CUPS (v6)                  ALLOW       Anywhere (v6)               27017 (v6)                 ALLOW       Anywhere (v6)               8080 (v6)                  ALLOW       Anywhere (v6) 3. If the firewall is inactive, the following commands will allow OpenSSH and turn it back on: $ sudo ufw allow OpenSSH  $ sudo ufw enable We can finish the initial configuration with Jenkins installed and our firewall configured. Note: If you decide to continue to use Jenkins, use a Nginx Reverse Proxy at Ubuntu 18.04 to configure Jenkins with SSL when your exploration has been completed to protect your passwords and any sensitive system or product information sent between the machine and the server in plain text. Step 5: Setting Up Jenkins 1. To set up installation, visit Jenkins on its default 8080 port with your server domain name or IP address: http://your_server_ip_or_domain:8080 You should see the Unlock Jenkins screen, which displays the initial password's location:2. You can use the cat command to display the password: $ sudo cat /var/lib/jenkins/secrets/initialAdminPassword 3. Copy the alphanumeric terminal 32-character password and paste into the Administrator Password field, then click Continue. Output: 0aaaf00d9afe48e5b7f2a494d1881326 The following screen shows the ability to install or select certain plugins: 4. We will click on the option to install proposed plugins to start the installation process immediately. 5. When the installation is done, the first administrative user will be prompted. You can save this step and use your initial password to continue as an Admin. However, we will take some time to create the user. The Jenkins default server is NOT encrypted to prevent data from being protected. Use the Nginx Reverse Proxy on Ubuntu 18.04 to configure Jenkins with SSL. This protects the information of users and builds transmitted through the web interface. 6. You will see a configuration instance page, which asks you to confirm your Jenkins instance's URL of choice. Confirm either your server's domain name or the IP address of your server.  7. Click Save and Finish once you have confirmed the relevant information. A confirmation page will show you that "Jenkins is ready!"  Hit Start using Jenkins button and it will take you to the Jenkins dashboard.  Congratulations! You have completed the installation of Jenkins. Step 6: Creation of New Build Jobs in Jenkins: The freestyle job is a highly versatile and user-friendly choice. It's easy to set up and many of its options appear in many other build jobs. For all projects, you can use it. Follow the following steps: You have to login to your Jenkins Dashboard by visiting2) Create New item: Click on the New Item on the left-hand side of the dashboard.3) Fill the project description: You can enter the job details as per your need.4) Source Code Management: Under source code management, enter the repository URL.You can also use a Local repository. 5) Build Environment: Now in the Build section, Click on the “Add build Setup” Select "Execute Windows batch command".Now, add the java commands. In this article, we have used javac HelloWorld.java and java HelloWorld.   6) Save the project: Click Apply and save the project. 7) Build Source Code and check its status: Click on “Build Now” on the left-hand side of the screen to create the source code. 8) Console Output: Select the build number and click on “Console Output” to check the status of the build run. When it shows success, it means that we have successfully run the HelloWorld program from the cGitHub Repository. In case of failure, you can check the job logs by clicking on failure icon and debug the root cause.Uninstall Jenkins Follow the instructions to uninstall Jenkins: $ sudo apt-get remove jenkins Uninstall Jenkins: $ sudo apt-get remove --auto-remove jenkins Purging your data: $ sudo apt-get purge jenkins or you can use: $ sudo apt-get purge --auto-remove jenkins Conclusion: Installing Jenkins on Ubuntu is really that easy. Jenkins has a low learning curve and so you can start to work with it as quickly as possible. In the above article we have learned how to install Jenkins in an Ubuntu machine where all the steps are explained clearly. In case you want to learn more about the core concepts of Jenkins Jobs, Pipelines, Distributed System, Plugins, and how to use Jenkins in depth you can enroll for our course Jenkins Certification Course. 
5535
How To Install Jenkins on Ubuntu

Jenkins is a Java-built open-source Continuous In... Read More

Useful links