1.History of DevOps
First time pitched in Toronto Conference in 2008 by Patrick Debois and Andrew Shafer. He proposed that there is a better approach can be adopted to resolve the conflicts we have with dev and operation time so far. This again took a boom when 2 flickr employee delivered a seminar that how they are able to place 10+ deployment in a day. They came up with a proven model which can resolve the conflicts of Dev and operation having component build, test and deploy this should be an integrated development and operations process.
2.Definition of DevOps
DevOps is a practice culture having a union of people, process, and tools which enables faster delivery. This culture helps to automate the process and narrow down the gaps between development and IT. DevOps is a kind of abstract impression that focuses on key principles like Speed, Rapid Delivery, Scalability, Security, Collaboration & Monitoring etc.
A definition mentioned in Gartner
“DevOps represents a change in IT culture, focusing on rapid IT service delivery through the adoption of agile, lean practices in the context of a system-oriented approach. DevOps emphasizes people (and culture) and seeks to improve collaboration between operations and development teams. DevOps implementations utilize technology— especially automation tools that can leverage an increasingly programmable and dynamic infrastructure from a life cycle perspective.”
3.How DevOps Works with a toolchain?
4.Benefits of DevOps
Adding some benefits which we can see after adopting DevOps :
a) Horizontal and vertical growth: When I’m using “Horizontal and Vertical Growth” I’m keeping customer satisfaction on X, Business on Y2 and time on Y axis. Now the question is how it helps to populate growth in 2 axes, and my answer will be the quick turnaround time for minor and major issues. Once we adopt DevOps we scale and build in such fashion that in less time the graph shows a rapid jump.
b) Scalability & Quality: If a business starts reaching to more user we start looking to increase infrastructure and bandwidth. But in other hands, it starts popping up a question whether we are scaling our infra in right way and also if a lots people are pushing changes (Your code commits/builds) are having the same or great quality we have earlier. Both of the questions are somehow now reserved by the DevOps team. If your business pitch that we might be going to hit 2000+ client and they will be having billion of traffic and we are ready to handle, DevOps take this responsibility and says yes can scale infra at any point of time. And if the same time internal release team says I want to deliver 10 feature in next 10 days independently, DevOps says quality can be maintained.
c) Agility & Velocity: They key param of adopting DevOps is to improve the velocity of product development. DevOps enables Agility and if both are in sync we can easily observe the velocity. The expectation of end users are always high and at the same time, the deliverable time span is short. To achieve this we have to ensure that we are able to our rollout new features to customers at much higher frequencies otherwise your competitors may win the market
d) Improving ROI of Data: Having DevOps in an organization ensures that we can design a decent ROI from data at an early stage more quickly. If we will do a raw survey now Software industry is playing with data and have control over there a team should have an end to end control on data. And if we define DevOps it will help a team to crunch data in various ways by automating small jobs. Automation we can segregate and justify data and this helps to populate either in Dashboard or can present offline to a customer.
5.How DevOps enables CI/CD (Continuous Integration/Continuous Delivery)
When we say “continuous” it doesn’t translate that “always running” but of course “always ready to run”.
Continuous integration is nothing but the development philosophy and practices that drive teams to check-in code to version control system as often as possible. So keeping your build clean and QA ready developer's changes need to be validated by running automated tests against the build that could be a Junit, iTest. The goal of CI is to place a consistent and automated way to build and test applications which results in a better collaboration between teams, and eventually a better-quality product.
Continuous Delivery is an adjunct of CI which enables a facility to make sure that we can release new changes to your customers quickly in a sustainable way. Typical CD involves below steps:
Below there is a table which will help us better to understand what we need to put an effort and what we will gain if we enable CI/CD in place:
|Practice test||Effort Required||Gain|
|Continuous integration||a) Need to prepare automated your team will|
need to write automated tests for each new feature, improvement or bug fix
b)You need a continuous integration server that can monitor the main repository and run the tests automatically for every new commits pushed.
c)Developers need to merge their changes as often as possible, at least once a day.
|a) Will give control on regressions which can be captured in early stage of automated testing.|
b)Less context switching as developers are alerted as soon as they break the build and can work on fixing it before they move to another task.
c)Building the release is easy as all integration issue have beer solved early.
d)Testing costs are reduced drastically- your Cl server can run hundreds of tests in the matter 0 seconds.
e)Your QA team spend less time testing and can focus or significant improvement to the quality culture.
|Continuous Delivery||a)You need a strong foundation in continuous|
integration and your test suite needs to cover enough of your codebase.
b)Deployments need to be automated. The tigger is still manual but once a deployment is started there shouldn't be a need for human intervention.
c)Your team will most likely need to embrace feature flags so that incomplete feature do not affect customer in production.
|a)The complexity of deploying software has been taken away your team doesn't have to spend days preparing for a release anymore.|
b)You can release more often thus accelerating the feedback loop with your customer.
C)There is much less pressure on decisions for small changes hence encouraging iterating faster.
Sample Flow of CI /CD plan:-
6. Devops Tools
Jenkins: Jenkins is an open source automation server which is used to automate the software build, and deliver or deploy the build. It can be installed through native system packages, Docker, or even run standalone by any machine with a Java Runtime Environment (JRE) installed. In short, Jenkins enables continuous integration which helps to accelerate the development. There are ample of plugins available which enables integration for Various DevOps stages. For example Git, Maven 2 project, Amazon EC2, HTML publisher etc.
Pingdom: Pingdom is a platform which enables monitoring to check the availability, performance, transaction monitoring (website hyperlink) and incident collection of your websites, servers or web applications. Beauty is if you are using collaboration tool like slack or flock you can just integrate by using the webhook (Pretty much simple no code required
)you can easily get notified at any time. Pingdom also provides API so you can have your customized dashboard (Recently started) and the documentation is having enough details and self-explanatory.
Image copyright Pingdom - D-3
Nagios: It’s an Open Source Monitoring Tool to monitor the computer network. We can monitor server, applications, incident manager etc and you can certainly configure email, SMS, Slack notifications and phone calls even. Nagios is licensed under GNU GPLv2. Listing some major components which can be monitored with Nagios:
Image copyright stack driver- D-4
Stackdriver: Stackdriver is again a Monitoring tool to get the visibility of performance, uptime, and overall health for cloud-powered applications. Stackdriver monitoring collects events and metadata from Google Cloud Platform, Amazon Web Services (AWS). Stackdriver consumes data and generates insights via dashboards, charts, and alerts. And for alerting we can integrate to collaboration tools like Slack, PagerDuty, HipChat, Campfire, and more.
Image copyright stack driver- D-2
Adding one sample log where we can see what all parameter it collects and also i have just separated them in the fashion which will help us to understand what all it logs it actually collects:
User Details and Authorization Info
|Request Type and Caller IP|
Resource and Operation Details
TimeStamp and Status Details
Grafana: It is an open source visualization tool and can be used on top of different data stores like InfluxDB,Elasticsearch and Logz.io.We can create comprehensive charts with smart axis formats (such as lines and points) as a result of Grafana’s fast, client-side rendering — even over long ranges of time — that uses Flot as a default option. We can get the 3 different level of access, watcher, Editor and Admin, even we can enable G-Auth for having good access control. A detail information guide can be found here
Image copyright stack driver- D-5ELK:
Elasticsearch: It's an open source realtime distributed, RESTful search and analytics engine. It collects unstructured data and stores in a cultivated format which is optimized and available for language based search. The beauty of Elastic is scalability, speed, document-oriented, schema-free. It scales horizontally to handle kajillions of events per second, while automatically managing how indices and queries are distributed across the cluster for smooth operations.
Logstash: Logstash is used to gather logging messages, convert them into JSON documents and store them in an ElasticSearch cluster.
reOptimize.io: Once we run ample of servers we usually end up with burning good amount not intentionally but of course because of not have a clear visualization. At reoptimize helps thereby providing a detailed insight about the cloud expense the integration can be done with 3-4 simple steps but before that you might need to look into the request which can be accessed here. Just a heads up that they only need a read access for all these and permission docs can be found here .
Image copyright reOptimizeD-6
7 DevOps vs Agile
|DevOps culture can be enabled in the software industry to deliver reliable build.|
Agile is a generic culture which can be deployed in any department
|The key focus area is to have involvement at the end to end process|
Agile helps management team to push the frequent release
|Enables quality build with rapid delivery|
Keep team aware of frequent changes for any release and feature
|Agile sprints work within the immediate future, A sprint lifecycle varies between 7-30 days.|
DevOps don’t have such scheduled matrix, they work to avoid such unscheduled disruptions.
|Team size also differs, in Agile wee can minimal resource can be one as well.|
DevOps works on collaboration and bridge a big set of the team.
8 What are a DevOps engineer & roles and responsibilities of it?
Don’t want to be specific but I should say when they build or code they keep the business logic in mind. DevOps engineer should also seek for why this but why not this. They add value when they communicate infra which help business to scale. DevOps team should have control on data matrix collection which could be business driven and the automation pipeline would be Dev driven. As current demand of the cloud is on the pick they should also have a control over the scalability.
DevOps engineer are expert collaborators and also they are often known as a mentor for a software developer as they help them to build a scalable architecture. At the same time, they join IT and security team to ensure that there is enough work has been done to push quality code and to avoid data breach.
9 DevOps engineer salary
As per the current market, trend salary doesn’t seem to be a bar for right candidate. The basic expectation from companies are, the candidate should be a good learner and at the same time, he or she should be able to implement that expertise to resolve the real-time problem. And for cracking good no’s you must have the ability to build an automated pipeline of DevOps
Above table is a kind of current trending no’s which we recently heard from candidates, companies and also the analytical firms who all participate in this (Hiring Firms). Nowadays there is we can see there is a demand for DevOps engineer in almost every organization. And also I must say Startup always beat MNC in salary though they also beat in work, which I believe it is always a good sign to see a growth in your skill and no’s.
DevOps by vmware (Credit: VmWare)
Your email address will not be published. Required fields are marked *
What is DevOps?DevOps is nothing but the combinati... Read More