We are “Evolving” to “DevOps”

What was the business problem?

Company X envisioned a product to cater the needs of manufacturing companies that would automate the process of data integration, reports, transaction systems etc.

The stakeholders, business had ever changing requirements\priorities and aggressive timelines to launch the project for an upcoming event.

Company X had always worked in waterfall model with more than 3-4 months of development + testing iterations while using TFS as source control, MPP for project planning and other home grown templates to document requirements and other project artifacts.

What challenges we anticipated

The Product manager, Dev manager and leads anticipated the challenges due to

• Ever changing business requirements
• Limited availability of stakeholders and business. Imagine the time required by business at the end of 3-4 month iteration cycle
• Aggressive timelines for product launch

The team took few strong decision and we started evolving

Step 1: Adopt Agile

The team brainstormed and decided

• Go Agile, No waterfall anymore
• Use tools best suited. Picked TFS with the existing licenses.
• TFS provides project management using SCRUM
• TFS also provides tools for CI, Build Automation etc.
• Dev Tool – Visual Studio

 

Benefits Delivered

• Fortnightly presentation of the product developed with each sprint
• Immediate absorption of changes suggested by business
• Acceptance of product by business due to frequent presentation and UAT
• Transparency with stakeholders on progress and budgets
• Projection of product delivery date
• Regular test cycle in each sprint ensured the quality product.

Step 2: Test Automation

 

The team gradually moved from manual testing to automation testing using MS Test Project

 

Benefits Delivered

• Reduced testing effort of new features
• Reduced testing effort of regression testing
• Consistency in each sprint

 

Step 3: Continuous Integration and build automation

 

The next step was to adopt continuous integration. While the test engineers mastered in automation testing the knowledge was readily available to Dev for creating the automation testing for continuous integration.

The test scenarios developed by the test engineers were leveraged by the Dev team and we used again the TFS for continuous integration

Benefits Delivered

• Reduced effort on integrations
• Prevention of check-in of the buggy code
• Prevention of build failure due to partial check-in
• No effort for regression test

 

Step 3.1: No Test Engineers

 

Surprised??? Me too when I first learnt about having no test engineers. Finally we took a bold step and enabled each developer to be the owner of his\her own code. They have to include the efforts of automation testing for the tasks they own.

The Devs think all the scenarios for the functionality they are developing, write automation test cases and ensure the quality delivery. 

The test engineers started working closely with the Dev team and started taking over development tasks as well.

Benefits Delivered

• Reduced testing effort and test iterations
• The build is shipped directly to UAT.

 

Step 4: Deployment Automation

 

The team witnesses manual errors while doing the deployments. Not done yet and we are in process of automating our deployment process.

Expected Benefits

• Reduced\no efforts to troubleshoot the deployment issues
• Frequent releases to production
• Reduced downtime

 

Step 5: Integral Dev and Support

 

The Dev team will continue to support the product for P1 issues while working on sprints.

How it would work

• The Devs will provide 24x7 support on rotation basis (1-2 weeks).
• The Dev on support will not be part of sprints during that period.
• He\She is required the address P1 issues only.
• The Dev should be able to fix the code developed by others.
• The continuous integration and deployment will release the fix to production immediately
• After completion of support duration the Dev joins the sprint and another Dev takes over the support

 

Step 6: Other Aspects

 

A change in mindset

Move to DevOps is not quick, it takes time to adjust. As a team you have to be fully committed and keep pursuing. Initially you will be involved in lot of late night calls, debugging, code drops, etc. but slowly your product will become better and VERY reliable. Every developer will start to feel more accountable as they get involved in live-site issues. It will reflect in a great quality software each developer will produce from now on

 

ELK

The software we are developing is integrated with ELK and logging each step. The logging is based on KPIs like Errors, Security etc.

 

Expected Benefits

• Kibana will help analyze the software health, user experience etc. which will eventually help us to improve the product
• Integral Dev and Support – The Dev on support role will get the details description with steps on the issue reported. This will help him\her to locate the issue quickly.
• Reduced effort in fixing the issues

 

 

While the steps I have mentioned above may not be complete to achieve maturity in DevOps but we are evolving. There would be more benefits in each step but I am mentioning the most important ones we experienced.

I find it exciting to evolve on DevOps. Few of best practices we had followed in other projects in the past also but they were done in silos. We are now adopting all of them under the hood of DevOps.

Our target is to be mature in next 3-6 months.