CI/CD Pipeline for a sample project

Things to remember:

• Analyze the existing infrastructure and process
• Find the best places/situations to automate in small chunks.
• Brainstorm & plan the processes thinking about the cost, time, quality, quantity in delivery.
• Combine all the micro steps and make the delivery pipelines seamless.
• Actions should not break current system, so it should increase the efficiency.

High level workflow of a CI/CD pipeline for a web-based trading system for a financial institution:

• Focus on all the environments.

1. Development
2. QA
3. UAT
4. Staging/Pre-Production
5. Production

• Upgrade the OS, apply patches continuously.

Use the latest stable (LTS) versions/AMIs (Images)

• Segment the existing application architecture in to micro services.
• Integrate Continuous integration tools. Create an application build pipeline (having multiple jobs) and make it seamless.

1. Jenkins/Hudson
2. Build tools - Ant, Maven, Gradle
3. Replay stock exchange price feeds
4. Setup trade simulators
5. Version Controlling - Subversion, Git, Github, Bitbucket etc.
6. Artifact Repositories - Nexus, Apache Archiva, Artifactory
7. Unit/Integration/Coding standard Testing tools - PMD, Cobertura, Rspec, Behat
8. Quality Control Tools
9. Alerting tools

• Integrate Continuous Delivery/deployment tools. Create a delivery pipeline for the trading platform (having multiple jobs) and make it seamless.

1. Identify the sequence of deployment at which frequency
2. Configuration Management - Chef/Puppet/Ansible
3. Infrastructure Automation/IAAS/PAAS - AWS Cloudformation/Local cloud flavour
4. Monitoring Tools
5. Log rotating & Management Tools
6. Blue Green Deployments

• Implement/Integrate Security Management Tools

1. Multiple security layers
2. Multiple VPCs, VPC Peerings
3. API Gateways, Micro services
4. Encryption
5. Bastion Hosts
6. Port opening/Closing documents
7. Updated diagrams for existing and proposing enterprise architecture
8. Firewalls, VPNs, VPGs if necessary
9. Active Directories or Citrix etc.

• Enhance the compliance to best practices
• Documentation (Api, Code, Infrastructure, SLAs)
• Implement Non Disclosure Agreements (NDA)

Systems that are on premise:

• Can setup jenkins to build and deploy in local test environments.
• Implement proper service access procedure and IT policies
• Implement data protection policies to prevent manipulations and unauthorized access to sensitive data
• Implement firewalls/VPNs if connect/allow access from/to remote locations/home etc.
• Local data centre hardware/software needs to be encrypted and use keep updating password sensitivity or password life cycle.

Systems that are on AWS:

• Blue-Green deployment plans to reduce deployment related risks
• Have proper roll back plans
• Implement extra security measures
• Implement password management tools/Key management tools etc.
• Implement good AWS governance.