Abstract:With growing demand for containers in the enterprise, build pipelines are a bottleneck to success. Traditional workflows can't release application candidates quickly enough to fulfill demand. With over 400 development teams across many different business units, Pearson had to move away from massive installs of traditional build pipeline tools and rethink the entire concept. In this talk we'll demonstrate how we have built in security compliance, performance testing, quality assurance, abstracted away complexity, reduced overhead, aim to recover 10% of developers time and turned build tools into cattle.
This represents the story to date of an in-flight engineering project to modernise the digital estate of a global enterprise organisation and how scale of the operation is leading us to challenge many established beliefs. Attendees will walk away with everything from workflows to code which they can use to get started in their own endeavors.
Problem Statement of container builds at scale:
- Significant developer time managing build process
- Bottleneck across 400+ development teams
- Lack of security compliance across the enterprise footprint
- Little to no standardization
- Significant resource consumption
- Build pipelines are pets requiring constant attention
- CAB approvals requiring weeks of effort to ensure stability
- Manual processes contributing to stability issues
- Development teams don't want to manage their own build process
Solution:
- Standardized base images
- Allows for deltas for additional libraries
- Reduce time for testing with standard images being pretested
- CISO requirements baked into base images
- Pre-approved for deployment
- Every dev team gets their own build tools
- Prevents build process from bottlenecking other teams
- Abstract build servers away and turn them into cattle
- Build servers contain no long-term information
- Configured through repos
- Requires no direct dev interaction
- Build done automatically on git push or timed event
- Build pipeline deploys and kicks off QA/CISO/PERF tests automatically
- Provides feedback loops to necessary engineers
- Automated gates to approve/reject deployment
- Automated processes establish confidence for CAB approval
- Easy oversight to view process
