DevOps is your garage, not your mechanicRecently, our team has been focused on stabilizing and expanding those tools, and then getting out of the way so people can use them when they need them. Here’s what that looks like.
Cleaning up the garageFirst, we want developers to be able to understand and work on our tools. To that end, we’ve invested heavily in refactoring our codebase to reduce complexity, as well as apply more strict coding principles like clean coding and true object-oriented programming. We’ve paid particular attention to a central component called “the seed job,” reducing that file from 3028 lines to 1276 lines, and we’re not even close to done yet.
Safety firstMuch of that refactoring was done while we worked to stabilize and simplify the posting pipelines. Aside from making the jobs reliable, we’ve drastically improved the fault-tolerance, including implementing what we’re calling a “zero failure system”—when a pipeline hits a snag, it pauses and pings us in a dedicated chat channel with a link to the job and some other info and options for how to handle it. This allows us to remediate the error in-place and then let everything continue as if nothing happened.
More power!A big part of our mindset centers on “Self-Serve DevOps.” For us, the best part about self-serve DevOps is that both our team as well as the teams we work with all benefit from it. DevOps was spending too much time doing the same things over and over, and teams were spending too much time waiting for DevOps to help them do the things they needed, like, yesterday. On both sides we were moving slowly, and mistakes were common because the systems were entirely manual, meaning human error was inevitable.
In recent months, DevOps eliminated the need for about 90% of our internal service desk tickets by converting them into tasks that teams can perform by themselves, with DevOps serving only as a “parameter reviewer.” This allows DevOps to focus on strategic work, while also allowing scrum teams to quickly conduct production operations without waiting on a ticket to be picked up. This also means that fewer errors occur because these things happen in a predictable manner every single time. Finally, this has allowed us to add guardrails around dangerous operations, like adding a step to validate a result set before performing a deletion of data in prod.
The upshot of what DevOps offers is this: “you can do more, faster, with less work, and fewer hoops to jump through.” As stated above, this is the heart of DevOps at Silverchair, and we’re committed to keep applying that mindset to more pieces of the development lifecycle, making it easier for teams to provide better service to our clients, with faster turnaround times, and fewer side effects.