Automation technology: ways out of the automation disaster

Software developer Benjamin Willenbring was thrilled when his employer Autodesk introduced automated software testing in 2015. However, this enthusiasm subsided relatively quickly because communication with the small automation team was difficult and the results of the efforts did not meet expectations.

“My colleagues talked about test runs failing for unpredictable reasons and left no doubt that they weren’t particularly convinced of the whole thing,” says Willenbring out of the box. Getting the test phase up and running was difficult: there was no documentation and files in abundance. Automated testing was supposed to simplify Willenbring’s work – instead, the automation project posed so many new problems that the developer had to invest the majority of his energy in the following years to solve it.

However, the experience that Willenbring has gained in automation is not uncommon. The higher the level of automation in IT departments, the more instructive examples are that show how automation should not be tackled. Regardless of whether it is automated DevOps workflows, robotic process automation or process automation – the technology should actually ensure that specialists are freed from repetitive tasks. But inadequate framework conditions and rollouts with failures can quickly turn the automation dream into a nightmare. We’ll show you six ways to avoid automation disaster.

Willenbring’s testing automation nightmare was particularly triggered by one circumstance: the only ones who were able to understand the automated tests were the ones who set them up. Unfortunately the team in question was located in another city.

“One problem with the test framework was that it didn’t provide really good feedback in the event of errors,” says Willenbring. “If an error occurred, the first step was to take Slack to ask the test lead for the cause. The lead then repeated the test manually and communicated the cause of the error.”

The testing framework thus violated essential rules with which every automation initiative should be aligned, says Robert Haas, DevOps Product Manager at DXC: “Automation code must be documented, regardless of whether you use modern methods such as Documentation-as-a-Code or link Visio diagrams. Solving problems is much easier if you can understand what the original state looked like. “

Because that didn’t happen in Willenring’s case, the test results were unusable for his team. This in turn led to a decline in trust in the tests and the team that set them up. “Sometimes the developers just blocked and ignored obvious errors,” says the software developer.

Due to the spatial distance between the developer and testing team, additional problems arose: “You need experts who know what they are doing and select what is being tested. Lots of tests without a real plan generally do not lead to usable and meaningful results”, so Willenbring.

Only a new head of the development department was able to free Willenbring and his team from this vicious cycle – he made it clear that quality has top priority, even when it comes to testing automation. Since then, centralized testing has been a thing of the past – individual departments now set up tests for “their” code. In this way, Willenbring and his team were able to launch tests that were adapted to their needs and ultimately integrate this process into everyday work. Willenbring’s conclusion on this experience: “You shouldn’t even allow an attitude in the sense of ‘that is not part of my job’.”

Tripwire, a provider of automation solutions in the area of ​​IT
security and compliance, was recently amazed at the processes at a major customer in the financial sector. “Our solutions are rolled out automatically,” says Irfahn Khimji, Country Manager for Canada at Tripwire. “So at some point we wondered why it was taking so long for this customer. We saw no increase in license usage, which was not the normal pattern.”

As it turned out, the ideal of an almost completely automated CI / CD pipeline was caught up in reality: each of the many different departments at the customer’s own, individually designed set of software components.

“The customer wanted to integrate more than 30 applications – which led to numerous problems, for example when it came to the various libraries,” says Khimi. “Adjusting the pipeline so that it reliably covers all requirements slowed down the automation process significantly.”

There is no magic bullet to solve such a problem. However, you should be aware that with an increasing number of components that are to be integrated into the automation process, the complexity increases – after all, these components have to be linked with one another and corresponding adjustments have to be made. This, in turn, will result in the automation initiative taking more time and resources. It also plays a role where these components come from: The majority of the pipelines and RPA environments contain a heterogeneous mix of in-house and third-party components. The latter in particular can lead to real problems if errors occur.

Robotic process automation and chatbots are particularly popular in the financial sector. In this context, Muddu Sudhakar, CEO and co-founder of the software company Aisera, warns of a common scenario: Processes are seen as a monolithic functional unit, the internal processes of which are difficult to separate and thus become a black box: “In a monolithic structure, the customer wants check your account balance, withdraw money or transfer money – all of this happens in one step, “explains Sudhakar. “If something goes wrong and there are no monitoring measures, all that remains is a call to customer service or a physical visit to the branch.”

According to the experts, this type of architecture is typical of the initial efforts of companies with automation technology. Such projects could produce good results, says Sudhakar, but only as long as everything goes according to plan. If not, all that remains is to remove the individual functions from the black box – which is why you should avoid them from the outset.

“Divide each process into building blocks that can be audited and checked,” recommends the automation specialist from Aisera.

Many automated processes can benefit from the poka-yoke principle. The error prevention strategy invented in Japan is used, for example, in the Toyota production system. Here, a single process step is divided into two different ones – the second step depends on the first. Paradoxically, the multiplication of the individual steps leads to efficiency gains because mistakes do not flow into the production chain – where their effects can multiply.

Individual automation steps should not only be auditable, but should be constantly checked by other automated processes. This brings you closer to the AIOps area: automated platforms relieve human specialists of the burden of IT management.

Automated CI / CD pipelines often have a dirty secret: Many were initially rolled out by shadow IT to circumvent security guidelines. “Developers just want to develop and move forward quickly with their code,” explains Irfahn Khimji from Tripwire. “When rigorous IT and security processes get in the way, they immediately have ideas in their head to avoid them.”

This does not mean that such CI / CD pipelines are unsafe per se – but you should definitely keep an eye on which security practices have been sacrificed to gain automation efficiency. In addition, you should not forget that every automated process is always another attack vector. Autonomous processes, which are equipped with extended authorizations, can be particularly attractive targets for attacks. Muddu Sudhakar can not only remember one case in which intruders misused the DevOps pipeline to infiltrate malware. This could make its way into the production environment and paralyzed the entire system.

Every hype comes with misunderstandings in tow. Ajeet Dhaliwal, lead developer and founder of Tesults, has seen many companies whose vision of automated testing contrasts quite clearly with best practices. Smaller companies in particular, whose management level has no technical background, seem to be affected by this phenomenon. In the logic of these people, automated tests are simply manual tests without human involvement.

“So they encourage traditional QA testers with no knowledge of software development to automate testing,” summarizes Dhaliwal. “The robustness and flexibility of such approaches is far behind what a software developer with automation know-how can do.” Of course, it is no problem to involve young developers too, a
ccording to the testing expert. But these should be guided by experienced developers to ensure good results. (fm)

This article is based on an article from our US sister publication CIO.com.