Generative AI and resilient hybrid cloud systems

As enterprises invest their time and money into digitally transforming their business operations, and move more of their workloads to cloud platforms, their overall systems organically become largely hybrid by design. A hybrid cloud architecture also means too many moving parts and multiple service providers, therefore posing a much bigger challenge when it comes to maintaining highly resilient hybrid cloud systems.

The business impact of system outages

Let’s look at some data points regarding system resiliency over the last few years. Several studies and client conversations reveal that major system outages over the last 4-5 years have either remained flat or have increased slightly, year over year. Over the same timeframe, the revenue impact of the same outages has gone up significantly.

There are several factors contributing to this increase in business impact from outages.

Increased rate of change

One of the very reasons to invest in digital transformation is to have the ability to make frequent changes to the system to meet business demand. It is also to be noted that 60-80% of all outages are usually attributed to a system change, be it functional, configuration or both. While accelerated changes are a must-have for business agility, this has also caused outages to be a lot more impactful to revenue.

New ways of working

The human element is mostly under rated when to comes to digital transformation. The skills needed with Site Reliability Engineering (SRE) and hybrid cloud management are quite different from a traditional system administration. Most enterprises have invested heavily in technology transformation but not so much on talent transformation. Therefore, there is a glaring lack of skills needed to keep systems highly resilient in a hybrid cloud ecosystem.

Over-loaded network and other infrastructure components

With highly distributed architecture comes the challenges of capacity management, especially network. A large portion of hybrid cloud architecture usually includes multiple public cloud providers, which means payloads traversing from on-premises to public cloud and back and forth. This can add disproportionate burden on network capacity, especially if not properly designed leading to either a complete breakdown or unhealthy responses for transactions. The impact of unreliable systems can be felt at all levels. For end users, downtime could mean slight irritation to significant inconvenience (for banking, medical services etc.). For IT Operations team, downtime is a nightmare when it comes to annual metrics (SLA/SLO/MTTR/RPO/RTO, etc.). Poor Key Performance Indicators (KPIs) for IT operations mean lower morale and higher degrees of stress, which can lead to human errors with resolutions. Recent studies have described the average cost of IT outages to be in the range of $6000 to $15,000 per minute. Cost of outages is usually proportionate to the number of people depending on the IT systems, meaning large organization will have a much higher cost per outage impact as compared to medium or small businesses.

AI solutions for hybrid cloud system resiliency

Now let’s look at some potential mitigating solutions for outages in hybrid cloud systems. Generative AI, when combined with traditional AI and other automation techniques can be very effective in not only containing some of the outages, but also mitigating the overall impact of outages when they do occur.

Release management

As stated earlier, rapid releases are a must-have these days. One of the challenges with rapid releases is tracking the specific changes, who did them, and what impact they have on other sub-systems. Especially in large teams of 25+ developers, getting a good handle of changes through change logs is a herculean task, mostly manual and prone to error. Generative AI can help here by looking at bulk change logs and summarizing specifically what changed and who made the change, as well as connecting them to specific work items or user stories associated with the change. This capability is even more relevant when there is a need to rollback a subset of changes because of something being negatively impacted due to the release.

Toil elimination

In many enterprises, the process to take workloads from lower environments to production is very cumbersome, and usually has several manual interventions. During outages, while there are “emergency” protocols and process for rapid deployment of fixes, there are still several hoops to go through. Generative AI, along with other automation, can help greatly speed up phase gate decision-making (e.g., reviews, approvals, deployment artifacts, etc.), so deployments can go through faster, while still maintaining the quality and integrity of the deployment process.

Virtual agent assist

IT Operations personnel, SREs and other roles can greatly benefit by engaging with virtual agent assist, usually powered by generative AI, to get answers for commonly occurring incidents, historical issue resolution and summarization of knowledge management systems. This often means issues can be resolved faster. Empirical evidence suggests a 30-40% productivity gain by using generative AI powered virtual agent assist for operations related tasks.

AIOps

As an extension to the virtual agent assist concept, generative AI infused AIOps can help with better MTTRs by creating executable runbooks for faster issue resolution. By leveraging historical incidents and resolutions and looking at current health of infrastructure and applications (apps), generative AI can also help prescriptively inform SREs of any potential issues that may be brewing. In essence, generative AI can take operations from being reactive to predictive and get ahead of incidents.

Challenges with generative AI implementation

While there are strong use cases for implementing generative AI to improve IT Operations, it would be remiss if some of the challenges weren’t discussed. It is not always easy to figure out what Large Language Model (LLM) would be the most appropriate for the specific use case being solved. This area is still evolving rapidly, with newer LLMs becoming available almost daily.

Data lineage is another issue with LLMs. There needs to be total transparency on how models were trained so there can be enough trust in the decisions the model will recommend.

Finally, there are additional skill requirements for using generative AI for operations. SREs and other automation engineering will need to be trained on prompt engineering, parameter tuning and other generative AI concepts for them to be successful.

Next steps for generative AI and hybrid cloud systems

In conclusion, generative AI can bring in significant productivity gains when augmented with traditional AI and automation for many of the IT Operations tasks. This will help hybrid cloud systems to be more resilient and, in due course, help mitigate outages that are impacting business operations.