Approaching the Clouds-Watch for turbulence!

After reading the article, ‘Achieving cloud-native operability with Microservices, DevOps, and continuous delivery’, by Casey West, I feel it is necessary to add a few points on the difficulties some established companies have embracing cloud-native operability.  I have been fortunate to work in IT Infrastructure for the past 35+ years, across several industries (Financial, Consulting, Energy, Oil & Gas upstream and downstream and transportation) and have seen these similar problems across companies:

  • Only a few companies actually maintained architectural guiding principles and had processes to validate projects establishing new IT services for the business
  • Most of the companies I worked for had established applications (five years or older) which have been customized beyond OTS (Off-The-Shelf) recognition or were developed in-house
  • Some applications, during development, ignored architectural principles and considered only the platform they were currently on instead of designing portability to other OS platforms
  • The majority of the companies I was at had not heard of DevOps (it was still very new or non-existent). We believed if we were performing the components of ITIL and ITSM, that was good enough!
  • Most companies I worked at also did not have established continuous improvement programs, although while I was in consulting, we always made sure CI was listed as a deployment phase
  • Probably the most difficult was replacing hardware platforms. This usually required considerable investment and was obtained by
    • Repeated outages of existing hardware where business departments constantly complain to Executive Management — or —
    • Lack of security compliance and repeated audit findings were reported to Executive Management and some Boards!
  • Employing automation throughout IT and Business areas was difficult since many people believe it would mean lost jobs, after all, if I am not performing the tasks manually, what ELSE am I going to do?!

With the companies I worked for, it was a struggle to improve IT Services to a noticeable level by the business.  How did we do it (I say “we” because nobody does it by themselves)?  It can seem insurmountable and a constant uphill battle, but the way I would start was to make an honest assessment of three key areas within IT in the company: People; Processes; Technology.  In most cases, the people I led were very good technically, needed additional training (which is usually the first item cut in budgets) to maintain pace with changing technology.  The second, standard policies and procedures were the biggest issues.  It is SO easy for IT Infrastructure to just get on with day-to-day operations and not bother to document WHO and HOW operations happen.  It is always too late once the auditor shows up and wants to see you execute the procedures!  In my opinion, this is where most IT in companies fail on processes. Not focusing on 2 primary areas:

  1. Maintaining policies and procedures up to date (at LEAST have them updated in the current year!)
  2. Not following the procedures as documented (quickest way to fail an IT audit)

Make sure your documentation is up to date.  As for Technology / Tools, I have experienced a couple of ways to improve automation and collaboration.

  1. If your IT infrastructure areas are outsourced (IaaS or PaaS), make sure the internal IT management team has the ability to access the tools the service provider uses to manage and monitor the environments. Several companies I worked for, we were at the mercy of the service provider to keep us informed when issues (either outage or capacity) occurred.
  2. If the IT infrastructure areas are internal, leverage OTS tools with capability to interface and automate workflows as well as tasks
  3. Having the tools integrated with incident and problem management process/tools is a big step towards establishing automation

For collaboration, it is a difficult process to get IT to adopt.  All of the companies I was a part of had many silos with high towering walls – and that was just within the IT Infrastructure teams!  Network Services did not see any need to communicate with the server/storage teams, Applications-Development teams did not communicate unless absolutely necessary with IT Infrastructure/Operations and Architecture groups don’t feel compelled to work with Operations. In a couple of companies, Architecture would develop guiding instructions, but would not help Operations build it or document it.  Talk about a lack of collaboration!

Mr. West states that using Microservices doesn’t make it easier to build apps – he is right and for that reason, many companies skip the architectural exercises because it slows down the speed to market or costs too much.  This is not fixed by adding more complexity, it is fixed by increasing the awareness (and modifying the culture) to see the long term benefits.  In every company I worked at, speed-to-market to get a jump on the competition was a primary concern, right after cost.  Management wanted it “out-the-door”, we’ll fix it later.  I agree with Mr. West; culture is critically important to the success of Microservices or DevOps.  A company’s culture is one of the hardest and longest set of beliefs and behaviors to change and can take years to modify.

It is easier for a startup company to deploy new platforms and establishes a culture that embraces a cloud based enterprise architecture, however, getting a company built with legacy applications, infrastructure and culture, is an uphill climb to adopt Microservices, DevOps and Continuous Delivery.  Not impossible, just takes more time and planning.

Couple of points to note, however, when transitioning applications to a cloud environment:

  1. Make sure your cloud service provider will provide full range of operational services
  2. Make sure you have a RACI matrix identifying WHO is going to do WHAT on how often
  3. Understand what services your cloud service provider is NOT going to do – you will have to probably hire a third party for operational tasks

If companies have these items documented, approaching the cloud environments should be smoother….