Fidelity Investments

Fostering Technology Recovery and Resiliency

Fidelity Investments was committed to creating a new type of enterprise business resilience program to help it manage increasing disruptions and meet looming regulatory deadlines. To do this required the company to evolve from a legacy disaster recovery mindset, better collaborate across the business and technology teams, and create a more proactive program that allows the business to seamlessly detect, address, and rectify risk and more easily absorb small- and medium-sized “shocks” to the system.

The company had historically used manual processes to assess risk, which required substantial full-time equivalent hours, produced inaccurate data, and made it difficult to work within cost measures when the volume of data and work increased. While the firm had made it a priority to build a culture of operational resilience, changing business and resilience requirements required it to reassess its approach over the years, ultimately morphing its disaster recovery program into a new technology, resiliency, and recovery approach. Even with these thoughtful activities, subsequent events proved too challenging for the business to handle with in-house capabilities.

When Fidelity moved several of its critical systems to the cloud, it created a subsequent 4x increase in the number of components, requiring end-to-end testing to measure and manage resiliency. Tests were run on weekends and did not always yield the most accurate results as the conditions over the weekend test were not the same conditions in which the regular business services would run. At the same time, the company saw a staggering growth in data from multiple sources as it attempted to capture consumer behavior patterns through social media and other tools to properly target clients and prospects, and also saw a 32% increase in its critical applications.

Fidelity soon realized the dramatic growth in critical applications and data volumes meant it was impossible for it to keep pace, and it was time to call in outside help. The firm sought out a collaborative software partner that could provide a flexible and scalable solution to effectively capture data, automate processes, and conduct testing to create a new level of resilience. Any resilience partner would need to meet a clear set of requirements and understand the company’s vision for its ongoing resilience journey. After a competitive evaluation, the company selected the Fusion Framework to help it create a new enterprise business resilience program that could grow and scale along with new technologies.

A New Way to Engage and Collaborate With Enterprise Technology

Fidelity recognized it needed to make a fundamental change from a disaster recovery approach to a technology recovery and resiliency approach. That required it to identify a new way to engage and collaborate with its enterprise technology and design for resiliency in everything it does, from mainframe resiliency to transferring workloads within and outside of regions to container and platform management.

A top challenge for the business was how to ensure resiliency in a dynamic technology landscape. When the business adopted new technology and capabilities, the established enterprise business resiliency program was challenged. This made it necessary to better understand the new technology and how it operates and, more importantly, where it fits into the resilience program and how resilience can be ensured. Some key challenges were the dynamic transfer of compute workload and elasticity, containerization, platforms and micro services, and exponential data growth.

By working with Fusion’s advanced automation and guided workflows, the company minimized the need for extensive man hours. Previously, a full-time employee was needed to capture data. With Fusion’s advanced automation and guided workflows, the company can generate data and results at the push of a button and capture large amounts of data from the cloud. Fusion worked closely with the business to establish a process that automatically captured data in real time, thus reducing manual data entry. All data collected is subsequently scored and tracked to align with the company’s needs and establish resilience. The flexibility offered by Fusion enabled the company to decide when to close out its testing cycle and fostered a heightened focus on outliers.

Fidelity worked with Fusion to create a resiliency index for proactive risk assessment before deploying new cloud applications. Prior, the company had minimum proactive measures to test systems before they went live. Fusion and the company collaborated and established set parameters to test and promote resilience confidence to the CIO stack. To ensure accurate results, the company tests under conditions reminiscent of real-world expectations. Scenario tests are based on service composite rather than individual applications and focus on actual production traffic and shocks. Before any new products deploy, the company requires mandatory testing to proactively mitigate risk.

The company will take advantage of Fusion’s advanced automation and guided workflows. If an application does not meet the criteria for deployment throughout the process of moving systems and applications to the cloud, automated workflow will help flag systems that do not meet resiliency requirements. Once the cloud group sends an initial trigger, Fusion uses workflow automation to put the information in motion. Fusion’s holistic approach to resilience and automated workflows enables the company to detect alerts and mitigate risk factors before impacting the business.

Resilience in Action

The Fusion Framework^® System™ fostered heightened resilience at Fidelity. By utilizing Fusion, the company is prepared to continue operating and deliver on their client promise through any disruption. The company saw the power of true resilience during an interruption earlier this year when they learned about an outage with their pricing platform at AWS 10 minutes after the outage occurred. With the proper tools for resilience in place, the company continued business as usual. Typically, this would result in a halt in services. Automatically, services were rerouted to an alternative hosting location which absorbed the entire workload of the downed system. Once the regular environment was thoroughly checked, services seamlessly flipped back to their regular host location.

Operational Resilience to Build Trust

By partnering with Fusion, Fidelity implemented a holistic operational resilience program with automation that reduced the amount of manual intervention needed to ensure resilient systems. It can now also assess risk proactively, intervene to minimize potential disruption, and engage CISOs by demonstrating that systems were ready to deploy, thus raising its organizational resilience competency. The outcome has been a new resilience program that enables the business to drive heightened customer value by automating tasks and seamlessly absorbing shocks, thereby fostering customer trust through a culture of resilience that allows it to consistently deliver on brand promises.