Sharing data across applications sounds almost trivial from a technical perspective, but having conducted several hundred due diligences across many industries, we are constantly struck by the persistence of data silos. These silos extract a massive, sometimes hidden cost that personifies itself in suboptimal decision making and missed opportunities.
Having a thoughtful and practical strategy for sharing data and integrating systems pays massive financial dividends, and is also a prerequisite for advance capabilities such AI / ML.
Why Do Data Silos Persist?
There has been no shortage of research related to the ways data evolves. A statement made back in 1967 that has withstood the test of time is Conway’s Law, named after programmer Melvin Conway. His original wording was,
“Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure.”
Conway’s law suggests that organizational and social boundaries are built into systems, something we’ve all observed (e.g. sales uses CRM, finance uses a financials application, HR uses an HR application). As noted, this leads to siloed business processes and data, which produces suboptimal results.
Integrations Center of Excellence (CoE)
One way to offset this phenomenon and help ensure an organization is prepared to optimize processes and efficiently implement digital initiatives is to build an integration center of excellence.
This doesn’t have to be a dedicated group and costly initiative only meant for large enterprises. For many small and medium size businesses, this may simple be a list of approved patterns or an engagement model with your 3rd party IT service provider. In all scenarios, however, the goal is the same, to enable data sharing across the enterprise.
To effectively set up an integrations CoE, we have identified six key building blocks:
- Establish alignment between corporate strategy and required integrations: It sounds obvious, but many organizations fail to explicitly align their corporate strategy with their technology and digital strategies. Doing so helps you clarify the gaps that need to be addressed and relative priorities. This exercise, which should take days, not weeks, and should then be memorialized via a simple written document that is shared throughout the organization.
- Assemble the right team: Organizations should take stock in the current team, and then consider a people strategy, which includes reuse/repurpose existing resources, upskilling/training, hiring, or renting third party skills. As mentioned above, some organizations might simply need their third-party IT service partner to provide the IT development skills to build integrations and share data.
- Define your architecture: Modern packaged solutions often have pre-built integrations or APIs, and any custom build should have a similar ability to interface with the application and underlining data. There are integration platforms as a service (iPaaS) which depending on the organization’s needs, can provide a complete data management suite (Informatica, Dell Boomi) or if the need is simply consistent plumbing for interfaces there are several reliable open source options (RabbitMQ, Kafka). The important part is to have an architecture that can guide the software delivery to ensure consistency and extendibility.
- Have consistent processes for delivering software changes: The software delivery lifecycle (SDLC) is the path for deploying software changes to your business’ operations. The ability to quickly expose and share data in a consistent manner is paramount in the new World, and your SDLC should take in consideration the approved architectural patterns. Speed and throughput KPIs are important measures of a modern IT organization
- Define the data boundaries and owners: Access to the right information at the right time is the overarching goal of efficient IT systems, and there is much to say about the data strategy beyond the scope of this article. In relation to sharing data, the integration capability needs to be mindful of data ownership. An important activity is to map out the data ownership by organizational groups. Spoiler alert – don’t expect alignment the first time around!
- Reach beyond your current internal systems: One of the biggest drivers of innovation and efficiencies are enabled through external data integrations. Explore how you get data from your customers, how you share data with your trading partners and vendors, and look for ways to automate data sharing beyond your organizational boundaries.
Read Dell Boomi’s Cloudflare case study where they were able to automate the high volume and high velocity backend billing process using Boomi’s cloud solution to orchestrate the process across systems. Boomi’s iPaaS provided a low-code, cloud native solution to quickly deliver results and reduce the complexity across Cloudflare’s system landscape.
There are few investments with better payback than a thoughtful, intentional integrations CoE. The work required to improve in this area is often quick hit in nature, and the use cases that can be unlocked are immense.