The complexity of creating a viable data foundation suitable for AI and IA adoption is critical, further exacerbated by the fact that most organizations had to fast track their digitisation journey because of COVID-19 and therefore aren't working from scratch. For any sizable business enterprise or government institution, it is almost inevitable that its data will be spread across multiple systems, often with some or much of it siloed.
New functions and technological solutions may have been rapidly bolted on, thanks to the frantic pace of global innovation that drives organizations to keep up or fall by the wayside when operating through a pandemic Accordingly, complete data integration and visibility is perhaps nothing more than a dream for most organizations.
Data silos are a major challenge in AI and any digital transformation adoption. Silos by definition restrict innovation. The companies that have successfully created significant value from their data have done so mainly due to data democratisation.
However, data democratisation still requires rigorous data governance, privacy, and security commitments. Organisations are very slow in moving towards opening their silos for analytics purposes, which is a problem because one cannot find new patterns in the data if there is only a limited set of data available.
Another challenge is that the type of data required for some AI and IA projects is different from the data with which most organisations are accustomed to working. For example, some solutions depend on access to a certain amount of unstructured data that may have been only for record-keeping but were never planned to put to use for analysis.
Getting the data required for an AI and IA project, preparing it for analysis, protecting privacy, and ensuring security can be a hugely time-consuming and costly process. But a necessary one.