Organisations wrestling with staggering levels of 'dark data'

Editorial Type: Research Date: 2022-08-03 Views: 1,183 Tags: Storage, Cloud, Strategy, Digital Transformation, Quest Software PDF Version:
New report from Quest/ESG finds companies struggling with data quality, visibility and accessibility

Quest Software in collaboration with ESG has released the '2022 State of Data Governance and Empowerment Report', an annual report highlighting the top challenges and innovations in data governance, data management and DataOps.

ESG conducted a research study of 220 business and IT professionals responsible for and/or familiar with data governance and empowerment strategies, investments, and operations at their organisations. All the organisations represented in the research have 1,000+ employees and annual revenues of $100M+. Key aspects of the project included:

  • Understanding how organisations define, adopt, and prioritise data governance.
  • Uncovering the drivers and challenges of data governance implementations across key data lifecycle and data technology integration points.
  • Providing YoY comparisons/data of fundamental data governance functions and trends (2021 versus 2022).
  • Investigating the adoption levels and efficacy of DataOps and its relationship to organisations' data governance endeavours.
  • The report found that data quality has overtaken data security as the top driver of data governance initiatives, with 41% of those surveyed agreeing that their business decision-making relies fundamentally on trustworthy, quality data. At the same time, however, 45% of IT leaders say that data quality is the biggest detractor from ROI in data governance efforts. While they recognise its importance, they're struggling to improve the quality of their data, and thus the ability to strategically and maximally leverage data in practice.

    "Business users at all technical levels have an edge when they have full visibility into, control over and confidence in their data," said Patrick Nichols (pictured), CEO of Quest Software. "Trustworthy data and efficient data operations have never been more influential in determining the success or failure of business goals. When people lack access to high-quality data and the confidence and guidance to use it properly, it's virtually impossible for them to reach their desired outcomes."

    The report also revealed that business leaders struggle not only to make sense of their data, but to locate it and use it in the first place, with 42% of survey respondents saying at least half of their data was "dark data" - retained by the organisation, but unused, unmanageable and unfindable. An influx in dark data and a lack of data visibility often leads to downstream bottlenecks, impeding the accuracy and effectiveness of operational data.

    "Businesses can't utilise data, much less optimise it for the benefit of their organisation, if they can't actually see it," said Nichols. "IT leaders must make data empowerment their first priority, enabling their organisations to leverage business intelligence, creating a single source of the truth to succeed in today's data-driven world."

    While the challenges of data visibility and observability differ across industries, DataOps was overwhelmingly recognised as the primary solution to drive forward data empowerment. 9 in 10 people surveyed agreed that strengthening DataOps capabilities improves data quality, visibility and access issues across their businesses. The biggest opportunities to improve DataOps accuracy and efficiency lie in investing in automated technologies and deployment of time-saving tools, such as metadata management. Currently, only 37% of respondents describe their DataOps processes as automated, and a similarly small proportion report having automated data cataloguing and mapping today (36% and 35% respectively).

    "Today's businesses are all but forced to be data-driven and evidence-based in their strategies, yet still face significant obstacles that prevent their people from being fully empowered to bring data to every decision," said Mike Leone, senior analyst at ESG. "Organisations that invest in building a data-first culture - fuelled by automation in DataOps processes, high-quality data, holistic governance, and enterprise-wide accessibility - to drive business success, will have the advantage."

    More info: www.quest.com