TL;DR

In a recent workshop on responsible AI governance for local governments, Open North convened a group of experts from the public sector and research organizations to engage with the tangible use cases and challenges for AI in local government. The conversation was immensely insightful, and it confirmed much of what we at Open North have been arguing: that data governance is a crucial and overlooked component of any local government’s AI governance strategy.

A strong data governance framework, developed with intention and guided by values, is essential to the data and digital maturity required by any organization to effectively and responsibly use AI tools. It is the most important tool for solving challenges around data quality, data silos, data stewardship, transparency, and accountability, all of which are issues that are magnified by any AI tool.

However, data governance is also essential to AI governance for additional, more strategic reasons:

First, while regulations and ethical principles are debated and (very) slowly implemented at national or provincial levels, data governance is an area in which local governments can actually take immediate, effective action and define their own approach to adopting AI technologies.
Second, we do not know how AI tools will develop in the future, but we do know they will require advanced data and digital maturity. As such, data governance is also an investment in an organization’s capacity to engage with technologies as they emerge and evolve.
Third, data governance and the resultant maturity is essential for any organization far beyond the issues of AI. It is an investment whose benefits will be broad, long-lasting, and impact multiple areas of operation at once.
Fourth, Open North’s stance on data governance is that, unlike private corporations, local governments are responsible to residents. Data governance is, in part, about demonstrating that responsibility through transparent, inclusive, and accountable decision-making around data use. A key public sector concern with AI is a lack of trust; trust is gained by having such a data governance model in place. Data governance is foundational to responsible public engagement on AI use.

In this blog post we will unpack our stance on data governance for responsible AI in greater detail, summarize our research partners at Concordia University’s Applied AI Institute’s talk on AI governance, describe the workshop’s findings, and conclude with our insights for local governments and next steps.

The Community of Practice Workshop Series

On April 17, 2023, Open North’s Community Solutions Network held its first community of practice workshop specifically on responsible AI governance in local municipalities. The workshop, the results of which can be read about in detail here, was attended by civil servants from coast to coast to coast and concluded that local governments were struggling with three key challenges:

The emergent debate around AI had pushed them into uncharted policy and regulatory territory.
Education, both internally and externally with residents, is particularly complex for AI, as it needs to balance informing with learning.
They needed not just new strategies to handle these wicked problems, but new types of strategies to effectively manage the many interconnected dimensions of the challenge.

Our conclusion from this workshop and our extensive engagement with local governments throughout the Community Solutions Network was that, while the high level philosophical and regulatory debates currently underway are important, we needed to dig in from the perspective of local governments’ specific challenges, needs, and opportunities. To that end, we organized a second workshop to do just that.

On August 23, 2023, we held a second workshop together with our partners at Concordia University’s Applied AI Institute. The introductory talks given by Prof. Fenwick McKelvey and Open North’s Thomas Linder can be viewed here. Both talks provided practical perspectives for unpacking the challenge of AI into constituent, manageable components. Professor McKelvey described AI governance as a pyramid with technological and data governance underpinning the tip of AI governance, and as a “Rubik’s cube” in which AI governance can be understood as trying to solve for six different aspects (or colours) at the same time: type, law, standards, ethics, supply chains, and expertise.

Thomas Linder’s talk unpacked those bottom two layers of the pyramid further. At these foundational levels, AI must be understood as more than just the individual tools. The tools are just one component within a broader social and technological infrastructure and ecosystem, all of which must be considered when assessing an AI technology. The talk introduces three key methods for local governments to aid this kind of consideration: decentering the uniqueness of AI, recentering digital maturity, and working responsibly from problem to solution. These three methods form the core of Open North’s stance on data governance for responsible AI.

Workshop Discussion

The workshop discussion covered a number of critical issues, but as a whole – and this is the primary purpose of our community of practice workshop series – what it underscored was the importance of engaging with medium-sized and smaller cities, towns, and communities. A great deal of the “AI and cities” debate is driven by very large cities with extensive resources, like Amsterdam, London, Barcelona, New York, etc. These cities are pioneering important use cases and governance approaches, but their needs and challenges are nothing like those of a city like Hamilton or Kelowna. Yet, cities like these are also confronted with the need to consider governing for and with AI – and we heard an important range of issues from them in this workshop, two of which we will highlight here.

First, local governments are considering a wide range of different AI tools, and each comes with significant and significantly different governance needs. Through a discussion of what use cases were being implemented or being considered, it became clear that, despite the current generative AI hype, these local governments were also assessing non-generative AI chatbots as well as GPT-3 or -4 based tools for internal as well as public-facing projects, a range of traffic analysis and control machine learning tools, various smart CCTV and smart streetlamp tools, and more. Participants discussed how difficult it was to assess the different risk profiles of each tool, to responsibly experiment with and test them, to govern and currently manage the data inputs and outputs, and to effectively communicate internally and externally about their use and purpose. The complexity of what is becoming known as “algorithmic impact assessment” as well as the data governance required for responsible AI use was exceeding their capacity. This finding supports the research conducted in Canada (Wan & Sieber, 2023) and internationally (Davidovic et al., 2023; Marcucci, Kalkar, & Verhulst, 2022).

Second, across the board participants are specifically deeply concerned with their digital and data maturity in the face of artificial intelligence technologies. These are the main questions that arose in our conversation:

How to identify the best suited data sets?
How is responsible collection governed, particularly if more or new data is needed? Are all potentially impacted voices at the table?
How is ongoing data collection governed?
How are data silos best broken down and data collaborations established to ensure the data flows that the tool requires?
How is the data flow managed?
What are the appropriate metrics for data quality, and how is sufficient data quality ensured?
What level of master data management is needed for the tool?
How can trust, internally and publicly, in the data use – both on the collection and input side as well as the output and usage side – best be established and maintained?

What strikes us at Open North is that these issues are in no way new or unique to AI. These are well-established data governance considerations that have been growing in salience for the public sector for over a decade. Open North works and writes extensively on the importance of data governance for local governments; our stance is that data governance represents far more than the compliant management of data; it is a critical lever through which to readjust the balance of power between corporations, governments, and people and to enable deeper, more transparent, inclusive, accountable, participatory, and democratic control over what data is collected, how it is used, what risks are incurred, and who benefits.

However, what has changed is that AI tools have raised the level of risk around data and so dramatically thrust their importance to the forefront. And thus, while there are other crucial components to responsible AI governance, “data governance for AI” is also just good data governance! These data governance considerations are a) essential to responsible AI governance, b) existing areas in which local governments can immediately and effectively act, and c) doing so not only enables better AI governance but also vastly improves the maturity of the local government as a whole.

Finally, the need for governments to establish and maintain trust was vitally important before the current AI debate, and it is of paramount importance now. There are numerous mechanisms needed to build and maintain trust in AI tools, from algorithmic impact assessments to auditing requirements and procurement contracts that counteract black-boxed algorithms; however data will always be the crux of the issue. As Open North has continuously argued, data governance for the public sector is always also about the relationship between government and the public. Data governance for AI is a necessary (although not sufficient) mechanism to ensure the public can see and be involved in the decision-making for what data is being used, how, and with what effects. We will be producing a comprehensive assessment of specific data governance for AI measures soon — in the mean time we finish this workshop series update with six foundational questions for data governance for AI:

Public education: how are you teaching and learning from residents?
Data sovereignty: between residents, governments, and private partners, how is the data use governed? Are all voices represented?
Data collaboration: how is data effectively shared across sectors and organizations?
Data transparency: how open are the data sets used for AI tools?
Data quality: is the data accuracy and fit demonstrated?
Privacy: what privacy principles are operationalized, and how?