When AI Inherits Access, Does It Inherit Judgment?

Reflections from a facilitated discussion at Cool/Scary AI Toronto.

Close your eyes and think about how information is shared across your organization.

An employee joins a project and receives access to a shared folder. A manager shares a working document with colleagues in another department. Someone creates a link to make a report easier to find. Over time, people change roles, projects conclude, and documents accumulate.

Some of the permissions governing access to that information are deliberate while others are inherited, outdated, or simply broader than they need to be. So organizations rely on more than those permissions to determine how information should be used. They rely on human judgment.

An employee might have access to a colleague’s working documents and recognize that they aren’t approved for wider circulation. A manager might be able to view sensitive financial information while understanding that it shouldn’t be included in a report to an external funder. Someone preparing a presentation might encounter conflicting figures and know which colleague to consult before deciding what to share.

In each situation, technical access goes hand-in-hand with people exercising judgment about the information’s accuracy, authority, sensitivity, and intended use.

Now consider what happens when an AI assistant enters that environment. AI assistants like Copilot, Gemini, and Zia are embedded in some of the most popular tools that nonprofits use today (Microsoft 365, Google Workspace, and Zoho Workplace) and they can make it considerably easier to find, summarize, and combine information. Depending on their configuration, they can draw on documents and data that employees have permission to access, including information those employees might not have thought to look for themselves.

In this sense, the assistant isn’t necessarily bypassing any permissions. It may be operating exactly as configured, but those permissions don’t always reflect the organizational context that people have historically relied on to decide whether information should be used for a particular purpose. An AI assistant may be able to retrieve a document without knowing that its figures are preliminary, that its contents were shared for a limited purpose, or that someone needs to approve the information before it is circulated.

How confident are you that the permissions in your organization’s systems reflect how you actually want AI assistants to access and use your information?

That was the question behind an interactive exercise I facilitated at the Cool/Scary AI conference at the Centre for Social Innovation in Toronto on October 19, 2026.

With a room full of experienced nonprofit professionals, I wanted to explore how organizations might respond when an AI assistant encounters information that is technically accessible but requires judgment to use appropriately.

The exercise produced a range of thoughtful suggestions, some agreement, and some unresolved differences. Reflecting on the approaches participants proposed after the fact, I began to see an important pattern emerge.

A familiar organizational dilemma

The scenario I presented was straightforward. An employee at a nonprofit organization asks Microsoft Copilot to summarize progress against the organization’s annual goals for an external funder. Copilot identifies three documents the employee can access.

An official quarterly report. This is the organization’s most recently approved report. It indicates that progress against annual goals is behind target.

A more recent working document from Finance. This suggests that performance has improved, but includes detailed financial information and notes about the likelihood of individual invoices being paid.

A more recent working document from Operations. This also suggests better performance, but uses calculations that differ from those in the approved report.

Neither working document has been formally reviewed or approved. Both are accessible to the employee, but neither is clearly identified as an authoritative source for external reporting.

The assistant has found information relevant to the request, but the sources differ in their authority, sensitivity, and conclusions.

Which information should it use?

Should it prioritize the approved report, even though it is less current? Should it incorporate the newer figures? Should it identify the discrepancies and ask for clarification? And what should happen to the sensitive financial details it has encountered?

I asked participants to consider two questions:

1. What would you want the AI assistant to do, and why?
   
2. What safeguards would help ensure the assistant behaves appropriately?

The goal wasn’t to arrive at a single correct answer. It was to explore the decisions, assumptions, and safeguards that might be necessary when AI assistants encounter situations that have relied on human judgment.

What happened in the room

Participants immediately recognized the ambiguity in the scenario. Rather than accepting that the assistant should simply use whatever information it could retrieve, groups began exploring ways to reduce uncertainty and establish clearer boundaries around how information should be handled.

There were different perspectives on what the assistant should do. Some groups reached agreement, while others continued to disagree. The disagreements highlighted how many decisions sit behind a seemingly straightforward request to summarize information.

As I circulated among the groups listening to the discussion, I overheard people exploring a range of practical responses.

1. Some focused on preventing sensitive information from reaching the AI assistant in the first place. They discussed anonymizing and aggregating information, maintaining separate versions of documents, and using isolated accounts or environments.
   
2. Other groups explored configuring AI assistants to recognize approved sources, follow prescribed instructions, and behave according to established expectations.
   
3. Still others emphasized keeping people involved, particularly by reviewing AI-generated information before it was shared externally.

Across these conversations, participants repeatedly returned to the importance of human judgment, organizational policies, and thoughtfully designed processes.

What grabbed me was that many of the proposed solutions involved changing how the organization prepared, managed, and reviewed information—not simply managing how the AI assistant performed the desired task.

A pattern emerged later

As I reflected on the discussion, I began to see a common thread connecting the different approaches. Each approach addressed the same underlying challenge: how do we ensure that appropriate judgments are made when AI assistants participate in workflows built for humans?

The approaches seemed to fall into three groups, each representing a different way of keeping human judgment in AI-assisted workflows.

1. Some sought to reduce the judgments an AI assistant would need to make.
   
2. Others attempted to equip the assistant to make more appropriate decisions.
   
3. Still others relied on people to evaluate the decisions reflected in its output.

Each approach has its strengths and each approach also leaves certain risks unresolved.

1. Limit access: Reduce the need for judgment

Much of the conversation focused on controlling what information an AI assistant could encounter. Participants proposed preparing anonymized or aggregated datasets, creating separate versions of documents for AI use, and restricting access through dedicated accounts or environments. These suggestions connect to established information security practices.

The principle of least privilege, for example, involves giving users and systems only the access necessary to perform their intended tasks. Applying that principle to AI-assisted workflows can help organizations reduce the amount of information available to an assistant.

Organizations might also use separate accounts, restricted data repositories, or isolated environments to establish clearer boundaries around information access. Depending on the architecture, containerization and other isolation techniques can help separate workloads. 

Another possibility is to prepare information specifically for AI-assisted analysis, removing unnecessary personal or confidential details before it becomes available to the assistant.

The suggestions all sought to have people establish boundaries, and shift important decisions earlier in the workflow rather than relying on the assistant to determine whether every piece of information is sensitive or appropriate.

Let’s go back to our original scenario. If Copilot could access only the approved quarterly report, it wouldn’t encounter the potentially conflicting calculations in the Operation Manager’s working document. That would reduce one significant judgment call.

But it would also mean the assistant couldn’t identify potentially important developments reflected in newer documents that might form an important part of the funder report. For example, if a program began slowly in Q1 and is exceeding expectations in Q2, then it might not be accurate or helpful to report to the funder that the program is underperforming. Even if Copilot had access only to the approved report, it could still misinterpret the figures, overlook qualifications, or produce an inaccurate summary.

Restricting access requires ongoing organizational work. Someone needs to determine which information is appropriate, establish permissions, maintain them as responsibilities change, and ensure that information prepared for AI use remains current.

Creating separate versions of documents can introduce duplication and additional maintenance. Anonymization can reduce privacy risks, but doesn’t necessarily eliminate the possibility of identifying individuals from the remaining information if you are working with small data sets or within close-knit communities.

Limiting access can substantially reduce the risk of inappropriate information exposure, but it doesn’t necessarily resolve questions about how the remaining information should be interpreted or used.

2. Configure the AI: Guide the judgments it makes

Other participants explored ways to configure AI assistants to behave more appropriately. Suggestions included identifying preferred information sources, establishing instructions about how information should be handled, and configuring agents or other safeguards to guide the assistant’s actions. Rather than preventing the assistant from encountering every situation that might require judgment, this approach attempts to give it the context and direction necessary to make better judgment calls.

An organization might configure an assistant to prioritize approved reporting documents, identify conflicting figures, or request clarification before incorporating unverified information into an external report.

This is where distinctions between different types of configuration become important. Some configurations establish enforceable boundaries, such as restricting which repositories an assistant can search or which tools it can access. Other configurations influence behaviour through instructions, prompts, or policies that the assistant is expected to follow. Both can be useful, but they aren’t equivalent. An instruction to use only approved information isn’t necessarily the same as a technical control that prevents access to unapproved information.

Returning to our scenario, imagine that Copilot has access to all three documents. We might instruct it to prioritize the most recent available information. But the most recent documents haven’t been approved for external reporting.

We could instead instruct it to use only approved sources. That would address the authority question, but might overlook newer information that materially changes the accuracy of the report.

Perhaps we want it to identify the discrepancy, use the approved report as its baseline, and flag the newer information for review. That seems reasonable, but it introduces further questions.

How does the assistant recognize which documents are approved? What happens when approval status isn’t recorded consistently? Who determines whether the discrepancy is significant enough to escalate?

The effectiveness of the configuration depends partly on the quality of the organizational information and rules supporting it. AI assistants can evaluate information and make context-dependent decisions, but they don’t automatically possess the institutional knowledge, authority, or accountability that inform an employee’s judgment. Configuring an assistant can help it navigate these situations. It doesn’t guarantee that every decision will be appropriate, particularly when the underlying organizational expectations remain ambiguous.

3. Review the output: Verify the judgments made

A third approach focused on keeping people involved in reviewing AI-generated information, particularly before information was shared outside the organization. This is often talked about as keeping a human in the loop.

In practice, this too can mean different things. A person might review an assistant’s response before sending it. A workflow might require formal approval before information is published. Or an assistant might be configured to stop and request a decision when it encounters a situation outside its defined authority.

In our scenario, a knowledgeable employee could review the summary, compare it with the approved report, investigate discrepancies, and determine which information was appropriate to share with the funder. That review could address problems the assistant wasn’t equipped to resolve independently. But it also raises questions about what the reviewer can actually verify. Does the employee know which sources the assistant used? Can they see whether the assistant omitted important qualifications? Do they understand how the figures were calculated?

Source attribution, audit trails, and access to supporting documents can make review more meaningful. Without sufficient visibility into how a response was produced, human oversight can become little more than a final glance at a polished document.

There is also an important distinction between reviewing information for accuracy and reviewing it for appropriate disclosure. A summary might be factually accurate while containing confidential information that shouldn’t be shared externally. Alternatively, it might contain no sensitive information but still misrepresent the organization’s performance. Both require judgment, but not necessarily the same expertise or authority.

And there is another limitation. By the time someone reviews an AI-generated response, the assistant may already have accessed and processed the information used to produce it. If that access or processing was inappropriate, reviewing the output doesn’t undo what has already happened.

Human review can prevent some errors and inappropriate disclosures from reaching their intended audience. But it cannot substitute for the decisions and safeguards needed before the information reaches the assistant.

Bringing the approaches together

Looking at the three approaches individually helped me recognize something that wasn’t immediately apparent during the exercise. They address different parts of the same problem.

1. Limiting access reduces the information an assistant can encounter, but doesn’t guarantee that the information it uses will be interpreted accurately.
2. Configuring the assistant can guide how information is selected and handled, but instructions and configurations have limitations, particularly when organizational expectations aren’t clearly defined.
3. Human review can identify problems before information is used or shared, but it can’t reverse inappropriate access or processing that has already occurred.

As you may have inferred, these three approaches aren’t mutually exclusive strategies. They can actually function as complementary layers, with each placing responsibility for organizational judgment at a different point.

1. Before AI processing, people determine what information should be available and establish boundaries around its use.
2. During AI processing, organizations configure systems, instructions, and controls to guide or constrain how information is handled.
3. After AI processing, people evaluate the results and determine whether they’re accurate, appropriate, and ready for their intended purpose.

Consider what this might look like in the original scenario. The organization could use least-privilege permissions to restrict access to sensitive financial details that aren’t necessary for preparing a progress report. It could establish a clear distinction between approved reporting documents and working materials, then configure the assistant to prioritize authoritative sources and flag conflicting information for review. Finally, it could require an authorized employee to examine the resulting summary, investigate noted discrepancies, and approve the information before it is shared with the funder.

None of those safeguards is particularly unusual on its own. What matters is how they work together. The appropriate combination will depend on the sensitivity of the information, the consequences of an error, and the organization’s capacity to maintain the necessary processes and controls. A low-risk internal brainstorming exercise may need relatively lightweight safeguards. Preparing a report containing sensitive financial or personal information for an external audience warrants greater care.

The goal isn’t to introduce elaborate approval processes around every AI-assisted task. It’s to recognize where risks arise and design safeguards that are proportionate to them and are consistent across many/all AI-assisted tasks.

What changes when AI enters the workflow?

One of my biggest takeaways from the exercise is that introducing AI doesn’t eliminate the need for human judgment. It just shifts where that judgment needs to happen.

Organizations have always relied on people to navigate situations where technical permissions, formal policies, and practical realities don’t align perfectly. An employee may know that a document is accessible but not approved. They may recognize that information shared for one purpose shouldn’t automatically be reused for another. They may understand when a decision requires consultation with a colleague or approval from leadership.

Some of that knowledge is documented. Much of it may be informal, embedded in relationships, established through experience, or simply understood as part of how the organization works. AI assistants make it increasingly important to examine those assumptions.

If an organization’s information-handling expectations depend on people recognizing context that isn’t reflected in its systems or processes, how will those expectations be made explicit when an AI assistant participates in the work?

That question has implications beyond the AI tool itself. It may require organizations to revisit how access is granted and maintained, how documents are classified, which information sources are authoritative, how policies are communicated, and where review and approval responsibilities sit.

For nonprofits, those decisions also need to be practical. Many organizations don’t have dedicated information security teams or extensive technical resources. But before deciding which safeguards to introduce, organizations need to understand where they’re already relying on human judgment—and what happens to those judgments when AI becomes part of the workflow.