AI engineering
Designing Permissions for AI Tools
Give AI tools only the authority required for a task. Separate reading from writing, bind actions to user identity, and make consequential changes reviewable.
An AI tool can be persuasive in its explanation and still propose an action the user is not authorized to perform. Permission decisions belong to the application and connected services. OWASP’s authorization guidance recommends default denial and validation on requests. Applying that principle to AI tools means defining the allowed action scope independently of the model’s interpretation.
Build an action inventory
List each tool, the resources it can reach, and the effects it can cause. Separate read operations, draft creation, reversible updates, and actions with external consequences. A broad connector credential should not automatically translate into broad authority for every model request.
Derive user and organization identity from trusted session state. Where possible, use scoped service credentials or delegated access appropriate to the operation. For background work, store a reference to the authorized job context and decide what happens if that authorization changes before execution. A queued task should not silently outlive a revoked entitlement.
Make the proposed action concrete
Imagine a hypothetical account-management assistant that can prepare a customer follow-up. Reading account notes, drafting text, and sending that text are distinct capabilities. The application can allow the first two while requiring a specific user action to send. The review screen should show the destination, message, and attachments that will actually be submitted.
If an approved draft changes, determine whether its approval remains valid. Bind approval to the relevant content and target rather than a vague conversation-level permission. For routine low-impact actions already authorized by the workflow, avoid repetitive prompts that train users to approve without reading. Permission design should reflect the effect of the action.
Test the service boundary
Try unauthorized resource identifiers, cross-organization targets, expired sessions, and requests made after a role change. Assert that the server rejects them even when the model supplies syntactically valid arguments. Include direct API calls in testing so security does not depend on the chat interface hiding a button.
Log who initiated an action, which permission was checked, and the resulting operation identifier. Keep sensitive content out of routine logs. Provide a way to revoke a connector and stop pending work. Review the inventory when adding a new tool: a seemingly convenient export operation may create a much wider data-access surface than an ordinary record lookup.
