Setup guide
Everything you must provision before this skill can run โ the exact credentials, permissions, and where to click. Links go to the official consoles and docs, which stay current.
Register an Entra application, grant it admin-consented Microsoft Graph application permissions for Conditional Access and audit logs, and create a break-glass emergency access group to exclude from every policy.
You need to already have
- Global Administrator or Privileged Role Administrator on the tenant (required to grant admin consent for application permissions)
- Microsoft Entra ID P1 or P2 license on the tenant (Conditional Access is not available on the free tier; sign-in log access via Graph also requires a premium license)
- At least two cloud-only emergency access (break-glass) accounts in a dedicated security group, each with a registered phishing-resistant credential (FIDO2 passkey or certificate-based authentication)
- Python 3.9+ with the requests package, or any HTTP client capable of the OAuth 2.0 client credentials flow
How the pieces connect
Credentials this skill needs
Set these as environment variables. Never paste secrets into a chat or commit them.
| Variable | What it is | Where to get it |
|---|---|---|
AZURE_TENANT_ID | Directory (tenant) ID format: GUID (or a verified domain such as contoso.onmicrosoft.com) | Entra admin center > Entra ID > Overview > Overview tab > Tenant ID; also shown on Entra ID > App registrations > your app > Overview > Directory (tenant) ID |
AZURE_CLIENT_ID | Application (client) ID format: GUID | Entra admin center > Entra ID > App registrations > your app > Overview > Application (client) ID |
AZURE_CLIENT_SECRETsecret | Client secret value format: Opaque string, typically 40 characters. Copy the Value column, not the Secret ID; the value is shown only once immediately after creation. | Entra admin center > Entra ID > App registrations > your app > Manage > Certificates & secrets > Client secrets > New client secret > copy the Value column |
ENTRA_BREAKGLASS_GROUP_ID | Break-glass exclusion group object ID format: GUID | Entra admin center > Entra ID > Groups > All groups > select your emergency access group > Overview > Object Id |
Permissions to grant
Policy.Read.AllApplicationadmin consent requiredEnumerates all Conditional Access policies, policy templates, named locations and authentication strength policies so the skill can snapshot the tenant before making changes and detect overlapping policies.
โณ Policy.Read.ConditionalAccess covers reading Conditional Access policies and running What If, but not named locations, templates or authentication strength policies. Use it if the skill is being run in audit-only mode.
Policy.ReadWrite.ConditionalAccessApplicationadmin consent requiredCreates policies in report-only state, patches state and displayName during promotion or rollback, and deletes a policy that fails the break-glass What If gate.
โณ Omit entirely for read-only auditing runs. There is no narrower write scope for Conditional Access; when a human operator is present, prefer delegated access via the Conditional Access Administrator role instead of an app-only secret.
AuditLog.Read.AllApplicationadmin consent requiredReads auditLogs/signIns to measure the real-world impact of a report-only policy, and auditLogs/directoryAudits to confirm the policy change was recorded under the expected actor.
โณ No narrower scope exists for sign-in logs. Note that appliedConditionalAccessPolicies is returned only to callers that also hold a Conditional Access read permission.
Group.Read.AllApplicationadmin consent requiredReads the break-glass exclusion group and its membership so the preflight can assert the group exists, has at least two members, and that all members are enabled.
โณ GroupMember.Read.All is sufficient if the skill only needs membership and not group properties.
User.Read.AllApplicationadmin consent requiredResolves user principal names to object IDs for What If personas and checks accountEnabled on break-glass accounts.
โณ No narrower application scope exists for reading arbitrary users in the directory.
RoleManagement.Read.DirectoryApplicationadmin consent requiredReads directory role membership to confirm at least one break-glass account actually holds Global Administrator, and to resolve role template IDs used in policy include/exclude conditions.
โณ RoleManagement.Read.All is broader and covers other RBAC providers; prefer the Directory-scoped variant.
Application.Read.AllApplicationadmin consent requiredResolves the application GUIDs used in includeApplications and excludeApplications to human-readable names so the impact report is legible.
โณ Optional. Skip it and report raw application GUIDs if the operator will not grant it.
Step by step
- 1
Create the break-glass emergency access accounts and group first
Create at least two cloud-only accounts on the *.onmicrosoft.com domain, assign them the Global Administrator role permanently (not via PIM eligibility), register a FIDO2 passkey or certificate-based credential on each, store the credentials offline, and place both accounts in a dedicated security group named for example 'CA-BreakGlass-Exclusions'. Copy the group's Object Id. Do this before any policy work.
Open in the console / docs โ - 2
Register the application
Entra admin center > Entra ID > App registrations > New registration. Name it e.g. 'ca-policy-engineer'. Under Supported account types choose 'Accounts in this organizational directory only (Single tenant)'. Leave Redirect URI empty (app-only flow). Select Register, then copy Application (client) ID and Directory (tenant) ID from the Overview blade.
Open in the console / docs โ - 3
Add the Microsoft Graph application permissions
On the app > Manage > API permissions > Add a permission > Microsoft Graph > Application permissions. Search for and check each of: Policy.Read.All, Policy.ReadWrite.ConditionalAccess, AuditLog.Read.All, Group.Read.All, User.Read.All, RoleManagement.Read.Directory, Application.Read.All. Select Add permissions. Remove the default delegated User.Read entry to keep the app surface minimal.
Open in the console / docs โ - 4
Grant admin consent
On the same API permissions blade select 'Grant admin consent for <tenant name>' and confirm. Every permission row must show Status = 'Granted for <tenant name>' with a green check. Without this, token acquisition succeeds but every Graph call returns 403 Authorization_RequestDenied.
Open in the console / docs โ - 5
Create a client secret or, preferably, upload a certificate
App > Certificates & secrets > Client secrets > New client secret. Set Description and an Expires value of 6 months or less. Copy the Value column immediately - it is masked after you leave the blade. For production, use Certificates > Upload certificate with an X.509 public key (.cer/.pem/.crt) and authenticate with a signed client assertion instead of a secret.
Open in the console / docs โ - 6
Exclude the break-glass group from every existing enforcing policy
Entra admin center > Entra ID > Conditional Access > Policies (older tenants show this under Protection > Conditional Access). Open each policy whose State is 'On' and whose Grant control blocks access or requires MFA or a compliant device. Under Assignments > Users > Exclude > Groups, add the break-glass group, then Save. The skill's preflight refuses to run until this is true for every enforcing policy.
Open in the console / docs โ - 7
Confirm the app itself is not blocked by Conditional Access
Single-tenant service principals can be targeted by Conditional Access. If the tenant has such a policy, exclude this app registration or the client credentials call fails with AADSTS53003. Under Entra ID > Conditional Access > Policies, check every policy whose Assignments > Users or workload identities > 'What does this policy apply to' is set to 'Workload identities'. Note that adding a service principal to a group does not scope a policy to it; it must be assigned directly.
Open in the console / docs โ - 8
Export the environment variables and verify
Set AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET and ENTRA_BREAKGLASS_GROUP_ID in the environment or secret store the agent reads. Then run the verify command below; it must return HTTP 200 and a JSON body containing a 'value' array of policies.
Open in the console / docs โ
Check it worked
TOKEN=$(curl -s -X POST "https://login.microsoftonline.com/$AZURE_TENANT_ID/oauth2/v2.0/token" -d "client_id=$AZURE_CLIENT_ID" -d "client_secret=$AZURE_CLIENT_SECRET" -d 'grant_type=client_credentials' -d 'scope=https://graph.microsoft.com/.default' | python -c 'import sys,json;print(json.load(sys.stdin)["access_token"])') && curl -s -o /dev/null -w '%{http_code}\n' -H "Authorization: Bearer $TOKEN" 'https://graph.microsoft.com/v1.0/identity/conditionalAccess/policies?$select=id,displayName,state'If you hit an error
403 Forbidden - {"code":"Authorization_RequestDenied","message":"Insufficient privileges to complete the operation."}Cause: The application permission was added to the app registration but admin consent was never granted, or the wrong permission type (Delegated instead of Application) was selected.
Fix: Entra admin center > Entra ID > App registrations > your app > API permissions. Confirm each row's Type column reads 'Application', then select 'Grant admin consent for <tenant>'. Acquire a fresh token afterwards - existing tokens do not pick up new consent.
AADSTS7000215: Invalid client secret provided. Ensure the secret being sent in the request is the client secret value, not the client secret ID, for a secret added to app '<guid>'.Cause: The Secret ID GUID was copied instead of the secret Value, or the secret has expired.
Fix: Create a new client secret under Certificates & secrets > Client secrets and copy the Value column immediately. Update AZURE_CLIENT_SECRET.
AADSTS700016: Application with identifier '<guid>' was not found in the directory '<tenant>'.Cause: AZURE_CLIENT_ID and AZURE_TENANT_ID belong to different tenants, or the app was registered in a personal directory rather than the target tenant.
Fix: Re-copy both values from the same app's Overview blade in the target tenant. If the app is multi-tenant, provision its service principal in the target tenant first.
AADSTS65001: The user or administrator has not consented to use the application with ID '<guid>'. Send an interactive authorization request for this user and resource.Cause: A delegated flow was used against an app whose permissions are configured for app-only, or consent was granted for a different set of scopes than requested.
Fix: For app-only use, request scope=https://graph.microsoft.com/.default with grant_type=client_credentials, and grant admin consent on the API permissions blade. Do not request individual scope strings in the client credentials flow.
AADSTS53003: Access has been blocked by Conditional Access policies. The access policy does not allow token issuance.Cause: A Conditional Access policy targeting workload identities, or a location/device policy, is blocking the token request for this app or the signed-in administrator.
Fix: Exclude the app's service principal from the workload identity policy, or sign in from a permitted location and device. Use a break-glass account to make the change if the administrator is also blocked.
403 Forbidden - "Neither tenant is B2C or tenant doesn't have premium license"Cause: GET /auditLogs/signIns requires a Microsoft Entra ID P1 or P2 license on the tenant. The Graph permission alone is not enough.
Fix: Assign a premium license to the tenant. Until then, skip the report-only impact measurement step and rely on What If evaluation only - and say so explicitly in the report.
Sign-in results return "result": "unknownFutureValue" instead of reportOnlySuccess / reportOnlyFailureCause: The reportOnly* members of appliedConditionalAccessPolicyResult are part of an evolvable enumeration and are hidden unless the caller opts in.
Fix: Add the request header Prefer: include-unknown-enum-members to the GET /auditLogs/signIns call.
400 Bad Request - {"code":"Request_BadRequest","message":"Invalid value specified for property 'state' of resource 'ConditionalAccessPolicy'."}Cause: The state value was misspelled or cased wrongly. The only legal values are enabled, disabled and enabledForReportingButNotEnforced - 'reportOnly' is the portal label, not the API value.
Fix: Send "state": "enabledForReportingButNotEnforced" exactly. Also strip read-only properties (id, createdDateTime, modifiedDateTime) from PATCH bodies, which trigger the same 400.
429 Too Many Requests with a Retry-After response headerCause: Microsoft Graph throttling, usually from paging the sign-in logs too aggressively during impact measurement.
Fix: Honour the Retry-After header value in seconds, use $top=1000 to reduce page count, and narrow the createdDateTime filter window.
GET /identity/conditionalAccess/policies returns 403 even though the policy was created successfullyCause: The app holds Policy.ReadWrite.ConditionalAccess but not Policy.Read.All. The write scope permits creating policies without granting the ability to read the full policy set back.
Fix: Add Policy.Read.All as an application permission and grant admin consent.
Official documentation: https://learn.microsoft.com/en-us/graph/api/resources/conditionalaccesspolicy?view=graph-rest-1.0 โ
API operations this skill uses (17)
Every call the skill makes, linked to its official reference page.
| Operation | Call | Permission | Ref |
|---|---|---|---|
| Acquire app-only access token Obtains the bearer token used for every subsequent Microsoft Graph call in this skill. | POST https://login.microsoftonline.com/{tenant}/oauth2/v2.0/token (grant_type=client_credentials, scope=https://graph.microsoft.com/.default) | None - authenticates with client_id plus client_secret or a signed client assertion | docs โ |
| List Conditional Access policies Snapshots every existing policy before any mutation, detects overlapping conditions, and verifies that enforcing policies already exclude the break-glass group. | GET /v1.0/identity/conditionalAccess/policies | Policy.Read.All | docs โ |
| Get a single Conditional Access policy Re-reads a specific policy after creation or promotion to confirm the state and condition set actually stored by the service. | GET /v1.0/identity/conditionalAccess/policies/{policyId} | Policy.Read.All | docs โ |
| Create a policy in report-only state Stages the new policy with "state": "enabledForReportingButNotEnforced" and the break-glass group in excludeGroups. | POST /v1.0/identity/conditionalAccess/policies | Policy.ReadWrite.ConditionalAccess | docs โ |
| Promote or roll back a policy state Flips state to enabled after human approval, and is the rollback path back to enabledForReportingButNotEnforced if enforcement causes unexpected failures. | PATCH /v1.0/identity/conditionalAccess/policies/{policyId} | Policy.ReadWrite.ConditionalAccess | docs โ |
| Delete a failed staged policy Removes a just-created report-only policy that failed the break-glass What If gate, so no defective policy is left in the tenant. | DELETE /v1.0/identity/conditionalAccess/policies/{policyId} | Policy.ReadWrite.ConditionalAccess | docs โ |
| What If sign-in evaluation Simulates a sign-in for each persona and returns whatIfAnalysisResult objects with policyApplies and analysisReasons, proving break-glass accounts are not caught by the new policy. | POST /v1.0/identity/conditionalAccess/evaluate | Policy.Read.ConditionalAccess | docs โ |
| List Conditional Access policy templates Starts a new policy from a Microsoft baseline condition set instead of hand-authoring conditionalAccessConditionSet from scratch. | GET /v1.0/identity/conditionalAccess/templates?$select=name,description,id,scenarios | Policy.Read.All | docs โ |
| List named locations Resolves the location IDs referenced in policy conditions and checks whether a trusted-IP location already exists before creating a duplicate. | GET /v1.0/identity/conditionalAccess/namedLocations | Policy.Read.All | docs โ |
| Create an IP named location Defines the ipNamedLocation or countryNamedLocation a location-based policy will reference, before the policy that consumes it is created. | POST /v1.0/identity/conditionalAccess/namedLocations | Policy.ReadWrite.ConditionalAccess | docs โ |
| List authentication strength policies Resolves the ID of a built-in or custom authentication strength (for example phishing-resistant MFA) to place in grantControls.authenticationStrength. | GET /v1.0/policies/authenticationStrengthPolicies | Policy.Read.All | docs โ |
| Read sign-in logs for report-only impact Aggregates appliedConditionalAccessPolicies per sign-in to count reportOnlyFailure events and name the users who would be blocked on enforcement. | GET /v1.0/auditLogs/signIns?$filter=createdDateTime ge {isoTimestamp}&$top=1000 (header: Prefer: include-unknown-enum-members) | AuditLog.Read.All plus a Conditional Access read permission for appliedConditionalAccessPolicies to be returned | docs โ |
| Read directory audit log for policy changes Confirms every policy create, update and delete performed by this skill was recorded under the expected service principal, for change-control evidence. | GET /v1.0/auditLogs/directoryAudits?$filter=activityDisplayName eq 'Update conditional access policy' | AuditLog.Read.All | docs โ |
| List break-glass group members Preflight assertion that the emergency access group holds at least two enabled accounts before any policy is written. | GET /v1.0/groups/{groupId}/members?$select=id,userPrincipalName,accountEnabled | Group.Read.All | docs โ |
| List Global Administrator role members Confirms at least one break-glass account actually holds Global Administrator, and enumerates the admin population a privileged-access policy will target. | GET /v1.0/directoryRoles(roleTemplateId='62e90394-69f5-4237-9190-012177145e10')/members?$select=id,userPrincipalName | RoleManagement.Read.Directory | docs โ |
| Resolve a user for a What If persona Converts an operator-supplied UPN into the objectId that the evaluate action's signInIdentity.userId property requires. | GET /v1.0/users/{userPrincipalName}?$select=id,userPrincipalName,accountEnabled | User.Read.All | docs โ |
| Resolve a target application by appId Turns the GUIDs in includeApplications and in sign-in log rows into readable application names for the impact report. | GET /v1.0/servicePrincipals?$filter=appId eq '{appId}'&$select=id,appId,displayName | Application.Read.All | docs โ |