Skip to main content
Version: Latest

16. Operational Notes

16.1 Deployment checklist

  • Shumoul.MultiTenancyApi.Host and Shumoul.Api are configured with the same JwtSettings:key.
  • OnboardingApplyBridgeSettings:InternalApiKey (MultiTenancy host) exactly matches InternalServiceAuthSettings:OnboardingApplyApiKey (BackEnd host).
  • OnboardingApplyBridgeSettings:Enabled is true and BackEndBaseUrl points at the correct BackEnd environment (dev bridge settings must never point at production, and vice versa).
  • Both hosts trust each other's TLS certificate in whatever environment they're deployed to (a self-signed/untrusted dev certificate on either side will surface as an opaque SSL handshake failure from the bridge call, not a clear error).
  • The default survey (default-business-setup) seeds successfully on Shumoul.MultiTenancyApi.Host startup — check for "Seeded the onboarding survey." in the startup log.

16.2 Diagnosing a failed apply

  1. Check the MultiTenancy host's logs for "Onboarding apply bridge: calling Shumoul.BackEnd" and the matching "...HTTP call to Shumoul.BackEnd failed" or "...reported failure" line — both are logged with the same correlation id.
  2. If the failure is an SSL/HttpRequestException — check the BackEnd host's certificate is trusted from wherever the MultiTenancy host runs.
  3. If the failure message is "Tenant could not be resolved." — confirm the internal bridge is actually sending a TenantId header matching the request body's tenantId (this exact bug existed once — see Phase History — and would recur if the header-forwarding code regressed).
  4. If the failure is 401/403 from the BackEnd host — the two InternalApiKey/OnboardingApplyApiKey values have drifted apart; re-sync them.
  5. Once the bridge call succeeds but individual settings groups end up in ignoredGroups unexpectedly — check Chapter 11 for the exact whitelist; a rule referencing an unsupported group or property name will always be silently ignored, never a hard failure.

16.3 Monitoring signals worth alerting on

  • A sustained increase in apply calls returning succeeded: false with bridge-originated errors (as opposed to expected business-rule failures like "already applied") — usually indicates a configuration drift between the two hosts (see checklist above).
  • Any request to GET /api/Saas/SharedDbConnectionStrings/GetActive (or any of that controller's other actions) that does not carry a valid Authorization header should be a 401 today — a 200 response to that route from any log source is a signal something has regressed the Phase 1.5 fix.

16.4 Historical credential exposure risk

SharedDbConnectionStringsController.GetActiveListAsync() was reachable anonymously (no token, no permission check) prior to the Phase 1.5 fix, and returned raw ServerName, UserName, Password, and ConnectionString values for every active shared database connection string on the platform. This was found during this feature's authorization review, not as a result of a reported incident — there is no evidence either way of it having been exploited.

As a precaution, operators should:

  • Rotate the credentials for any shared database connection strings that existed before this fix was deployed, out of caution.
  • Review API/web-server access logs for anonymous GET requests to /api/Saas/SharedDbConnectionStrings/GetActive (or its sibling actions), especially from IP ranges outside expected internal/admin access.
  • Check whether any monitoring, logging, or APM tooling captured full response bodies for that route while it was unprotected, and purge/redact those captures if so.

These actions are not performed automatically by this documentation or by the code fix — they are operational follow-ups for whoever owns credential rotation and log review in your environment.

16.5 Rollback

There is no destructive migration associated with this feature — the 6 onboarding tables are additive, and a rollback of the application code alone (without a database rollback) leaves the tables in place, unused, which is safe. No tenant's existing AppSettings values are altered by rolling back — only future apply calls would stop working.