16. Operational Notes
16.1 Deployment checklist
-
Shumoul.MultiTenancyApi.HostandShumoul.Apiare configured with the sameJwtSettings:key. -
OnboardingApplyBridgeSettings:InternalApiKey(MultiTenancy host) exactly matchesInternalServiceAuthSettings:OnboardingApplyApiKey(BackEnd host). -
OnboardingApplyBridgeSettings:EnabledistrueandBackEndBaseUrlpoints at the correct BackEnd environment (dev bridge settings must never point at production, and vice versa). - Both hosts trust each other's TLS certificate in whatever environment they're deployed to (a self-signed/untrusted dev certificate on either side will surface as an opaque SSL handshake failure from the bridge call, not a clear error).
- The default survey (
default-business-setup) seeds successfully onShumoul.MultiTenancyApi.Hoststartup — check for"Seeded the onboarding survey."in the startup log.
16.2 Diagnosing a failed apply
- Check the MultiTenancy host's logs for
"Onboarding apply bridge: calling Shumoul.BackEnd"and the matching"...HTTP call to Shumoul.BackEnd failed"or"...reported failure"line — both are logged with the same correlation id. - If the failure is an
SSL/HttpRequestException— check the BackEnd host's certificate is trusted from wherever the MultiTenancy host runs. - If the failure message is
"Tenant could not be resolved."— confirm the internal bridge is actually sending aTenantIdheader matching the request body'stenantId(this exact bug existed once — see Phase History — and would recur if the header-forwarding code regressed). - If the failure is
401/403from the BackEnd host — the twoInternalApiKey/OnboardingApplyApiKeyvalues have drifted apart; re-sync them. - Once the bridge call succeeds but individual settings groups end up in
ignoredGroupsunexpectedly — check Chapter 11 for the exact whitelist; a rule referencing an unsupported group or property name will always be silently ignored, never a hard failure.
16.3 Monitoring signals worth alerting on
- A sustained increase in
applycalls returningsucceeded: falsewith bridge-originated errors (as opposed to expected business-rule failures like "already applied") — usually indicates a configuration drift between the two hosts (see checklist above). - Any request to
GET /api/Saas/SharedDbConnectionStrings/GetActive(or any of that controller's other actions) that does not carry a validAuthorizationheader should be a401today — a200response to that route from any log source is a signal something has regressed the Phase 1.5 fix.
16.4 Historical credential exposure risk
SharedDbConnectionStringsController.GetActiveListAsync() was reachable anonymously (no token, no
permission check) prior to the Phase 1.5 fix, and returned raw ServerName, UserName, Password, and
ConnectionString values for every active shared database connection string on the platform. This was found
during this feature's authorization review, not as a result of a reported incident — there is no evidence
either way of it having been exploited.
As a precaution, operators should:
- Rotate the credentials for any shared database connection strings that existed before this fix was deployed, out of caution.
- Review API/web-server access logs for anonymous
GETrequests to/api/Saas/SharedDbConnectionStrings/GetActive(or its sibling actions), especially from IP ranges outside expected internal/admin access. - Check whether any monitoring, logging, or APM tooling captured full response bodies for that route while it was unprotected, and purge/redact those captures if so.
These actions are not performed automatically by this documentation or by the code fix — they are operational follow-ups for whoever owns credential rotation and log review in your environment.
16.5 Rollback
There is no destructive migration associated with this feature — the 6 onboarding tables are additive, and a
rollback of the application code alone (without a database rollback) leaves the tables in place, unused,
which is safe. No tenant's existing AppSettings values are altered by rolling back — only future apply
calls would stop working.
