21. Onboarding Apply Bridge — Local Setup & Live Verification

This chapter closes a specific gap in Chapter 10 (wire protocol), Chapter 12 (config key names), and Chapter 16 (deployment checklist / troubleshooting): none of them previously said, in one place, "here is exactly how to turn the bridge on for a local dotnet run and confirm it really writes to a tenant's AppSettings." This chapter is that walkthrough, plus the literal environment-variable names and a live reproduction of the exact 501 scenario reported against a Shumoul Starter / مطعم / كوفي tenant.

Not to be confused with: the separate Tenant Operational Defaults phase, which addresses why WarehouseId/CashAccountId/SalesInvoiceTemplateId never got set even once this bridge is enabled — a different root cause (missing operational records + a deliberate Guid-block in this bridge's own patch engine, not a bridge-configuration issue) with its own dedicated apply bridge.

21.1 Is the reported 501 a bug?

No. It is the documented, intentional behavior of HttpOnboardingTenantSettingsApplierShould's disabled-bridge short-circuit (Chapter 10 §10.4, Chapter 12 §12.1) — confirmed again in this session by reproducing the exact scenario end-to-end:

1. Registered a fresh dev tenant on Shumoul Starter, answered the survey as a restaurant/cafe (POS, tables, kitchen, modifiers, VAT), and generated a recommendation.
2. Got the exact reported response: profile RestaurantCafe (مطعم / كوفي), confidence 95, a settingsPatch covering CashierAppsSettings/ProductSettings/DiscountSettings/TaxSettings, and the same four warnings quoted in the bug report (see §21.5).
3. Called apply with the bridge left at its committed, production-safe default (OnboardingApplyBridgeSettings:Enabled = false) — got the same 501, same message, byte-for-byte.
4. Restarted the local process with the bridge enabled and a matching dev-only internal key on both sides — the identical apply call returned 200, the session moved to Applied, and a TenantOnboardingApplyLogs row was written with the real applied patch.

No source change was required to make this work — the code already does exactly what Chapters 10/12 document. The only thing "not configured" was the local process's own environment.

21.2 Why Shumoul.Api calling itself is the correct local setup

Shumoul.Api (Shumoul.BackEnd/Shumoul.Api) hosts both stacks in a single process:

• It calls services.AddMultiTenancy(config) / app.UseMultiTenancy(config) (Shumoul.Infrastructure/Extensions/ServiceCollectionExtensions.cs, Shumoul.Api/Program.cs) — this registers every Shumoul.Framework.MultiTenancy.Api controller and service, including OnboardingController and HttpOnboardingTenantSettingsApplier.
• It also hosts OnboardingInternalController (Shumoul.Api/Controllers/Internal/) directly.

So on a developer machine there is only one running process, listening on one port (https://localhost:5001 per Shumoul.Api/Properties/launchSettings.json), and the bridge's outbound call is the process calling its own loopback address. This is not a shortcut or a test-only trick — it is exactly how staging/production also work whenever both stacks are deployed as the same Shumoul.Api instance (see §21.4). If a future topology splits the two into separate deployed services, only BackEndBaseUrl changes — no code changes.

Practical consequence: OnboardingApplyBridgeSettings:BackEndBaseUrl for local dev is simply the same URL the dev server itself listens on — already the committed default in Shumoul.Api/appsettings.Development.json:

"OnboardingApplyBridgeSettings": {
  "Enabled": false,
  "BackEndBaseUrl": "https://localhost:5001",
  "ApplySettingsPath": "/api/internal/onboarding/apply-settings-patch",
  "InternalApiKey": "**FROM_ENVIRONMENT**",
  "TimeoutSeconds": 120
}

Only Enabled and InternalApiKey need an override to turn the bridge on locally — never edit this file to put a real key in it.

21.3 Turning the bridge on locally (dev-only, never commit)

Set three process/session-scoped environment variables before dotnet run (PowerShell shown; the double-underscore is the standard ASP.NET Core nested-config-key binding):

$devKey = [guid]::NewGuid().ToString("N")     # any dev-only random value — never reuse a real key
$env:ASPNETCORE_ENVIRONMENT = "Development"
$env:OnboardingApplyBridgeSettings__Enabled = "true"
$env:OnboardingApplyBridgeSettings__InternalApiKey = $devKey
$env:InternalServiceAuthSettings__OnboardingApplyApiKey = $devKey   # MUST be the same value — shared secret

Then dotnet run --project Shumoul.Api. Both settings resolve into the same running process, so the two __InternalApiKey / __OnboardingApplyApiKey values above are really just one shared secret used twice — the loopback call authenticates to itself.

Never put a real value in appsettings.Development.json — the committed template's "**FROM_ENVIRONMENT**" placeholders are intentional and match the same convention already used for every other internal-service key in this codebase (AccountingChartTemplateApplyApiKey, TenantEntitlementProvisioningApiKey, etc. — see Chapter 12 §12.3).

21.4 Staging / production configuration

Same three settings, on the real deployed values:

Setting	Local dev	Staging/Production
OnboardingApplyBridgeSettings:Enabled	true (only after explicit local setup)	true — required for apply to ever succeed
OnboardingApplyBridgeSettings:BackEndBaseUrl	https://localhost:5001 (loopback)	The real Shumoul.Api internal/public URL — never localhost
OnboardingApplyBridgeSettings:InternalApiKey	any dev-only random value	A real, generated secret, stored in the server's secret store / environment, never in git
InternalServiceAuthSettings:OnboardingApplyApiKey	same value as above	Must exactly equal the value above — this is one shared secret, not two independent keys
OnboardingApplyBridgeSettings:TimeoutSeconds	default (60, or 120 per the committed dev template)	Keep generous — see §21.6 on first-call latency

Deployment checklist (adds to, does not replace, Chapter 16 §16.1):

• Enabled = true on whichever host actually serves the tenant-facing onboarding UI.
• BackEndBaseUrl points at the correct environment's own Shumoul.Api — a staging bridge pointed at production (or vice versa) would apply settings to the wrong tenant's database.
• Both InternalApiKey values are set from the platform's secret store (Azure Key Vault, AWS Secrets Manager, server-level environment variables, etc.), never committed, and rotated the same way any other internal service key is rotated.
• If BackEndBaseUrl uses https:// with a self-signed or otherwise untrusted certificate, confirm the calling host actually trusts it — this fails as an opaque SSL handshake error (§21.7), not a clean 401/403.
• After deploy, run one safe read: GET api/v1/onboarding/status for a single, clearly-marked test tenant — never mutate production AppSettings as part of a deployment smoke test without a separate, explicit approval.

Rollback

Setting OnboardingApplyBridgeSettings:Enabled = false (or removing the internal key) instantly reverts to the safe, documented 501 deferred-apply behavior — no code deploy, no migration, and no tenant's already-applied AppSettings are touched. This is the same safety property already documented in Chapter 16 §16.5.

21.5 The four warnings in the reported scenario — none of them are bugs

Warning (as shown in the reported scenario)	Category	Meaning	Blocks apply?
"WarehouseId was not set because no default warehouse was created in Phase 1."	Missing master data	WarehouseId is a Guid reference property — the engine never assigns entity references automatically (Chapter 11 §11.4)	No
"CashAccountId was not set because no default cash account was created in Phase 1."	Missing master data	Same Guid-reference rule, different property	No
"SalesInvoiceTemplateId was not set because automatic invoice template creation is outside Phase 1."	Missing master data	Same rule; invoice-template creation is explicitly out of scope	No
"Settings group LoyaltySettings is not included in the tenant's active subscription and was removed from the patch."	Subscription entitlement	Shumoul Starter does not include LoyaltySettings in its entitled groups (see subscription-packages-and-entitlements, Chapter 5) — removed defensively before the patch is even sent, then re-checked again on the BackEnd side (Chapter 11 §11.2, "independently re-checked")	No

All four are warnings, not errors — apply still returned succeeded: true with the remaining allowed groups (CashierAppsSettings, ProductSettings, DiscountSettings, TaxSettings) applied. A correct UI must render these as non-blocking notices ("here's what we couldn't set up automatically, and why"), never as a failure state — this matches how the existing recommendation response already separates warnings from a hard failure (Chapter 8 §8.4). No frontend change is required by this finding; it is confirmation that the existing warning/error separation in the response contract is sufficient.

Master-data warnings (Warehouse/CashAccount/SalesInvoiceTemplate) are resolved by a later phase creating that default master data (see the Tenant Starter Kit & Product Catalog guide's product/account templates) — not by this bridge. Entitlement warnings (LoyaltySettings) are resolved only by the tenant upgrading to a package that includes that settings group — never by changing onboarding code.

21.6 A latency note from this session's live run

The first apply call after enabling the bridge took ~30 seconds end-to-end (the internal endpoint itself logged responded 200 in 28041.98 ms). This is not a bridge defect — the bulk of the time is per-group IAppSettingService.GetAppSetting<T>(refreshCache: true) cache-refresh cost across four settings groups, ProvisionTenantEntitlementsAsync's post-apply best-effort work, and general first-call JIT/connection warm-up on a freshly-provisioned dev tenant database — not the HTTP hop itself. Keep TimeoutSeconds generous (the committed default of 120 is appropriate) rather than tightening it, and don't mistake a slow-but-successful call for a hung bridge.

21.7 Troubleshooting quick-reference

Symptom	Likely cause	Where to look
501 — "bridge is not configured on this host"	Enabled=false, missing BackEndBaseUrl, or missing InternalApiKey on the calling side	§21.3 / §21.4; HttpOnboardingTenantSettingsApplier.ApplyAsync's short-circuit check
502 — generic "Failed to apply onboarding settings"	The bridge reached Shumoul.Api but the internal call itself failed or timed out	MultiTenancy host log: "Onboarding apply bridge: HTTP call to Shumoul.BackEnd failed"
401 from the internal endpoint	X-Shumoul-Internal-Key header missing entirely	Confirm the calling host actually has a non-empty InternalApiKey configured
403 from the internal endpoint	Key present but doesn't match	The two InternalApiKey/OnboardingApplyApiKey values have drifted — re-sync them (§21.4)
"Tenant could not be resolved." (502)	TenantId header not forwarded, or forwarded value doesn't match the request body's tenantId	Chapter 16 §16.2 step 3 — this exact regression happened once before
Apply succeeds but a settings group is unexpectedly in ignoredGroups	Whitelist or entitlement filtering, not a bug	§21.5 above; Chapter 11
SSL handshake error, not a clean 401/403/501	Untrusted certificate on either host	Chapter 16 §16.1 checklist item on TLS trust

21.8 What this session did not need to change

No entity, controller, service, settings-patch rule, or subscription entitlement was modified. This chapter documents configuration and expected behavior only — the bridge, the internal endpoint, the whitelist, and the entitlement filter were all already correct.