1. Overview
1.1 What is the Background Jobs Framework?
The Shumoul Background Jobs Framework is a standalone, independently-versioned execution/runtime engine
— package family Shumoul.BackgroundJobs.* (repo Shumoul.Saas.BackgroundJobsFramework) — that provides
scheduling, cancellation-aware execution, tenant-scoped context, distributed locking, and failure
classification as generic capability, owned by no single business feature.
It does not own business logic, domain data, or a persistence layer of its own. It answers one question for any recurring or fire-and-forget unit of work in the ERP: how does this get scheduled, executed safely across a distributed deployment, tenant-scoped correctly, and reported on consistently — regardless of what the job actually does?
1.2 Why was it created?
Before this framework existed, recurring background execution in Shumoul.Saas.Api — Hangfire recurring
jobs, worker dispatch, retry, concurrency guarding — was implemented ad hoc, directly inside the ERP, with
no reusable runtime abstraction. Every new recurring job hand-rolled its own Hangfire registration, its own
concurrency assumptions, and its own exception-handling shape. Some of this code
(IBackgroundJobService/HangfireService, RecurringJobRegistry) was already business-independent in
spirit, but had never been extracted into its own versioned, independently-testable module.
1.3 Problems it solves
Before this initiative, every new recurring job meant re-solving the same problems from scratch: Hangfire registration boilerplate, ad hoc concurrency assumptions, inconsistent exception handling, and no shared testing pattern. That repeated cost is now paid once. All six production recurring jobs currently in scope — notification dead-letter cleanup, retry expiration, retry processing, campaign cleanup, campaign scheduling, and campaign workflow execution — run through the framework's pipeline today, each migrated with a cheap, proven rollback path and zero regressions.
A second, harder design question the initiative resolved: where does concurrency safety belong when a
generic runtime engine and a business domain both touch the same job? The answer, worked out for the
highest-risk migration (campaign-workflow-executor): duplicate-execution prevention belongs to whichever
domain owns the data being touched (the Notification Framework's own ClaimAsync atomic claim), never to the
generic runtime. See §9 — Distributed Lock for why a framework-level lock was
explicitly rejected as the fix.
1.4 Architecture philosophy
The framework follows the same Framework First principle as every Shumoul module: "is this capability generic enough that a second host or a second business feature would want it unmodified?" If yes, it is designed as an independent module, not built inside the ERP with an intention to extract later.
It also follows a distinct, explicitly-named companion principle: Module-First Design — a module's design must be validated against its own domain, not forced into the shape of whichever Golden Reference module came first. Copying the Notification Framework's adapter-placement rule verbatim would have been wrong for a runtime module — see §7 — Adapters § Two-Tier Adapter Model for the concrete divergence this produced.
What it shares with the Notification Framework: the same layering skeleton (Contracts → Abstractions → Core → Entry), the same "translate, delegate, return" thin-adapter rule, the same Level 1→2→3 certification bar, the same requirement for a frozen Ownership Matrix and Change Policy, and the same rule that a module never opens a database transaction it doesn't own.
What deliberately diverges: the Background Jobs Framework has no Persistence package — it owns only
runtime primitives, no domain data — and its infrastructure adapters (Hangfire, MultiTenancy accessor) ship
inside the framework's own Adapters package rather than living entirely in the ERP host, because they
wrap generic, host-agnostic infrastructure rather than ERP-specific data access. See
§7.2.
1.5 Golden Reference certification
The framework holds two separate, sequential certifications, both dated 2026-07-04:
| Certification | Phase | Status |
|---|---|---|
| Architecture certification | Phase 4 | Level 3 Certified — Golden Reference, designated the Golden Reference Module for the execution/runtime module shape (as distinct from the Notification Framework's data/delivery shape) |
| Productization certification | Phase 4.1 | LTS Certified |
The certification process required: Level 3 compliance, at least one production-integrated release, an audit against actual shipped code (not design documents) across 8 fixed areas (dependency, runtime/ownership boundary, adapter, package, anti-pattern, host-integration, test, and documentation audits), an explicit named designation for its module shape, and formal initiative closure.
Result: all Critical checklist items PASS; Major items PASS with documented deviation (the two-tier adapter model; a single consolidated Tests project rather than three); three sections marked N/A by design because the module owns no domain data.
A small number of documented, non-blocking gaps were found and are tracked openly rather than silently fixed
— see §20 — Appendix for the authoritative list (source
documents count these inconsistently as "two," "three," or "four" gaps depending on which document and
counting convention is used — this guide standardizes on the four-item AP-01–AP-04 list used by the
framework's own Roadmap and Anti-Patterns documents).
1.6 LTS status
LTS status is a documentation and process certification — it changes nothing about the runtime itself. Granted at Phase 4.1 (2026-07-04), immediately following the Phase 4 architecture certification, it makes five things explicit and binding:
- Frozen public surface — 36 Stable types, 0 Experimental, 0 Deprecated.
- Predictable semver contract — within the
1.xline, every Stable type keeps its signature, namespace, and behavior across MINOR/PATCH releases; only a MAJOR bump may break Stable surface, and only with an ADR plus a migration guide. - A documented, repeatable release process — though, honestly, this has not yet actually been exercised by a second engineer; every release to date has been executed by one person.
- A named support boundary —
.NET 9only (no multi-targeting),Hangfire.Core1.8.21,Shumoul.Framework.MultiTenancy1.0.77, with exactly one current consumer:Shumoul.Saas.Api. - A deprecation path with a minimum lifetime before removal of any Stable type.
All 5 packages release together at one synchronized version (currently 1.0.0). Maintenance is
reactive, not cadence-based: a new version is cut only for a real consumer need, a defect meeting the
certification bar, or a security/runtime advisory — never for version-number cosmetics.
1.7 Relationship with the Notification Framework
No package-level dependency exists in either direction — confirmed by repository-wide search on both sides.
What connects them is entirely at the ERP host: the six migrated jobs each have an ERP-owned adapter that
bridges the Background Jobs Framework's IBackgroundJob contract to the unchanged Notification Framework
worker/service that always did the actual work. See §14 — Job Migration Guide
for the full migration history, and §20 — Appendix for the one incomplete piece of this
relationship (a planned notification-dispatch adapter, ADR-006, that shipped as a permanent no-op stub).
1.8 Relationship with the ERP (Shumoul.Saas.Api)
The framework owns pipeline orchestration, execution wrapping, scheduling abstraction, tenant-context
scoping, the distributed-lock primitive, failure classification, and its own infrastructure adapters. The
ERP host owns every job-specific adapter, the Hangfire-callable bridges, recurring-job registration and
cutover, DI composition order, the Hangfire Dashboard's authorization and exposure (permanently, per
ADR-004), job-tracking/visibility persistence (UserJob, ImportJobHistory, ImportJobRowError —
permanently, per ADR-003), and all business logic for any job it hosts.
The ERP references the framework only via PackageReference — never ProjectReference — and the framework
has zero references to any ERP assembly. Exactly one consumer exists today: Shumoul.Saas.Api.
Shumoul.Saas.MultiTenancyApi is confirmed not a consumer.
1.9 Relationship with MultiTenancy
Two distinct things are involved here, and it matters not to conflate them:
The Shumoul.Framework.MultiTenancy package (v1.0.77) — genuinely used, but narrowly: the framework's
own BackgroundJobTenantResolverAdapter resolves tenant identity from IMultiTenantContextAccessor
(AsyncLocal-backed) rather than an HTTP-request-scoped mechanism, because an HTTP-scoped resolver "would be
meaningless inside a Hangfire worker." This is the only infrastructure package referenced anywhere in the
framework besides Hangfire.Core, and it is confined entirely to the Adapters package. See
§8 — Tenant Context.
The Shumoul.Saas.MultiTenancyApi repository/host — unrelated. It is confirmed to have zero integration
with the Background Jobs Framework; its own legacy notification code references old Shumoul.Notification.*
packages for an entirely separate, unconnected purpose.
All 6 currently migrated jobs run host-level (tenantId: null) by deliberate business-semantic design — they
operate across all tenants. Per-tenant scoping is proven at the framework level (dedicated E2E tenant-context
tests) but not yet exercised by a real, single-tenant production job — a completeness gap, not a readiness
gap.
1.10 Current final status
The initiative is formally closed, effective 2026-07-04, spanning Phases 0 through 4.1: "every original objective was met or explicitly, honestly deferred with a named reason." Closed does not mean frozen to extension — see §15 — Architecture Decisions § What "Closed" Permits for exactly what ordinary maintenance is still allowed without a new initiative, and what requires one.