UNPKG

@studyportals/sp-hs-misc

Version:

Miscellaneous code used in HouseStark's projects

1,262 lines (991 loc) 53.5 kB
# Campaigns Table Revamp & API Migration Plan This document describes how the `campaigns` table and related ServiceLayer-Private API endpoints will be changed, and what other services that **consume this API** must do to stay compatible. The core goals are: - Make all business-critical columns on `campaigns` mandatory and strongly typed. - Remove deprecated/unused columns from `campaigns`. - Treat the ServiceLayer `campaigns` endpoints as **read-only** (no inserts/updates/deletes through this API). - Preserve read behaviour for all dependent services with **no downtime**. - Delete all legacy data that does not conform to the new schema. - **Remove all Banner, BannerBox, Lead, LeadCampaign, and LeadCampaignStudy entities entirely** (deprecated functionality). - **Drop 17 related database tables** after code cleanup is deployed. > **Audience:** Developers of services that call ServiceLayer-Private campaign endpoints. --- ## AI Assistant Prompt Use this prompt when analyzing other repositories for required changes: > Here is a migration plan for ServiceLayer-Private campaigns API. Analyze this repository's code and identify all places that need to be updated according to Phase 1 requirements. Specifically look for: > > **ServiceLayer API Changes:** > - Any PUT/POST/DELETE calls to `/data/campaigns/` endpoints (ALL campaign writes are now blocked) > - Usage of `/data/campaigns/.../organisations/` or `/data/campaigns/.../countries/` sub-component endpoints for LINKING (REMOVED - use Campaign Management API for org linking) > - Usage of `countries` decorator data in campaign responses (REMOVED - country data no longer in responses) > - Usage of dropped fields listed in section 8 > - Usage of deprecated search parameters > - Code that assumes `crm_id` is a string > - Enums that include deprecated values (`advanced_nrbp`, `rbp_cpl`, `west`, `bne`, `global`, `act`) > > **Important distinction:** > - `organisations` decorator data **still available** in responses (read-only) > - `countries` decorator data **removed** from responses (deprecated feature) > - Organisation/country **linking operations** (PUT/DELETE) are removed from ServiceLayer > > **Legacy Endpoint Migration (see Section 10):** > - `/data/campaigns/all/` → Use `/data/campaigns/any/details/` > - `/data/campaigns/all/{id}/` → Use `/data/campaigns/any/details/{id}/` > - `/data/campaigns/details/` → Use `/data/campaigns/any/details/` > - `/data/campaigns/ids/` → Use `/data/campaigns/any/ids/` > - `/data/campaigns/search/` → Use `/data/campaigns/any/ids/` > - `/data/campaigns/sender/` → **REMOVED** (use `/any/details/` with `recipients` decorator) > - `/data/campaigns/all/{id}/banners/` → **REMOVED** (deprecated feature, no replacement) > - `/data/campaigns/any/details/{id}/banners/` → **REMOVED** (deprecated feature, no replacement) > - `/data/campaigns/all/{id}/lead-campaigns/` → **REMOVED** (deprecated feature, no replacement) > - `/data/campaigns/any/details/{id}/lead-campaigns/` → **REMOVED** (deprecated feature, no replacement) > - `/data/campaigns/all/{id}/organisations/` → **REMOVED** (use Campaign Management API) > - `/data/campaigns/all/{id}/countries/` → **REMOVED** (deprecated feature) > > **Package Updates Required:** > - Update `@studyportals/sp-millennium-falcon` to ^3.0.0 > - Update `@studyportals/sp-hs-misc` to ^3.0.5 (or ^3.0.1 minimum) > > **hs-misc Breaking Changes (see Section 11):** > - **For Node.js:** Imports from `bin-browser/` no longer work - use main entry point instead: `import { ... } from '@studyportals/sp-hs-misc'` > - **For browsers (v3.0.5+):** Browser build restored as ES modules - use `<script type="module">` with imports from `/bin-browser/` > - References to `externalNotes` on campaign objects — this property has been removed from `ICampaign` > - Usage of `CampaignType.ADVANCED_NRBP` or `CampaignType.RBP_CPL` — these enum values have been removed > > **Search patterns to use:** > ``` > /data/campaigns/all # legacy endpoint - migrate to /any/details/ > /data/campaigns/details # legacy endpoint - migrate to /any/details/ > /data/campaigns/ids # legacy endpoint - migrate to /any/ids/ > /data/campaigns/search # legacy endpoint - migrate to /any/ids/ > /data/campaigns/sender # legacy endpoint - REMOVED, no equivalent > sp-hs-misc/bin-browser # broken imports (Node.js) - use main entry point > externalNotes # removed field > external_notes # removed field (API response) > ADVANCED_NRBP # removed enum value > RBP_CPL # removed enum value > "/data/campaigns/" # check for POST/PUT/DELETE > /organisations/.*PUT # organisation linking removed (data still in responses) > /organisations/.*DELETE # organisation linking removed (data still in responses) > /countries/ # country linking AND data removed > campaign.countries # decorator removed - data no longer in responses > chain # chain decorator removed > banners # banner decorator and linking REMOVED > lead-campaigns # lead-campaign linking REMOVED > banner_ids # search parameter REMOVED > has_banner # search parameter REMOVED > lead_campaign_ids # search parameter REMOVED > automated # dropped field > tableau_reporting # dropped field > rating # dropped field > chain_id # dropped field > options # dropped field > adwords_start # dropped field > adwords_end # dropped field > capped_budget # search parameter REMOVED (budget always non-null) > ``` > > **Write operations status:** > - BLOCKED: ALL `PUT/POST/DELETE` on `/data/campaigns/` entity endpoints > - BLOCKED: Organisation linking (use Campaign Management API instead) > - BLOCKED: Country linking (deprecated, no longer supported) > - BLOCKED: Banner linking to campaigns (deprecated, no longer supported) > - BLOCKED: Lead-campaign linking to campaigns (deprecated, no longer supported) > - ALLOWED: `PUT/DELETE` on `/data/campaigns/any/details/{id}/customproducts/` > - ALLOWED: `PUT/DELETE` on `/data/campaigns/any/details/{id}/presetproducts/` > > **COMPLETELY REMOVED ENTITIES (see Section 12):** > - Banner entity and all `/data/banners/*` endpoints → **REMOVED** (no replacement) > - BannerBox entity and all `/data/banner-boxes/*` endpoints → **REMOVED** (no replacement) > - Lead entity and all `/data/leads/*` endpoints → **REMOVED** (no replacement) > - LeadCampaign entity and all `/data/lead-campaigns/*` endpoints → **REMOVED** (no replacement) > - LeadCampaignStudy entity and all `/data/lead-campaign-studies/*` endpoints → **REMOVED** (no replacement) > - Banner driver (`/banner/*`) → **REMOVED** (no replacement) > - LeadQualifier driver (`/lead-qualifier/*`) → **REMOVED** (no replacement) > - Publish lead routes (`/publish/lead-*`, `/publish/email/lead/*`) → **REMOVED** (no replacement) > > **Additional search patterns for removed entities:** > ``` > /data/banners/ # REMOVED - banner entity deleted > /data/banner-boxes/ # REMOVED - bannerbox entity deleted > /data/leads/ # REMOVED - lead entity deleted > /data/lead-campaigns/ # REMOVED - leadcampaign entity deleted > /data/lead-campaign-studies/ # REMOVED - leadcampaignstudy entity deleted > /banner/ # REMOVED - banner driver deleted > /lead-qualifier/ # REMOVED - leadqualifier driver deleted > /publish/lead- # REMOVED - publish lead routes deleted > /publish/email/lead/ # REMOVED - publish lead routes deleted > study.lead_campaigns # REMOVED - decorator deleted > has_lead_campaign # REMOVED - search parameter deleted > ``` --- ## 1. Schema Changes (MySQL) The `campaigns` table will be updated as follows: ### 1.1 Columns becoming NOT NULL / typed The following columns will become non-nullable and (re)typed as shown: ```sql ALTER TABLE campaigns MODIFY persist_gl_param_ref_link TINYINT(1) NOT NULL, MODIFY budget INT NOT NULL, MODIFY budget_total INT NOT NULL, MODIFY assistant_id INT NOT NULL, MODIFY handler_id INT NOT NULL, MODIFY crm_id INT NOT NULL, MODIFY custom_products_budget INT NOT NULL, MODIFY `type` ENUM('rbp','nrbp','hybrid_ebp','full_ebp','free_trial') NOT NULL, MODIFY region ENUM('other','uk','apac','emea','na','strp') NOT NULL, MODIFY currency ENUM('EUR','GBP','USD','AUD','CAD') NOT NULL, MODIFY campaign_type ENUM('custom_products','listing_products') NOT NULL, MODIFY start_date DATE NOT NULL, MODIFY end_date DATE NOT NULL; ``` **Important changes to note:** 1. **`crm_id`** changes from `VARCHAR(5)` to `INT NOT NULL` – this is a type change, not just a constraint change. 2. **`type` enum** loses these values: - `advanced_nrbp` (removed) - `rbp_cpl` (removed) 3. **`region` enum** loses these values: - `west` (removed) - `bne` (removed) - `global` (removed) - `act` (removed) 4. **`start_date` and `end_date`** change from nullable to `DATE NOT NULL`. **Important:** While the database stores these as DATE type, the ServiceLayer API **converts them to Unix timestamps** in responses using: ```sql UNIX_TIMESTAMP(c.start_date) AS start_date, UNIX_TIMESTAMP(c.end_date) AS end_date ``` This means consuming services will continue to receive integer Unix timestamps, not DATE strings. 5. Prior to this change, **all rows that do not satisfy these constraints will be deleted**. There will be no attempts to coerce or migrate invalid values – if your service relies on such rows, they will disappear. ### 1.2 Dropped columns The following columns will be removed from the `campaigns` table entirely: ```sql ALTER TABLE campaigns DROP COLUMN automated, DROP COLUMN tableau_reporting, DROP COLUMN is_scheduled, DROP COLUMN nullIfOld, DROP COLUMN automation_notes, DROP COLUMN options, DROP COLUMN external_notes, DROP COLUMN chain_id, DROP COLUMN rating, DROP COLUMN adwords_start, DROP COLUMN adwords_end; ``` After this migration: - These columns **no longer exist** in the database. - Any query or ORM mapping in your code that selects/references them must be removed or adapted. --- ## 2. ServiceLayer Campaign API Contract Changes ### 2.1 Campaign endpoints become read-only Historically, the ServiceLayer allowed creation, update and deletion of campaigns via endpoints such as: - `/data/campaigns/any/details/` - Legacy: `/data/campaigns/all/`, `/data/campaigns/details/`, `/data/campaigns/ids/`, `/data/campaigns/sender/` Going forward: - These endpoints will be treated as **read-only**. No new campaigns will be created, updated or deleted through ServiceLayer-Private. - The underlying `Campaign` entity in ServiceLayer will **not** call `INSERT`, `UPDATE` or `DELETE` on the `campaigns` table anymore. - Any attempt by client services to use `PUT`, `POST` or `DELETE` on campaign endpoints must be removed. **Sub-component linking changes:** | Operation | Status | Notes | |-----------|--------|-------| | Custom products linking | ✅ Still supported | Use `/data/campaigns/any/details/{id}/customproducts/` | | Preset products linking | ✅ Still supported | Use `/data/campaigns/any/details/{id}/presetproducts/` | | Banner linking | ❌ **REMOVED** | Deprecated feature, no longer supported | | Lead campaign linking | ❌ **REMOVED** | Deprecated feature, no longer supported | | Organisation linking | ❌ **REMOVED** | Use Campaign Management API instead | | Country linking | ❌ **REMOVED** | Deprecated feature, no longer supported | **Legacy endpoints removed:** All legacy campaign endpoints have been removed entirely (will return 404): - `/data/campaigns/all/` → Use `/data/campaigns/any/details/` - `/data/campaigns/details/` → Use `/data/campaigns/any/details/` - `/data/campaigns/ids/` → Use `/data/campaigns/any/ids/` - `/data/campaigns/search/` → Use `/data/campaigns/any/ids/` (was already an alias for `/ids/`, marked obsolete) - `/data/campaigns/sender/` → **Removed** (use `/any/details/` with `recipients` decorator) - All `/data/campaigns/all/{id}/...` sub-component endpoints → **Removed** **Action for consumers:** - Audit your usage of campaign endpoints. - If you are using `GET`, you are generally fine (subject to field removals below). - If you are using `PUT`, `POST` or `DELETE` on any `/data/campaigns/...` URL, you **must migrate away from this pattern**. Use your own write path or a new dedicated campaign service. - If you are linking banners to campaigns, **remove this functionality** (deprecated). - If you are linking lead-campaigns to campaigns, **remove this functionality** (deprecated). - If you are linking organisations to campaigns, **migrate to Campaign Management API**. - If you are linking countries to campaigns, **remove this functionality** (deprecated). - If you are using legacy `/all/` endpoints, **migrate to `/any/details/`**. ### 2.2 Organisation and Country Data Changes #### Organisation Data (Decorator Stays, Linking Removed) The `organisations` decorator **remains available** in campaign API responses. You can still read organisation data linked to campaigns via the decorator. **What changed:** - ✅ **READ** - Organisation data is still returned in campaign responses via the `organisations` decorator - ❌ **WRITE** - Organisation linking operations (PUT/DELETE on `/organisations/` sub-component) are removed **For organisation linking operations**, use the **Campaign Management API** instead of ServiceLayer. #### Country Data (Decorator and Linking Both Removed) The `countries` decorator is **removed** from campaign API responses. Country data will **no longer appear** in campaign responses. **What changed:** - ❌ **READ** - Country data is no longer returned in campaign responses - ❌ **WRITE** - Country linking operations are removed Country linking is a deprecated feature with no replacement. #### Chain Decorator (Removed) The `chain` decorator is removed because the `chain_id` column is dropped from the database. #### Banner Decorator and Linking (Removed) The `banners` decorator and all banner linking functionality have been **completely removed**: **What changed:** - ❌ **READ** - Banner data is no longer returned in campaign responses - ❌ **WRITE** - Banner linking operations (PUT/DELETE on `/banners/` sub-component) are removed - ❌ **SEARCH** - `banner_ids` and `has_banner` search parameters are removed The `campaigns_banners` linking table is deprecated. Banner functionality for campaigns should use custom products of type `banners` instead. #### Lead-Campaign Linking (Removed) Lead-campaign linking to campaigns has been **completely removed**: **What changed:** - ❌ **WRITE** - Lead-campaign linking operations (PUT/DELETE on `/lead-campaigns/` sub-component) are removed - ❌ **SEARCH** - `lead_campaign_ids` search parameter is removed The `lead_campaigns_campaigns` linking table is deprecated. **Summary of decorator changes:** | Decorator | Data in Response | Linking Operations | Notes | |-----------|------------------|-------------------|-------| | `organisations` | ✅ Still available | ❌ Removed | Use Campaign Management API for linking | | `banners` | ❌ Removed | ❌ Removed | Deprecated feature | | `countries` | ❌ Removed | ❌ Removed | Deprecated feature | | `chain` | ❌ Removed | N/A | `chain_id` column dropped | | `attached_organisations` | ✅ Still available | N/A | Different relationship, unaffected | ### 2.3 Fields removed from API responses Because the underlying columns are dropped, the following fields will **disappear** from campaign JSON responses (or be permanently `null` and then removed): - `automated` - `automated_over_performance` - `tableau_reporting` - `is_scheduled` - `nullIfOld` - `automation_notes` - `options` / `campaign_options` - `external_notes` - `chain_id` - `rating` - `adwords_start` - `adwords_end` This affects at least: - `/data/campaigns/any/details/` - `/data/campaigns/any/list/` - `/data/campaigns/any/export/` - `/data/campaigns/any/google-ads/` **Action for consumers:** - Search your code for these keys in JSON payloads from ServiceLayer campaigns. - Remove references, or replace them with alternative signals if your business logic still requires that information. - Remove any sorting/filtering logic that expects these fields to exist. ### 2.4 Fields with stricter semantics The following fields remain in responses but will now be **guaranteed non-null** and follow stricter domain rules: - `type` – now limited to `rbp`, `nrbp`, `hybrid_ebp`, `full_ebp`, `free_trial`. - `region` – now limited to `other`, `uk`, `apac`, `emea`, `na`, `strp`. - `currency` – now limited to `EUR`, `GBP`, `USD`, `AUD`, `CAD`. - `campaign_type` – now limited to `custom_products`, `listing_products`. - `start_date`, `end_date` – always present (no `null` values) and valid dates. - `budget`, `budget_total`, `assistant_id`, `handler_id`, `crm_id`, `custom_products_budget`, `persist_gl_param_ref_link` – always non-null numeric values. **Action for consumers:** - If your code assumed these fields might be `null` or empty, you can simplify it. However, you should: - Validate that your enums/typed models match the new value sets. - Remove any fallbacks that treat `null` as “unknown” for these fields. ### 2.4 Deletion of non-conforming campaigns As part of the migration, **all campaign rows that do not satisfy the new constraints will be deleted**. **Potential impact:** - If your service expects historical campaigns with incomplete/legacy data, those IDs may cease to exist. - Calls such as `/data/campaigns/any/details/{id}/` for such campaigns will return 404 (or simply no entry) after migration. **Action for consumers:** - Check whether you rely on very old campaigns or "draft" campaigns with partial data. - Adjust your code to: - Handle missing campaigns gracefully. - Not assume that *every* historic ID will always stay resolvable. --- ## 3. Impact on Related Entities and Endpoints Several other ServiceLayer entities and endpoints depend on data from `campaigns`. Examples include: - Decorators that attach campaign information to other entities (e.g. organisations, studies). - Linking tables such as `campaigns_banners`, `campaigns_countries`, `campaigns_products2`, `campaigns_entities_configs`, `lead_campaigns_campaigns`, `campaigns_recipients`. ### 3.1 Search/Filter Parameters Being Removed If you use any of these query parameters when calling campaign endpoints, they will **stop working**: | Parameter | Reason | |-----------|--------| | `automated` | Column dropped | | `automated_over_performance` | Column dropped | | `tableau_reporting` | Column dropped | | `has_adwords` | `adwords_start`/`adwords_end` columns dropped | | `ratings` | `rating` column dropped | | `options` / `no_options` | `options` column dropped | | `banner_ids` | Banner linking deprecated | | `has_banner` | Banner linking deprecated | | `lead_campaign_ids` | Lead campaign linking deprecated | | `capped_budget` | Budget is now always non-null, filter is meaningless | ### 3.2 Search/Filter Parameters Still Available These parameters will continue to work: - `types`, `regions`, `crm_id`, `statuses`, `is_running` - `organisation_ids`, `attached_organisation_ids`, `study_ids` - `assistant_ids`, `handler_ids`, `user_ids` - `exclude_from_campaign_lift` - `keywords`, `rbp` Key points: - Relationships (via linking tables) stay intact **only for campaigns that survive the data cleanup**. - Decorators and search maps that use dropped fields (`tableau_reporting`, `rating`, `adwords_*`, etc.) will be updated or removed in ServiceLayer. - Your service may see differences in filtered lists or counts if it relied on these filters indirectly. **Action for consumers:** - If you use endpoints that indirectly filter on `tableau_reporting`, `rating`, AdWords-related fields or other soon-to-be-dropped properties, expect those filters to stop having any effect. - If you use `capped_budget` parameter, remove it - budget is now always non-null so this filter is meaningless. - Review any business logic that interprets such filter results. --- ## 4. Required Actions for Dependent Services This section is the checklist you should follow if your service uses ServiceLayer campaigns. ### 4.1 Stop using ServiceLayer for campaign writes - [ ] Search your codebase for `"/data/campaigns/"` (including `any`, `all`, `details`, `sender`, etc.). - [ ] Ensure there are **no** `PUT`, `POST` or `DELETE` HTTP calls to campaign entity URLs. - [ ] If you find any write operations: - Migrate campaign creation/update/deletion to your own service or a new dedicated API that is not this ServiceLayer campaign entity. - Remove the old write calls and associated error handling. ### 4.2 Migrate organisation linking / Remove deprecated linking operations - [ ] **Organisation linking:** If you link organisations to campaigns via ServiceLayer: - Search for `/organisations/` in campaign endpoint paths (PUT/DELETE operations) - **Migrate linking operations to Campaign Management API** - Note: Organisation data is **still available** in responses via the `organisations` decorator (read-only) - [ ] **Country data:** If you use country data from campaign responses: - Search for `campaign.countries` in code that processes responses - **Remove this usage** - the `countries` decorator is removed, data no longer in responses - [ ] **Country linking:** If you link countries to campaigns via ServiceLayer: - Search for `/countries/` in campaign endpoint paths (PUT/DELETE operations) - **Remove this functionality** - country linking is deprecated and no longer supported - [ ] **Banner linking:** If you link banners to campaigns via ServiceLayer: - Search for `/banners/` in campaign endpoint paths (PUT/DELETE operations) - **Remove this functionality** - banner linking is deprecated and completely removed - The sub-component endpoint `/data/campaigns/any/details/{id}/banners/` no longer exists - [ ] **Lead-campaign linking:** If you link lead-campaigns to campaigns via ServiceLayer: - Search for `/lead-campaigns/` in campaign endpoint paths (PUT/DELETE operations) - **Remove this functionality** - lead-campaign linking is deprecated and completely removed - The sub-component endpoint `/data/campaigns/any/details/{id}/lead-campaigns/` no longer exists - [ ] **Legacy sub-component endpoints:** All `/data/campaigns/all/{id}/...` sub-component endpoints are **removed** ### 4.3 Remove use of dropped fields and decorators - [ ] Search for usage of the following JSON keys in your campaign-related code: - `automated` - `automated_over_performance` - `tableau_reporting` - `is_scheduled` - `nullIfOld` - `automation_notes` - `options` - `campaign_options` - `external_notes` - `chain_id` - `rating` - `adwords_start` - `adwords_end` - [ ] Search for usage of removed decorators: - `chain` (removed - chain_id column dropped) - `countries` (removed - deprecated feature, data no longer in responses) - `banners` (removed - deprecated feature, decorator and sub-component removed) - [ ] Note: `organisations` decorator **still available** (read-only data in responses) - [ ] Remove or replace them with alternative fields. - [ ] Remove any UI elements or reports that display these fields directly. - [ ] **Check for deprecated enum values:** - `type`: remove handling for `advanced_nrbp`, `rbp_cpl` - `region`: remove handling for `west`, `bne`, `global`, `act` ### 4.4 Align enums and required fields - [ ] Ensure your internal models for campaigns: - Treat `type`, `region`, `currency`, `campaign_type` as **closed enums** with the allowed values listed above. - Treat `start_date`, `end_date` as required fields. - Treat `budget`, `budget_total`, `assistant_id`, `handler_id`, `crm_id`, `custom_products_budget`, `persist_gl_param_ref_link` as required numeric fields. - [ ] Update validation and mapping code so that it: - Does not expect `null` values for these fields. - Fails fast on unexpected enum values (in case new values are added later). ### 4.4 Handle missing/removed campaigns - [ ] Review any logic that assumes a campaign ID from the past will always resolve. - [ ] Update code to handle cases where: - A previously-known campaign ID now returns 404 or is absent from results. - Counts or lists of historic campaigns shrink due to deletion of invalid/legacy rows. ### 4.5 Re-run and update tests - [ ] Update integration/end-to-end tests that rely on: - Deprecated fields in API responses. - Creating/updating/deleting campaigns through ServiceLayer. - [ ] Add tests that assert: - Your service does not attempt to write to `/data/campaigns/...` endpoints. - It can handle missing campaigns gracefully. --- ## 5. ServiceLayer Deployment Phases The ServiceLayer team will deploy changes in this order: > **Critical:** Data cleanup MUST happen before code deployment. Campaign decorators are present on many other entities (e.g., Product2, Study). If campaigns with invalid enum values (like deprecated `region` or `type` values) exist when the new code deploys, those endpoints will fail with validation errors. ### Phase 1: Data Cleanup (Database - FIRST) - Invalid campaign rows deleted (rows with deprecated `type` values like `advanced_nrbp`, `rbp_cpl`) - Invalid campaign rows deleted (rows with deprecated `region` values like `west`, `bne`, `global`, `act`) - Rows with NULL values in soon-to-be-required fields deleted - Linking table entries for deleted campaigns cleaned up - Deprecated linking tables truncated (`campaigns_banners`, `campaigns_countries`, `lead_campaigns_campaigns`) - **This ensures all remaining data is valid for the new schema and code** ### Phase 2: Code Deployment (ServiceLayer) - All code changes deployed in single release: - Write endpoints (`PUT`, `POST`, `DELETE`) disabled on `/data/campaigns/*` - Legacy endpoints removed (`/data/campaigns/all/`, `/data/campaigns/details/`, etc.) - Deprecated search parameters removed - Deprecated decorators removed (`banners`, `countries`, `chain`, `lead_campaigns`) - Banner, Lead, LeadCampaign entities completely removed - Deprecated properties removed from responses - **Your service should be ready for this before Phase 2 deploys** ### Phase 3: Schema Migration (Database - LAST) - Columns modified to `NOT NULL` - Deprecated columns dropped (`automated`, `chain_id`, etc.) - New enum constraints applied (reduced `type` and `region` values) - Deprecated tables dropped (17 tables listed in Section 12.5) --- ## 6. Migration Timeline for Dependent Services A suggested sequence for your service: 1. **Audit & refactor (NOW - before Phase 1):** - Stop using campaign write endpoints (`PUT`, `POST`, `DELETE`). - Remove dependencies on soon-to-be-dropped fields. - Remove usage of deprecated search parameters. 2. **Align models:** - Update enums to match new allowed values. - Treat all listed fields as required (non-null). - Make tests match the new expectations. 3. **Deploy your changes:** - Deploy before ServiceLayer Phase 1 if possible. - Otherwise, deploy immediately after Phase 1 (read-only mode). 4. **Verify after Phase 2:** - Confirm your service handles the new response format. - Check that missing campaigns are handled gracefully. The exact calendar dates for each phase will be communicated separately. --- ## 7. Questions to Answer Before You Start When updating your service, it helps to clarify: - Do you **currently** create, update or delete campaigns via ServiceLayer? - Do you rely on any of the fields that are being dropped (`automated`, `automated_over_performance`, `rating`, `tableau_reporting`, `chain_id`, `options`, AdWords fields, etc.)? - Do you use any of the deprecated search parameters (`automated`, `tableau_reporting`, `has_adwords`, `ratings`, `options`, `no_options`, `automated_over_performance`)? - Do you depend on very old or partially-filled campaigns that may be deleted as part of the cleanup? - Do you use the `chain` decorator to get related campaigns? - Do you use the `countries` decorator to get country data from campaigns? (This data is being removed) - Do you use the `banners` decorator to get banner data from campaigns? (This data is being removed) - Do you link banners or lead-campaigns to campaigns via PUT/DELETE on sub-component endpoints? (This functionality is removed) - Do you link organisations or countries to campaigns via PUT/DELETE on sub-component endpoints? - Do you have custom enums or models for `type`, `region`, `currency`, or `campaign_type` that might include deprecated values (`advanced_nrbp`, `rbp_cpl`, `west`, `bne`, `global`, `act`)? - Does your code handle `crm_id` as a string? (It's changing to an integer.) Having these answers ready will make it easier for an AI assistant (or another developer) in your other repository to propose specific code changes. --- ## 8. Quick Reference: Field Status ### Fields Being REMOVED ``` automated automated_over_performance tableau_reporting is_scheduled nullIfOld automation_notes options / campaign_options external_notes chain_id rating adwords_start adwords_end ``` ### Decorators Being REMOVED (Data No Longer in Responses) ``` banners (banner linking deprecated - data removed from responses) chain (chain_id column dropped) countries (deprecated feature - data removed from responses) ``` ### Decorators with LINKING REMOVED (Data Still Available) ``` organisations (read-only data stays in response, linking moved to Campaign Management API) ``` ### Sub-Component Endpoints Being REMOVED ``` /data/campaigns/any/details/{id}/banners/ (banner linking deprecated) /data/campaigns/any/details/{id}/lead-campaigns/ (lead-campaign linking deprecated) ``` ### Fields Becoming REQUIRED (non-null) ``` persist_gl_param_ref_link (TINYINT) budget (INT) budget_total (INT) assistant_id (INT) handler_id (INT) crm_id (INT - was VARCHAR) custom_products_budget (INT) type (ENUM: rbp, nrbp, hybrid_ebp, full_ebp, free_trial) region (ENUM: other, uk, apac, emea, na, strp) currency (ENUM: EUR, GBP, USD, AUD, CAD) campaign_type (ENUM: custom_products, listing_products) start_date (DATE) end_date (DATE) ``` ### Fields Unchanged ``` id name title exclude_from_campaign_lift internal_notes geo_targeted ``` ### Search Parameters Being REMOVED ``` automated automated_over_performance tableau_reporting has_adwords ratings options no_options banner_ids (banner linking deprecated) has_banner (banner linking deprecated) lead_campaign_ids (lead-campaign linking deprecated) capped_budget (budget is now always non-null) ``` --- ## 9. Example: Before and After API Response ### Before (current response from `/data/campaigns/any/details/{id}/`) ```json { "123": { "id": 123, "ego": "/data/campaigns/any/details/123/", "name": "Example Campaign", "title": "Example Campaign", "type": "rbp", "region": "emea", "currency": "EUR", "campaign_type": 1, "start_date": 1609459200, "end_date": 1640995200, "budget": 10000, "budget_total": 50000, "assistant_id": 42, "handler_id": 43, "crm_id": "AB123", "custom_products_budget": 5000, "persist_gl_param_ref_link": 1, "exclude_from_campaign_lift": 0, "internal_notes": "<p>Some notes</p>", "chain_id": 5, "options": "internal,fixed_end_date", "external_notes": "<p>External</p>", "automated": 1, "automated_over_performance": 10, "geo_targeted": 0, "tableau_reporting": 1, "rating": "A", "automation_notes": "<p>Auto notes</p>", "adwords_start": 1609459200, "adwords_end": 1640995200, "chain": { ... }, "organisations": [ ... ], "countries": [ ... ], "components": { ... } } } ``` ### After (new response format) ```json { "123": { "id": 123, "ego": "/data/campaigns/any/details/123/", "name": "Example Campaign", "title": "Example Campaign", "type": "rbp", "region": "emea", "currency": "EUR", "campaign_type": "listing_products", "start_date": 1609459200, "end_date": 1640995200, "budget": 10000, "budget_total": 50000, "assistant_id": 42, "handler_id": 43, "crm_id": 12345, "custom_products_budget": 5000, "persist_gl_param_ref_link": 1, "exclude_from_campaign_lift": 0, "internal_notes": "<p>Some notes</p>", "geo_targeted": 0, "organisations": [ ... ], "components": { ... } } } ``` **Key differences:** - `crm_id` is now an integer (was string `"AB123"`, now `12345`) - `campaign_type` is now a string enum (was integer `1`, now `"listing_products"`) - All dropped fields are gone (no `chain_id`, `options`, `automated`, etc.) - Removed decorators: `chain`, `banners`, `countries` no longer in response --- ## 10. Full List of Affected Endpoints ### Main Campaign Endpoints (Now Read-Only) | Endpoint | Current Methods | After Migration | |----------|-----------------|-----------------| | `/data/campaigns/any/details/` | GET, PUT, POST, DELETE | **GET only** | | `/data/campaigns/any/details/{id}/` | GET, POST, DELETE | **GET only** | | `/data/campaigns/any/list/` | GET | GET (unchanged) | | `/data/campaigns/any/ids/` | GET | GET (unchanged) | | `/data/campaigns/any/export/` | GET | GET (unchanged) | | `/data/campaigns/any/google-ads/` | GET | GET (unchanged) | ### Sub-Component Endpoints (Still Writable) These endpoints write to **linking tables**, not the `campaigns` table, so they remain fully functional: | Endpoint | Methods | Linking Table | |----------|---------|---------------| | `/data/campaigns/any/details/{id}/customproducts/` | GET, PUT, DELETE | `campaigns_products2` | | `/data/campaigns/any/details/{id}/presetproducts/` | GET, PUT, DELETE | `campaigns_products2` | ### Sub-Component Endpoints Being REMOVED (Deprecated Functionality) The following sub-component endpoints are being **completely removed** as the banner and lead-campaign linking functionality is deprecated: | Endpoint | Previous Methods | Replacement | |----------|------------------|-------------| | `/data/campaigns/any/details/{id}/banners/` | GET, PUT, DELETE | **REMOVED** - No replacement, functionality deprecated | | `/data/campaigns/any/details/{id}/lead-campaigns/` | GET, PUT, DELETE | **REMOVED** - No replacement, functionality deprecated | > **Note:** The linking tables (`campaigns_banners`, `lead_campaigns_campaigns`) are no longer writable via ServiceLayer campaigns endpoints. This functionality has been deprecated. ### Legacy Endpoints Being REMOVED The following legacy endpoints are being **completely removed** from ServiceLayer. You must migrate to the equivalent `/any/` endpoint or remove the functionality if no equivalent exists. #### Endpoints WITH Equivalents (Must Migrate) | Legacy Endpoint | Equivalent `/any/` Endpoint | Notes | |-----------------|----------------------------|-------| | `/data/campaigns/all/` | `/data/campaigns/any/details/` | Same properties available | | `/data/campaigns/all/{id}/` | `/data/campaigns/any/details/{id}/` | Same properties available | | `/data/campaigns/details/` | `/data/campaigns/any/details/` | Same functionality | | `/data/campaigns/ids/` | `/data/campaigns/any/ids/` | Identical functionality | | `/data/campaigns/search/` | `/data/campaigns/any/ids/` | Was already an alias for `/ids/`, marked obsolete in original code | #### Endpoints WITHOUT Equivalents (Must Remove) | Legacy Endpoint | Reason | Action Required | |-----------------|--------|-----------------| | `/data/campaigns/sender/` | Specialized endpoint for campaign sending | **Remove usage** - if you need recipient data, query `/data/campaigns/any/details/{id}/` with `recipients` decorator | #### Sub-Component Endpoints Migration | Legacy Endpoint | Equivalent | Notes | |-----------------|------------|-------| | `/data/campaigns/all/{id}/banners/` | **REMOVED** | Banner functionality deprecated | | `/data/campaigns/all/{id}/lead-campaigns/` | **REMOVED** | Lead-campaign functionality deprecated | | `/data/campaigns/all/{id}/organisations/` | **REMOVED** | Use Campaign Management API for linking | | `/data/campaigns/all/{id}/countries/` | **REMOVED** | Deprecated feature | ### Decorator Changes Summary | Decorator | Data in Response | Linking Sub-Component | Notes | |-----------|------------------|----------------------|-------| | `organisations` | ✅ Still available | ❌ Removed | Linking moved to Campaign Management API | | `banners` | ❌ Removed | ❌ Removed | Deprecated feature, entity deleted | | `countries` | ❌ Removed | ❌ Removed | Deprecated feature | | `chain` | ❌ Removed | N/A | `chain_id` column dropped | | `lead_campaigns` | ❌ Removed | ❌ Removed | Deprecated feature, entity deleted | **Note:** The `attached_organisations` decorator remains available (this is a different relationship used for filtering). --- ## 11. Using `@studyportals/sp-hs-misc` with the New Campaign API If your service uses `@studyportals/sp-hs-misc` to interact with ServiceLayer campaigns, this section explains what you need to do. ### 11.1 Required Package Updates Update to the latest versions that support the new campaign API: ```bash npm install @studyportals/sp-millennium-falcon@^3.0.0 npm install @studyportals/sp-hs-misc@^3.0.5 ``` **Important:** Update `sp-millennium-falcon` first, then `sp-hs-misc`. **Version notes:** - v3.0.1: Added proper entry point (`main` and `types` in package.json) - v3.0.5: **Restored browser build** for CDN usage (ES2020 modules) ### 11.2 Breaking Changes in v3.0.0 #### Browser Build Status The browser build (`bin-browser/`) was **removed in v3.0.0** but **restored in v3.0.5** as ES2020 modules. **Status by version:** - **v2.x:** Browser build available (global scripts) - **v3.0.0 - v3.0.4:** Browser build removed (CDN imports broken) - **v3.0.5+:** Browser build restored (ES2020 modules) #### Import Path Changes for Node.js For Node.js services, imports should use the main entry point: ```typescript // ❌ OLD — bin-browser no longer exists in v3.0.0+ import { ServiceLayerClient } from '@studyportals/sp-hs-misc/bin-browser/src/adapters/service-layer-client.class'; // ❌ OLD — direct bin/ path (still works but not recommended) import { ServiceLayerClient } from '@studyportals/sp-hs-misc/bin/src/adapters/service-layer-client.class'; // ✅ NEW — use main entry point (RECOMMENDED for Node.js) import { ServiceLayerClient } from '@studyportals/sp-hs-misc'; ``` #### Browser CDN Usage (v3.0.5+) For browser usage via CDN (jsDelivr, unpkg), use the restored `/bin-browser/` with ES modules: ```html <!-- v3.0.0 - v3.0.4: BROKEN --> <script src="https://cdn.jsdelivr.net/npm/@studyportals/sp-hs-misc@3.0.4/bin/index.js"></script> <!-- Error: exports is not defined --> <!-- v3.0.5+: WORKING - ES Modules --> <script type="module"> import { ServiceLayerClient, BaseSuperAgentRequestFactory } from 'https://cdn.jsdelivr.net/npm/@studyportals/sp-hs-misc@3.0.5/bin-browser/index.js'; const client = new ServiceLayerClient( new BaseSuperAgentRequestFactory(), 'https://servicelayer.example.com', 3 ); </script> ``` **Browser requirements:** - Must use `<script type="module">` - Modern browsers with ES2020 support - Direct class imports also supported from `/bin-browser/src/.../*.js` #### Campaign Model Changes The `ICampaign` interface and `CampaignDto` class no longer include `externalNotes`: ```typescript // ❌ OLD — no longer valid const campaign: ICampaign = { id: "123", title: "Example", externalNotes: "Some notes", // REMOVED // ... }; // ✅ NEW — externalNotes removed const campaign: ICampaign = { id: "123", title: "Example", // ... }; ``` #### Campaign Type Enum Changes The following `CampaignType` enum values have been removed: - `CampaignType.ADVANCED_NRBP` - `CampaignType.RBP_CPL` **Valid campaign types:** - `CampaignType.RBP` - `CampaignType.NRBP` - `CampaignType.HYBRID_EBP` - `CampaignType.FULL_EBP` - `CampaignType.FREE_TRIAL` ### 11.3 ServiceLayerDataModelsFactory The `createCampaign()` method automatically handles the new API response format: ```typescript import { ServiceLayerDataModelsFactory } from '@studyportals/sp-hs-misc'; const factory = new ServiceLayerDataModelsFactory(); const campaign = factory.createCampaign(serviceLayerResponse); // Returns ICampaign with: // - id, title, startDate, endDate, currency, budget, type // - NO externalNotes property ``` ### 11.4 ServiceLayerClient Usage The `ServiceLayerClient` is a generic HTTP client. After this migration: ```typescript import { ServiceLayerClient, BaseSuperAgentRequestFactory } from '@studyportals/sp-hs-misc'; const client = new ServiceLayerClient( new BaseSuperAgentRequestFactory(), 'https://servicelayer.example.com', 3 ); // ✅ READ operations — still work const campaigns = await client.get('/data/campaigns/any/details/'); const campaign = await client.getCached('/data/campaigns/any/details/123/'); // ❌ WRITE operations on campaign entity — will FAIL after Phase 1 await client.post('/data/campaigns/any/details/', data); // BLOCKED await client.put('/data/campaigns/any/details/123/', data); // BLOCKED await client.delete('/data/campaigns/any/details/123/'); // BLOCKED // ✅ WRITE operations on custom/preset products — still work await client.put('/data/campaigns/any/details/123/customproducts/', data); await client.put('/data/campaigns/any/details/123/presetproducts/', data); // ❌ WRITE operations on banners/lead-campaigns — REMOVED (deprecated) await client.put('/data/campaigns/any/details/123/banners/', data); // REMOVED await client.put('/data/campaigns/any/details/123/lead-campaigns/', data); // REMOVED ``` ### 11.5 Action Checklist for Your Service If your service uses `hs-misc` to interact with campaigns: - [ ] Update `@studyportals/sp-millennium-falcon` to ^3.0.0 - [ ] Update `@studyportals/sp-hs-misc` to ^3.0.5 (or ^3.0.1 minimum) - [ ] **For Node.js:** Search for `bin-browser` imports and replace with main entry point - [ ] **For browsers:** Update to v3.0.5+ and use ES module imports from `/bin-browser/` - [ ] Remove any references to `externalNotes` on campaign objects - [ ] Remove handling for `CampaignType.ADVANCED_NRBP` and `CampaignType.RBP_CPL` - [ ] Search for `ServiceLayerClient` write operations (`post`, `put`, `delete`) - [ ] Verify none target `/data/campaigns/any/details/` or legacy campaign endpoints - [ ] If any do, migrate to a dedicated campaign service ### 11.6 Deprecated Classes (Unrelated to This Migration) These classes in `hs-misc` are deprecated but **not affected** by the campaigns revamp: | Class | Replacement | |-------|-------------| | `CognitoAuthenticationServicesProvider` | `@studyportals/client-internal-platform-sso` | | `UserSessionCookieManager` | `@studyportals/client-internal-platform-sso` | | `K2SOUserPrivilegesDataHelper` | `@studyportals/client-internal-platform-authorization` | | `AWS4RequestSigner` | `@studyportals/mb-platform-http-requests` | | `SignedRequestSigner` | `@studyportals/mb-platform-http-requests` | --- ## 12. Complete Removal of Deprecated Entities As part of this migration, the following **entire entities** have been removed from ServiceLayer-Private. These were deprecated features that are no longer supported. ### 12.1 Entities Completely Removed | Entity | Description | Replacement | |--------|-------------|-------------| | `Banner` | Banner display management | **None** - deprecated feature | | `BannerBox` | Banner box container management | **None** - deprecated feature | | `Lead` | Lead (enquiry) data storage | **None** - deprecated feature | | `LeadCampaign` | Lead campaign configuration | **None** - deprecated feature | | `LeadCampaignStudy` | Study-to-lead-campaign linking | **None** - deprecated feature | ### 12.2 API Routes Completely Removed The following **entire routes** have been removed from ServiceLayer-Private. All paths under these routes will return 404. | Route | Description | |-------|-------------| | `/data/banners/*` | Banner entity endpoints | | `/data/banner-boxes/*` | Banner box entity endpoints | | `/data/leads/*` | Lead entity endpoints | | `/data/lead-campaigns/*` | Lead campaign entity endpoints | | `/data/lead-campaign-studies/*` | Lead campaign study entity endpoints | | `/banner/*` | Banner driver (non-data) | | `/lead-qualifier/*` | Lead qualifier driver | | `/publish/lead-notify-*` | Publish driver lead notification routes | | `/publish/email/lead/*` | Publish driver lead email routes | ### 12.3 Search Parameters Completely Removed The following search parameters have been removed from **all entities** that previously supported them: | Parameter | Removed From | Reason | |-----------|--------------|--------| | `banner_ids` | Campaigns, Countries, Disciplines | Banner entity removed | | `has_banner` | Campaigns | Banner entity removed | | `lead_campaign_ids` | Campaigns, Studies, Products | LeadCampaign entity removed | | `has_lead_campaign` | Studies | LeadCampaign entity removed | ### 12.4 Decorators Completely Removed | Decorator | Removed From | Reason | |-----------|--------------|--------| | `banners` | Campaign | Banner entity removed | | `countries` | Campaign | Deprecated feature | | `lead_campaigns` | Study | LeadCampaign entity removed | ### 12.5 Database Tables To Be Dropped After the code changes are deployed, the following **17 database tables** will be dropped: #### Banner Tables (7 tables) ```sql DROP TABLE banners; DROP TABLE banners_box; DROP TABLE banners_boxes; DROP TABLE banners_countries; DROP TABLE banners_countries_origin; DROP TABLE banners_disciplines; DROP TABLE banners_regions; DROP TABLE banners_scholarship; ``` #### Campaign Linking Tables (2 tables) ```sql DROP TABLE campaigns_banners; DROP TABLE campaigns_countries; ``` #### Lead Tables (8 tables) ```sql DROP TABLE leads; DROP TABLE lead_campaigns; DROP TABLE lead_campaigns_campaigns; DROP TABLE lead_campaigns_countries_nationality; DROP TABLE lead_campaigns_countries_residence; DROP TABLE lead_campaigns_pricings; DROP TABLE lead_campaigns_studies; ``` ### 12.6 Impact on Other Entity Responses The removal of these entities affects **responses from other endpoints**. This section details what data is no longer returned. #### Study Entity — `lead_campaigns` Decorator REMOVED **Affected endpoints:** - `/data/studies/any/details/` - `/data/studies/any/details/{id}/` - `/data/studies/public/details/` - Any endpoint that previously included the `lead_campaigns` decorator **Before (old response):** ```json { "123": { "id": 123, "name": "Computer Science MSc", "lead_campaigns": [ { "id": 45, "name": "CS Campaign 2024", "active": true } ] } } ``` **After (new response):** ```json { "123": { "id": 123, "name": "Computer Science MSc" } } ``` The `lead_campaigns` property **no longer exists** in study responses. **Search patterns:** ``` study.lead_campaigns studies[id].lead_campaigns response.lead_campaigns ?.lead_campaigns ``` **Action:** Remove all code that reads `lead_campaigns` from study responses. --- #### Study Entity — `lead_campaign_ids` and `has_lead_campaign` Search Parameters REMOVED **Affected endpoints:** - `/data/studies/any/details/?q=lead_campaign_ids-45` - `/data/studies/public/results/?q=has_lead_campaign-1` These query parameters **no longer work**. Using them will have no filtering effect. **Search patterns:** ``` lead_campaign_ids has_lead_campaign q=lead_campaign_ids q=has_lead_campaign ``` **Action:** Remove any filtering logic that uses these parameters. --- #### Campaign Entity — `banners` Decorator REMOVED **Affected endpoints:** - `/data/campaigns/any/details/` - `/data/campaigns/any/details/{id}/` **Before (old response):** ```json { "456": { "id": 456, "name": "Summer Campaign", "banners": [ { "id": 78, "name": "Homepage Banner" } ] } } ``` **After (new response):** ```json { "456": { "id": 456, "name": "Summer Campaign" } } ``` The `banners` property **no longer exists** in campaign responses. **Search patterns:** ``` campaign.banners campaigns[id].banners response.banners ?.banners ``` **Action:** Remove all code that reads `banners` from campaign responses. --- #### Campaign Entity — `banner_ids` and `has_banner` Search Parameters REMOVED **Affected endpoints:** - `/data/campaigns/any/details/?q=banner_ids-78` - `/data/campaigns/any/details/?q=has_banner-1` These query parameters **no longer work**. **Search patterns:** ``` banner_ids has_banner q=banner_ids q=has_banner ``` **Action:** Remove any filtering logic that uses these parameters. --- #### Country Entity — `banner_ids` Search Parameter REMOVED **Affected endpoints:** - `/data/countries/any/details/?q=banner_ids-78` This query parameter **no longer works**. **Search patterns:** ``` /data/countries.*banner_ids countries.*q=banner_ids ``` **Action:** Remove any country filtering logic that uses banner-related parameters. --- #### Discipline Entity — `banner_ids` Search Parameter REMOVED **Affected endpoints:** - `/data/disciplines/any/details/?q=banner_ids-78` This query parameter **no longer works**. **Search patterns:** ``` /data/disciplines.*banner_ids disciplines.*q=banner_ids ``` **Action:** Remove any discipline filtering logic that uses banner-related parameters. --- #### Product Entity — `lead_campaign_ids` Search Parameter REMOVE