@accounter/scraper-app
Version:
Scraper app with Fastify server and React UI
638 lines (550 loc) • 43.5 kB
Markdown
# @accounter/scraper-app
## 0.0.3-alpha-20260819194013-dcbb6aebad214e81bc64f34f6babc7ffeac3cc21
### Patch Changes
- [#3792](https://github.com/Urigo/accounter-fullstack/pull/3792)
[`1a8e0ea`](https://github.com/Urigo/accounter-fullstack/commit/1a8e0eaee582464aa9ec79b71c8aadd618fc5a15)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency [`fastify@5.9.0` ↗︎](https://www.npmjs.com/package/fastify/v/5.9.0) (from
`5.8.5`, in `dependencies`)
- [#3836](https://github.com/Urigo/accounter-fullstack/pull/3836)
[`059b638`](https://github.com/Urigo/accounter-fullstack/commit/059b638702116701a5a0062a7ac3a58a28eeb004)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - dependencies updates:
- Updated dependency [`fastify@5.10.0` ↗︎](https://www.npmjs.com/package/fastify/v/5.10.0) (from
`5.9.0`, in `dependencies`)
- [#3842](https://github.com/Urigo/accounter-fullstack/pull/3842)
[`f19c49c`](https://github.com/Urigo/accounter-fullstack/commit/f19c49cccfb84b5cd3b8b0455f241ef325b9facb)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency [`fastify@5.10.0` ↗︎](https://www.npmjs.com/package/fastify/v/5.10.0) (from
`5.9.0`, in `dependencies`)
- [#3871](https://github.com/Urigo/accounter-fullstack/pull/3871)
[`8a38200`](https://github.com/Urigo/accounter-fullstack/commit/8a38200368a8e0e9a80a325e3bb878f99a26c38d)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency [`msw@2.15.0` ↗︎](https://www.npmjs.com/package/msw/v/2.15.0) (from `2.14.6`,
in `dependencies`)
- [#3873](https://github.com/Urigo/accounter-fullstack/pull/3873)
[`246b1e4`](https://github.com/Urigo/accounter-fullstack/commit/246b1e4d41a6424c4b9f251ff0b8cf36c774ab30)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - dependencies updates:
- Updated dependency
[`@fastify/static@9.3.0` ↗︎](https://www.npmjs.com/package/@fastify/static/v/9.3.0) (from
`9.1.3`, in `dependencies`)
- Updated dependency
[`fast-xml-parser@5.10.0` ↗︎](https://www.npmjs.com/package/fast-xml-parser/v/5.10.0) (from
`5.9.3`, in `dependencies`)
- Updated dependency [`msw@2.15.0` ↗︎](https://www.npmjs.com/package/msw/v/2.15.0) (from `2.14.6`,
in `dependencies`)
- [#3881](https://github.com/Urigo/accounter-fullstack/pull/3881)
[`9556c7e`](https://github.com/Urigo/accounter-fullstack/commit/9556c7ed91f83629c76e6c502826aff948850354)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency
[`@fastify/static@9.3.0` ↗︎](https://www.npmjs.com/package/@fastify/static/v/9.3.0) (from
`9.1.3`, in `dependencies`)
- [#3882](https://github.com/Urigo/accounter-fullstack/pull/3882)
[`f9f80c8`](https://github.com/Urigo/accounter-fullstack/commit/f9f80c84fce8fbe32e58bd24e4ab0ae858e8fc46)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency
[`@fastify/websocket@11.3.0` ↗︎](https://www.npmjs.com/package/@fastify/websocket/v/11.3.0) (from
`11.2.0`, in `dependencies`)
- [#3911](https://github.com/Urigo/accounter-fullstack/pull/3911)
[`6c3406e`](https://github.com/Urigo/accounter-fullstack/commit/6c3406e6a5a8bd91f76f3b971db33f4a526de582)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency
[`fast-xml-parser@5.10.0` ↗︎](https://www.npmjs.com/package/fast-xml-parser/v/5.10.0) (from
`5.9.3`, in `dependencies`)
- [#3912](https://github.com/Urigo/accounter-fullstack/pull/3912)
[`494b8f8`](https://github.com/Urigo/accounter-fullstack/commit/494b8f87ffce00b37e1ebed571fb1026317a007f)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency
[`@fastify/static@10.1.0` ↗︎](https://www.npmjs.com/package/@fastify/static/v/10.1.0) (from
`9.3.0`, in `dependencies`)
- [#3923](https://github.com/Urigo/accounter-fullstack/pull/3923)
[`9ee7ab8`](https://github.com/Urigo/accounter-fullstack/commit/9ee7ab89299fe08b12a5ac639dc84dfd25a390c7)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - dependencies updates:
- Updated dependency
[`@fastify/websocket@11.3.0` ↗︎](https://www.npmjs.com/package/@fastify/websocket/v/11.3.0) (from
`11.2.0`, in `dependencies`)
- [#3925](https://github.com/Urigo/accounter-fullstack/pull/3925)
[`547e405`](https://github.com/Urigo/accounter-fullstack/commit/547e40567b02e0e16a7f6d3cfdbf12fe5ae819b6)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - dependencies updates:
- Updated dependency
[`@fastify/websocket@11.3.0` ↗︎](https://www.npmjs.com/package/@fastify/websocket/v/11.3.0) (from
`11.2.0`, in `dependencies`)
- [#3927](https://github.com/Urigo/accounter-fullstack/pull/3927)
[`485dae7`](https://github.com/Urigo/accounter-fullstack/commit/485dae7f363133e7c794bb2a6ae50fe50fef2b9f)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - dependencies updates:
- Updated dependency
[`@fastify/websocket@11.3.0` ↗︎](https://www.npmjs.com/package/@fastify/websocket/v/11.3.0) (from
`11.2.0`, in `dependencies`)
- [#3936](https://github.com/Urigo/accounter-fullstack/pull/3936)
[`5b2ffbb`](https://github.com/Urigo/accounter-fullstack/commit/5b2ffbbec76719fb0b6cd17dbe535755a830287d)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency [`ws@8.21.1` ↗︎](https://www.npmjs.com/package/ws/v/8.21.1) (from `8.21.0`, in
`dependencies`)
- [#3938](https://github.com/Urigo/accounter-fullstack/pull/3938)
[`2a0c041`](https://github.com/Urigo/accounter-fullstack/commit/2a0c041b0dc6104039304598f932e7990e84c2c0)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - dependencies updates:
- Updated dependency
[`@fastify/static@10.1.0` ↗︎](https://www.npmjs.com/package/@fastify/static/v/10.1.0) (from
`9.3.0`, in `dependencies`)
- [#3939](https://github.com/Urigo/accounter-fullstack/pull/3939)
[`aed9832`](https://github.com/Urigo/accounter-fullstack/commit/aed98323f04b757d0ad3d5b2b3b67280a1db61dc)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - dependencies updates:
- Updated dependency
[`@fastify/static@10.1.0` ↗︎](https://www.npmjs.com/package/@fastify/static/v/10.1.0) (from
`9.3.0`, in `dependencies`)
- [#3942](https://github.com/Urigo/accounter-fullstack/pull/3942)
[`57c9258`](https://github.com/Urigo/accounter-fullstack/commit/57c92581f2359697b84278ab40c56c9b0aa710f6)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - dependencies updates:
- Updated dependency
[`@fastify/static@10.1.0` ↗︎](https://www.npmjs.com/package/@fastify/static/v/10.1.0) (from
`9.3.0`, in `dependencies`)
- [#3955](https://github.com/Urigo/accounter-fullstack/pull/3955)
[`bdcc700`](https://github.com/Urigo/accounter-fullstack/commit/bdcc7001ddaa78524310952919ebf30d331447eb)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency
[`fast-xml-parser@5.10.1` ↗︎](https://www.npmjs.com/package/fast-xml-parser/v/5.10.1) (from
`5.10.0`, in `dependencies`)
- [#3964](https://github.com/Urigo/accounter-fullstack/pull/3964)
[`49cd281`](https://github.com/Urigo/accounter-fullstack/commit/49cd281e9aa63832cd1eda409472d6074ad83ba4)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - dependencies updates:
- Updated dependency
[`fast-xml-parser@5.10.1` ↗︎](https://www.npmjs.com/package/fast-xml-parser/v/5.10.1) (from
`5.10.0`, in `dependencies`)
- [#4041](https://github.com/Urigo/accounter-fullstack/pull/4041)
[`8b56b46`](https://github.com/Urigo/accounter-fullstack/commit/8b56b468e08154e0a496595c5c4a6b283bfb9bbb)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency
[`@fastify/static@10.1.2` ↗︎](https://www.npmjs.com/package/@fastify/static/v/10.1.2) (from
`10.1.0`, in `dependencies`)
- [#4042](https://github.com/Urigo/accounter-fullstack/pull/4042)
[`a71f169`](https://github.com/Urigo/accounter-fullstack/commit/a71f16930752deda421e7a8cd280e2ba1c42a79f)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency [`react@19.2.8` ↗︎](https://www.npmjs.com/package/react/v/19.2.8) (from
`19.2.7`, in `dependencies`)
- Updated dependency [`react-dom@19.2.8` ↗︎](https://www.npmjs.com/package/react-dom/v/19.2.8)
(from `19.2.7`, in `dependencies`)
- [#4057](https://github.com/Urigo/accounter-fullstack/pull/4057)
[`9d7b4ff`](https://github.com/Urigo/accounter-fullstack/commit/9d7b4ff93bcbd19146aca8a6957625632ad1a53d)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency [`fastify@5.11.0` ↗︎](https://www.npmjs.com/package/fastify/v/5.11.0) (from
`5.10.0`, in `dependencies`)
- [#4072](https://github.com/Urigo/accounter-fullstack/pull/4072)
[`9b0e91f`](https://github.com/Urigo/accounter-fullstack/commit/9b0e91ffa7916c70ad9ce769153ba6fd71186478)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency [`fastify@5.11.0` ↗︎](https://www.npmjs.com/package/fastify/v/5.11.0) (from
`5.10.0`, in `dependencies`)
- [#4101](https://github.com/Urigo/accounter-fullstack/pull/4101)
[`0e724c3`](https://github.com/Urigo/accounter-fullstack/commit/0e724c318b801b9ec616732cc1391da8915946c6)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency [`fastify@5.11.2` ↗︎](https://www.npmjs.com/package/fastify/v/5.11.2) (from
`5.11.0`, in `dependencies`)
- [#4120](https://github.com/Urigo/accounter-fullstack/pull/4120)
[`73316d3`](https://github.com/Urigo/accounter-fullstack/commit/73316d3a7704b22a493d5357097a283baf3567ea)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency
[`@fastify/static@10.1.3` ↗︎](https://www.npmjs.com/package/@fastify/static/v/10.1.3) (from
`10.1.2`, in `dependencies`)
- Updated dependency [`ws@8.21.2` ↗︎](https://www.npmjs.com/package/ws/v/8.21.2) (from `8.21.1`, in
`dependencies`)
- [#4121](https://github.com/Urigo/accounter-fullstack/pull/4121)
[`b28d745`](https://github.com/Urigo/accounter-fullstack/commit/b28d745456f135d952437ca5ad7056f6188aae33)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - dependencies updates:
- Updated dependency [`fastify@5.11.2` ↗︎](https://www.npmjs.com/package/fastify/v/5.11.2) (from
`5.11.0`, in `dependencies`)
- [#4131](https://github.com/Urigo/accounter-fullstack/pull/4131)
[`4cdb553`](https://github.com/Urigo/accounter-fullstack/commit/4cdb553058e07928815b9fedfd1ff7fb86e6e702)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency [`ws@8.21.2` ↗︎](https://www.npmjs.com/package/ws/v/8.21.2) (from `8.21.1`, in
`dependencies`)
- [#4146](https://github.com/Urigo/accounter-fullstack/pull/4146)
[`4b3cddc`](https://github.com/Urigo/accounter-fullstack/commit/4b3cddc633e8bcd1e4bf352623b8d59ecf0971e5)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency
[`@fastify/static@10.1.3` ↗︎](https://www.npmjs.com/package/@fastify/static/v/10.1.3) (from
`10.1.2`, in `dependencies`)
- [#4149](https://github.com/Urigo/accounter-fullstack/pull/4149)
[`40e131e`](https://github.com/Urigo/accounter-fullstack/commit/40e131ea47520879f220c4c7bec2f0b58a52d729)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency [`fastify@5.11.3` ↗︎](https://www.npmjs.com/package/fastify/v/5.11.3) (from
`5.11.2`, in `dependencies`)
- Updated dependency [`ws@8.21.3` ↗︎](https://www.npmjs.com/package/ws/v/8.21.3) (from `8.21.2`, in
`dependencies`)
- [#4206](https://github.com/Urigo/accounter-fullstack/pull/4206)
[`8c0814b`](https://github.com/Urigo/accounter-fullstack/commit/8c0814b5d871eb348484effdce6c2fe38b3fbd6d)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency [`fastify@5.12.0` ↗︎](https://www.npmjs.com/package/fastify/v/5.12.0) (from
`5.11.3`, in `dependencies`)
- [#4209](https://github.com/Urigo/accounter-fullstack/pull/4209)
[`34d4c12`](https://github.com/Urigo/accounter-fullstack/commit/34d4c126f017b92d02b56de4b473f3075baa1996)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - dependencies updates:
- Updated dependency [`fastify@5.12.0` ↗︎](https://www.npmjs.com/package/fastify/v/5.12.0) (from
`5.11.3`, in `dependencies`)
- [#4065](https://github.com/Urigo/accounter-fullstack/pull/4065)
[`b218e17`](https://github.com/Urigo/accounter-fullstack/commit/b218e17e1f96d9e12f6c9a825dd8bba7a32f6af6)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - Align the MAX credit card ingestion path
with what MAX actually returns.
The Zod payload schema for MAX was `.loose()` and covered only a handful of fields, so the nested
`dealData`, `merchantData` and `runtimeReference` blocks were never validated and were passed
through to the GraphQL mutation unflattened. `max.schema.ts` now models the full transaction
strictly, and `maxVars` flattens those nested blocks into the `MaxTransactionInput` columns the
server expects.
MAX omits large parts of `dealData` for some transaction types, so the `MaxTransactionInput`
fields backing those columns are now nullable, and the numeric ones (`dealDataAmount`,
`dealDataAmountIls`, `dealDataExchangeRate`, `originalAmount`, …) are typed as `Float` rather than
`String`. Boolean fields backing `bit` columns are converted to `1`/`0` via a new
`convertBooleanToBit` helper — previously a JS boolean was sent to a `bit` column, which Postgres
rejects.
Also fixes a few typing gaps in `scraper-app` that surfaced alongside this: the upload client
derives its result types from the generated mutation types instead of a hand-maintained
`ScraperUploadResult`, `filterPayload` handles the two `otsar-hahayal` source types instead of
falling through, the config screen gets an explicit `CONFIGURABLE_SOURCE_TYPES` list, and the
package's tsconfig resolves `@accounter/modern-poalim-scraper` to its build output.
- [#4194](https://github.com/Urigo/accounter-fullstack/pull/4194)
[`4eb395f`](https://github.com/Urigo/accounter-fullstack/commit/4eb395fc0436adeca029f2a8bb49e0aef8f22c66)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - Ingest static foreign-securities reference
data from Bank Hapoalim.
Poalim's "mytrade" portfolio app exposes the static details of every security held in a trading
account — name, symbol, exchange, currency, instrument type. Accounter already had a
`FOREIGN_SECURITIES` account type and recognised securities fees, but had no record of _which_
securities an account holds, so securities transactions could not be resolved against instrument
metadata. This adds the full ingestion path for that list. Live portfolio balances
(`View.Account`, `View.Account.AccountPosition.Balance`) and open orders (`View.Orders`) are out
of scope; nothing reads the stored rows yet.
`modern-poalim-scraper` gains `getSecurities(account)`, following the same
`{ data, isValid, errors }` contract as its siblings, plus the exported `HapoalimSecurities` /
`PoalimSecurity` types. Two things about this endpoint differ from the rest of the Poalim surface
and are worth knowing:
- It addresses the account as `branch-account` (no bank number), unlike the three-part `accountId`
every other method uses.
- It is only served to callers running on the mytrade SPA's own page, and requires a `session`
header holding a **server-issued** key — inventing one is rejected with
`InvalidSessionException`. So the request is issued from a short-lived sibling tab on the
logged-in browser context, and `captureMytradeSession` listens for the SPA's own `/mytrade/api/`
call to harvest the real `session` / `csession` headers rather than guessing where the SPA
stores them.
Accounts with no securities portfolio omit `View.Meta.Security` entirely; that is treated as an
empty portfolio rather than a malformed response, so it no longer fails the whole Poalim run.
`server` gains `uploadPoalimSecurities(securities: [PoalimSecurityInput!]!)` on the
scraper-ingestion module, backed by a new `accounter_schema.poalim_securities` table (migration
`2026-08-11T12-00-00.add-poalim-securities-table`) that mirrors the source fields one-to-one. The
table is owner-scoped with RLS and a `tenant_isolation` policy, and deduplicates on
`(bank_number, branch_number, account_number, security_key)`. Re-scrapes are no-ops, and changed
attributes are reported through the shared `changedTransactions` result — `as_of_date` is excluded
from that comparison since it moves on every scrape and would otherwise flag every row. These rows
are reference data, not cash movements, so there is deliberately no insert trigger and no
`transactions_raw_list` wiring.
`scraper-app` gets a per-source "Fetch foreign securities" option (off by default — the endpoint
is portfolio-specific), a `securities` column in the run progress table, and the fetch/upload
steps.
Also tightens two things that this work surfaced: the shared XSRF lookup now reads cookies from
the browser context instead of the deprecated page-level cookie API, filtered by cookie domain —
all scrapers share the default browser context, so an unfiltered lookup could pick up another
bank's `XSRF-TOKEN`.
- [#3949](https://github.com/Urigo/accounter-fullstack/pull/3949)
[`36a1832`](https://github.com/Urigo/accounter-fullstack/commit/36a1832ad9da82660379109bd688c8f4e4d60d9f)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - Surface Poalim schema validation failures
instead of silently swallowing them. The modern-poalim-scraper runs Zod validation internally
(`validateSchema: true`) and returns `{ data, isValid, errors }`, setting `data` to `null` and
`isValid` to `false` on a failed validation. `scrapePoalim` only read `data`, so a failed
`getAccountsData` validation looked like "no accounts" and the run reported success with no data;
the same silent swallow applied to the per-account ILS, foreign, and SWIFT requests. Each Poalim
request now checks `isValid` and throws a descriptive error (including the validation issues),
which propagates to the run runner and is emitted as a `task-error` so the user is notified.
- [#4234](https://github.com/Urigo/accounter-fullstack/pull/4234)
[`bb8cc37`](https://github.com/Urigo/accounter-fullstack/commit/bb8cc3743043d2bb11595ccc8f49e6e52eccedfc)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - Accept the Poalim securities feed's non-USD
currencies, and page past its 150-execution response cap
The executions schema only knew `שקל חדש` and `דולר ארה"ב`, so any non-US listing failed
validation; `אירו`, `לירה שטרלינג` and `ין יפני` are now accepted and mapped to `Currency` on
read. The endpoint also caps a response at 150 executions without saying it truncated, so a full
page is re-requested with the window's ceiling pulled back to the oldest day that page reached,
and the pages are merged and deduplicated.
- [#4203](https://github.com/Urigo/accounter-fullstack/pull/4203)
[`9a38125`](https://github.com/Urigo/accounter-fullstack/commit/9a3812588d0ed0ebcee304122073950e9990abc8)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - Ingest securities portfolio activity from
Bank Hapoalim.
The previous securities work stored _what_ an account holds; this adds _what happened_ in it —
buys, sells, dividend and interest payments, redemptions, deposit transfers and other corporate
actions — from the "mytrade" order executions history
(`/mytrade/api/v2/json2/order/executions/history`). Nothing reads the stored rows yet.
Because there are now two securities feeds, the static one is renamed throughout so the two are
never confused:
- `modern-poalim-scraper`: `getSecurities` → `getSecuritiesInfo`; `HapoalimSecuritiesSchema` →
`HapoalimSecuritiesInfoSchema`; the `HapoalimSecurities` / `PoalimSecurity` types →
`HapoalimSecuritiesInfo` / `PoalimSecurityInfo`. **Breaking** for direct consumers of those
exports.
- `scraper-app`: payload type `poalim-securities` → `poalim-securities-info`,
`poalimSecuritiesVars` → `poalimSecuritiesInfoVars`, WebSocket `txnType` `securities` →
`securitiesInfo`.
The server's `uploadPoalimSecurities` mutation and the `accounter_schema.poalim_securities` table
keep their names — they are persisted API and schema, and read fine as the reference feed
alongside the new table.
`modern-poalim-scraper` gains `getSecuritiesTransactions(account, range?)`, following the same
`{ data, isValid, errors }` contract as its siblings, plus the exported
`HapoalimSecuritiesTransactions` / `PoalimSecurityTransaction` types. It shares the sibling-tab
and `captureMytradeSession` mechanics with `getSecuritiesInfo` (both now go through a common
`fetchFromMytrade` helper), with two differences: this endpoint is a **GET** — sending it as POST
returns nothing, so `fetchPoalimMytradeWithinPage` took a `method` parameter — and it takes a
`ddMMyyyy` date range, defaulting to the scraper's configured `duration` window.
The response schema is strict rather than permissive, on both the scraper and the scraper-app
side: closed enums for the bank's own vocabularies (transaction and trade types, payment types,
currencies, security groups, the `כן`/`לא` flags), formats for timestamps, ISIN, security numbers
and branch/account strings, ranges for percentages and non-negative amounts, and the cross-field
invariants the bank's data obeys — a buy/sell `TradeType` iff the matching `TransactionType`, the
`PaymentType`/`PaymentDate`/`ExDate` block filled all-or-nothing and never on a trade, the
`FinancialAccount*` strings agreeing with `Branch`/`Account`, one currency across issue/trade/
settlement, and `IsUSEquity` agreeing with the issuer country. The intent is that a bank-side
change is a loud, located failure rather than a silently mis-typed column: every constraint
carries a message naming the field, the offending value and what to widen, and
`describeSecuritiesTransactionsError` annotates each issue with the security and trade date of the
row it came from (the raw Zod path points at `Account.Execution.37.TradeType`, which says nothing
about which execution that is). `PayloadValidationError` now prints a capped `path: message` list
instead of dumping the whole `ZodError` as JSON.
`server` gains
`uploadPoalimSecuritiesTransactions(transactions: [PoalimSecurityTransactionInput!]!)` on the
scraper-ingestion module, backed by a new `accounter_schema.poalim_securities_transactions` table
(migration `2026-08-13T12-00-00.add-poalim-securities-transactions-table`) that mirrors the source
fields one-to-one, bank misspellings included (`israe_tax_value`, `peyment_pecentage`,
`trade_currnecy_rate`, `last_tranaction_date`, `fund_plus_accumulated_inerest_value`) so the
mapping stays mechanical. It is owner-scoped with RLS and a `tenant_isolation` policy, and joins
to `poalim_securities` on `security` / `security_key`. The response carries no per-execution id,
so deduplication uses the natural key — account, security, the trade/value/settlement dates, trade
and transaction type, quantity, price, net value and the corporate-action dates — with
`NULLS NOT DISTINCT`, since most of those are null on a plain trade. That key is verified unique
across a full year of real executions; shorter keys are not (two same-day dividends on one
security collide). Re-scrapes are no-ops and restated values are reported through the shared
`changedTransactions` result. Like `poalim_securities`, these rows are not cash movements: no
insert trigger, no `transactions_raw_list` wiring.
The date columns are `DATE`, matching every other `poalim_*` table. The bank sends calendar dates
dressed as instants — always midnight, carrying the Israel offset of that date — so storing them
as `TIMESTAMPTZ` (as the first cut did, fixed by migration
`2026-08-14T10-00-00.poalim-securities-transactions-calendar-dates`) put the stored instant at
21:00 or 22:00 UTC the day before, and anything rendering it outside Israel time reported the
wrong day. For the same reason the ingestion provider compares and queries these as calendar dates
rather than converting through `Date`, which would shift the day on any server not running on
Israel time.
`scraper-app` fetches and uploads both securities feeds under the existing per-source option (now
labelled "Fetch securities portfolio (info + activity)"), with separate progress columns for each.
Also extends the foreign-transactions `metadata.messages[].messageCode` lists, which the
enumerated unions rejected on an account whose informational banners differed from the ones
already listed.
- [#3803](https://github.com/Urigo/accounter-fullstack/pull/3803)
[`9bd0c32`](https://github.com/Urigo/accounter-fullstack/commit/9bd0c32ec5279fd98e7c41d3a17e9cd64a943dfc)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - - Updated `amex.test.ts`, `cal.test.ts`,
`discount.test.ts`, and `isracard.test.ts` to set `vi.setConfig({ testTimeout: 30_000 })`
- Added explanatory comments referencing PR
[#3795](https://github.com/urigo/accounter-fullstack/issues/3795) which introduced the 2-5s
per-month delays in scrapers
- [#4246](https://github.com/Urigo/accounter-fullstack/pull/4246)
[`cab8a23`](https://github.com/Urigo/accounter-fullstack/commit/cab8a238ae7683e19629179b9a4adca85fade063)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - Skip Max transactions with
`runtimeReference.type === 0` — these are declarations of transactions not received by the credit
card company and are missing many required attributes, which previously failed payload validation
and aborted the whole Max scrape.
- Updated dependencies
[[`670f7cc`](https://github.com/Urigo/accounter-fullstack/commit/670f7cc2bd58c04da5dd151e4878382916700a8e),
[`50ca939`](https://github.com/Urigo/accounter-fullstack/commit/50ca939661d9eb5b31e134c54d12015e524fac1c),
[`7a7dc71`](https://github.com/Urigo/accounter-fullstack/commit/7a7dc71ba2dc9a660f150340f2352704d571f367),
[`fd69847`](https://github.com/Urigo/accounter-fullstack/commit/fd698477f09c7bbf245bcb011afc359f47521da3),
[`e3f2337`](https://github.com/Urigo/accounter-fullstack/commit/e3f233710a581a48dee6e45e956170dc3c2799c3),
[`568beb7`](https://github.com/Urigo/accounter-fullstack/commit/568beb7c49949f553c9e6cce54ab248123ba86b3),
[`3910f54`](https://github.com/Urigo/accounter-fullstack/commit/3910f548f6da8ec4130878766853ec326df37b94),
[`f899b37`](https://github.com/Urigo/accounter-fullstack/commit/f899b37a1561ef168398d122f214d9802178707d),
[`73316d3`](https://github.com/Urigo/accounter-fullstack/commit/73316d3a7704b22a493d5357097a283baf3567ea),
[`9571ea2`](https://github.com/Urigo/accounter-fullstack/commit/9571ea22c35a030112b2631a8a4e2c0c212098fa),
[`75b5f17`](https://github.com/Urigo/accounter-fullstack/commit/75b5f17f1d642d2ac8f97d62f7e7d1cbb20cda24),
[`4eb395f`](https://github.com/Urigo/accounter-fullstack/commit/4eb395fc0436adeca029f2a8bb49e0aef8f22c66),
[`87e374e`](https://github.com/Urigo/accounter-fullstack/commit/87e374e340733845f3c6ff9616f2a475d390788d),
[`34d4c12`](https://github.com/Urigo/accounter-fullstack/commit/34d4c126f017b92d02b56de4b473f3075baa1996),
[`4eb395f`](https://github.com/Urigo/accounter-fullstack/commit/4eb395fc0436adeca029f2a8bb49e0aef8f22c66),
[`bb8cc37`](https://github.com/Urigo/accounter-fullstack/commit/bb8cc3743043d2bb11595ccc8f49e6e52eccedfc),
[`9a38125`](https://github.com/Urigo/accounter-fullstack/commit/9a3812588d0ed0ebcee304122073950e9990abc8),
[`c406f51`](https://github.com/Urigo/accounter-fullstack/commit/c406f5151625b26768c3764ee302ef5e1aa80c05)]:
- @accounter/modern-poalim-scraper@0.11.0-alpha-20260819194013-dcbb6aebad214e81bc64f34f6babc7ffeac3cc21
## 0.0.2
### Patch Changes
- [#3454](https://github.com/Urigo/accounter-fullstack/pull/3454)
[`0b654ef`](https://github.com/Urigo/accounter-fullstack/commit/0b654ef08495e1632ccc75942d6a1a3142924d9f)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency
[`fast-xml-parser@5.7.3` ↗︎](https://www.npmjs.com/package/fast-xml-parser/v/5.7.3) (from
`5.7.2`, in `dependencies`)
- Updated dependency [`msw@2.14.3` ↗︎](https://www.npmjs.com/package/msw/v/2.14.3) (from `2.14.2`,
in `dependencies`)
- Updated dependency [`react@19.2.6` ↗︎](https://www.npmjs.com/package/react/v/19.2.6) (from
`19.2.5`, in `dependencies`)
- Updated dependency [`react-dom@19.2.6` ↗︎](https://www.npmjs.com/package/react-dom/v/19.2.6)
(from `19.2.5`, in `dependencies`)
- [#3465](https://github.com/Urigo/accounter-fullstack/pull/3465)
[`89e36f3`](https://github.com/Urigo/accounter-fullstack/commit/89e36f3f76ac3a1c315bd305d55c0f1704efe8ac)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency [`msw@2.14.2` ↗︎](https://www.npmjs.com/package/msw/v/2.14.2) (from `2.13.6`,
in `dependencies`)
- [#3481](https://github.com/Urigo/accounter-fullstack/pull/3481)
[`9fcf835`](https://github.com/Urigo/accounter-fullstack/commit/9fcf8357867e437f39c33fc7c056cdb3f749a5fc)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency [`zod@4.4.3` ↗︎](https://www.npmjs.com/package/zod/v/4.4.3) (from `4.3.6`, in
`dependencies`)
- [#3503](https://github.com/Urigo/accounter-fullstack/pull/3503)
[`dab1752`](https://github.com/Urigo/accounter-fullstack/commit/dab17529ee68305203d59d8a195763191baea3d5)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency [`msw@2.14.5` ↗︎](https://www.npmjs.com/package/msw/v/2.14.5) (from `2.14.3`,
in `dependencies`)
- [#3506](https://github.com/Urigo/accounter-fullstack/pull/3506)
[`b68dc3a`](https://github.com/Urigo/accounter-fullstack/commit/b68dc3ab5b3653de616077734ffa6ae9401d4307)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency [`msw@2.14.5` ↗︎](https://www.npmjs.com/package/msw/v/2.14.5) (from `2.14.3`,
in `dependencies`)
- [#3517](https://github.com/Urigo/accounter-fullstack/pull/3517)
[`771084b`](https://github.com/Urigo/accounter-fullstack/commit/771084ba04be2aa0cf42904ab5591534b60af680)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency [`msw@2.14.6` ↗︎](https://www.npmjs.com/package/msw/v/2.14.6) (from `2.14.5`,
in `dependencies`)
- [#3524](https://github.com/Urigo/accounter-fullstack/pull/3524)
[`dda0668`](https://github.com/Urigo/accounter-fullstack/commit/dda066884da59c5b3a1b962a1d8e1364972e965d)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency
[`fast-xml-parser@5.8.0` ↗︎](https://www.npmjs.com/package/fast-xml-parser/v/5.8.0) (from
`5.7.3`, in `dependencies`)
- [#3534](https://github.com/Urigo/accounter-fullstack/pull/3534)
[`54c0e4b`](https://github.com/Urigo/accounter-fullstack/commit/54c0e4b840e1efea5744647745a1b031cfa0f5de)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency [`ws@8.20.1` ↗︎](https://www.npmjs.com/package/ws/v/8.20.1) (from `8.20.0`, in
`dependencies`)
- [#3548](https://github.com/Urigo/accounter-fullstack/pull/3548)
[`c6555f5`](https://github.com/Urigo/accounter-fullstack/commit/c6555f5d6a731b05f26ae58a47c485a905e4ec55)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency [`date-fns@4.2.1` ↗︎](https://www.npmjs.com/package/date-fns/v/4.2.1) (from
`4.1.0`, in `dependencies`)
- [#3551](https://github.com/Urigo/accounter-fullstack/pull/3551)
[`9d9ce57`](https://github.com/Urigo/accounter-fullstack/commit/9d9ce57f4058c0d66b8c1ea383030a57692fe607)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency [`date-fns@4.2.1` ↗︎](https://www.npmjs.com/package/date-fns/v/4.2.1) (from
`4.1.0`, in `dependencies`)
- [#3559](https://github.com/Urigo/accounter-fullstack/pull/3559)
[`43b3575`](https://github.com/Urigo/accounter-fullstack/commit/43b3575f981c01e3caafc7170cb30358ed227c3b)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency [`date-fns@4.3.0` ↗︎](https://www.npmjs.com/package/date-fns/v/4.3.0) (from
`4.2.1`, in `dependencies`)
- [#3561](https://github.com/Urigo/accounter-fullstack/pull/3561)
[`4298298`](https://github.com/Urigo/accounter-fullstack/commit/429829867e1f9c70619c6a18d8bd9dee8cab1ea4)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency [`ws@8.21.0` ↗︎](https://www.npmjs.com/package/ws/v/8.21.0) (from `8.20.1`, in
`dependencies`)
- [#3603](https://github.com/Urigo/accounter-fullstack/pull/3603)
[`cca7787`](https://github.com/Urigo/accounter-fullstack/commit/cca778797ff2252e52bf9dcf0777e2d385ac68d3)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency [`date-fns@4.4.0` ↗︎](https://www.npmjs.com/package/date-fns/v/4.4.0) (from
`4.3.0`, in `dependencies`)
- [#3613](https://github.com/Urigo/accounter-fullstack/pull/3613)
[`7397aad`](https://github.com/Urigo/accounter-fullstack/commit/7397aad6a9bf7d120218e6a8274a539eb1e76160)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency [`react@19.2.7` ↗︎](https://www.npmjs.com/package/react/v/19.2.7) (from
`19.2.6`, in `dependencies`)
- Updated dependency [`react-dom@19.2.7` ↗︎](https://www.npmjs.com/package/react-dom/v/19.2.7)
(from `19.2.6`, in `dependencies`)
- [#3687](https://github.com/Urigo/accounter-fullstack/pull/3687)
[`2130c0d`](https://github.com/Urigo/accounter-fullstack/commit/2130c0d38bd8f53b16a8d95a1eb87d363d6644ab)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency
[`fast-xml-parser@5.9.0` ↗︎](https://www.npmjs.com/package/fast-xml-parser/v/5.9.0) (from
`5.8.0`, in `dependencies`)
- [#3720](https://github.com/Urigo/accounter-fullstack/pull/3720)
[`452d0ed`](https://github.com/Urigo/accounter-fullstack/commit/452d0ede302ac26f9aad5d76afc53e72d6d99977)
Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates:
- Updated dependency
[`fast-xml-parser@5.9.3` ↗︎](https://www.npmjs.com/package/fast-xml-parser/v/5.9.3) (from
`5.9.0`, in `dependencies`)
- [#3554](https://github.com/Urigo/accounter-fullstack/pull/3554)
[`5d1b2a3`](https://github.com/Urigo/accounter-fullstack/commit/5d1b2a373dd76f12186fff325d57d80bfd286fec)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - - **New Scraper Implementation**: Added a
comprehensive scraper for Otsar Hahayal, supporting ILS, foreign currency, and credit card
transactions.
- **Infrastructure Integration**: Integrated the new scraper into the existing system, including
GraphQL mutation updates and UI configuration support.
- [#3463](https://github.com/Urigo/accounter-fullstack/pull/3463)
[`2d320ea`](https://github.com/Urigo/accounter-fullstack/commit/2d320ea2340a57b9bac1b8f597a2a16aff61cf5b)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - - **Granular Progress Events**: Replaced
legacy progress messages with 10 precisely-typed WebSocket events, providing real-time visibility
into per-month and per-account scraping progress.
- **Non-Fatal Scrape Loops**: Updated Isracard, Amex, Discount, and Cal scrapers to use a
non-fatal month-by-month loop, allowing tasks to continue even if a specific month fails.
- **UI Progress Tracking**: Added a new TaskSteps sub-component to the UI, enabling users to
monitor granular progress via collapsible sub-rows for months or accounts.
- **Poalim Account Discovery**: Implemented per-account and per-transaction-type progress
reporting for Poalim, including vault status checks.
- [#3515](https://github.com/Urigo/accounter-fullstack/pull/3515)
[`0c2b8d1`](https://github.com/Urigo/accounter-fullstack/commit/0c2b8d1076b5ca1a478d21f98f64d9db723358a7)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - - **New Vault API Endpoints**: Added GET
/api/vault/env-path, GET /api/vault/download, and POST /api/vault/upload to support vault file
management, including atomic writes and binary streaming.
- **Vault Path Configuration**: Introduced a configurable vaultPath in SettingsSchema, allowing
users to specify and update the storage location of their vault file.
- **Atomic Vault Migration**: Implemented moveVaultFile in the server store to atomically relocate
the vault file on disk when the vaultPath setting is updated.
- **UI Integration**: Added file upload functionality to both the setup and unlock screens, and
enabled vault downloading and path editing in the settings tab.
- [#3474](https://github.com/Urigo/accounter-fullstack/pull/3474)
[`1e70928`](https://github.com/Urigo/accounter-fullstack/commit/1e709282f39c9f722bb3ce5289a8edcda826012c)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - - Emit detailed currency rates and Max
scrapers task status
- For Max accounts, the websocket logic was updated to process accounts individually
- [#3459](https://github.com/Urigo/accounter-fullstack/pull/3459)
[`6bd3b40`](https://github.com/Urigo/accounter-fullstack/commit/6bd3b40e1d128429e2fdcf8c06a16af2fbabe3a4)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - - **Scraper App Package**: Added the new
`@accounter-helper/scraper-app` package to the monorepo, including its build configuration and
initial server/UI structure.
- **Vault Configuration**: Updated the codegen configuration to include the new scraper app's
GraphQL mutations and added a comprehensive implementation blueprint for the refactor.
- [#3627](https://github.com/Urigo/accounter-fullstack/pull/3627)
[`4f0c31c`](https://github.com/Urigo/accounter-fullstack/commit/4f0c31c03761d0f915ddbdd5104bcaec0a042dfe)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - - Updated `parseForeignXls` to support
`subAccount` identification, ensuring foreign transactions are correctly associated with their
respective sub-accounts.
- Improved browser launch configuration to explicitly check for `headless` mode.
- Updated types to include `subAccount` in `ForeignAccountMetadata`.
- **Database Migrations:**
- Added a new migration to flag fees in Otsar Hahayal transactions (ILS, foreign, and credit
card) based on description and name patterns.
- Updated the migration runner to include the new fee-flagging migration.
- [#3466](https://github.com/Urigo/accounter-fullstack/pull/3466)
[`249ad87`](https://github.com/Urigo/accounter-fullstack/commit/249ad87809e104c6c0ffc02971f3cee5dac478b1)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - - **Scraper Enhancements**: Updated
currency rate and Poalim scrapers to accept specific date ranges, improving flexibility and data
accuracy.
- **Transaction Ingestion Logic**: Implemented advanced transaction diffing and change detection
for Isracard, Amex, and currency rates to identify updates to existing records rather than just
inserting new ones.
- **Data Type Normalization**: Standardized card identifiers to strings across the ingestion
pipeline and updated GraphQL schemas to reflect these changes.
- [#3497](https://github.com/Urigo/accounter-fullstack/pull/3497)
[`ce62b25`](https://github.com/Urigo/accounter-fullstack/commit/ce62b2576b1bfc86344fe40e272f58cd3b45d956)
Thanks [@gilgardosh](https://github.com/gilgardosh)! - Enhance OTP dialog: enable manual closing,
auto-close on done
- Updated dependencies
[[`b4d8625`](https://github.com/Urigo/accounter-fullstack/commit/b4d8625fcdec44818ffc81f363eab652074a2fa6),
[`82f07c2`](https://github.com/Urigo/accounter-fullstack/commit/82f07c25cd88fbcef8d29ef3d32230cfce168caa),
[`a5e6d19`](https://github.com/Urigo/accounter-fullstack/commit/a5e6d19bf96173dde9097985a8df0065b9c3e499),
[`a92aeb2`](https://github.com/Urigo/accounter-fullstack/commit/a92aeb20551558fc2e32d9ac427e47eab52ddb52),
[`f94b2da`](https://github.com/Urigo/accounter-fullstack/commit/f94b2dadb2431f5a386217d716b72c15cff0ddd0),
[`46f9fc5`](https://github.com/Urigo/accounter-fullstack/commit/46f9fc5592590a765770592efab440a0374e9bde),
[`ba0531a`](https://github.com/Urigo/accounter-fullstack/commit/ba0531acbe4936cca371e553cfe7120329acec25),
[`1ad8a10`](https://github.com/Urigo/accounter-fullstack/commit/1ad8a10f3dd92b38c04ccea15b90cc7f9dc43887),
[`0b654ef`](https://github.com/Urigo/accounter-fullstack/commit/0b654ef08495e1632ccc75942d6a1a3142924d9f),
[`e76cee0`](https://github.com/Urigo/accounter-fullstack/commit/e76cee0916c6a9f6cc445d228642bc3caf767f86),
[`771084b`](https://github.com/Urigo/accounter-fullstack/commit/771084ba04be2aa0cf42904ab5591534b60af680),
[`b7634ad`](https://github.com/Urigo/accounter-fullstack/commit/b7634ad71fa879662cbcbc2ccca796016dc3f8e1),
[`c6555f5`](https://github.com/Urigo/accounter-fullstack/commit/c6555f5d6a731b05f26ae58a47c485a905e4ec55),
[`9d9ce57`](https://github.com/Urigo/accounter-fullstack/commit/9d9ce57f4058c0d66b8c1ea383030a57692fe607),
[`ae5aa31`](https://github.com/Urigo/accounter-fullstack/commit/ae5aa311e4beb3d5b9ded7a0c0fadaf00b796e2e),
[`5d1b2a3`](https://github.com/Urigo/accounter-fullstack/commit/5d1b2a373dd76f12186fff325d57d80bfd286fec),
[`43b3575`](https://github.com/Urigo/accounter-fullstack/commit/43b3575f981c01e3caafc7170cb30358ed227c3b),
[`5d9a79a`](https://github.com/Urigo/accounter-fullstack/commit/5d9a79aab5c5b75cf09c2dcc0df7f76513a05ba9),
[`f7a8d24`](https://github.com/Urigo/accounter-fullstack/commit/f7a8d244660fb59627da16b90ef2263fa8aa7718),
[`cca7787`](https://github.com/Urigo/accounter-fullstack/commit/cca778797ff2252e52bf9dcf0777e2d385ac68d3),
[`7397aad`](https://github.com/Urigo/accounter-fullstack/commit/7397aad6a9bf7d120218e6a8274a539eb1e76160),
[`fb74946`](https://github.com/Urigo/accounter-fullstack/commit/fb7494648b7e0a73d72e802c4dbff791e3583767),
[`452d0ed`](https://github.com/Urigo/accounter-fullstack/commit/452d0ede302ac26f9aad5d76afc53e72d6d99977),
[`64119ef`](https://github.com/Urigo/accounter-fullstack/commit/64119ef336f825399c7cf46bdc33e3f7ae1009dd),
[`5d1b2a3`](https://github.com/Urigo/accounter-fullstack/commit/5d1b2a373dd76f12186fff325d57d80bfd286fec),
[`4f0c31c`](https://github.com/Urigo/accounter-fullstack/commit/4f0c31c03761d0f915ddbdd5104bcaec0a042dfe)]:
- @accounter/modern-poalim-scraper@0.10.6