UNPKG

@accounter/scraper-app

Version:

Scraper app with Fastify server and React UI

github.com/Urigo/accounter-fullstack/packages/scraper-app

Urigo/accounter-fullstack

131 lines (113 loc) • 8.96 kB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
# @accounter/scraper-app ## 0.0.2-alpha-20260520080303-e6c2745307f7481b06a9ea46607cbcea363ff3e9 ### Patch Changes - [#3454](https://github.com/Urigo/accounter-fullstack/pull/3454) [`0b654ef`](https://github.com/Urigo/accounter-fullstack/commit/0b654ef08495e1632ccc75942d6a1a3142924d9f) Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates: - Updated dependency [`fast-xml-parser@5.7.3` ↗︎](https://www.npmjs.com/package/fast-xml-parser/v/5.7.3) (from `5.7.2`, in `dependencies`) - Updated dependency [`msw@2.14.3` ↗︎](https://www.npmjs.com/package/msw/v/2.14.3) (from `2.14.2`, in `dependencies`) - Updated dependency [`react@19.2.6` ↗︎](https://www.npmjs.com/package/react/v/19.2.6) (from `19.2.5`, in `dependencies`) - Updated dependency [`react-dom@19.2.6` ↗︎](https://www.npmjs.com/package/react-dom/v/19.2.6) (from `19.2.5`, in `dependencies`) - [#3465](https://github.com/Urigo/accounter-fullstack/pull/3465) [`89e36f3`](https://github.com/Urigo/accounter-fullstack/commit/89e36f3f76ac3a1c315bd305d55c0f1704efe8ac) Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates: - Updated dependency [`msw@2.14.2` ↗︎](https://www.npmjs.com/package/msw/v/2.14.2) (from `2.13.6`, in `dependencies`) - [#3481](https://github.com/Urigo/accounter-fullstack/pull/3481) [`9fcf835`](https://github.com/Urigo/accounter-fullstack/commit/9fcf8357867e437f39c33fc7c056cdb3f749a5fc) Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates: - Updated dependency [`zod@4.4.3` ↗︎](https://www.npmjs.com/package/zod/v/4.4.3) (from `4.3.6`, in `dependencies`) - [#3503](https://github.com/Urigo/accounter-fullstack/pull/3503) [`dab1752`](https://github.com/Urigo/accounter-fullstack/commit/dab17529ee68305203d59d8a195763191baea3d5) Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates: - Updated dependency [`msw@2.14.5` ↗︎](https://www.npmjs.com/package/msw/v/2.14.5) (from `2.14.3`, in `dependencies`) - [#3506](https://github.com/Urigo/accounter-fullstack/pull/3506) [`b68dc3a`](https://github.com/Urigo/accounter-fullstack/commit/b68dc3ab5b3653de616077734ffa6ae9401d4307) Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates: - Updated dependency [`msw@2.14.5` ↗︎](https://www.npmjs.com/package/msw/v/2.14.5) (from `2.14.3`, in `dependencies`) - [#3517](https://github.com/Urigo/accounter-fullstack/pull/3517) [`771084b`](https://github.com/Urigo/accounter-fullstack/commit/771084ba04be2aa0cf42904ab5591534b60af680) Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates: - Updated dependency [`msw@2.14.6` ↗︎](https://www.npmjs.com/package/msw/v/2.14.6) (from `2.14.5`, in `dependencies`) - [#3524](https://github.com/Urigo/accounter-fullstack/pull/3524) [`dda0668`](https://github.com/Urigo/accounter-fullstack/commit/dda066884da59c5b3a1b962a1d8e1364972e965d) Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates: - Updated dependency [`fast-xml-parser@5.8.0` ↗︎](https://www.npmjs.com/package/fast-xml-parser/v/5.8.0) (from `5.7.3`, in `dependencies`) - [#3534](https://github.com/Urigo/accounter-fullstack/pull/3534) [`54c0e4b`](https://github.com/Urigo/accounter-fullstack/commit/54c0e4b840e1efea5744647745a1b031cfa0f5de) Thanks [@renovate](https://github.com/apps/renovate)! - dependencies updates: - Updated dependency [`ws@8.20.1` ↗︎](https://www.npmjs.com/package/ws/v/8.20.1) (from `8.20.0`, in `dependencies`) - [#3463](https://github.com/Urigo/accounter-fullstack/pull/3463) [`2d320ea`](https://github.com/Urigo/accounter-fullstack/commit/2d320ea2340a57b9bac1b8f597a2a16aff61cf5b) Thanks [@gilgardosh](https://github.com/gilgardosh)! - - **Granular Progress Events**: Replaced legacy progress messages with 10 precisely-typed WebSocket events, providing real-time visibility into per-month and per-account scraping progress. - **Non-Fatal Scrape Loops**: Updated Isracard, Amex, Discount, and Cal scrapers to use a non-fatal month-by-month loop, allowing tasks to continue even if a specific month fails. - **UI Progress Tracking**: Added a new TaskSteps sub-component to the UI, enabling users to monitor granular progress via collapsible sub-rows for months or accounts. - **Poalim Account Discovery**: Implemented per-account and per-transaction-type progress reporting for Poalim, including vault status checks. - [#3515](https://github.com/Urigo/accounter-fullstack/pull/3515) [`0c2b8d1`](https://github.com/Urigo/accounter-fullstack/commit/0c2b8d1076b5ca1a478d21f98f64d9db723358a7) Thanks [@gilgardosh](https://github.com/gilgardosh)! - - **New Vault API Endpoints**: Added GET /api/vault/env-path, GET /api/vault/download, and POST /api/vault/upload to support vault file management, including atomic writes and binary streaming. - **Vault Path Configuration**: Introduced a configurable vaultPath in SettingsSchema, allowing users to specify and update the storage location of their vault file. - **Atomic Vault Migration**: Implemented moveVaultFile in the server store to atomically relocate the vault file on disk when the vaultPath setting is updated. - **UI Integration**: Added file upload functionality to both the setup and unlock screens, and enabled vault downloading and path editing in the settings tab. - [#3474](https://github.com/Urigo/accounter-fullstack/pull/3474) [`1e70928`](https://github.com/Urigo/accounter-fullstack/commit/1e709282f39c9f722bb3ce5289a8edcda826012c) Thanks [@gilgardosh](https://github.com/gilgardosh)! - - Emit detailed currency rates and Max scrapers task status - For Max accounts, the websocket logic was updated to process accounts individually - [#3459](https://github.com/Urigo/accounter-fullstack/pull/3459) [`6bd3b40`](https://github.com/Urigo/accounter-fullstack/commit/6bd3b40e1d128429e2fdcf8c06a16af2fbabe3a4) Thanks [@gilgardosh](https://github.com/gilgardosh)! - - **Scraper App Package**: Added the new `@accounter-helper/scraper-app` package to the monorepo, including its build configuration and initial server/UI structure. - **Vault Configuration**: Updated the codegen configuration to include the new scraper app's GraphQL mutations and added a comprehensive implementation blueprint for the refactor. - [#3466](https://github.com/Urigo/accounter-fullstack/pull/3466) [`249ad87`](https://github.com/Urigo/accounter-fullstack/commit/249ad87809e104c6c0ffc02971f3cee5dac478b1) Thanks [@gilgardosh](https://github.com/gilgardosh)! - - **Scraper Enhancements**: Updated currency rate and Poalim scrapers to accept specific date ranges, improving flexibility and data accuracy. - **Transaction Ingestion Logic**: Implemented advanced transaction diffing and change detection for Isracard, Amex, and currency rates to identify updates to existing records rather than just inserting new ones. - **Data Type Normalization**: Standardized card identifiers to strings across the ingestion pipeline and updated GraphQL schemas to reflect these changes. - [#3497](https://github.com/Urigo/accounter-fullstack/pull/3497) [`ce62b25`](https://github.com/Urigo/accounter-fullstack/commit/ce62b2576b1bfc86344fe40e272f58cd3b45d956) Thanks [@gilgardosh](https://github.com/gilgardosh)! - Enhance OTP dialog: enable manual closing, auto-close on done - Updated dependencies [[`b4d8625`](https://github.com/Urigo/accounter-fullstack/commit/b4d8625fcdec44818ffc81f363eab652074a2fa6), [`82f07c2`](https://github.com/Urigo/accounter-fullstack/commit/82f07c25cd88fbcef8d29ef3d32230cfce168caa), [`a5e6d19`](https://github.com/Urigo/accounter-fullstack/commit/a5e6d19bf96173dde9097985a8df0065b9c3e499), [`a92aeb2`](https://github.com/Urigo/accounter-fullstack/commit/a92aeb20551558fc2e32d9ac427e47eab52ddb52), [`f94b2da`](https://github.com/Urigo/accounter-fullstack/commit/f94b2dadb2431f5a386217d716b72c15cff0ddd0), [`46f9fc5`](https://github.com/Urigo/accounter-fullstack/commit/46f9fc5592590a765770592efab440a0374e9bde), [`ba0531a`](https://github.com/Urigo/accounter-fullstack/commit/ba0531acbe4936cca371e553cfe7120329acec25), [`1ad8a10`](https://github.com/Urigo/accounter-fullstack/commit/1ad8a10f3dd92b38c04ccea15b90cc7f9dc43887), [`0b654ef`](https://github.com/Urigo/accounter-fullstack/commit/0b654ef08495e1632ccc75942d6a1a3142924d9f), [`e76cee0`](https://github.com/Urigo/accounter-fullstack/commit/e76cee0916c6a9f6cc445d228642bc3caf767f86), [`771084b`](https://github.com/Urigo/accounter-fullstack/commit/771084ba04be2aa0cf42904ab5591534b60af680), [`b7634ad`](https://github.com/Urigo/accounter-fullstack/commit/b7634ad71fa879662cbcbc2ccca796016dc3f8e1)]: - @accounter/modern-poalim-scraper@0.10.6-alpha-20260520080303-e6c2745307f7481b06a9ea46607cbcea363ff3e9