UNPKG

datapilot-cli

Version:

Enterprise-grade streaming multi-format data analysis with comprehensive statistical insights and intelligent relationship detection - supports CSV, JSON, Excel, TSV, Parquet - memory-efficient, cross-platform

github.com/Mrassimo/datapilot

Mrassimo/datapilot

237 lines (143 loc) • 10.8 kB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
## [1.7.1](https://github.com/Mrassimo/datapilot/compare/v1.7.0...v1.7.1) (2025-07-06) ### Bug Fixes * add missing package-lock.json for CI/CD pipeline ([feb6242](https://github.com/Mrassimo/datapilot/commit/feb6242c8f8aa87e4c98f3f1927799a94dd224ab)) * comprehensive CLI testing and critical bug fixes ([0a3cfb4](https://github.com/Mrassimo/datapilot/commit/0a3cfb411413abae2514200b8ed381497c967b06)) * Enhance Jest configuration for CI compatibility and fix E2E test timeouts ([54c5231](https://github.com/Mrassimo/datapilot/commit/54c5231e63d1ebc61144d06bde7dd2c31aca8f01)) * Move build step before tests in CI workflow to fix E2E test failures ([875968c](https://github.com/Mrassimo/datapilot/commit/875968cb3ca36f17e4703116a24b9b554b1ec975)) * update hardcoded version references for v1.7.0 ([85d5262](https://github.com/Mrassimo/datapilot/commit/85d526224f75a94561af8d2331d8be1228cdb498)) # [1.7.0](https://github.com/Mrassimo/datapilot/compare/v1.6.7...v1.7.0) (2025-07-06) ### Bug Fixes * resolve Jest worker crashes in CLI integration tests ([a97ca74](https://github.com/Mrassimo/datapilot/commit/a97ca748587ccd2e758410b6f525a0dc527fde24)) ### Features * comprehensive testing and cleanup improvements ([8ce02ab](https://github.com/Mrassimo/datapilot/commit/8ce02ab57f42de3d0b01a94f157a106d1c5e731a)) ## [1.6.6](https://github.com/Mrassimo/datapilot/compare/v1.6.5...v1.6.6) (2025-06-29) ### Bug Fixes * complete minor issue resolution for v1.6.5+ perfection ([5cd467b](https://github.com/Mrassimo/datapilot/commit/5cd467b8bd686240a12ffb9da71819f8941e1514)) * display actual sample values in Section 5 schema analysis ([27c0a24](https://github.com/Mrassimo/datapilot/commit/27c0a24a3c173b6e6554b387cce808844de21717)) ## [1.6.5](https://github.com/Mrassimo/datapilot/compare/v1.6.4...v1.6.5) (2025-06-29) ### Bug Fixes * calibrate Section 2 quality scoring thresholds for realistic assessment ([380691e](https://github.com/Mrassimo/datapilot/commit/380691e8ab1c14cd8ff9839ea79229d3267ca40a)) * resolve Section 5 schema accuracy issues - use real statistics ([2791eae](https://github.com/Mrassimo/datapilot/commit/2791eae1e88aea9947763c5caf22602669702ef6)) ## [1.6.4](https://github.com/Mrassimo/datapilot/compare/v1.6.3...v1.6.4) (2025-06-29) ### Bug Fixes * comprehensive resolution of inter-section uniqueness consistency bugs ([737d659](https://github.com/Mrassimo/datapilot/commit/737d65968b206c09344c4ca811a67bde919134f3)), closes [#46](https://github.com/Mrassimo/datapilot/issues/46) ## [1.6.3](https://github.com/Mrassimo/datapilot/compare/v1.6.2...v1.6.3) (2025-06-29) ### Bug Fixes * resolve inter-section data consistency bug ([#46](https://github.com/Mrassimo/datapilot/issues/46)) ([b1d5305](https://github.com/Mrassimo/datapilot/commit/b1d53058b64953d734fa90542fdb428bc69f04c8)) * update yaml-formatter test to expect version 1.6.2 ([adb00a6](https://github.com/Mrassimo/datapilot/commit/adb00a613d5c5b64d2c4f7b1703c837085a17958)) ## [1.6.2](https://github.com/Mrassimo/datapilot/compare/v1.6.1...v1.6.2) (2025-06-29) ### Bug Fixes * COMPLETE resolution of all 4 critical issues from review v1.6.1 ([64f2cdb](https://github.com/Mrassimo/datapilot/commit/64f2cdb21671ba2cc3efd006dc820abd38c0d821)), closes [#42](https://github.com/Mrassimo/datapilot/issues/42) [#43](https://github.com/Mrassimo/datapilot/issues/43) [#44](https://github.com/Mrassimo/datapilot/issues/44) [#45](https://github.com/Mrassimo/datapilot/issues/45) ## [1.6.1](https://github.com/Mrassimo/datapilot/compare/v1.6.0...v1.6.1) (2025-06-29) ### Bug Fixes * resolve 4 critical issues for v1.6.1 bug-fix release ([98380ec](https://github.com/Mrassimo/datapilot/commit/98380eccad33471112ceb904dcc6d79267b15e38)) # [1.6.0](https://github.com/Mrassimo/datapilot/compare/v1.5.0...v1.6.0) (2025-06-29) ### Features * consolidate CI and Release workflows into unified CI/CD pipeline ([ea2c045](https://github.com/Mrassimo/datapilot/commit/ea2c04568b6a493f709141ba67152a293dbfd10e)) # [1.5.0](https://github.com/Mrassimo/datapilot/compare/v1.4.17...v1.5.0) (2025-06-28) ### Features * implement enhanced error handling with stack traces and contextual debugging ([f7fee81](https://github.com/Mrassimo/datapilot/commit/f7fee816c8e8aabaf15d6ea7e9f175613585aa47)) ## [1.4.17](https://github.com/Mrassimo/datapilot/compare/v1.4.16...v1.4.17) (2025-06-28) ### Bug Fixes * COMPLETE Section 6 fix - all modeling task failures resolved ([7fe3541](https://github.com/Mrassimo/datapilot/commit/7fe3541fcb512bc46b2ce4af02b78a83192ba989)), closes [#36](https://github.com/Mrassimo/datapilot/issues/36) ## [1.4.16](https://github.com/Mrassimo/datapilot/compare/v1.4.15...v1.4.16) (2025-06-28) ### Bug Fixes * Correct tgz version mismatch in GitHub releases ([fca273e](https://github.com/Mrassimo/datapilot/commit/fca273e3f6043997dcc7068e83a8d549a04f6b5a)) ## [1.4.15](https://github.com/Mrassimo/datapilot/compare/v1.4.14...v1.4.15) (2025-06-28) ### Bug Fixes * CRITICAL - Section 6 ML task identification completely rewritten ([1d4c860](https://github.com/Mrassimo/datapilot/commit/1d4c8606aef3e01a0a09d08f22ab278a6e145ea6)), closes [#36](https://github.com/Mrassimo/datapilot/issues/36) ## [1.4.14](https://github.com/Mrassimo/datapilot/compare/v1.4.13...v1.4.14) (2025-06-28) ### Bug Fixes * resolve Issues [#34](https://github.com/Mrassimo/datapilot/issues/34) and [#35](https://github.com/Mrassimo/datapilot/issues/35) with critical runtime bug fixes ([75b7c72](https://github.com/Mrassimo/datapilot/commit/75b7c724b67d2e5f4714b358a4fa04c98c03a70e)), closes [#36](https://github.com/Mrassimo/datapilot/issues/36) ## [1.4.13](https://github.com/Mrassimo/datapilot/compare/v1.4.12...v1.4.13) (2025-06-28) ### Bug Fixes * optimize npm package size by excluding source maps and development files ([44ba35f](https://github.com/Mrassimo/datapilot/commit/44ba35fa91a685687df76c331e46689a1f9212c3)) * resolve 400+ files in releases by fixing semantic-release configuration ([1fcc844](https://github.com/Mrassimo/datapilot/commit/1fcc844eec6ca87bd91720198d374f1167be6ece)) ## [1.4.12](https://github.com/Mrassimo/datapilot/compare/v1.4.11...v1.4.12) (2025-06-28) ### Bug Fixes * resolve critical v1.4.11 issues and add automated tgz publishing ([3644926](https://github.com/Mrassimo/datapilot/commit/3644926feb7ff5af70e5f58d4df630599ac71052)), closes [#34](https://github.com/Mrassimo/datapilot/issues/34) [#35](https://github.com/Mrassimo/datapilot/issues/35) [#36](https://github.com/Mrassimo/datapilot/issues/36) ## [1.4.11](https://github.com/Mrassimo/datapilot/compare/v1.4.10...v1.4.11) (2025-06-28) ### Bug Fixes * add NPM token authentication test documentation ([c4771f7](https://github.com/Mrassimo/datapilot/commit/c4771f7f29aaf5745c8296b21bffd4e738d2552e)) * correct NPM token authentication in CI workflow ([eecdd29](https://github.com/Mrassimo/datapilot/commit/eecdd29fbf6a67f7ca00d41befacd7915ffeb340)) * test NPM token configuration with automated release ([e96186c](https://github.com/Mrassimo/datapilot/commit/e96186ccdea5266afa1b5eb7649c582b74457e33)) # Changelog All notable changes to DataPilot will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [1.4.0] - 2025-06-15 ### Added - **Major Feature**: Unsupervised ML recommendations system (GitHub Issue #22) - UnsupervisedAnalyzer with 1,875 lines of advanced ML capabilities - 5 types of synthetic target generation (clustering, outlier, composite, temporal, domain-derived) - AutoML platform recommendations (H2O, AutoGluon, etc.) - Feature engineering recipe automation - Section 6 now always provides modeling opportunities, never returns "0 tasks" - **Major Feature**: Smart performance auto-configuration (GitHub Issue #23) - SmartResourceManager for automatic system resource detection and optimization - SectionCacheManager with intelligent memory/disk caching and TTL management - CLI performance options: --auto-config, --preset, --threads, --cache, --streaming - 5 performance presets (ultra-large-files, large-files, balanced, speed-optimized, memory-constrained) - Performance dashboard (datapilot perf) and cache management (datapilot clear-cache) - Automatic performance optimization based on file size and system capabilities ### Technical Implementation - Added 4 new TypeScript files with 3,000+ lines of production-ready code - Full integration with existing DataPilot architecture and dependency injection - Comprehensive error handling, logging, and memory management - Zero TypeScript compilation errors, full type safety maintained - British English spelling consistency throughout - Backward compatibility preserved ### Fixed - ESLint empty block statement error in section-cache-manager - UnsupervisedAnalyzer column property access errors - CI/CD pipeline linting stage failures ## [1.3.3] - 2024-12-XX ### Fixed - Comprehensive CI/CD pipeline test failures resolution - E2E CLI command parsing errors - Windows privacy mode test compatibility across CI environments - Large file test performance optimization with caching system - Security vulnerability management (CVE-2023-30533, CVE-2024-22363) - GitHub Actions workflow permissions for binary uploads - Jest test infrastructure and coverage reporting ### Enhanced - Cross-platform testing compatibility (Windows, macOS, Linux) - Test performance with reduced file sizes and caching - Repository organization and file structure cleanup - GitHub Release automation for git tags ## [1.3.2] - 2024-12-XX ### Fixed - Critical test infrastructure issues - Jest functionality restoration - CI/CD pipeline resolution with 97.5% test success rate ## [1.3.1] - 2024-12-XX ### Added - Complete WCAG accessibility compliance implementation - Comprehensive visualization system with accessibility features - Advanced statistical analysis capabilities ### Fixed - Accessibility module test coverage - Visualization rendering performance - Output formatting consistency ## [1.2.0] - 2024-11-XX ### Added - Enhanced engineering analysis features - Multi-file relationship analysis - Streaming statistical computation engine - 6-section analysis pipeline architecture ### Enhanced - Memory management and resource optimization - Error handling and reporting system - Configuration system with environment overrides ## [1.1.x] - 2024-10-XX ### Added - Core CSV processing and analysis functionality - Basic CLI interface - Statistical analysis modules - Quality assessment features ## [1.0.x] - 2024-09-XX ### Added - Initial release of DataPilot - Basic CSV parsing and analysis - Command-line interface - Core statistical functions