UNPKG

arehs

Version:

The arehs ensures the best possible large batch processing, which is oriented towards event-driven chunk processing.

132 lines (97 loc) β€’ 5.33 kB
<p align="center"> <img src="https://github.com/seongjin605/arehs/blob/main/arehs.svg" width="400px" alt="Arehs Logo" /> <p> # πŸ›οΈ Arehs `μ•„λ ˆμŠ€`λŠ” 이벀트 μ€‘μ‹¬μ˜ chunk processing을 μ§€ν–₯ν•˜λŠ” μ΅œμƒμ˜ λŒ€κ·œλͺ¨ 배치 처리λ₯Ό 보μž₯ν•©λ‹ˆλ‹€. μ΄λŠ” 첫 번째 비동기 μž‘μ—… 호좜이 μ™„λ£Œλ  λ•ŒκΉŒμ§€ 기닀리지 μ•Šκ³  λ‹€μŒ 비동기 μž‘μ—… ν˜ΈμΆœμ„ μ¦‰μ‹œ ν• λ‹Ήν•˜μ—¬ μ‘°λ°€ν•˜κ²Œ ν”„λ‘œμ„ΈμŠ€λ₯Ό μˆ˜ν–‰ν•©λ‹ˆλ‹€. **이λ₯Ό 톡해 λ‹€μŒκ³Ό 같은 μ—¬λŸ¬ κ°€μ§€λ₯Ό 달성할 수 μžˆμŠ΅λ‹ˆλ‹€:** * ν”„λ‘œλ―ΈμŠ€ ν’€μ˜ λ™μ‹œμ„±μ„ μ„€μ •ν•˜μ—¬ μ„œλΉ„μŠ€ μ²˜λ¦¬λŸ‰μ„ μ œμ–΄ν•  수 μžˆμŠ΅λ‹ˆλ‹€. * ν”„λ‘œλ―ΈμŠ€ ν’€μ˜ λ™μ‹œμ„±μ„ μ„€μ •ν•˜μ—¬ λ‹€μš΄μŠ€νŠΈλ¦Ό μ„œλΉ„μŠ€μ˜ λΆ€ν•˜λ₯Ό κ΄€λ¦¬ν•©λ‹ˆλ‹€. * μ• ν”Œλ¦¬μΌ€μ΄μ…˜μ˜ μ„±λŠ₯ ν–₯상 * CPU 유휴 μ‹œκ°„ κ°μ†Œ λ“± ## πŸ“š Getting Started `arehs`λŠ” CommonJS와 ES Modulesλ₯Ό μ§€μ›ν•©λ‹ˆλ‹€. ### CommonJS ```javascript const { Arehs } = require("arehs"); ``` ### ES Modules ```javascript import { Arehs } from "arehs"; ``` ### Example * `create`: create λ©”μ„œλ“œμ˜ λͺ©μ μ€ νŠΉμ •ν•œ 데이터 λ°°μ—΄λ‘œλΆ€ν„° Arehs μΈμŠ€ν„΄μŠ€λ₯Ό μƒμ„±ν•˜κΈ° μœ„ν•¨μž…λ‹ˆλ‹€. * `withConcurrency`: 병렬 처리 값을 μ„€μ •ν•˜κ³  ν˜„μž¬ μΈμŠ€ν„΄μŠ€λ₯Ό λ°˜ν™˜ν•˜λŠ” λ©”μ„œλ“œμž…λ‹ˆλ‹€. (default: 10) * `timeoutLimit`:Default 값은 0 μž…λ‹ˆλ‹€. (0보닀 크면 μ˜΅μ…˜μ΄ λ™μž‘ν•˜λ©°, νƒ€μž„μ•„μ›ƒ μ‹œκ°„(ms)보닀 μž‘μ—…μ‹œκ°„μ΄ κΈΈλ©΄ μ—λŸ¬κ°€ λ°œμƒν•©λ‹ˆλ‹€. * `mapAsync`: mapAsync ν•¨μˆ˜λ₯Ό ν˜ΈμΆœν•˜λ©΄ μž…λ ₯ 데이터λ₯Ό λΉ„λ™κΈ°μ μœΌλ‘œ μ²˜λ¦¬ν•˜κ³  κ²°κ³Όλ₯Ό λ°˜ν™˜ν•˜λŠ” ν”„λ‘œμ„ΈμŠ€κ°€ μ‹œμž‘λ©λ‹ˆλ‹€. μ΄λ•Œ 각 μž‘μ—…μ€ λ™μ‹œμ— μ—¬λŸ¬ μž‘μ—…μ΄ 싀행될 수 μžˆμ§€λ§Œ, concurrency 섀정에 따라 μ œν•œλ©λ‹ˆλ‹€. 이것은 λŒ€κ·œλͺ¨ 데이터 처리 μž‘μ—…μ„ 효과적으둜 κ΄€λ¦¬ν•˜κ³  μ œμ–΄ν•˜κΈ° μœ„ν•œ μœ μš©ν•œ λ„κ΅¬λ‘œ μ‚¬μš©λ  수 μžˆμŠ΅λ‹ˆλ‹€. ```typescript import { Arehs } from "arehs"; const dataArr = [ { id: 1, name: "John" }, { id: 2, name: "Alice" }, { id: 3, name: "Bob" } ]; const result = await Arehs.create(dataArr) .withConcurrency(10) .mapAsync(async data => { return await someAsyncFunction(data); }); ``` ## ⚑️ Performance ν…ŒμŠ€νŠΈ κ²°κ³Ό `Arehs`λŠ” `Promise.all`에 λΉ„ν•΄ μ•½ 30% 이상 ν–₯상될 수 μžˆλŠ” κ²ƒμœΌλ‘œ λ‚˜νƒ€λ‚¬μŠ΅λ‹ˆλ‹€. ```javascript import { Arehs } from "arehs"; const delay = (i) => { return new Promise((res, rej) => { setTimeout(() => { res(i); }, 150 + Math.random() * 1000); }); }; (async () => { const tasks = Array.from({ length: 1000 }).map((d, i) => i); const startArehs = performance.now(); await Arehs.create(tasks).withConcurrency(50).mapAsync(delay); const endArehs = performance.now(); console.log(`Arehs: ${endArehs - startArehs}ms`); const startPromiseAll = performance.now(); while (tasks.length > 0) { const chunkedTasks = tasks.splice(0, 50); await Promise.all(chunkedTasks.map(delay)); } const endPromiseAll = performance.now(); console.log(`Promise.all: ${endPromiseAll - startPromiseAll}ms`); })(); ``` ```bash promiseAllTime: 19.859867874979972(s) promisePoolTime: 13.55725229203701(s) ``` ### Promise.all λ³΄μ‹œλ‹€μ‹œν”Ό, `Promise.all`은 λ°°μΉ˜μ—μ„œ κ°€μž₯ 느린 ν”„λ‘œλ―ΈμŠ€λ§ŒνΌ 였래 μ‹€ν–‰λ©λ‹ˆλ‹€. λ”°λΌμ„œ 메인 μŠ€λ ˆλ“œλŠ” 기본적으둜 "아무것도 ν•˜μ§€ μ•ŠλŠ”" μƒνƒœμ΄λ©° κ°€μž₯ 느린 μš”μ²­μ΄ μ™„λ£Œλ˜κΈ°λ₯Ό 기닀리고 μžˆμŠ΅λ‹ˆλ‹€. Promise λ°°μ—΄μ—μ„œ κ°€μž₯ κΈ΄ ν”„λ‘œλ―ΈμŠ€μΈ **4번**이 청크의 μ‹€ν–‰ μ‹œκ°„μ΄ λ©λ‹ˆλ‹€. 이둜 인해 κ°€μž₯ κΈ΄ ν”„λ‘œλ―ΈμŠ€κ°€ μ™„λ£Œλ  λ•ŒκΉŒμ§€ λ‹€μŒ ν”„λ‘œλ―ΈμŠ€κ°€ 아무 μž‘μ—…λ„ μˆ˜ν–‰ν•˜μ§€ μ•ŠλŠ” λΉ„νš¨μœ¨μ μΈ λ¬Έμ œκ°€ λ°œμƒν•©λ‹ˆλ‹€. <p align="center"> <img src="https://velog.velcdn.com/images/preciou_star/post/d35894aa-4ce4-4bf1-aa04-3486745964ed/image.webp" width="700px" alt="Code Crafters Logo" /> <p> ### Arehs `Arehs`λŠ” ν”„λ‘œλ―ΈμŠ€ ν’€ νŒ¨ν„΄μ„ μ‹€ν–‰ν•˜μ—¬ Node.js의 메인 μŠ€λ ˆλ“œλ₯Ό μ΅œλŒ€ν•œ ν™œμš©ν•˜λŠ” 것이 ν•΅μ‹¬μž…λ‹ˆλ‹€. ν™œμš©λ„λ₯Ό 높이렀면 API 호좜(λ˜λŠ” λ‹€λ₯Έ 비동기 μž‘μ—…)을 μ‘°λ°€ν•˜κ²Œ νŒ¨ν‚Ήν•˜μ—¬ κ°€μž₯ κΈ΄ 호좜이 μ™„λ£Œλ˜λŠ” λ™μ•ˆ 기닀리지 μ•Šλ„λ‘ ν•΄μ•Ό ν•©λ‹ˆλ‹€. 기닀리지 μ•Šκ³  첫 번째 호좜이 μ™„λ£Œλ˜λŠ” μ¦‰μ‹œ λ‹€μŒ ν˜ΈμΆœμ„ μ˜ˆμ•½ν•©λ‹ˆλ‹€. <p align="center"> <img src="https://velog.velcdn.com/images/preciou_star/post/e49061b9-f18a-4d59-8c6d-aaf240fd9085/image.webp" width="700px" alt="Code Crafters Logo" /> <p> ## πŸ™‹β€β™€οΈFAQ **`Arehs`λŠ” 항상 `Promise.all`보닀 μ’‹λ‚˜μš”?** 아뇨, **No Silver Bullet(μ€νƒ„ν™˜μ€ μ—†λ‹€)**. λ§Žμ€ API 호좜과 비동기 μž‘μ—…μ„ ν•  λ•Œ μ• ν”Œλ¦¬μΌ€μ΄μ…˜μ˜ μ„±λŠ₯을 ν–₯μƒμ‹œν‚¬ 수 μžˆμŠ΅λ‹ˆλ‹€. λ˜ν•œ, 각 ν”„λ‘œλ―ΈμŠ€μ˜ μž‘μ—… μ‹œκ°„μ΄ 거의 λ™μΌν•œ μƒν™©μ—μ„œλŠ” 큰 차이λ₯Ό λ§Œλ“€μ§€ λͺ»ν•  μˆ˜λ„ μžˆμŠ΅λ‹ˆλ‹€. μ‚¬μš© 쀑인 ν™˜κ²½μ—μ„œ `Promise.all`둜 더 이상 μ„±λŠ₯ ν–₯상을 얻을 수 μ—†λ‹€λ©΄ μ‹œλ„ν•΄ λ³Ό μˆ˜λŠ” μžˆμ§€λ§Œ, `Promise.all`둜 μΆ©λΆ„ν•˜λ‹€λ©΄ ꡳ이 μ‚¬μš©ν•  ν•„μš”λŠ” μ—†μŠ΅λ‹ˆλ‹€. λ”°λΌμ„œ μ„±λŠ₯ κ°œμ„ μ΄ ν•„μš”ν•œ ν”„λ‘œμ νŠΈμ— `Arehs`λ₯Ό μ‚¬μš©ν•˜λ €λ©΄ μΆ©λΆ„ν•œ ν…ŒμŠ€νŠΈλ₯Ό 거친 후에 μ‚¬μš©ν•΄ λ³΄μ„Έμš”. 도움이 되길 λ°”λžλ‹ˆλ‹€. κ°μ‚¬ν•©λ‹ˆλ‹€. ## πŸ‘¨β€πŸ‘©β€πŸ‘§β€πŸ‘¦ Contributors - Author: Jin Park