@basetenlabs/performance-client/benchmark_results_nodejs.csv

This library provides a high-performance Node.js client for Baseten.co endpoints including embeddings, reranking, and classification. It was built for massive concurrent POST requests to any URL, also outside of baseten.co. The PerformanceClient is built

client,length,duration,maxCpu,avgCpu,readings,maxRam
"PerformanceClient",64,0.04548364100000003,0.9246000000000001,0.9246000000000001,0,0
"AsyncOpenAI",64,0.06454851300000018,5.1243,5.1243,0,0
"PerformanceClient",128,0.03271391899999981,1.3634,1.3634,0,0
"AsyncOpenAI",128,0.030114540000000033,1.3593,1.3593,0,0
"PerformanceClient",256,0.06927135699999963,3.6139,3.6139,0,0
"AsyncOpenAI",256,0.03783893699999999,3.9244999999999997,3.9244999999999997,0,0
"PerformanceClient",384,0.06793542900000012,4.325699999999999,4.325699999999999,0,0
"AsyncOpenAI",384,0.08412554199999976,7.8441,7.8441,0,0
"PerformanceClient",512,0.06311456700000008,4.9881,4.9881,0,0
"AsyncOpenAI",512,0.0791323110000003,7.1956999999999995,7.1956999999999995,0,0
"PerformanceClient",2048,0.22536020300000018,24.4927,24.4927,0,0
"AsyncOpenAI",2048,0.23796466699999974,36.0376,36.0376,2,239.51953125
"PerformanceClient",8192,0.8055539630000003,128.8116,128.8116,0,0
"AsyncOpenAI",8192,1.2579907789999998,150.9049,150.9049,6,411.46484375
"PerformanceClient",32768,4.564146041,648.3926,648.3926,0,0
"AsyncOpenAI",32768,3.635188868000001,472.697,472.697,10,1030.58984375
"PerformanceClient",131072,42.14760924,9293.930400000001,9293.930400000001,0,0
"AsyncOpenAI",131072,8.379361671000005,1067.0226,1067.0226,29,3253.23046875