msnodesqlv8
Version:
Microsoft Driver for Node.js SQL Server compatible with all versions of Node.
1,612 lines (1,219 loc) • 94 kB
Markdown
# NODE-SQLSERVER-V8 #
[](https://ci.appveyor.com/project/TimelordUK/node-sqlserver-v8/branch/master)
[](https://github.com/TimelordUK/node-sqlserver-v8/stargazers)
[](https://github.com/TimelordUK/node-sqlserver-v8/issues)
[]
[]
1. [What is this library for?](#what-is-library-for)
1. [What Platforms does it support?](#supported-platforms)
1. [Running on macOS (darwin)](#darwin)
1. [Docker images for Ubuntu, Debian and Alpine](#docker-images)
1. [Linux troubleshoot SEGV (Ubuntu, Debian)](#ubuntu)
1. [Sequelize Compatibility?](#sequelize-compatibility)
1. [Is the library production quality?](#production-quality)
1. [How to install](#install)
1. [Does this library support electron?](#electron-support)
1. [electron troubleshoot](#electron-trouble-shoot)
1. [How do I set the app name on connection string?](#set-app-name)
1. [How does the driver handle SQL_VARIANT types](#handling-variant)
1. [Errors when passing strings > 2k in length.](#long-strings)
1. [Api](#api)
1. [Read BigInt numbers as JS strings](#bigint-strings)
1. [Sybase Adaptive Server](#sybase-adaptive-server)
1. [thread pooling](#thread-pooling)
1. [promises](#promises)
1. [Table Value Parameters TVP](#table-value-parameters)
1. [User Binding](#user-binding)
1. [Async Patterns](#async-patterns)
1. [Connecting](#connecting)
1. [ConnectionPool](#pool)
1. [Executing A Query](#executing-a-query)
1. [Geography Types](#geography)
1. [Pause A Query](#pause-a-query)
1. [Cancel A Query](#cancel-a-query)
1. [Subscribing To Driver Events](#subscribing-to-driver-events)
1. [Capturing Print Output](#capture-print)
1. [Register Stored Procedure](#register-stored-procedure)
1. [Executing Stored Procedures](#executing-stored-procedures)
1. [Table Binding](#table-binding)
1. [BCP](#bcp)
1. [Prepared Statements](#prepared-statements)
1. [Easiest Way To See Library Being Used](#easiest-way-to-see-library-being-used)
1. [Testing The Driver](#testing-the-driver)
1. [Compiling The Driver](#compiling-the-driver)
1. <a href="#setup">Setup a development environment</a>
## What Is Library For ##
This wiki describes [NodeJS](https://nodejs.org) npm module [msnodesqlv8](https://www.npmjs.com/package/msnodesqlv8). It has been forked from the Microsoft SQL server ODBC module msnodesql and has been reworked to be compatible with later versions of Node. New features are being introduced.
This npm module will allow a windows instance of Node JS to connect to a SQL server database and submit SQL queries via a javascript api.
The module is a mixture of C++ and javascript. Compiled binaries are supplied, therefore it should be up and running quickly for use against your database. Those familiar with the Microsoft original library will feel at home as the api is a super-set of existing functionality.
Please give the module a try and feel free to offer suggestions for improvement.
## Supported Platforms ##
[msnodesqlv8](https://www.npmjs.com/package/msnodesqlv8) will run on 32 bit and 64 bit versions of Node JS hosted on the Windows operating system. All major versions of Node are supported and tested i.e. >= 10 [download node here](https://nodejs.org/en/download/). [previous node releases](https://nodejs.org/en/download/releases/).
if running on Linux, the odbc driver needs to be installed as outlined here [ODBC 17](https://docs.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server?view=sql-server-ver15). Please use version >= 17.5 which has been tested with this library. We are running test suite for Linux on AppVeyor which you can see via the badge at top of this page. Linux distros tested so far are Ubuntu 18.04, Alpine 3.12, Ubuntu 20.04, Debian 10.5 and Fedora 32. The driver also works under windows linux subsystem 2 (WLS).
## docker images ##
cd docker\debian-msnodesqlv8\
from this folder build image
```bash
docker build . -t msnodesqlv8-debian
```
or with human readable log and clean rebuild
```
docker build --no-cache --progress=plain . 2>&1 | tee build.log
```
list the docker images
```
docker images
```
list these images only
```
docker images msnodesql*
```
REPOSITORY TAG IMAGE ID CREATED SIZE
msnodesqlv8-alpine latest 94aa468ed1b3 21 hours ago 1.25GB
msnodesqlv8-debian latest c1eee90b2e4d 26 hours ago 2.42GB
start an interactive shell in a new container
```
docker run -it msnodesqlv8-debian bash
```
go to driver folder git repo
```
cd ~/app/driver/node_modules/msnodesqlv8
```
change the connection string in
```
cat .env-cmdrc
```
exit shell
```
exit <ret>
```
list containers
```
docker container ls
```
start a container
```
docker start abc6f443950c
```
connect interactive shell to the container
```
docker exec -it upbeat_easley bash
```
list all container running or stopped
```
docker ps -a
```
## debian crash - ssl stack trace ##
if the library crashes with segmentation fault it is likely related to version of ssl
if you have access to brew
```bash
brew install openssl
cd /usr/lib
sudo ln -s /home/linuxbrew/.linuxbrew/opt/openssl@3/lib/libssl.so libssl.so.1.1
export LD_PRELOAD=/usr/lib/libssl.so.1.1
me@me-pc:~/dev/js/published/samples/javascript$ export LD_PRELOAD=/usr/lib/libssl.so.1.1
me@me-pc:~/dev/js/published/samples/javascript$ ps -ef | grep node
me 6350 1576 54 15:57 pts/0 00:00:39 node pooling.js
me 6634 6370 0 15:58 pts/1 00:00:00 grep node
me@me-pc:~/dev/js/published/samples/javascript$ pldd 6350
6350: /home/me/.nvm/versions/node/v16.13.0/bin/node
linux-vdso.so.1
/usr/lib/libssl.so.1.1
/lib/x86_64-linux-gnu/libdl.so.2
/lib/x86_64-linux-gnu/libstdc++.so.6
/lib/x86_64-linux-gnu/libm.so.6
/lib/x86_64-linux-gnu/libgcc_s.so.1
/lib/x86_64-linux-gnu/libpthread.so.0
/lib/x86_64-linux-gnu/libc.so.6
/lib64/ld-linux-x86-64.so.2
/home/linuxbrew/.linuxbrew/Cellar/openssl@1.1/1.1.1l_1/lib/libcrypto.so.1.1
/home/me/dev/js/published/node_modules/msnodesqlv8/build/Release/sqlserverv8.node
/lib/x86_64-linux-gnu/libodbc.so.2
/usr/lib/x86_64-linux-gnu/gconv/UTF-16.so
/opt/microsoft/msodbcsql17/lib64/libmsodbcsql-17.8.so.1.1
/lib/x86_64-linux-gnu/librt.so.1
/lib/x86_64-linux-gnu/libodbcinst.so.2
/lib/x86_64-linux-gnu/libkrb5.so.3
/lib/x86_64-linux-gnu/libgssapi_krb5.so.2
/lib/x86_64-linux-gnu/libk5crypto.so.3
/lib/x86_64-linux-gnu/libcom_err.so.2
/lib/x86_64-linux-gnu/libkrb5support.so.0
/lib/x86_64-linux-gnu/libkeyutils.so.1
/lib/x86_64-linux-gnu/libresolv.so.2
/lib/x86_64-linux-gnu/libnss_files.so.2
me@me-pc:~/dev/js/published/samples/javascript$
me@me-pc:~/dev/js/published/samples/javascript$ uname -a
Linux me-pc 4.19.0-18-amd64 #1 SMP Debian 4.19.208-1 (2021-09-29) x86_64 GNU/Linux
me@me-pc:~/dev/js/published/samples/javascript$ node simple-demo.js
rows.length 36 elapsed 48
rows.length 36 elapsed 30
me@me-pc:~/dev/js/published/samples/javascript$
```
## darwin ##
Note the driver has been tested on macOS High Sierra. MacOS native binaries are now supplied and should be available via NPM
You need to install Xcode before running those steps
It is highly recommended to install nvm if not already present - launch a terminal
```bash
touch ~/.bash_profile # you need a profile even an empty one
curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.0/install.sh | bash
# profile should contain
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm
# restart terminal .... check nvm is installed
nvm --version
# install latest node e.g.
nvm install 15.5.0
# check node is installed
node --version
```
install g++ if not already present - below will invite you to install if not yet present.
```bash
g++ --version
```
Install Microsoft ODBC driver via HomeBrew
```bash
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
brew tap microsoft/mssql-release https://github.com/Microsoft/homebrew-mssql-release
brew update
# build odbc driver
HOMEBREW_NO_ENV_FILTERING=1 ACCEPT_EULA=Y brew install msodbcsql17 mssql-tools
```
fetch the node driver
```bash
# assume folder such as dev/js/sql/node_modules
git clone https://github.com/TimelordUK/node-sqlserver-v8.git msnodesqlv8
cd msnodesqlv8/
node-gyp rebuild
npm install
```
run a sample
```bash
cd samples/javascript
# edit the jaon file and set a valid connection - change local or add a new one and edit code
node throttle
```
```txt
stream fetch query with throttle back to server to slow results
on.submitted select top 750 * from master..syscolumns;
22:31:37 - [0] (rowCount 100): pause and dispatch 100 rows ...
22:31:37 - [0] (rowCount 100): ... done, resume query
22:31:39 - [1] (rowCount 200): pause and dispatch 100 rows ...
22:31:39 - [1] (rowCount 200): ... done, resume query
22:31:40 - [2] (rowCount 300): pause and dispatch 100 rows ...
22:31:40 - [2] (rowCount 300): ... done, resume query
22:31:42 - [3] (rowCount 400): pause and dispatch 100 rows ...
22:31:42 - [3] (rowCount 400): ... done, resume query
22:31:43 - [4] (rowCount 500): pause and dispatch 100 rows ...
22:31:43 - [4] (rowCount 500): ... done, resume query
22:31:45 - [5] (rowCount 600): pause and dispatch 100 rows ...
22:31:45 - [5] (rowCount 600): ... done, resume query
22:31:47 - [6] (rowCount 700): pause and dispatch 100 rows ...
22:31:47 - [6] (rowCount 700): ... done, resume query
on.done
[7] (rowCount 750): last dispatch 50 rows
done
```
## Ubuntu ##
tested platforms
| OS | Node Version | ODBC Version | SSL version | action |
|----------------|--------------------|--------------|----------------|----------------------|
| Windows 10 | 12, 14, 16, 18, 20 | 17, 18 | | leave default |
| Windows 11 | 18, 20 | 18 | | leave default |
| Ubuntu 22.04.2 | 18, 20 | 17, 18 | openssl 3.2 | e.g. tool/openssl.sh |
| Ubuntu 20.04 | 18, 20 | 18 | openssl 3.2 | e.g. tool/openssl.sh |
| Ubuntu 20.04 | 12, 14, 16 | 17, 18 | openssl 1.1 | leave default |
| Lubuntu 20.04 | 18, 20 | 18 | openssl 3.2 | e.g. tool/openssl.sh |
| Debian | 18, 20 | 17, 18 | openssl 3.2 | e.g tool/openssl.sh |
| Alpine 3.16 | 18 | 18 | openssl 1.1.1t | leave default |
| MacOS Big Sur | 18, 20 | 18 | libressl 2.8.3 | leave default |
for reference check [appveyor](https://ci.appveyor.com/project/TimelordUK/node-sqlserver-v8/branch/master) - note for Node 18, openssl 3.2 is installed to allow Node 18 and this driver to work.
you must have odbc installed on system
```sh
odbcinst -j
```
```txt
unixODBC 2.3.11
DRIVERS............: /etc/odbcinst.ini
SYSTEM DATA SOURCES: /etc/odbc.ini
FILE DATA SOURCES..: /etc/ODBCDataSources
USER DATA SOURCES..: /home/me/.odbc.ini
SQLULEN Size.......: 8
SQLLEN Size........: 8
SQLSETPOSIROW Size.: 8
```
```sh
sudo apt install unixodbc
sudo apt install odbcinst
```
ensure ms driver [odbc](https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server?view=sql-server-ver16&tabs=ubuntu18-install%2Calpine17-install%2Cdebian8-install%2Credhat7-13-install%2Crhel7-offline) is installed.
check you now have ODBC entry
```sh
cat /etc/odbcinst.ini
```
```txt
[ODBC Driver 18 for SQL Server]
Description=Microsoft ODBC Driver 18 for SQL Server
Driver=/opt/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1
UsageCount=1
```
check which platform version you are running on
```sh
lsb_release -rs
```
using appveyor as reference providing above are installed, then using Node 12,14,16 should work - appveyor test is not doing anything other than install the git branch and run.
if you see a SEGV running node versions <= 16 then try node 18,20. If one of these versions work you very likely have open ssl 3 installed
this has been set up below to show the problem
```sh
me@asuszen:~/dev/js/sql/v8/node_modules/msnodesqlv8$ node samples/javascript/streaming.js
Segmentation fault
me@asuszen:~/dev/js/sql/v8/node_modules/msnodesqlv8$ node --version
v16.16.0
```
look at the stack trace to see the violation occurs in openssl
```sh
gdb node
(gdb) run samples/javascript/streaming
Starting program: /home/me/.nvm/versions/node/v16.16.0/bin/node samples/javascript/streaming
warning: File "/usr/lib/x86_64-linux-gnu/libthread_db-1.0.so" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$
```
from within gdb having seen SEGV
```sh
Thread 8 "node" received signal SIGSEGV, Segmentation fault.
[Switching to LWP 31950]
0x000000000179eaf0 in EC_GROUP_order_bits ()
(gdb) bt
```
```txt
Thread 8 "node" received signal SIGSEGV, Segmentation fault.
[Switching to LWP 31950]
0x000000000179eaf0 in EC_GROUP_order_bits ()
(gdb) bt
#0 0x000000000179eaf0 in EC_GROUP_order_bits ()
#1 0x00007fffd631f60d in engine_unlocked_init () from /usr/local/ssl/lib64/libcrypto.so.3
#2 0x00007fffd631f796 in ENGINE_init () from /usr/local/ssl/lib64/libcrypto.so.3
#3 0x00007fffd636b7e7 in EVP_PKEY_CTX_new_from_pkey () from /usr/local/ssl/lib64/libcrypto.so.3
#4 0x00007fffd66f325b in ssl_setup_sigalgs () from /usr/local/ssl/lib64/libssl.so.3
#5 0x00007fffd66e858f in SSL_CTX_new_ex () from /usr/local/ssl/lib64/libssl.so.3
#6 0x00007ffff44aa4f7 in ?? () from /opt/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1
#7 0x00007ffff44a50d2 in ?? () from /opt/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1
#8 0x00007ffff44a599b in ?? () from /opt/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1
#9 0x00007ffff446f5cf in ?? () from /opt/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1
#10 0x00007ffff446c8cc in ?? () from /opt/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1
#11 0x00007ffff446d675 in ?? () from /opt/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1
#12 0x00007ffff446dcd4 in ?? () from /opt/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1
#13 0x00007ffff43db23c in ?? () from /opt/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1
#14 0x00007ffff440ff8e in ?? () from /opt/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1
#15 0x00007ffff43d9dfa in SQLDriverConnectW () from /opt/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1
#16 0x00007ffff4f952fd in SQLDriverConnectW (hdbc=0x7fffd0000bf0, hwnd=0x0, conn_str_in=0x4bb2e70, len_conn_str_in=110, conn_str_out=0x0,
conn_str_out_max=0, ptr_conn_str_out=0x0, driver_completion=0) at SQLDriverConnectW.c:776
#17 0x00007ffff521f003 in mssql::OdbcConnection::try_open(std::shared_ptr<std::vector<unsigned short, std::allocator<unsigned short> > >, int) () from /home/me/dev/js/sql/v8/node_modules/msnodesqlv8/build/Release/sqlserverv8.node
#18 0x00007ffff5231ffe in mssql::OpenOperation::TryInvokeOdbc() ()
from /home/me/dev/js/sql/v8/node_modules/msnodesqlv8/build/Release/sqlserverv8.node
#19 0x00007ffff522411f in mssql::OdbcOperation::Execute() ()
from /home/me/dev/js/sql/v8/node_modules/msnodesqlv8/build/Release/sqlserverv8.node
#20 0x0000000001560704 in worker (arg=0x0) at ../deps/uv/src/threadpool.c:122
#21 0x00007ffff7c51609 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#22 0x00007ffff7b76133 in clone () from /lib/x86_64-linux-gnu/libc.so.6
```
note in this case where ssl is being loaded from
```sh
/usr/local/ssl/lib64/
```
```sh
me@asuszen:~/dev/js/sql/v8/node_modules/msnodesqlv8$ /usr/local/ssl/bin/openssl version
OpenSSL 3.2.0-dev (Library: OpenSSL 3.2.0-dev )
```
we are running Node 16 on Ubuntu with ms driver ODBC 18 and openssl 3.2. This is not a compatible configuration. In this case we must use node 18 or 20 to fix the problem.
```sh
me@asuszen:~/dev/js/sql/v8/node_modules/msnodesqlv8$ nvm use 20.0.0
Now using node v20.0.0 (npm v9.6.4)
me@asuszen:~/dev/js/sql/v8/node_modules/msnodesqlv8$ npm install
> msnodesqlv8@3.1.0 install
> prebuild-install || node-gyp rebuild
up to date, audited 877 packages in 4s
132 packages are looking for funding
run `npm fund` for details
8 vulnerabilities (3 moderate, 5 high)
To address issues that do not require attention, run:
npm audit fix
To address all issues (including breaking changes), run:
npm audit fix --force
Run `npm audit` for details.
me@asuszen:~/dev/js/sql/v8/node_modules/msnodesqlv8$ node samples/javascript/streaming.js
run with Driver={ODBC Driver 18 for SQL Server}; Server=DESKTOP-VIUCH90;UID=linux; PWD=linux; Database=node;Encrypt=no;
submitted select top 5 * from master..syscolumns elapsed 2 ms
meta = [
{
"size": 128,
"name": "name",
"nullable": true,
"type": "text",
"sqlType": "nvarchar"
},
```
if you see SEGV whilst using Node 18, 20 you must either downgrade Node to version 16 assuming you are not on openssl 3.2 - or you must install openssl 3.2 which is done for app veyor Node 18 testing on Ubuntu.
you can use script used by appveyor as a reference. Note this script requires su access for install.
```sh
# note this should only be run by SA after checking dependencies for other systems
tool/openssl.sh
```
this script will download openssl from git, configure, build and install it to /usr/local/ssl/bin/openssl. Note it is important to install to its own folder under local should you wish to easily reverse the effects of install.
should you encounter a problem e.g. another previkously working system now fails with new installed version of openssl then remove the ld dependency
```sh
me@asuszen:~/dev/js/sql/v8/node_modules/msnodesqlv8$ cd /etc/ld.so.conf.d/
me@asuszen:/etc/ld.so.conf.d$ ls
fakeroot-x86_64-linux-gnu.conf ld.wsl.conf libc.conf openssl-3.2.0.1s.conf x86_64-linux-gnu.conf
me@asuszen:/etc/ld.so.conf.d$ sudo mv openssl-3.2.0.1s.conf ~/
[sudo] password for me:
me@asuszen:/etc/ld.so.conf.d$ sudo ldconfig -v > /dev/null
/sbin/ldconfig.real: Can't stat /usr/local/lib/x86_64-linux-gnu: No such file or directory
/sbin/ldconfig.real: Path `/usr/lib/x86_64-linux-gnu' given more than once
/sbin/ldconfig.real: Path `/lib/x86_64-linux-gnu' given more than once
/sbin/ldconfig.real: Path `/usr/lib/x86_64-linux-gnu' given more than once
/sbin/ldconfig.real: Path `/usr/lib' given more than once
/sbin/ldconfig.real: /lib/x86_64-linux-gnu/ld-2.31.so is the dynamic linker, ignoring
```
## Sequelize Compatibility ##
the library now has direct support for [sequelize](https://www.npmjs.com/package/sequelize), the popular JS ORM. Simply configure sequelize to point to dialectModulePath directly under msnodesqlv8/lib/sequelize
[example](https://github.com/TimelordUK/node-sqlserver-v8/tree/master/samples/javascript/sequelize.js)
```js
const Sequelize = require('sequelize')
let sequelize = new Sequelize({
dialect: 'mssql',
dialectModulePath: 'msnodesqlv8/lib/sequelize',
dialectOptions: {
'user': '',
'password': '',
'database': 'scratch',
'connectionString': 'Driver={SQL Server Native Client 11.0};Server= np:\\\\.\\pipe\\LOCALDB#2DD5ECA9\\tsql\\query;Database=scratch;Trusted_Connection=yes;',
'options': {
'driver': 'SQL Server Native Client 11.0',
'trustedConnection': true,
'instanceName': ''
}
},
pool: {
min: 0,
max: 5,
idle: 10000
}
})
function createUserModel () {
return sequelize.define('user', {
username: {
type: Sequelize.STRING
},
job: {
type: Sequelize.STRING
}
})
}
function userModel () {
return new Promise(async (resolve, reject) => {
let user = createUserModel()
// force: true will drop the table if it already exists
await user.sync({ force: true })
await Promise.all([
user.create({
username: 'techno01',
job: 'Programmer'
}),
user.create({
username: 'techno02',
job: 'Head Programmer'
}),
user.create({
username: 'techno03',
job: 'Agile Leader'
})
]).catch((e) => reject(e))
let id1 = await user.findByPk(3)
console.log(JSON.stringify(id1, null, 4))
let agile = await user.findOne({
where: { job: 'Agile Leader' }
})
console.log(JSON.stringify(agile, null, 4))
let all = await user.findAll()
console.log(JSON.stringify(all, null, 4))
let programmers = await user
.findAndCountAll({
where: {
job: {
[Sequelize.Op.like]: '%Programmer'
}
},
limit: 2
})
console.log(programmers.count)
const dataValues = programmers.rows.reduce((aggregate, latest) => {
aggregate.push(latest.dataValues)
return aggregate
}, [])
console.log(dataValues)
resolve()
})
}
userModel().then(() => {
sequelize.close()
console.log('done')
})
```
## Production Quality ##
The library is subjected to extensive testing in continuous test cycles to ensure stability and accuracy. It is being used in production environments.
## install ##
ensure you have an up to date version of node installed on your computer first :-
``` javascript
npm install msnodesqlv8
```
to use the code in your node javascript having installed :-
``` javascript
var sql = require('msnodesqlv8');
```
## Electron Support ##
Yes. [Electron](http://electron.atom.io/) is a framework for creating native applications with web technologies. This library now ships with compiled binaries that are compatible with Electron.
## Electron Trouble shoot ##
some information in this thread may be of use in trying to complile against electron
[electron-issue](https://github.com/TimelordUK/node-sqlserver-v8/issues/80)
``` json
"scripts": {
"copybinding": "copy .\\node_modules\\msnodesqlv8\\bindingdotgyp.old .\\node_modules\\msnodesqlv8\\binding.gyp",
"rebuild": "cd .\\node_modules\\.bin\\ && electron-rebuild -f -w msnodesqlv8 --module-dir ..\\..\\",
"movenode": "move .\\node_modules\\msnodesqlv8\\bin\\win32-x64-69\\msnodesqlv8.node .\\node_modules\\msnodesqlv8\\lib\\bin\\msnodesqlv8.node",
"postinstall": "npm run copybinding && npm run rebuild && npm run movenode",
...
},
```
here is a good starting point for React running in Electron using the superb [boilerplate](https://github.com/electron-react-boilerplate/electron-react-boilerplate) which works particularly well using visual studio code.
```cmd
git clone --depth 1 --single-branch https://github.com/electron-react-boilerplate/electron-react-boilerplate.git erb-msnodesqlv8
cd .\erb-msnodesqlv8\
yarn
cd .\app\
yarn
yarn add --dev electron-rebuild
yarn add msnodesqlv8
.\node_modules\.bin\electron-rebuild.cmd
cd ..
yarn dev
```
for example replace app\components\Home.tsx with following to show module running in the renderer. This has been tested on both Windows and Linux. In the sample look at the console output to see list of databases selected assuming a valid connection string.
```ts
import React from 'react';
import { Link } from 'react-router-dom';
import routes from '../constants/routes.json';
import { SqlClient } from 'msnodesqlv8/types';
import styles from './Home.css';
export default function Home(): JSX.Element {
const sql: SqlClient = require('msnodesqlv8');
const connectionString =
'Driver={SQL Server Native Client 11.0}; Server=(localdb)\\node; Database={master}; Trusted_Connection=Yes;';
const query = 'SELECT name FROM sys.databases';
sql.query(connectionString, query, (err, rows) => {
console.log(rows);
});
return (
<div className={styles.container} data-tid="container">
<h2>Home</h2>
<Link to={routes.COUNTER}>to Counter</Link>
</div>
);
}
```
## Set App Name ##
Use the 'APP=my-application' such as in the example below. [issue](https://github.com/TimelordUK/node-sqlserver-v8/issues/25)
```sql
Driver={SQL Server Native Client 11.0}; APP=msnodesqlv8; Server=np:\.\pipe\LOCALDB#8704E301\tsql\query; Database={scratch}; Trusted_Connection=Yes
```
## Handling Variant ##
```typescript
c.query("select cast(10000 as sql_variant) as data;", (err, res) => {});
```
the driver on detecting variant type will query underlying type information and proceed to interpret column as that underlying type. In the example above the column therefore will be presented as javascript int type.
## Long Strings ##
from 0.6 this is not necessary, the driver will automatically choose correct binding.
see [issue](https://github.com/TimelordUK/node-sqlserver-v8/issues/24)
use one of the user binding functions which will force the driver to use the specified binding type rather than guessing from the input data. In this case WLongVarChar should resolve the issue of binding large strings.
``` typescript
function largeText() {
var len = 2200;
var s = "A".repeat(len);
sql.open(connStr, (err, conn) => {
conn.query("declare @s NVARCHAR(MAX) = ?; select @s as s", [sql.WLongVarChar(s)],
(err, res) => {
assert.ifError(err);
var ss = res[0].s;
assert(ss.length == len);
});
});
}
```
## Api ##
### typescript type based information for this library. [api](https://github.com/TimelordUK/node-sqlserver-v8/blob/master/lib/MsNodeSqlDriverApiModule.ts) ###
```javascript
import {MsNodeSqlDriverApiModule as v8} from './lib/MsNodeSqlDriverApiModule'
import v8Connection = v8.v8Connection;
import v8PreparedStatement = v8.v8PreparedStatement;
import v8BindCb = v8.v8BindCb;
import v8BulkMgr = v8.v8BulkTableMgr;
import v8Error = v8.v8Error;
export const sql: v8.v8driver = require('msnodesqlv8');
```
#### including library for use in node ####
```javascript
var sql = require('msnodesqlv8');
export const sql: v8.v8driver = require('msnodesqlv8'); // typescript
```
#### open a connection with standard connection string ####
```typescript
sql.open(conn_str, function (err, conn) { });
```
#### open a connection with timeout ####
```typescript
sql.open( {conn_str : '', conn_timeout : 10}, function (err, conn) {});
```
#### close an open connection ####
```typescript
conn.close(function () { });
```
#### query, error and the more flag ####
When executing a query, the more flag may be used to indicate no further results will be returned. This optional paramater will be set true for example when queries of form 'select ....; select .....' are issued.
It is worth noting behaviour when a query such as below is executed.
```sql
let severity = 9; // or 14 for immediate terminate
RAISERROR(`'User JS Error', ${severity}, 1);SELECT 1+1;`;
```
here the first statement to run will be RAISEERROR. When submitting severity code less than 14, the driver has an oppertunity to read the error code and proceed to returning results on the follwing query. In this case the more flag will be set true and the error condition set. A further callback can be expected with the results and err code not set.
When error code is >= 14 the driver immediately terminates as odbc will not allow results to be obtained from further queries. In this case the more flag will be false and the err condition set to the RAISEERROR.
This behaviour is demonstrated in following test code which loops forever executing this example to prove the connection will recover from a termination of this kind.
```cmd
tool\t-busy14.bat
node test\edge-case.js -t busy --delay=500 --severity=14
tool\t-busy9.bat
node test\edge-case.js -t busy --delay=500 --severity=9
```
#### query an opened connection no input parameters ####
```typescript
var q = conn.query(sql, function (err, results, more) { });
```
#### call a stored proc direct from connection ####
```typescript
// call a proc - omit callback and listen to events
const q = conn.callproc (name, paramsOrCb, function (err, results, output, more) { }))
// helper promise to accumulate results, info messages, output variables
const q = await conn.callprocAggregator(procName, o, options)
```
#### query an opened connection with input parameters ####
```typescript
var q = conn.query(sql, [], function (err, results, more) { });
```
#### adhoc query i.e. open, query, close, callback ####
```typescript
sql.query(conn_str, q, [], function (err, results, more) { });
```
#### query an opened connection with timeout ####
```typescript
conn.query({query_str: '',query_timeout: 10}, (err, results, more) => {});
```
#### prepare a reusable statement on an open connection ####
```typescript
conn.prepare(select, function (err, ps) { });
```
#### execute using prepared statement and input vector of parameters ####
```typescript
ps.preparedQuery([], function (err, res) { });
```
#### free a prepared statement when no longer required ####
```typescript
ps.free(function () { });
```
#### listening to query events ####
```typescript
q.on('error', function(err) { }); // Error type i.e. e.message
q.on('meta', function(meta) { }); // array of column meta data relating to query.
q.on('column', function(col, v) { }); // value relating to column for current row
q.on('info', function(e) { }); // Error type i.e. e.message
q.on('rowcount', function(count) { });
q.on('row', function(row) { }); // the start of index row
q.on('done', function() { }); // no more results - statement still natively exists
q.on('submitted', function() { }); // statement has been sent to native driver.
q.on('output', function() { }); // stored proc call unbinds output params
q.on('open', function() { }); // new connection is open
q.on('closed', function() { });
q.on('free', function() { }); // when native driver releases resources on statement
```
#### obtain procedure manager ####
```typescript
var pm = conn.procedureMgr();
```
#### invoke a stored procedure with input params ####
```typescript
pm.callproc(sp_name, [], function(err, results, output) { });
```
#### obtain table manager ####
```typescript
var tm = conn.tableMgr();
```
#### bind the manager to a target table ####
```typescript
tm.bind(table_name, function (bm) {});
```
#### drive types sent to driver from column definitions - not derived from data ####
```typescript
bm.useMetaType(true)
```
#### use local timestamps not adjusted to UTC ####
``` javascript
test('use tableMgr bulk insert single non UTC based date with datetime col', testDone => {
async function runner () {
const helper = new TypeTableHelper(theConnection, 'datetime')
const testDate = new Date('Mon Apr 26 2021 22:05:38 GMT-0500 (Central Daylight Time)')
const expected = helper.getVec(1, () => testDate)
theConnection.setUseUTC(false)
const table = await helper.create()
const promisedInsert = util.promisify(table.insertRows)
const promisedSelect = util.promisify(table.selectRows)
try {
await promisedInsert(expected)
const res = await promisedSelect(expected)
res.forEach(a => {
delete a.col_a.nanosecondsDelta
})
assert.deepStrictEqual(res, expected)
} catch (e) {
assert.ifError(e)
}
}
runner().then(() => {
testDone()
})
})
```
#### insert using bulk manager with array of objects ####
```typescript
bm.insertRows([], function () {});
```
#### select with array of designated keys ####
```typescript
bm.selectRows([], function (err, res) {});
```
#### delete rows with array of designated keys ####
```typescript
bm.deleteRows([], function () {});
```
#### modify rows via designated keys ####
```typescript
bm.updateRows([], function () {});
```
#### change the set of update columns to be modified via updateRows ####
```typescript
bm.setUpdateCols(updateCols);
```
#### return a summary object representing the bound table ###
```typescript
res = bm.getSummary();
```
## Table Value Parameters ##
From version 0.4.1 the library node supports Table Value Parameters
please refer to [tvp.js](https://github.com/TimelordUK/node-sqlserver-v8/blob/master/unit.tests/tvp.js) for examples
assuming a type is created in the database
```sql
CREATE TYPE TestTvpType AS TABLE (username nvarchar(30), age int, salary real)
```
use a connection object to get a Table representing this type.
```javascript
function (asyncDone) {
theConnection.getUserTypeTable(tableTypeName, function (err, t) {
assert.ifError(err)
table = t
assert(table.columns.length === 3)
asyncDone()
})
}
```
add some rows to the table.
``` javascript
var vec = [
{
username:'santa',
age:1000,
salary:0
},
{
username:'md',
age:28,
salary:100000
}
]
function (asyncDone) {
// see above
table.addRowsFromObjects(vec)
asyncDone()
}
```
create a table parameter from the Table. This method can also accept an mssql Table type.
```javascript
var tp = sql.TvpFromTable(table)
```
create a table in the datbase representing the type.
```sql
create TABLE TestTvp (
username nvarchar(30),
age int,
salary real
)
```
add a stored procedure to insert to the table from the new type
```sql
create PROCEDURE InsertTestTvp
@tvp TestTvpType READONLY
AS
BEGIN
set nocount on
INSERT INTO TestTvp
(
[username],
[age],
[salary]
)
SELECT
[username],
[age],
[salary]
n FROM @tvp tvp
END
```
invoke the stored procedure to insert records.
```javascript
function (asyncDone) {
// see above to create a table
var tp = sql.TvpFromTable(table)
table.rows = []
theConnection.query('exec insertTestTvp @tvp = ?;', [tp], function (err) {
assert.ifError(err)
asyncDone()
})
},
function (asyncDone) {
theConnection.query('select * from ' + tableName, function (err, res) {
assert.ifError(err)
assert.deepEqual(vec, res)
asyncDone()
})
}
```
if an existing table exists. i.e. Employee - bind to it using tablem manager
```javascript
function (asyncDone) {
var tableName = 'Employee'
var tm = theConnection.tableMgr()
tm.bind(tableName, function (bulk) {
bulkMgr = bulk
asyncDone()
})
}
```
then can get a sql representation of the table
``` javascript
function (asyncDone) {
var sql = bulkMgr.asUserType()
theConnection.query(sql, function (err) {
assert.ifError(err)
asyncDone()
})
}
```
which is represented as below
```sql
CREATE TYPE EmployeeType AS TABLE (BusinessEntityID int, NationalIDNumber nvarchar(15), LoginID nvarchar(256), OrganizationNode hierarchyid, OrganizationLevel smallint, JobTitle nvarchar(50), BirthDate date, MaritalStatus nchar, Gender nchar, HireDate date, SalariedFlag bit, VacationHours smallint, SickLeaveHours smallint, CurrentFlag bit, rowguid uniqueidentifier, ModifiedDate datetime)
```
can also get a Table representing the database table
```javascript
function (asyncDone) {
var parsedJSON = helper.getJSON()
// construct a table type based on a table definition.
var table = bulkMgr.asTableType()
// convert a set of objects to rows representing the table
table.addRowsFromObjects(parsedJSON)
// use a type the native driver can understand, using column based bulk binding.
var tp = sql.TvpFromTable(table)
// can now use the tvp with the driver and bind all data in one go.
theConnection.query('select * from ?;', [tp], function(err, res) {
assert.deepEqual(res,parsedJSON)
asyncDone()
})
}
```
## BigInt Strings ##
configure either the oonnection to return all numbers as strings.
```js
test('query a numeric - configure connection to return as string', testDone => {
async function runner () {
const num = '12345678.876'
theConnection.setUseNumericString(true)
const q = `SELECT CAST(${num} AS numeric(11, 3)) as number`
const res = await theConnection.promises.query(q)
try {
assert.deepStrictEqual(res.first[0].number, num)
} catch (e) {
assert.ifError(e)
}
}
runner().then(() => {
testDone()
})
})
```
or issue as a query
```js
test('query a -ve numeric - configure query to return as string', testDone => {
async function runner () {
const num = '-12345678'
const q = `select ${num} as number`
const res = await theConnection.promises.query({
query_str: q,
numeric_string: true
})
try {
assert.deepStrictEqual(res.first[0].number, num)
} catch (e) {
assert.ifError(e)
}
}
runner().then(() => {
testDone()
})
})
```
## Sybase Adaptive Server ##
the library runs most features against Sybase such as below with promised query or legacy callback.
```js
const sql = require('msnodesqlv8')
const { GetConnection } = require('./get-connection')
const connectionString = new GetConnection().connectionString
const query = 'SELECT top 5 * FROM syscomments'
// "Driver={Adaptive Server Enterprise}; app=myAppName; server=localhost port=5000; db=pubs3; uid=sa; pwd=ooooo;"
function legacyQuery () {
return new Promise((resolve, reject) => {
sql.open(connectionString, (err, con) => {
if (err) {
reject(err)
}
con.query(query, (err, rows) => {
if (err) {
reject(err)
}
con.close(() => {
resolve(rows)
})
})
})
})
}
async function promised () {
const connection = await sql.promises.open(connectionString)
const res = await connection.promises.query(query)
console.log(`promised ${JSON.stringify(res, null, 4)}`)
await connection.promises.close()
return res
}
async function q1 () {
const d = new Date()
try {
const rows = await legacyQuery()
const elapsed = new Date() - d
console.log(`legacyQuery rows.length ${rows.length} elapsed ${elapsed}`)
console.log(`legacyQuery ${JSON.stringify(rows, null, 4)}`)
} catch (err) {
console.error(err)
}
}
```
## thread pooling ##
the library can now be used by a thread worker as outlined below.
master worker
```js
const path = require('path')
const filePath = path.resolve(__dirname, './worker-item.js')
const { Worker } = require('worker_threads')
const worker1 = new Worker(filePath)
const worker2 = new Worker(filePath)
function dispatch (worker) {
worker.on('message', msg => {
switch (msg.command) {
case 'task_result': {
console.log(JSON.stringify(msg, null, 4))
}
}
})
worker.on('error', error => {
console.log(error)
})
}
dispatch(worker1)
dispatch(worker2)
function sendTask (worker, num) {
worker.postMessage(
{
command: 'task',
num: num
})
}
function clean () {
setTimeout(async () => {
console.log('exit.')
await Promise.all([
worker1.terminate(),
worker2.terminate()
])
}, 5000)
}
for (let i = 0; i < 40; i += 2) {
sendTask(worker1, i)
sendTask(worker2, i + 1)
}
clean()
```
worker
```js
const { parentPort } = require('worker_threads')
const sql = require('msnodesqlv8')
const { GetConnection } = require('./get-connection')
const connectionString = new GetConnection().connectionString
async function compute (msg) {
try {
console.log(`worker receive task ${msg.num}`)
const conn = await sql.promises.open(connectionString)
const query = `select ${msg.num} as i, @@SPID as spid`
const res = await conn.promises.query(query)
await conn.promises.close()
parentPort.postMessage(
{
command: 'task_result',
data: `spid ${res.first[0].spid}`,
num: msg.num,
fib: getFib(msg.num)
})
} catch (e) {
parentPort.emit('error', e)
}
}
parentPort.on('message', async msg => {
switch (msg.command) {
case 'task': {
await compute(msg)
break
}
default: {
console.log(`unknown command ${msg.command}`)
break
}
}
})
function getFib (num) {
if (num === 0) {
return 0
} else if (num === 1) {
return 1
} else {
return getFib(num - 1) + getFib(num - 2)
}
}
```
## promises ##
see promises.ts under samples/typescript for some example code of how to use these promise methods.
Some promises have been added to the API for a more modern async approach. They are all collected under 'object.promises.promise' (pool.promises.open(), sql.promises.query(..), sql.promises.callProc(..), connection.promises.query(..) etc)
see index.js for definitions
```ts
export interface AggregatorPromises {
query(sql: string, params?: any[], options?: QueryAggregatorOptions): Promise<QueryAggregatorResults>
callProc(name: string, params?: any, options?: QueryAggregatorOptions): Promise<QueryAggregatorResults>
}
interface SqlClientPromises {
query(conn_str: string, sql: string, params?: any[], options?: QueryAggregatorOptions): Promise<QueryAggregatorResults>
callProc(conn_str: string, name: string, params?: any, options?: QueryAggregatorOptions): Promise<QueryAggregatorResults>
open(conn_str: string): Promise<Connection>
}
export interface PoolPromises extends AggregatorPromises {
open(): Promise<Pool>
close(): Promise<any>
}
interface ConnectionPromises extends AggregatorPromises {
prepare(sql: string): Promise<PreparedStatement>
getTable(name: string): Promise<BulkTableMgr>
close(): Promise<any>
cancel(name: string): Promise<any>
}
export interface BulkTableMgrPromises
{
select(cols: any[]): Promise<any[]>
insert(rows: any[]): Promise<any>
delete(rows: any[]): Promise<any>
update(rows: any[]): Promise<any>
}
export interface PreparedPromises {
free(): Promise<any>
query(params?: any[], options?: QueryAggregatorOptions) : Promise<QueryAggregatorResults>
}
```
for example using the connection pool using promises.
### connection pool ###
``` ts
async function pool() {
try {
const connStr: string = getConnection()
const size = 4
const options: PoolOptions = {
connectionString: connStr,
ceiling: size
}
const pool: Pool = new sql.Pool(options)
await pool.promises.open()
const all = Array(size * 2).fill(0).map((_, i) => pool.promises.query(`select ${i} as i, @@SPID as spid`))
const promised: QueryAggregatorResults[] = await Promise.all(all)
const res = promised.map(r => r.first[0].spid)
await pool.promises.close()
console.log(`pool spids ${res.join(', ')}`)
} catch (e) {
console.log(e)
}
}
```
### query ###
```ts
async function adhocQuery() {
try {
const connStr: string = getConnection()
const res: QueryAggregatorResults = await sql.promises.query(connStr, 'select @@SPID as spid')
console.log(`ashoc spid ${res.first[0].spid}`)
} catch (e) {
console.log(e)
}
}
async function openSelectClose() {
try {
const connStr: string = getConnection()
const conn: Connection = await sql.promises.open(connStr)
const res: QueryAggregatorResults = await conn.promises.query('select @@SPID as spid')
console.log(JSON.stringify(res, null, 4))
await conn.promises.close()
} catch (e) {
console.log(e)
}
}
```
### procedure ###
use a promise to open connection, call a proc and close all from one promise - or call from a connection. Note all results are aggregated i.e. you are returned a result containing all queries etc
```ts
async function adhocProc() {
try {
const connStr: string = getConnection()
const proc = new ProcTest(connStr, sampleProc)
await proc.create()
const msg = 'hello world'
const res: QueryAggregatorResults = await sql.promises.callProc(connStr, sampleProc.name, {
param: msg
})
await proc.drop()
console.log(`adhocProc returns ${res.returns} from param '${msg}''`)
} catch (e) {
console.log(e)
}
}
async function proc() {
try {
const connStr: string = getConnection()
const proc = new ProcTest(connStr, sampleProc)
await proc.create()
const conn: Connection = await sql.promises.open(connStr)
const promises: ConnectionPromises = conn.promises
const msg = 'hello world'
const res: QueryAggregatorResults = await promises.callProc(sampleProc.name, {
param: msg
})
console.log(`proc returns ${res.returns} from param '${msg}''`)
await proc.drop()
await promises.close()
} catch (e) {
console.log(e)
}
}
```
### table manager ###
use a promise to fetch a table and insert rows to it.
```ts
async function table() {
try {
const connStr: string = getConnection()
const connection = await sql.promises.open(connStr)
const tm: BulkTableTest = new BulkTableTest(connection, sampleTableDef)
const table: BulkTableMgr = await tm.create()
const vec: SampleRecord[] = getInsertVec(10)
console.log(`table = ${tm.createTableSql}`)
await table.promises.insert(vec)
const read = await connection.promises.query(tm.selectSql)
console.log(`table ${read.first.length} rows from ${tm.tableName}`)
console.log(JSON.stringify(read.first, null, 4))
await tm.drop()
await connection.promises.close()
} catch (e) {
console.log(e)
}
}
```
## Compiling The Driver ##
1. ensure visual studio 2015 is installed, community edition should be fine. If you do not want the full IDE it is also possible to build with the tools available from Microsoft [here](https://www.microsoft.com/en-us/download/details.aspx?id=48159)
1. make sure you have latest version of node installed.
1. install node-gyp globally :- npm install -g node-gyp
1. download and install the git [client](https://git-scm.com/downloads)
1. clone the code base :- git clone [git-src](https://github.com/TimelordUK/node-sqlserver-v8.git) msnodesqlv8
1. start a shell command and cd into new folder :- copy bindingdotgyp.old binding.gyp
1. make sure [Python27](https://www.python.org/downloads/) is installed
1. node-gyp clean configure build --verbose --arch=x64
1. if you wish to edit and compile code for Debug, open the generated solution file in VS2015. i.e. build\binding.sln
1. copy the node.lib to Debug for example C:\Users\<usr>\.node-gyp\6.10.0\Debug from x64 folder.
1. build from visual studio.
1. to run this target change to debug mode from file [sqlserver.native.js](https://github.com/TimelordUK/node-sqlserver-v8/blob/master/lib/sqlserver.native.js)
## Async Patterns ##
can convert the API easily into a promise based API
```javascript
async function test (request, response) {
response.statusCode = 200
response.setHeader('Content-Type', 'text/plain')
let sqlOpen = toPromise(sql.open)
try {
let connection = await sqlOpen(connectionString)
let connectionQuery = toPromise(connection.queryRaw)
try {
let d = new Date()
let data = await connectionQuery(query)
let elapsed = new Date() - d
response.end(JSON.stringify(data, null, 4))
let close = toPromise(sql.close)
await close()
} catch (err) {
response.end(err.message)
}
} catch (err) {
response.end(err.message)
}
}
\\ use any conversion utility ....
function toPromise (f) {
return function (args) {
return new Promise((resolve, reject) => {
function handler (err, res) {
if (err) {
reject(err)
} else {
resolve(res)
}
}
if (args) {
f(args, handler)
} else {
f(handler)
}
})
}
}
```
the msnodesqlv8 library works nicely with the brilliant library [asynquence](https://github.com/getify/asynquence) written by