jiny/README.md

Jiny - runs tests/jobs in parallel

# Jiny - runs tests/jobs in parallel

`jiny` allows to run your tests/jobs/commands in parallel on the different servers.
Main goal of `jiny` is horizontal scalability for testing.

### It has 4 main features:

* upload project folder to the all servers (slaves) - `jiny upload`
* prepare project on the all slaves - `jiny run "./prepare.sh"`
* make job queue based on mask and pattern and run in parallel `jiny feed *.js "mocha -R spec @"`
* make job queue based on pipe and run in parallel `find ... | jiny jobs`


### Install
- Require: `tar`
- `npm install -g jiny`

### Picture

![jiny](https://bytebucket.org/slavahatnuke/jiny/raw/ea714392ff660fecc78b12c280800fc64bab4bc1/doc/jiny.png)

### Help 
```
> jiny 


  Usage: jiny [options] [command]

  Commands:

    master
       Starts master

    slave
       Starts slave

    upload
       Upload dir to the slaves

    run [command]
       Runs command on the all slaves. Prepare project on the all slaves. `jiny run "./prepare.sh"`

    feed [mask] [pattern]
       Find files by mask, build job based on pattern and runs jobs in parallel on different slaves. Replaces `@` -> `[file name]`. Example: `jiny --dir tests feed *.js "mocha -R spec @"`

    jobs
       Runs commands in  in parallel on the different slaves. Uses pipe: `find tests/ | grep .js | awk '{print "echo "$1" && sleep 1"}' | jiny jobs`

    stop
       Stops slaves and master

    wait.slaves [quantity]
       Waits for slaves registered in the master


  Options:

    -h, --help     output usage information
    -V, --version  output the version number
    --port [port]  master port
    --host [host]  master host
    --dir [dir]    work dir
    --detach       runs detached process: master / slave
    --live         master option to save alive slaves after exit
    --skip-upload  slave option to skip upload on this slave, if you want to use slave on the master

```

### Local example

#### Make dirs
- `mkdir ~/my-project # make project dir`
- `mkdir ~/my-project-1 # make slave dir 1`
- `mkdir ~/my-project-2 # make slave dir 2`

#### Start master and slaves (in different terminals)
- `jiny master --dir ~/my-project/ # start master`
- `jiny slave --dir ~/my-project-1/ # start slave #1`
- `jiny slave --dir ~/my-project-2/ # start slave #2`

#### Prepare project (in another terminal)
```
> jiny run "echo prepare-project && pwd && sleep 1" # run same command on the both slaves in same time

I am client http://10.0.2.15:7500
#1 10.0.2.15 > echo prepare-project && pwd && sleep 1
prepare-project
/home/vagrant/my-project-1

#2 10.0.2.15 > echo prepare-project && pwd && sleep 1
prepare-project
/home/vagrant/my-project-2

PASSed 1s
```

#### Make fake tests
- `mkdir ~/my-project/tests # make like tests dir in the project`
- `touch ~/my-project/tests/test1.js # make like test1`
- `touch ~/my-project/tests/test2.js # make like test2`

#### Upload tests to slaves
```
> jiny upload # upload project to slaves

I am client http://10.0.2.15:7500
#1 10.0.2.15 uploaded dir: /home/vagrant/my-project-1/ time: 0s
#2 10.0.2.15 uploaded dir: /home/vagrant/my-project-2/ time: 0s
done 0s

#### Check uploaded data
> jiny run "pwd && ls tests" # check that tests have been uploaded

I am client http://10.0.2.15:7500
#1 10.0.2.15 > pwd && ls tests
/home/vagrant/my-project-1
test1.js
test2.js

#2 10.0.2.15 > pwd && ls tests
/home/vagrant/my-project-2
test1.js
test2.js

PASSed 0s
```

#### Find files by mask, build job based on pattern and runs jobs in parallel
- it will run commands one by one on the free slaves
- slave #1 executes test1.js, slave #2 executest test2.js in same time.
- in this example I use mocha to run these tests
- real examples looks like:
    - `jiny feed --dir features *.feature "cucumber.js features/@"`
    - `jiny feed --dir src *Test.php "phpunit -c app/ src/@"`

```

> jiny --dir ~/my-project/tests feed *.js "mocha -R spec tests/@"

I am client http://10.0.2.15:7500
#1 10.0.2.15 > mocha -R spec tests/test1.js


  0 passing (5ms)


#2 10.0.2.15 > mocha -R spec tests/test2.js


  0 passing (5ms)


PASSed 0s

```

### Stop all: slaves and master
```
> jiny stop # stops all, slaves and master
```



### Notes

#### Start master and slaves (in same terminal)
- if you want to work in one terminal use `--detach` option.
    - `jiny master --dir ~/my-project/ --detach # start master`
    - `jiny slave --dir ~/my-project-1/ --detach # start slave #1`
    - `jiny slave --dir ~/my-project-2/ --detach # start slave #2`

 
#### Find files by `find` and `grep` and pipe it to jiny jobs
- if you want to make job queue by pipe you can use this example
    
```
> find ~/my-project/tests | grep .js | awk '{print "echo "$1" && sleep 1"}' | jiny jobs

I am client http://10.0.2.15:7500

#1 192.168.1.131 > echo /home/vagrant/my-project/tests/test1.js && sleep 1
/home/vagrant/my-project/tests/test1.js

#2 192.168.1.131 > echo /home/vagrant/my-project/tests/test2.js && sleep 1
/home/vagrant/my-project/tests/test2.js

PASSed 1s

```


Run your tests in parallel, be scalable horizontal! Enjoy and be happy!