imago-azure-storage/README.md

An opinionated async wrapper for azure-storage for working with Azure Storage such as tables and queues.

# Imago Azure Storage

This is an opinionated wrapper for Microsoft's `azure-storage` package for working with
Azure storage. It provides a more convenient syntax and `async` support.

## Performance ##

It is best to cache the files stored in the cloud storage, because the
retrieval times are 20-200 ms from within the same region, and 200-600 ms
if the data is transferred between regions.

## Initialization ##

The constructor takes the storage account name (e.g. `myaccount`) and the
storage service key that you can find in Azure Portal after creating a
storage account.

These credentials are better stored in the `.env` file. Then you can initialize
the service as follows:

```js
    const { Blobs } = require('imago-azure-storage');
    global.blobs = new Blobs(
      process.env.AZURE_STORAGE_ACCOUNT, process.env.AZURE_STORAGE_ACCESS_KEY);
```

The reason to store the blobs service in the `global` variable is to
avoid reestablishing the connection to the server each time the service is
accessed from different parts of the code.

## Methods ##

### async initializeContainers(containers) ###

Initializes the containers, meaning it checks that the containers are created,
or creates them.

The `containers` is an object in the following format:

```js
const AZURE_BLOB_STORAGE_CONTAINERS = {
  MY_CONTAINER_1: { name: 'my-container-1', type: 'private' },
  MY_OTHER_CONTAINER: { name: 'my-other-container', type: 'public' },
};
```

The key is irrelevant and is for the programmer's convenience when
calling other methods like `writeFile`. The `name` is the container name in
Azure. The `type` can be `private` or `public` (allows anyone to download files
via a public URL).

### async writeFile(container, filename, content) ###

Write a file to storage.

### async append(container, filename, content) ###

Append to a file in storage.

### async readFile(container, filename) ###

Read a file from storage.

### async exists(container, filename) ###

Check if file exists in storage.

Since each operation costs money, in some cases you can simply try to read
a file using `readFile()`, and catch an exception if it doesn't exist.

## getPublicDownloadLink(container, filename) ##

Get a public download URL for a filename. The container type must be set to
`public` when it was first created.

# Azure Tables

Azure Table storage is also supported.

## Methods ##

### constructor(storageAccount, storageAccessKey) ###

### async initializeTables(tables) ###

Create tables if they don't exist. `tables` is an array of strings (table names).

### async retrieveEntity(table, partition, key) ###

Read an entity from table.

### async retrieveAllEntitiesByPartitionKey(table, partition, maxPages = 0) ###

Return all entities from a table that have the specified partition key.

The `maxPages` controls the maximum number of queries sent to the server to retrieve
data (each query may up to 1000 rows). Set to `0` to retrieve all data. Note that
`maxPages = 1` does not guarantee that some data will be returned: some queries may
return empty data because the requested data resides on another server. For safety,
retrieve a few pages and then trim the data if necessary.

### async retrieveEntireTable(table, maxPages = 0) ###

Returns all data from a table. Each 1000 entities will be a separate API
request (charged by Azure), and retrieving a large number of entities may
take a significant time. Do it only when the table is small enough, at most a
few thousand records.

The `maxPages` controls the maximum number of queries sent to the server to retrieve
data (each query may up to 1000 rows). Set to `0` to retrieve all data. Note that
`maxPages = 1` does not guarantee that some data will be returned: some queries may
return empty data because the requested data resides on another server. For safety,
retrieve a few pages and then trim the data if necessary.

### async saveStringOnlyEntity(table, partition, key, data) ###

Save or replace entity with the specified key in database. `data` is an object
where each key is a column name.

### async saveStringOnlyEntity(table, data) ###

Same as above. The `data` must contain the `PartitionKey` and `RowKey` properties.

### async query(table, partition, conditions, maxPages = 0) ###

Queries a table based on the column values. `conditions` is an array of arrays, where each condition
is an array containing two or more elements, for example ['userName eq ?', 'Anton']
or ['hitCount >= ? and hitCount <= ?', '2', '100'].

If `partition` is null or empty string, then all partitions will be searched.

The `maxPages` controls the maximum number of queries sent to the server to retrieve
data (each query may up to 1000 rows). Set to `0` to retrieve all data. Note that
`maxPages = 1` does not guarantee that some data will be returned: some queries may
return empty data because the requested data resides on another server. For safety,
retrieve a few pages and then trim the data if necessary.

### async deleteEntity(table, partition, key) ###

Deletes the specified entity from table by row key.

# Azure Queues

## Example ##

```js
const { Queues } = require('imago-azure-storage');
const q = new Queues(storageAccount, storageKey);

const QUEUE_NAME = 'testqueue';

(async() => {
    await q.initializeQueues([QUEUE_NAME]);
    
    // Write:
    await q.put(QUEUE_NAME, { test: 12345 });
    
    // Retrieve:
    let items = await q.fetch(QUEUE_NAME);
    for (let { item, message } of items) {
        console.log(item); // { test: 12345 }

        // Delete from queue:
        await q.commit(message);
    }
})();

```

### async initializeQueues(queues) ###

Creates queues if they don't exist yet. `queues` is an array of strings which
are names of the queues.

### async put(queue, message) ###

Places a message into the queue. `message` can be string or object, but must
be less than 48 KB in size (for objects - when serialized).

### async fetch(queue, numOfMessages, visibilityTimeout) ###

Locks and fetches available items from queue.

Optional arguments:

- `numOfMessages`: How many items to retrieve (up to 32)
- `visibilityTimeout`: Timeout in seconds after which messages will be requeued if not committed. Default 30 seconds.

It returns the array of objects, where each object contains `item` which is your
data that you placed into the queue, and `message` that can be used to delete
the item from the queue once your processing is done. If you don't delete the 
item, it will be placed back into the queue when the timeout expires.

### commit(message) ###

Deletes an item from the queue. `message` must be the object returned as part of
the data from `fetch()`.