@techolution-ai/computer-vision/README.md

A JavaScript/TypeScript library for computer vision applications, providing tools for image processing, scanning, and MQTT-based messaging.

# @techolution-ai/computer-vision

A JavaScript/TypeScript library for computer vision applications, providing tools for image processing, scanning, and MQTT-based messaging.

## Table of Contents

- [Features](#features)
- [Usage](#usage)
  - [Stream the video](#stream-the-video)
  - [Listening to MQTT messages](#listening-to-mqtt-messages)
  - [Scanner with MQTT with UI](#scanner-with-mqtt-with-ui)
- [API Reference](#api-reference)
- [Requirements](#requirements)
- [License](#license)
- [Contributing](#contributing)
- [Support](#support)



## Features

The library provides several modules:

- Core computer vision functionality
- Scanner module for image/video scanning
- MQTT-based messaging system for real-time communication

## Usage

### Stream the video

Use scanner to stream the camera's live recording to web using `useScanner` hook.

```tsx
import { useMemo } from 'react';

import { useScanner } from '@techolution-ai/computer-vision/scanner';

function App(){

  const scanner = useScanner({ baseUrl: 'http://0.0.0.0:8501' })
  const streamUrl = useMemo(()=>scanner.getVideoStreamUrl(),[scanner]);

  // render the image with steaming url
  return <ScanImage src={streamUrl} />
}
```


### Listening to MQTT messages

We can use `useMessages` to connect, subscribe to certain topics, and can receive the messages on `onMessage` callback.

We must use the `MessagesProvider` to use `useMessages` hook. It can be put at the root (or one level up) to the component's tree.

```tsx
import {
  MessagesProvider,
  useMessages,
} from '@techolution-ai/computer-vision/messages'


function MessagesListener() {

  const handleReceiveMessage = (topic: string, message: any) => {
    // your topic and message is here
    // add your logic here
  }

  useMessages({
    topics: ['current_status', 'products_scanned'], // subscribing to topics
    onMessage: handleReceiveMessage, // message handler
  })
}

function App(){
  <MessagesProvider url="ws://0.0.0.0:9001">
    <MessagesListener />
  </MessagesProvider>
}
```

### Scanner with MQTT with UI

We can combine above examples to do a full integration.

We also have a ready-to-use react component: `ScannerStatus` with default options added which can be used to show different statuses in the UI.

```tsx

import { useMemo, useState } from 'react';

import {
  ScanImage,
  useScanner,
  ScannerStatus,
  defaultStatusMap,
  TStatusMap,
} from '@techolution-ai/computer-vision/scanner';

import {
  MessagesProvider,
  useMessages,
} from '@techolution-ai/computer-vision/messages';

function MessagesListener() {
  const scanner = useScanner({ baseUrl: 'http://0.0.0.0:8501' })

  const streamUrl = useMemo(() => scanner.getVideoStreamUrl(), [scanner])

  const [statusMap, setStatusMap] = useState<TStatusMap | null>(
    defaultStatusMap,
  )

  const [status, setStatus] = useState('idle')
  const [messages, setMessages] = useState<{ topic: string; message: any }[]>([])

  const handleReceiveMessage = (topic: string, message: any) => {
    try {
      const value = JSON.parse(message.toString())

      if (topic === 'current_status') {
        setStatus(value.status)
      } else if (topic === 'item_scanned') {
        setMessages((prev) => [...prev, { topic, message: value }])
      }
    } catch (e) {
      console.error('Error parsing message:', e)
    }
  }

  useMessages({
    topics: ['current_status', 'item_scanned'],
    onMessage: handleReceiveMessage,
  })

  return (
    <ScannerStatus
      style={{
        border: '4px solid #4b2f99',
        borderRadius: 16,
        overflow: 'hidden',
        position: 'relative',
        width: '100%',
        height: '100%',
      }}
      status={status}
      statusMap={statusMap}
    >
      <ScanImage
        src={streamUrl}
        style={{
          width: '100%',
          height: '100%',
          objectFit: 'cover',
          position: 'absolute',
          top: 0,
          left: 0,
          bottom: 0,
          right: 0,
        }}
      />
    </ScannerStatus>
  )
}

function App() {
  return (
    <MessagesProvider url="ws://0.0.0.0:9001">
      <MessagesListener />
    </MessagesProvider>
  )
}

```

## API Reference

The library exposes three main modules:

1. **Scanner Module**
   - Specialized scanning capabilities
   - Import via `@techolution-ai/computer-vision/scanner`

2. **Messages Module**
   - MQTT-based messaging system
   - Import via `@techolution-ai/computer-vision/messages`



### MessagesProvider (component) 

Provider for MQTT connection.

#### Props

Prop | Type |	Default Value	| Remarks
| ----- | ----- | ----- | ----- |
| url | string | -- | Required  |
| enableDebugging | boolean | false | Optional, to enable logging |



### ScannerStatus (component) 

Take all `HTMLDivElement` props as root

#### Props

Prop | Type |	Default Value	| Remarks
| ----- | ----- | ----- | ----- |
| status | string | -- | Required  |
| statusMap | TStatusMap (Object) | null | Optional, uses default statusMap internally if not passed |


### useMessages (react hook) 

#### Props

| Prop | Type |	Default Value	| Remarks |
| ----- | ----- | ----- | ----- |
| topics | string[] | -- | Required  |
| onMessage | (topic:string,message:Buffer)=>void | callback to receive messages | Required |
| onError | function | Callback function when an error occurs.  (not implemented) |
| onConnect | function | Callback function when the connection is established. (not implemented) |
| onClose | function |Callback function when the connection is closed. (not implemented) |


### useScanner (react hook)

#### Props

| Prop | Type |	Default Value	| Remarks | 
| ----- | ----- | ----- | ----- |
| baseUrl | string | -- | Required  | The base URL for the scanner API.
| endpoints | object | Custom endpoints for the scanner API. (Optional)



#### APIs (returned functions)

| API |  Remarks |
| ----- | ----- |
| `getVideoStreamUrl()` | Returns the URL for the video stream. |
| `startInference()` | Starts the inference process. |
| `stopInference()` | Stops the inference process. |
| `captureFrame()` | Captures the current frame. |




## Requirements

- React ≥ 16.8.0
- Node.js ≥ 14.0.0
- TypeScript ≥ 4.0.0 (for TypeScript users)
- MQTT server running along with AI backend service


## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

---

## Contributing

Please read our contributing guidelines and code of conduct before submitting pull requests.

## Support

For issues and feature requests, please use the GitHub issue tracker.