textanalysis-tool
Version:
A TypeScript module providing text analysis functionalities with various operations.
774 lines (605 loc) • 27.1 kB
Markdown
# 🌟 Text Analysis Tools 🌟
[](https://www.npmjs.com/package/textanalysis-tool) [](https://opensource.org/licenses/MIT) [](https://www.typescriptlang.org/)
> 🚀 A lightning-fast ⚡ TypeScript library for 🤖 text analysis and 🔧 manipulation. Bundled with everything from simple cleanup to deep linguistic insights! 📦
---
## 🗂️ Table of Contents
- [🌟 Text Analysis Tools 🌟](#-text-analysis-tools-)
- [🗂️ Table of Contents](#️-table-of-contents)
- [⬇️ Installation](#️-installation)
- [🧩 Usage](#-usage)
- [🔰 Basic Usage](#-basic-usage)
- [⚙️ Working with Operations](#️-working-with-operations)
- [🧠 Advanced Analysis](#-advanced-analysis)
- [🏭 Using Factory Methods](#-using-factory-methods)
- [📦 Batch Processing](#-batch-processing)
- [🛠️ Available Operations](#️-available-operations)
- [🗑️ Text Removal](#️-text-removal)
- [📤 Text Extraction](#-text-extraction)
- [🔄 Text Transformation](#-text-transformation)
- [🔢 Text Counting](#-text-counting)
- [🔮 Advanced Analysis Options](#-advanced-analysis-options)
- [🎨 Custom Operations](#-custom-operations)
- [➕ Adding Custom Operations](#-adding-custom-operations)
- [📊 With Metadata Extraction](#-with-metadata-extraction)
- [🚀 Advanced Usage](#-advanced-usage)
- [🔘 Toggling Operations](#-toggling-operations)
- [🔀 Managing Multiple Operations](#-managing-multiple-operations)
- [🔄 Resetting Text](#-resetting-text)
- [✂️ Truncating Text](#️-truncating-text)
- [📘 API Reference](#-api-reference)
- [🧰 Tools.Analyser Class](#-toolsanalyser-class)
- [🧩 Tools.Operations Enum](#-toolsoperations-enum)
- [🔍 Tools.ToolsConstant Class](#-toolstoolsconstant-class)
- [📋 Interface Types](#-interface-types)
- [🔌 Extensions](#-extensions)
- [😊 SentimentAnalyzer](#-sentimentanalyzer)
- [📝 TextSummarizer](#-textsummarizer)
- [📊 TextStatistics](#-textstatistics)
- [🌐 LanguageDetector](#-languagedetector)
- [🔍 TextDiff](#-textdiff)
- [🧩 Interfaces \& Types](#-interfaces--types)
- [😀 SentimentResult](#-sentimentresult)
- [🏷️ SentimentClassification](#️-sentimentclassification)
- [📖 ReadabilityResult](#-readabilityresult)
- [🗣️ LanguageDetectionResult](#️-languagedetectionresult)
- [🔄 TextDiffResult](#-textdiffresult)
- [🤝 Contributing](#-contributing)
- [📝 License](#-license)
---
## ⬇️ Installation
> _Install in one click! ✨_
```bash
npm install textanalysis-tool
```
or via Yarn:
```bash
yarn add textanalysis-tool
```
---
## 🧩 Usage
### 🔰 Basic Usage
```typescript
import { Tools } from 'textanalysis-tool';
const text = "This is a sample text with 123 numbers and https://example.com URL!";
// Create analyzer with specific operations enabled
const analyser = new Tools.Analyser(text, {
[Tools.Operations.CountCharacters]: true,
[Tools.Operations.CountWords]: true,
[Tools.Operations.ExtractUrls]: true
});
// Run the analysis
analyser.main()
.then(result => {
console.log("Text analysis results:");
console.log(`- Character count: ${result.metadata.counts.characterCount}`);
console.log(`- Word count: ${result.metadata.counts.wordCount}`);
console.log(`- URLs found: ${result.metadata.urls?.join(', ')}`);
console.log(`- Operations performed: ${result.operations.join(', ')}`);
console.log(`- Execution time: ${result.executionTime}ms`);
})
.catch(error => {
console.error('Analysis failed:', error);
});
```
### ⚙️ Working with Operations
```typescript
import { Tools } from 'textanalysis-tool';
const analyser = new Tools.Analyser("Hello, world! 123 #hashtag @mention", {
// Enable specific operations
[Tools.Operations.RemovePunctuations]: true,
[Tools.Operations.RemoveNumbers]: true,
[Tools.Operations.ConvertToUppercase]: true
});
analyser.main().then(result => {
console.log(result.output); // "HELLO WORLD HASHTAG MENTION"
});
```
### 🧠 Advanced Analysis
```typescript
import { Tools } from 'textanalysis-tool';
const paragraph = `This tool is amazing 🎯. Sometimes you need exact control.`;
const analyser = new Tools.Analyser(paragraph, {
[Tools.Operations.AnalyzeSentiment]: { positive: ['amazing'], negative: ['need'] },
[Tools.Operations.SummarizeText]: { sentenceCount: 2 },
[Tools.Operations.CalculateReadability]: true,
[Tools.Operations.DetectLanguage]: true,
[Tools.Operations.CompareTexts]: { compareWith: 'Other example text.' }
});
analyser.main().then(result => {
console.log('❤️ Sentiment:', result.sentiment);
console.log('✂️ Summary:', result.summary);
console.log('📖 Readability:', result.readability);
console.log('🌍 Language:', result.language);
console.log('🔍 Text Diff:', result.diff);
});
```
### 🏭 Using Factory Methods
```typescript
import { Tools } from 'textanalysis-tool';
// Create an analyzer with specific operations enabled
const analyser = Tools.Analyser.createWithEnabledOperations(
"Hello, world! 123",
['CountCharacters', 'CountWords', 'RemovePunctuations']
);
analyser.main().then(result => {
console.log(result.output); // "Hello world 123"
console.log(`Words: ${result.metadata.counts.wordCount}`);
console.log(`Characters: ${result.metadata.counts.characterCount}`);
});
```
### 📦 Batch Processing
```typescript
import { Tools } from 'textanalysis-tool';
const texts = [
"First sample with https://example1.com",
"Second sample 12345 with #hashtags",
"Third sample with @mentions and emails@example.com"
];
const options = {
[Tools.Operations.CountWords]: true,
[Tools.Operations.ExtractUrls]: true,
[Tools.Operations.ExtractHashtags]: true,
[Tools.Operations.ExtractMentions]: true,
[Tools.Operations.ExtractEmails]: true
};
Tools.Analyser.batch(texts, options)
.then(results => {
results.forEach((result, index) => {
console.log(`\nAnalysis of text #${index + 1}:`);
console.log(`- Word count: ${result.metadata.counts.wordCount}`);
console.log(`- URLs: ${result.metadata.urls?.join(', ') || 'None'}`);
console.log(`- Hashtags: ${result.metadata.hashtags?.join(', ') || 'None'}`);
console.log(`- Mentions: ${result.metadata.mentions?.join(', ') || 'None'}`);
console.log(`- Emails: ${result.metadata.emails?.join(', ') || 'None'}`);
});
});
```
## 🛠️ Available Operations
### 🗑️ Text Removal
| Operation | Description | Example Input | Example Output |
| --------------------- | -------------------------- | ------------------- | ----------------- |
| `RemovePunctuations` | 🧹 Remove punctuation | "Hello, world!" | "Hello world" |
| `RemoveNumbers` | 🔢 Remove numbers | "abc123def" | "abcdef" |
| `RemoveAlphabets` | 🔡 Remove alphabets | "abc123def" | "123" |
| `RemoveSpecialChars` | ✨ Remove special chars | "Hi @you #1!" | "Hi you 1" |
| `RemoveNewlines` | ↩️ Remove newlines | "Hello\nWorld" | "Hello World" |
| `RemoveExtraSpaces` | 📏 Trim extra spaces | " Hi there " | "Hi there" |
### 📤 Text Extraction
| Operation | Description | Example Input | Example Output |
| ---------------------- | ------------------------ | ------------------------------------ | ------------------------- |
| `ExtractUrls` | 🌐 Extract URLs | "Visit https://a.com and b.org" | ["https://a.com"] |
| `ExtractEmails` | ✉️ Extract emails | "Email me at user@test.com" | ["user@test.com"] |
| `ExtractPhoneNumbers` | 📞 Extract phone numbers | "Call 123-456-7890" | ["123-456-7890"] |
| `ExtractHashtags` | #️⃣ Extract hashtags | "#fun #code" | ["#fun","#code"] |
| `ExtractMentions` | Extract mentions | "Hi @user!" | ["@user"] |
### 🔄 Text Transformation
| Operation | Description | Example Input | Example Output |
| --------------------- | ----------------------- | ---------------------- | ---------------- |
| `ConvertToUppercase` | 🔠 UPPERCASE conversion | "Hello World" | "HELLO WORLD" |
| `ConvertToLowercase` | 🔡 lowercase conversion | "Hello World" | "hello world" |
| `ConvertToTitleCase` | 🆎 Title Case | "hello world" | "Hello World" |
| `ReverseText` | 🔁 Reverse text | "abcde" | "edcba" |
| `Truncate` | ✂️ Truncate text | (maxLength=5) "abcdef"| "abcde..." |
### 🔢 Text Counting
| Operation | Description | Example Input | Example Output |
| ---------------------- | ------------------------ | ------------- | ------------------- |
| `CountCharacters` | 🔠 Count non-space chars | "Hi!" | 3 |
| `CountAlphabets` | 📝 Count letters | "A1b2C" | 3 |
| `CountNumbers` | 🔢 Count digits | "A1b2C" | 2 |
| `CountAlphanumeric` | 🔤 Letters+digits count | "A1 b2!" | { alph:3, num:2 } |
| `CountWords` | 📝 Count words | "Hello world"| 2 |
| `CountSentences` | 📑 Count sentences | "Hi. Bye?" | 2 |
### 🔮 Advanced Analysis Options
| Operation | Description | Example Input | Example Output |
| ----------------------- | -------------------------------- | ------------------------------------ | ------------------------------- |
| `AnalyzeSentiment` | ❤️ Sentiment analysis | "I love this!" | { score:1, positive:["love"] }|
| `SummarizeText` | ✂️ Extractive summary | (3 sentences) long paragraph | (2 sentence summary) |
| `CalculateReadability` | 📖 Flesch-Kincaid score | "The quick brown fox jumps..." | { score:70, grade:5 } |
| `DetectLanguage` | 🌍 Language detection | "Bonjour le monde" | "fr" |
| `CompareTexts` | 🔍 Text diff | { text1, text2 } | { added:[...], removed:[...] } |
---
## 🎨 Custom Operations
_Easily plug in your own workflows! ✨_
### ➕ Adding Custom Operations
You can extend functionality by adding your own custom operations:
```typescript
import { Tools } from 'textanalysis-tool';
const analyser = new Tools.Analyser("Sample text for custom operation");
// Add a simple custom operation
await analyser.addCustomOperation(
"surroundWithAsterisks", // Command name
"Surround With Asterisks", // Log name
{
operation: (text) => `*${text}*`, // Operation function
isEnabled: true, // Enable immediately
metadata: { decorationType: "asterisks" } // Additional metadata
}
);
// Run the analysis with the custom operation
const result = await analyser.main();
console.log(result.output); // "*Sample text for custom operation*"
console.log(result.metadata.custom.surroundWithAsterisks); // { decorationType: "asterisks" }
```
### 📊 With Metadata Extraction
```typescript
import { Tools } from 'textanalysis-tool';
const analyser = new Tools.Analyser("The code is 12345 and the pin is 6789");
// Add a custom operation with metadata extraction
await analyser.addCustomOperation(
"extractNumericCodes",
"Extract Numeric Codes",
{
operation: (text) => text, // Operation doesn't change the text
isEnabled: true,
metadataExtractor: (text) => {
const allNumbers = text.match(/\d+/g) || [];
return {
codes: allNumbers,
codeCount: allNumbers.length
};
}
}
);
const result = await analyser.main();
console.log(result.metadata.extractNumericCodes);
// Output: { codes: ["12345", "6789"], codeCount: 2 }
```
---
## 🚀 Advanced Usage
### 🔘 Toggling Operations
Enable or disable operations dynamically:
```typescript
import { Tools } from 'textanalysis-tool';
const analyser = new Tools.Analyser("Sample text with 123 numbers");
// Enable specific operations
await analyser.toggleOperation(Tools.Operations.RemoveNumbers, true);
await analyser.toggleOperation(Tools.Operations.CountCharacters, true);
// Run analysis
let result = await analyser.main();
console.log(result.output); // "Sample text with numbers"
// Disable and enable different operations
await analyser.toggleOperation(Tools.Operations.RemoveNumbers, false);
await analyser.toggleOperation(Tools.Operations.ConvertToUppercase, true);
// Run analysis again with new settings
result = await analyser.main();
console.log(result.output); // "SAMPLE TEXT WITH 123 NUMBERS"
```
### 🔀 Managing Multiple Operations
```typescript
import { Tools } from 'textanalysis-tool';
const analyser = new Tools.Analyser("Hello, world! 123");
// Enable all available operations
await analyser.enableAllOperations();
// Run with all operations
let result = await analyser.main();
console.log("With all operations:", result.output);
// Disable all operations
await analyser.disableAllOperations();
// Enable only specific operations
await analyser.toggleOperation(Tools.Operations.RemovePunctuations, true);
await analyser.toggleOperation(Tools.Operations.ConvertToUppercase, true);
// Run with only selected operations
result = await analyser.main();
console.log("With selected operations:", result.output); // "HELLO WORLD 123"
```
### 🔄 Resetting Text
```typescript
import { Tools } from 'textanalysis-tool';
const analyser = new Tools.Analyser("Original text", {
[Tools.Operations.ConvertToUppercase]: true
});
// Run first analysis
let result = await analyser.main();
console.log(result.output); // "ORIGINAL TEXT"
// Reset with new text
await analyser.resetText("New content");
// Run analysis again
result = await analyser.main();
console.log(result.output); // "NEW CONTENT"
```
### ✂️ Truncating Text
```typescript
import { Tools } from 'textanalysis-tool';
const longText = "This is a very long text that needs to be truncated to a reasonable length.";
const analyser = new Tools.Analyser(longText, {
[Tools.Operations.Truncate]: {
maxLength: 20,
suffix: "..." // Optional, defaults to "..."
}
});
analyser.main().then(result => {
console.log(result.output); // "This is a very long..."
});
```
---
## 📘 API Reference
### 🧰 Tools.Analyser Class
The main class for text analysis operations.
**Constructor:**
```typescript
constructor(raw_text: string, options: AnalyserBuiltInOptions = {})
```
**Properties:**
| Property | Type | Description |
|----------|------|-------------|
| `raw_text` | string | The input text being analyzed |
| `count` | number | The character count |
| `alphacount` | number | The alphabetic character count |
| `numericcount` | number | The numeric character count |
| `wordCount` | number | The word count |
| `sentenceCount` | number | The sentence count |
| `urls` | string[] | Extracted URLs |
| `emails` | string[] | Extracted email addresses |
| `phoneNumbers` | string[] | Extracted phone numbers |
| `hashtags` | string[] | Extracted hashtags |
| `mentions` | string[] | Extracted mentions |
| `operations` | string[] | Log of operations performed |
| `availableOperations` | Record<string, string> | All available operations |
| `options` | AnalyserBuiltInOptions | Current operation options |
**Methods:**
| Method | Parameters | Return Type | Description |
|--------|------------|-------------|-------------|
| `main` | None | Promise<AnalyserResult> | Executes all enabled operations |
| `addCustomOperation` | commandName: string, logName: string, config: object | Promise<void> | Adds a custom operation |
| `toggleOperation` | commandName: string, isEnabled: boolean | Promise<void> | Enables/disables an operation |
| `enableAllOperations` | None | Promise<void> | Enables all operations |
| `disableAllOperations` | None | Promise<void> | Disables all operations |
| `resetText` | newText?: string | Promise<void> | Resets text to original or new value |
**Static Methods:**
| Method | Parameters | Return Type | Description |
|--------|------------|-------------|-------------|
| `createWithEnabledOperations` | text: string, operations: (keyof typeof Operations)[] | Analyser | Creates instance with specific operations |
| `batch` | texts: string[], options: AnalyserBuiltInOptions | Promise<AnalyserResult[]> | Processes multiple texts with same options |
### 🧩 Tools.Operations Enum
Enum of all built-in operations:
```typescript
export enum Operations {
RemovePunctuations = "removepunc",
RemoveNumbers = "removenum",
RemoveAlphabets = "removealpha",
RemoveSpecialChars = "removespecialchar",
RemoveNewlines = "newlineremover",
RemoveExtraSpaces = "extraspaceremover",
ExtractUrls = "extractUrls",
ExtractEmails = "extractEmails",
ExtractPhoneNumbers = "extractPhoneNumbers",
ExtractHashtags = "extractHashtags",
ExtractMentions = "extractMentions",
ConvertToUppercase = "fullcaps",
ConvertToLowercase = "lowercaps",
ConvertToTitleCase = "titlecase",
CountCharacters = "charcount",
CountAlphabets = "alphacount",
CountNumbers = "numcount",
CountAlphanumeric = "alphanumericcount",
CountWords = "wordcount",
CountSentences = "sentencecount",
ReverseText = "reversetext",
Truncate = "truncate",
AnalyzeSentiment = "analyzeSentiment",
SummarizeText = "summarizeText",
CalculateReadability = "calculateReadability",
DetectLanguage = "detectLanguage",
CompareTexts = "compareTexts",
}
```
### 🔍 Tools.ToolsConstant Class
Contains regular expression patterns used throughout the library:
```typescript
export class ToolsConstant {
static readonly regex = {
alphabets: /[a-zA-Z]/g,
numbers: /\d/g,
punctuations: /[!"#$%&'()*+,\-./:;<=>?@[\\\]^_`{|}~]/g,
specialCharacters: /[^a-zA-Z0-9\s!"#$%&'()*+,\-./:;<=>?@[\\\]^_`{|}~]/g,
urls: /https?:\/\/\S+/gi,
newlines: /^\s*$(?:\r\n?|\n)/gm,
extraSpaces: / +/g,
character: /[^\s\p{Cf}]/gu,
email: /[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}/g,
phoneNumber: /(?:\+\d{1,3}[-\s]?)?\(?\d{3}\)?[-\s]?\d{3}[-\s]?\d{4}/g,
hashtags: /#[a-zA-Z0-9_]+/g,
mentions: /@[a-zA-Z0-9_]+/g
};
}
```
### 📋 Interface Types
**AnalyserBuiltInOptions:**
```typescript
type AnalyserBuiltInOptions = Partial<Record<Operations | string, boolean | any>>;
```
**AnalyserResult:**
```typescript
interface AnalyserResult {
purpose: string;
output: string;
metadata: {
counts: {
characterCount: number;
alphabetCount: number;
numericCount: number;
wordCount?: number;
sentenceCount?: number;
};
urls?: string[];
emails?: string[];
phoneNumbers?: string[];
hashtags?: string[];
mentions?: string[];
custom?: {
[key: string]: any;
};
};
operations: string[];
builtInOperations: string[];
customOperations: string[];
executionTime?: number;
}
```
**AnalyserResult (Extended):**
```typescript
interface AnalyserExtendedResult extends AnalyserResult {
sentiment?: SentimentResult;
summary?: string;
readability?: ReadabilityResult;
languageDetection?: LanguageDetectionResult;
textComparison?: TextDiffResult;
}
```
**TruncateConfig:**
```typescript
interface TruncateConfig {
maxLength: number;
suffix?: string;
}
```
### 🔌 Extensions
## 😊 SentimentAnalyzer
A utility class for analyzing text sentiment.
**Constructor:**
```typescript
constructor()
```
**Properties:**
| Property | Type | Description |
|----------------|----------------|--------------------------------------|
| `positiveWords`| `Set<string>` | The set of positive sentiment words |
| `negativeWords`| `Set<string>` | The set of negative sentiment words |
**Methods:**
| Method | Parameters | Return Type | Description |
|--------------------|---------------------------------------------------------|-------------------|--------------------------------------------------|
| `analyze` | `text: string` | `SentimentResult` | Executes sentiment analysis on the input text |
| `addCustomLexicon` | `lexicon: { positive?: string[]; negative?: string[] }` | `void` | Adds custom positive/negative words to the lexicons |
**Static Methods:**
_None_
---
## 📝 TextSummarizer
A utility class for generating extractive summaries from text.
**Constructor:**
```typescript
constructor()
```
**Properties:**
| Property | Type | Description |
|-------------|----------------|--------------------------------------------------|
| `stopWords` | `Set<string>` | The set of stop words to ignore when scoring sentences |
**Methods:**
| Method | Parameters | Return Type | Description |
|----------------------|------------------------------------------|-------------|--------------------------------------------------------|
| `extractiveSummarize` | `text: string, sentenceCount?: number` | `string` | Generates an extractive summary with the top sentences |
| `addStopWords` | `words: string[]` | `void` | Adds custom words to the stop‑word list |
**Static Methods:**
_None_
---
## 📊 TextStatistics
A utility class for computing readability metrics such as the Flesch–Kincaid score.
**Constructor:**
```typescript
// No constructor parameters; all methods are stateless
constructor()
```
**Properties:**
_None_
**Methods:**
| Method | Parameters | Return Type | Description |
|----------------------------|----------------|---------------------|-----------------------------------------------|
| `fleschKincaidReadability` | `text: string` | `ReadabilityResult` | Calculates readability and grade level scores |
**Static Methods:**
_None_
---
## 🌐 LanguageDetector
A trigram‑based language detection utility supporting English, Spanish, French, and German.
**Constructor:**
```typescript
constructor()
```
**Properties:**
| Property | Type | Description |
|--------------------|------------------------------------|--------------------------------------------------|
| `languageProfiles` | `Map<string, Map<string, number>>` | Mapping of language names to frequency profiles |
**Methods:**
| Method | Parameters | Return Type | Description |
|----------------------|----------------------------------------------|---------------------------|------------------------------------------------|
| `detect` | `text: string` | `LanguageDetectionResult` | Detects the most likely language for the text |
| `addCustomLanguage` | `language: string, profile: Record<string, number>` | `void` | Registers a new language profile |
**Static Methods:**
_None_
---
## 🔍 TextDiff
A utility class for comparing two texts and computing similarity metrics.
**Constructor:**
```typescript
// No constructor parameters; all methods are invoked directly
constructor()
```
**Properties:**
_None_
**Methods:**
| Method | Parameters | Return Type | Description |
|-----------|--------------------------------|------------------|----------------------------------------------------------|
| `compare` | `text1: string, text2: string` | `TextDiffResult` | Computes similarity, edit distance, and common substrings |
**Static Methods:**
_None_
---
## 🧩 Interfaces & Types
### 😀 SentimentResult
```typescript
interface SentimentResult {
score: number;
positiveWordCount: number;
negativeWordCount: number;
totalWords: number;
classification: SentimentClassification;
}
```
### 🏷️ SentimentClassification
```typescript
type SentimentClassification = "positive" | "negative" | "neutral";
```
### 📖 ReadabilityResult
```typescript
interface ReadabilityResult {
readabilityScore: number;
gradeLevel: number;
wordCount: number;
sentenceCount: number;
syllableCount: number;
avgWordsPerSentence: number;
avgSyllablesPerWord: number;
complexity: string;
}
```
### 🗣️ LanguageDetectionResult
```typescript
interface LanguageDetectionResult {
detectedLanguage: string;
confidence: number;
scores: Record<string, number>;
}
```
### 🔄 TextDiffResult
```typescript
interface TextDiffResult {
similarity: number;
editDistance: number;
commonSubstrings: Array<{ substring: string; length: number }>;
wordDifference: {
added: string[];
removed: string[];
unchanged: string[];
addedCount: number;
removedCount: number;
unchangedCount: number;
};
}
```
---
## 🤝 Contributing
Love it? Spread the word! 🌍
1. Fork the repo 🍴
2. Create a branch `git checkout -b feature/amazing-feature` 🌿
3. Commit your changes `git commit -m 'Add some amazing feature'` ✨
4. Push and create a PR `git push origin feature/amazing-feature` 🚀
---
## 📝 License
This project is licensed under the MIT License - see the [LICENSE](https://github.com/abindent/textanalyser/blob/master/LICENSE) file for details.# 🌟 Text Analysis Tools 🌟