twx-media-scrapper/README.md

Twitter/x.com media content extractor via Puppeteer and TwitterDL

# twx-media-scrapper

A professional Node.js module to extract direct video URLs from [X (formerly Twitter)](https://x.com), with automated authentication using Puppeteer. It supports headless login, session caching, and CLI-based two-factor code input (OTP) when required.

---


## 🚀 Features

- ✅ Headless login to [x.com](https://x.com)
- 🔐 Automatic token (`Bearer`) and cookie retrieval
- 🧠 CLI prompt for verification code (if X.com asks for OTP)
- 📦 Auth session cache saved in `auth_cache.json`
- 🎥 Extracts all available video qualities using [`twitter-downloader`](https://www.npmjs.com/package/twitter-downloader)
- 🧰 Fully compatible with server/VPS environments

---


## 📦 Installation

```bash
npm install twx-media-scrapper puppeteer twitter-downloader
```

> `puppeteer` and `twitter-downloader` are required as peer dependencies.

---


## ✨ Usage

```js
const { listTwitterMediaPath } = require("twx-media-scrapper");

// Ganti dengan akun dummy Twitter/X kamu
const email = "your@email.com";
const username = "yourUsername";
const password = "yourPassword";

const twitterVideoUrl = "https://x.com/elonmusk/status/1911984444459667639";

listTwitterMediaPath(twitterVideoUrl, email, username, password)
  .then(videos => console.log("🎥 List Videos:", videos))
  .catch(err => console.error("❌ Error:", err.message));

```

---




## 📊 Sample Output

Here is an example of the JSON result returned by the `listTwitterMediaPath()` function:

```json
[
  {
    "type": "video",
    "cover": "https://pbs.twimg.com/amplify_video_thumb/1911851777877487616/img/295hD94p38eqAET1.jpg",
    "duration": "1:18",
    "expandedUrl": "https://x.com/RedWave_Press/status/1911852100230771088/video/1",
    "videos": [
      {
        "bitrate": 288000,
        "content_type": "video/mp4",
        "quality": "482x270",
        "url": "https://video.twimg.com/amplify_video/1911851777877487616/vid/avc1/482x270/0P1rL4U7-Jm1PSk6.mp4?tag=16"
      },
      {
        "bitrate": 832000,
        "content_type": "video/mp4",
        "quality": "644x360",
        "url": "https://video.twimg.com/amplify_video/1911851777877487616/vid/avc1/644x360/CTT8V1u8HsSsauzc.mp4?tag=16"
      },
      {
        "bitrate": 2176000,
        "content_type": "video/mp4",
        "quality": "1288x720",
        "url": "https://video.twimg.com/amplify_video/1911851777877487616/vid/avc1/1288x720/_Uy9S8dPYHCzAVMK.mp4?tag=16"
      }
    ]
  }
]
```

This output includes:
- **type**: Content type (e.g. `video`)
- **cover**: Thumbnail image of the video
- **duration**: Video length in minutes and seconds
- **expandedUrl**: Direct link to the video on X.com
- **videos**: List of available video streams with different bitrates and resolutions

## 💾 Session Caching

Upon successful login, a local cache file `auth_cache.json` is created, storing the session token and cookie:

```json
{
  "authorization": "Bearer AAAAAAA...",
  "cookie": "auth_token=...; ct0=...; ..."
}
```

This file is reused automatically for subsequent requests.  
If the token is invalid or expired, the module will re-authenticate.

---


## 🔐 OTP Handling (Two-Factor Auth)

When logging in from a new IP address or location, X.com may request an OTP sent to your email or phone.  
The module automatically detects this challenge and prompts the user in the terminal:

```
Twitter requires a verification code.
Enter the OTP sent to your email/phone: ██████
```

The login process continues once the code is entered.

---


## ⚠️ Notes

- Intended for use with non-private accounts.
- Recommended to use a dummy account for scraping or automation purposes.
- Not intended for downloading protected or copyrighted content.

---


## 📄 License

MIT License  
© 2025 [Harris Munahar]

---


## 🙋 Support & Contribution

Feel free to open issues or pull requests for improvements or fixes.  
If you encounter login issues due to layout changes on X.com, updates may be required in the login automation.

---