crawler-ninja

Version:

A web crawler made for the SEO based on plugins. Please wait or contribute ... still in beta

6 lines (5 loc) • 154 B

HTML

<body> <p>This is the page 4</p> <a href="/timeout">This is a timeout</a> <a href="/">it should crawl this link even if a timeout</a> </body>