crawler-ninja

Version:

A web crawler made for the SEO based on plugins. Please wait or contribute ... still in beta

6 lines (5 loc) • 180 B

HTML

<body> <p>This is the page 5</p> <a href="http://www.thisnotcorrect.abc">This is a dns error</a> <a href="/">it should crawl this link even if an dns error</a> </body>