UNPKG

crawlkit

Version:

A crawler based on Phantom. Allows discovery of dynamic content and supports custom scrapers.

github.com/crawlkit/crawlkit

crawlkit/crawlkit

1 folder, 12 files

	Filename	Content Type	Size
	../
	worker/	–	16.6 kB
	Finder.js	text/javascript	2.15 kB
	Runnable.js	text/javascript	1.84 kB
	Runner.js	text/javascript	2.22 kB
	Scope.js	text/javascript	1.47 kB
	applyUrlFilterFn.js	text/javascript	1.63 kB
	crawl.js	text/javascript	2.58 kB
	createPhantomPool.js	text/javascript	2.18 kB
	errors.js	text/javascript	646 B
	index.js	text/javascript	8.86 kB
	isPhantomError.js	text/javascript	1.03 kB
	logger.js	text/javascript	273 B
	timedRun.js	text/javascript	615 B