What can you do with this plugin?
Crawlomatic Multisite Scraper Post Generator Plugin for WordPressIs a breakthroughWebsite scraping and scraping, post generator auto blog pluginIt uses website crawling and crawling to turn your website into an automatic blog or even a money-making machine!Get content from almost any webpage! You no longer need to register and provide APIs with limited access. You can also retrieve data from websites that don’t provide APIs.Schedule it once and let it autopilot your posts for you like a master 7/24!
How does it work?
This plugin will crawl the seed URL you give it (crawling means it will search all the links the page contains) and will visit and extract content from each crawled URL.The crawling process is customizable: you can set the crawl depth, crawl rate, maximum number of articles to be crawled, crawl only links with a specific class or ID, and more custom settings.
Crawlomatic v2.0 Update
In the v2.0 update, a new shortcode for live scraping was added to the plugin: [crawlomatic-scraper]. This new feature makes this plugin an easy-to-implement WordPress web data extractor.Therefore, it can be used to display real-time data from any website directly into your post, page or sidebar.It also temporarily caches scraped content so your site doesn't overcommit resources.You can use this plugin to include live stock quotes, cricket or football scores or any other generic content from the public domain!New features included in this update:
- The scraped output can be displayed via custom template tags, shortcodes in pages, posts and sidebars (via text widgets).
- Configurable cache for crawled data. The cache timeout can be defined in minutes for each captured data.
- Configurable user agent for scraper that can be set for each scrape.
- Configurable default settings such as enable, user agent, timeout, caching, error handling.
- Multiple ways to query content - CSS selectors, XPath or regular expressions, auto-detection.
- Various parameters for parsing the content.
- Option to pass post parameters to the URL to crawl.
- The captured content is dynamically converted into a specified character encoding to use different character sets to capture data from the site.
- Create scraped pages on the fly with dynamically generated URLs to crawl or post parameters based on the page's fetch or post parameters.
- Callback function for advanced analysis of deleted data.
an examinationv2 updated official documentation, Browse through the examples and check out the FAQ to make a perfectly optimized web crawler.
More information about plugins
You can scrape content from almost every website you open in your browser.If the content is loaded using JavaScript, the plugin can be combined with PhantomJS to crawl the content generated by JavaScript.Additionally, you can automatically generate an unlimited number of custom website crawls and crawls.
Other plug-in functions:
- mise à jour v2.3.5 :Execute your own JavaScript code on the scraped HTML and scrape the results - this feature is only available when scraping with a headless browser (Puppeteer/Tor/PhantomJS) or HeadlessBrowserAPI
- mise à jour v2.2.1 :Crawl the RSS feed for links and crawl the articles listed there
- actualización v2.2.0: Use HeadlessBrowserAPIScrape JavaScript-generated HTML content from any website on the internet without installing anything on your server (except this plugin) -Tutorial video
- actualització v2.1.0:shave. Dark web onion sites using Tor Browser and Puppeteer! -Tutorial video
- mise à jour v2.0.0 :Added Live Scraper shortcode to provide more crawling control and crawling capabilities: [crawlomatic-scraper]
- v1.7.1-Update:Support for sitemap scraping -Video tutorial
- atualização v1.6.5:Added visual content selector support -Video tutorial
- v1.6.0-Update:Added the ability to make screenshots of the grab page and use them in the generated post content-Video tutorial
- v1.5.2-Update:Ability to shorten outgoing (published source) links (and monetize them), useCorto circuito. YingshiLink shortening service-Shortened link example
- actualización v1.4.8:Added JavaScript execution support for scraping pages - requires PhantomJS to be installed on the server -How to install PhantomJs? – Video tutorial
- actualización v1.4.4:Added ability to set multiple proxies for scraping pages. The plugin will randomly choose one on each page visit
- Actualización v1.4.0:Added the function of paging crawling (the crawling of the article will continue on the next page of the seed page).
- Actualización v1.4.0:Added ability to import product prices for scraped products (WooCommerce compatible) + dropshipping price auto modification -Video tutorial
- Actualización v1.4.0:Added the function of increasing the price of imported products by a fixed number or multiplying it with a predefined number (very valuable for direct sales!)
- v1.2.8 update:Added paginated post import support (to a single crawled post) check:VIDEO.
- v1.2.4 update:Added ability to set proxy for scraping pages
- v1.2.3 update:Added option to fetch pages from google cache when direct fetch fails (blocked)
- Google Translate Support-Choose the language in which you want to publish your article
- Text Spinner Support - Automatically modify generated text, change words with synonyms - Built-in, Best Spinner, SpinRewriter, WordAI, TurkceSpin and others - Huge SEO value!
- Customizable post status (published, draft, pending, private, junk)
- Shortcode to list all posts generated by this plugin: [crawlomatic-list-posts type => 'any', order => 'ASC', 'orderby' => 'date', 'posts' => 50, 'category ' => '', 'ruleid' => '']
- Crawl and grab can be set to respect bots. txt files for websites and bots crawl HTML headers of pages
- Automatically generate post categories or tags from market items
- Manually add post categories or tags to items
- Choose if you want to update published posts
- Send custom cookie with request to crawled web page (authentication)
- Generate posts or pages or any custom post types
- Embed videos from YouTube, Vimeo, Flickr, IGN, Ustream. tv and DailyMotion use website scraping and scraping
- Define publishing restrictions: Do not publish posts without pictures, do not publish posts with short/long titles/contents
- Automatically generate featured images for posts
- Enable/disable comments, pingbacks or quotes for generated posts
- Customize the post title and content (including various related post shortcodes)
- "Keyword Substitution Tool"-its purpose is to define keywords that can be automatically replaced with your affiliate links. These keywords appear anywhere in the content of your website.For example, you can define the keyword "codecanyon" and replace it to point tohttp://www.codecanyon.net/?ref=user_nameIt appears anywhere in the content of your website.
- 'Random Sentence Generator Tool' (Related sentence-as you defined)
- Option to automatically delete generated posts after a period of time
- Detailed plug-in activity record
- Scheduled rules to run
- Custom field support for generating posts
- Custom taxonomy support for generated posts
- Unlimited scraping variable import (crawl pages section of unlimited import)
- Choose to copy the image locally or not
- Option to add canonical meta tags to generated posts
- Maximum/minimum title length post limit
- Maximum/minimum content length post limit
- Only add posts when pre-defined required keywords are found in the title/content
- Only add posts if the predefined prohibited keywords are not found in the title/content
- Save and restore plug-in list from file
Test this plugin
- You can use 'to test the functionality of the pluginTest site generator'. Here you can try all the functions of the plug-in. Please note that the generated test blog will be automatically deleted after 24 hours.
Plug-in requirements
- PHP DOM -> How to install it (if you don't have it, but probably you already have it):http://php.net/manual/en/dom.setup.php
- PHP5.0 or higher
- dom, mbstring, iconv and json extensions (enabled by default)
For more information on how to configure the plugin, check out this1 hour long tutorial video, It covers the full feature set of the plugin.
Need support?
Please visit usknowledge base, It may answer your question or solve your problem. If not, please email meSupport @coderevolution. LuoI will reply as soon as possible.Changelog:
Version 1.0 Release Date 2017-08-15First version released!Version 1.1 Release Date 2017-08-16
Fixed some small issuesVersion 1.2 Release Date 2017-08-17
Added the ability to crawl page by div class or idVersion 1.2.1 Release Date 2017-08-18
Fixed incompatibility with some WordPress installsVersion 1.2.2 Release Date 2017-08-22
Added a shortcode to display post generated by this pluginVersion 1.2.3 Release Date 2017-08-30
Added an option to crawl the page from Google cache when direct crawling fails (blocked)Version 1.2.4 Release Date 2017-08-31
Added the ability to set proxies for crawling pagesVersion 1.2.5 Release Date 2017-09-04
Added the canonicalization for generated articlesVersion 1.2.6 Release Date 2017-09-13
Made the plugin timezone awareVersion 1.2.7 Release Date 2017-09-14
Fixed post date for non gmt blogsVersion 1.2.8 Release Date 2017-09-23
Added paginated post importing supportVersion 1.2.9 Release Date 2017-09-27
BugfixesVersion 1.3.0 Release Date 2017-09-28
Fixed rule restoreVersion 1.3.1 Release Date 2017-10-20
Fixed featured image generationVersion 1.3.2 Release Date 2017-10-22
Added crawling helperVersion 1.3.3 Release Date 2017-11-06
Fixed a memory issueVersion 1.3.4 Release Date 2017-11-07
BugfixesVersion 1.3.5 Release Date 2017-12-14
Fixed class selector not working in all casesVersion 1.3.6 Release Date 2017-12-18
Added the ability to specify a custom user agent for each crawled webpageVersion 1.3.7 Release Date 2018-01-20
Added a new text spinner service: SpinrewriterVersion 1.3.8 Release Date 2018-01-22
Plugin can now continuously import contentVersion 1.3.9 Release Date 2018-02-02
Fixed issue when multiple crawl classes where specifiedVersion 1.4.0 Release Date 2018-02-22
Major update: added the ability to crawl imported product prices (WooCommerce compatible) Added the ability to crawl serial content (paged crawling - crawling for articles will continue on the next page)Version 1.4.1 Release Date 2018-03-07
BugfixesVersion 1.4.2 Release Date 2018-03-21
Fixed a duplicate posting issueVersion 1.4.3 Release Date 2018-03-22
Fixed a critical issue with multiple rule runningVersion 1.4.4 Release Date 2018-04-04
Added the ability to define multiple proxies. The plugin will select one at random at each page accessVersion 1.4.5 Release Date 2018-07-13
Updated built-in readability moduleVersion 1.4.6 Release Date 2018-07-16
Critical bugfixesVersion 1.4.7 Release Date 2018-07-19
Added the ability to not translate linksVersion 1.4.8 Release Date 2018-09-05
Added JavaScript execution support for crawled pages - requires PhantomJS installed on serverVersion 1.4.9 Release Date 2018-09-18
BugfixesVersion 1.5.0 Release Date 2018-09-24
Added the ability to add custom post taxonomies from crawled content Added the ability to add unlimited crawled variables to posts's content/ meta/ taxonomiesVersion 1.5.1 Release Date 2018-10-16
Fixed issue when importing large pagesVersion 1.5.2 Release Date 2018-10-24
Added the ability to shorten links using Shorte.stVersion 1.5.3 Release Date 2018-10-29
Fixed issue when importing paginated postsVersion 1.5.4 Release Date 2018-11-06
Added the ability to strip HTML elements by tag name (div,a,span,etc.)Version 1.5.5 Release Date 2018-11-07
Added WooCommerce product category creation supportVersion 1.5.6 Release Date 2018-12-16
Added nested importing support - import mixed content into a single post, from multiple plugins created by CodeRevolutionVersion 1.5.7 Release Date 2018-12-16
Added the ability to define a list of URLs to skip from crawling and importingVersion 1.5.8 Release Date 2019-01-08
Added the ability to import royalty free images for created postsVersion 1.5.9 Release Date 2019-01-12
Added Gutenberg blocks supportVersion 1.6.0 Release Date 2019-02-01
Added the ability to make screenshots of scraped pagesVersion 1.6.1 Release Date 2019-02-06
Improved compatibility with some crawled pagesVersion 1.6.2 Release Date 2019-04-19
Security updateVersion 1.6.3 Release Date 2019-05-15
Fixed some recently found bugs with post paginationVersion 1.6.4 Release Date 2019-05-17
Added support for TurkceSpin content spinnerVersion 1.6.5 Release Date 2019-05-27
Added a much demanded new feature: Visual Content Selector for assigning scraped page content Added the ability to scrape pages from bottom to top Added the ability to replace words in scraped content Other minor bug fixes and functionality improvementsVersion 1.6.6 Release Date 2019-07-26
Fixed timeout issue with some crawled pages Many small issues fixed and features improvedVersion 1.6.7 Release Date 2019-08-05
Fixed issue with Google TranslateVersion 1.6.8 Release Date 2019-11-15
WordPress 5.3 compatibility updateVersion 1.6.9 Release Date 2020-05-11
New features added for content templates Bugfix updateVersion 1.7.0 Release Date 2020-07-21
Added support for scraping more sitesVersion 1.7.1 Release Date 2020-09-28
Added the ability to crawl sitemaps and to scrape posts linked in them Added the ability to respect the directives set in the robots.txt filesVersion 2.0.0 Release Date 2020-12-08
Added a new shortcode and Gutenberg block alternative that will enable live scraping of any website Major performance improvement Fixed reported bugsVersion 2.1.0 Release Date 2021-01-02
Added support for using the Tor Browser to crawl dark web sites! Scrape .onion websites like you would scrape any other public website!Version 2.1.1 Release Date 2021-01-04
Added the ability to crawl and scrape pages using POST requests (POST form submission scraping support)Version 2.2.0 Release Date 2021-01-14
Added support for HeadlessBrowserAPI to scrape JavaScript rendered content with easeVersion 2.2.1 Release Date 2021-01-16
PHP 8 compatibility update Added support for crawling links from RSS feedsVersion 2.2.2 Release Date 2021-01-28
Fixed rare issue when saving importing rule settings on some PHP 8 configurationsVersion 2.2.3 Release Date 2021-02-01
Improved content extraction algorithmVersion 2.2.4 Release Date 2021-02-17
Added the ability to not spin posts generated by specific rulesVersion 2.2.5 Release Date 2021-03-07
Added the ability to enter multiple URLs (one per line) to be crawled and scrapedVersion 2.2.6 Release Date 2021-03-07
Visual Selector improvements - now it will be able to use HeadlessBrowserAPI/Puppeteer/PhantomJS/Tor to visualize scrape contentVersion 2.2.7 Release Date 2021-04-02
Fixed rare issues when crawling links with URL parametersVersion 2.2.8 Release Date 2021-04-07
Fixed rare issues with relative URL paths in crawled contentVersion 2.2.9 Release Date 2021-05-03
Added the ability to skip publishing of new posts if not images found (separately, for each rule)Version 2.3.0 Release Date 2021-05-19
Added the ability to make screenshots of websites using the HeadlessBrowserAPI featureVersion 2.3.1 Release Date 2021-06-10
Fixed content extracting/stripping in case of some websites with dynamically generated contentVersion 2.3.2 Release Date 2021-07-15
Added multiple Regex expression support (for content stripping and replacement)Version 2.3.3 Release Date 2021-07-18
Added SpinnerChief to the supported premium text spinners (SpinRewriter, The Best Spinner, WordAI, TurkceSpin)Version 2.3.4 Release Date 2021-07-19
Added Bing Translator support (next to Google Translator and DeepL Translator)Version 2.3.5 Release Date 2021-08-06
Added the ability to execute your own custom JavaScript on scraped pages when using headless browsers (PhantomJS/Puppeteer/Tor) or HeadlessBrowserAPI (XSS - cross site scripting feature) and scrape the resulting HTML contentVersion 2.3.6 Release Date 2021-08-30
Added the ability to set featured images of posts from website screenshots Added the ability to remove HTML content (leave text only) of XPath matched contentVersion 2.3.7 Release Date 2021-09-02
Added the ability to set local storage objects when scraping websites (these are similar to cookies, their usage is supported only when using headless browsers or HeadlessBrowserAPI in conjunction with the plugin)Version 2.3.8 Release Date 2021-09-15
Added the ability to set the WPML language to created postsVersion 2.3.9 Release Date 2021-10-19
WooCommerce product scraping related improvements
Are you already a customer?
If you have already purchased and tried it, please contact me in the comments section of the project and give me feedback so that I can turn it into a better WordPress plugin!WordPress 5.8 er blevet testet!
Disclaimer
Do you like our work and want more?
CheckThis MEGA plug-in package.
Contents
Crawlomatic Multisite Scraper Post Generator Plugin for WordPress [Free Download]
Crawlomatic Multisite Scraper Post Generator Plugin for WordPress [Nulled]
WordPress » Miscellaneous