It's been a thousand years since I've had to script anything, and basically, I'd have to relearn everything. Is there a software I can use that is simple and will allow me to quickly script and automate tasks?
I need to scrape google, but I want a bit more than an out-of-the-box scraper. Ideally, I'd like to be able to make the script quickly. GUI format preferred because I'm too damn lazy to get the *nix vi book out.
I'm going to try to hack Automator for Mac OSX, but I'd like to be able to do on a PC as well.
Most of the ones I come across these days are pricey. There's one, I'm blanking on the name, that's $500
import.io is $300/month
I've used Selenium IDE a bit. It's mostly a testing tool, but it will log you in and do various things, but saving the data is the problem there.
I've had mixed results with pulling HTML/XML into Google sheets (often chokes, or goes bad if a minor thing on the page changes). I've never tried to pull search results. But that's the idea here
https://www.labnol.org/internet/google-web-scraping/28450/
This (which I've never heard of, but which was linked on the web page of a now outdated tool) is $14/mo... might be an option
http://scraping.services/
iMacros might work for you.
Quote from: littleman on January 17, 2018, 05:32:41 PM
iMacros might work for you.
That's the one I was trying to remember. And now I remember the problem was I needed a feature of the Pro version, which is $500, but there is a personal edition for $99 and a free version for the simplest tasks.
I worked with a guy that was really into using it and was able to do quite a lot with the free version, but he ended up writing scripts and then feeding them to the plugin.
I just came across this too:
https://wildfire.ai/
Seems to have a lot of features, but I don't have any experience with it.
I was doing things with Selenium IDE that needed a few hundred steps and had multiple user login/logout actions. But it looks like even the Personal edition would handle that. I don't know what it was that needed Pro.
Anyway, thanks for the reminder... I have a couple ideas revived.
I think the main thing is that I wanted scripts to run at given times (i.e. that did not require a computer to be running), which Selenium (not Selenium IDE) can do.