Dupe content checker?

Started by Rumbas, January 21, 2015, 12:58:27 PM

Previous topic - Next topic

Rumbas

We run quite a few audits for clients and I'm always looking for dupe content or near dupe content. However I could really use a tool to automate the process.

Anyone know of anything worth working with out there?

littleman

That would be a very useful tool.  You'd think a company like Majestic or another link analysis company would have such a thing because they are already spidering the web anyway.

buckworks


BoL

I've used Copyscape's API in the past, a co-worker would look at the results to check whether content was unique or not. iirc they have some kind of similarity/unique metric.

ergophobe

I always think of Copyscape as a tool for catching plagiarists, so I don't know as it's what you're looking for

Copyscape is a 1:many tool - you input the one text you care about and it searches many texts to see if your text is plagiarized

It sounds like you're looking for a many:many tool - something that will look at many texts and see if there are matches among them.

I think to use Copyscape that way you're going to need to pipe stuff in - some sort of

bash>Screaming Frog http://example.com | Copyscape

type of workflow.