Alternate Subject Line:  The firehose is going to kill me.

My Craigslist search engine is based on the simplepie php class for feed readers. Most people use rss readers to read news articles, I just bastardized it to read CL, then sliced & diced the info once I pulled it in.  What the layman doesn't understand is that the reader isn't the really hard part, it's the subsequent file-handling and slicing&dicing that provides the superpowers.

BUT I also read a metric sh##-ton of news articles on any given day.  So I'm taking what I've learned about the superpowers I needed in CL and applying them to my news sources.  With CL, the heaviest lifting was in building, maintaining, and applying positive and/or negative filters.  I see the same issues with news. For instance, given the recent independence vote, I'd like to be able to quickly mask out any mention of 'Scotland' for the next few days and *permanently* ban 'Kardashian' or 'Kayne' while green-flagging for possible reading any article mentioning 'North Carolina.'

We have a few heavy-readers here. What's on your wish list?

Johnny Five:

This is awesome.  Conservatively/truthfully, this *exponentially* reduces time spent on my reading list. (and I've roughly doubled the routine sources while testing).

All this and I've not yet added the filters.

Surely there is only so much one man can read!

Next you will be working out a way to connect directly to the brain, matrix styley

>connect directly

It may end up connecting somewhere.  I'm databasing (flatfiles, of course) on the fly.

So I need a somewhat active, noisy/spammy artcile source to use for live filtering tests while not having to care if I get my IP banned ...I can now confirm that nothing meets that requirement better than foxnews. Followed hotly by hufingtonpost.

>only so much

Within the context of US news, take a look at the filter arrays just for the headline field that I've harvested so far just today. By the time I get through, there may be no news left.

array('bieber','clinton','christie','cuomo','de blasio','gun control','hillary','jennifer lawrence','jimmy fallon','john stewart','kardash','kayne','nra ',' palin ','sarah palin','taylor swift','tim cook')

array('airstrike','air strike','amazon','apple ',' apple','behead','blackberry','civilian','comcast','ebola ','espn ','facebook','fatah',' gaza','guinea',' hamas ','iphone',' iran',' iraq','iraqi',' isis','islam ','islamic','israel','jerusalem',' jews','jews ','jewish','kim jong','korea','liberia','muslim','nigeria','palestin','pentagon',' rebel ',' rebels ','samsung','shiite','sierra leone','syria','terror','time-warner',' twc',' uae ','ukraine','white house','yemeni')

array('eric holder','halloween','pumpkin',' ammo ','ammunit','assault','bisexual','breast cancer','buffett','carbon dioxide','charged','co2','colbert','college parent','college prof','college student','conan ','conservative',' cop ',' cops ','custody','debate','death row',' drone','drone ','drones ','emission','entertainm','firearm','fraternity',' freed ',' gay ','greenhouse gas',' gmo',' gop ',' gun ',' guns ',' hiv ','inmate ','jail',' juror',' killed','killer','killing','koch ','lottery','mayor ',' melting','murder','olympic','opinion:','prison','senate','senator',' shoot','sorority','steam','suspect ','transex','transgend',' trial',' tv ','tweet','twitter','vaccinat','valve','video:','walking dead','warming','zombie')

>only so much

Headline/Title filtering is working.  As of last night, I upped the feeds to 97 from 50 sources --full firehose force.  As anticipated, if you scrape the iphones, war & pestilence away there is only a trickle of news left.

>this *exponentially* reduces time spent
>I've roughly doubled the sources
>upped the feeds to 97 from 50 sources
>only a trickle of news left

Feed me, Seymour!

I'm not sure I can help.  On tablets I use News Republic apps which are okay for a quick morning overview of what is shaking the world.  I also use Zite, which is fantastic for coming up with fascinating articles that the others miss.  I have Zite further divided into topics like Archeology, History, World War II, Military, Naval and it is amazing what it dredges up and what the mainstream media ignores.  Comes down to sources, News Republic is mainstream sources mainly, whereas Zite is carefully curated expert sources wherever possible.  I know you already know this but mind the sources.


On computer, nothing beats NewsNow.co.uk, their sources are extensive and I have about 20 favorite topics bookmarked. I like that it updates every 5 minutes.

>not sure if

Yep, you did.

The feature set is pretty well defined now.  What I need is feeds.  Judging from the post-filter trickle, I can probably handle something in the range of 100. As it turned out, the biggest PITA is/was building a dynamic TOC that aligns with both the feeds themselves and how & when I want to read them  ...morning, afternoon, weekday, etc.

Spin-off thread:

http://th3core.com/talk/water-coolerextra/feed-me-seymour!-good-rss-feeds/

update: related idea at Disrupt Europe 2014 in London

http://techcrunch.com/2014/10/19/newsly-hack/

<added>
code is on github

https://github.com/mombi93/newsly