The Core
Why We Are Here => Water Cooler => Topic started by: BoL on November 30, 2018, 12:49:19 AM
-
Part of my remit with Mojeek is to identify how to improve search engine results.
There's an emphasis on being a strong alternative to main search engines just now.
Obviously within this group there's an insane amount of experience in information retrieval and Google's weaknesses.
Free hand, how you you improve search results from your own perspective?
One line of thought I see often is simply offering an alternative to the strong brands that prevail in lots of results, and offering 10 different blue links. Separately, do you think there's a utility for offering a toolbar/form that provides a way to customise the ranking algorithm? Is it just an interesting toy or real world useful option?
-
Tim Mayer once told me that less than 2% of users used any advanced search features. So, it'd be a great feature for a very few users. Is it worth it? Maybe, if it generates buzz with the tech sites.
-
Most definitely with the algo tweaking thing, for surely a gimmick, but one that might resonate with people like yourself who understand the algo is a black box between you and the rest of the web.
-
What I would like is a right-hand checkbox listing of domains on a serp. Checking them would delete all pages from that domain in the serps, even if I went to deeper pages. Basically, it'd be like adding -domain to the search term, just more convenient. Even the 98% who never use advanced search might catch on. Ajax re-rendering when something is checked would be cool.
-
right-hand checkbox listing of domains on a serp
That ties in with my thinking. When I search I know if I want amazon and ebay in there or not and generally search directly on their domains. Often when looking for specialist shops, they are buried so deep as to be impossible to find, usually buried by ebay and amazon. I would be looking at making those 2 pay for the ads, and getting boutiques in listing for free.
Also Google is now often unable or unwilling to separate shopping search from enquiry. The word "review" still bring in the shopping images, the ads, and amazon, so I am below the fold before I find Wired. There are some words that are obvious I am not shopping yet. Thats one. As a merchant I have gamed that successfully in the past, and it can work. Now its so heavily gamed its of less value imho.
-
I will be the first to admit I don't know if anything I say is practical, business smart or search smart but I'll throw some thoughts out and you can filter them.
1, There is an old thread on WmW where the CEO (Marc?) mentions that a lot of big media sites block all crawlers except Google and Bing and that was a problem for Mojeek. I heard the same from the guys at the late Findx search engine. So if those doors are closed to you maybe Mojeek needs to deep spider and rank sites that Google only ranks in the long tail searches. Stuff like really good blogs, wikis and knowledge bases. Maybe white list say 10,000 good big noncommercial blogs. Not just one or two pages but deep crawl. There will be a lot of garbage posts but there also ought to be some cool stuff like software reviews, product reviews and geeky stuff. If you are blocked from the big mainstream, if you've spidered the 3rd and 4th tier sites but they are not enough why not bring in the artisans? If it looks promising expand but also play it up among the geeky communities for buzz.
2. Maybe it's just the way I search, but, Wikipedia aside, I rarely see any results from the interior pages of wikis, knowledge bases or directories. (Okay I recognize that most directories left standing are crap.) It seems to me that the good ones are nodes of expertise. I do see a lot of specialized forums in the G and B serps.
3. Related to #2. Find unique expert sites and list results from them in the sidebar for appropriate queries. Go deeper than just Wikipedia and Wikihow. Heck, Mojeek could do a publicity campaign to crowd source best expert sites on a list of topics and then weed it down to the good stuff.
4. Broaden the web savvy communities you talk to beyond search and marketing. What do they want in a general search engine?
More later.
-
>being a strong alternative to main search engines
It might be unpopular with this crowd, but I think there is opportunity for media-rich results pages. Thumbnail photos and videos next to the page titles, etc...
Basically: https://old.reddit.com/
-
Ugh! But if you go the media-rich route, build in hoverzoom.
-
BoL, check out this group. https://e.foundation/
Here's why: They are developing an ecosystem of cloud services for their fork of Android. One of they things they are trying to do is improve the Searx open source meta script to be the search engine in that ecosystem. I don't think it will scale. Privacy is their thing.
Talk to them. See what they envision for a search engine. The founder and leader: Gaël Duval comes across as being approachable. Talk with them. You got nothing to lose except a little time. By being first at the table and taking them seriously, maybe they will include Mojeek as a search on their mobile browser. Maybe you can work out a greater role. It's a small niche and they may fail but they might also succeed - network. You never know.
-
And then I would repeat that, find all the privacy minded groups and just talk to them ask how they use search engines and what they like.
-
It's hard to know the users intent which does alter the desired results.
I suppose removing or pushing non relevant stuff out would be good.
You could make using an advanced search easier, so upon a search you could offer a few key buttons:
Refine your results by saying what you are doing:
"shopping"
"research"
"news"
It's a tough one. I suppose why Google are trying to guess intent based on search.
-
Cheers all, your thoughts as always very much appreciated.
I hope to resurrect this thread now and again and pick your brains on various aspects. Measuring intent is indeed a bit hard and it remains to be seen how much of Google's tracking allows them certain tactics in improving SERPs, e.g. retargeting based on query history, intent based on previous searches.
RE: privacy groups, cheers. There's definitely some efforts on that front.
All taken on board. Feel free to add more gripes about what could be done better.
Slightly OT, DDG is now doing 1% of Google's volume and their traffic chart shows exponential growth.
-
>growth
I was going to say now is the time to push. Every new revelation has people looking.
-
All this privacy stuff is going to hurt the big players.
Now is the beginning of the right time.
-
BoL - I can tell you, that interest in Mojeek is increasing. Traffic to my Mojeek posts have gone up sharply on Bing and DDG. Every once in awhile Bing leaks a search query "mojeek search engine" was the latest. Or maybe Bing had an update? Dunno but people are searching for info and reviews of Mojeek.
-
Most people don't even use G's search ops so it may be a waste of time offering that feature.
Being an AI company i'm surprsied at G's bad natural language processing, sentences that a 6 year old could parse it fails to.
I think it would be good if you could have a seperate 'commerce search', keep it split from knowledge searches.
-
Tim Mayer once told me that less than 2% of users used any advanced search features. So, it'd be a great feature for a very few users. Is it worth it? Maybe, if it generates buzz with the tech sites.
DuckDuckGo has done a great job of differentiating in part by marketing stuff most people don't care much about.
I'm a bit surprised it hasn't cost them the Google ad syndication partnership by now given how many studies they've done flaming Google.
Maybe it's just the way I search, but, Wikipedia aside, I rarely see any results from the interior pages of wikis, knowledge bases or directories. (Okay I recognize that most directories left standing are crap.) It seems to me that the good ones are nodes of expertise. I do see a lot of specialized forums in the G and B serps.
Through directly tracking over 1 billion end users daily, Google can see what sites users are actively looking for, how well people engage on them, and which sites people repeatedly choose to visit.
A forum is highly interactive. People actively seek them out to learn, to answer questions & to participate in some sense of community. Whereas a static directory typically doesn't add enough value & isn't differentiated enough to be a sought after destination which people seek out as an alternative to a search engine. A lot of wikis are overrun by spam or so adverse to the risk of spam that they close themselves off to edits from new users & thus die slowly each time an old user quits for whatever reason.
If you edit / write / contribute there is some sense of ownership & identity wrapped in the content consumption. If you are exclusively a reader the commitment is often much less unless you are reading things vital to work or things that confirm your identity & political outlook & such.