Search engine battle heats up
April 17, 2008 3:52 AM Subscribe
According to ComScore, Google takes 59.8% of search traffic in the US, leaving Yahoo, MSN and smaller players to fight for the scraps. Pretty pie-chart here. Slightly different numbers are available from Compete and Hitwise, but Google still rules the roost.
I remember HotBot, Altavista, Infoseek, and Northern Light fondly. Especially Hotbot - it used to be positively amazing when it was first launched. And then Google came and swept them all away.
posted by SharQ at 4:09 AM on April 17, 2008
posted by SharQ at 4:09 AM on April 17, 2008
Crazy seeing Yahoo so high. They weren't even that good back in the day.
posted by DU at 4:28 AM on April 17, 2008
posted by DU at 4:28 AM on April 17, 2008
AltaVista made themselves irrelevant. Their nested Boolean searches + NEAR operator kicked Google's butt for years until Google's page rank algorithm was sufficiently tweaked. However, sometime after Compaq bought them, they practically stopped indexing pages and Google crushed them.
posted by BrotherCaine at 4:32 AM on April 17, 2008
posted by BrotherCaine at 4:32 AM on April 17, 2008
Also, Google deserves to rule the roost not just for their search algorithms, but their clean interface and consistent design.
posted by BrotherCaine at 4:34 AM on April 17, 2008
posted by BrotherCaine at 4:34 AM on April 17, 2008
40.2% of the market that google does not (yet) have is hardly scraps.
That Google can still grow in this dimension will make Google shareholders happy.
But the brief history of this industry suggests that search engine leadership is ephmeral. AltaVista, Yahoo!, AskJeeves saw their market share quickly evaporate as new competitors entered the fray. Google, like every other would be Caesar, is mortal.
posted by three blind mice at 5:31 AM on April 17, 2008
That Google can still grow in this dimension will make Google shareholders happy.
But the brief history of this industry suggests that search engine leadership is ephmeral. AltaVista, Yahoo!, AskJeeves saw their market share quickly evaporate as new competitors entered the fray. Google, like every other would be Caesar, is mortal.
posted by three blind mice at 5:31 AM on April 17, 2008
Apple has what... 8% of the computer market? 40% of anything isn't scraps.
What on earth is the point of this FPP?
/me looks forward to tomorrow on Metafilter when someone tells me Coke sells more than Pepsi and McDonald's sells a fuck of a lot of toadburgers.
posted by dobbs at 5:44 AM on April 17, 2008
What on earth is the point of this FPP?
/me looks forward to tomorrow on Metafilter when someone tells me Coke sells more than Pepsi and McDonald's sells a fuck of a lot of toadburgers.
posted by dobbs at 5:44 AM on April 17, 2008
Ask Jeeves was pretty nice back in the day, but it has fallen into the hands of scammers and villans. Ask Jeeves was been repurposed as a spyware company, under the IAC Search and Media. These are the folks behind the MyWay/MyWeb searchbar and those tacky Zwinky doll things. Most malware diagnostics report it as such. It's not horrible malware, but it does unknown DNS redirects and phones home browser histories.
posted by bonehead at 6:13 AM on April 17, 2008
posted by bonehead at 6:13 AM on April 17, 2008
What, no dogpile?
posted by Hildegarde at 6:21 AM on April 17, 2008
posted by Hildegarde at 6:21 AM on April 17, 2008
Dogpile is a metasearch engine -- it broadcasts your queries to the search engines on the ComScore list plus a few others. It effectively makes itself irrelevant by incrementing each primary search engine's traffic equally.
posted by ardgedee at 6:53 AM on April 17, 2008
posted by ardgedee at 6:53 AM on April 17, 2008
Standard disclaimer: ComScore are not just an unreliable source of data (as with all the others who claim to provide internet metrics), but also a load of stealth-installing spyware scumbags. Do not trust.
posted by BobInce at 7:33 AM on April 17, 2008
posted by BobInce at 7:33 AM on April 17, 2008
The real story here is that Microsoft still only has 9% share. Despite years of significant investment in a first tier search engine and an enormous starting advantage in owning the defaults on Windows. And last I checked a big chunk of that 9% was garbage searches, so their real share is lower.
Yahoo's search engine also has impressive technology and very good quality. I don't quite understand why more people don't use it.
posted by Nelson at 7:53 AM on April 17, 2008
Yahoo's search engine also has impressive technology and very good quality. I don't quite understand why more people don't use it.
posted by Nelson at 7:53 AM on April 17, 2008
altavista was shit from the start, and google would have crushed them merely because their default search was AND not OR. Why, yes, altavista when I search for "mac jaguar" results about jaguar cars and jaguar cats are tippity top. Idiots
posted by bonaldi at 8:58 AM on April 17, 2008
posted by bonaldi at 8:58 AM on April 17, 2008
> altavista was shit from the start,
On the contrary, Altavista was the best available for years. OR searches made more sense when the web was considerably smaller. "mac OR jaguar" generates pretty random results, but in 1997 you could use it to see every page on the internet about the Mac and jaguars in an afternoon. OR searches also make more sense for search patterns like "osx OR macosx". Default ANDing queries makes more sense now but it's hard to commit such a radical change without alienating their existing userbase. Fortunately for Altavista, queries can have fairly sophisticated logical structures, compared to the other major search engines. It's still the best at certain kinds of searches, such as date ranges.
Yahoo also had a general web search engine from fairly early on, usually provided by third party vendors that Yahoo would cashier from time to time. And it was always considered a second-tier service compared to their indexing. Yahoo's primary focus was indexing the web, and Altavista hit its stride at about the time Yahoo became incapable of keeping its index up to date, first telling webmasters that their sites would be added to a queue several weeks long, and then charging webmasters to be in a priority queue.
Yahoo and Altavista both fell to Google when their homepages swelled to require monster load times. Query results turned into craptasms of advertising indistinguishably interleaving search results. Metasearch engines became popular among power users, partly because each search engine was missing large chunks of web content, and partly because it meant quicker page load times.
Google's search was better, but that wasn't the main reason why it won the search engine wars; it won because users felt they could trust what they saw, starting from the uncluttered single-use homepage and ending at the uncluttered clearly-marked results page. Ever since, Google's been nudging the boundaries of that trust but they haven't thoroughly broken it yet. Yahoo took a lesson from the shakeup and stripped its site back, slowly building it up to a better-designed, more moderated version of what it used to be. The homepage is still overloaded but is at least navigable now. Altavista didn't learn but it was the victim of circumstances. It was sold off, and while the Altavista query page is probably the best designed its results pages continue to be borderline useless due to advertising.
posted by ardgedee at 9:27 AM on April 17, 2008
On the contrary, Altavista was the best available for years. OR searches made more sense when the web was considerably smaller. "mac OR jaguar" generates pretty random results, but in 1997 you could use it to see every page on the internet about the Mac and jaguars in an afternoon. OR searches also make more sense for search patterns like "osx OR macosx". Default ANDing queries makes more sense now but it's hard to commit such a radical change without alienating their existing userbase. Fortunately for Altavista, queries can have fairly sophisticated logical structures, compared to the other major search engines. It's still the best at certain kinds of searches, such as date ranges.
Yahoo also had a general web search engine from fairly early on, usually provided by third party vendors that Yahoo would cashier from time to time. And it was always considered a second-tier service compared to their indexing. Yahoo's primary focus was indexing the web, and Altavista hit its stride at about the time Yahoo became incapable of keeping its index up to date, first telling webmasters that their sites would be added to a queue several weeks long, and then charging webmasters to be in a priority queue.
Yahoo and Altavista both fell to Google when their homepages swelled to require monster load times. Query results turned into craptasms of advertising indistinguishably interleaving search results. Metasearch engines became popular among power users, partly because each search engine was missing large chunks of web content, and partly because it meant quicker page load times.
Google's search was better, but that wasn't the main reason why it won the search engine wars; it won because users felt they could trust what they saw, starting from the uncluttered single-use homepage and ending at the uncluttered clearly-marked results page. Ever since, Google's been nudging the boundaries of that trust but they haven't thoroughly broken it yet. Yahoo took a lesson from the shakeup and stripped its site back, slowly building it up to a better-designed, more moderated version of what it used to be. The homepage is still overloaded but is at least navigable now. Altavista didn't learn but it was the victim of circumstances. It was sold off, and while the Altavista query page is probably the best designed its results pages continue to be borderline useless due to advertising.
posted by ardgedee at 9:27 AM on April 17, 2008
I'm quite astonished the other search engines have as much as 40%. Who is using them?
The days when we resorted to multiple search engines to find stuff ended in the late 1990's. I can't remember the last time I used any search engine other than google and cannot think of any reason I would want to.
posted by bap98189 at 9:41 AM on April 17, 2008 [1 favorite]
The days when we resorted to multiple search engines to find stuff ended in the late 1990's. I can't remember the last time I used any search engine other than google and cannot think of any reason I would want to.
posted by bap98189 at 9:41 AM on April 17, 2008 [1 favorite]
On the contrary, Altavista was the best available for years.
That it was the best of a selection of shit doesn't mean it wasn't heaving crap nonetheless.
OR searches made more sense when the web was considerably smaller. "mac OR jaguar" generates pretty random results, but in 1997 you could use it to see every page on the internet about the Mac and jaguars in an afternoon.
Nor is it any excuse that it's OK for the results to be bizarrely oblique to the two key terms you're searching for if there just aren't that many of them.
Google won because users felt they could trust it, I agree, but I think a huge part of that trust, and the apparent quality of the results, was because you didn't get pages about Macs turning up when you wanted pages about big cats -- whether that was down to advertising results, poor crawling or better logic was completely irrelevant to them.
I switched the very first time I saw the little grey text that said something like "No need for AND: Google searches for all terms by default", even though I couldn't understand the original query url format (I'd taken to typing in Altavista queries by hand to try and avoid the homepage)
posted by bonaldi at 10:05 AM on April 17, 2008
That it was the best of a selection of shit doesn't mean it wasn't heaving crap nonetheless.
OR searches made more sense when the web was considerably smaller. "mac OR jaguar" generates pretty random results, but in 1997 you could use it to see every page on the internet about the Mac and jaguars in an afternoon.
Nor is it any excuse that it's OK for the results to be bizarrely oblique to the two key terms you're searching for if there just aren't that many of them.
Google won because users felt they could trust it, I agree, but I think a huge part of that trust, and the apparent quality of the results, was because you didn't get pages about Macs turning up when you wanted pages about big cats -- whether that was down to advertising results, poor crawling or better logic was completely irrelevant to them.
I switched the very first time I saw the little grey text that said something like "No need for AND: Google searches for all terms by default", even though I couldn't understand the original query url format (I'd taken to typing in Altavista queries by hand to try and avoid the homepage)
posted by bonaldi at 10:05 AM on April 17, 2008
I used to work at Google, doing log analysis. One of the ranking companies (I don't think it was comScore, but this was back in '01, and my memory is a bit hazy) actually came in and talked to us about how they were planning on measuring traffic to the search engines.
If, for some reason, you really need to rank search engines, I guess these numbers are better than you're likely to get. But I wouldn't put too much faith in them: search engines regularly do things that mess them up, and the samples they are based on often aren't representative. The comment above about comScore being spyware scumbags is relevant to this: in the early days of Google, I think one of the reasons it wasn't "highly ranked" was that some of these companies were only measuring traffic coming from Windows users, and Google had much better market penetration in techie circles (where many people were running Linux.)
I recall one period where Google hadn't grown in the rankings as much as we had expected it to. A bit of looking into it turned up the fact that one of Google's competitors had added code to their results page to make it reload every few minutes. Since these traffic numbers were based on number of pages downloaded, that skewed the rankings. (I wonder if the bragging rights in the rankings were worth the extra load on their servers; it certainly is true that their results weren't changing from minute to minute, so there's not much reason to do this refreshing.)
Also, some of these reports are a little bit unclear on the traffic they're measuring. Does gmail count as traffic to Google? Do all of the yahoo subproperties count as yahoo? Do mistypes in Internet Explorer count as searches on MSN? (I suspect that for comScore, the answers are no, no, and yes, but it's been a while since I looked into this.)
Just to point out some of the shoddy analysis done, there was a Reuters article back in March that mentioned that Google's traffic had decreased about 5% in February. Which isn't particularly surprising to me, given that the month of February was about 6.5% shorter than January this year (meaning that Google's traffic actually grew if you think about it)... There are a couple of other reasons I would have expected traffic to be lower in February, but I think the point that many of the people doing this analysis seem to be unable to understand the data has been made.
posted by grae at 10:31 AM on April 17, 2008 [1 favorite]
If, for some reason, you really need to rank search engines, I guess these numbers are better than you're likely to get. But I wouldn't put too much faith in them: search engines regularly do things that mess them up, and the samples they are based on often aren't representative. The comment above about comScore being spyware scumbags is relevant to this: in the early days of Google, I think one of the reasons it wasn't "highly ranked" was that some of these companies were only measuring traffic coming from Windows users, and Google had much better market penetration in techie circles (where many people were running Linux.)
I recall one period where Google hadn't grown in the rankings as much as we had expected it to. A bit of looking into it turned up the fact that one of Google's competitors had added code to their results page to make it reload every few minutes. Since these traffic numbers were based on number of pages downloaded, that skewed the rankings. (I wonder if the bragging rights in the rankings were worth the extra load on their servers; it certainly is true that their results weren't changing from minute to minute, so there's not much reason to do this refreshing.)
Also, some of these reports are a little bit unclear on the traffic they're measuring. Does gmail count as traffic to Google? Do all of the yahoo subproperties count as yahoo? Do mistypes in Internet Explorer count as searches on MSN? (I suspect that for comScore, the answers are no, no, and yes, but it's been a while since I looked into this.)
Just to point out some of the shoddy analysis done, there was a Reuters article back in March that mentioned that Google's traffic had decreased about 5% in February. Which isn't particularly surprising to me, given that the month of February was about 6.5% shorter than January this year (meaning that Google's traffic actually grew if you think about it)... There are a couple of other reasons I would have expected traffic to be lower in February, but I think the point that many of the people doing this analysis seem to be unable to understand the data has been made.
posted by grae at 10:31 AM on April 17, 2008 [1 favorite]
If you asked me 15 years ago what search engine I would be using in 2008, I probably would have answered "Betty" or some other Archie character who hadn't been used yet.
Strange what changes a decade and a half can bring.
posted by Dr-Baa at 1:39 PM on April 17, 2008 [1 favorite]
Strange what changes a decade and a half can bring.
posted by Dr-Baa at 1:39 PM on April 17, 2008 [1 favorite]
I think the main reason for me posting this as a FPP was that I was very surprised by the 'other' 40%. Because I work in the on-line industry, I use yahoo / google / msn about 40/50/10% each, but Google consistently gives me better results... And I'm curious who actually decides to use anything but Google for genuine research.
posted by SharQ at 8:43 AM on April 18, 2008
posted by SharQ at 8:43 AM on April 18, 2008
at least some of the other 40% can be explained by the fact that people don't change their homepages. For a long while, yahoo was very aggressively partnering with ISPs (and may still be--I'm not paying close attention any more.) Also, Internet Explorer generally defaults to MSN, and at times has connected to MSN on DNS lookup failures (if I was actually at a windows box right now I'd confirm it before posting, but I'm not.) I think this is a fine behavior for browsers to have: it's remarkable how many people don't distinguish between their browser's address bar and search boxes. (I couldn't ever come up with an alternate explanation of why so many people search for 'google' on google. Or 'yahoo' and 'msn' and 'cnn'.)
posted by grae at 9:14 AM on April 18, 2008
posted by grae at 9:14 AM on April 18, 2008
And I'm curious who actually decides to use anything but Google for genuine research.
You assume everyone searches from the address bar or search toolbar which isn't the case. A large portion of Internet users set portals as their homepage. In terms of portals, Yahoo has everyone beat by a larger margin. So for all the "casual" users out there who don't discriminate on where their search results are coming from, they're going to default to the search engine that's baked right into their homepage portal. That's a huge chunk of Yahoo's 21.3% right there. And I wouldn't be surprised if a large majority of Microsoft/MSN's 9.4% search engine share comes searches through the MSN portal and searches from Internet Explorer's toolbar and sidebar.
posted by junesix at 4:26 PM on April 18, 2008
You assume everyone searches from the address bar or search toolbar which isn't the case. A large portion of Internet users set portals as their homepage. In terms of portals, Yahoo has everyone beat by a larger margin. So for all the "casual" users out there who don't discriminate on where their search results are coming from, they're going to default to the search engine that's baked right into their homepage portal. That's a huge chunk of Yahoo's 21.3% right there. And I wouldn't be surprised if a large majority of Microsoft/MSN's 9.4% search engine share comes searches through the MSN portal and searches from Internet Explorer's toolbar and sidebar.
posted by junesix at 4:26 PM on April 18, 2008
Bonaldi, if you didn't use Boolean searches, I grant you that AltaVista sucked. If you did, Google took a few years to catch up in terms of search results, and even after they caught up there were occaisionally times where I missed a feature that AltaVista had.
posted by BrotherCaine at 4:35 PM on April 18, 2008
posted by BrotherCaine at 4:35 PM on April 18, 2008
> And I wouldn't be surprised if a large majority of Microsoft/MSN's 9.4% search engine share comes searches through the MSN portal and searches from Internet Explorer's toolbar and sidebar.
For that matter, inserting a space or typoing without an http:// in the address bar will redirect you to MSN's search results page (in IE) or Google's search results page (in Firefox and Safari). A few points of success for each party come through indirection. More points for MSN Search simply because IE has the dominant user share, and probably mitigated by those who see the MSN Search results page and try something else in the address bar or hit their Google bookmark.
> And I'm curious who actually decides to use anything but Google for genuine research.
Depends on what you mean by 'genuine research'. Google's query structure is limited in scope compared to AltaVista's, and it's a shame that AV's shitted up its results page nigh unto uselessness. Google's good for what it is, but it is frustratingly dumbed down compared to what's available in proprietary information resources. However, those resources are really only useful after you have training in them. You can make a modest living simply by being good at LexisNexis searching.
posted by ardgedee at 8:41 AM on April 20, 2008
For that matter, inserting a space or typoing without an http:// in the address bar will redirect you to MSN's search results page (in IE) or Google's search results page (in Firefox and Safari). A few points of success for each party come through indirection. More points for MSN Search simply because IE has the dominant user share, and probably mitigated by those who see the MSN Search results page and try something else in the address bar or hit their Google bookmark.
> And I'm curious who actually decides to use anything but Google for genuine research.
Depends on what you mean by 'genuine research'. Google's query structure is limited in scope compared to AltaVista's, and it's a shame that AV's shitted up its results page nigh unto uselessness. Google's good for what it is, but it is frustratingly dumbed down compared to what's available in proprietary information resources. However, those resources are really only useful after you have training in them. You can make a modest living simply by being good at LexisNexis searching.
posted by ardgedee at 8:41 AM on April 20, 2008
« Older All Our Favorite Toxins | A Virtually Virtual Terminal Newer »
This thread has been archived and is closed to new comments
posted by The Monkey at 4:04 AM on April 17, 2008