Hi Matt,

Hitwise uses HTTP packet sniffing that extracts the usage data
directly from an ISPs network. In other words, it only has access to
the data from its partner ISPs. This could be a very large sample of
web users (7-8 million users in the US), however, isn't necessarily
representative of the US online population.

For example, if an ISP is geo-biased or audience segment specific
(e.g. most its customers are home users with little or no corporate
clients) then the data would be skewed towards those user profiles.
Another bias I noticed is towards websites affiliated with the main
ISP/s in the Hitwise network.

For example, assume AOL is one of Hitwise's biggest ISPs. Given that
many of AOL customers would have the AOL homepage as their default
browser page, they are more likely to click through to the AOL fantasy
football partner FanHouse.

There is a good likelihood that Hitwise would over estimate FanHouse
traffic vs other fantasy football sites.

Hitwise also partners a network of browser plug-in providers. Here too
it is reasonable to assume there would be a biased towards more savvy
technically minded (did I say geeks?) web users. Same problem as
Alexa.

On their own admission Hitwise has a very small At Work panel, less
than 1%. This, by the way, is a problem for the panel measurement
companies as well (no matter what they say). The big corporates are
very sensitive about being measured.

The panel measurement companies have much greater control over their
panel audience because they actually have significant information
about these people (e.g. demographic data). Both comScore and Nielsen
recruit their core panel prospects offline via telephone interviews
(Random Digit Dial method or RDD). They supplement the RDD panel with
cheaper online recruitment which is not as good "quality" panel but
helps bolster panel sizes.

They will then use an enumeration survey to chart the US online
population and will periodically weigh their panels to match the
current US online population map.

These two constantly argue about the validity of each other's
recruitment methods but in general should have a smaller bias in terms
of representing the US online population (vs Hitwise).

As I mentioned above as part of their recruitment process, comScore
and Nielsen collect demographic data about their panel members. They
then monitor these members' online behaviour on a 1-to-1 basis.

This is similar to a registered user login into your site. With the
right web analytics tool you should be able to track them individually
and produce a stat such as "50% of traffic to section X is from users
below that age of 30" (if you initially collected this customer info).

Hitwise doesn't have this depth of data about their panel members (the
ISPs are not allowed by law to provide this data to Hitwise without
opt-in from their customers). It only has their zip/post codes, age
and gender.

Hitwise matches credit rating demographic data (from credit scoring
companies like Experien – which recently acquired Hitwise) per zip
code with the recorded web data for that zip code.
In some cases Hitwise also has an opt-in panel but normally very small
compared to their sample (3%).

The credit rating data might be comprehensive but isn't always
indicative. You could have different demographic profiles living under
the same zip code, especially in densely populated areas.

Can you accurately tell whether a family comprises of two adults with
kids with a love for outdoor sports based on a zip code and credit
rating data?

I have used the Hitwise Search Term Gap Analysis to mix success. But
it is more to do with the site I was using it for than being a true
reflection on the quality of the tool (it was a brand new site).

Don't get me wrong, this is not to say that the Hitwise data is
useless. on the contrary, it has many advantages: daily data available
next data, large search data (for long tail analysis) and wide reach
(for smaller sites). As with any other data source it has its
limitations and short comings.

For search data I'm more inclined to "live" with these data issues.
I'm assuming that demographics and geo locations would have some
impact on search patterns. However, I'm not familiar with any
conclusive research in this area. Would love to hear from anyone that
does have such research available.

Hope this helps.

Michael Feiner
AEP Convert
www.aepconvert.com


--- In webanalytics@yahoogroups.com, "mattjames8080"
<mattjames8080@...> wrote:
>
> Hi Michael,
>
> Can you elaborate on Hitwise's sample as bias? I thought panel
based
> measurement would be more biased. and how does hitwise combine
their
> data with demographics?
> Have you also used the Search Term Gap Analysis on Hitwise and found
> it useful?
>
> Thanks again,
> Matt
>
>
> --- In webanalytics@yahoogroups.com, "mfeiner17" <mfeiner17@> wrote:
> >
> > Hi Matt,
> >
> >
> > I've worked with both comScore and Hitwise. I haven't, however,
used
> > the comScore search product.
> >
> > If the main focus is search I would favour Hitwise. Their sample
size
> > is significantly larger than comScore which is critical for search
> > (think long tail).
> >
> > Factors to consider: Hitwise's sample is biased reflecting the
profile
> > of the ISP customers from which the data is sniffed. Given that
> > Hitwise does not disclose their ISP list it is hard to assess
whether
> > these customers are truly reflective of the wider national online
> > audience.
> >
> > I'm very suspicious of Hitwise's demographic data. I think the
method
> > they use to merge their (valuable) web measurement data with
> > demographic data is flawed. I would ignore any Hitwise demographic
> > data.
> >
> > comSocre on the other hand would likely face sampling and accuracy
> > issues for less frequent search terms. The data could fluctuate
> > significantly month on month.
> >
> >
> > Disclaimer: I used to work for Nielsen Online. I categorise the
big
> > three's greatest strengths as:
> > Nielsen for demographic information, comScore for their global
reach
> > (even taking into account the measurement accuracy issues they
face)
> > and Hitwise for search.
> >
> > Let me know if you have any other questions.
> >
> > Michael Feiner
> > AEP Convert
> > www.aepconvert.com
> >
> >
> >
> >
> > --- In webanalytics@yahoogroups.com, "mattjames8080"
> > <mattjames8080@> wrote:
> > >
> > > Hi,
> > >
> > > Anybody have experience using Comscore Marketer and Hitwise?
Which
> > > one would you recommend? I'm mainly into search. They both
have
> > > similar features (search terms to a website - paid/organic
breakdown,
> > > websites receiving traffic from a particular term), but Comscore
> > > Marketer seems to offer very little compared to Hitwise. I'm
not
> > even
> > > sure what I can do with ComScore Marketer. Any thoughts or
> > > experiences?
> > >
> > > Thanks in advance,
> > > Matt
> > >
> >
>