Reviewer #1:

============================================================================
REVIEWER #1
============================================================================


---------------------------------------------------------------------------
Reviewer's Scores
---------------------------------------------------------------------------

Originality: 4
Contribution: 4
Technical Correctness: 4
Overall Score: 4


---------------------------------------------------------------------------
Comments
---------------------------------------------------------------------------

This paper conducts extensive measurements on Web Advertising Networks,
with inspiring results on network- and content-level performance of ad
networks, which is seldom investigated in literatures. I see that the
evaluation metrics are carefully chosen, the adopted measuring approach is
appropriate, and the experiments are complete.

The paper exhibits a good structure and fluent English, and is hence well
written. The statement is clear and easy to follow. Moreover, the
problem being investigated (understanding ad networks) is highly
motivated. Finally, The reasoning process are coherent and the
conclusions made are convincing.

The techniques adopted, i.e., using vantage point to investigate the
network performance, is of no fundamental difference from the previous work.
Though some adaptations have been applied to fit the scenario studied here,
the novelty is limited.

Besides, the paper is a bit verbose in the sense of repeated information.
For example, some information at the beginning of Section VI has been
introduced before.

There are some typos and grammar mistakes appeared in the context. For
example, in the second paragraph of Section V-A, "so that it do not have..."
should be "so that it does not have...". Also in the second paragraph, but
for Section VI-B, the data for the Google-Goolge case should be "31.58%"
instead of "31.38%".

But in all, I believe this is a solid work.




Reviewer #2:

============================================================================
REVIEWER #2
============================================================================


---------------------------------------------------------------------------
Reviewer's Scores
---------------------------------------------------------------------------

Originality: 4
Contribution: 4
Technical Correctness: 4
Overall Score: 4


---------------------------------------------------------------------------
Comments
---------------------------------------------------------------------------

This paper presents an interesting study on the network characteristics of Web
advertising networks. They select three representatives: Google, AOL, and
Adblade. Extensive measurements are performed to evaluate the client perceived
latency, the personalization of Web advertising, the loading time difference
between the Web content and the ads themselves, etc..

While an interesting and fairly detailed study, there is no big surprise. For
example, since Google and AOL use far more servers than Adblade, it's expected
that they achieve better performance which is confirmed by the data in the
paper. Similarly, it is common knowledge that Web advertising company target
users based on their geographic locations and use cookies to personalize the
ads they see. The paper quantifies this effect which I think can be considered
as part of the contribution.

Overall, I think this is a solid and well executed paper, even if it mostly
just confirms what people already know or can easily guess.



Reviewer #3:

============================================================================
REVIEWER #3
============================================================================


---------------------------------------------------------------------------
Reviewer's Scores
---------------------------------------------------------------------------

Originality: 4
Contribution: 4
Technical Correctness: 4
Overall Score: 4


---------------------------------------------------------------------------
Comments
---------------------------------------------------------------------------

This paper presents some interesting study on the infrastructures of ads
commissioners and their ads-targeting policies. It is among the first batch of
work that systematically studies ads delivery networks.

I enjoy the paper very much, because it provides many insights on how ads
commissioners operate in the wild. I believe the authors have done a thorough
study on the ads platforms. One minor comment: in Section VI.A, you mention you
check for "identical" ads seen by different vantage points. However, recent
studies have shown that even the same ad may have different versions due to
minor wording changes (IMC'10, Challenges in Measuring Online Advertising
Systems). It might be worthwhile to say more about how you perform the check,
and how will ads content randomness affect the results.

Also, in Section VI.C, the methodology of identifying behavioral targeting is
interesting. Maybe the study can be further extended to cover other targeting
groups separated by, for instance, genders and education levels.



Reviewer #4:


============================================================================
REVIEWER #4
============================================================================


---------------------------------------------------------------------------
Reviewer's Scores
---------------------------------------------------------------------------

Originality: 5
Contribution: 4
Technical Correctness: 4
Overall Score: 5


---------------------------------------------------------------------------
Comments
---------------------------------------------------------------------------

This paper describes a measurement driven study to understand the
performance of commissioners, the network service providers who match
web ad publishers with advertisers. This paper uses a combination
of measurement and active probing to understand the user-perceived
performance of these commissioners.

First and foremost, the paper was enjoyable to read, both because
the paper was well written and the technical content was novel and
insightful.

The problem was well-described, relevant, and novel. There are vast
numbers of large, planet-wide services which deserve the attention
of measurement, analysis, and evolution. Web advertising is one of
them.

The authors describe a reasonable methodology for data collection and
describe how they deal with the inevitable measurement weaknesses.

The analysis is clear, the results insightful with some interesting
surprises.

Most of the weaknesses are nits. For example, it would have been
useful to more clearly describe who would use which of the results.
Is the only criteria that a publisher or advertiser would use
performance? Is there not also the question of how effectively a
commissioner would match ads with a user? A bit more of an
explanation on the roles and needs of each participant would have
been helpful.

It would be useful to identify the non-technical questions that might
also be asked. While these questions are outside the scope of what
a measurement study could provide, it would be useful to understand
how much of a decision to select a particular commissioner is just
based on performance.