======= Review 1 =======
*** Contributions: What are the major issues addressed in the paper? Do you consider them important? Comment on the novelty, creativity, impact, and technical depth in the paper.
The paper addresses an important and hot topic: unified communication solutions. This is well-written and organized.
The proposed solution is important for both academic and industry groups, including it will be used in the next release of the Microsoft Office Communicator solution.
The proposed scheme, named SureCall, was implemented on over 80 machines across a global enterprise and many residential networks. The experiments were performed for more than 6 months and show interesting results. The authors analyze packet traces to quantitatively characterize the impact of various network scenarios on the performance of real-time audio/video conferencing.
The paper contribution and motivation are clear. A related work analysis is presented to highlight the contribution of the proposed solutions. SureCall is a unique data collection of large-scale end-to-end packet level traces.
*** Familiarity: Rate your familiarity with the topic of the paper.
Some knowledge (I am marginally aware of research work in this topic) (2)
*** Strengths: What are the major reasons to accept the paper?
The paper presents several requirements to be accepted.
The paper motivation and contribution are clear. The paper is well-written and organized. A deep performance evaluation in a large-scale scenario was performed and the results show how SureCall can serve as an ideal platform to design, experiment and validate new schemes and algorithms.
The implementation of SureCall in real softwares show a good contribution also from the industry point-of-view.
*** Weaknesses: What are the major reasons NOT to accept the paper?
The references should be updated.
SureCall was implemented in 2008 and the authors should present recents advanced in the proposed solution to show its novelty.
The quality of the figures should be improved.
Audio quality of experience testes could be evaluated by using E-Model and other assessment metrics. In addition, QoS-based metrics such as packet loss rate, packet delay rate and packet jitter rate, are typically used to indicate the impact on the video/audio quality level from the network's point of view, but do not reflect the user's perception. New QoE-based assessment metrics are now available in literature and the results could be extended with QoE measurements.
The author said that in future work, video quality assessment will be performed, but it could be done in the current version of the paper to improve it.
*** Recommendation: Your overall rating
Likely accept (top 20% but not top 10%, significant contribution) (4)
*** Detailed Comments: Please provide detailed comments that will help the TPC assess the paper and help provide feedback to the authors.
The paper discusses an important and hot topic, as well as, it is well-written and organized.
The proposed solution is important for both academic and industry groups, including it will be used in the next release of the Microsoft Office Communicator solution.
Performance eveluation presented the benefits of the proposed solution and a good contribution for future researches.
The proposed solution was implemented on over 80 machines across a global enterprise and many residential networks. Traces were analysed for audio and video conferences and interesting results are presented.
New QoE-based assessment metrics are now available in literature and should be explored in this papers. It could really improved the paper quality.
======= Review 2 =======
*** Contributions: What are the major issues addressed in the paper? Do you consider them important? Comment on the novelty, creativity, impact, and technical depth in the paper.
This paper presents a new platform for Audio/video network testing. It's testbed is very extense and includes all types of Internet connections all over the world.
It also presents a new for of detecting an inminent loss of voice packets based on jitter measures.
*** Familiarity: Rate your familiarity with the topic of the paper.
Familiar (I am well aware of research work in this topic) (3)
*** Strengths: What are the major reasons to accept the paper?
They have developed a good testbed scenario for VoIP, Videoconferencing tests.
Due the huge test clients they have used, results showed are very interesting in terms of client diferenciation (home / enterprise, and wired / wireless).
Also they present interesting results about jitter-loss correlation.
*** Weaknesses: What are the major reasons NOT to accept the paper?
They mention that have used clients all over the world but at the end they only use US-US clients to calculate jitter-loss correlation.
They present a new method of detecting inminent packet loss but they do not compare obtained results whith other clasifiers, or if they do not exist they doesn't mention anything.
Also, in Section VI they do not mention anything about the voice loss detector they have deployed in the previous page.
*** Recommendation: Your overall rating
Likely accept (top 20% but not top 10%, significant contribution) (4)
*** Detailed Comments: Please provide detailed comments that will help the TPC assess the paper and help provide feedback to the authors.
It's a good paper, they present a new VoIP testbed platform and some results obtained from it. They also present in last page an audio quality classifier.
They could have had more results obtained from the testbed,(eg. International VoIP calls) Instead of presenting the classifier in so little space.
======= Review 3 =======
*** Contributions: What are the major issues addressed in the paper? Do you consider them important? Comment on the novelty, creativity, impact, and technical depth in the paper.
The paper presents a study on the impact of different network conditions on audio and video calls over IP. In particular, the authors set up an experimental testbed composed of 80 machines placed all over the world in residential and enterprise locations; such machines periodically performed active testing by simulating audio-video calls.
*** Familiarity: Rate your familiarity with the topic of the paper.
Familiar (I am well aware of research work in this topic) (3)
*** Strengths: What are the major reasons to accept the paper?
The paper presents a large scale active analysis of audio and video call performances on a real world scenario. The analysis is ample both in time, number of collected traces. and number of interested host.
*** Weaknesses: What are the major reasons NOT to accept the paper?
- Authors should say how jitter and packet loss values presented in Section IV are calculated from collected data presented in section II.E. For example, it is not clear to me how jitter is defined, because the term jitter is imprecise. Is jitter the standard deviation of packet delay or it is the instantaneous packet delay variation? Please refer to RFC 3393 for a more detailed discussion.
- The jitter measurement in absence of synchronized clocks is done by applying a clock skew estimation algorithm, but no tests are made to have an idea of the actual accuracy of such an algorithm (for example, by performing some tests by means of synchronized reference clock on the two end of the connection). The authors only compared two different algorithms. The author should have done some testing by means of GPS or NTP synchronized hosts.
*** Recommendation: Your overall rating
Accept if room (top 30% but not top 20%, borderline) (3)
*** Detailed Comments: Please provide detailed comments that will help the TPC assess the paper and help provide feedback to the authors.
- The acronym I/SP/P is presented three times before being defined. The author should define it the first time it appears, or give a pointer to the definition. Furthermore, such frames are typical of a given protocol (what protocol?) or are generated only for testing purpose?
- The sum of second row of Table II is 1196, should be 1212 as reported at page 2, second column.
- Figure 12 is before Figure 10 and Figure 11