Changes to our geographic annotations have resulted in changes to geographic filters.
On March 16 we hosted a town hall discussion regarding M-Lab’s collection of IP addresses and asked for feedback on potential alternatives. We are accepting comments on the techniques we proposed until April 1.
Presentation on the benefits and challenges of M-Lab’s collection of IP addresses and feedback solicitation for potential alternatives
In this second “analysis recommendations in context” post, we will explore the refined research questions from the first post, resulting from our discussion of how to design specific questions with understanding of available data source(s) and the context of what each contains. We emphasized the importance of selecting a data source that matches the goal of the research question. This is critical for analyses of broadband measurement data, particularly when the research goal is to compare the results to one another, to national broadband standards or specific funding requirements, or to align with advertised terms of ISP service.
A while back, our team published some analysis recommendations for anyone working with our data from the Network Diagnostic Tool (NDT), comparing it to other Internet measurement data sets, and drawing conclusions or inferences about the data. These recommendations are intended to provide guidance about analyzing crowdsourced data, because we know that it’s easy for analyses to end up with what looks like a striking comparison or finding, but that may not actually be supported by the underlying measurements or data. But because recommendations are only that, we’re now beginning a series of posts to unpack those recommendations with some context and examples. First, we’ll recap our previous recommendations post with more context, and finish with an example that we’ll continue working with in subsequent posts.
M-Lab is pleased to share our selection of Research Fellows for Spring 2022, generously supported by Internet Society. The fellows were selected for the merit, rigor, and feasibility of their proposed project as well as their research’s relevance to improvement for end-user Internet performance. Read below for more information about the fellows and their proposed projects
How can M-Lab’s data and platform be used to improve the experience of the Internet for the end user?
Measurement Lab is seeking three Research Fellows to expand Internet performance research beyond the measurement and optimization for bandwidth. Fellows will utilize M-Lab’s longitudinal, open dataset and/or platform to identify under-recognized Internet performance metrics that can be used to improve end user performance.
Many people know M-Lab and our TCP performance test, NDT, from running it in a web browser. Perhaps the largest single source of NDT tests comes from its integration by the Google Search team. While M-Lab is known for the large volume of crowdsourced test data resulting from people running our tests, over the past few years we’ve developed new ways to run our tests and open source Internet measurement tests from other platforms using a tool we’ve called Murakami.
In our recent roadmap post, we shared a list of milestones that the team is working on this and last quarter. Our Datatype migration and Standardized Columns milestone references the gardener service, which maintains and reprocesses M-Lab data, as well as the UUID annotator, that generates and saves per-connection metadata as annotations to user-conducted measurements. This post provides more detailed information about how these services have annotated measurements with geographic and network information in the past and present, and expands on what current work is happening now as mentioned in our roadmap post.
For researchers, policy makers, governing bodies, advocacy groups, or anyone who wants to understand M-Lab data and how it compares to other internet measurement data sets, M-Lab has begun to develop recommendations for appropriate use of our data in their analyses and reports.
NDT measures “bulk transport capacity”: the maximum date rate that TCP can reliably deliver data using the unreliable IP protocol over an end-to-end Internet path. TCP’s goal is to send data at exactly the optimal rate for the network, containing just the right mix of new data and retransmissions such that the receiver gets exactly one copy of everything. Since its creation, the TCP protocol has consistently made improvements to the way it accomplishes this task, consequently, NDT has also incrementally changed to reflect these improvements. The most recent improvements, including support for TCP BBR, are available in ndt7. On July 24th, we announced the start of migration of NDT clients to the latest protocol version. As of today, approximately 50% of clients are using ndt7. As the ndt7 measurements become the majority of the NDT dataset, the M-Lab team is considering what we do and do not know about whether and how changes to the NDT protocol have affected M-Lab’s longitudinal NDT dataset over time.
Following the general availability of the ndt7 protocol, we will be working with NDT client integrators to support their migration to ndt7. As they do, the NDT dataset will shift from predominantly ndt5 to predominantly ndt7. As part of assessing our readiness for this larger effort, a pilot was started on July 8.
The new ndt7 protocol for the Network Diagnostic Tool (NDT) is now generally available on the M-Lab platform. Since 2009, NDT has been the premier TCP performance measurement service test hosted by M-Lab. During its history on the platform, NDT has produced the largest test volume to date, spanning the longest history. Since late 2018, M-Lab has worked with researcher Simone Basso to develop the ndt7 protocol and archival data format.
OONI was recently invited to participate in a NetGain Partnership webinar (titled “Surging Demand and The Global Internet Infrastructure”) to discuss the changing landscape for internet infrastructure and technology in the wake of the COVID-19 pandemic.
As part of our preparation for this webinar, we looked at network performance measurements collected from northern Italy over the last months (i.e. when Italy was hit hard by the COVID-19 pandemic) in an attempt to understand whether and to what extent there was a correlation between increased internet use and reduced network performance. As our observations may be of public interest, we decided to share them through this blog post.
We deployed the new M-Lab platform to 1/3rd of the M-Lab fleet, and now we need to assess whether or not it is a performance regression, relative to the old platform. As long as we can be sure the performance of the new platform does not constitute a regression, then we can roll out the new platform and be confident that we have not made anything worse.
- Stock Linux 4.19 LTS kernels with modern TCP and Cubic congestion control
- Standard instrumentation for all experiments using tcp-info
- Virtualization and container management using Kubernetes and Docker
- Reimplementation of the NDT server
Last year, we outlined our plans to Modernize the M-Lab Platform. This year, we’re bringing them to life. Here’s a summary of why the platform update is so valuable and what you can expect throughout the year.
AFRICOMM 2018. Left to right: ￼Amreesh Phokeer (AFRINIC)￼, Josiah Chavula (University of Capetown), Georgia Bullen (M-Lab), Antoine Delvaux (perfSonar), Stephen Soltesz (M-Lab).
In late November 2018, M-Lab was invited to the Internet Measurement Workshop at AFRINIC-29 in Tunisia and to give a keynote about M-Lab and open internet measurement at AFRICOMM 2018 in Senegal. Both trips were a fantastic opportunity to deepen our relationship with researchers focused on the African Internet, learn more about how our platform is serving community needs, foster conversation around open Internet measurement, and identify opportunities for further collaboration, research and tool development to better support the Internet measurement, research and policy community in Africa.
M-Lab had the pleasure of attending the first ever SIGCOMM hackathon on August 25, 2018, at the Nokia Skypark headquarters in Budapest, Hungary. The hackathon, sponsored by Nokia, DECIX, and Netflix, invited network research faculty, students, and industry professionals from around the world to form teams and develop tools, new features or analyses during the Saturday following the SIGCOMM conference.
We’ve reached a point in human history where, for many of us, the Internet has become a standard presence in our daily lives. In the United States, the Internet is simply part of how many of us engage with the world. In other countries (and parts of this one), the Internet remains unaffordable, unreliable, and inaccessible. The Internet unites us in many ways, and at the center of work on the future of the Internet is a dedicated community of experts exploring the questions that will move the Internet to the next level of its evolution: What is an open Internet? What is a healthy Internet? What factors contribute to the Internet ecosystem’s health?
On February 1st, 2018, during a regular data quality review, we identified an increase in switch discards at sites with 10Gbps equipment connected to 1Gbps uplinks. We used our switch telemetry data to assess whether there were any negative consequences for tests contained in our SideStream or NDT data sets, and then we used the same data sets to determine whether our remediation strategy had any negative effects. In both cases, we found no observable effects, indicating that everything was below the noise floor for Internet performance data.
- When: Saturday, August 25, 2018
- Where: SIGCOMM, Budapest, Hungary
- When: Aug. 7, 2018 - Aug. 8, 2018, 9AM - 5PM
- Where: New America, 740 15th St NW #900, Washington, D.C. 20005
Measurement Lab is turning 10! On August 7 and 8, we look forward to gathering the Measurement Lab community to showcase how the platform has evolved, learn from you about how you are using M-Lab, and discuss how we plan for the next 10 years of measuring the Internet and providing public data to the world. So much has changed over the last 10 years (and that’s not just our expanding volume of longitudinal data!), come celebrate, brainstorm, analyze, and share with us.
Since June 2016, M-Lab has collected high resolution switch telemetry for each M-Lab server and site uplink.
Originally designed to detect switch discards from server traffic microbursts, we now support the DIScard COllection (a.k.a. DISCO) dataset as a standard M-Lab BigQuery table:
The Measurement Lab team has always tried to make it as easy as possible to run network measurements. Currently, most users run tests either directly from the M-Lab website, or through a 3rd party integration. Over the years, many users have requested the ability to run tests on a regular basis, e.g. daily or weekly to collect data over time. Today, we’re releasing a tool that will help you do just that.
As a platform committed to producing empirical data for the public, Measurement Lab (M-Lab) has historically supplied regulators and other governmental entities with technical facts pertinent rule-making processes. In our February 2015 submission to the FCC’s Open Internet docket, we committed to research on the state of broadband and performance impact of interconnection in the United States. Earlier this year, the FCC began the process of re-evaluating its authority over broadband Internet services, and opened a Notice of Proposed Rulemaking. This blogpost is a shortened version comments that M-Lab filed in the docket regarding its continued research on the impact of interconnection on consumer broadband. The full filing in the FCC docket includes an elaboration of our research with additional supporting evidence and charts.
M-Lab data is collected from distributed experiments hosted on servers all over the world, processed in a pipeline, and published for free in both raw and parsed (structured) formats. The back end processing component for this has served us well for many years, but it’s been showing its age recently. As M-Lab collects an increasing amount of data thanks to new partnerships, we have been concerned that it will not be as reliable.
Last October, Measurement Lab released the Internet Observatory, a data-visualization tool that enables consumers, policymakers, and researchers to better understand the impact of ISP relationships on Internet access and performance. The Observatory provides easier access to M-Lab’s rich dataset on network performance to reproduce the analysis in our report on “ISP Interconnection and its Impact on Consumer Internet Performance.”
On Thursday Nov. 6, Collin Anderson gave a talk at the RIPE meeting in London. Collin has been immersed in the M-Lab data for some time, and helped lead the recently published technical report, ISP Interconnection and its Impact on Consumer Internet Performance. He presented a lyrical overview of these research findings for the assembled crowd of network operators and researchers, which is archived on the RIPE website.
Prior to the publication of our recent report, ISP Interconnection and its Impact on Consumer Internet Performance, we shared review copies with members of the research community, broadening the set of eyes on our methods and challenging our assertions. We received valuable feedback that allowed us to improve what we published, along with a list of research questions that we will be exploring and documenting over the coming months in our Research Updates series of blog posts adding insights to Research Reports.
ISP Interconnection and its Impact on Consumer Internet Performance: Introducing A New M-Lab Consortium Technical Report
We are happy to announce the release of a long-term collaborative research effort using M-Lab’s data to understand how interconnection impacts end-user performance. The report, ISP Interconnection and its Impact on Consumer Internet Performance examines years of network measurement data from across the United States to determine the effects of network interconnection on the Internet performance of customers subscribing to specific access ISPs. Alongside this report, we are also pleased to release the Internet Observatory – a dynamic data visualization tool that will allow consumers, policymakers, and researchers to better understand the impact of ISP relationships on their own Internet access and performance. The Internet Observatory will be updated regularly, allowing future monitoring and comparison against past performance.