Back in January 2015, the Postal Service proposed a major change in the way it measures on-time service performance for single-piece First Class Mail. Instead of contracting out to a third-party as it had done for many years, the Postal Service wanted to take advantage of the increased use of barcoding and do the measuring itself.
The Postal Service began testing the new internal measurement system during the first quarter of fiscal year 2016, and last week it shared the results for the second and third quarters with the Postal Regulatory Commission.
According to this PRC order, the Commission will host an off-the-record technical conference to discuss the data represented in the quarterly service performance reports on Friday, August 26.
At this point, it appears that the Postal Service may have some explaining to do. In many instances, the on-time performance data produced by the two measurement systems seem significantly different, and it’s not clear what the discrepancies mean.
The two measurement systems: EXFC and SPM
For years now, the Postal Service has been using what it calls the External First-Class Measurement (EXFC) system to measure the end-to-end (mail deposit to delivery) length of time it takes for mail to be delivered. The system is “external” because it’s conducted by an independent third-party contractor, i.e., IBM.
In the EXFC system, the contractor employs “droppers” who deposit letters and flats addressed to “reporters.” The participants record when mail was sent and received, and based on the transit times for those mailpieces the system generates service performance data showing the percent of mail that was delivered on time, i.e., within the service standard.
Because it’s expensive to track individual pieces of mail this way, the EXFC system uses a comparatively small sample of test mailpieces, and then extrapolates to make estimates for mail volumes as a whole. The test mail is statistically analyzed based on sample volume, mail characteristics, and the location where the mail was entered and delivered.
The new internal system is called Service Performance Measurement (SPM). It takes advantage of the fact that much of the mail is now barcoded and can be tracked on automation equipment by the Postal Service itself, which will save the expense of contracting out.
Unlike the EXFC system, the new SPM system doesn’t track particular pieces of mail, end to end. Instead, it uses a statistical composite that averages, weights, and combines live mailpieces on three different stages of the delivery process — First Mile (collection), Processing Operation, and Last Mile (delivery). The data are combined to measure overall transit time for what amounts to “virtual” mailpieces, as opposed to the actual mailpieces tracked by EXFC.
Thanks to barcoding and automation equipment, the SPM system can track a significantly larger volume of mail than EXFC. In fact, since the SPM tracks such a large portion of the mail flow, the Postal Service considers it a “census” rather than “sampling” methodology (just as the U.S. Census tries to include every household, not just a representative sample). As stated in the USPS report describing the new system, “the proposed measurement system changes will increase the number of pieces in measurement – moving from a seed/sample system to more of a census-measurement system.”
Since it’s not tracking actual pieces, the reliability of SPM is more dependent on the quality of the statistical design and the accuracy of sampling. The reliability also depends on whether the categories of mail that are measured (i.e., barcoded mail) are also indicative of the categories of mail that are not measured (i.e., non-barcoded mail). Read More