Scoring and tracking what drives store pickup experiences

Not that long ago when I was managing digital experience for a large online pickup program, I routinely read through hundreds of customer feedback comments. I tallied up survey scores weekly in an effort to gauge customer sentiment in general and to look for any signal, positive or negative. While these methods were helpful in identifying areas for improvement, I had always considered them to be inadequate feedback of the true pickup experience, not least because there always tends to be a negative feedback bias in survey responses. I wished that I had access to a more quantitative pickup experience score — one I could track for an order, for a customer, for a store, for a market and for the program as a whole.

While I did not devise a metric then, I took some time recently to think about how online order pickup experience could be measured and came up with a method that uses a few simple calculations. If you are a product manager or business owner looking to get a better handle over in-store or curbside pickup customer experience, I hope this can provide some inspiration.

There are more than a handful of factors that constitute pickup experience, but I have focused on just four that I believe customers care about the most.

Lead Time — When is the earliest I can pick up?

Fill Rate — Can you fill all the items I ordered?

On-Time — Is my order ready for pick up on time as promised?

Wait Time — When I am here to pick up, don’t make me wait

For each of the above factors, I will attempt to compute an experience score that lies between 0 and 1, the perfect score being 1.

Lead Time
Pickup programs must strive to offer a quick turnaround to be effective. The longer the duration between click (order) and collect (pickup), the lower the value a customer sees in the service.

Say lead time (LT) is the earliest turnaround time I am able to offer for an online order pickup. If I can make a 1 hour promise, LT =1. For a 2 hour lead time, LT = 2, and so on. For the purposes of computing this metric, let us consider only business hours between 10 AM and 5 PM. So, a customer promise made at 4:30 PM for a next day pickup at 10:30 AM will have a lead time of 1 hour,

Pickup experience score for lead time pex(LT) = 1/(1+ (LT — 1)/k), where k is a factor that can be scaled up or down depending on the weightage I want to assign to pickup availability. Suggested value of k is 25. A lower k value increases the weight for this score and a higher value does the opposite.

Consequently, when I can offer 1 hour pickup my pex(LT) score is a perfect 1. For a 2 hour lead time, the score is 0.96 when k = 25, and a 24 hour lead time for pickup yields a score of 0.52.

Fill Rate
Fill Rate is a relatively straightforward metric. It is the ratio of units fulfilled to units ordered. 3 units fulfilled out of 4 units ordered yields a fill rate 75% or 0.75.

pex(FR) = Units Fulfilled/Units Ordered

A high fill rate for a pickup program is generally a reflection of good inventory management practices at the store, as well as the depth in technology used to compute inventory available to promise in real time.

On-Time
If a pickup ready notification goes to the customer on or before the promised pickup time, this experience score should be perfect 1. A delay in pickup readiness should yield a score lower than 1. The longer the delay, the lower this score.

If D is number of hours by which the order is delayed,

pex(OT) = 1/ (1+ D/m), where m is a factor that can be scaled up or down based on the weightage given to timeliness. A suggested value of m is 5. For an order ready for pickup on time, D = 0 and pex(OT) = 1. An order delayed by 1 hour has a pex(OT) of 0.83. For a delay of 4 hours, pex(OT) = 0.56

Wait Time
While the three metrics above can be computed with data that are easily accessible, it is harder to measure how long customers are waiting to pick up their orders. A mobile app or kiosk based check-in feature allows one to accurately calculate wait times. Video analytics can be used to measure the length of time customers wait in a parking lot for curbside pickup. If adequate technology is unavailable, just asking customers how long their wait was and recording that will be a good starting point.

For the sake of simplicity, let us assign wait time scores in 5 minute increments. Wait time of 5 minutes or less gets a perfect score of 1. Wait time of 5–10 minutes gets a lower score and so on.

Say WT is the actual pick up wait time in minutes.

WT(adj) = ROUNDUP(WT/5)

pex(WT) = 1/(1+(WT(adj) -1)/h), where h is a factor used to adjust the weightage of this KPI. A suggested value of h is 5.

pex(WT) for wait times less than 5 minutes = 1. For a wait time of 8 minutes, pex(WT) = 0.83. A 30 minute wait time results in a score of 0.5

Composite Pickup Experience Score
Once we have all of the above KPIs measured individually, we can use them to compute a composite pickup experience score. We can use a weighted average of these scores, or as I prefer, simply multiply them to arrive at the overall score pex(Comp).

pex(Comp) = pex(LT) * pex(FR) * pex(OT) * pex(WT)

The table below shows the individual pickup experience scores and the corresponding composite scores for various scenarios.

If I am promised a 1 hour pickup, but only 4 of the 5 items ordered were filled, the order was late by 2 hours and I was made to wait 20 minutes at the curbside to pick up, will I give a 3.5 out of 10 for the overall experience? If that sounds about right, this approach may be close to how I want to track pickup experience scores. If not, the weightages in the formulae can be tweaked to get to a closer approximation for my business and relative priorities.

In Practice
Measuring the pickup experience score for each order is a good place to start. The same method may then be extended to compute lifetime pickup experience scores for customers, which in turn can inform recovery and retention initiatives. If tracked for each of the stores offering pickup, this score can also help identify factors (such as staffing, training, space, and inventory control) that impact store performance with respect to pickup.