The Long Search for Ice Cream Sauce

I’ve recently been doing my food shopping online. No more wandering around the store and queuing. This week though I swapped online supermarkets in a bid to save some money and all I did was waste a load more time.

In my opinion, search is at the heart of everything in the world of online shopping. I search for stuff to buy. I search some more. I search again. I find things to add to my basket via search.

Sure, I could use the menus and structure the company have added to the site but I don’t usually. Or I try not to. I search.

So it was a great surprise to find that my newly chosen supermarket’s search failed my expectations.

Ice cream sauce and cigarettes

The first search I did was for “ice cream sauce”. It’s been crazy hot here recently and my kids are getting bored with plain old vanilla ice cream.

The results of the search surprised me.

There was no ice cream sauce in the top ten results but the returned list did include green tea, yoghurts, dark chocolate and cigarettes.

Yep, you read that right. A search for “ice cream sauce” returned two different brands of cigarettes, but no ice cream sauce.

Ironically ice cream sauce was listed at the bottom of the page where suggested products (based on what other people bought after searching for “ice cream sauce”) are shown. Are they taunting me?

I then did a search based on the exact title of the “Toffee Ice Cream sauce” that other people bought (I also included the brand name in a second search) and still I didn’t get the product returned in the results. Eh?

So what is the search algorithm using to find products?

It doesn’t seem to be using the title or the description otherwise it would have returned the sauce…right?

Is it using keywords? Or tags? And has someone got it wrong and labelled cigarettes as “ice” or “cream” or “sauce” in the database?

I then searched for “blueberries”. It didn’t return a direct hit.

I then searched for “eggs”. The first pack of eggs was about six rows down in the results grid. Not exactly what I would call a direct hit.

I then searched for “children’s toothbrush” and didn’t find one.

I started using the menus to find the toothbrush.

Children’s toothbrushes didn’t exist under Dental > Toothbrushes, or Dental > Children. Neither did they exist under Baby and Child section either. So they basically don’t appear to sell them, yet they do sell them in-store.

Should they have tested more?

To say that they should have tested more, or done more of X testing is a statement too far. I don’t know that they didn’t do the testing, find the bugs and then a product owner say “ship it” anyway.

What I do know though is that the search was not throwing an error. It handled all of the stuff I threw at it, but it didn’t return what I wanted.

Would this have passed a functional test? Maybe… probably. Depends on who ran it, what values they used and what their level of engagement was.

As a user I wasn’t frustrated because of crashes or exceptions or errors; things we typically talk about finding as testers. I was frustrated because I couldn’t do my shopping. The result of this was that I went elsewhere to do my shopping. I went to my usual supermarket where I found 100% of the items I searched for straight away.

If the site is well designed in the background and people are keen on improving the process then I suspect they are tracking the success rate of searches and tweaking the algorithms over time. I hope they are doing this.

What is your product’s main purpose?

Your product doesn’t have to just work it has to serve its main purpose.

It’s much easier now to switch suppliers or providers or services, so as testers we need to make sure that when we test, we’re testing that the product is fit for purpose, not just that it accepts '£%£@$@££!@@£@@FFDSIFJE21398389198237' as an input to a field.

I see testers digging around in the detail, trying to find the exceptions, the errors and the data misconfiguration and missing the big picture. It’s good to find the gnarly bugs things, but pointless if the basics don’t work.

How useful is a functionally spot-on working product that is used by no-one?

This week when I did my online food shopping, I didn’t find any exceptions, or stack overflows, or errors using the search, but I also didn’t find my groceries. Which result is more important?