Episode 19: Market Research & Usability Testing

Stories of Market Research: The Insightrix Podcast
Episode 19: Market Research & Usability Testing

For this nineteenth episode of Stories of Market Research, we’re taking a dive into the subject of usability testing.

Sometimes referred to as UX testing, usability testing is an essential part of the digital development process. By testing websites or apps, organizations can be assured their product will behave as it was intended with its target user group, but further than that, it assures that the product will meet the user experience needs of that group. UX testing can produce insights that can guarantee better end usability and a better experience for end users.

For this episode, we’re joined by Petrina Holoszyc, the Director, Research Partner at Validately.

Validately is a platform many researchers will already be well aware of for its use in UX research and usability testing. In our conversation, Petrina tells us all about how the Validately platform produces insights for clients and how researchers use it to capture the essence of a user’s UX journey. Then, later on, Petrina offers some tips and tricks for folks who may be new to the Validately platform. 

Later in the episode, our CRO, Shonna Caldwell, joins the podcast to fill us in on how UX research is done at Insightrix and to talk about the benefits Insightrix clients see from engaging in this type of testing. Then, Shonna shares some information about an exciting recent UX project in which we performed usability testing for a client in the process of developing a crop science website (read the case study). We also discuss some of the challenges researchers can encounter when doing UX research, as well as some of the ways these challenges can be overcome.

If you’re interested in learning more about Insightrix UX Testing, check out this case study where we describe how we helped a Canadian provincial government agency develop a professional services website to serve the labour market needs of a regional municipality that met the usability needs of its target user groups.

Insightrix Releases New Content Regularly

Check out the articles on the Insightrix blog for informative articles about everything and anything market research related.

You can also access Stories of Market Research on your favourite podcast player app, including SpotifyApple Podcast and YouTube. Make sure you subscribe to be notified the moment a new episode is published!

Schedule update, Insightrix Research Facebook button, InsightrixMarket Research and Data Visualization, Insightrix Research Twitter, InsightrixMarket Research & Data Visualization, Insightrix Research, Insightrix PodcastCOVID-19, Insightrix Research, Insightrix Podcast


Episode Transcript


[DUNCAN] Let’s talk about a hypothetical situation. Let’s say you’re a developer with the next big app that you’re sure is going to shake up the world. You’ve developed your app along lines you think will be useful for your users, with a design you think will be easy for them to learn and then use after they learn it. You’ve tested the code and you know that it’s bulletproof. Now what? Well now you could simply say “Done” and release the app out into the wilds of the app stores, then just hope for the best. Or, you could test it with real users before it’s released to make sure the app will work the way you want it to in the real world, with real users using it. In testing, you may find out that the experiences had by actual end users may differ from what the app experienced when it was going through testing in your organization by its developers.

Further, you could test your app with real end users to measure their user journey as they get to know your app, and learn about what their experience is actually like. Do they like it? What are they engaging with most? What do they find most useful? And what pain points could be removed from the user journey? With answers to these questions, developers can address what they learn from their test and ultimately release an app into the world that they know they can be confident in because it’s been tested.

Hello, I’m Duncan McGregor, the Marketing and Communications Coordinator at Insightrix Research in Saskatoon, Canada, and your podcast host. In this nineteenth episode of Stories of Market Research, we’re talking about usability testing. Sometimes called UX testing or user experience testing, usability testing happens when businesses want to know how their website or apps end user experience or user journey will perform with people who would be similar to, or who actually make up the actual user base of that website or app.

For this episode, we speak to Petrina Holoszyc. Petrina’s the director, research partner at Validately. Validately is a tool many researchers will be pretty well aware of already, for its use in UX research and usability testing. In our chat, Petrina tells us about the Validately platform itself and how researchers use it to capture the real essence of a user’s UX journey. And Petrina even provides new Validately users some handy tips and tricks for using the platform.

Then, later in the episode, we talk to Shonna Caldwell, the CRO at Insightrix. Shonna tells us about some of the more common usability testing scenarios we see in our own research, and we discuss all of the benefits our clients see from conducting UX testing. We also discuss some of the challenges researchers can face when doing this kind of research, as well as some of the ways these challenges could be overcome. Measuring UX journeys, validating assumptions, and saving time and resources in the long run, all that more on this episode of Stories of Market Research, the Insightrix podcast.


[DUNCAN] First, let’s talk to Petrina Holoszyc from Validately about how Validately is used by researchers in the UX testing process.


[DUNCAN] Hi Petrina, thanks for appearing on our podcast. Really glad to have you on to talk about how Validately helps researchers perform usability testing.

[PETRINA] Thanks so much for having me.

[DUNCAN] Oh, no, it’s great to have you on. Perhaps, could I ask you to tell our audience a little bit more about yourself and your role at Validately?

[PETRINA] Sure. So, my name is Petrina Holoszyc, and I run our customer success and support teams at Validately, which pretty much means that it’s my job to make sure customers are getting the most out of our product as well as understanding what their feedback is and working with our product teams and sales teams and marketing teams to make sure that information is passed around so that we can build a product that our customers really want.

[DUNCAN] Awesome! That’s really cool. And for folks who don’t know about Validately, the company itself, could you tell us a little bit about it?

[PETRINA] Sure. So, Validately was started in 2013. It’s actually a really good story in terms of the same reason a lot of people have come to use us. So our founder originally had their one company and it went great. And so with all that confidence went and started another company, which did not go so great. And when that didn’t go so great, he started talking to people, trying to figure out what people do to make good products, what people do to make good digital content. And the more he talked to people, the more he learned it was understanding user experiences. And with that, he started Validately, which was to be able to understand user experiences for digital content, be it an application, be it software, be it a website, anything like that. So, yeah, we’ve been around since 2013 at this point, and I’ve actually been part of a company for about five years.

[DUNCAN] Cool. Now, the platform itself. Could you tell us about it and what it is?

[PETRINA] Sure. So Validately is a platform to be able to do UX research as well as, well any type of really understanding user experiences. So you can do one-on-one interviews, you can do unmoderated testing, which is just going through the prototypes, if that’s what you’re testing. It’s really understanding why, so with a customer, why do they want to use your product? What are they looking for in a product? Does this make sense? Anything for that type of stuff. What’s really great is it’s purpose built specifically for that kind of moderated one-on-one remote research, which is actually incredibly important these days because people can’t travel. So you’re able to actually understand what people are trying to get out of that product or that website or whatever kind of thing. It’s also equally important when, for example, you’re located in one area, but you have a national or international reach of customers. And instead of bringing people in to be able to talk to them just there, which only people around where that headquarters is, you can talk to people anywhere in the world.

[DUNCAN] That sounds super useful. Now, million dollar question, how does it help market researchers perform usability testing?

[PETRINA] Yeah. So again, it’s understanding pretty much what people are coming for. You can do message testing, for example, you can understand calls to actions. They often say, and I mentioned this before, the understanding of that why. You’re looking to build a new website or build a new experience in terms of shopping or even build a new marketing website in terms of explaining what your product is. Does it make sense? Do people understand what they’re looking at? Are they finding the information where they need to find it? Is it even what they expected to be on that website?

And that’s where prototyping comes in, incredibly important so that you can test it out before you spend a lot of time building something, using a lot of dev resources and using a lot of even designer resources. So, it’s a great way specifically to get up and running quick and easy. That’s also kind of what’s important with Validately is it’s an easy tool to use and an easy tool to get that information you need fast.

[DUNCAN] How does it get you that information?

[PETRINA] So, you’re able to do the interview on our platform. So, if you sign up, you create a study, you go and talk to somebody, you’ll be able to do the interview. You can take notes, they’re all timestamped notes. You’ll be able to make clips afterwards. You can do all that analysis inside. You can share that analysis. So, it’s an easy place to get all that information done, do that analysis and get it out to your stakeholders.

[DUNCAN] That’s really cool. I can see how that’d be really handy when you’re communicating research findings, that kind of thing, for sure. How does that translate to better research outcomes to clients though? From a research point of view, I can see how it’d be super useful. But if you’re translating that to better research outcomes, how does that usually work?

[PETRINA] So, your stakeholders are always going to be, they can be internal or external stakeholders, right?

[DUNCAN] OK, yeah.

[PETRINA] We had a customer, for example, that was looking at understanding properly, how one of their patrons were being used in one of their apps. And the product team all had an assumption and other people had assumptions. And they’re like, “Maybe before we redesign this entire new app, let’s make sure it’s being used the way we think it’s being used.” And with that, they did a whole bunch of quick research, tasked about six or seven of their own users. And it turns out it was being used completely differently. And with that, they actually found a feature and a need that they didn’t even know was there because people had been not hacking, but using it in a way in which they didn’t assume it was going to be used. And so when building these experiences, it’s so incredibly important, not to just understand what people might use, but people who aren’t using your current software or website to see if the need is there and what you’re looking for.

[DUNCAN] Yeah, I could see how that would be super useful because really it just takes the assumptions right out of it, right? And make sure that what you’re working on is actually what people need.

[PETRINA] Exactly, and it’s so incredibly important because they’re spending so much time. And also, the expectations now are very different. It’s not like the early days of you can be your own testing department, right? The expectations of what people need, people want to be able to get what they want pretty easily and maybe feel that they can do that is what we’re here to help.

[DUNCAN] Yeah, that’s awesome. Now, do you have any examples you could share or maybe a case study that you can share with our audience about a time somebody used Validately to help one of their clients get better and more actionable research outcomes?

[PETRINA] Yeah, so one of the ones I just mentioned is actually a good one in terms of that.


[PETRINA] Making sure that you’re building stuff that the product team needs. I can also say that we’ve used it even internally. We do research as much with our own users, as I think by the end, everybody hopefully should do, right? So when we build new features, we also decide and try and talk to people to see what the need is. We go through the designs, we go through wireframes, we go back and understand what people are expecting to be able to do. And then we’ll talk to them later as well. So that’s kind of the research is being done by us. We preach what we sell in that sense, and being done by a lot of our other customers who just want to make sure that that quick understanding of what’s going on is possible. And what’s great with it is that you can have somebody in a meeting be able to say, “You know what? I’m really looking for to understanding this,” and you can get that done pretty quickly. Instead of it taking six months, it can take literally two weeks.

[DUNCAN] Wow! I know we’ve actually just used it ourselves, the Validately platform, on a case that we were just working on, where we were designing some portals for some CX research for one of our financial services clients. And frankly, our team really, really liked it. That’s one of the reasons we wanted to get you on. So, we’re going to be talking about that a little later in the episode, but since I’ve got you here, I think it’d be a bit of a wasted opportunity to not ask if you could share any pro user tips with our listeners that someone new to Validately could use to get the most out of the software.

[PETRINA] Totally, and this is an oldie, but a goodie, but preview your study before you send anything out. It’s like one of the things when you’re back in college and you’re writing your own term paper, and then you’re reading it over and you know what it’s supposed to say, and you understand it. And then you send it out and you start talking to people and you’re like, “Oh, they don’t understand it.” We have preview options in our tool for a reason. Send it out, make sure, send it to your spouse, send to co-worker, send it to whoever. If they understand what you’re trying to get out of it, then so will the tester you’re trying to actually get feedback from.

[DUNCAN] That’s a really good pro tip! Honestly, I’d say that goes for everything, but especially research. But as a copy editor, as one of the hats I wear, I always tell people, make sure that you get everything you’re working on looked at by somebody else, because often you won’t be able to see the forest for the trees and you’ll think that folks will understand you whe,n frankly, that you’re not being very cogent.

[PETRINA] Exactly. As the data says, garbage in, garbage out, right?

[DUNCAN] Fully, fully. You’d mentioned, do you have a free trial of Validately going, can you tell us a little bit about it?

[PETRINA] Yeah, totally. So, we want to make sure that everybody test that out our platform before they decide to do anything. And so, if you sign up on Validately.com, you can get a free trial for two weeks, where you can test out our unmoderated and moderated features and play around with the platform. It’s a great way to get your feet wet and see what we have to offer. And we’re always here to help if you have any questions.

[DUNCAN] That’s awesome. I’ll make sure to add some information about that and maybe a link on the Insightrix website so folks will find it a little easier.

[PETRINA] Perfect.

[DUNCAN] Awesome. Well, this has been super interesting talking about Validately with you and a lot of fun actually. Thanks for coming on the podcast.

[PETRINA] No worries. Loved talking to you too.




[DUNCAN] Next we speak with Shonna Caldwell, our CRO about usability testing at Insightrix.

Hi Shonna, Thanks for coming back on the podcast. Really looking forward to talking about UX testing with you.

[SHONNA] Yeah, for sure. Thanks for having me.

[DUNCAN] It’s going to be great. So, when it comes to doing UX testing, do we do much usability testing here at looking? What kind of projects do we usually work on, websites, apps, that kind of thing?

[SHONNA] Yeah, so our primary focus is on website usability testing. We can also test things like apps or prototypes, website. The way that it would kind of work is, yeah, we would talk to customers of that particular say website or service that we were looking to test and have them walk through the website and observe their steps in terms of getting certain places, their experience with the website or the app. And yeah, it’s very user-friendly.

[DUNCAN] That’s cool. Now how does UX testing differ from other kinds of prototype testing? Because I imagine they’re pretty similar. How does that differ in UX testing to that kind of just prototype testing?

[SHONNA] Yeah, so the way that UX testing differs is it’s more of a qualitative approach. So, if you think of it similar to an online focus group, or even just an in-person focus group, a moderator’s guide is developed where we, as a researcher, would walk the customer or the user, in this case, through a series of questions and tasks and ask them to walk through step by step, how they would get somewhere. So, for instance, if they were searching something on the search bar and we wanted to get them to a certain page, we would ask these series of questions that would not necessarily lead them, but want to observe how each user was doing it, and if there was any different ways that certain users were accessing certain information.

So yeah, it’s very much in terms of a discussion, and observation. We’re able to see exactly what the user is doing through a shared screen capability. So, you can communicate through video, through chat, through audio, as well as if you wanted to take the share screen capability off and control that screen yourself as a moderator, like what you would do in a focus group. You can do that and walk the customer through how you would do it and ask for their feedback. So, there’s endless possibilities in terms of how we would get them to test the user experience.

[DUNCAN] Yeah, that makes sense because every product, every user experience is going to have a different journey. So, you’re going to be looking at that journey differently every time. Correct?

[SHONNA] For sure.

[DUNCAN] Right on. Now, I know we recently wrapped up a usability testing project for an ad agency who was developing a new crop science website for their client. Can you tell us a little bit about that project?

[SHONNA] Sure. So yeah, so this, as I mentioned, similar to what a qualitative, maybe online focus group would be, we recruited, I think, nine farmers or those that would be in the crop science area to participate in a user, so in an hour or so, 45 minutes to an hour usability test. And so, each one of those were done as an in-depth interview. So, they were done individually and we recruit them to the website, walk them through that moderator guide, share the site that we wanted them to test as well, so then in our end, having the researchers sitting in the back end, being able to observe exactly where they’re going. So, you can watch their mouse move on the screen as we walked them through specific tasks. This one in particular we did in the Prairie provinces. So, we had participants from Saskatchewan, Alberta and Manitoba participate in the research. Variety of ages, anywhere from 35 to 55 plus, that kind of thing. And we also tested both it as a mobile app, as well as a desktop.

[DUNCAN] That’s really cool. What did you find out in the case itself? During the project, did you learn anything interesting, that you can share of course?

[SHONNA] Yeah, I guess more on a broad level, that as you mentioned earlier, each user has their own experience with how they might get to certain information. So, for instances, we had probably a portion of the sample, when we asked them to search for something, would go right into that search bar, similar to what you would find on Google and just type in the word. Whereas on this site as well, there was a map tool. So, some of the respondents instantly went to more visual versus text-based, which I thought was very interesting in terms of how their brain works to search for certain information.

Yeah, some other things that we found out was obviously, as I mentioned, we did anywhere from 30, 35 plus, so a variety of ages and their technology comfortability level. That’s also had an impact in terms of how they would look for information or how they would do certain tasks, which I think is just something you always have to take in consideration anytime you’re doing research and especially if it’s technology-based. So, from a researcher’s point of view, there’s just a little bit more, I guess handholding, to make sure that each respondent feels comfortable with the technology in terms of what it’s doing and what we’re asking them to do. And so, there’s a little bit of that in terms of you really get to know the participants. So, working with them even before the test to make sure that everything’s working properly.

Yeah, I guess those are some of the really interesting things about this type of work. You also get to have really in-depth conversations. And so, as you mentioned, you can walk them through a task, you’re watching their screen and where they’re going to next, and then you can immediately ask them, “Why did you go there?” And then they’ll just read it. You get a variety of different answers in terms of like, “Oh, well it just made sense to me” or, “Oh, I didn’t even think of something else.” So yeah, you really get to explore and see their facial expressions as you can have a video camera going on as well, which is funny, you can see if someone’s getting a little bit confused by the website, but they might not be technically wanting to communicate that. But by their facial expressions, you can see, OK, well, I’m not really sure what I’m supposed to do next. And that’s kind of a finding in itself.

[DUNCAN] Was this project typical of the UX testing projects that we usually do at Insightrix?

[SHONNA] Yeah, I would say, like I mentioned, the majority of our UX experience is with websites. So, this is what this project was as well. And we do a variety of either testing a website on mobile or desktop. So, this is primarily what our clients have come to us for. However, like I said, we can do app testing or prototype testing as well. So, I think as more and more clients are developing their own apps, this would be a very good opportunity to look at that experience. So hopefully, that’s an area that we can start driving a little bit more forward.

[DUNCAN] That makes sense. Now I know we were just talking about how each project is different when we’re doing our UX testing at Insightrix, but do we have any kind of standardized process that we sometimes follow?

[SHONNA] Yeah, I guess more so in terms of the setup of the project.


[SHONNA] So, like I said, it’s similar to qualitative research where you would want to talk to a specific target audience. And so how we would recruit those people, we would obviously want to offer certain groups an incentive to participate – so, some of those harder to reach segments. So, in this case, if we’re looking at farmers or maybe a target that’s not necessarily as easy, like a general population target, we might increase that incentive.

But that set up phase would take probably about a week or so, or week and a half, just to make sure we get the right questions, who we’re recruiting, having our recruiters reach out, confirm appointments, set up those times for either that’s online focus group or those IDIs. So, our researchers would work really closely with a client to make sure that that was all done. And that’s kind of the same, no matter what the project is, because once you get into the usability test itself, that’s where you have the creativity. But in terms of setting it up, make sure everyone’s comfortable with the technology itself. So, in this case, we use Validately. So, making sure that that app is downloaded on their systems, they’re able to log in, they’re able to communicate because you want to eliminate any of those technology issues prior to the test so that you’re not wasting any of the time.

[DUNCAN] Now you were saying technology, earlier, as a challenge that sometimes people can have to overcome during UX testing. Were there any other challenges that you had to overcome during the project?

[SHONNA] Not really, no.

[DUNCAN] Awesome.

[SHONNA] I guess the only – I think it would be a piece of advice after working on some of these projects is knowing when you’re developing any sort of moderator guide or the types of questions. So, just like a user might have their own way of looking for information, they also have their own way of understanding the questions that you’re asking. So, you need to be able to express exactly what you’re looking for them to do without leading them down a certain path. And so, I think it just comes with experience and our researchers are very in-depth in knowing how to ask the right questions and get the user to go to the place that we need them to without actually having to tell them.

[DUNCAN] What would you say of the biggest benefits are for our clients who do usability testing before their app or their websites launched?

[SHONNA] Honestly, I think it’s the fact – so I didn’t mention it before and I should have – but as much as researchers can observe, the clients can actually observe as the test is happening themselves. So, they can sit in, it’s almost like a back room, but they just go and they can listen and they can watch and they can see so they can get their insights right away without having to wait for that final report or that presentation. It also offers the researchers and our clients to have those conversations upfront. So, for instance, you might get two or three tests, out of say 20, and you’re noticing, OK, well this starts to be a trend. Can we steer them in a different direction, or can we dig a little bit deeper? And you can make those calls as you go through the research. So being able to have the client and observe and provide updates as you’re going through, I think just makes the insights even more valuable and save some of that having to fix things too late, so you’re being a bit more risk adverse in this way.

[DUNCAN] Yeah, it makes sense too, that you’d want to be able to spot those issues before you get the app out. I know it’s a head-slappingly obvious kind of thing, but a lot of these issues are things that you don’t even think of until you actually in there watching somebody, a user’s journey on the app or say on the website. Having that ability as a client to actually watch the participants use that stuff and go through their journey, as the research is unfolding, must be really, really useful.

[SHONNA] It is. Yeah, it’s similar to, like I said, a focus group, but it’s like you get that real-time interaction. Like I said, being able to observe facial expressions, seeing when certain users might have a challenge with a certain task, starts to just form the insights as you’re going. And I think that that’s a really cool opportunity, not just for our clients, but our researchers as well, to build that report at the end, just to make it more valuable for our clients.

[DUNCAN] 100%. What would you say is the average turnaround time on a UX project like this?

[SHONNA] Well, I would say you get your standard setup, a week to two weeks, let’s say that would be with setup, moderator guide, recruitment, that kind of thing. And then, it just really depends on the availability and how many tests you’re doing. So in this case, I think I’m just going to see if I can find it here. Yeah, in this case we did it probably in about three to four weeks from start to finish. And so yeah, that was doing all the tests, writing a report and delivering that presentation back to the clients. I think it can be turned around quicker than that. It just really depends how quickly you get that set up. And the recruitment is one of the most important pieces. You want to make sure that you have the right people and availability of everyone.

[DUNCAN] So, from a research perspective, especially as a researcher, where do you get your joy out of doing UX testing? What’s the most interesting point of doing it?

[SHONNA] That’s a good question. I would say for me personally, as a researcher, it’s getting to observe in real time what’s happening and connecting with the user themselves. It often can be a very funny conversation. The feedback that you get, also very insightful. So in this case, I wouldn’t call myself an expert by any means in crop science, but having the ability to talk to all of these people who are experts and who deal with it, it’s a learning process and it’s very interesting to see how other people get to the information that they need to make their own business decisions. So I think that’s probably the most interesting part.

[DUNCAN] That’s awesome. And I guess getting a chance to actually meet your participants and talk to them is always a lot of fun too. So, that’s really cool – awesome. Well, I really want to thank you for coming on and chatting with me about UX testing. This has been really informative for me and I really appreciate it.

[SHONNA] No problem. Thanks for having me.




[DUNCAN] And there you have it. I’d like to thank our guests for this episode. Thanks to Petrina Holoszyc from Validately for joining us to chat about Validately and how it can be used to help in the UX testing process. Having Petrina on the podcast was really informative, and I really enjoyed speaking with her. You can try Validately on a trial basis right now, just head over to validately.com for more information or check out the podcast episode webpage on our website for a link.

And I’d like to thank Shonna Caldwell for coming back on the podcast to talk about UX testing at Insightrix. Having her share her perspective on usability testing was really informative and it was really great to have her back on the podcast again. If you’d like to know more about the usability testing at Insightrix, head over to our website, www.Insightrix.com. There you’ll find several case studies all about how we’ve helped our clients develop better UX for their end users.

And of course, we’d like to thank you, our huge and super supportive fan base. Thanks for listening everybody. If you’re looking for a new way to access Stories of Market Research, you can find the podcast on Apple Podcasts, Spotify, Stitcher, or pretty much everywhere else podcast content can be found. You can also find us on YouTube or access the podcast on our website archive, insightrix.com/podcast, if you’d like to explore past episodes of Stories of Market Research.

If you have any questions about this episode or past episodes or about the podcast in general, or if you think you know somebody who you believe would be the perfect guest for the Insightrix podcast, send us an email and let us know. You can reach the Insightrix podcast at podcast@insightrix.com. We’d love to hear from you.

Thanks again for listening. We’ll be back again in another couple of weeks with another episode of Stories of Market Research: The Insightrix Podcast.