5 Key Concepts to Understand About Tracking, Testing, and Making Decisions

The Transcript

5 Key Concepts to Understand About Tracking, Testing, and Making Decisions

Voiceover: You’re listening to The Digital Entrepreneur, the show for folks who want to discover smarter ways to create and sell profitable digital goods and services. This podcast is a production of Digital Commerce Institute, the place to be for digital entrepreneurs. DCI features an in-depth, ongoing instructional academy, plus a live education and networking summit where entrepreneurs from across the globe meet in person. For more information go to rainmaker.fm/digitalcommerce, that’s rainmaker.fm/digitalcommerce.

Brian Clark: Hey there everyone, welcome to The Digital Entrepreneur. I am Brian Clark, founder and CEO of Rainmaker Digital. Today, as always, we have Jerod Morris, VP of marketing for Rainmaker Digital. We also have our very own data ninja, Loryn Thompson. Don’t ever call yourself a ninja. We’ve talked about this. I can call you a ninja, right?

Loryn Thompson: Exactly.

Jerod Morris: Yes.

Brian Clark: Okay, this is a podcast, you have to talk. Today we’re flipping things around a little bit to close out our series on the elements of the modern website. We’re going to talk about testing. Instead of me rambling on about it, I’ve decided to flip the switch and let Jerod and Loryn talk about testing. As we came out of our launch and build phase — which, when we launched Digital Commerce Institute in the fall, that was the last piece of the puzzle. We don’t plan on building anything brand new or launching something new. What we’re doing now is shifting into our ‘optimize and grow’ phase.

A big part of that initiative that we knew we wanted to tackle as soon as we could — it was at the beginning of the year — was to move from tracking and testing when we thought it might be a good idea, to a culture of tracking and testing. There’s a big difference there. A one-off test here or there can give you valuable information. A process, a system, a culture that basically tracks and tests everything and then tries to make sense of the resulting data and apply that back into our own intuitive creativity, that’s where we needed to go next. That was really what I tasked Jerod and Loryn with, so who better to talk to about it than these two. Are you guys ready?

Jerod Morris: We are ready. Yes.

Loryn Thompson: Yup, ready.

Brian Clark: All right. I’ve got here some notes from you two. This catchy headline, I wonder where you got that from.

Jerod Morris: I know.

Brian Clark: “5 Key Concepts to Understand About Tracking, Testing and Making Decisions. Very nicely done, Jerod was that you?

Jerod Morris: That was me, yes, but with Loryn’s help.

Brian Clark: Good job. Oh, with Loryn’s help, okay. We know Loryn’s the brains behind this but you’re the front man. She’s got to feed you the smart stuff.

Jerod Morris: Right, and I do want to say, Brian, as the lead-in to this conversation — when we talk about data and testing I think it’s so important, as you just said, to create a culture of testing. It’s so important to have two things, number one you’ve got to have commitment from the top. Because if you aren’t committed to this and to doing this, then something like this isn’t really going to happen. I think you’ve really got to have organizational commitment to this being important.

The second thing I think you’ve got to have is enthusiasm. I think that’s really what Loryn has brought, a real enthusiasm for the subject, for learning more, for figuring out where we can use it. It’s really helped and infused what we’re doing with a lot more information and insight. I think as we talk about this, this isn’t listed here as one of the things we’re going to talk about, but it’s very important if you want have this culture of testing. Having that commitment from the top down and then individuals with enthusiasm for actually doing the work that it will take is really important.

Loryn Thompson: Mm-hmm (affirmative).

Brian Clark: Yeah. You not only have my commitment, but I’ve been nerding out right along with you. I think we’re all enthusiastic about it because it’s fascinating.

Jerod Morris: Yeah.

Brian Clark: It’s fascinating when your intuition actually gets proven correct. It’s also just as fascinating when you’re completely wrong. You just have to be open to being wrong because …

Loryn Thompson: Whichever one makes you more money.

Brian Clark: Exactly. Blindly going along with a losing strategy is not that smart in my book. Yeah, I think the enthusiasm is there as soon as people get over their trepidation about it. People think split testing is complicated, but with the right tools it’s becoming easier and easier. Even I can do it when you let me.

Jerod Morris: Yes.

Brian Clark: All right, let’s go through these 5 key concepts. What is number 1?

The Best Time to Start is Now

Jerod Morris: Number 1 is starting now. Again, this is one of those that almost seems so simple you don’t want to say it, but it’s really important. It’s similar to what we tell people on the The Showrunner with podcasting who see that there are people who’ve been podcasting since 2008 so they feel intimidated. It’s like, “Yeah, okay. The best time to start a podcast was back then — 2007, 2008 — but the next best time is now.” Same thing with a blog.

Think about if you haven’t been working out or if you’ve gotten out of shape. Sometimes you’ll think about it and it’s like, “Man, I don’t even want to get on the treadmill for the first time because I haven’t been doing it for so long.” You can’t think that way, because you can take a positive step in that direction today to start moving forward in a better way.

I think it’s the same thing with data. Maybe you’ve had analytics on your site for five years but you’ve never done anything with it and now you get in there and you’re intimidated because there’s all this old data and you don’t know what to do with it. Start today. Take a step today to understand something better or to set up just one simple test to get going.

You’re going to be a lot further along in six months if you take that first step today than if you continue to lament the fact that you haven’t started yet and continue to allow that to keep you from getting going. Again, it’s basic but it’s an important mental hurdle for some people to get over. Whatever you’ve done in the past, it’s okay. Just start today moving forward in a better way.

Loryn Thompson: Yeah, I want to add to that. Even if you don’t think that you really need the data right now or anything, or now is not the best time to devote the energy to putting everything together. It can be really time consuming to get set up. tThe important thing to remember is that with data — having the historical data, having three months of data — that puts you way ahead already. It’s always good to just go ahead and get it set up, even if you think, “Oh, Well I need to devote more time to this right now.” Just put the data in place. You’ll be glad you did in a few months.

Jerod Morris: Mm-hmm (affirmative).

Brian Clark: Yeah, that’s good advice. The next thing I want to talk about here is something I’ve seen over and over. When a company decides, “Okay, we’re going to embrace testing. We’re doing this.” The first thing they then do is look for some turnkey off-the-shelf system that they’re going to slap onto their particular business. Silly or not?

Context is Everything

Jerod Morris: I think the number one thing that you have to realize when you’re going to start doing this is you’ve got to understand what you’re tracking and what data and what reports and what numbers are going to be important for you. I think back to my time when I was running Midwest Sports Fans five or six years ago and all of our monetization was based off of display ads.

For Midwest Sports Fans the page view number was huge. Having Google Analytics driving page views — that’s how we were going to drive revenue, because we weren’t doing anything else with it. But you think about what we’re doing now and think about all of our properties — page views are a nice number and they can tell us something, but it doesn’t tell us very much. We need to convert people onto an email list. We need people to start trials of the Rainmaker Platform.

You’ve really got to start with where you want to end up. What your business objectives are, what numbers drive those objectives. And then what’s going to allow you to track that. The context is everything. I think that’s why you really need to start thinking there and that will help tell you what you need then to be tracking and what tools will help you do that.

Loryn Thompson: Definitely. One thing, when you start looking at that data it’s really useful to start breaking that down into different kinds of segments. You had one session number, you got ‘X’ many sessions last month. That’s great, but what does that actually mean? Maybe you start by looking at people who came from a certain email, or people who landed on the home page, or people who landed on a different page. Looking at how their behavior compares to behavior of people in other segments on this site — breaking that down and figuring out what those sessions mean will get you a lot more mileage out of your data.

Brian Clark: Mm-hmm (affirmative). Yeah, it’s interesting. I think page views are completely irrelevant to a content marketing-driven business. With the state of online advertising they may before irrelevant to everyone. But there are certain key metrics that I think anyone who’s following along with this series … Converting the email, that’s key. Converting to customers based on various adaptive funnels that you create — the kind of stuff that we’re going through here with this series. These are all the things you have to not only see what approach works best, but then you’ve got to be able to tweak and optimize. You can’t do that blindly. You don’t just decide to change a word, you see what word works better.

Jerod Morris: Yeah, exactly.

Loryn Thompson: Yeah.

Understanding What Your Tools are Telling You

Brian Clark: All right, what about tools? People get a little bit hung up on that. What’s your advice about understanding how to use this technology to best effect?

Jerod Morris: Yeah, this is such an important one. Loryn, this is where I’ve learned a lot from just in the time that we’ve been working together. Especially this concept of, “your tools are telling you certain things, but there are also things that they are not telling you.” It’s very important to understand this. Because you can be getting this number, but if you don’t know where this number is coming from and what data is coming in to give you this number, there could be giant glaring holes that mean that it’s painting you an incomplete picture. Again, I’ve learned a lot from you in this sense. And you’ve got a couple of good examples as far as this goes which help illustrate that.

Loryn Thompson: Yeah, for sure. One of the main things is that looking at website data, you really do want to focus on users. It’s nice to know how many sessions you have, how many page views, but really what you’re looking for is when people get to your site, what are they doing? One of the biggest limitations with Google Analytics — which is a super popular website analytics tool — is that the tracking is cookie-based, which pretty much the best we can do right now.

But that means that each user, when you look at that user’s number, it’s only recognized with the exact same browser and device configuration as they were the first time. If you are on Firefox and go to our site one day, and then the next day you go on Safari, Google Analytics isn’t going to know that you’re the same person. That’s why it’s really important to do things like … There are a few things you can do to your website like user ID tracking.

Whenever you look at that user number if you don’t have user ID tracking in place it’s likely over inflated. My best suggestion with things like this is you can’t always fix these huge holes, you just have to be more aware that they exist. As you’re moving through the data if you see something that indicates that, “Look, we had so many more users this time,” it’s, “maybe people are switching from their desktop to their phone to access our site because we started putting more podcasts on,” or something like that.

One of the other things that I want to touch on is that there are a lot of platforms that will give you proprietary metrics that they don’t really tell you how they came up with them. They’re like, “This is the magical impressions reach engagement number that you need to be paying attention to. It’s fantastic.” Okay, what does it mean? If you only have that number and you don’t really have anything to compare it with, then it doesn’t really do anything to help your business. I personally would never rely on any metrics that I don’t know how they got to be there, how they were counted.

Jerod Morris: I think what’s important there is it doesn’t necessarily mean that the metric itself is bad. It could be good, but for you to make an informed decision on it you have to know what it means. It’s very important that you can wrap your arms around it, embrace it, understand what it means so you can use it to make a smart decision.

Loryn Thompson: Exactly. A lot of these companies it’s like a whole ‘business secrets’ thing, and that’s fine. But if you don’t know what’s behind it then how do you know what that number’s limitations are? How do you know what it’s really getting at if you can’t know how it’s calculated?

Jerod Morris: Yeah.

Balancing Qualitative and Quantitative

Brian Clark: That’s interesting. Okay, so everyone is obsessed with big data, the quantitative side of things. Is there still room for qualitative data?

Jerod Morris: Yes, and you need it. I think you can fall into that trap when you start talking about data and start talking about testing. That all you do is look at things from a numbers perspective. Numbers can tell you a lot. They should tell you a lot. But that doesn’t mean that asking questions — having actual conversations — doesn’t matter.

We started putting up a survey on the Rainmaker Platform site to get some information on people when they left a certain page to find out why they were or were not doing something, and we’ve gotten some good insight from that. Now that, obviously, gets boiled down to into data. But it’s not just tracking how someone comes and goes from a site, it’s asking them a question. Brian, just 15 minutes ago we got off the call and we were talking about reaching out to some people to get a testimonial. In that email that I sent out to a few folks I asked them, “Hey, how are things going in Digital Commerce Academy so far?”

I’ve already gotten three replies with people telling me a little bit about their experience and things they liked and things that they wish that it had. It’s really good information, because we’re reaching out to our most active people. Are we going to jump and go do something with that? No, but it’s the kind of information — hearing it from people who are clearly engaged and really using the product.

Now we take that, we combine that with quantitative data that we have about usage and different things, and now you really inform that intuition, which we’re going to talk about here in a minute. But I think you really need both. If you just rely on one or just rely on the other you’ve got an incomplete picture. Incomplete information that you’re making a decision with.

Loryn Thompson: Definitely. I think one of the major things when you’re looking at data and you’re looking at testing, the tests you can do are kind of limited to your perspective and what you can come up with. When you ask those open-ended questions you might be completely blindsided and say, “Oh, this actually what’s really important to our customer.” You are only now giving them the chance to speak up about it, whereas if you had just decided to test things and you came up with a bullet point list of five things you wanted to test, what’s the likelihood that the thing that the customer wanted would actually be on that list? It really does matter to get out there and talk to your customers and know your customers because that’s what’s going to inform which tests you actually should run.

Brian Clark: It’s amazing that I went a decade with never doing surveys. I really prefer to observe people in the wild, if you will, and comments in social media and forums — all of that stuff. It worked really well for us. Yet now that we’ve started doing this — now I’m still against multiple choice questions that aren’t carefully crafted because you can suggest answers. But the open-ended question thing? I love it. It’s unwieldy. It’s not scalable to a certain degree. But that’s where its power is. It’s real people telling you real stuff and you can figure things out from that.

Like Jerod said, one answer is not going to make you go act, but when you start seeing the same kind of pattern recognition that I would see out there in the wild, that’s where the answers come from. We’ve gotten to the point where we have more customers — 170,000 now — we have more customers than some people have prospects, so we have to talk to them. We made that shift starting a couple years ago, and it’s really intensifying now. Yeah, I totally agree with you guys on that one.

Intuition Still Matters

Brian Clark: Then finally, we’re shifting to a true data-driven company, yet we got here on intuition and creativity, which really stems from knowing your market. Knowing your audience. Knowing who these people are and what their pain points are, and doing your best to try to help them with that. Are we compromising what got us here with this data approach?

Jerod Morris: No, I think we’re informing it better. It’s like you said, you’ve done so much off of intuition, and that’s still important. You’ve got to know your customer, because ultimately you still have to make a decision. It’s not like we’re turning the decision making over to the data. These are reports, and you read the reports and you allow them to inform your decision. But it’s really important with data — it’s like the reports that Loryn sends us. These are reports of past events which we’re then using to predict the future events as much as we can, but there’s no guarantee.

A lot can change from the time that the data was collected to when the decision that you’re going to make is going to have its impact. There may be different environmental factors that change. Shifts that you’ve seen. Things that happen. “Okay this works with this data, but we know that this has changed. So let’s tweak this, but this stays the same,” and you move forward. That intuition and that knowledge of what to do is still extremely important.

We go back and we talk about that qualitative information that you gain. All of that goes into it and is combined into that gut, that instinct, that intuition that eventually helps tell you what the best decision is. But you do yourself a disservice to close yourself off to all the good information that you can get. We’ve drawn a distinction between good and bad in here. As long as it’s good information then you’ll be able to make a well informed decision, but I don’t think there’s any replacement for good intuition. But intuition is not just something you’re born with or that you wake up with, it’s …

Brian Clark: It’s informed creativity. We’re just informing ourselves better.

Jerod Morris: Exactly.

Loryn Thompson: Exactly. That’s why it’s so important to look at your data as real events. What does ‘sessions’ actually mean? It means that your website was opened and viewed this number of times. When you start to break down the data into the concrete real world as close as we can get to events that actually happened, that helps your intuition that much more. When you start looking into some of your tools you might get, “What does this mean? What does time on site mean?” It’s important to know how all of that is calculated, what all that means, so that you can go forward and say, “Yes, this is what my instinct was before we did this promotion, or this campaign. This is what we learned from the campaign. And this is how I’m modifying my instinct to move forward.”

I definitely think that that’s the way forward. You can’t really just rely on the data or just rely on going off on your own. You have to be able to marry the two and have a feedback loop where you start a campaign, a promotion, or some sort of event and then you execute it. And then you come back and say, “Okay, were those presumptions we had at the beginning correct? Were those the right ones? If not then why? If they were then cool, how do we take that and move forward with it?”

Brian Clark: Yeah, it’s interesting because from a very simple standpoint when you do iterative content development you’re really looking at very simple data — social sharing, comments — just indications that it resonated. Then you do more of that and less of the stuff that didn’t get that kind of response. It’s really not that alien to me, it’s just better data. More sophisticated predictive analytics.

Loryn Thompson: Yeah. The important part of intuition for me is the exploration. Because if you just follow what the data says about what you’ve done in the past to a T, you’re going to end up doing the same things over and over that work, and it’s going to end up not working in the end.

Brian Clark: That’s so key, because you have to try new stuff to see if it’s going to work. And if you’re completely married to the data, you truly are non-creative in a literal sense because you stop innovating. You stop moving forward. Great answers. Loryn, we talked briefly about tools and I think people get hung up at this level. We use the testing tools built into Rainmaker, Google Analytics, and what else?

Loryn Thompson: As far as website data goes, we use Google Tag Manager so I can easily implement things without bugging developers.

Brian Clark: So another free tool from Google.

Loryn Thompson: Yes.

Brian Clark: Okay, my point being here that you do need the set of testing tools but you don’t need necessarily a fancy analytics package. If you know how to work with Google Analytics it’s all there for you.

Jerod Morris: Yeah.

Loryn Thompson: Definitely. Then we also use Hotjar, which is what we’ve been doing our surveys with.

Brian Clark: Cool, okay.

Loryn Thompson: That’s been working really well. They also have heat mapping which is pretty cool.

Brian Clark: Interesting. Cool. All right. Well, I think that covers the 5 elements that we all agreed needed to be covered. Jerod, what do we have today for the folks at home as a free offer?

Jerod Morris: We have the free membership at Digital Commerce Academy, which, if you have not yet started, you really should get over there and get started today. You can do that at rainmaker.fm/digitalcommerce. That’s where you can go get your free membership activated.

Once you do that you have access to a wealth of content, including free lessons in Brian’s course on Building an Online Training Business the Smart Way. Free lessons in Chris Garrett and Tony Clark’s course on marketing funnels. As well as some case study webinars that we’ve done that you have access to. You want to get in there and start seeing everything that’s available, because again as soon as you register — it’s free — then you’ll also get a newsletter from us every week which has very interesting and useful information in it as well.

Brian Clark: Which includes these very podcast episodes, right?

Jerod Morris: Exactly. Yes.

Brian Clark: Also, if you are listening over at iTunes, if you would — before you head over to get your free registration — if you could leave a rating or review for us over at iTunes we would most appreciate that. Helps us out to find new people in the iTunes ecosystem and we definitely appreciate any help you can give us on that. All right.

Jerod Morris: Hey, Brian, real quick, we probably should let people know the reason why we have the URL rainmaker.fm/digitalcommerce is because we’re tracking conversions from this podcast episode. Instead of saying digitalcommerce.com/free, or whatever it would be, we have that URL which will then redirect you. That way we can track whether people are actually paying attention to it.

Brian Clark: It’s a little inside baseball right there.

Jerod Morris: Yeah, exactly.

Brian Clark: Okay. Well, Jerod, where are we going from here? We’re done with our series. I guess we’re going to have to come up with some new topics.

Jerod Morris: We are. I think this is where Loryn takes over the show.

Brian Clark: Yeah, Loryn, you don’t have stuff to do right? Just remaking the culture of the company, it’s not a big deal. All right, we will be back next week with a new episode and we promise to come up with something good. Until then, take it easy, keep going. Jerod, Loryn, get back to work.

Jerod Morris: Thank you Brian.

Loryn Thompson: Always a pleasure.