The sample size bias is one of the biggest pitfalls I've witnessed with YouTubers (particularly NewTubers) that I speak to. While the term isn't technically "real" in a sense (as trolls love to tell me), the issue is still prevalent.

What is a "Sample Size Bias?"

It's pretty self-explanatory (which is the exact reason I chose that name for the concept).

It's a statistical bias, that forms due to something to do with a dataset's sample size. Shocking, I know.

Snarkiness aside, a sample size bias (in the context of content creation) occurs when a creator starts looking at their calculated statistics such as Click-Through Rate (CTR), Average View Duration (AVD), Viewed Vs Swiped Away (VVSA), etc. and begins making major decisions or judgements about their video which has low views.

The issue here is that the "low views" in question are not representative of a substantial or even relevant audience, making any "objective statistics" to be misleading at best.

Example

For example, if a thousand random people off the street probably won't want to watch a Minecraft video, so if you gave them the option, maybe only tens of them (let's pretend, 50 people) would watch.

This translates to a 50/1000*100 = 5% CTR. Which hey, is pretty average!

Now let's say that of these 50 people, 35 of them clicked off within the first 30 seconds (which is VERY realistic!). That 70% drop off is going to absolutely tank the video's AVD.

So we now have this video with an alright CTR, and a terrible AVD. So this means that the packaging (title and thumbnail) is good, but the video is pretty bad, right?

Wrong, or well, not necessarily.

To avoid more napkin math, just consider the fact that this hypothetical creator has decided that the video is "bad" due to the judgement of just 50 people. In a world where there are millions of views to be had, does that sound reasonable?

Don't let a miniscule of your potential audience sway how you fundamentally make videos. Those hypothetical 50 people might only like Minecraft PvP videos, while the video itself is a building guide.

Would MrBeast let 50 random Grandmas dictate his editing style? Are you going to let some snotty-nosed child with no attention span tell you that your video is boring?

Wait until you have some real numbers before looking at those stats.

But I got x amount of views!

Okay, shut up.

...Oh you're still here? Great. Scroll down to this section.

Why should abide by this arbitrarily made up term?

Well, while the phrase's existence is arguably real, the raw concept itself is perfectly valid.

Bias skews the results, whereas random errors increase the variance but do not skew the results. In a random sample, larger sample size can help reduce the influence of random noise

- Stanford University

• Larger sample size = reducing the random noise and outlier inaccuracies
• Small sample = higher noise and outlier inaccuracies
• Sample size = The size of the sample (whether it be large or small)

Again:

Bias skews the results

So... Sample size bias = A bias that is caused due to the provided sample size

If you don't trust the massive, internationally known University, then here's something from The Australian Association of Mathematics:

Okay, I get it, when can I look at my stats?

The consensus from large creators that I've spoken to is 10 thousand (or more).

That's 10,000 impressions for the CTR to be judged, and 10,000 views for AVD and other related metrics.

More is always better of course.

Now... you could start looking once the views get into the thousands, (i.e. 1k, 2k, etc.) but remember that the dataset is still not good or even particularly representative.

So if I shouldn't look at my stats, how can I improve then?

Get some feedback

Whether it's from a help community like NewTubers, your friends, family, or even the comment section, getting feedback from real people can be invaluable.

Even if someone is not particularly "YouTube knowledgeable", they still give insights on parts they found boring, overstimulating, misleading, etc.

Read the page about asking for feedback for some guidance.

While this is technically a super-duper-horrible sample size bias in itself, a conversation can lead to critical thinking and smart, justified decisions, as opposed so some random number on a screen causing some sort of mixed interpretation.

Give it some time

Many people get worried about stats and such very early onto a video's life (i.e. a few days). Refer to the previous section for advice about poor stats/sample size bias in the first few hours of an upload.

Take a deep breath and be critical on yourself

Truth is, your video probably sucks. The sooner you realise that they do, and the sooner the realise why, the better.

Keep uploading

Stop worrying and go make your next video. Pick one (or more, whatever you can handle) thing that you should improve on, and do it. Over time you will be making substantially better videos.

TL;DR

Oh, so you're lazy. Okay, here's what you can look at and/or share to your friends:

"Sample size bias" is sort of a summarised phrase I use that encapsulates the concept that undersized datasets are not usable to draw valid conclusions.

For example, 200 viewers is not a decent representation of "the masses" on YouTube. 200 random adults on the street probably won't watch a Minecraft video even if you paid them to, but there are millions upon millions of people on Youtube who queue up to watch Minecraft.