Another probability question

I would want to see several thousand trials.

Although, the sample mean of the sampling distribution is approximately equal to the mean of the sampled population for all n >=30

The approximation tightens as the sample size increases.

I tried a new (to me) method for this (Bayes), I think you can say:

After 264 trials (138wins/118losses) you can be 90% sure that your edge is positive (ie true strike rate >50%).
After 422 trials you can be 95% sure that your edge is positive (ie true strike rate >50%).

Also, out of interest:
After 1022 trials you can be 90% sure that your edge is true strike rate >52%.
After 1684 trials you can be 95% sure that your edge is true strike rate >52%.

Okay, i've been trying to do a bit more working out. I make it that 100 trials gives the probability of the success continuing as 82%, after 250 trials 91%, 500 - 97 and 1000 - 99%.

Does that sound right?

Doh! bit slow there!

Interesting, we have come at it slightly differently but the figures seem roughly similar.

JLivermore,

Can you post the Bayes' formula used? It's been 9 years since I've seen one!

I always thought the best way to trial such an edge's significance was to calculate the p-value [p < 0.01]

Steeringjob - do you really think it needs thousands of trials to be confident?

Also, can you explain the following in simpleton terms as I don't really understand!

Although, the sample mean of the sampling distribution is approximately equal to the mean of the sampled population for all n >=30

^ Bayesian formula even

I asked a Ph.D statistican the sample size required forany such type of experiement - and the answer was "5,000 trials".

It tallies with JL's analysis above i.e. the level of significance increases as the number of trials rises.

What my comment means is this:-

- take the average answer from a sample of trials [i.e.at least 30
   bets];
- the trials should be taken from the phenomenon you are looking to
   model [i.e. your edge];
- the average answer for your trials will approximate the average answer for the phenomenon as a whole;
- the scope for error falls as the number of trials rise

I used an Beta distribution, which I *think* is the distribution of the Bayesian posterior.

So for the probability your strike rate is > 50%:
=1-BETA.DIST(0.5,

ty - yes, that tallies.

Okay, I'm not sure I get that. When you say take the average answer from a sample of trials [i.e.at least 30
   bets]; are you saying, to take average answers from trial runs of the system over different numbers of sample bets?

Also, when you say - I always thought the best way to trial such an edge's significance was to calculate the p-value [p < 0.01]
are you saying to calculate the number of trials until there is a

it didn't print all my formula - here is what I meant:

=1-BETA.DIST(0.5, number of wins, number of losers, TRUE)

Not sure what happened there, half my post disappeared.

Okay, I'm not sure I get that. When you say take the average answer from a sample of trials [i.e.at least 30
   bets]; are you saying, to take average answers from trial runs of the system over different numbers of sample bets?Crazy

Also, when you say - I always thought the best way to trial such an edge's significance was to calculate the p-value [p < 0.01]
are you saying to calculate the number of trials until there is a

It did it again!!

If you are saying to calculate the figure so that the chances of the true figure being less is

If you are saying to calculate the figure so that the chances of the true figure being less is smaller than 1%  then I calculate that number to be 839

got called away from  my desk............

Thanks for the earlier replies, i'm still not really sur of a number I shoul aim for though.

What complicates the matter is that the system only throws up a bet approx 2-3 times a week and there is no way to back-test it prior to March this year so waiting for meaningful stats could be a long time!!!

statistical significance can be a pretty high bar.

If I was you I would bet it with my uncertainty about the edge taken into account and track the edge as you get more bets.

Judge R
You say " What complicates the matter-----only throws up a bet 2-3 times a week-----so waiting for meaningful stats could be a long time ".
Now I'm no statistician, but couldn't you therefore just keep tracking all future events  and if they ( being those that do not throw up bets) keep running at a flat stake bet loss ( as they must do if the ones that throw up a bet strike at 52% rate), then your system is still theoretically intact, despite not getting any actual bets for the period concerned.
Or is that false logic ?