Inference with Observational Data

Binomial model

The problem can be modeled using a Binomial distribution. The binomial distribution is frequently used to model the number of successes (\(p\)) in a sample of size \(n\) drawn with replacement from a population of size \(N\). In other words, it can be thought of as simply the probability of a SUCCESS or FAILURE outcome in an experiment that is repeated multiple times. In our case, we are going to consider each eye surgery as an independent experiment.

So for our particular sample, we can say that From 45 observed surgeries only 5 show an outcome=0 and the rest outcome=1.

Can we get an idea of the variability of such proportion?

One proportion z-test

Since our random variable \(X\) follows a binomial distribution we can estimate the ‘margin of error’ or to use the proper term, confidence interval for the observed \(p\) using a one-proportion z-test. Under a one-proportion \(z\)-test, the binomial distribution is approximated to a normal distribution (see Figure 1), then we can use this approximation to estimate the confidence intervals.

We only need to obtain \(\mu\) (expected value for \(p\)) and standard deviation \(\sigma\) and then calculate the confidence intervals for a given \(\alpha\) (the probability of rejecting the null hypothesis when it is true) using the \(z\) statistic.

\[ z=\frac{\mu-p}{\sigma} \]

In R we can use the function prop.test() that implements the Z test for proportions using Wilson’s ideas about approximating a binomial to normal distribution.

n<-45 # the total number of surgeries (trials)
p<-40 # the number of surgeries where outcome 1 was observed.
mu<-0.1 # the expected probability for observing outcome 1. 

prop.test(p,n,mu,correct = FALSE, alternative = "two.sided")

## Warning in prop.test(p, n, mu, correct = FALSE, alternative = "two.sided"): Chi-
## squared approximation may be incorrect

## 
##  1-sample proportions test without continuity correction
## 
## data:  p out of n, null probability mu
## X-squared = 311.17, df = 1, p-value < 2.2e-16
## alternative hypothesis: true p is not equal to 0.1
## 95 percent confidence interval:
##  0.7650090 0.9515953
## sample estimates:
##         p 
## 0.8888889

So according to the one-proportion \(z\)-test results, we can say that surgeries observing an outcome=1 will be between 76% and 95% of the times (with 95% confidence level) In other words, if someone repeats this process (i.e. it takes another sample from the population of eyes surgeries using the same technique), there will be 95% of chance that 75 to 96 percent of the surgeries have outcome=1.

The \(z\)-test for proportions provides us not only with information about confidence interval but also let us to test different hypothesis. For instance, in the above example we are testing that on average the outcome will be observed 10% of the times. So the null hypothesis is \(H_{0}=0.1\) and the alternative hypothesis is \(H_{A}!=0.1\). According to z-test p-value we can reject the null hypothesis. In other words, if someone else samples another eye surgeries using the same technique, there is 95% chance of not observing the outcome 10% of the time.

Notice that most of the time we would check if there were 30 data points or more, but for a proportion, we do something slightly different. When data is binary, like we have here, the central limit theorem kicks in slower than usual. The standard thing to check is whether:

\[n.(p/n)>5 \] \[n.(1−p/n)>5\] So if we do: \[ 45 . (40/45) ~= 40 \] \[ 45 . (1-(40/45)) ~= 5\] Oops, we have a problem… Actually if we look carefully at prop.test() output we can see a Warning message saying something like Chi-squared approximation may be incorrect So, what we can do?