[Solved] Ads May Actually Improve

To date, you have staunchly avoided placing any advertisements on your website. However, with revenues declining, you are considering relenting on this position. Before making a full commitment, you decide to try to determine whether the presence of ads has any negative impacts on your sales. Therefore, you make a deal with an advertiser to show a 10-second pop-up ad (that pops up and plays when the site is first visited and then becomes a banner ad on the page) intermittently over a period of one month. Suppose you’ve decided to show the ad during business hours (9:00 a.m.– 5:00 p.m.), and not show the ad any other time during the month.

What are the “subjects” in this (field) experiment?
What is the treatment?
What is the relevant outcome?
Is treatment assignment random?

Suppose for each visitor to your site, whether the visitor sees the ad is determined randomly. After the month is completed, you have the following information:

Mean sales when ad was shown: $26.75

Standard deviation of sales when ad was shown: $18.21

Number of times ad was shown: 8,172

Mean sales when ad was not shown: $27.21

Standard deviation of sales when ad was not shown: $19.20

Number of time ad was not shown: 10,437

Does showing the ad affect your sales? Explain your reasoning.

Using the data from above, the advertiser claims there is evidence that running the ads may actually improve your sales. Is there evidence for this? Explain your reasoning, why or why not