Assignment 4 Implement for both SPY and your stock i

Assignment 4

Implement for both SPY and your stock i.e. CRM

A set of numbers satisfies Bernford’s law if the leading digit d occurs with probability

Therefore, the frequency distribution of the leading digit d in such a set is the following: π1 = 30.1%, π2 = 17.6%, π3 = 12.5%, π4 = 9.7%, π5) = 7.9%, π6, π7 = 5.8%, π8 = 5.1% and π9 = 4.6%.

In this assignment, you will examine this law when applied to a large online retail dataset. You have price data in ”on- line retail.csv” for about 500,000 items from different countries. You will ignore all items that start with 0.

You compute the real distribution F = (f1, f2, . . . , f9) of frequencies of the leading digit in these prices and compare F to two models:

1. Model 1: equal-weight distribution: each leading digit has the same frequency 1/9 = 11.1%. In other words, your

predicted model of frequencies in this model is a 9-digit vector P = (1/9,1/9,...,1/9)

2. Model 2: leading digit follows the Bernford’s law. In this model, the prediction is a 9-digit vector π = (π1, π2, . . . , π9)

Questions:

1. plot 3 histograms for the frequencies for real distribution, equal-weight and Bernford (for each digit)
2. plot 3 histograms for the relative errors for Models 1 and 2 (for each digit)
3. compute RMSE (root mean squared error) for model 1, 2. Which model is closer to the real distribution?
4. take 3 countries of your choice: one from Asia, one from Europe and one from the Middle East. For each of these countries do the following:

(a) compute F, P and π

(b) using RMSE as a ”distance” metric, for which of these chosen three countries is the distribution ”closest” to equal weight P?

5. discuss your findings