Talk:Law of the iterated logarithm

Iterated logarithm?

Do Iterated logarithm and Law of the iterated logarithm have something in common despite the name? --Abdull 13:31, 3 March 2006 (UTC)

No, nothing in common, I believe. Boris Tsirelson (talk) 08:50, 14 May 2009 (UTC)

Leo Breiman

I put the late Leo Breiman's name in Wiki cites. Would someone with better acquaintance with Breiman and his works be willing to start an article about him? He was a distinguished statistician, and deserves recognition. Bill Jefferys 22:40, 26 April 2006 (UTC)

Puzzling picture

I like the idea of the picture added recently by User:Dean P Foster, but looking more closely I got puzzled. The law of the iterated logarithm states that S_n is sometimes of order $\pm {\sqrt {n\log \log n}}$ (but not more). Most of the time, however, it is of order $\pm {\sqrt {n}}.$ The latter is probably what we see on the picture. The former is probably what we should see on a relevant picture but do not see on this picture. The logarithmic curve should be the envelope of rare large deviations (short peaks). Boris Tsirelson (talk) 08:43, 14 May 2009 (UTC)

I start to understand; it happens because of a very nonlinear (logarithmic, in fact) vertical axis. The point is that on such a plot the graph of ${\sqrt {n\log \log n}}$ is quite close to the graph of ${\sqrt {n}}$ even though their ratio tends to infinity. For not puzzling the reader it is probably better to show both curves on the picture and to add some words of explanation. Boris Tsirelson (talk) 09:00, 14 May 2009 (UTC)

Oops... The problem is considerably harder! The log log factor becomes relevant only for huge n (such as 10¹⁰¹⁰). Straightforward simulation of a random walk on this scale is unfeasible. On the other hand, on this scale the random walk is so close to Brownian motion that their difference cannot be seen on a picture. Thus, one should simulate Brownian motion instead. Further, logarithmic axes turn the Brownian motion into an Ornstein-Uhlenbeck process (see Wiener process#Related processes). The latter behaves in the large like a sequence of i.i.d N(0,1) random variables. Thus the envelope of its large deviations is of order ${\sqrt {\log t}}$ where $t=\log n\;$. Boris Tsirelson (talk) 10:23, 14 May 2009 (UTC)

I will generate another picture with both the log(log(n)) line and the sqrt(n) line. I agree that n has to be huge for this difference to matter. But what I do like about the graph is that it shows how to prove the theorem (as Boris points out). Namely, about each order of magnitude, where it is, above or below, is independent of where it was (this is seen by the switching about that often). Dean P Foster (talk)

It would help if the plot indicated exactly what transformation is being used for the x-axis. Otherwise it looks fairly good. For some of the above discussion, and with the extra line included, it would be good to have a second version concentrating on the right hand end, say from 10⁸ onwards.Melcombe (talk) 09:35, 15 May 2009 (UTC)

Done. I think putting both graphs on the main page may be too much. The description of the transformations is in the discussion of the picture. If you think it should be in the caption on the main page, please add what you think best. Thanks everone for the suggestions! Dean P Foster (talk) —Preceding undated comment added 12:08, 15 May 2009 (UTC).

Interesting! I guess it is in fact Ornstein-Uhlenbeck simulated. Boris Tsirelson (talk) 14:41, 15 May 2009 (UTC)

These figures are now excellent and the information about the transformation is in the right place. If possible it would be good to copy the caption down so that it can be seen when looking at the expanded graphs. Melcombe (talk) 11:13, 18 May 2009 (UTC)

practical use?

So, can this be used to construct the confidence intervals? For example, instead of saying that

${\frac {1}{\sqrt {n}}}|S_{n}|<1.96\qquad {\text{with probability }}95\%,$

we could have been saying that

${\frac {1}{\sqrt {n}}}|S_{n}|<{\sqrt {2\log \log n}}\qquad {\text{almost surely}},$

which is a nice alternative, because frankly speaking the number “95%” is quite arbitrary… For practical sample sizes the two quantities are quite similar (which could be the reason why 0.95 was chosen in the first place), in particular they are same when n = 921. Of course this all would depend on the rate of convergence of the left-hand side to the limit — anybody knows the performance of the limiting quantity in the mid-size samples? … stpasha » 22:12, 6 December 2009 (UTC)

I am afraid, it could be drastically different from the whole philosophy of statistics; definitely, original research (to be discussed elsewhere, if you like). "Arbitrary" numbers like 0.95 are inevitable (according to that philosophy) and have important practical meaning: the probability of rejecting a true hypothesis. Boris Tsirelson (talk) 20:30, 22 December 2009 (UTC)

Wrong claim

The article claims, that

$\sup _{n\geq m}{\frac {|S_{n}|}{\sqrt {n\log \log n}}}\ {\xrightarrow {p}}\ 0,\qquad \sup _{n\geq m}{\frac {|S_{n}|}{\sqrt {n\log \log n}}}\ {\xrightarrow {a.s.}}\ {\sqrt {2}},\qquad {\text{as}}\ m\to \infty .$

However, the article on convergence of random variables claims, that from almost sure convergence follows convergence in probability. How is the above claim possible?

You are right: the "Discussion" section added recently by User:Stpasha is not accurate. Could Stpasha provide either references or proofs? Boris Tsirelson (talk) 20:24, 22 December 2009 (UTC)

This is a very valid point, so I went ahead and removed the supremums from the first formula. I guess somebody more knowledgeable with statistics would have to provide further explanation of what's going on with this law. The reference i used was: A.W. van der Vaart. Asymptotic statistics. ISBN 9780521784504.  … stpasha »  20:36, 22 December 2009 (UTC)

Now better, but still a bit inaccurate. Boris Tsirelson (talk) 20:33, 22 December 2009 (UTC)

---

Also, the text that follows the above equality, is

Thus, although the quantity $|S_{n}|/{\sqrt {n\log \log n}}$ is less than any predefined ε > 0 with probability approaching one, that quantity will nevertheless be dropping out of that interval infinitely often, and in fact will be visiting the neighborhoods of any point in the interval (0,√2) almost surely.

and it implies that in fact

${\frac {|S_{n}|}{\sqrt {n\log \log n}}}\ {\xrightarrow {p}}\ 0,\qquad {\text{as}}\ m\to \infty .$

Is this true? How does it follow from the Law of iterated logarithm? —Preceding unsigned comment added by 193.77.126.73 (talk) 16:52, 21 December 2009 (UTC)

It follows from the central limit theorem. Since by the CLT, the $|S_{n}|/{\sqrt {n}}$ is asymptotically normal, the probability for $|S_{n}|/{\sqrt {n\log \log n}}$ to lie within the interval (−ε, ε) will be approximately equal

$2\Phi \!\left({\frac {\varepsilon }{\sigma }}{\sqrt {\log \log n}}\right)-1\ \to \ 1$

And that implies the convergence in probability claim.  … stpasha »  20:36, 22 December 2009 (UTC)

Now all formulas are basically correct (to the best of my knowledge), but still, two remarks. First: "...converge in distribution to a standard normal, which implies that these quantities do not converge to anything neither in probability nor almost surely" – really? Yes, in fact they do not converge, but does it follow from convergence in distribution? A sequence of random variables can converge (very easily) in distribution and also in probability (and also almost surely). Second: I never saw such notation ${\nrightarrow }\ \forall ;$ is it a good idea? Boris Tsirelson (talk) 21:56, 22 December 2009 (UTC)

Expansion

I don't know the first thing about advanced math, but I do know this article's a little puny. Anyone with those sources should be able to build it up some. Ten Pound Hammer, his otters and a clue-bat • ^{(Otters want attention)} 20:50, 20 January 2011 (UTC)

Notation

Could we get a reference or link for the notation used, particularly the "-> forall" bit? Intuitively I know what it means (and I kinda like it) but more info would be nice, especially for people visiting this page who might not be familiar with that symbol. 24.220.188.43 (talk) 13:28, 15 April 2011 (UTC)

I repeat (see several lines above): "I never saw such notation ${\nrightarrow }\ \forall ;$ is it a good idea?" I suspect it is someone's neologism (neo-notation :-) ). --Boris Tsirelson (talk) 15:08, 15 April 2011 (UTC)

Error in statements about Laws of Large Numbers

One passage reads:

"Let {Y_n} be independent, identically distributed random variables with means zero and unit variances. Let S_n = Y₁ + … + Y_n. . . . . . . There are two versions of the law of large numbers — the weak and the strong — and they both claim that the sums Sn, scaled by n⁻¹, converge to zero, respectively in probability and almost surely:

${\frac {S_{n}}{n}}\ {\xrightarrow {p}}\ 0,\qquad {\frac {S_{n}}{n}}\ {\xrightarrow {a.s.}}0,\qquad {\text{as}}\ \ n\to \infty .$"

But, these statements about the two kinds of limiting behavior of S_n/n ought to be saying that as n →∞, S_n/n → 1 (not 0).

Maybe someone knowledgeable on the subject can fix this.Daqu (talk) 16:34, 11 February 2013 (UTC)

Sorry, I did not understand why do you believe they tend to 1. They tend to the expectation of $Y_{1},$ assumed to be 0. Boris Tsirelson (talk) 17:10, 11 February 2013 (UTC)

The reason I thought they tend to 1 was because my brain wasn't working right. Of course they should tend to the expectation of any of the i.i.d. Y_i's. My apologies for my stupid error.Daqu (talk) 17:51, 11 February 2013 (UTC)

Another completely erroneous statement

At the end of the Discussion section, this statement appears:

"Thus, although the quantity $S_{n}/{\sqrt {n\log \log n}}$ is less than any predefined ε > 0 with probability approaching one, that quantity will nevertheless be dropping out of that interval infinitely often, and in fact will be visiting the neighborhoods of any point in the interval (0,√2) almost surely."

1) Clearly the first part of this sentence (before the comma) cannot be true, since for any positive integer n ≥ 2, the quantity in question is positive. It appears that some condition on the index n is omitted here.

2) The part after the comma makes no sense at all, because "that quantity will nevertheless be dropping out of that interval " is meaningless gobbledygook. I have no idea which interval is meant here, and dropping out of an interval makes no mathematical sense whatsoever.

3) Finally, will be dropping out uses the future progressive tense -- a completely inappropriate tense for a mathematical statement.

Maybe someone who is knowledgeable on the subject and who can explain things clearly can fix this.Daqu (talk) 17:01, 11 February 2013 (UTC)

Well, some condition on the index n is indeed omitted there, but is clear from the context (see the formula): n tends to infinity, of course. Accordingly, "will be dropping out" means "as n tends to infinity" (which is treated as running in time, which is not rigorous but rather usual). "Which interval?" The interval $(-\epsilon ,\epsilon ),$ according to the inequality mentioned before. Boris Tsirelson (talk) 17:17, 11 February 2013 (UTC)

Okay -- fine, I'm glad that whoever wrote this had some sensible meaning in mind.

But because this is intended for readers who are not necessarily familiar with the material in advance, it is much better to spell things out exactly, and not to expect people to know what the writer was thinking. There is no place for untrue statements -- regardless of which true statement was actually "meant".

If quantifiers are needed for a statement to be literally true, then they certainly need to be there.Daqu (talk) 20:43, 12 February 2013 (UTC)