If you see this, something is wrong
To get acquainted with the document, the best thing to do is to select the "Collapse all sections" item from the "View" menu. This will leave visible only the titles of the top-level sections.
Clicking on a section title toggles the visibility of the section content. If you have collapsed all of the sections, this will let you discover the document progressively, from the top-level sections to the lower-level ones.
Generally speaking, anything that is blue is clickable.
Clicking on a reference link (like an equation number, for instance) will display the reference as close as possible, without breaking the layout. Clicking on the displayed content or on the reference link hides the content. This is recursive: if the content includes a reference, clicking on it will have the same effect. These "links" are not necessarily numbers, as it is possible in LaTeX2Web to use full text for a reference.
Clicking on a bibliographical reference (i.e., a number within brackets) will display the reference.
Speech bubbles indicate a footnote. Click on the bubble to reveal the footnote (there is no page in a web document, so footnotes are placed inside the text flow). Acronyms work the same way as footnotes, except that you have the acronym instead of the speech bubble.
By default, discussions are open in a document. Click on the discussion button below to reveal the discussion thread. However, you must be registered to participate in the discussion.
If a thread has been initialized, you can reply to it. Any modification to any comment, or a reply to it, in the discussion is signified by email to the owner of the document and to the author of the comment.
The blue button below that says "table of contents" is your tool to navigate in a publication.
The left arrow brings you to the previous document in the publication, and the right one brings you to the next. Both cycle over the publication list.
The middle button that says "table of contents" reveals the publication table of contents. This table is hierarchical structured. It has sections, and sections can be collapsed or expanded. If you are a registered user, you can save the layout of the table of contents.
First published on Monday, Jun 30, 2025 and last modified on Monday, Jun 30, 2025
Department of Mathematics, University of Cergy-Pontoise, F-95000 Cergy-Pontoise, France Email
Measure theory, Haar integration
This is an introduction to measure theory, integration and function spaces, with all the needed preliminaries included, and with some applications included as well. We first discuss some basic motivations, coming from discrete probability, that we develop in detail, as a preliminary to general measure theory. Then we discuss measure theory, integration and function spaces, all developed in a standard way, and with emphasis on the explicit computation of various integrals. Finally, we come back to probability, discrete and continuous, with a more advanced discussion, of quantum flavor.
You certainly know some calculus, in one and several variables, along with some basic algebra, geometry and probability. For most questions in science and engineering, that knowledge is enough, although learning new methods and tricks, from time to time, say from older colleagues that you will work with, will always improve your science.
This being said, why not learning some new methods and tricks right away, now that you are still studying. You would perhaps say, why bothering with this, I already had enough troubles with math, and learned enough of it, as to get started in life. But let me ask you a few questions, about your current knowledge, and judge yourself:
(1) The probability for a randomly picked \( x\in\mathbb R\) to be \( x=1\) is, and you surely know that, \( P=0\) . But what does this exactly mean, mathematically speaking?
(2) Along the same lines, but more concerning, everything that happens in the real life, happens with probability \( P=0\) . And this is something that we must clarify.
(3) Even worse, in certain physics disciplines like quantum mechanics, or statistical mechanics, \( P\) is the only tool available, and \( P=0\) , your daily nightmare.
(4) Another interesting thing that you surely know is that the reals are uncountable, \( |\mathbb R|>|\mathbb N|=\infty\) . So, how many possible \( \infty\) beasts are there, in mathematics?
(5) What about arbitrarily small infinitesimals, how many \( 1/\infty\) beasts are there? And with the comment that these can be, technically speaking, very useful.
(6) Less philosophically now, you are surely a heavy user of \( dxdy=dydx\) , called Fubini theorem, for integrating 2-variable functions. But, does Fubini always work?
(7) Also, when using polar coordinates, the formula is \( dxdy=Jdrdt\) , with \( J=r\) . But no one ever fully proves this in class, so is this formula really correct, and why?
(8) You surely know about the Dirac \( \delta_x\) function from electrostatics, or other physics. But, what is the precise mathematics behind this delta function?
(9) In fact, even mathematicians can pull out tricks based on \( \delta_x\) . For instance they have a theory where the basic step function is differentiable, with derivative \( \delta_0\) .
(10) Still talking classical analysis, you might know from Fourier transform theory that functions can be of type \( L^1,L^2,L^\infty\) and so on, and clarifying all this matters.
(11) As even worse classical analysis, arbitrary functions can be non-integrable at all, at least in theory. And with these being non-negligible in quantum mechanics.
(12) In fact, talking worse things that can happen, these are wrong formulae, obtained via correct proofs. So, make sure that you know as much theory as possible.
In the hope that I convinced you that learning some more analysis would be a great investment, and the present book will be here for that. The above questions all fall under the banner “measure theory”, and we will discuss here measure theory, with an introduction to the subject, and answers, from partial to full, to most of the above questions.
This book was concieved as a complement to my calculus book [1], or to any other modern calculus book, which is typically weak on measure theory. Of course, if you are brave enough for learning full calculus directly, the old way, with measure theory included, go with some classical Cold War books, such as those of Rudin [2], [3].
Many thanks go to my students, who had to endure several beta versions of the material presented here, once for instance, long ago, with a measure theory class featuring a full proof for the Zorn Lemma. Thanks as well go to my cats, who seem to deal with everything, successfully, just by using fast binary logic. But we are not all that fast.
Cergy, June 2025
Teo Banica
Low lie
The fields of Athenry
Where once we watched
The small free birds fly
Welcome to measure theory. What we will be doing in this book, namely measure theory and function spaces, is standard advanced undergraduate mathematics, sometimes erring on the graduate side, and can be best described as being a mixture of “advanced calculus” and “rigorous calculus”. That is, you are supposed to know reasonably well basic calculus, in one and several variables, learned from one of the many calculus books available, such as those of Lax and Terrell [4], [5], or why not mine [1], and looking for a continuation of that, more advanced of course, but more rigorous too.
I can hear you thinking “why bothering with more rigorous calculus, what I want to learn is more advanced calculus, that is what looks the most useful”. Good point, and if you are really into this, I can only recommend you, as a continuation of your calculus experience, to start reading some physics. You have here the famous course of Feynman [6], [7], [8], or the lovely books of Griffiths [9], [10], [11], or those of Weinberg [12], [13], [14], if you already know some physics. So go with them, for advanced calculus, physicists know best, that is their job, to come up via advanced calculus to all sorts of amazing formulae, and you will certainly learn many interesting things from them.
This being said, even if you are genuinely interested in physics, a bit of rigor, that you will not necessarily learn from Feynman, Griffiths or Weinberg, can help. To make my point, let me formulate the following question, which is something very natural:
Question 1
The probability for a randomly picked \( x\in\mathbb R\) to be \( x=1\) is obviously \( P=0\) . But what does this mean, mathematically speaking?
And I hope that you agree with me, this is a very good question, for anyone having a bit of calculus background, and looking to learn more, regardless of the precise goals. For instance, don’t expect to understand any sort of truly advanced physics, which often relies on probabilistic techniques, without having such things perfectly understood.
We will discuss this question several times in this book, first at the end of the present chapter, with a quick but rather advanced explanation, and then on several occasions afterwards, once our knowledge in measure theory and probability improves.
Getting started now, let us first talk about numbers. The commonly accepted fact, by most humans, is that the set \( \mathbb N=\{0,1,2,3,…\}\) was invented by God. According however to certain mathematicians, God only invented the empty set \( \emptyset\) , and then \( \mathbb N\) naturally came afterwards, somehow by inventing itself, according to the following scheme:
Along the same lines, also alternatively, according to certain physicists, God only invented the Big Bang, and everything including \( \mathbb N\) naturally came afterwards. But shall we trust all this modern science, better stick with good old traditional religion.
Once you have \( \mathbb N\) , solving \( a+b=c\) naturally leads you to the set of all integers \( \mathbb Z\) . Then, once you have \( \mathbb Z\) , solving \( ab=c\) naturally leads you to the set of all rationals \( \mathbb Q\) . So, this will be our starting point, the set of rationals \( \mathbb Q\) , defined as follows:
Definition 1
The rational numbers are the quotients \( r=a/b\) , with \( a,b\in\mathbb Z\) and \( b\neq0\) , identified according to the usual rule for quotients, namely:
These quotients add, multiply and invert according to the following formulae:
We denote the set of rational numbers by \( \mathbb Q\) , standing for “quotients”.
As a comment here, in the hope that you love fractions. This is what makes the difference between science and humanities, in science we all love fractions, and are particularly excited whenever we must do any computations with them, useful or unuseful.
In more advanced mathematical terms, the above operations, namely sum, product and inversion, tell us that \( \mathbb Q\) is a field, in the following sense:
Definition 2
A field is a set \( F\) with a sum operation \( +\) and a product operation \( \times\) , subject to the following conditions:
Apparently, what we did so far, with our philosophical discussion regarding creation, \( \diamondsuit\to\mathbb N\to\mathbb Z\to\mathbb Q\) , was to construct the simplest possible field, \( \mathbb Q\) . However, this is not exactly true, because by a strange twist of fate, the numbers \( 0,1\) , whose presence in a field is mandatory, \( 0,1\in F\) , can form themselves a field, with addition as follows:
To be more precise, according to our field axioms, we certainly must have:
Thus, everything regarding the addition and multiplication of \( 0,1\) is uniquely determined, except for the value of \( 1+1\) . And here, you would say that we should normally set \( 1+1=2\) , with \( 2\neq0\) being a new field element, but the point is that \( 1+1=0\) is something natural too, this being the addition modulo 2. And, what we get is a field:
Let us summarize this finding, along with a bit more, obtained by suitably replacing our 2, used for addition, with an arbitrary prime number \( p\) , as follows:
Theorem 1
The following happen:
Proof
All this is basic number theory, the idea being as follows:
(1) This is clear, because \( 1+…+1\neq0\) tells us that we have an embedding \( \mathbb N\subset F\) , and then by taking inverses with respect to \( +\) and \( \times\) we obtain \( \mathbb Q\subset F\) .
(2) Again, this is clear, because assuming \( 1+…+1=0\) , with \( p=ab\) terms, chosen minimal, we would have a formula as follows, which is a contradiction:
(3) This follows a bit as in (1), with the copy \( \mathbb F_p\subset F\) consisting by definition of the various sums of type \( 1+…+1\) , which must cycle modulo \( p\) , as shown by (2).
Getting back now to our philosophical discussion regarding creation, what we have in Theorem 1.4 is not exactly good news, suggesting that, on purely mathematical grounds, there is a certain rivalry between \( \mathbb Q\) and \( \mathbb F_p\) , as being the simplest field. So, which of them shall we study, as being created first? Not an easy question, and as answer, we have:
Answer 1
Ignoring what pure mathematics might say, and trusting instead physics and chemistry, we will choose to trust in \( \mathbb Q\) , as being the simplest field.
In short, welcome to analysis, and with this being something quite natural, analysis being the main topic of the present book. Moving ahead now, many things can be done with \( \mathbb Q\) , but getting straight to the point, one thing that fails is solving \( x^2=2\) :
Theorem 2
The field \( \mathbb Q\) does not contain a square root of \( 2\) :
In fact, among integers, only the squares, \( n=m^2\) with \( m\in\mathbb N\) , have square roots in \( \mathbb Q\) .
Proof
This is something very standard, the idea being as follows:
(1) In what regards \( \sqrt{2}\) , assuming that \( r=a/b\) with \( a,b\in\mathbb N\) prime to each other satisfies \( r^2=2\) , we have \( a^2=2b^2\) , and so \( a\in2\mathbb N\) . But then by using again \( a^2=2b^2\) we obtain \( b\in2\mathbb N\) as well, which contradicts our assumption \( (a,b)=1\) .
(2) Along the same lines, any prime number \( p\in\mathbb N\) has the property \( \sqrt{p}\notin\mathbb Q\) , with the proof here being as the above one for \( p=2\) , by congruence and contradiction.
(3) More generally, our claim is that any \( n\in\mathbb N\) which is not a square has the property \( \sqrt{n}\notin\mathbb Q\) . Indeed, we can argue here that our number decomposes as \( n=p_1^{a_1}… p_k^{a_k}\) , with \( p_1,…,p_k\) distinct primes, and our assumption that \( n\) is not a square tells us that one of the exponents \( a_1,…,a_k\in\mathbb N\) must be odd. Moreover, by extracting all the obvious squares from \( n\) , we can in fact assume \( a_1=…=a_k=1\) . But with this done, we can set \( p=p_1\) , and the congruence argument from (2) applies, and gives \( \sqrt{n}\notin\mathbb Q\) , as desired.
In short, in order to advance with our mathematics, we are in need to introduce the field of real numbers \( \mathbb R\) . You would probably say that this is very easy, via decimal writing, like everyone does, but before doing that, let me ask you a few questions:
(1) Honestly, do you really like the addition of real numbers, using the decimal form? Let us take, as example, the following computation:
This computation can surely be done, but, annoyingly, it must be done from right to left, instead of left to right, as we would prefer. I mean, personally I would be most interested in knowing first what happens at left, if the integer part is 39 or 40, but go do all the computation, starting from the right, in order to figure out that. In short, my feeling is that this addition algorithm, while certainly good, is a bit deceiving.
(2) What about multiplication. Here things become even more complicated, imagine for instance that Mars attacks, with \( \delta\) -rays, which are something unknown to us, and \( 100,000\) stronger than \( \gamma\) -rays, and which have paralyzed all our electronics, and that in order to protect Planet Earth, you must do the following multiplication by hand:
This does not look very inviting, doesn’t it. In short, as before with the addition, there is a bit of a bug with all this, the algorithm being too complicated.
(3) Getting now to the problem that we were interested in, namely extracting the square root of 2, here the algorithm is as follows, not very inviting either:
In short, quite concerning all this, and don’t count on such things, mathematics of the decimal form, if Mars attacks. Let us record these findings as follows:
Fact 1
The real numbers \( x\in\mathbb R\) can be certainly introduced via their decimal form, but with this, the field structure of \( \mathbb R\) remains something quite unclear.
Well, it looks like we are a bit stuck. Fortunately, there is a clever solution to this, due to Dedekind. His definition for the real numbers is as follows:
Definition 3
The real numbers \( x\in\mathbb R\) are formal cuts in the set of rationals,
with such a cut being by definition subject to the following conditions:
These numbers add and multiply by adding and multiplying the corresponding cuts.
This might look quite original, but believe me, there is some genius behind this definition. As a first observation, we have an inclusion \( \mathbb Q\subset\mathbb R\) , obtained by identifying each rational number \( r\in\mathbb Q\) with the obvious cut that it produces, namely:
As a second observation, the addition and multiplication of real numbers, obtained by adding and multiplying the corresponding cuts, in the obvious way, is something very simple. To be more precise, in what regards the addition, the formula is as follows:
As for the multiplication, the formula here is similar, namely \( A_{xy}=A_xA_y\) , up to some issues with positives and negatives, which are quite easy to untangle, and with this being a good exercise. We can also talk about order between real numbers, as follows:
But let us perhaps leave more abstractions for later, and go back to more concrete things. As a first success of our theory, we can formulate the following theorem:
Theorem 3
The equation \( x^2=2\) has two solutions over the real numbers, namely the positive solution, denoted \( \sqrt{2}\) , and its negative counterpart, which is \( -\sqrt{2}\) .
Proof
By using \( x\to-x\) , it is enough to prove that \( x^2=2\) has exactly one positive solution \( \sqrt{2}\) . But this is clear, because \( \sqrt{2}\) can only come from the following cut:
Thus, we are led to the conclusion in the statement.
More generally, the same method works in order to extract the square root \( \sqrt{r}\) of any number \( r\in\mathbb Q_+\) , or even of any number \( r\in\mathbb R_+\) , and we have the following result:
Theorem 4
The solutions of \( ax^2+bx+c=0\) with \( a,b,c\in\mathbb R\) are
provided that \( b^2-4ac\geq0\) . In the case \( b^2-4ac<0\) , there are no solutions.
Proof
We can write our equation in the following way:
Thus, we are led to the conclusion in the statement.
Summarizing, we have a nice abstract definition for the real numbers, that we can certainly do some mathematics with. As a first general result now, which is something very useful, and puts us back into real life, and science and engineering, we have:
Theorem 5
The real numbers \( x\in\mathbb R\) can be written in decimal form,
with \( a_i,b_i\in\{0,1,…,9\}\) , with the convention \( … b999…=…(b+1)000…\)
Proof
This is something non-trivial, even for the rationals \( x\in\mathbb Q\) themselves, which require some work in order to be put in decimal form, the idea being as follows:
(1) First of all, our precise claim is that any \( x\in\mathbb R\) can be written in the form in the statement, with the integer \( \pm a_1… a_n\) and then each of the digits \( b_1,b_2,b_3,…\) providing the best approximation of \( x\) , at that stage of the approximation.
(2) Moreover, we have a second claim as well, namely that any expression of type \( x=\pm a_1… a_n.b_1b_2b_3……\) corresponds to a real number \( x\in\mathbb R\) , and that with the convention \( … b999…=…(b+1)000…\,\) , the correspondence is bijective.
(3) In order to prove now these two assertions, our first claim is that we can restrict the attention to the case \( x\in[0,1)\) , and with this meaning of course \( 0\leq x<1\) , with respect to the order relation for the reals discussed in the above.
(4) Getting started now, let \( x\in\mathbb R\) , coming from a cut \( \mathbb Q=A_x\sqcup B_x\) . Since the set \( A_x\cap\mathbb Z\) consists of integers, and is bounded from above by any element \( q\in B_x\) of your choice, this set has a maximal element, that we can denote \( [x]\) :
It follows from definitions that \( [x]\) has the usual properties of the integer part, namely:
Thus we have \( x=[x]+y\) with \( [x]\in\mathbb Z\) and \( y\in[0,1)\) , and getting back now to what we want to prove, namely (1,2) above, it is clear that it is enough to prove these assertions for the remainder \( y\in[0,1)\) . Thus, we have proved (3), and we can assume \( x\in[0,1)\) .
(5) So, assume \( x\in[0,1)\) . We are first looking for a best approximation from below of type \( 0.b_1\) , with \( b_1\in\{0,…,9\}\) , and it is clear that such an approximation exists, simply by comparing \( x\) with the numbers \( 0.0,0.1,…,0.9\) . Thus, we have our first digit \( b_1\) , and then we can construct the second digit \( b_2\) as well, by comparing \( x\) with the numbers \( 0.b_10,0.b_11,…,0.b_19\) . And so on, which finishes the proof of our claim (1).
(6) In order to prove now the remaining claim (2), let us restrict again the attention, as explained in (4), to the case \( x\in[0,1)\) . First, it is clear that any expression of type \( x=0.b_1b_2b_3…\) defines a real number \( x\in[0,1]\) , simply by declaring that the corresponding cut \( \mathbb Q=A_x\sqcup B_x\) comes from the following set, and its complement:
(7) Thus, we have our correspondence between real numbers as cuts, and real numbers as decimal expressions, and we are left with the question of investigating the bijectivity of this correspondence. But here, the only bug that happens is that numbers of type \( x=… b999…\) , which produce reals \( x\in\mathbb R\) via (6), do not come from reals \( x\in\mathbb R\) via (5). So, in order to finish our proof, we must investigate such numbers.
(8) So, consider an expression of type \( … b999…\) Going back to the construction in (6), we are led to the conclusion that we have the following equality:
Thus, at the level of the real numbers defined as cuts, we have:
But this solves our problem, because by identifying \( … b999…=…(b+1)000…\) the bijectivity issue of our correspondence is fixed, and we are done.
The above theorem was of course quite difficult, but this is how things are. Alternatively, we have the following definition for the real numbers:
Theorem 6
The field of real numbers \( \mathbb R\) can be defined as well as the completion of \( \mathbb Q\) with respect to the usual distance on the rationals, namely
and with the operations on \( \mathbb R\) coming from those on \( \mathbb Q\) , via Cauchy sequences.
Proof
There are several things going on here, the idea being as follows:
(1) Getting back to Definition 1.2, we know from there what the rational numbers are. But, as a continuation of the material there, we can talk about the distance between such rational numbers, as being given by the formula in the statement, namely:
(2) Very good, so let us get now into Cauchy sequences. We say that a sequence of rational numbers \( \{r_n\}\subset\mathbb Q\) is Cauchy when the following condition is satisfied:
Here of course \( \varepsilon\in\mathbb Q\) , because we do not know yet what the real numbers are.
(3) With this notion in hand, the idea will be to define the reals \( x\in\mathbb R\) as being the limits of the Cauchy sequences \( \{r_n\}\subset\mathbb Q\) . But since these limits are not known yet to exist to us, precisely because they are real, we must employ a trick. So, let us define instead the reals \( x\in\mathbb R\) as being the Cauchy sequences \( \{r_n\}\subset\mathbb Q\) themselves.
(4) The question is now, will this work. As a first observation, we have an inclusion \( \mathbb Q\subset\mathbb R\) , obtained by identifying each rational \( r\in\mathbb Q\) with the constant sequence \( r_n=r\) . Also, we can sum and multiply our real numbers in the obvious way, namely:
We can also talk about the order between such reals, as follows:
Finally, we can also solve equations of type \( x^2=2\) over our real numbers, say by using our previous work on the decimal writing, which shows in particular that \( \sqrt{2}\) can be approximated by rationals \( r_n\in\mathbb Q\) , by truncating the decimal writing.
(5) However, there is still a bug with our theory, because there are obviously more Cauchy sequences of rationals, than real numbers. In order to fix this, let us go back to the end of step (3) above, and make the following convention:
(6) But, with this convention made, we have our theory. Indeed, the considerations in (4) apply again, with this change, and we obtain an ordered field \( \mathbb R\) , containing \( \mathbb Q\) . Moreover, the equivalence with the Dedekind cuts is something which is easy to establish, and we will leave this as an instructive exercise, and this gives all the results.
Very nice all this, so have have two equivalent definitions for the real numbers. Finally, getting back to the decimal writing approach, that can be recycled too, with some analysis know-how, and we have a third possible definition for the real numbers, as follows:
Theorem 7
The real numbers \( \mathbb R\) can be defined as well via the decimal form
with \( a_i\in\{0,1,…,9\}\) , with the usual convention for such numbers, namely
and with the sum and multiplication coming by writing such numbers as
and then summing and multiplying, in the obvious way.
Proof
This is something which looks quite intuitive, but which in practice, and we insist here, is not exactly beginner level, the idea with this being as follows:
(1) Let us first forget about the precise decimal writing in the statement, and define the real numbers \( x\in\mathbb R\) as being formal sums as follows, with the sum being over integers \( k\in\mathbb Z\) assumed to be greater than a certain integer, \( k\geq k_0\) :
(2) Now by truncating, we can see that what we have here are certain Cauchy sequences of rationals, and with a bit more work, we conclude that the \( \mathbb R\) that we constructed is precisely the \( \mathbb R\) that we constructed in Theorem 1.12. Thus, we get the result.
(3) Alternatively, by getting back to Theorem 1.11 and its proof, we can argue, based on that, that the \( \mathbb R\) that we constructed coincides with the old \( \mathbb R\) from Definition 1.8, the one constructed via Dedekind cuts, and this gives again all the assertions.
Is the field \( \mathbb R\) that we constructed good enough, for doing analysis? Certainly yes for many questions, as we both know, but not so quick with conclusions, because we have:
Fact 2
It would be desirable to have complex numbers, \( z=a+ib\) with \( a,b\in\mathbb R\) and with \( i^2=-1\) , as for the following dreams to become true:
In short, and you get the point I hope, multivariable calculus begs for complex numbers. So, let us introduce the complex numbers \( \mathbb C\) . Many interesting things can be said here, and you certainly know all of them, with their summary being as follows:
Theorem 8
The complex numbers, \( z=a+ib\) with \( a,b\in\mathbb R\) and with \( i\) being a formal number satisying \( i^2=-1\) , form a field \( \mathbb C\) . Moreover:
Proof
We have a field, with \( z^{-1}=(a-ib)/(a^2+b^2)\) , and regarding the rest:
(1) This is clear.
(2) Again, this is clear.
(3) Again, this is clear. Observe also that we have \( r^2=z\bar{z}\) , with \( \bar{z}=a-ib\) .
(4) We need here the formulae for the sines and cosines of sums, which are as follows, coming from some trigonometry, done the old way, with triangles in the plane:
Indeed, with these formulae in hand, we have the following computation, as desired:
(5) In order to prove \( e^{it}=\cos t+i\sin t\) , consider the following function \( f:\mathbb R\to\mathbb C\) :
By using \( \sin'=\cos\) and \( \cos'=-\sin\) , coming from the formulae in (4), we have:
We conclude that \( f:\mathbb R\to\mathbb C\) is constant, equal to \( f(0)=1\) , as desired.
(6) This is clear from (5), with \( z=w^k\) , with \( w=e^{2\pi i/N}\) and \( k=0,1,…,N-1\) .
(7) This follows in the usual way, with \( \sqrt{re^{it}}=\pm\sqrt{r}e^{it/2}\) at the end, using (5).
Quite remarkably, we have in fact the following result, generalizing what we know in degree 2, and telling us that with \( \mathbb C\) we are safe, no need to look for more:
Theorem 9
Any polynomial \( P\in\mathbb C[X]\) decomposes as
with \( c\in\mathbb C\) and with \( a_1,…,a_N\in\mathbb C\) .
Proof
The problem is that of proving that our polynomial has at least one root, because afterwards we can proceed by recurrence. We prove this by contradiction. So, assume that \( P\) has no roots, and pick a number \( z\in\mathbb C\) where \( |P|\) attains its minimum:
Since \( Q(t)=P(z+t)-P(z)\) is a polynomial which vanishes at \( t=0\) , this polynomial must be of the form \( ct^k\) + higher terms, with \( c\neq0\) , and with \( k\geq1\) being an integer. We obtain from this that, with \( t\in\mathbb C\) small, we have the following estimate:
Now let us write \( t=rw\) , with \( r>0\) small, and with \( |w|=1\) . Our estimate becomes:
Now recall that we have assumed \( P(z)\neq0\) . We can therefore choose \( w\in\mathbb T\) such that \( cw^k\) points in the opposite direction to that of \( P(z)\) , and we obtain in this way:
Now by choosing \( r>0\) small enough, as for the error in the first estimate to be small, and overcame by the negative quantity \( -|c|r^k\) , we obtain from this:
But this contradicts our definition of \( z\in\mathbb C\) , as a point where \( |P|\) attains its minimum. Thus \( P\) has a root, and by recurrence it has \( N\) roots, as stated.
As a concrete question, let us try to solve a degree 3 equation, \( aX^3+bX^2+cX+d=0\) . By linear transformations we can always assume \( a=1,b=0\) , and then it is convenient to write \( c=3p,d=2q\) . Thus, our equation becomes \( x^3+3px+2q=0\) , and regarding such equations, we have the following famous result, due to Cardano:
Theorem 10
For a normalized degree \( 3\) equation, namely
the discriminant is \( \Delta=-108(p^3+q^2)\) . Assuming \( p,q\in\mathbb R\) and \( \Delta<0\) , the numbers
with \( w=1,e^{2\pi i/3},e^{4\pi i/3}\) are the solutions of our equation.
Proof
There are several things going on here, as follows:
(1) The formula of \( \Delta\) comes the theory of the discriminant. If you know that, good, and if you don’t, no worries, because we will only need \( p^3+q^2>0\) , in what follows.
(2) With \( z\) as in the statement, by using \( (a+b)^3=a^3+b^3+3ab(a+b)\) , we have:
Thus, we are led to the conclusion in the statement.
In degree 4 now, we have the following result, which is famous as well:
Theorem 11
The roots of a normalized degree \( 4\) equation, written as
are as follows, with \( y\) satisfying the equation \( (y^2-3r)(y-3p)=2q^2\) ,
and with \( y\) being computable via the Cardano formula.
Proof
This is something quite tricky, the idea being as follows:
(1) To start with, let us write our equation in the following form:
Now assume that we have a number \( y\) satisfying the following equation:
With this magic number \( y\) in hand, our equation takes the following form:
(2) Which looks very good, leading us to the following degree 2 equations:
Now let us write these two degree 2 equations in standard form, as follows:
(3) Regarding the first equation, the solutions there are as follows:
As for the second equation, the solutions there are as follows:
(4) Now by cutting a \( \sqrt{2}\) factor from everything, this gives the formulae in the statement. As for the last claim, regarding the nature of \( y\) , this comes from Cardano.
We still have to compute the number \( y\) appearing in the above via Cardano, and the result here, adding to what we already have in Theorem 1.18, is as follows:
Theorem 12 (continuation)
The value of \( y\) in the previous theorem is
where the number \( t\) is given by the formula
with \( a=p^2+r\) and \( b=2p^2-3pr+q^2\) .
Proof
The legend has it that this is what comes from Cardano, but depressing and normalizing and solving \( (y^2-3r)(y-3p)=2q^2\) makes it for too many operations, so the most pragmatic way is to simply check this equation. With \( y\) as above, we have:
With this in hand, we have the following computation:
Now by using the formula of \( t\) in the statement, this gives:
Thus, we are led to the conclusion in the statement.
In degree 5 and more, things become fairly complicated, and we have:
Theorem 13
There is no general formula for the roots of polynomials of degree \( N=5\) and higher, with the reason for this, coming from Galois theory, being that the group \( S_5\) is not solvable. The simplest numeric example is \( P=X^5-X-1\) .
Proof
This is something quite tricky, the idea being as follows:
(1) Given a field \( F\) , assume that the roots of \( P\in F[X]\) can be computed by using iterated roots, a bit as for the degree 2 equation, or the degree 3 and 4 equations. Then, algebrically speaking, this gives rise to a tower of fields as folows, with \( F_0=F\) , and each \( F_{i+1}\) being obtained from \( F_i\) by adding a root, \( F_{i+1}=F_i(x_i)\) , with \( x_i^{n_i}\in F_i\) :
(2) In order for Galois theory to apply to this situation, we must make all the extensions normal, which amounts in replacing each \( F_{i+1}=F_i(x_i)\) by its extension \( K_i(x_i)\) , with \( K_i\) extending \( F_i\) by adding a \( n_i\) -th root of unity. Thus, with this replacement, we can assume that the tower in (1) in normal, meaning that all Galois groups are cyclic.
(3) Now by Galois theory, at the level of the corresponding Galois groups we obtain a tower of groups as follows as follows, which is a resolution of the last group \( G_k\) , the Galois group of \( P\) , in the sense of group theory, in the sense that all quotients are cyclic:
As a conclusion, Galois theory tells us that if the roots of a polynomial \( P\in F[X]\) can be computed by using iterated roots, then its Galois group \( G=G_k\) must be solvable.
(4) In the generic case, the conclusion is that Galois theory tells us that, in order for all polynomials of degree 5 to be solvable, via square roots, the group \( S_5\) , which appears there as Galois group, must be solvable, in the sense of group theory. But this is wrong, because the alternating subgroup \( A_5\subset S_5\) is simple, and therefore not solvable.
(5) Finally, regarding the polynomial \( P=X^5-X-1\) , some elementary computations here, based on arithmetic over \( \mathbb F_2,\mathbb F_3\) , and involving various cycles of length \( 2,3,5\) , show that its Galois group is \( S_5\) . Thus, we have our counterexample.
(6) Finally, let us mention that all this shows as well that a random polynomial of degree 5 or higher is not solvable by square roots, and with this being an elementary consequence of the main result from (4), via some standard analysis arguments.
There is a lot of further interesting theory that can be developed here, following Galois and others. For more on all this, we recommend any number theory book.
All that we have been talking about so far, while certainly beautiful and potentially useful, was a bit abstract. So, what about true applications, say for the scientist or engineer using a computer for everything that is heavy mathematics?
So, let us talk now about integrating functions, which is a favorite pastime of our friends, the computers. As definition for the integral, which is something simple and straightforward, that our computer friend will certainly appreciate, we have:
Theorem 14
We have the Riemann integration formula,
which can serve as a definition for the integral, for the continuous functions.
Proof
This is standard, by drawing rectangles. We have indeed the following formula, which can stand as a definition for the signed area below the graph of \( f\) :
Thus, we are led to the formula in the statement.
Observe that the above formula suggests that \( \int_a^bf(x)dx\) is the length of the interval \( [a,b]\) , namely \( b-a\) , times the average of \( f\) on the interval \( [a,b]\) . Thinking a bit, this is indeed something true, with no need for Riemann sums, coming directly from definitions, because area means side times average height. Thus, we can also formulate:
Theorem 15
The integral of a function \( f:[a,b]\to\mathbb R\) is given by
where \( A(f)\) is the average of \( f\) over the interval \( [a,b]\) .
Proof
As explained above, this is clear from definitions, via some geometric thinking. Alternatively, this is something which certainly comes from Theorem 1.21.
Going ahead with more interpretations of the integral, we have:
Theorem 16
We have the Monte Carlo integration formula,
with \( x_1,…,x_N\in[a,b]\) being random.
Proof
We recall from Theorem 1.21 that the idea is to use a formula as follows, with the points \( x_1,…,x_N\in[a,b]\) being uniformly distributed:
But this works as well when the points \( x_1,…,x_N\in[a,b]\) are randomly distributed, for somewhat obvious reasons, and this gives the result.
Observe that the Monte Carlo integration works better than Riemann integration, for instance when trying to improve the estimate, via \( N\to N+1\) . Indeed, in the context of Riemann integration, assume that we managed to find an estimate as follows, which in practice requires computing \( N\) values of our function \( f\) , and making their average:
In order to improve this estimate, any extra computed value of our function \( f(y)\) will be unuseful. For improving our formula, what we need are \( N\) extra values of our function, \( f(y_1),…,f(y_N)\) , with the points \( y_1,…,y_N\) being precisely the midpoints of the previous division of \( [a,b]\) , so that we can write an improvement of our formula, as follows:
With Monte Carlo, things are far more flexible. Assume indeed that we managed to find an estimate as follows, which again requires computing \( N\) values of our function:
Now if we want to improve this, any extra computed value of our function \( f(y)\) will be helpful, because we can set \( x_{n+1}=y\) , and improve our estimate as follows:
And isn’t this potentially useful, and powerful, when thinking at practically computing integrals, either by hand, or by using a computer. Let us record this finding as follows:
Conclusion 1
Monte Carlo integration works better than Riemann integration, when it comes to computing as usual, by estimating, and refining the estimate.
As another interesting feature of Monte Carlo integration, this works better than Riemann integration for functions having various symmetries, because Riemann integration can get “fooled” by these symmetries, while Monte Carlo remains strong.
As an example for this phenomenon, chosen to be quite drastic, let us attempt to integrate, via both Riemann and Monte Carlo, the following function \( f:[0,\pi]\to\mathbb R\) :
The first few Riemann sums for this function are then as follows:
Based on this evidence, we will conclude, obviously, that we have:
With Monte Carlo, however, such things cannot happen. Indeed, since there are finitely many points \( x\in[0,\pi]\) having the property \( \sin(120 x)=0\) , a random point \( x\in[0,\pi]\) will have the property \( |\sin(120 x)|>0\) , so Monte Carlo will give, at any \( N\in\mathbb N\) :
Again, this is something interesting, when practically computing integrals, either by hand, or by using a computer. So, let us record, as a complement to Conclusion 1.24:
Conclusion 2
Monte Carlo integration is smarter than Riemann integration, because the symmetries of the function can fool Riemann, but not Monte Carlo.
All this is good to know, and in practice now, we are led into the following question, coming from these Monte Carlo considerations, and from a myriad other reasons:
Question 2
How to produce random numbers?
And good question this is. The first thought goes into producing first random digits \( x\in\{0,1,…,9\}\) , that we can combine afterwards into random real numbers, by using the decimal form. But, how to produce these random digits \( x\in\{0,1,…,9\}\) ?
You would say, easy, just pick the digits of some reasonably complicated real number \( r\in\mathbb R\) . In practice, however, this is not easy at all, the reasons being as follows:
(1) To start with, any rational number \( r\in\mathbb Q\) , be that a very complicated one, will not do, because the decimal expansion of rationals is periodic.
(2) Then, we can try beasts as \( \sqrt{2}\) , or more generally, roots of polynomials \( P\in\mathbb Q[X]\) . But here, we potentially run again into troubles, because, say in the context of a Monte Carlo application, if our function \( f\) to be integrated happens to be related to \( P\) , using random numbers coming from solutions of \( P=0\) might flaw our computation.
(3) So, we are seemingly led into transcedental numbers, those which are not roots of polynomials \( P\in\mathbb Q[X]\) . But here, again, lots of problems waiting for us. For instance being transcedental is certainly not enough, because there are such numbers having only \( 0,1\) as digits. Also, the simplest such numbers are \( e,\pi\) , and we don’t want to use these for integrating our functions \( f\) , which usually have something with do with \( e,\pi\) .
So, what to do? Not clear. Question 1.26 is actually one of the points where modern mathematics fails, and for telling you everything, what we scientists do is to use for our simulations whatever random numbers are provided by the software. And, in case the output starts going completely wrong, pick random numbers from another batch.
We have now a quite good knowledge of numbers, and randomness. So, let us go back to the question that we raised in the beginning of this chapter, namely:
Question 3
The probability for a randomly picked \( x\in\mathbb R\) to be \( x=1\) is obviously \( P=0\) . But what does this mean, mathematically speaking?
And surprise here, this is something quite tricky. Indeed, the uniform measure on \( \mathbb R\) , that you first think of, when it comes to randomly pick numbers \( x\in\mathbb R\) , has mass \( \infty\) , so is not a probability measure. Damn. So our formula in Question 1.27, namely \( P(x=1)=0\) , while being something undoubtedly true, and it would be foolish to deny this, and think otherwise, looks more like a “social science fact”, coming from our human methods of picking random numbers \( x\in\mathbb R\) , than something mathematical.
So, in order to answer our question, here we are into some deep thinking, both in social science, and in abstract mathematics. And the situation here is as follows:
(1) Humanities first. If you ask people on the street for random numbers \( x\in\mathbb R\) , it is most likely that most of them will choose small numbers, say \( x\in[-1000,1000]\) , and basically no one will think at choices of type \( x=872,563,267,127,167,176\) .
(2) Of course, there is a similar phenomenon involving decimals too, but let us ignore this. Our finding is that \( P\) should come from some sort of bell-shaped curve around the origin 0, and that can only be, say by the Central Limit Theorem, a normal law.
(3) Regarding now the mathematics, this is quite clear. Once we have our normal law, the probability \( P(x=1)\) that we want to compute is obtained by integrating the density of this law between 1 and 1. And we obtain of course 0, no matter what \( t>0\) is.
As a conclusion to this, mystery solved, our answer to Question 1.27 being:
Answer 2
The probability for a randomly picked \( x\in\mathbb R\) to be \( x=1\) is
which means, by computing the integral, \( P=0\) .
Well done, but that wasn’t trivial, wasn’t it. Which leads us now into a bit of a philosophy scare, so if dealing with such a simple question like Question 1.27 was non-trivial, what about more complicated questions of the same type, based on the obvious fact that “everything that happens in the real life, happens with probability \( P=0\) ”.
Very concerning all this, hope you agree with me. In short, we have to review now all our basic probability knowledge, by putting that on a fully rigorous basis, in order to never ever get annoyed by such \( P=0\) things, and have these fully understood, in a fully rigorous way. And good news, this is what we will be doing in this book, by systematically developing measure theory. So, let us record this finding, as follows:
Fact 3
Everything that happens in the real life, happens with probability \( P=0\) , and in order to clarify this, we must develop measure theory.
Which is very nice, at least we know one thing, and have now a plan.
Before developing measure theory, however, let us further explore Question 1.27, with the aim of bringing even more mess to what we have, namely Answer 1.28. Indeed, there are in fact some bugs with that Answer 1.28, the rationale being as follows:
(1) If you ask people on the street for random numbers \( x\in\mathbb R\) , frankly, you will certainly have some answering \( x=1\) . Thus, we have in fact \( P(x=1)>0\) .
(2) Looking back at our reasoning leading to Answer 1.28, the modeling error there was by ignoring what happens to the decimals. People don’t like them, for sure.
(3) Thus, by simplifying, we must rather understand what happens when picking random integers \( x\in\mathbb Z\) . Or even better, by picking numbers \( x\in\mathbb N\) .
In view of this, let us formulate a more realistic version of Question 1.27, as follows:
Question 4
The probability for a randomly picked \( x\in\mathbb N\) to be \( x=1\) is obviously \( P>0\) . But what does this mean, mathematically speaking, and what exactly is \( P\) ?
So, here we go again with modeling questions, belonging to both social science, and to mathematics. To be more precise, from a pure mathematics viewpoint, the first measure that comes to mind, namely the uniform measure on \( \mathbb N\) , has mass \( \infty\) , and so is not a probability measure. Thus, wrong way, and we must throw in a bit of social science.
But social science tells us that, exactly as before in the continuous setting, people prefer small numbers, say \( x\in[0,1000]\) , to beasts such as \( x=872,563,267,127,167,176\) . Thus, we can only expect to have a density decreasing from 0 to \( \infty\) , and a bit of thinking, say based on the Poisson Limit Theorem, tells us that what we need is a Poisson law.
Time now to answer Question 1.30. Based on the above, we can formulate:
Answer 3
The probability for a randomly picked \( x\in\mathbb N\) to be \( x=1\) is
which means, by computing the integral, \( P=t/e^t\) .
Of course, this was something a bit mathematical, with the \( P>0\) phenomenon being definitely explained, but with the actual figure \( P=t/e^t\) still depending on a parameter \( t>0\) , whose realistic value remains to be computed, based on social science considerations. But we will not get here into this latter question, which looks quite difficult.
As a conclusion to this, let us complement Fact 1.29 with:
Fact 4
Some things happen in the real life with probability \( P>0\) , but even these need, mathematically speaking, measure theory, for dealing with the Dirac masses.
Long story short, no matter what we want to do advanced probability, or even basic probability, to be fully honest, we must develop measure theory first.
This was a quite crowded preliminary chapter, and as exercises, we have:
Exercise 1
Learn more about fields, and their characteristic.
Exercise 2
Write down a self-contained approach to \( \mathbb R\) , using the decimal writing.
Exercise 3
Learn about what can be done with numeration bases \( \neq10\) .
Exercise 4
Check all the details for the Cardano formulae in degree \( 3\) and \( 4\) .
Exercise 5
Learn more about fields, and their Galois theory.
Exercise 6
Prove that \( e\) is irrational, then look up the fact that \( e\) is transcendental.
Exercise 7
Learn about random numbers, and how to produce them.
Exercise 8
Experiment a bit with Monte Carlo, on a computer.
As bonus exercise, work more on the material from the last section.
We have seen in chapter 1 that even very simple philosophical questions regarding the real numbers \( \mathbb R\) require some knowledge of advanced probability, or at least of probability theory going beyond what our daily life experience teaches us. Time now to do this, and in this chapter and in the next one we will discuss the discrete probability measures. Then, in chapter 4 we will discuss the continuous probability measures.
As starting point, we have basic things such as playing with coins, dice, or cards. You surely know a bit about this, both from real life and from previous math classes, and for our purposes here, it is convenient to axiomatize things in the following way:
Definition 4
A discrete probability space is a set \( X\) , usually finite or countable, whose elements \( x\in X\) are called events, together with a function
called probability function, which is subject to the condition
telling us that the overall probability for something to happen is \( 1\) .
As a first comment, our condition \( \sum_{x\in X}P(x)=1\) perfectly makes sense, and this even if \( X\) is uncountable, because the sum of positive numbers is always defined, as a number in \( [0,\infty]\) , and this no matter how many positive numbers we have.
As a second comment, we have chosen in the above not to assume that \( X\) is finite or countable, and this for instance because we want to be able to regard any probability function on \( \mathbb N\) as a probability function on \( \mathbb R\) , by setting \( P(x)=0\) for \( x\notin\mathbb N\) .
As a third comment, once we have a probability function \( P:X\to[0,\infty)\) as above, with \( P(x)\in[0,1]\) telling us what the probability for an event \( x\in X\) to happen is, we can compute what the probability for a set of events \( Y\subset X\) to happen is, by setting:
But more on this, mathematical aspects of discrete probability theory, later, when further building on Definition 2.1. Before that, let us explore the basic examples, coming from the real life. And here, there are many things to be learned, as follows:
Example 1
Flipping coins.
Here things are simple and clear, because when you flip a coin the corresponding discrete probability space, together with its probability measure, is as follows:
In the case where the coin is biased, as to land on heads with probability \( 2/3\) , and on tails with probability \( 1/3\) , the corresponding probability space is as follows:
More generally, given any number \( p\in[0,1]\) , we have an abstract probability space as follows, where we have replaced heads and tails by win and lose:
Finally, things become more interesting when flipping a coin, biased or not, several times in a row. We will be back to this in a moment, with details.
Example 2
Throwing dice.
Again, things here are simple and clear, because when you throw a die the corresponding probability space, together with its probability measure, is as follows:
As before with coins, we can further complicate this by assuming that the die is biased, say landing on face \( i\) with probability \( p_i\in[0,1]\) . In this case the corresponding probability space, together with its probability measure, is as follows:
Also as before with coins, things become more interesting when throwing a die several times in a row, or equivalently, when throwing several identical dice at the same time. In this latter case, with \( n\) identically biased dice, the probability space is as follows:
Observe that the sum 1 condition in Definition 2.1 is indeed satisfied, and with this proving that our dice modeling is bug-free, due to the following computation:
We will be back to dice in a moment, with further details and computations.
Example 3
Playing cards.
As a third and last example, which is more advanced, and also more dangerous, don’t try this at home, let us play some poker. Consider a deck of 32 cards:
Now let us pick 5 cards, and compute for instance the probability for having four of a kind. The total number of possibilities for a poker hand, 5 cards out of 32, is:
On the other hand, for having four of a kind, we have to pick that kind, then 4 cards of that kind, and then the 5th card, so the number of possibilities is:
Thus, the probability of having four of a kind is:
So far, so good, but you might argue, what if we model our problem as for our poker hand to be ordered, do we still get the same answer? In answer, sure yes, but let us check this. The probability for having four of a kind, computed in this new way, is then:
To be more precise, here on the bottom \( 32\cdot 31\cdot 30\cdot 29\cdot 28\) stands for the total number of possibilities for an ordered poker hand, 5 out of 32, and on top, exercise for you to figure out what the above numbers 8, 5, then \( 4\cdot3\cdot2\) , and 28, stand for.
Getting back now to theory, in the general context of Definition 2.1, we can see that what we have there is very close to the biased die, from Example 2.3. Indeed, in the general context of Definition 2.1, we can say that what happens is that we have a die with \( |X|\) faces, which is biased such that it lands on face \( i\) with probability \( P(i)\) . Which is quite interesting, so as a conclusion, let us record this finding as follows:
Conclusion 3
Discrete probability can be understood as being about throwing a general die, having an arbitrary number of faces, and which is arbitrarily biased too.
Moving ahead now, let us go back to the context of Definition 2.1, which is the most convenient one, technically speaking. As usual in probability, we are mainly interested in winning. But, winning what? In case we are dealing with a usual die, what we win is what the die says, and on average, what we win is the following quantity:
In case we are dealing with the biased die in Example 2.3, again what we win is what the die says, and on average, what we win is the following quantity:
With this understood, what about coins? Here, before doing any computation, we have to assign some numbers to our events, and a standard choice here is as follows:
With this choice made, what we can expect to win is the following quantity:
Of course, in the case where the coin is biased, this computation will lead to a different outcome. And also, with a different convention for \( f\) , we will get a different outcome too. Moreover, we can combine if we want these two degrees of flexibility.
In short, you get the point. In order to do some math, in the context of Definition 2.1, we need a random variable \( f:X\to\mathbb R\) , and the math will consist in computing the expectation of this variable, \( E(f)\in\mathbb R\) . Alternatively, in order to do some business in the context of Definition 2.1, we need some form of “money”, and our random variable \( f:X\to\mathbb R\) will stand for that money, and then \( E(f)\in\mathbb R\) , for the average gain.
Let us axiomatize this situation as follows:
Definition 5
A random variable on a probability space \( X\) is a function
and the expectation of such a random variable is the quantity
which is best thought as being the average gain, when the game is played.
Here the word “game” refers to the probability space interpretation from Conclusion 2.5. Indeed, in that context, with our discrete set of events \( X\) being thought of as corresponding to a generalized die, and by thinking of \( f\) as representing some sort of money, the above quantity \( E(f)\) is what we win, on average, when playing the game.
We have already seen some good illustrations for Definition 2.6, so time now to get into more delicate aspects. Imagine that you want to set up some sort of business, with your variable \( f:X\to\mathbb R\) . You are of course mostly interested in the expectation \( E(f)\in\mathbb R\) , but passed that, the way this expectation comes in matters too. For instance:
(1) When your variable is constant, \( f=c\) , you certainly have \( E(f)=c\) , and your business will run smoothly, with not so many surprises on the way.
(2) On the opposite, for a complicated variable satisfying \( E(f)=c\) , your business will be more bumpy, with wins or loses on the way, depending on your skills.
In short, and extrapolating now from business to mathematics, physics, chemistry and everything else, we must complement Definition 2.6 with something finer, regarding the “quality” of the expectation \( E(f)\in\mathbb R\) appearing there. And the first thought here, which is the correct one, goes to the following number, called variance of our variable:
However, let us not stop here. For a total control of your business, be that of financial, mathematical, physical or chemical type, you will certainly want to know more about your variable \( f:X\to\mathbb R\) . Which leads us into general moments, constructed as follows:
Definition 6
The moments of a variable \( f:X\to\mathbb R\) are the numbers
which satisfy \( M_0=1\) , then \( M_1=E(f)\) , and then \( V(f)=M_2-M_1^2\) .
And, good news, with this we have all the needed tools in our bag for doing some good business. To put things in a very compacted way, \( M_0\) is about foundations, \( M_1\) is about running some business, \( M_2\) is about running that business well, and \( M_3\) and higher are advanced level, about ruining all the competing businesses.
As a further piece of basic probability, coming this time as a theorem, we have:
Theorem 17
Given a random variable \( f:X\to\mathbb R\) , if we define its law as being
regarded as probability measure on \( \mathbb R\) , then the moments are given by the formula
with the usual convention that each Dirac mass integrates up to \( 1\) .
Proof
There are several things going on here, the idea being as follows:
(1) To start with, given a random variable \( f:X\to\mathbb R\) , we can certainly talk about its law \( \mu\) , as being the formal linear combination of Dirac masses in the statement. Our claim is that this is a probability measure on \( \mathbb R\) , in the sense of Definition 2.1. Indeed, the weight of each point \( y\in\mathbb R\) is the following quantity, which is positive, as it should:
Moreover, the total mass of this measure is \( 1\) , as it should, due to:
Thus, we have indeed a probability measure on \( \mathbb R\) , in the sense of Definition 2.1.
(2) Still talking basics, let us record as well the following alternative formula for the law, which is clear from definitions, and that we will often use, in what follows:
(3) Now let us compute the moments of \( f\) . With the usual convention that each Dirac mass integrates up to \( 1\) , as mentioned in the statement, we have:
Thus, we are led to the conclusions in the statement.
The above theorem is quite interesting, because we can see here a relation with integration, as we know it from calculus. In view of this, it is tempting to further go this way, by formulating the following definition, which is something purely mathematical:
Definition 7
Given a set \( X\) , which can be finite, countable, or even uncountable, a discrete probability measure on it is a linear combination as follows,
with the coefficients \( \lambda_i\in\mathbb R\) satisfying \( \lambda_i\geq0\) and \( \sum_i\lambda_i=1\) . For \( f:X\to\mathbb R\) we set
with the convention that each Dirac mass integrates up to \( 1\) .
Observe that, with this, we are now into pure mathematics. However, and we insist on this, it is basic probability, as developed before, which is behind all this. Now by staying abstract for a bit more, with Definition 2.9 in hand, we can recover our previous basic probability notions, from Definition 2.1 and from Theorem 2.8, as follows:
Theorem 18
With the above notion of discrete probability measure in hand:
Proof
This might look a bit scary, but is in fact a collection of trivialities, coming straight from definitions, the details being as follows:
(1) Nothing much to say here, with our assertion being plainly clear, just by comparing Definition 2.1 and Definition 2.9. As a interesting comment, however, in the general context of Definition 2.9, a probability measure \( \mu=\sum_{x\in X}\lambda_x\delta_x\) as there depends only on the following function, called density of our probability measure:
And, with this notion in hand, our equation \( P(x)=d\nu(x)\) simply says that the probability function \( P\) is the density of \( \nu\) . Which is something which is good to know.
(2) Pretty much the same story here, with our first assertion being clear, just by comparing Theorem 2.8 and Definition 2.9. As for the second assertion, consider more generally a probability space \( (X,\nu)\) , and a function \( f:X\to Y\) . We can then construct a probability measure \( \mu=f_*\nu\) on \( Y\) , called push-forward of \( \nu\) by \( f\) , as follows:
Alternatively, at the level of the corresponding measures of the parts \( Z\subset Y\) , we have the following abstract formula, which looks more conceptual:
In any case, one way or another we can talk about push-forward measures \( \mu=f_*\nu\) , and in the case of a random variable \( f:X\to\mathbb R\) , we obtain in this way the law of \( f\) .
Very nice all this, and needless to say, welcome to measure theory. In what follows we will rather go back to probability theory developed in the old way, as in the beginning of the present chapter, and keep developing that material, because we still have many interesting things to be learned. But, let us keep Definition 2.9 and Theorem 2.10, which are quite interesting, somewhere in our head. We will be back to these later.
Let us go back now to real-life probability, and try to understand what independence, which can only be something quite simple and intuitive, really means. The most basic occurrence of independence if when throwing a die several times in a row. In order to discuss this, let us first convene for the following rules for the game:
Rules 1
Rolling the die is played with the following rules:
Of course, this is something very particular, but we will complicate the rules later on, no worries for that. Now with these rules agreed on, we have the following result:
Theorem 19
When rolling a die \( k\) times what you can win are quantities of type \( 6w-k\) , with \( w=0,1,…,k\) , with the probability for this to happen being:
Geometrically, your winning curve starts with probability \( (5/6)^k\) of losing \( k\) , then increases, up to some point, and then decreases, up to probability \( 1/6^k\) of winning \( 5k\) .
Proof
There are several things going on here, the idea being as follows:
(1) When rolling the die \( k\) times, what will happen is that you will win \( w\) times and lose \( l\) times, with \( k=w+l\) . And in this situation, your profit will be, as stated:
(2) As for the probability for this to happen, this is the total number of possibilities for you to win \( w\) times, which is \( 5^{k-w}\binom{k}{w}\) , because this amounts in choosing the \( w\) times when you will win, among \( k\) , then multiplying by \( 5^{k-w}\) possibilities, at places where your partner wins, and finally dividing by the total number of possibilities, which is \( 6^k\) :
(3) As usual when doing complicated math, let us doublecheck all this, matter of being sure that we did not mess up our counting. First, the sum of all probabilities involved must be 1, and 1 that sum is, as shown by the following computation:
(4) Let us triplecheck this as well. Obviously, Rules 2.11 do not favor you, nor your partner, so on average, you should win 0. And this is the case indeed, because:
(5) This last computation was hot, wasn’t it, but in math, triplechecks are mandatory. In any case theorem proved, and the final conclusions in the statement are clear too.
Quite interestingly, Theorem 2.12 is best seen, both at the level of the statement, and of the proof, from the viewpoint of your partner. Let us record this, as follows:
Theorem 20
When rolling a die \( k\) times what you can win are quantities of type \( 5k-6l\) , with \( l=0,1,…,k\) , with the probability for this to happen being:
Geometrically, your winning curve starts with probability \( (5/6)^k\) of losing \( k\) , then increases, up to some point, and then decreases, up to probability \( 1/6^k\) of winning \( 5k\) .
Proof
As before, when rolling the die \( k\) times, you will win \( w\) times and lose \( l\) times, with \( k=w+l\) . And in this situation, your profit will be, as stated:
As for the rest, we already know all this from Theorem 2.12, but the point is that the proof of Theorem 2.12 becomes slightly simpler when using \( l\) instead of \( w\) .
Now with Theorem 2.13 in hand, it is quite clear that the basic \( 1/6-5/6\) probabilities at dice can be repaced with something of type \( p-(1-p)\) , with \( p\in[0,1]\) being arbitrary. We are led in this way to the following notions, which are quite general:
Definition 8
Given \( p\in[0,1]\) , the Bernoulli law of parameter \( p\) is given by:
More generally, the \( k\) -th binomial law of parameter \( p\) , with \( k\in\mathbb N\) , is given by
with the Bernoulli law appearing at \( k=1\) , with \( s=1,0\) here standing for win and lose.
Let us try now to understand the relation between the Bernoulli and binomial laws. Indeed, we know that the Bernoulli laws produce the binomial laws, simply by iterating the game, from 1 throw to \( k\in\mathbb N\) throws. Obviously, what matters in all this is the “independence” of our coin throws, so let us record this finding, as follows:
Theorem 21
The following happen, in the context of the biased coin game:
Proof
Obviously, this is something a bit informal, but let us prove this as stated, and we will come back later to it, with precise definitions, general theorems and everything. In what regards the first assertion, nothing to be said there, this is what life teaches us. As for the second assertion, the formula \( \mu_{bin}=\mu_{ber}^{*k}\) there certainly looks like mathematics, so job for us to figure out what this exactly means. And, this can be done as follows:
(1) The first idea is to encapsulate the data from Definition 2.14 into the probability measures associated to the Bernoulli and binomial laws. For the Bernoulli law, the corresponding measure is as follows, with the \( \delta\) symbols standing for Dirac masses:
As for the binomial law, here the measure is as follows, constructed in a similar way, you get the point I hope, again with the \( \delta\) symbols standing for Dirac masses:
(2) Getting now to independence, the point is that, as we will soon discover abstractly, the mathematics there is that of the following formula, with \( *\) standing for the convolution operation for the real measures, which is given by \( \delta_x*\delta_y=\delta_{x+y}\) and linearity:
(3) To be more precise, this latter formula does hold indeed, as a straightforward application of the binomial formula, the formal proof being as follows:
(4) Summarizing, save for some uncertainties regarding what independence exactly means, mathematically speaking, and more on this in a moment, theorem proved.
Getting to formal mathematical work now, let us start with the following straightforward definition, inspired by what happens for coins, dice and cards:
Definition 9
We say that two variables \( f,g:X\to\mathbb R\) are independent when
happens, for any \( x,y\in\mathbb R\) .
As already mentioned, this is something very intuitive, inspired by what happens for coins, dice and cards. As a first result now regarding independence, we have:
Theorem 22
Assuming that \( f,g:X\to\mathbb R\) are independent, we have:
More generally, we have the following formula, for the mixed moments,
and the converse holds, in the sense that this formula implies the independence of \( f,g\) .
Proof
We have indeed the following computation, using the independence of \( f,g\) :
As for the last assertion, this is clear too, because having the above computation work, for any \( k,l\in\mathbb N\) , amounts in saying that the independence formula for \( f,g\) holds.
Regarding now the convolution operation, motivated by what we found before, in Theorem 2.15, let us start with the following abstract definition:
Definition 10
Given a space \( X\) with a sum operation \( +\) , we can define the convolution of any two discrete probability measures on it,
as being the discrete probability measure given by the following formula:
That is, the convolution operation \( *\) is defined by \( \delta_x*\delta_y=\delta_{x+y}\) , and linearity.
As a first observation, our operation is well-defined, with \( \mu*\nu\) being indeed a discrete probability measure, because the weights are positive, \( a_ib_j\geq0\) , and their sum is:
Also, the above definition agrees with what we did before with coins, and Bernoulli and binomial laws. We have in fact the following general result:
Theorem 23
Assuming that \( f,g:X\to\mathbb R\) are independent, we have
where \( *\) is the convolution of real probability measures.
Proof
We have indeed the following straightforward verification, based on the independence formula from Definition 2.16, and on Definition 2.18:
Thus, we are led to the conclusion in the statement.
Before going further, let us attempt as well to find a proof of Theorem 2.19, based on the moment characterization of independence, from Theorem 2.17. For this purpose, we will need the following standard fact, which is of certain theoretical interest:
Theorem 24
The sequence of moments
uniquely determines the law.
Proof
Indeed, assume that the law of our variable is as follows:
The sequence of moments is then given by the following formula:
But it is then standard calculus to recover the numbers \( \lambda_i,x_i\in\mathbb R\) , and so the measure \( \mu\) , out of the sequence of numbers \( M_k\) . Indeed, assuming that the numbers \( x_i\) are \( 0<x_1<…<x_n\) for simplifying, in the \( k\to\infty\) limit we have the following formula:
Thus, we got the parameters \( \lambda_n,x_n\in\mathbb R\) of our measure \( \mu\) , and then by substracting them and doing an obvious recurrence, we get the other parameters \( \lambda_i,x_i\in\mathbb R\) as well. We will leave the details here as an instructive exercise, and come back to this problem later in this book, with more advanced and clever methods for dealing with it.
Getting back now to our philosophical question above, namely recovering Theorem 2.19 via moment technology, we can now do this, the result being as follows:
Theorem 25
Assuming that \( f,g:X\to\mathbb R\) are independent, the measures
have the same moments, and so, they coincide.
Proof
We have the following computation, using the independence of \( f,g\) :
On the other hand, we have as well the following computation:
Thus, job done, and theorem proved, or rather Theorem 2.19 reproved.
Getting back now to the basic theory of independence, here is now a second result, coming as a continuation of Theorem 2.19, which is something more advanced:
Theorem 26
Assuming that \( f,g:X\to\mathbb R\) are independent, we have
where \( F_f(x)=E(e^{ixf})\) is the Fourier transform.
Proof
We have the following computation, using Theorem 2.19:
Thus, we are led to the conclusion in the statement.
Of course, you might wonder what \( i\in\mathbb C\) has to do with all this. Good point, this is something quite subtle, and more on this later, trust me in the meantime.
As a continuation of the above, and at a more advanced level, let us discuss now the key limiting result in discrete probability, which is the Poisson Limit Theorem (PLT). This result involves the Poisson laws \( p_t\) , constructed as follows:
Definition 11
The Poisson law of parameter \( 1\) is the following measure,
and the Poisson law of parameter \( t>0\) is the following measure,
with the letter “p” standing for Poisson.
We are using here, as usual, some simplified notations for these laws. Observe that our laws have indeed mass 1, as they should, due to the following key formula:
We will see in the moment why these measures appear a bit everywhere, in the discrete context, the reasons for this coming from the Poisson Limit Theorem (PLT), which is closely related to our previous investigations regarding the Bernoulli and binomial laws.
For the moment, let us first develop some general theory, for these Poisson laws. We first have the following result, regarding their mean and variance:
Proposition 1
The mean and variance of \( p_t\) are given by:
In particular for the Poisson law \( p_1\) we have \( E=1,V=1\) .
Proof
We have two computations to be performed, as follows:
(1) Regarding the mean, this can be computed as follows:
(2) For the variance, we first compute the second moment, as follows:
Thus the variance is \( V=M_2-E^2=(t^2+t)-t^2=t\) , as claimed.
At the theoretical level now, we first have the following result:
Theorem 27
We have the following formula, for any \( s,t>0\) ,
so the Poisson laws form a convolution semigroup.
Proof
By using \( \delta_k*\delta_l=\delta_{k+l}\) and the binomial formula, we obtain:
Thus, we are led to the conclusion in the statement.
Next in line, we have the following result, which is fundamental as well:
Theorem 28
The Poisson laws appear as formal exponentials
with respect to the convolution of measures \( *\) .
Proof
By using the binomial formula, the measure on the right is:
Thus, we are led to the conclusion in the statement.
Regarding now the Fourier transform computation, this is as follows:
Theorem 29
The Fourier transform of \( p_t\) is given by
for any \( t>0\) .
Proof
We have indeed the following computation:
Thus, we obtain the formula in the statement.
Observe that the above result provides us with an alternative proof for Theorem 2.25, due to the fact that the logarithm of the Fourier transform is linear in \( t\) .
We can now establish the Poisson Limit Theorem, as follows:
Theorem 30 (PLT)
We have the following convergence, in moments,
for any \( t>0\) .
Proof
Let us denote by \( \nu_n\) the measure under the convolution sign. We have the following computation, for the Fourier transform of the limit:
Thus, we obtain indeed the Fourier transform of \( p_t\) , as desired.
Many further things can be said here, mixing Bernoulli laws and variables, binomial laws and variables, and Poisson laws and variables, and in particular clarifying the speculations from chapter 1. For all this, we recommend any specialized probability book.
In what concerns us, the above will do. We have Theorem 2.28, which is something nice and mathematical, and reasonably connected to real-life probability, in the obvious way, and this is what we will need, for further developing our mathematics.
Let us discuss now the moments of the Poisson laws. As we will soon discover, things are quite tricky here, in relation with a lot of subtle mathematics, and understanding this type of mathematics will be actually a main theme of discussion, for this book.
Let us start with something nice, and seemingly unrelated, namely:
Definition 12
We denote by \( P(k)\) the set of partitions of \( \{1,…,k\}\) , with these partitions \( \pi\in P(k)\) being most conveniently being drawn as diagrams,

with the strings joining the numbers belonging to the same block of \( \pi\) . That is, the above diagram represents the partition \( \{1,…,8\}=\{1,3,4\}\cup\{2,5\}\cup\{6,7\}\cup\{8\}\) .
Observe that there is a bit of care to be taken with this convention, in respect to the crossings. We can either proceed as above, with the \( \{2,5\}\) block being respresented “under” the block \( \{1,3,4\}\) , or use different types of strings, as for instance:

Both conventions are good, and we will be mostly using the first one, that from Definition 2.29. Now, let us study these partitions. And here, surprise, instead of pulling a theorem, as you would expect, we must formulate something quite modest, as follows:
Proposition 2
The Bell numbers \( B_k=|P(k)|\) satisfy the recurrence relation
with initial data \( B_0=1\) , \( B_1=1\) , and are numerically as follows:
However, there is no mathematical formula for \( B_k\) .
Proof
There are several things going on here, the idea being as follows:
(1) Experiments first, before anything, let us compute a few Bell numbers. Obviously \( B_1=1\) , and then we have \( B_2=2\) , the partitions being as follows:
Next, we have \( B_3=5\) , the partitions at \( k=3\) being as follows:
At \( k=4\) now, things become more complex, and it is better to trick. We can count the partitions up to permutations of the corresponding diagrams, and with this convention made, here are the relevant partitions and their multiplicities, leading to \( B_4=15\) :
The same method works at \( k=5\) , with the block distributions and multiplicities, which are simpler to draw than partitions, being as follows, leading to \( B_5=52\) :
As for the case \( k=6\) , where \( B_6=203\) , we will leave this as an instructive exercise.
(2) Let us try now to find a recurrence for these Bell numbers. Since a partition of \( \{1,…,k+1\}\) appears by choosing \( s\) partners for \( 1\) , among the \( k\) numbers available, and then partitioning the \( k-s\) elements left, we have the following formula:
Observe that this formula forces us to talk about \( B_0=1\) , as done in the statement.
(3) As for the last assertion, regarding the non-computability of the Bell numbers, take this as a physics fact. Mankind has tried to find a formula for these numbers, had not found anything, and we are reporting here this finding, which is of course rock-solid.
In relation now with the Poisson laws, we first have the following result:
Theorem 31
The moments of \( p_1\) are the Bell numbers,
where \( P(k)\) is the set of partitions of \( \{1,…,k\}\) .
Proof
The moments of \( p_1\) are given by the following formula:
We have the following recurrence formula for these moments:
Let us try now to find a recurrence for the Bell numbers:
A partition of \( \{1,…,k+1\}\) appears by choosing \( r\) neighbors for \( 1\) , among the \( k\) numbers available, and then partitioning the \( k-r\) elements left. Thus, we have:
Thus, the numbers \( M_k\) satisfy the same recurrence as the numbers \( B_k\) . Regarding now the initial values, for the moments of \( p_1\) , these are:
Indeed, the formula \( M_0=1\) is clear, and the formula \( M_1=1\) follows from:
Now by using the above recurrence we obtain from this:
Thus, we can say that the initial values for the moments are:
As for the Bell numbers, here the initial values are as follows:
Thus the initial values coincide, and so these numbers are equal, as stated.
More generally now, we have the following result:
Theorem 32
The moments of \( p_t\) are given by
where \( |.|\) is the number of blocks.
Proof
Observe first that the formula in the statement generalizes the one in Theorem 2.31, because at \( t=1\) we obtain, as we should:
In general now, the moments of \( p_t\) with \( t>0\) are given by:
We have the following recurrence formula for the moments of \( p_t\) :
As for the initial values of these moments, we know from Proposition 2.24 and its proof that these are given by the following formulae:
On the other hand, consider the numbers in the statement, namely:
Since a partition of \( \{1,…,k+1\}\) appears by choosing \( r\) neighbors for \( 1\) , among the \( k\) numbers available, and then partitioning the \( k-r\) elements left, we have:
As for the initial values of these numbers, these are as follows:
Thus the initial values coincide, so these numbers are the moments, as stated.
We had a lot on interesting theory in this chapter, and as exercises, we have:
Exercise 9
Compute all the probabilities at poker.
Exercise 10
Learn more about the Dirac masses, from the physicists.
Exercise 11
Meditate on the variance \( V\) , and its probabilistic meaning.
Exercise 12
Meditate on \( M_3,M_4,…\,\) , and their probabilistic meaning.
Exercise 13
Develop some mathematics for the formal exponential formula for \( p_t\) .
Exercise 14
Learn more about the PLT, and its probabilistic consequences.
Exercise 15
Work out, using calculus, the asymptotics of the Bell numbers.
Exercise 16
Fill in the missing details for the moments of \( p_t\) , with \( t>0\) .
As bonus exercise, set up a small business, with the theory that you learned here.
We have seen so far that, in what regards the discrete laws, passed a lot of general theory, that you should of course master well, and don’t hesitate here to learn and relearn more, and do more exercises, it all comes to combinatorics. In fact, and believe me here, at the advanced level, probability and combinatorics are the same thing.
With this idea in mind, let us examine more in detail the relation between probability and combinatorics. As a starting point, we have the following definition:
Definition 13
A group is a set \( G\) with a multiplication operation \( (g,h)\to gh\) , which must satisfy the following conditions:
When the multiplication is commutative, \( gh=hg\) , we say that \( G\) is abelian.
As a first observation, with this notion in hand, we can now properly talk about the convolution of measures, provided that our probability space is a group \( G\) , or at least a semigroup, and with the best results being obtained when \( G\) is an abelian group, with a basic example here being \( G=\mathbb R\) , or \( G=\mathbb R^N\) . Let us record this finding as follows:
Proposition 3
When our probability space is an abelian group, we have:
When our probability space is a non-abelian group, only the first three formulae hold.
Proof
This is just a reformulation of the group axioms, from Definition 3.1, in the abelian case, and then in general, in terms of Dirac masses, and with the group elements denoted \( x,y,z,…\,\) , and the unit denoted, as usual in the abelian case, by \( 0\) .
However, it is not about such things, which remain quite abstract, that we would like to talk about, in this chapter. In order to get to truly exciting connections between probability and combinatorics, let us get back to Definition 3.1, as stated, and see where that definition brings us. That is, let us develop some group theory, somehow with focus on combinatorial aspects, and for connections with probability, we will see later.
Getting started with our program, let us first discuss the finite abelian groups, which are the simplest ones. As basic examples, we have the cyclic groups:
Proposition 4
The following constructions produce the same group, denoted \( \mathbb Z_N\) , which is finite and abelian, and is called cyclic group of order \( N\) :
Proof
Here the equivalence between (1) and (2) is obvious. More complicated, however, is the question of finding the good philosophy and notation for this group. In what concerns us, we will use the notation (1), and often prefer the interpretation (2).
We can construct further examples by making products between various cyclic groups \( \mathbb Z_N\) . Quite remarkably, we obtain in this way all finite abelian groups:
Theorem 33
The finite abelian groups are precisely the products of cyclic groups:
Moreover, there are technical extensions of this result, going beyond the finite case.
Proof
This is something quite tricky, the idea being as follows:
(1) In order to prove our result, assume that \( G\) is finite and abelian. For any prime number \( p\in\mathbb N\) , let us define \( G_p\subset G\) to be the subset of elements having as order a power of \( p\) . Equivalently, this subset \( G_p\subset G\) can be defined as follows:
(2) It is then routine to check, based on definitions, that each \( G_p\) is a subgroup. Our claim now is that we have a direct product decomposition as follows:
(3) Indeed, by using the fact that our group \( G\) is abelian, we have a morphism as follows, with the order of the factors when computing \( \prod_pg_p\) being irrelevant:
Moreover, it is routine to check that this morphism is both injective and surjective, via some simple manipulations, so we have our group decomposition, as in (2).
(4) Thus, we are left with proving that each component \( G_p\) decomposes as a product of cyclic groups, having as orders powers of \( p\) , as follows:
But this is something that can be checked by recurrence on \( |G_p|\) , via some routine computations, and we are led to the conclusion in the statement. See Lang [15].
Moving away now from finite or abelian groups, the next thought goes to matrices. So, let us call \( GL_N(\mathbb C)\) the group formed by the \( N\times N\) matrices having nonzero determinant, with \( GL\) standing for “general linear”. By further imposing the condition \( \det A=1\) we obtain a subgroup \( SL_N(\mathbb C)\) , with \( SL\) standing for “special linear”, and then we can talk as well about the real versions of these groups, and also intersect everything with the group of unitary matrices \( U_N\) . We obtain in this way 8 groups, as follows:
Theorem 34
We have groups of invertible matrices as follows,

with \( S\) standing here for “special”, meaning having determinant \( 1\) .
Proof
This is clear indeed from the above discussion. As a comment, we can talk in fact about \( GL_N(F)\) and \( SL_N(F)\) , once we have a ground field \( F\) , but in what regards the corresponding orthogonal and unitary groups, things here are more complicated.
As a continuation, let us discuss now the finite non-abelian groups. As basic examples, we have the symmetric group \( S_N\) , and its various subgroups. Let us record here:
Proposition 5
We have finite non-abelian groups, as follows:
Proof
The fact that we have indeed groups is clear from definitions, and the non-abelianity of these groups is clear as well, provided of course that in each case \( N\) is chosen big enough, and with exercise for you to work out all this, with full details.
For constructing further examples of finite non-abelian groups, the best is to “look up”, by regarding \( S_N\) as being the permutation group of the \( N\) coordinate axes of \( \mathbb R^N\) . Indeed, this suggests looking at the symmetry groups of all sorts of geometric beasts inside \( \mathbb R^N\) , or even \( \mathbb C^N\) , and we end with a whole menagery of groups, as follows:
Theorem 35
We have groups of unitary matrices as follows,

for the most finite, and non-abelian, called complex reflection groups.
Proof
The above statement is of course something informal, and here are explanations on all this, including definitions for all the groups involved:
(1) To start with, \( S_N\) is the symmetric group \( S_N\) that we know, but regarded now as permutation group of the \( N\) coordinate axes of \( \mathbb R^N\) , and so as subgroup \( S_N\subset O_N\) .
(2) Similarly, \( A_N\) is the alternating group \( A_N\) that we know, but coming now geometrically, as \( A_N=S_N\cap SO_N\) , with the intersection being computed inside \( O_N\) .
(3) Regarding \( H_N\subset O_N\) , this is a famous group, called hyperoctahedral group, appearing as the symmetry group of the hypercube \( \square_N\subset\mathbb R^N\) .
(4) Regarding \( K_N\subset U_N\) , this is the complex analogue of \( H_N\) , consisting of the unitary matrices \( U\in U_N\) having exactly one nonzero entry, on each row and each column.
(5) We have as well on our diagram the groups \( SH_N,SK_N\) , with \( S\) standing as usual for “special”, that is, consisting of the matrices in \( H_N,K_N\) having determinant 1.
(6) In what regards now the diagram itself, sure I can see that \( S_N,A_N\) appear twice, but nothing can be done here, after thinking a bit, at how the diagram works.
(7) Let us mention too that the groups \( \mathbb Z_N,D_N\) have their place here, in \( N\) -dimensional geometry, but not exactly on our diagram, as being the symmetry groups of the oriented cycle, and unoriented cycle, with vertices at the simplex \( X_N=\{e_i\}\subset\mathbb R^N\) .
(8) Finally, in what regards finiteness, non-abelianity, and also the name “complex reflection groups”, many things to be checked here, left to you as an exercise.
Very nice all this. As a conclusion, which is something quite interesting, we have:
Conclusion 4
Most groups are best seen as being groups of matrices,
and even as groups of unitary matrices, \( G\subset U_N\) , in most cases.
So long for abstract group theory, and we will be back to this, on numerous occasions, in what follows. Going now towards probability, with somehow the above conclusion in mind, let us start with the following definition, putting everything on a solid basis:
Definition 14
A unitary representation of a compact group \( G\) is a morphism
which can be faithful or not. The character of such a representation is the function
where \( Tr\) is the usual, unnormalized trace of the \( N\times N\) matrices.
At the level of examples, most of the compact groups that we met so far naturally appear as subgroups \( G\subset U_N\) . In this case, the embedding \( G\subset U_N\) is of course a representation, called fundamental representation. Many other examples of representations can be constructed, by using various operations for the representations, and we will be back to this, with details, in chapter 14 below, when systematically discussing group theory.
Getting now to the point, the character \( \chi_u:G\to\mathbb C\) of a representation \( u:G\to U_N\) can be regarded as a random variable, and the advanced theory of compact groups, which is something that we will discuss later, in chapter 14 below, leads us into computing the law of \( \chi_u\) . Moreover, this problem is of particular interest when \( u\) is the fundamental representation of a matrix group \( G\subset U_N\) , so we are led into the following question:
Question 5
Given a compact group \( G\subset U_N\) , what is the law of
with respect to the uniform, mass \( 1\) measure on \( G\) ?
Observe that this latter question is something elementary, save for some details with the uniform measure on \( G\) , whose existence is something non-trivial. But, in what follows we will mostly deal with this question for the finite groups \( G\) , where the measure in question is simply the counting one. So, as a conclusion to this, pick \( G\subset U_N\) finite, solve Question 3.10 for it, and the advanced theory of finite groups guarantees that you are into very interesting mathematics. We will see in what follows that this is indeed the case.
Getting back now to probability, we would like to investigate Question 3.10 for the various reflection groups from Theorem 3.7. For this purpose, we will need:
Proposition 6
We have the following formula,
called inclusion-exclusion principle.
Proof
This is quite clear, by thinking a bit, as follows:
(1) In order to count \( (\cup_iA_i)^c\) , we certainly have to start with \( |A|\) .
(2) Then, we obviously have to remove each \( |A_i|\) , and so remove \( \sum_i|A_i|\) .
(3) But then, we have to put back each \( |A_i\cap A_j|\) , and so put back \( \sum_{i<j}|A_i\cap A_j|\) .
(4) Then, we must remove each \( |A_i\cap A_j\cap A_k|\) , so remove \( \sum_{i<j<k}|A_i\cap A_j\cap A_k|\) .
\( \vdots\)
(5) And so on, which leads to the formula in the statement.
We can now formulate the following key result, regarding the symmetric group \( S_N\) , making the link with the Poisson laws, that we met in chapter 2:
Theorem 36
The probability for a random \( \sigma\in S_N\) to have no fixed points is
in the \( N\to\infty\) limit, where \( e=2.7182…\) is the usual constant from analysis. More generally, the main character of \( S_N\) , which counts the fixed points, and is given by
via the standard embedding \( S_N\subset O_N\) , follows the Poisson law \( p_1\) , in the \( N\to\infty\) limit. Even more generally, the truncated characters of \( S_N\) , given by
with \( t\in(0,1]\) , follow the Poisson laws \( p_t\) , in the \( N\to\infty\) limit.
Proof
Many things going on here, the idea being as follows:
(1) In order to prove the first assertion, which is the key, and probably the most puzzling one, we will use the inclusion-exclusion principle. Let us set:
The set of permutations having no fixed points, called derangements, is then:
Now the inclusion-exclusion principle tells us that we have:
Thus, the probability that we are interested in, for a random permutation \( \sigma\in S_N\) to have no fixed points, is given by the following formula:
Since on the right we have the expansion of \( 1/e\) , this gives the result.
(2) Let us construct now the main character of \( S_N\) , as in the statement. The permutation matrices being given by \( \sigma_{ij}=\delta_{i\sigma(j)}\) , we have the following formula:
In order to establish now the asymptotic result in the statement, regarding these characters, we must prove the following formula, for any \( r\in\mathbb N\) , in the \( N\to\infty\) limit:
We already know, from (1), that this formula holds at \( r=0\) . In the general case now, we have to count the permutations \( \sigma\in S_N\) having exactly \( r\) points. Now since having such a permutation amounts in choosing \( r\) points among \( 1,…,N\) , and then permuting the \( N-r\) points left, without fixed points allowed, we have:
By dividing everything by \( N!\) , we obtain from this the following formula:
Now by using the computation at \( r=0\) , that we already have, from (1), it follows that with \( N\to\infty\) we have the following estimate:
Thus, we obtain as limiting measure the Poisson law of parameter 1, as stated.
(3) Finally, let us construct the truncated characters of \( S_N\) , as in the statement. As before in the case \( t=1\) , we have the following computation, coming from definitions:
Also before in the proofs of (1) and (2), we obtain by inclusion-exclusion that:
Now with \( N\to\infty\) , we obtain from this the following estimate:
More generally, by counting the permutations \( \sigma\in S_N\) having exactly \( r\) fixed points among \( 1,…,[tN]\) , as in the proof of (2), we obtain:
Thus, we obtain in the limit a Poisson law of parameter \( t\) , as stated.
As a first comment, we can approach the above problems as well directly, by using:
Proposition 7
The arbitrary integrals over the symmetric group \( S_N\) are given, modulo linearity, by the formula
where \( \ker i\) denotes as usual the partition of \( \{1,…,k\}\) whose blocks collect the equal indices of \( i\) , and where \( |.|\) denotes the number of blocks.
Proof
The first assertion follows from the Stone-Weierstrass theorem, because the standard coordinates \( g_{ij}\) separate the points of \( S_N\) , and so we have:
Regarding now the second assertion, according to the definition of the matrix coordinates \( g_{ij}\) , the integrals in the statement are given by:
Now observe that the existence of \( \sigma\in S_N\) as above requires \( i_m=i_n\iff j_m=j_n\) . Thus, the above integral vanishes when the following condition is satisfied:
Regarding now the case \( \ker i=\ker j\) , if we denote by \( b\in\{1,…,k\}\) the number of blocks of this partition \( \ker i=\ker j\) , we have \( N-b\) points to be sent bijectively to \( N-b\) points, and so \( (N-b)!\) solutions, and the integral is \( \frac{(N-b)!}{N!}\) , as claimed.
We can now recover the computation of the asymptotic law of \( \chi_t\) , as follows:
Theorem 37
For the symmetric group \( S_N\subset O_N\) , regarded as a compact group of matrices, \( S_N\subset O_N\) , via the standard permutation matrices, the truncated character
counts the number of fixed points among \( \{1,…,[tN]\}\) , and its law with respect to the counting measure becomes, with \( N\to\infty\) , a Poisson law of parameter \( t\) .
Proof
We already know this, from Theorem 3.12, but we can use as well the integration formula in Proposition 3.13. Indeed, with \( S_{kb}\) being the Stirling numbers, counting the partitions of \( \{1,…,k\}\) having exactly \( b\) blocks, we have the following formula:
In particular with \( N\to\infty\) we obtain the following formula:
But this is the \( k\) -th moment of the Poisson law \( p_t\) , and so we are done.
As another result now regarding \( S_N\) , here is a useful related formula:
Theorem 38
We have the law formula
where \( g_{ij}\) are the standard coordinates of \( S_N\subset O_N\) .
Proof
We have the following moment formula, where \( m_f\) is the number of permutations of \( \{1,… ,N\}\) having exactly \( f\) fixed points in the set \( \{1,… ,s\}\) :
Thus the law in the statement, say \( \nu_{sN}\) , is the following average of Dirac masses:
Now observe that the permutations contributing to \( m_f\) are obtained by choosing \( f\) points in the set \( \{1,… ,s\}\) , then by permuting the remaining \( N-f\) points in \( \{1,… ,n\}\) in such a way that there is no fixed point in \( \{1,…,s\}\) . But these latter permutations are counted as follows: we start with all permutations, we substract those having one fixed point, we add those having two fixed points, and so on. We obtain in this way:
We can proceed as follows, by using the new index \( p=f+k\) :
Here \( *\) is convolution of real measures, and the assertion follows.
Observe that the above formula is finer than most of our previous formulae, which were asymptotic, because it is valid at any \( N\in\mathbb N\) . We can use this formula as follows:
Theorem 39
Let \( g_{ij}\) be the standard coordinates of \( C(S_N)\) .
Proof
We can use indeed the formula in Theorem 3.15, as follows:
(1) With \( s\) fixed and \( N\to\infty\) we have the following estimate:
But the law on the right is that of a projection of trace \( s/N\) , as desired.
(2) We have a law formula of the following type:
The coefficients \( c_p\) can be estimated by using the Stirling formula, as follows:
We can now compute the Fourier transform with respect to a variable \( y\) :
But this being the Fourier transform of \( p_t\) , this gives the second assertion.
In order to further extend our results regarding the Poisson laws, we will need a number of standard probabilistic preliminaries. We have the following notion:
Definition 15
Associated to any discrete positive measure \( \nu\) on \( \mathbb C\) , not necessarily of mass \( 1\) , is the probability measure
where \( t=mass(\nu)\) , called compound Poisson law.
As a basic example, the measure \( \nu=t\delta_1\) with \( t>0\) produces the Poisson laws, via the PLT. The following standard result allows us to detect compound Poisson laws:
Proposition 8
For \( \nu=\sum_{i=1}^st_i\delta_{z_i}\) with \( t_i>0\) and \( z_i\in\mathbb C\) , we have
where \( F\) denotes the Fourier transform.
Proof
Let \( \eta_n\) be the measure in Definition 3.17, under the convolution sign:
We have then the following computation:
Thus, we have obtained the formula in the statement.
We have as well the following result, providing an alternative to Definition 3.17, and which will be our formulation here of the Compound Poisson Limit Theorem:
Theorem 40 (CPLT)
For \( \nu=\sum_{i=1}^st_i\delta_{z_i}\) with \( t_i>0\) and \( z_i\in\mathbb C\) , we have
where the variables \( \alpha_i\) are Poisson \( (t_i)\) , independent.
Proof
Let \( \alpha\) be the sum of Poisson variables in the statement, namely:
By using our standard Fourier transform formulae, we have:
Thus we have indeed the same formula as in Proposition 3.18, as desired.
Getting now to the examples, let us start with the following definition:
Definition 16
The Bessel law of level \( s\in\mathbb N\cup\{\infty\}\) and parameter \( t>0\) is
with \( \varepsilon_s\) being the uniform measure on the \( s\) -th roots of unity.
Of particular interest are the cases \( s=1,2,\infty\) , where we obtain the Poisson laws \( p_t\) , and then certain measures \( b_t,B_t\) , called real and purely complex Bessel laws:
As a basic result on the Bessel laws, generalizing those about \( p_t\) , we have:
Theorem 41
The Fourier transform of \( b^s_t\) is given by
where \( \exp_sz\) is the level \( s\) exponential function, given by the formula
so in particular the measures \( b^s_t\) have the property \( b^s_t*b^s_{t'}=b^s_{t+t'}\) .
Proof
We know from Theorem 3.19 that \( b^s_t\) appears as follows, with \( a_1,…,a_s\) being independent, each of them following the Poisson law of parameter \( t/s\) , and \( w=e^{2\pi i/s}\) :
We have the following computation, for the corresponding Fourier transform:
Thus, we are led to the conclusions in the statement.
Let us study now the density of \( b^s_t\) . We have here the following result:
Theorem 42
We have the formula
where \( w=e^{2\pi i/s}\) , and the \( \delta\) symbol is a Dirac mass.
Proof
The Fourier transform of the measure on the right is given by:
We multiply now by \( e^t\) , and we compute the derivative with respect to \( t\) :
By using the variable \( u=r-1\) , we obtain from this the following formula:
But this gives \( \log F=t(\exp_sz-1)\) , as in Theorem 3.21, as desired.
In analogy now to what we did before involving \( S_N\) , let us start with:
Proposition 9
Consider the hyperoctahedral group \( H_N\subset O_N\) , consisting of the symmetries of the standard hypercube in \( \mathbb R^N\) .
Proof
This is routine algebra, or perhaps geometry, as follows:
(1) This is indeed clear from definitions.
(2) Each of the permutations \( \sigma\in S_N\) of the \( N\) coordinate axes of \( \mathbb R^N\) can be further “decorated” by a sign vector \( \varepsilon\in\{\pm1\}^N\) , consisting of the possible \( \pm1\) flips which can be applied to each coordinate axis, at the arrival. In matrix terms, this gives the result.
(3) We have indeed, using (2), the formula \( |H_N| =|S_N|\cdot|\mathbb Z_2^N| =N!\cdot2^N\) .
(4) We know from (3) that at the level of cardinalities we have \( |H_N|=|S_N\times\mathbb Z_2^N|\) , and with a bit more work, we obtain that we have \( H_N=S_N\rtimes\mathbb Z_2^N\) , as claimed.
(5) This is simply a reformulation of (4), in terms of wreath products.
By doing now character computations, a bit as for \( S_N\) , we are led to:
Theorem 43
For the hyperoctahedral group \( H_N\subset O_N\) , the law of the truncated character \( \chi=g_{11}+… +g_{ss}\) with \( s=[tN]\) is, in the \( N\to\infty\) limit, the measure
which is the real Bessel law of parameter \( t>0\) .
Proof
We regard \( H_N\) as being the symmetry group of the graph \( I_N=\{I^1,… ,I^N\}\) formed by \( n\) segments. The diagonal coefficients are then given by:
Let us denote by \( F_g,R_g\) the number of segments among \( \{I^1,… ,I^s\}\) which are fixed, respectively returned by an element \( g\in H_N\) . With this notation, we have:
We denote by \( P_N\) probabilities computed over \( H_N\) . The density of the law of the variable \( u_{11}+…+u_{ss}\) at a point \( r\geq 0\) is then given by the following formula:
Thus, the asymptotic density can be computed as follows:
Together with \( D(-r)=D(r)\) , this gives the formula in the statement. As for the last assertion, this follows from the Fourier transform formula found above.
In order now to unify and extend our various results, we will need:
Definition 17
The complex reflection group \( H_N^s\subset U_N\) is the group of permutations of \( N\) copies of the \( s\) -simplex. Equivalently, we have
telling us that \( H_N^s\) consists of the permutation-type matrices with \( s\) -th roots of unity as entries. Also equivalently, we have the formula \( H_N^s=\mathbb Z_s\wr S_N\) .
To be more precise, the equivalence between the various viewpoints on \( H_N^s\) comes as in Proposition 3.23, which corresponds to the case \( s=2\) . In fact, we already met in Theorem 3.7 some of these groups, namely \( S_N,H_N,K_N\) , appearing at \( s=1,2,\infty\) .
Getting now to probability computations, we have here the following result:
Theorem 44
For the complex reflection group \( H_N^s\) we have, with \( N\to\infty\) :
Moreover, the asymptotic moments of this variable are the numbers
where \( P^s(k)\) are the partitions of \( \{1,…,k\}\) satisfying \( \#\circ=\#\bullet(s)\) , in each block.
Proof
This is something quite technical, the idea being as follows:
(1) At \( s=1\) the reflection group is \( H_N^1=S_N\) , the Bessel law is the Poisson law, \( b_t^1=p_t\) , and the formula \( \chi_t\sim p_t\) with \( N\to\infty\) is something that we know. As for the moment formula, where \( P^1=P\) , this is something that we know too.
(2) At \( s=2\) the reflection group is \( H_N^2=H_N\) , the Bessel law is \( b_t^2=b_t\) , and the formula \( \chi_t\sim b_t\) with \( N\to\infty\) is something that we know. As for the moment formula, where \( P^2=P_{even}\) , this is something more technical, which can be established too.
(3) At \( s=\infty\) the reflection group is \( H_N^\infty=K_N\) , the Bessel law is \( b_t^\infty=B_t\) , and the formula \( \chi_t\sim B_t\) with \( N\to\infty\) is something that can be proved as for \( S_N,H_N\) . As for the moment formula, where \( P^\infty=\mathcal P_{even}\) , this can be established too.
(4) In the general case, \( s\in\mathbb N\cup\{\infty\}\) , the formula \( \chi_t\sim b_t^s\) with \( N\to\infty\) can be established like for \( S_N,H_N\) , and the moment formula is something more technical. For details on all this, and for the whole story, you can have a look at my book [16].
We have seen a lot of interesting combinatorics in this chapter, but this is not the end of the story. Following Rota, let us formulate now the following definition:
Definition 18
Associated to any real probability measure \( \mu=\mu_f\) is the following modification of the logarithm of the Fourier transform \( F_\mu(\xi)=E(e^{i\xi f})\) ,
called cumulant-generating function. The Taylor coefficients \( k_n(\mu)\) of this series, given by
are called cumulants of the measure \( \mu\) . We also use the notations \( k_f,K_f\) for these cumulants and their generating series, where \( f\) is a variable following the law \( \mu\) .
In other words, the cumulants are the coefficients of \( \log F_\mu\) , up to some normalizations. As a first observation, the sequence of cumulants \( k_1,k_2,k_3,…\) appears as a modification of the sequence of moments \( M_1,M_2,M_3,…\,\) , the numerics being as follows:
Proposition 10
The sequence of cumulants \( k_1,k_2,k_3,…\) appears as a modification of the sequence of moments \( M_1,M_2,M_3,…\,\) , and uniquely determines \( \mu\) . We have
in one sense, and in the other sense we have
with in both cases the correspondence being polynomial, with integer coefficients.
Proof
We know from Definition 3.27 that the cumulants are given by:
By exponentiating, we obtain from this the following formula:
Now by looking at the terms of order \( 1,2,3,4\) , this gives the above formulae.
The interest in cumulants comes from the fact that \( \log F_\mu\) , and so the cumulants \( k_n(\mu)\) too, linearize the convolution. To be more precise, we have the following result:
Theorem 45
The cumulants have the following properties:
Proof
Here (1) and (2) are both clear from definitions, because we have:
As for (3), this follows from the fact that the Fourier transform \( F_f(\xi)=E(e^{i\xi f})\) satisfies the following formula, whenever \( f,g\) are independent random variables:
Indeed, by applying the logarithm, we obtain the following formula:
With the change of variables \( \xi\to-i\xi\) , we obtain the following formula:
Thus, at the level of coefficients, we obtain \( k_n(f+g)=k_n(f)+k_n(g)\) , as claimed.
At the level of examples now, we have the following key result:
Theorem 46
For a compound Poisson law \( p_\nu\) we have
valid for any integer \( n\geq1\) .
Proof
We can assume, by using a continuity argument, that our measure \( \nu\) is discrete, as follows, with \( t_i>0\) and \( z_i\in\mathbb R\) , and with the sum being finite:
By using now the Fourier transform formula for \( p_\nu\) , we obtain:
Thus, we are led to the conclusion in the statement.
Getting back to theory now, the sequence of cumulants \( k_1,k_2,k_3,…\) appears as a modification of the sequence of moments \( M_1,M_2,M_3,…\,\) , and understanding the relation between moments and cumulants will be our next task. Let us start with:
Definition 19
The M
specialChar{34}obius function of any lattice, and so of \( P\) , is given by
with the construction being performed by recurrence.
As an illustration here, for \( P(2)=\{||,\sqcap\}\) , we have by definition:
Also, \( ||<\sqcap\) , with no intermediate partition in between, so we obtain:
Finally, we have \( \sqcap\not\leq||\) , and so we have as well the following formula:
The main interest in the M
specialChar{34}obius function comes from the M
specialChar{34}obius inversion formula, which in linear algebra terms can be stated and proved as follows:
Theorem 47
We have the following implication,
valid for any two functions \( f,g:P(n)\to\mathbb C\) .
Proof
Consider the adjacency matrix of \( P\) , given by the following formula:
Our claim is that the inverse of this matrix is the M
specialChar{34}obius matrix of \( P\) , given by:
Indeed, the above matrix \( A\) is upper triangular, and when trying to invert it, we are led to the recurrence in Definition 3.31, so to the M
specialChar{34}obius matrix \( M\) . Thus we have:
Thus, in practice, we are led to the inversion formula in the statement.
With these ingredients in hand, let us go back to probability. We first have:
Definition 20
We define quantities \( M_\pi(f),k_\pi(f)\) , depending on partitions
by starting with \( M_n(f),k_n(f)\) , and using multiplicativity over the blocks.
To be more precise, the convention here is that for the one-block partition \( 1_n\in P(n)\) , the corresponding moment and cumulant are the usual ones, namely:
Then, for an arbitrary partition \( \pi\in P(k)\) , we decompose this partition into blocks, having sizes \( b_1,…,b_s\) , and we set, by multiplicativity over blocks:
With this convention, following Rota and others, we can now formulate a key result, fully clarifying the relation between moments and cumulants, as follows:
Theorem 48
We have the moment-cumulant formulae
or, equivalently, we have the moment-cumulant formulae
where \( \mu\) is the M
specialChar{34}obius function of \( P(n)\) .
Proof
There are several things going on here, the idea being as follows:
(1) According to our conventions above, the first set of formulae is equivalent to the second set of formulae. Also, due to the Möbius inversion formula, in the second set of formulae, the two formulae there are in fact equivalent. Thus, the 4 formulae in the statement are all equivalent. In what follows we will focus on the first 2 formulae.
(2) Let us first work out some examples. At \( n=1,2,3\) the moment formula gives the following equalities, which are in tune with the findings from Proposition 3.28:
At \( n=4\) now, which is a case which is of particular interest for certain considerations to follow, the computation is as follows, again in tune with Proposition 3.28:
As for the cumulant formula, at \( n=1,2,3\) this gives the following formulae for the cumulants, again in tune with the findings from Proposition 3.28:
Finally, at \( n=4\) , after computing the M
specialChar{34}obius function of \( P(4)\) , we obtain the following formula for the fourth cumulant, again in tune with Proposition 3.28:
(3) Time now to get to work, and prove the result. As mentioned above, the formulae in the statement are all equivalent, and it is enough to prove the first one, namely:
In order to do this, we use the very definition of the cumulants, namely:
By exponentiating, we obtain from this the following formula:
(4) Let us first compute the function on the left. This is easily done, as follows:
(5) Regarding now the function on the right, this is given by:
But the point now is that all this leads us into partitions. Indeed, we are summing over indices \( s_1,…,s_p\in\mathbb N\) , which can be thought of as corresponding to a partition of \( n=s_1+…+s_p\) . So, let us rewrite our sum, as a sum over partitions. For this purpose, recall that the number of partitions \( \nu\in P(n)\) having blocks of sizes \( s_1,…,s_p\) is:
Also, when resumming over partitions, there will be a \( p!\) factor as well, coming from the permutations of \( s_1,…,s_p\) . Thus, our sum can be rewritten as follows:
(6) We are now in position to conclude. According to (3,4,5), we have:
Thus, we have the following formula, valid for any \( n\in\mathbb N\) :
We are therefore led to the conclusions in the statement.
Very nice all this, good work that we did in this chapter. We will be back to this, relation between combinatorics and probability, on many occasions, in what follows.
Here are some exercises on the above, all combinatorics, of various flavors:
Exercise 17
Learn more about finite abelian groups, and Pontrjagin duality.
Exercise 18
Learn more about group representations, and their characters.
Exercise 19
We learned how to compute \( e\) , using \( S_N\) . What about \( \pi\) ?
Exercise 20
Learn about Bessel functions, of the first and second kind.
Exercise 21
Clarify all the details, for our computation involving \( H_N\) .
Exercise 22
Clarify all the details, for our computation involving \( H_N^s\) .
Exercise 23
Compute the M
specialChar{34}obius matrix for \( P(n)\) , at \( n=2,3,4\) .
Exercise 24
Compute more cumulants, for some measures of your choice.
As bonus exercise, learn some group theory. All beautiful things, and very useful.
We have seen that some interesting theory can be developed for the discrete measures, notably with a lot of exciting results regarding the Poisson laws, and their versions. Normally this would be the end of what we have to say, in the discrete case, and our job now would be that of getting into the continuous case, first with measure theory systematically developed, and then with some applications of this, to continuous probability.
However, we cannot leave basic probability without talking, in one form or another, about central limits. You have certainly heard about bell-shaped curves, and perhaps even observed them in physics or chemistry class, because any routine measurement leads to such curves. Mathematically, here is the question that we would like to solve:
Question 6
Given random variables \( f_1,f_2,f_3,…\) , say taken discrete, which are i.i.d., centered, and with common variance \( t>0\) , do we have
in the \( n\to\infty\) limit, for some bell-shaped density \( g_t\) ? And, what is the formula of \( g_t\) ?
Observe that this question perfectly makes sense, with the probability theory that we know, by assuming that our random variables \( f_1,f_2,f_3,…\) are discrete, as said above. As for the \( 1/\sqrt{n}\) factor, there is certainly need for a normalization factor there, as for things to have a chance to converge, and the good factor is \( 1/\sqrt{n}\) , as shown by:
Proposition 11
In order for a sum of the following type to have a chance to converge, with \( f_1,f_2,f_3,…\) being i.i.d., centered, and with common variance \( t>0\) ,
we must normalize this sum by a \( 1/\sqrt{n}\) factor, as in Question 4.1.
Proof
The idea here is to look at the moments of \( S\) . Since all variables \( f_i\) are centered, \( E(f_i)=0\) , so is their sum, \( E(S)=0\) , and no contradiction here. However, when looking at the variance of \( S\) , which equals the second moment, due to \( E(S)=0\) , things become interesting, due to the following computation:
Thus, we are in need a normalization factor \( \alpha\) , in order for our sum to have a chance to converge. But, repeating the computation with \( S\) replaced by \( \alpha S\) gives:
Thus, the good normalization factor is \( \alpha=1/\sqrt{n}\) , as claimed.
So far, so good, we have a nice problem above, and time now to make a plan, in order to solve it. With the tools that we have, from chapters 2-3, here is such a plan:
Plan 1
In order to solve our central limiting question, we have to:
Which sounds very good, but since we will be navigating here in troubled waters, with \( g_t\) being continuous, as opposed to the discrete densities that we are used to, perhaps time to ask the cat, before embarking on our trip. And good news, cat is already onboard, hired to catch mice on our ship, so let’s ask him what he has to say. And cat says:
Cat 1
Capitain, that \( g_t\) we are looking for is something two-dimensional, and all this normally comes after multivariable calculus, well done. But we can still go ahead, as long as there’s enough booze for you guys, and mice for me, things fine.
Nice advice, especially for the last part, but in what regards the first part, I must admit that I am completely puzzled. What are that two dimensions cat is talking about? Hope he has not confused mice with booze. So thanks cat, but please don’t touch human food during our trip, mice only for you, and let’s leave now, time to go.
So, back to our Plan 4.3, let us start with (1) there. Things are quickly done here, by using the linearization results for convolution from chapter 2, which lead to:
Theorem 49
Given discrete variables \( f_1,f_2,f_3,…\) , which are i.i.d., centered, and with common variance \( t>0\) , we have
with \( n\to\infty\) , with \( g_t\) being the law having \( F(x)=e^{-tx^2/2}\) as Fourier transform.
Proof
There are several things going on here, the idea being as follows:
(1) Observe first that in terms of moments, the Fourier transform of an arbitrary random variable \( f:X\to\mathbb R\) is given by the following formula:
(2) In particular, in the case of a centered variable, \( E(f)=0\) , as those that we are interested in, the Fourier transform formula that we get is as follows:
Moreover, by further assuming that the Fourier variable is small, \( x\simeq0\) , the Fourier transform formula that we get, that we will use in what follows, becomes:
(3) In addition to this, we will also need to know what happens to the Fourier transform when rescaling. But the formula here is very easy to find, as follows:
(4) Good news, we can now do our computation. By using the above formulae in (2) and (3), the Fourier transform of the variable in the statement is given by:
(3) We are therefore led to the conclusion in the statement, modulo the fact that we do not know yet that a density \( g_t\) having as Fourier transform \( F(x)=e^{-tx^2/2}\) really exists, plus perhaps some other abstract issues, related to the continuous measures, to be discussed too. But too late to go back, both cat and sailors are happy, we will go ahead. So, theorem proved, modulo finding that law \( g_t\) , which still remains to be done.
Getting now to step (2) of our Plan 4.3, that is easy to work out too, via some elementary one-variable calculus, with the result here being as follows:
Theorem 50
The “normal” law \( g_t\) , having as Fourier transform
must have all odd moments zero, and its even moments must be the numbers
where \( k!!=(k-1)(k-3)(k-5)…\) , for \( k\in2\mathbb N\) .
Proof
Again, several things going on here, the idea being as follows:
(1) To start with, at the level of formalism and notations, in view of Question 4.1 and of Theorem 4.5, we have adopted the term “normal” for the mysterious law \( g_t\) that we are looking for, the one having \( F(x)=e^{-tx^2/2}\) as Fourier transform.
(2) Getting towards the computation of the moments, as a first useful observation, according to Theorem 4.5 this normal law \( g_t\) must be centered, as shown by:
Moreover, the same argument works by replacing “centered” with “having an even function as density”, and this shows, via some standard calculus, that we will leave here as an exercise, that the odd moments of our normal law must vanish:
Thus, first assertion proved, and we only have to care about the even moments.
(3) As a comment here, as we will see in a moment, our study below of the moments computes in fact the odd moments too, as being all equal to 0, this time without making reference to Theorem 4.5. Thus, definitely no worries with the odd moments.
(4) Getting to work now, we must reformulate the equation \( F(x)=e^{-tx^2/2}\) , in terms of moments. We know from the proof of Theorem 4.5 that we have:
On the other hand, we have the following formula, for the exponential:
Thus, our equation \( F(x)=e^{-tx^2/2}\) takes the following form:
(5) As a first observation, the odd moments must vanish, as said in (2) above. As for the even moments, these can be computed as follows:
Thus, we are led to the formula in the statement.
The moment formula that we found is quite interesting, and before going ahead with step (3) of our Plan 4.3, let us look a bit at this, and see what we can further say.
To be more precise, in analogy with what we know about the Poisson laws, and about the Bessel laws too, making reference to interesting combinatorics and partitions, when it comes to moments, we have the following result, regarding the normal laws:
Theorem 51
The moments of the normal law \( g_t\) are given by
for any \( k\in\mathbb N\) , with \( P_2(k)\) standing for the pairings of \( \{1,…,k\}\) .
Proof
This is a reformulation of Theorem 4.6, the idea being as follows:
(1) We know from Theorem 4.6 that the moments of the normal law \( M_k=M_k(g_t)\) that we are interested in are given by the following formula, with the convention \( k!!=0\) for \( k\) odd, and \( k!!=(k-1)(k-3)(k-5)…\) for \( k\) even, for the double factorials:
Now observe that, according to our above convention for the double factorials, these are subject to the following recurrence relation, with initial data \( 1!!=0,2!!=1\) :
We conclude that the moments of the normal law \( M_k=M_k(g_t)\) are subject to the following recurrence relation, with initial data \( M_1=0,M_2=t\) :
(2) On the other hand, let us first count the pairings of the set \( \{1,…,k\}\) . In order to have such a pairing, we must pair \( 1\) with one of the numbers \( 2,…,k\) , and then use a pairing of the remaining \( k-2\) numbers. Thus, we have the following recurrence formula for the number \( P_k\) of such pairings, with the initial data \( P_1=0,P_2=1\) :
Now by multiplying by \( t^{k/2}\) , the resulting numbers \( N_k=t^{k/2}P_k\) will be subject to the following recurrence relation, with initial data \( N_1=0,N_2=t\) :
(3) Thus, the moments \( M_k=M_k(g_t)\) and the numbers \( N_k=t^{k/2}P_k\) are subject to the same recurrence relation, with the same initial data, so they are equal, as claimed.
Still in analogy with what we know about the Poisson laws, and about the Bessel laws too, we can further process what we found in Theorem 4.7, and we are led to:
Theorem 52
The moments of the normal law \( g_t\) are given by
where \( P_2(k)\) is the set of pairings of \( \{1,…,k\}\) , and \( |.|\) is the number of blocks.
Proof
This is a quick reformulation of Theorem 4.7, with the number of blocks of a pairing of \( \{1,…,k\}\) being trivially \( k/2\) , independently of the pairing.
It is possible to do some more combinatorics here, again in relation with what we know about the Poisson laws, for instance by looking at cumulants, and we have:
Theorem 53
The cumulants of the normal law \( g_t\) are the following numbers:
In particular, the normal laws satisfy \( g_s*g_t=g_{s+t}\) , for any \( s,t>0\) .
Proof
We have two assertions here, the idea being as follows:
(1) For the normal law \( g_t\) we have the following computation:
But the plain coefficients of this series are the numbers \( 0,t/2,0,0,…\,\) , and so the Taylor coefficients of this series are the numbers \( 0,t,0,0,…\,\) , as claimed.
(2) As for the last assertion, regarding the semigroup property of the normal laws, this actually follows from Theorem 4.5, the log of the Fourier transform being linear in \( t\) , but is best seen by looking at the cumulants, which are obviously linear in \( t\) .
(3) However, as a technical remark here, the linearization results for the convolution that we have, be them in terms of the Fourier transform, or of the cumulants, were formally established in chapters 2-3 only for the discrete measures. So, instead of further thinking at all this, let us pull out a third, elementary proof for \( g_s*g_t=g_{s+t}\) .
(4) In order to do this, consider, as in Theorem 4.5, on one hand i.i.d. centered variables \( f_1,f_2,f_3,…\) having variance \( s>0\) , and on the other hand i.i.d. centered variables \( h_1,h_2,h_3,…\) having variance \( t>0\) . According to Theorem 4.5, we have:
Now let us sum these formulae. Assuming that the variables \( f_1,f_2,f_3,…\) that we used were independent from the variables \( h_1,h_2,h_3,…\) , we obtain in this way:
On the other hand, yet another application of Theorem 4.5, with the remark that by independence, the variance of \( f_i+h_i\) is indeed \( s+t\) , gives the following formula:
Thus, we are led to the semigroup formula \( g_s*g_t=g_{s+t}\) , as desired.
As a philosophical conclusion now to all this, let us formulate:
Conclusion 5
The normal laws \( g_t\) have properties which are quite similar to those of the Poisson laws \( p_t\) , and combinatorially, the passage
appears by replacing the partitions with the pairings.
Which sounds quite conceptual, hope you agree with me. According to Question 4.1, compared to the Poisson Limit Theorem, the normal law \( g_t\) should be rather the “continuous version” of the Poisson law \( p_t\) , and here we are into some sort of quite advanced mathematics, telling us that the passage from discrete to continuous appears, combinatorially speaking, by replacing the partitions with the pairings.
We will explore such things, which are indeed quite advanced, later in this book. In the meantime, however, we still need to know what the density of \( g_t\) is.
So, let us get now to step (3) of our Plan 4.3. This does not look obvious at all, but some partial integration know-how leads us to the following statement:
Theorem 54
The normal laws are given by
with the constant on the bottom being \( I=\int_\mathbb Re^{-x^2}dx\) .
Proof
This comes from partial integration, as follows:
(1) Let us first do a naive computation. Consider the following quantities:
It is quite obvious that by partial integration we will get a recurrence formula for these numbers, similar to the one that we have for the moments of the normal laws. So, let us do this. By partial integration we obtain the following formula, for any \( k\in\mathbb N\) :
(2) Thus, we are on the good way, with the recurrence formula that we got being the same as that for the moments of \( g_{1/2}\) . Now let us fine-tune this, as to reach to the same recurrence as for the moments of \( g_t\) . Consider the following quantities:
By partial integration as before, we obtain the following formula:
(3) Thus, almost done, and it remains to discuss normalization. We know from the above that we must have a formula as follows, with \( I_t\) being a certain constant:
But the constant \( I_t\) must be the one making \( g_t\) of mass 1, and so:
Thus, we are led to the formula in the statement.
What we did in the above is good work, and it remains to compute the constant \( I\) appearing in Theorem 4.11, given by the following formula, and called Gauss integral:
Which does not look obvious at all, so time to ask again the cat. Unfortunately cat is nowhere to be found, probably gone with the sailors, for some partying, now that we are again on land, for a short break. This being said, walking through this debauchery, I can spot a small kitty, behind a pile of harbor garbage. She’s really small, and can barely meow, but never knows, let’s ask her about mathematics. And kitty answers:
Kitty 1
The normal laws are naturally two-dimensional, and it is
the quantity that you should compute. I bet you can do that.
Thanks kitty, this sounds wise, so again that mysterious two-dimensionality, that cat was talking about. So, I will try this, and by the way, come with me. Cat is surely very good at both catching mice and helping with trigonometry and other math, but these days he’s learning more and more bad things from the sailors, I am a bit worried about this, and in any case, we need a backup. You’ll like it, plenty of mice on our ship.
So, following now kitty’s advice, we are led to the following result:
Theorem 55
We have the following formula,
called Gauss integral formula.
Proof
As already mentioned, this is something which is nearly impossible to prove, with bare hands. However, this can be proved by using two dimensions, as follows:
Thus, we are led to the conclusion in the statement.
Very nice, so as a final conclusion to our study, started long ago, in the beginning of this chapter, we can now formulate the Central Limit Theorem (CLT), as follows:
Theorem 56 (CLT)
Given discrete random variables \( f_1,f_2,f_3,…\) , which are i.i.d., centered, and with common variance \( t>0\) , we have
in the \( n\to\infty\) limit, in moments, with the limiting mesure being
called normal, or Gaussian law of parameter \( t>0\) .
Proof
This follows indeed from our various results above, and more specifically from Theorem 4.5, Theorem 4.6 for the terminology, Theorem 4.11 and Theorem 4.13.
Let us study now more in detail the laws that we found. Normally we already have everything that is needed, but it is instructive at this point to do some computations, based on the explicit formula of \( g_t\) found above, and on Theorem 4.13. We first have:
Proposition 12
We have the variance formula
valid for any \( t>0\) .
Proof
We already know this, but we can establish this as well directly, starting from our formula of \( g_t\) from Theorem 4.14. Indeed, the first moment is 0, because our normal law \( g_t\) is centered. As for the second moment, this can be computed as follows:
We conclude from this that the variance is \( V=M_2=t\) , as claimed.
More generally, we can recover in this way the computation of all moments:
Theorem 57
The even moments of the normal law are the numbers
where \( k!!=(k-1)(k-3)(k-5)…\,\) , and the odd moments vanish.
Proof
Again, we already know this, but we can establish this as well directly, starting from our formula above of \( g_t\) . Indeed, we have the following computation:
Thus by recurrence, we are led to the formula in the statement.
Here is another result, which is the key one for the study of the normal laws:
Theorem 58
We have the following formula, valid for any \( t>0\) :
In particular, the normal laws satisfy \( g_s*g_t=g_{s+t}\) , for any \( s,t>0\) .
Proof
As before, we already know this, but we can establish now the Fourier transform formula as well directly, by using the explicit formula of \( g_t\) , as follows:
As for the last assertion, this follows from the fact that \( \log F_{g_t}\) is linear in \( t\) .
Observe that, thinking retrospectively, the above computation formally solves the question raised by Theorem 4.5, and so could have been used there, afterwards. However, and here comes the point, all this is based on Theorem 4.13, and also, crucially, on our work from Theorem 4.11, which in turn was based on moments and so on.
Thus, no really way of doing things in a simpler way. There is however one simplification, because if you know well multivariable calculus, the Gauss formula comes right away via polar coordinates, the computation here being as follows:
And, based on this, it is quite natural to introduce the normal laws \( g_t\) directly, then compute Fourier, and prove the CLT, as probabilists do. More on this later, once we will know well multivariable calculus, and in particular the formula \( dxdy=rdrdt\) , which was crucially used in the above computation, we will certainly come back to this.
As another comment, as explained in the previous section, before even knowing the formula of the densities, the normal laws \( g_t\) have a quite interesting combinatorics, quite similar to that of the Poisson and Bessel laws. We will be back to this later, in Part IV below, when discussing on a more systematic basis probability theory, in general. In the meantime, however, a few words about groups, which are still missing from our picture. In analogy with what we know about the Poisson and Bessel laws, we have:
Fact 5
For the orthogonal group \( O_N\) , the main character
follows with \( N\to\infty\) the normal law \( g_1\) . More generally, the truncated character
follows with \( N\to\infty\) the normal law \( g_t\) , for any \( t\in(0,1]\) .
Which looks very good, and in tune with what we know about the Poisson and Bessel laws, with the overall conclusion being that the passage \( p_t\to g_t\) , from discrete to continuous, amounts in performing a passage \( S_N\to O_N\) at the group theory level.
Moreover, with some group theory helping, it is possible to derive from this, directly, our previous Conclusion 4.10, in regards with partitions and pairings.
However, and here comes the point, proving such things is quite difficult, because unlike in the finite group case, where we were able to get away with elementary counts on \( S_N,H_N\) and other reflection groups, in order to deal with continuous groups like \( O_N\) , we have to know more about the uniform integration on such groups, and this is something non-trivial. So, all this will have to wait a bit, we will be back to it in chapter 14.
Let us discuss now the complex analogues of all the above, with a notion of complex normal, or Gaussian law. To start with, we have the following definition:
Definition 21
A complex random variable is a variable \( f:X\to\mathbb C\) . In the discrete case, the law of such a variable is the complex probability measure
given by the following formula, with \( P\) being the probability over \( X\) ,
with the sum being finite or countable, as per our discretness assumption.
Observe the similarity with the analogous notions introduced in chapter 2, for the real variables \( f:X\to\mathbb R\) . In fact, what we are doing here is to extend the formalism from chapter 2, from real to complex, in a straightforward way. As a basic example for this, any real variable \( f:X\to\mathbb R\) can be regarded as a complex variable \( f:X\to\mathbb C\) .
In order to understand the precise relation with the real theory, from chapter 2, we can decompose any complex variable \( f:X\to\mathbb C\) as a sum, as follows:
With this done, we have the following computation, for the corresponding law:
In the case where the real and imaginary parts \( g,h:X\to\mathbb R\) are independent, we can say more about this, with the above computation having the following continuation:
To be more precise, we have used here in the beginning the independence of the variables \( h,g:X\to\mathbb R\) , and at the end we have denoted the measure on the right, which is obtained from \( \mu_h\) by putting this measure on the imaginary axis, by \( i\mu_h\) .
All this is quite interesting, going beyond what we know so far about basic probability, in the real case, so let us record this finding, along with a bit more, as follows:
Theorem 59
For a discrete complex random variable \( f:X\to\mathbb C\) , decomposed into real and imaginary parts as \( f=g+ih\) , and with \( g,h\) assumed independent, we have
with \( *\) being the usual convolution operation, \( \delta_z*\delta_t=\delta_{z+t}\) , and with \( \mu\to i\mu\) denoting the rotated version, \( \mathbb R\to i\mathbb R\) . If \( g,h\) are not independent, this formula does not hold.
Proof
We already know that the first assertion holds, as explained in the above. As for the second assertion, this follows by carefully examining the above computation. Indeed, we have used only at one point the independence of \( g,h\) , so for the formula \( \mu_f=\mu_g*i\mu_h\) to hold, the equality used at that point, which is as follows, must hold:
But this is the same as saying that the following must hold, for any \( x,y\) :
We conclude that, in order for the decomposition formula \( \mu_f=\mu_g*i\mu_h\) to hold, the real and imaginary parts \( g,h:X\to\mathbb R\) must be independent, as stated.
Many other things can be said, along the same lines, inspired by the basic theory of the complex numbers. Indeed, what we used in the above was the fact that any complex number decomposes as \( z=x+iy\) with \( x,y\in\mathbb R\) , but at a more advanced level, we can equally use formulae of type \( z=re^{it}\) , or \( |z|^2=z\bar{z}\) and so on, and we are led in this way to a whole collection of results, connecting real and complex probability theory.
Going now straight to the point, probabilistic limiting theorems, let us discuss the complex analogue of the CLT. We have the following statement, to start with:
Theorem 60
Given discrete complex variables \( f_1,f_2,f_3,…\) whose real and imaginary parts are i.i.d., centered, and with common variance \( t>0\) , we have
with \( n\to\infty\) , in moments, where \( C_t\) is the law of a complex variable whose real and imaginary parts are independent, and each following the law \( g_t\) .
Proof
This follows indeed from the real CLT, established in Theorem 4.14, simply by taking the real and imaginary parts of all the variables involved.
It is tempting at this point to call Theorem 4.21 the complex CLT, or CCLT, but before doing that, let us study a bit more all this. We would like to have a better understanding of the limiting law \( C_t\) at the end, and for this purpose, let us look at a sum as follows, with \( a,b\) being real independent variables, both following the normal law \( g_t\) :
To start with, this variable is centered, in a complex sense, because we have:
Regarding now the variance, things are more complicated, because the usual variance formula from the real case, which is \( V(c)=E(c^2)\) in the centered case, will not provide us with a positive number, in the case where our variable is not real. So, in order to have a variance which is real, and positive too, we must rather use a formula of type \( V(c)=E(|c|^2)\) , in the centered case. And, with this convention for the variance, we have then the following computation, for the variance of the above variable \( c\) :
But this suggests to divide everything by \( \sqrt{2}\) , as to have in the end a variable having complex variance \( t\) , in our sense, and we are led in this way into:
Definition 22
The complex normal, or Gaussian law of parameter \( t>0\) is
where \( a,b\) are real and independent, each following the law \( g_t\) .
In short, the complex normal laws appear as natural complexifications of the real normal laws. As in the real case, these measures form convolution semigroups:
Proposition 13
The complex Gaussian laws have the property
for any \( s,t>0\) , and so they form a convolution semigroup.
Proof
This follows indeed from the real result, namely \( g_s*g_t=g_{s+t}\) , established in Theorem 4.9, simply by taking real and imaginary parts.
We have as well the following complex analogue of the CLT:
Theorem 61 (CCLT)
Given discrete complex variables \( f_1,f_2,f_3,…\) whose real and imaginary parts are i.i.d. and centered, and having variance \( t>0\) , we have
with \( n\to\infty\) , in moments.
Proof
This follows indeed from our previous CCLT result, from Theorem 4.21, by dividing everything by \( \sqrt{2}\) , as explained in the above.
Finally, in relation with groups, we have the following analogue of Fact 4.18:
Fact 6
For the unitary group \( U_N\) , the main character
follows with \( N\to\infty\) the normal law \( G_1\) . More generally, the truncated character
follows with \( N\to\infty\) the normal law \( G_t\) , for any \( t\in(0,1]\) .
Which looks of course very good and conceptual, but as before with Fact 4.18, when it comes to prove such things, this is no easy business. More on this later in this book.
Regarding now the moments, the situation here is more complicated than in the real case, because in order to have good results, we have to deal with both the complex variables, and their conjugates. Let us formulate the following definition:
Definition 23
The moments a complex variable \( f\in L^\infty(X)\) are the numbers
depending on colored integers \( k=\circ\bullet\bullet\circ…\,\) , with the conventions
and multiplicativity, in order to define the colored powers \( f^k\) .
As an illustration for this notion, which is something very intuitive, here are the formulae of the four possible order 2 moments of a complex variable \( f\) :
Observe that, since \( f,\bar{f}\) commute, we have the following identity, which shows that there is a bit of redundancy in our above definition, as formulated:
In fact, again since \( f,\bar{f}\) commute, we can permute terms, in the general context of Definition 4.26, and restrict the attention to exponents of the following type:
However, our results about the complex Gaussian laws, and other complex laws, later on, not to talk about laws of matrices, random matrices and other noncommuting variables, that will appear later too, will look better without doing this. So, we will use Definition 4.26 as stated. Getting to work now, we first have the following result:
Theorem 62
The moments of the complex normal law are given by
where \( k=\circ\bullet\bullet\circ…\) is called uniform when it contains the same number of \( \circ\) and \( \bullet\) .
Proof
We must compute the moments, with respect to colored integer exponents \( k=\circ\bullet\bullet\circ…\) as above, of the variable from Definition 4.22, namely:
We can assume that we are in the case \( t=1\) , and the proof here goes as follows:
(1) As a first observation, in the case where our exponent \( k=\circ\bullet\bullet\circ…\) is not uniform, a standard rotation argument shows that the corresponding moment of \( f\) vanishes. To be more precise, the variable \( f'=wf\) is complex Gaussian too, for any complex number \( w\in\mathbb T\) , and from \( M_k(f)=M_k(f')\) we obtain \( M_k(f)=0\) , in this case.
(2) In the uniform case now, where the exponent \( k=\circ\bullet\bullet\circ…\) consists of \( p\) copies of \( \circ\) and \( p\) copies of \( \bullet\) , the corresponding moment can be computed as follows:
(3) In order to finish now the computation, let us recall that we have the following formula, coming from the generalized binomial formula, or from the Taylor formula:
By taking the square of this series, we obtain the following formula:
Now by looking at the coefficient of \( t^p\) on both sides, we conclude that the sum on the right equals \( 4^p\) . Thus, we can finish the moment computation in (2), as follows:
We are therefore led to the conclusion in the statement.
Before going further, let us record the following consequence, of the above:
Theorem 63
The moments of the Rayleigh law, given by
are given by the following formula, at the parameter value \( t=1\) ,
and are given by the formula \( M_p=t^pp!\) , in general.
Proof
This follows indeed from Theorem 4.27, or simply from the computation (2) in the above proof, at \( t=1\) , and then by rescaling by \( t\) , in general.
Many other things can be said about the Rayleigh laws, which are quite interesting mathematical objects, notably with some further formulae, regarding the Fourier transform, and the cumulants, which can be obtained as a consequence of Theorem 4.28. We will be back to these remarkable laws, on several occasions, in what follows.
Let us record, however, the following statement, called Rayleigh Central Limiting Theorem, which is something of theoretical importance:
Theorem 64 (RCLT)
Given discrete complex variables \( f_1,f_2,f_3,…\) whose real and imaginary parts are i.i.d. and centered, and having variance \( t>0\) , we have
with \( n\to\infty\) , in moments.
Proof
This follows indeed from our previous Central Limiting result, namely the CCLT from Theorem 4.24, by taking absolute values on both sides.
As a comment here, observe that the Rayleigh laws, while being certainly real, capture the essentials of the two-dimensional nature of the normal laws. So, probably time now for some further thinking, in relation with what cat and kitty were saying. But with this being something a bit complicated, and with our advisors being not really available, one with a hangover, and the other chasing mice, we will not further explore this subject.
Getting back now to the complex normal laws, what we have in Theorem 4.27 about them is usually what is needed in practice, when dealing with moments. But, as before with the real Gaussian laws, or even further before with the Poisson and Bessel laws, a better-looking statement regarding the moments is in terms of partitions.
Indeed, given a colored integer \( k=\circ\bullet\bullet\circ…\,\) , let us say that \( \pi\in P_2(k)\) is matching when it pairs \( \circ-\bullet\) symbols. With this convention, we have the following result:
Theorem 65
The moments of the complex normal law are the numbers
where \( \mathcal P_2(k)\) are the matching pairings of \( \{1,…,k\}\) , and \( |.|\) is the number of blocks.
Proof
This is a reformulation of Theorem 4.27. Indeed, we can assume that we are in the case \( t=1\) , and here we know from Theorem 4.27 that the moments are:
On the other hand, the numbers \( |\mathcal P_2(k)|\) are given by exactly the same formula. Indeed, in order to have a matching pairing of \( k\) , our exponent \( k=\circ\bullet\bullet\circ…\) must be uniform, consisting of \( p\) copies of \( \circ\) and \( p\) copies of \( \bullet\) , with \( p=|k|/2\) . But then the matching pairings of \( k\) correspond to the permutations of the \( \bullet\) symbols, as to be matched with \( \circ\) symbols, and so we have \( p!\) such pairings. Thus, we have the same formula as for the moments of \( f\) , and we are led to the conclusion in the statement.
In practice, we also need to know how to compute joint moments. We have here:
Theorem 66 (Wick formula)
Given independent variables \( f_i\) , each following the complex normal law \( G_t\) , with \( t>0\) being a fixed parameter, we have the formula
where \( k=k_1… k_s\) and \( i=i_1… i_s\) , for the joint moments of these variables, where \( \pi\leq\ker i\) means that the indices of \( i\) must fit into the blocks of \( \pi\) , in the obvious way.
Proof
This is something well-known, which can be proved as follows:
(1) Let us first discuss the case where we have a single variable \( f\) , which amounts in taking \( f_i=f\) for any \( i\) in the formula in the statement. What we have to compute here are the moments of \( f\) , with respect to colored integer exponents \( k=\circ\bullet\bullet\circ…\,\) , and the formula in the statement tells us that these moments must be:
But this is the formula in Theorem 4.30, so we are done with this case.
(2) In general now, when expanding the product \( f_{i_1}^{k_1}… f_{i_s}^{k_s}\) and rearranging the terms, we are left with doing a number of computations as in (1), and then making the product of the expectations that we found. But this amounts in counting the partitions in the statement, with the condition \( \pi\leq\ker i\) there standing for the fact that we are doing the various type (1) computations independently, and then making the product.
The above statement is one of the possible formulations of the Wick formula, and there are many more formulations, which are all useful. For instance, we have:
Theorem 67 (Wick formula 2)
Given independent variables \( f_i\) , each following the complex normal law \( G_t\) , with \( t>0\) being a fixed parameter, we have the formula
for the non-vanishing joint moments of these variables.
Proof
This follows from the usual Wick formula, from Theorem 4.31. With some changes in the indices and notations, the formula there reads:
Now observe that we have \( \mathcal P_2(K)=\emptyset\) , unless the colored integer \( K=K_1… K_s\) is uniform, in the sense that it contains the same number of \( \circ\) and \( \bullet\) symbols. Up to permutations, the non-trivial case, where the moment is non-vanishing, is the case where the colored integer \( K=K_1… K_s\) is of the following special form:
So, let us focus on this case, which is the non-trivial one. Here we have \( s=2k\) , and we can write the multi-index \( I=I_1… I_s\) in the following way:
With these changes made, the above usual Wick formula reads:
The point now is that the matching pairings \( \sigma\in\mathcal P_2(K)\) , with \( K=\circ…\circ\bullet…\bullet\,\) , of length \( 2k\) , as above, correspond to the permutations \( \pi\in S_k\) , in the obvious way. With this identification made, the above modified usual Wick formula becomes:
Thus, we have reached to the formula in the statement, and we are done.
Finally, here is one more formulation of the Wick formula, useful as well:
Theorem 68 (Wick formula 3)
Given independent variables \( f_i\) , each following the complex normal law \( G_t\) , with \( t>0\) being a fixed parameter, we have the formula
for the non-vanishing joint moments of these variables.
Proof
This follows from our second Wick formula, from Theorem 4.32, simply by permuting the terms, as to have an alternating sequence of plain and conjugate variables. Alternatively, we can start with Theorem 4.31, and then perform the same manipulations as in the proof of Theorem 4.32, but with the exponent being this time as follows:
Thus, we are led to the conclusion in the statement.
Many other things can be said about the normal laws, and also about Rayleigh laws. We will be back to this, notably in chapter 15 below, when doing random matrices.
Strange and exciting chapter that we had here, with our explorations involving ships, drunken sailors and mysterious cats, and as exercises on all this, we have:
Exercise 25
Do some abstract simulations, of your choice, for the CLT.
Exercise 26
Do as well some computer simulations for the CLT.
Exercise 27
Prove that the density is even when the odd moments vanish.
Exercise 28
Find a genius method for dealing with the Gauss integral.
Exercise 29
How to best integrate \( g_t\) , over a bounded interval \( [a,b]\) ?
Exercise 30
Learn more about the Rayleigh laws, and their properties.
Exercise 31
Work out our RCLT program, outlined in the above.
Exercise 32
Do some probabilistic computations for \( O_N,U_N\) .
As bonus exercise, learn some systematic probability, say from probabilists, or from physicists. Be aware tough that the books there are also based on some cat knowledge.
And it’s no nay never
No nay never no more
Will I play the wild rover
No never no more
Welcome to measure theory, take two. We have seen in Part I a lot of interesting mathematics, mostly of probabilistic nature, coming as a complement to the basic calculus that you surely know, and often pushing us into developing measure theory.
So, this is what we will do here, in this Part II, developing mesure theory, in a very standard, rigorous and mathematical way. Then in Part III we will talk about function spaces, by using measure theory tools, again in a very standard way, and with this being traditionally part of measure theory too. In short, expect now 200 pages of standard mathematics, which will be something useful, improving your knowledge of calculus.
As for mysterious probability, and other strange manipulations involving \( \infty\) , in case you got addicted to that, do not worry, we will be back to this in Part IV, that time with even more exciting considerations, mixing measure theory, math and physics.
In short, you got it, this book is concieved a bit like a sandwich, with philosophical buns and meat content, so time to get into the meat. Getting started now, before even talking measures, we need spaces. And here, we have the following question:
Question 7
What is a space?
And good question this is. There are of course a myriad answers to it, but remembering that we are analysts, let us consider instead a more modest question, as follows:
Question 8
What are the spaces where we can do analysis?
But, this looks again like an excessively general question. Indeed, examples of such spaces include for instance \( \mathbb R,\mathbb C\) , then the powers \( \mathbb R^N,\mathbb C^N\) , and then all sorts of curves, surfaces and so on \( X\subset\mathbb R^N,X\subset\mathbb C^N\) . And with the story being not over here, because we should normally be able to allow \( N=\infty\) too in all this. Plus, there might be other spaces \( X\) where we can do analysis, which do not necessarily embed into \( \mathbb R^N,\mathbb C^N\) .
In order to see more clearly what is going on, let us forget about spaces, and think instead at what doing analysis means. And here, for doing analysis, we obviously need convergence, with the usual definition of convergence being as follows:
Long story short, for doing analysis on a space \( X\) , we need a distance on \( X\) . Thus, as answer to Question 5.2, which is something very intuitive, we have:
Answer 4
In order to be able to do analysis on \( X\) , we need a distance on \( X\) :
Indeed, once we have \( d\) , we can talk about convergence, then continuity and so on.
Very good all this, and the question is now, what are the exact axioms on \( (X,d)\) . But these can only be the usual properties satisfied by the distances on \( \mathbb R^N,\mathbb C^N\) , and a bit of thinking here leads to the following definition, fine-tuning Answer 5.3:
Definition 24
A metric space is a set \( X\) with a distance function \( d:X\times X\to\mathbb R_+\) , having the following properties:
As a basic example, we have \( \mathbb R^N\) , as well as any of its subsets \( X\subset\mathbb R^N\) . Indeed, the first two axioms are clear, and for the third axiom, we must prove that:
Now by raising to the square, this is the same as proving that:
But this latter inequality is one of the many equivalent formulations of the Cauchy-Schwarz inequality, that we know well from calculus, and which follows by using the fact that \( f(t)=\sum_i(a_i+tb_i)^2\) being positive, its discriminant must be negative.
As another example, we have \( \mathbb C^N\) , as well as any of its subsets \( X\subset\mathbb C^N\) . Indeed, this follows either from \( \mathbb C^N\simeq\mathbb R^{2N}\) , or directly, along the lines of the above proof for \( \mathbb R^N\) . To be more precise, after some algebra, we are left with proving the following inequality:
But this is the complex version of the Cauchy-Schwarz inequality, that we know well from calculus, and which follows by using the fact that the function \( f(t)=\sum_i|a_i+twb_i|^2\) with \( t\in\mathbb R\) and \( |w|=1\) being positive, its discriminant must be negative.
Here is now another example, which at first looks new and interesting, but is in fact not new, because it appears as a subspace of a suitable \( \mathbb R^N\) :
Proposition 14
Given a finite set \( X\) , the following function is a metric on it, called discrete metric:
This metric space is in fact the \( N\) -simplex, which can be realized as a subspace of \( \mathbb R^{N-1}\) , or, more conveniently, as a subspace of \( \mathbb R^N\) .
Proof
There are several things going on here, the idea being as follows:
(1) First of all, the axioms from Definition 5.4 are trivially satisfied, and with the main axiom, namely the triangle inequality, basically coming from:
(2) At the level of examples, at \( |X|=1\) we obtain a point, at \( |X|=2\) we obtain a segment, at \( |X|=3\) we obtain an equilateral triangle, at \( |X|=4\) we obtain a regular tetrahedron, and so on. Thus, what we have in general, at \( |X|=N\) , is the arbitrary dimensional generalization of this series of geometric objects, called \( N\) -simplex.
(3) In what regards now the geometric generalization of the \( N\) -simplex, our above examples, namely segment, triangle, tetrahedron and so on, suggest to look for an embedding \( X\subset\mathbb R^{N-1}\) . Which is something which is certainly possible, but the computations here are quite complicated, involving a lot of trigonometry, as you can check yourself by studying the problem at \( N=4\) , that is, parametrizing the regular tetrahedron in \( \mathbb R^3\) .
(4) However, mathematics, or perhaps physics come to the rescue, via the idea “add a dimension, for getting smarter”. Indeed, when looking for an embedding \( X\subset\mathbb R^N\) things drastically simplify, because we can simply take \( X\) to be the standard basis of \( \mathbb R^N\) :
Indeed, we have by definition \( d(e_i,e_j)=1\) for any \( i\neq j\) . So, we have solved our embedding problem, just like that, without doing any computations or trigonometry.
Moving ahead now with some theory, and allowing us a bit of slopiness, we have:
Proposition 15
We can talk about limits inside metric spaces \( X\) , by saying that
and we can talk as well about continuous functions \( f:X\to Y\) , by requiring that
and with these notions in hand, all the basic results from the cases \( X=\mathbb R,\mathbb C\) extend.
Proof
All this is very standard, and we will leave this as an exercise, namely carefully checking what you learned in basic calculus, in relation with limits and continuity, in the cases \( X=\mathbb R,\mathbb C\) , and working out the metric space extensions of this.
More interestingly now, we can talk about open and closed sets inside metric spaces \( X\) , again in analogy with what we did for \( X=\mathbb R,\mathbb C\) , but with a whole lot of interesting new phenomena appearing. So, we will do this in detail. Let us start with:
Definition 25
Let \( X\) be a metric space.
At the level of examples, you can quickly convince yourself, by working out a few of them, that our notions above coincide with the usual ones, that we know well, in the case \( X=\mathbb R,\mathbb C\) . We will be back to this later, with some general results in this sense, confirming all this. But for the moment, let us work out the basics. We first have the following result, clarifying some terminology issues from Definition 5.7:
Proposition 16
The open balls are open, and the closed balls are closed.
Proof
This might sound like a joke, but it is not one, because this is the kind of thing that we have to check. Fortunately, all this is elementary, as follows:
(1) Given an open ball \( B_x(r)\) and a point \( y\in B_x(r)\) , by using the triangle inequality we have \( B_y(r')\subset B_x(r)\) , with \( r'=r-d(x,y)\) . Thus, \( B_x(r)\) is indeed open.
(2) Given a closed ball \( \bar{B}_x(r)\) and a point \( y\in B_x(r)^c\) , by using the triangle inequality we have \( B_y(r')\subset B_x(r)^c\) , with \( r'=d(x,y)-r\) . Thus, \( \bar{B}_x(r)\) is indeed closed.
Here is now something more interesting, making the link with our intuitive understanding of the notion of closedness, coming from our experience so far with analysis:
Theorem 69
For a subset \( F\subset X\) , the following are equivalent:
Proof
We can prove this by double implication, as follows:
\( (1)\implies(2)\) Assume by contradiction \( x_n\to x,x_n\in F\) with \( x\notin F\) . Since we have \( x\in F^c\) , which is open, we can pick a ball \( B_x(r)\subset F^c\) . But this contradicts our convergence assumption \( x_n\to x\) , so we are done with this implication.
\( (2)\implies(1)\) Assume by contradiction that \( F\) is not closed in our sense, meaning that \( F^c\) is not open. Thus, we can find \( x\in F^c\) such that there is no ball \( B_x(r)\subset F^c\) . But with \( r=1/n\) this provides us with a point \( x_n\in B_x(1/n)\cap F\) , and since we have \( x_n\to x\) , this contradicts our assumption (2). Thus, we are done with this implication too.
Here is another basic theorem about open and closed sets:
Theorem 70
Let \( X\) be a metric space.
Moreover, both \( (3)\) and \( (4)\) can fail for infinite intersections and unions.
Proof
We have several things to be proved, the idea being as follows:
(1) This is clear from definitions, because any point \( x\in\cup_iU_i\) must satisfy \( x\in U_i\) for some \( i\) , and so has a ball around it belonging to \( U_i\) , and so to \( \cup_iU_i\) .
(2) This follows from (1), by using the following well-known set theory formula:
(3) Given an arbitrary point \( x\in\cap_iU_i\) , we have \( x\in U_i\) for any \( i\) , and so we have a ball \( B_x(r_i)\subset U_i\) for any \( i\) . Now with this in hand, let us set:
As a first observation, this is a ball around \( x\) , \( B=B_x(r)\) , of radius given by:
But this ball belongs to all the \( U_i\) , and so belongs to their intersection \( \cap_iU_i\) . We conclude that the intersection \( \cap_iU_i\) is open, as desired.
(4) This follows from (3), by using the following well-known set theory formula:
(5) Finally, in what regards the counterexamples at the end, these can be both found on \( \mathbb R\) , and we will leave this as an instructive exercise.
Still in relation with the open and closed sets, let us formulate as well:
Definition 26
Let \( X\) be a metric space, and \( E\subset X\) be a subset.
These notions are quite interesting, because they make sense for any set \( E\) . That is, when \( E\) is open, that is open and end of the story, and when \( E\) is closed, that is closed and end of the story. In general, however, a set \( E\subset X\) is not open or closed, and what we can best do to it, in order to study it with our tools, is to “squeeze” it, as follows:
In practice now, in order to use the above notions, we need to know a number of things, including that fact that \( E\) open implies \( E^\circ=E\) , the fact that \( E\) closed implies \( \bar{E}=E\) , and many more such results. But all this can be done, and the useful statement here, summarizing all that we need to know about interiors and closures, is as follows:
Theorem 71
Let \( X\) be a metric space, and \( E\subset X\) be a subset.
Proof
We have several things to be proved, the idea being as follows:
(1) Let us first prove that the interior \( E^\circ\) is open. For this purpose, pick \( x\in E^\circ\) . We know that we have a ball \( B_x(r)\subset E\) , and since this ball is open, it follows that we have \( B_x(r)\subset E^\circ\) . Thus, the interior \( E^\circ\) is open, as claimed.
(2) Let us prove now that the closure \( \bar{E}\) is closed. For this purpose, we will prove that the complement \( \bar{E}^c\) is open. So, pick \( x\in\bar{E}^c\) . Then \( x\) cannot appear as a limit of a sequence \( x_n\to x\) with \( x_n\in E\) , so we have a ball \( B_x(r)\subset \bar{E}^c\) , as desired.
(3) Finally, the maximality and minimality assertions regarding \( E^\circ\) and \( \bar{E}\) are both routine too, coming from definitions, and we will leave them as exercises.
As a continuation of this, we can talk as well about density, as follows:
Definition 27
We say that a subset \( E\subset X\) is dense when:
That is, any point of \( X\) must appear as a limit of points of \( E\) .
Obviously, this is something which is in tune with what we know so far from this book, and with the intuitive notion of density. As a basic example, we have \( \bar{\mathbb Q}=\mathbb R\) .
Again in analogy with what we know about \( X=\mathbb R,\mathbb C\) , we can talk about compact sets. However, things here are quite tricky, in the general metric space framework, substantially deviating from what we know, and we will do this in detail. Let us start with:
Definition 28
A set \( K\subset X\) is called compact if any cover with open sets
has a finite subcover, \( K\subset(U_{i_1}\cup…\cup U_{i_n})\) .
This definition, which is probably new to you, might seem overly abstract, but our claim is that this is the correct definition, and that there is no way of doing otherwise. Let us start with some examples, with \( X=\mathbb R\) . The situation here is as follows:
(1) A point is obviously compact, and we can choose that finite subcover with \( n=1\) . Similarly, 2 points are compact, and we can choose the subcover with \( n=2\) . More generally, \( N\) points are compact, and we can choose the subcover with \( n=N\) .
(2) In contrast, the set \( \mathbb N\subset\mathbb R\) is not compact, because we can cover it with of a suitable union of small open intervals around each point, and this open cover has no finite subcover. For the same reasons, the set \( \{1/n|n\in\mathbb N\}\) is not compact either.
(3) However, and here comes an interesting point, the following set is compact:
Indeed, any open cover of it \( \cup_iU_i\) has to cover 0, and by selecting an open set \( U_i\) covering 0, this set \( U_i\) will cover the whole \( K\) , except for finitely many points, due to \( 1/n\to0\) . But these finitely points left, say \( N\) of them, can be covered by suitable sets \( U_{i_1},…,U_{i_N}\) , and by adding this family to the set \( U_i\) , we have our finite subcover.
As a conclusion to this, Definition 5.14 seems to be in tune with what we know about the compact subsets \( K\subset\mathbb R\) , namely that these are the sets which are closed and bounded. However, and here comes our point, such things are wrong in general, due to:
Proposition 17
Given an infinite set \( X\) with the discrete distance on it, namely \( d(p,q)=1-\delta_{pq}\) , which can be modeled as the basis of a suitable Hilbert space,
this set is closed and bounded, but not compact.
Proof
Here the first part, regarding the modeling of \( X\) , that we will actually not need here, is something that we already know. Regarding now the second part:
(1) \( X\) being the total space, it is closed. In fact, since the points of \( X\) are open, any subset \( E\subset X\) is open, and by taking complements, any set \( E\subset X\) is closed as well.
(2) \( X\) is also bounded, because all distances are smaller than \( 1\) .
(3) However, our set \( X\) is not compact, because its points being open, as noted above, \( X=\cup_{x\in X}\{x\}\) is an open cover, having no finite subcover.
Let us develop now the theory of compact sets, and see what we get. We first have:
Proposition 18
The following hold:
Proof
These assertions are all clear from definitions, as follows:
(1) Assume that \( K\subset X\) is compact, and let us prove that \( K\) is closed. For this purpose, we will prove that \( K^c\) is open. So, pick \( p\in K^c\) . For any \( q\in K\) we set \( r=d(p,q)/3\) , and we consider the following balls, separating \( p\) and \( q\) :
We have then \( K\subset\cup_{q\in K}V_q\) , so we can pick a finite subcover, as follows:
With this done, consider the following intersection:
This intersection is then a ball around \( p\) , and since this ball avoids \( V_{q_1},…,V_{q_n}\) , it avoids the whole \( K\) . Thus, we have proved that \( K^c\) is open at \( p\) , as desired.
(2) Assume that \( F\subset K\) is closed, with \( K\subset X\) being compact. For proving our result, we can assume, by replacing \( X\) with \( K\) , that we have \( X=K\) . In order to prove now that \( F\) is compact, consider an open cover of it, as follows:
By adding the set \( F^c\) , which is open, to this cover, we obtain a cover of \( K\) . Now since \( K\) is compact, we can extract from this a finite subcover \( \Omega\) , and there are two cases:
– If \( F^c\in\Omega\) , by removing \( F^c\) from \( \Omega\) we obtain a finite cover of \( F\) , as desired.
– If \( F^c\notin\Omega\) , we are done too, because in this case \( \Omega\) is a finite cover of \( F\) .
(3) This follows from (1) and (2), because if \( K\subset X\) is compact, and \( F\subset X\) is closed, then \( K\cap F\subset K\) is closed inside a compact, so it is compact.
As a second batch of results, which are useful as well, we have:
Proposition 19
The following hold:
Proof
Again, these are elementary results, which can be proved as follows:
(1) Assume by contradiction \( \cap_iK_i=\emptyset\) , and let us pick \( K_1\in\{K_i\}\) . Since any \( x\in K_1\) is not in \( \cap_iK_i\) , there is an index \( i\) such that \( x\in K_i^c\) , and we conclude that we have:
But this can be regarded as being an open cover of \( K_1\) , that we know to be compact, so we can extract from it a finite subcover, as follows:
But this contradicts our non-empty intersection assumption, and we are done.
(2) This is a particular case of (1), proved above.
(3) We prove this by contradiction. So, assume that \( E\) has no limit point in \( K\) . This means that any \( p\in K\) can be isolated from the rest of \( E\) by a certain open ball \( V_p=B_p(r)\) , and in both the cases that can appear, \( p\in E\) or \( p\notin E\) , we have:
Now observe that these sets \( V_p\) form an open cover of \( K\) , and so of \( E\) . But due to \( |V_p\cap E|=0,1\) and to \( |E|=\infty\) , this open cover of \( E\) has no finite subcover. Thus the same cover, regarded now as cover of \( K\) , has no finite subcover either, contradiction.
(4) This follows from (3) that we just proved, with \( E=\{x_n\}\) .
(5) This is a reformulation of (4), that we just proved.
Getting now to more exciting theory, here is a key result about compactness:
Theorem 72
For a subset \( K\subset\mathbb R^N\) , the following are equivalent:
Proof
This is something quite tricky, the idea being as follows:
\( (1)\implies(2)\) As a first task, let us prove that any product of closed intervals is indeed compact. We can assume by linearity that we are dealing with the unit cube:
In order to prove that \( C_1\) is compact, we proceed by contradiction. So, assume that we have an open cover as follows, having no finite subcover:
Now let us cut \( C_1\) into \( 2^N\) small cubes, in the obvious way, over the \( N\) coordinate axes. Then at least one of these small cubes, which are all covered by \( \cup_iU_i\) too, has no finite subcover. So, let us call \( C_2\subset C_1\) one of these small cubes, having no finite subcover:
We can then cut \( C_2\) into \( 2^N\) small cubes, and by the same reasoning, we obtain a smaller cube \( C_3\subset C_2\) having no finite subcover. And so on by recurrence, and we end up with a decreasing sequence of cubes, as follows, having no finite subcover:
Now since these decreasing cubes have edge size \( 1,1/2,1/4,…\,\) , their intersection must be a point. So, let us call \( p\) this point, defined by the following formula:
But this point \( p\) must be covered by \( \cup_iU_i\) , so we can find an index \( i\) such that \( p\in U_i\) . Now observe that \( U_i\) must contain a whole ball around \( p\) , and so starting from a certain \( K\in\mathbb N\) , all the cubes \( C_k\) will be contained in this ball, and so in \( U_i\) :
But this is a contradiction, because \( C_K\) , and in fact the smaller cubes \( C_k\) with \( k>K\) as well, were assumed to have no finite subcover. Thus, we proved our claim. But with this claim in hand, the result is now clear. Indeed, assuming that \( K\subset\mathbb R^N\) is closed and bounded, we can view it as a subset as a suitable big cube, of the following form:
But, what we have here is a closed subset inside a compact set, which by Proposition 5.16 follows to be compact, as desired.
\( (2)\implies(3)\) This is something that we already know, not needing \( K\subset\mathbb R^N\) .
\( (3)\implies(1)\) We have to prove that \( K\) as in the statement is both closed and bounded, and we can do both these things by contradiction, as follows:
– Assume first that \( K\) is not closed. But this means that we can find a point \( x\notin K\) which is a limiting point of \( K\) . Now let us pick \( x_n\in K\) , with \( x_n\to x\) , and consider the set \( E=\{x_n\}\) . According to our assumption, \( E\) must have a limiting point in \( K\) . But this limiting point can only be \( x\) , which is not in \( K\) , contradiction.
– Assume now that \( K\) is not bounded. But this means that we can find points \( x_n\in K\) satisfying \( ||x_n||\to\infty\) , and if we consider the set \( E=\{x_n\}\) , then again this set must have a limiting point in \( K\) , which is impossible, so we have our contradiction, as desired.
So long for compactness. As a last piece of general topology, in our metric space framework, we can talk as well about connectedness, as follows:
Definition 29
We can talk about connected sets \( E\subset X\) , as follows:
All this looks a bit technical, and indeed it is. To start with, (1) is something quite natural, but the separation condition there \( E_1\cap\bar{E}_2=\bar{E}_1\cap E_2=\emptyset\) can be weakened into \( E_1\cap E_2=\emptyset\) , or strengthened into \( \bar{E}_1\cap\bar{E}_2=\emptyset\) , depeding on purposes, and with our (1) as formulated being the good compromise, for most purposes. As for (2), this condition is obviously something stronger, and we have in fact the following implications:
Anyway, leaving aside the discussion here, once all these questions clarified, the idea is that any set \( E\) can be written as a disjoint union of connected components:
Getting back now to more concrete things, remember that we are here in this book for studying functions, and doing calculus. And, regarding functions, we have:
Theorem 73
Assuming that \( f:X\to Y\) is continuous, the following happen:
Proof
This is something fundamental, which can be proved as follows:
(1) This is clear from the definition of continuity, written with \( \varepsilon,\delta\) . In fact, the converse holds too, in the sense that if \( f^{-1}({\rm {open}})={\rm open}\) , then \( f\) must be continuous.
(2) This follows from (1), by taking complements. And again, the converse holds too, in the sense that if \( f^{-1}({\rm {closed}})={\rm closed}\) , then \( f\) must be continuous.
(3) Indeed, given an open cover \( f(K)\subset\cup_iU_i\) , we have an open cover \( K\subset\cup_if^{-1}(U_i)\) , and so by compactness of \( K\) , a finite subcover \( K\subset f^{-1}(U_{i_1})\cup…\cup f^{-1}(U_{i_n})\) , and so finally a finite subcover \( f(K)\subset U_{i_1}\cup…\cup U_{i_n}\) , as desired.
(4) This can be proved via the same trick as for (3). Indeed, any separation of \( f(E)\) into two parts can be returned via \( f^{-1}\) into a separation of \( E\) into two parts, contradiction.
Observe the power of (3,4) in the above result, which among others prove the mean value theorem, which is something non-trivial. Good mathematics that we have here.
We are now ready to attack measure theory. We first need measurable sets, and on the bottom line, we would like our open and closed sets, and their “combinations”, to be measurable. But these “combinations” can be understood and axiomatized, and are called Borel sets. So, this will be the idea in what follows, talking about abstract measurable sets, then about Borel sets, and then coming with measures, and integration theory.
Let us start with the abstract measurable sets. We have here the following definition, which is something very general, not making reference to any metric on our space \( X\) , nor making reference to any measure on \( X\) , measuring these measurable sets:
Definition 30
An abstract measured space is a set \( X\) , given with a set of subsets \( M\subset P(X)\) , called measurable sets, which form an algebra, in the sense that:
Obviously, this is something quite abstract. If the last axiom, (3), is only satisfied for the finite unions and intersections, we say that \( M\subset P(X)\) is a finite algebra.
As a first observation, some of the axioms above are redundant. Indeed, assuming that (2) holds, we have the following equivalence, which can help in verifying (1):
The same goes for the axiom (3), with only one of the conditions there being in need to be verified, in practice, and this due to the following formula:
However, it is most convenient to write Definition 5.21 as above, in a symmetric way with respect to the two operations involved, namely union and intersection.
At the level of examples of abstract measured spaces, there are many of them, and more on this in a moment, usually coming via the following result:
Theorem 74
Given a set of subsets \( S\subset P(X)\) , there is a smallest algebra
containing it, called algebra generated by \( X\) .
Proof
This can be viewed in two possible ways, as follows:
(1) According to the axioms in Definition 5.21, what we have to do in order to construct \( M=\bar{S}\) is to start with \( S\) , then add \( \emptyset,X\) to it, along with the complements \( E^c\) of all the sets \( E\in S\) , and then take countable unions and intersections of such sets. And, some elementary verifications show that what we get in this way is indeed an algebra.
(2) Alternatively, we can define \( M=\bar{S}\) as being the intersection of all algebras containing \( S\) , and with the remark that we have at least one such algebra, namely \( P(X)\) itself. It is then clear from definitions that \( M\) is an algebra, as desired.
Getting now to the concrete examples of abstract measured spaces, we have:
Definition 31
Any metric space \( X\) is automatically an abstract measured space, with the algebra of measurable sets being
that is, the smallest algebra containing the open sets, called Borel algebra of \( X\) .
Observe that the Borel sets include all open sets, all closed sets, as well as all countable unions of closed sets, and all countable intersections of open sets. As an example here, in the case \( X=\mathbb R\) , with its usual topology, all kinds of intervals are Borel sets:
Indeed, the first interval is open, and the second one is closed, so these are certainly Borel sets. As for the third and fourth intervals, these appear as countable unions of closed intervals, or as countable intersections of open intervals, so they are Borel too.
Getting back now to the general case, following Definition 5.21, we have:
Definition 32
Given a measured space \( (X,M)\) , a measure on it is a function
which is countably additive, in the sense that we have
for any countable family of disjoint measurable sets \( E_i\in M\) .
This definition, which will play a key role in what follows, is something quite tricky, and there are several comments to be made about it, as follows:
(1) Obviously, what we axiomatized above are the positive measures, and for the moment this will do, and we will omit the term “positive”. More on this later, when we will talk about differences \( \mu-\eta\) of such measures, and about complex measures too.
(2) In contrast, we did not assume that our measures are finite, and this because many interesting spaces, such as \( X=\mathbb R\) itself, are naturally of infinite measure, \( \mu(X)=\infty\) . In the case where the measure \( \mu\) happens to be bounded, up to a rescaling we can assume \( \mu(X)=1\) , and we say in this case that we have a probability measure on \( X\) .
(3) Yet another subtlety comes in relation with the countable additivity condition at the end, the point being that for many measured spaces \( X\) , the finite additivity condition is something strictly weaker, and leads to a wrong theory. More on this later.
Looking at what we have so far, Definition 5.23 and Definition 5.24, many natural questions appear, and leaving aside anything too specialized, we are led to:
Question 9
Given a metric space \( X\) , such as \( X=\mathbb R\) , or \( X=\mathbb R^N\) , how to construct a measure on it? Also, once we have such a measure, how to integrate the functions \( f:X\to\mathbb R\) , or \( f:X\to\mathbb C\) , with respect to this measure?
As you can see, we have two questions here, and none is trivial:
(1) Indeed, regarding the first question, this is something that we do not know yet how to solve, even this even for very simple spaces like \( X=\mathbb R\) . Indeed, while we certainly know how to measure the real intervals, simply by setting \( \mu(a,b)=b-a\) , and then unions of such intervals too, by decomposing then into disjoint unions of intervals, and making sums, nothing guarantees that we can measure any Borel set \( E\in B\) , in this way.
(2) As for the second question, experience with the usual Riemann integral, that you know well from calculus, shows that such things can be quite tricky too. Observe also that this second question is more general than the first one, because we have \( \mu(E)=\int\chi_E\) for any \( E\in M\) , so if we know how to integrate, we know how to measure.
In short, we are facing non-trivial questions here, and we need a plan. And, perhaps a bit surprisingly, the best plan in order to deal with Question 5.25 is as follows:
Plan 2
We will jointly develop measure and integration theory, as follows:
As you can see, all this is quite tricky, the main idea behind this plan being functional analysis, that is, using functions and their integrals, instead of just fighting with abstract measure theory first, and looking at functions and their integrals afterwards.
Getting started now, let us first talk about measurable functions, in the general context of Definition 5.21, with no measure involved. We have here the following notion:
Definition 33
Given a measured space \( X\) , and a metric space \( Y\) , a function
is called measurable when it satisfies the following condition:
When \( X\) comes with a measure, we also call such functions integrable.
Obviously, this is something simplified, because for doing abstract measurability theory, our spaces should be abstract measured spaces in the sense of Definition 5.21, and so the functions that we should normally care about should be functions \( f:X\to Y\) , with both \( X,Y\) being abstract measured spaces. However, in view of Definition 5.23, this is more or less that what we are doing here, by restricting the attention to the target spaces \( Y\) which are metric, with this being the case that really matters.
Many things can be said about the measurable functions. We first have:
Proposition 20
The measurable functions have the following properties:
Proof
This is something very standard, the idea being as follows:
(1) This is clear from definitions, because we have:
(2) By using (1), it is enough to check that the function \( k(x)=(f(x),g(x))\) is measurable. But this follows by writing any open set \( U\subset\mathbb R^2\) as a union of rectangles. Indeed, the preminage of each rectangle \( R=I\times J\) is measurable, as shown by:
But then, with this in hand, since any open set \( U\subset\mathbb R^2\) can be written as a union of rectangles, it follows that \( k^{-1}(U)\) is measurable, as desired.
(3) This follows indeed by using (2), with the following functions:
(4) In the real case, \( f,g:X\to\mathbb R\) , the result follows by using (2), with:
Then, the result can be extended to the complex case, \( f,g:X\to\mathbb C\) , by using (3).
Next, we have the following useful characterization of the real measurable functions:
Proposition 21
A function \( f:X\to[-\infty,\infty]\) is mesurable precisely when
for any interval of type \( I=(a,\infty]\) , with \( a\in\mathbb R\) .
Proof
Consider the following set, which is easily seen to be an algebra:
We want to prove that \( \Omega\) contains all the open sets, and this can be done as follows:
(1) Pick \( a\in\mathbb R\) , and then pick an increasing sequence \( a_n\to a\) . We have then the following formula, which shows that we have \( [-\infty,a)\in\Omega\) :
(2) But with this in hand, we obtain, for any \( a<b\) , that we have:
(3) Now since any open set \( U\subset[-\infty,\infty]\) can be written as a union of open intervals, we conclude that we have \( U\in\Omega\) , as desired.
Getting now to limits of measurable functions, we have the following result:
Proposition 22
Given an abstract measured space \( X\) , the measurable functions \( f:X\to[-\infty,\infty]\) have the following properties:
Proof
This is again something very standard, the idea being as follows:
(1) For the function \( g=\sup_nf_n\) we can use the measurability criterion from Proposition 5.29, along with the following fact, valid for any \( a\in\mathbb R\) :
By symmetry we obtain that the function \( k=\inf_nf_n\) is measurable as well. But with these results in hand, the last assertion follows too, by using the following formula:
(2) Here the fact that \( k=\inf_nf_n\) is measurable was already proved in the above, and for \( l=\liminf_nf_n\) we can use the same argument, symmetry, or the following formula:
(3) This follows indeed from (1), or from (2), and from the following fact:
(4) This is a trivial application of (1) and (2).
(5) This follows from (4), and from the fact that if \( f\) is measurable, so is \( -f\) .
As a main result now regarding the measurable functions, which will be of key importance in what follows, we have the following statement, with the convention that a step function means a function which takes finitely many values:
Theorem 75
Given a measurable function \( f:X\to[0,\infty]\) , we can write
with \( 0\leq\varphi_1\leq\varphi_2\leq…\leq f\) , and with each \( \varphi_i\) being a measurable step function.
Proof
As a first observation, the converse holds too, thanks to the results in Proposition 5.30. Regarding now the proof, this goes as follows:
(1) First, it is clear by drawing a picture that we can approximate the identity of \( [0,\infty]\) with step functions as in the statement. That is, we can obviously find an increasing sequence of measurable step functions \( 0\leq\psi_1\leq\psi_2\leq…\leq id\) , satisfying:
(2) Now let us set \( \varphi_n=\psi_n\circ f\) . According to the limiting formula above, we have:
On the other hand, by using the measurability criterion from Proposition 5.29, it follows that our truncation functions \( \varphi_n=\psi_n\circ f\) are measurable, as desired.
Good news, with the above general theory understood, we can now integrate functions, by following the good old method of Riemann, that you know well from one-variable calculus. To be more precise, we have the following result, which is of course stated a bit informally, with some of the details being left to you, as an instructive exercise:
Theorem 76
We can integrate the measurable functions \( f:X\to\mathbb R_+\) by setting
with sup over measurable step functions, then extend this by linearity.
Proof
This is something very standard, and we will leave the clarification of all this, both precise statement, and proof, as an instructive exercise. To be more precise:
(1) We can certainly integrate the step functions \( \varphi:X\to\mathbb R_+\) , by writing each such function as a linear combination of characteristic functions, as follows:
Indeed, with this formula in hand, we can integrate our function \( \varphi\) , as follows:
The integral of step functions constructed in this way has then all the linearity and positivity properties that you might expect, and behaves well with respect to limits.
(2) Next, consider an arbitrary measurable function \( f:X\to[0,\infty]\) . We know from Theorem 5.31 that we can write this function as an increasing limit, as follows, with \( 0\leq\varphi_1\leq\varphi_2\leq…\leq f\) , and with each \( \varphi_i\) being a measurable step function:
But this suggests to define the integral of \( f\) by the formula in the statement, namely:
Indeed, we can see that the integral constructed in this way has all the linearity and positivity properties that you might expect, and behaves well with respect to limits.
More in detail now, here are some basic properties of the integrals, which are very similar to those of the Riemann integral, that you know from calculus:
Proposition 23
The integrals of measurable functions \( f:X\to\mathbb R_+\) have the following properties:
Proof
All this is indeed very standard, all routine verifications.
Summarizing, we know how to integrate the real positive functions, in our abstract measure theory setting. In general, we can use the following formula:
We can integrate as well the complex functions, by setting:
All this is, indeed, very standard, exactly as for the Riemann integral. Let us record these findings as an upgrade of Theorem 5.32, as follows, once again with the statement being a bit informal, and with some of the details being left as instructive exercises:
Theorem 77
We can integrate the measurable functions \( f:X\to\mathbb C\) by setting
with sup over measurable step functions, for \( f:X\to\mathbb R_+\) then extend this by linearity.
Proof
This follows indeed from what we already know from Theorem 5.32, and from the above discussion, exactly as for the Riemann integral.
Many other things can be said here, following Lebesgue, Fatou and others. We first have the following result, regarding the monotone convergence, due to Lebesgue:
Theorem 78 (Lebesgue)
Given an increasing sequence of measurable functions
which converges pointwise, \( f_n\to f\) , their limit is measurable, and we have
for any positive measure on \( X\) .
Proof
This is indeed something very standard, the idea being as follows:
(1) We first have the following obvious implication, showing that the sequence of integrals on the right converges, to a certain number in \( [0,\infty]\) :
Moreover, since we have \( f_n\to f\) , it follows that we have the following inequality:
(2) In order to prove now the reverse inequality, pick a measurable simple function \( 0\leq\varphi\leq f\) , pick also a number \( c\in(0,1)\) , and consider the following sets:
These sets are then measurable, we have \( E_1\subset E_2\subset…\) , and our claim is that:
Indeed, given \( x\in X\) , if \( f(x)=0\) then \( x\in E_1\) and things fine. Otherwise, we have \( f(x)>0\) , and so \( f(x)>c\varphi(x)\) , since \( c<1\) , and so \( x\in E_n\) for some \( n\) , as desired.
(3) Now observe that, with \( 0\leq\varphi\leq f\) and \( c\in(0,1)\) as above, we have:
By taking the limit of this estimate, with \( n\to\infty\) , we obtain:
Now with \( c\to1\) , we obtain from this the following estimate:
But this being true for any simple function \( 0\leq\varphi\leq f\) , we conclude that we have:
Thus we have the reverse of the estimate found in (1), which finishes the proof.
Regarding now the series of functions, again following Lebesgue, we have:
Theorem 79
Given measurable functions \( f_n:X\to[0,\infty]\) , their sum
is measurable, and we have the formula
for any positive measure on \( X\) .
Proof
This follows indeed from Theorem 5.35, applied to the partial sums.
Following now Fatou, we have as well the following key result:
Theorem 80 (Fatou)
Given measurable functions \( f_n:X\to[0,\infty]\) , their limit
is measurable, and we have the formula
for any positive measure on \( X\) .
Proof
The first assertion is something that we know, from Proposition 5.30. As for the second assertion, this can be proved by using the same trick as there, namely:
Indeed, we can apply Theorem 5.35 to the following sequence of functions:
Thus, we are led to the conclusion in the statement.
Regarding the above result of Fatou, let us mention that there are examples where the inequality can be strict, with the standard example here being as follows:
Finally, as a last basic result, due to Lebesgue again, we have:
Theorem 81 (Lebesgue)
Assuming \( f_n\to f\) pointwise, and assuming too
with \( \int g<\infty\) , the following happen:
Proof
This is something very standard, using Fatou, the idea being as follows:
(1) This follows from \( |f_n|\leq g\) , which in the limit gives \( |f|<g\) , so \( \int|f|<\infty\) .
(2) Since \( |f_n-f|\leq 2g\) , we can apply Theorem 5.37 to the following functions:
We obtain in this way the following estimate:
Now by substracting \( \int_X2g\) , this estimate gives the following formula:
We conclude that this limit must be 0, as claimed in (2).
(3) This follows indeed from (2).
Many other things can be said, along these lines, which are more specialized, and such knowledge can be very useful when doing applied probability, and statistics. For our purposes here, which will be mostly pure mathematical, with a touch of theoretical physics, the above general theory, and results of Lebesgue and Fatou, will do.
This was a very standard chapter, all fundamentals of advanced analysis, and as exercises on all this, numerous, and all mandatory, we have:
Exercise 33
Master Cauchy-Schwarz, and all its generalizations.
Exercise 34
Find the coordinates of the regular \( N\) -simplex, inside \( \mathbb R^{N-1}\) .
Exercise 35
Learn more about connectedness, and connected components.
Exercise 36
Learn about topological spaces, generalizing the metric spaces.
Exercise 37
Learn also about measure theory, on such topological spaces.
Exercise 38
Have some fun with Borel and non-Borel sets, on \( \mathbb R\) .
Exercise 39
Check all the details for our Riemann integration method.
Exercise 40
Look up and solve exercises, using Lebesgue and Fatou.
As bonus exercise, as already suggested, have a look into what probabilists and statisticians say too, about Lebesgue, Fatou and related topics. All this is good learning.
Welcome to measure theory, again. What we did in the previous chapter, while certainly useful, was not very connected to life and reality, because even for simple measured spaces like \( X=\mathbb R\) , we still don’t know what the measure \( \mu\) exactly is.
So, time to discuss such questions, following Riesz, Lebesgue and others. All this will be quite tricky and technical, and as a guiding principle, we will have:
Principle 1
The following are the same thing:
Now back to our question, constructing measures \( \mu\) on spaces like \( X=\mathbb R\) , or more generally \( X=\mathbb R^N\) , and perhaps on some other metric spaces \( X\) too, this does not look obvious to do, with bare hands. However, and here comes the point, in view of Principle 6.1, it is normally enough to know how to integrate, or equivalently, to know how to do probability, on our spaces \( X=\mathbb R\) , or more generally \( X=\mathbb R^N\) , and so on.
And here, we certainly know from calculus how to integrate over \( X=\mathbb R\) , and that is excellent knowledge, that we should normally be able to covert into systematic measure theory on \( \mathbb R\) , in particular with \( \mu\) fully constructed. As for \( X=\mathbb R^N\) , and more complicated spaces, such as various curves, surfaces and other manifolds \( X\subset\mathbb R^N\) , here we certainly have some integration knowledge too, from multivariable calculus, so with a bit of luck, we should be able to improve that knowledge, and convert it into measure theory too.
But probably too much talking, let us make a concrete plan, based on this:
Plan 3
In order to fully develop measure theory:
And afterwards, we can have a look at more complicated spaces \( X\subset\mathbb R^N\) too.
Getting to work now, what can be the “Riesz theorem”, answering (1) above? Ask Riesz, of course, or intercept a cat, and ask the cat, or simply meditate on all this, so many techniques are available to us. In practice, however, simplest is to have a look at a good measure theory book, such as Rudin [3], which reveals the following answer:
Fact 7 (Riesz theorem)
Any positive functional \( I:C_c(X)\to\mathbb R\) comes by integrating with respect to a certain measure on \( X\) ,
which is unique, modulo null sets. In addition, the following happen:
Which looks a bit scary, but quite reasonable. Indeed, the main assertion, with \( C_c(X)\) standing as usual for the continuous, compactly supported functions on \( X\) , is more or less the equivalence between (1) and (2) in our guiding Principle 6.1. As for the various properties (1,2,3), which are part of the whole story too, because they sort of tell us how to construct the measure \( \mu\) , from the knowledge of the integration \( I\) , and more on this in a moment, these are certainly very reasonable properties, that we can expect to hold.
So, very good all this, we have a concrete theorem now, waiting to be proved, and the rest will naturally come afterwards, according to Plan 6.2. Now by looking more in detail at what our theorem exactly says, we are led to the following battle plan:
Strategy 1
In order to prove the Riesz theorem, we must:
In short, and bad news here, not yet time to get to work and do (3), we have to deal with (1) first, and then especially with (2), which suspiciously reminds what mathematicans call “Lemmas", and with these having the reputation of being the worst of mathematics, unless of course you are a pure mathematician yourself.
Nevermind. As I always say to my students, one day you will have to work, and work can be quite varied, sometimes you have to move big furniture around the office, install or debug computers, fix electricity or even plumbing problems, wait for someone at the airport, calm down a colleague who gets crazy, encourage another one who starts crying, and so on. So, doing what is to be done, and we will not be scared about some Lemmas, if they are on the menu for today, we will prove Lemmas, no problem with that.
Getting started now, according to our Strategy 6.4, we first need to talk about null sets. So, let us first review the material from the previous chapter, by paying particular attention to the null sets, meaning sets of zero measure. We first have here:
Theorem 82
Any measured space can be completed,
by saturating everything with subsets of null sets.
Proof
This is something quite self-explanatory, the idea being as follows:
(1) To start with, once we have a null set, \( \mu(A)=0\) , we can declare any subset \( B\subset A\) to be measurable too, and of course of mass zero, \( \mu(B)=0\) .
(2) But then, we can add the sets \( B\) found above to all the measurable sets \( E\in M\) , and we obtain in this way a bigger algebra, \( M\subset\bar{M}\) .
(3) Finally, our original measure \( \mu\) extends from the original algebra \( M\) to the bigger algebra \( \bar{M}\) in the obvious way, according to the formula \( \mu(B)=0\) from (1).
(4) Thus, result basically proved, modulo a number of technical details, which look quite straightforward, and whose clarification we will leave, as an instructive exercise.
Along the same lines, but at the level of functions now, we have:
Theorem 83
At the level of functions on a measured space, we can talk about integration by avoiding null sets, based on the equivalence
for any positive function \( f\geq0\) , and then on \( f=g-h\) with \( g,h\geq0\) , in general.
Proof
This is indeed something very standard, coming as a functional analytic complement to Theorem 6.5, the idea with all this being as follows:
(1) To start with, we have indeed the equivalence in the statement, for any \( f\geq0\) , with this being something standard, coming from the general theory from chapter 5.
(2) Now based on this, we can develop some theory, by neglecting the null sets, which in practice means reformulating Theorem 6.5, in functional analysis terms.
(3) To be more precise, given \( (X,M,\mu)\) , we certainly know from chapter 5 how to integrate the measurable functions \( f:X\to\mathbb C\) . And the point is that, by using the equivalence in the statement, this tells us in fact how to integrate the measurable functions \( f:X\to\mathbb C\) , with respect to the saturated space \( (X,\bar{M},\mu)\) from Theorem 6.5.
(4) So, this was for the idea, and we will leave clarifying the details here, both precise statement, and few things that are needed for the proof, as an instructive exercise.
As before with Fatou and Lebesgue, many other things can be said, along these lines, which are more specialized, and such knowledge can be very useful when doing applied probability, and statistics. For our purposes here, the above general theory will do.
We will be actually back to this in Part III of this book, when doing more systematically functional analysis, which is the good framework for such things. The idea there will be indeed that of defining various spaces of functions by neglecting the null sets.
Moving now towards what we want to do, namely measuring and integrating over \( \mathbb R\) , in full generality, the idea here will be that of squeezing the arbitrary measurable sets \( E\subset X\) between compact sets \( K\subset X\) and open sets \( U\subset X\) , as follows:
In order to do this, which is something quite tricky, we will need a number of technical preliminaries, as a continuation of the material from chapter 5. First, we have:
Lemma 1
Given \( K\subset U\) , compact inside open, we can find inclusions
with the set \( V\) being open, with compact closure \( \bar{V}\) .
Proof
This is something elementary, the idea being as follows:
(1) For any \( x\in K\) pick a neighborhood \( W_x\) having compact closure. Since \( K\) is compact, we can find finitely many such neighborhoods, covering it:
With this done, consider now the set on the right, namely:
This is then an open set containing our compact set \( K\) , having compact closure, that we will use in what follows, in our constructions.
(2) In order to prove now our result, observe first that if we are in the case \( U=X\) we are done, because here we can simply take \( V=W\) . So, assume \( U\neq X\) .
(3) Given a point \( x\in U^c\) , since this point is compact, and our set \( K\) not containing it is compact as well, we can find an open set \( V_x\) having the following properties:
Now consider the following family of sets, with \( W\) being the open set in (1):
This is then a family of compact sets having empty intersection, so we can find a finite subfamily having empty intersection too. That is, we can find points \( x_i\in U^c\) such that:
(4) With this done, consider now the following open set:
Then \( V\) has compact closure, and we have the following inclusion:
Thus, \( V\) is the open set we are looking for, and we are done.
The above result, separating \( K\subset U\) , compact set inside open set, via an open set \( V\) with compact closure \( \bar{V}\) , is something quite useful, in practice. However, for our measure theory purposes here, we will need more, along the same lines.
To be more precise, under the same assumptions, \( K\subset U\) , compact inside open, we would like to come up with a continuous, compactly supported function \( f\) such that:
Which is something quite non-trivial, when thinking a bit, and so again, and with our apologies here, we are in need of some more technical preliminaries.
To be more precise, we will need the following standard notions:
Definition 34
A function \( f:X\to\mathbb R\) , with \( X\) topological space, is called:
These notions are actually quite interesting on their own, having many uses in many contexts in analysis, and as basic illustrations for them, we have:
(1) A characteristic function \( \chi_E\) is upper semicontinuous when \( E\subset X\) is closed. This follows indeed from the above definition of the upper semicontinuity.
(2) Also, a characteristic function \( \chi_E\) is lower semicontinuous when \( E\subset X\) is open. Again, this follows from definitions, or simply from (1), by using the set \( E^c\) .
(3) Observe also that a continuous function is trivially both upper and lower semicontinuous. The converse of this holds too, and more on this in a moment.
Many things can be said about the upper and lower semicontinuous functions, which some knowledge here being something quite useful, when doing in analysis in general. In what follows we will only need a handful of basic results on the subject, as follows:
Proposition 24
The upper and lower semicontinuous functions \( f:X\to\mathbb R\) have the following properties:
Proof
All this is elementary, the idea being as follows:
(1) This assertion, already mentioned in the above, follows from definitions.
(2) This assertion, also mentioned in the above, also follows from definitions.
(3) This assertion is also trivial, also coming from definitions.
(4) And this assertion is trivial too, also coming from definitions.
(5) In one sense, this is clear, as already mentioned in the above. In the other sense, assuming that \( f:X\to\mathbb R\) is both upper and lower semicontinuous, we can see that the preimage of any open interval \( (a,b)\subset\mathbb R\) is open, due to the following formula:
Now since any open set \( U\subset\mathbb R\) can be written a union of open intervals \( (a,b)\subset\mathbb R\) , it follows that \( f^{-1}(U)\) is open, and so that \( f\) is continuous, as desired.
Many other things can be said, about the upper and lower semicontinuous functions, notably with many examples. However, for our purposes here, the above will do.
Good news, we can now formulate the main technical result that we will need, for our measure theory purposes here. With the convention that the support of a function \( f\) is the closure of the set \( \{x|f(x)\neq0\}\) , this key statement is as follows:
Lemma 2 (Urysohn)
Given \( K\subset U\) , compact inside open, we can find
with \( f\) being continuous, and compactly supported.
Proof
This is something very standard, the idea being as follows:
(1) Given sets \( K\subset U\) as in the statement, by using Lemma 6.7 we can find an open set \( V_0\) , having compact closure \( \bar{V}_0\) , such that:
But then, by using again Lemma 6.7, applied this time to the inclusion \( K\subset V_0\) , we can find a second open set \( V_1\) , having compact closure \( \bar{V}_1\) , such that:
And so on, would be the idea. In practice now, by using the fact that the rational numbers are countable, we can construct in this way a whole family of open sets \( V_r\) , having compact closures \( \bar{V}_r\) , one for each rational number \( r\in[0,1]\) , such that:
(2) Time now to construct our function \( f\) . Let us set, for any \( r\in[0,1]\) rational:
Then, we can construct our function \( f\) in the following way:
It is clear then that \( f\) is lower semicontinuous, that we have \( \chi_K\leq f\leq\chi_U\) , and also that \( f\) is compactly supported, with support included in the compact set \( \bar{V}_0\) . Thus, we are almost there, and it remains to prove that \( f\) is upper semicontinuous as well.
(3) For this purpose, let us set as well, for any \( r\in[0,1]\) rational:
Then, we can construct another function \( g\) , in the following way:
It is then clear, exactly as in (2), that \( g\) is lower semicontinuous, and also that we have \( \chi_K\leq g\leq\chi_U\) , and that \( g\) is compactly supported. Of interest for us is the lower semicontinuity of \( g\) , because in order to finish the proof, it is enough to show that:
(4) So, let us prove this, \( f=g\) . By definition of the functions \( f_r\) and \( g_s\) , we have:
Thus \( f\leq g\) . Now assume that we have somewhere a strict inequality, \( f(x)<g(x)\) . Then, we can find two rational numbers \( r,s\) in between, as follows:
But the first inequality tells us that \( x\notin V_r\) , and the last inequality tells us that \( x\in\bar{V}_s\) , and this contradicts the condition in (1) on the sets \( V_r\) , as desired.
Very nice all this, but in fact, contrary to what was advertised before, we will actually need in what follows, besides the Urysohn lemma, another technical result, which is something useful too, and actually of independent interest too, as follows:
Theorem 84
Given a compact set inside a union of open sets
we can find an associated partition of unity, that is, a decomposition of type
with each \( f_i\) being continuous, supported on \( U_i\) .
Proof
This follows by using the Urysohn lemma, as follows:
(1) For any \( x\in K\) , let us pick a neighborhood \( V_x\) such that \( \bar{V}_x\subset U_i\) , for some \( i\) . Since \( K\) is compact, we can find finitely many points \( x_1,…,x_m\in K\) such that:
Now for any index \( i\in\{1,…,n\}\) let us consider the following union:
We can apply then the Urysohn lemma to the inclusion \( K_i\subset U_i\) , and we obtain in this way a continuous, compactly supported function \( g_i\) , such that:
(2) With this done, consider the following sequence of functions:
Then each \( f_i\) is continuous, supported on \( U_i\) , and we have:
(3) On the other hand, recall from the construction of \( K_i\) above that we have:
We conclude from this that we have the following formula:
Thus, we have our partition of the unity, as desired.
Good news, eventually, the above is all that we need, as set theoretic preliminaries. We can now formulate the key result in abstract measure theory, as follows:
Theorem 85 (Riesz)
Any positive functional \( I:C_c(X)\to\mathbb R\) comes by integrating with respect to a certain measure on \( X\) ,
which has the following additional properties,
and which is unique with these properties, modulo the null sets.
Proof
This is something quite long and tricky, the idea being as follows:
(1) Let us first prove the uniqueness. Assuming that our measure \( \mu\) produces \( I\) , and has the various properties in the statement, it is clear that \( \mu\) is uniquely determined by its values on the compact sets \( K\subset X\) . Thus, we must show that given two measures \( \mu_1,\mu_2\) as in the statement, and a compact set \( K\subset X\) , we have:
For this purpose, let us pick \( \varepsilon>0\) . By using the properties (1,3) in the statement, for the measure \( \mu_2\) , we can find \( K\subset U\) open such that:
Now by using the Urysohn lemma for the inclusion \( K\subset U\) , we obtain a certain continuous, compactly supported function \( f\) , such that:
But, with this choice of \( f\) , we have the following computation:
Thus \( \mu_1(K)\leq\mu_2(K)\) , and by interchanging \( \mu_1,\mu_2\) we have the reverse inequality as well, so we obtain, as desired, \( \mu_1(K)=\mu_2(K)\) , proving the uniqueness statement.
(2) With this done, let us get now to the main part, and discuss the construction of the algebra \( M\) , that we will choose to be saturated in the sense of Theorem 6.5, and of the measure \( \mu\) . What we have as data is a positive functional, as follows:
We must first measure the Borel sets \( E\subset X\) . And here, to start with, in order to measure the open sets \( U\subset X\) , the formula is straightforward, namely:
Now observe that with this definition in hand, for the open sets, we have:
We deduce that the following happens, for the open sets:
But this can serve as the definition of \( \mu\) , for all the subsets \( E\subset X\) .
(3) Regarding now the measurable sets, let us first consider the class \( N\) of subsets \( E\subset X\) which are of finite measure, \( \mu(E)<\infty\) , and satisfy the following condition:
Then, we can define the class of measurable sets \( M\subset P(X)\) as follows:
Summarizing, done with definitions, and it remains now to prove that \( M\) is an algebra, and that \( \mu\) is a measure on it, along with the other things claimed in the statement.
(4) In order to prove that \( \mu\) is indeed a measure, our first claim is that we have the following inequality, for any two open sets \( U_1,U_2\subset X\) :
In order to prove this claim, choose an arbitrary function \( g:X\to[0,1]\) , supported on \( U_1\cup U_2\) . By using Theorem 6.11 we obtain a certain partition of unity, \( f_1+f_2=1\) on the support of \( g\) , and it follows that we have the following equality:
Now by applying our integration functional \( I\) , we obtain from this:
Now since this holds for any \( g:X\to[0,1]\) as above, this proves our claim.
(5) Our second claim is that we have in fact, more generally, the following inequality, valid for any family of subsets \( E_1,E_2,E_3,…\) of our space \( X\) :
Indeed, this inequality trivially holds if \( \mu(E_i)=\infty\) for some \( i\) , so we can assume \( \mu(E_i)<\infty\) for any \( i\) . Now pick \( \varepsilon>0\) , and choose open sets \( U_i\supset E_i\) such that:
Consider also the union of these open sets \( U_i\) , chosen as above:
Now pick an arbitrary function \( f:X\to[0,1]\) , supported on this open set \( U\) . Since \( f\) has compact support, we can find a certain integer \( n\in\mathbb N\) such that:
Now by using what we found in (4), recursively, we obtain:
Since this holds for any \( f:X\to[0,1]\) as above, we deduce that we have:
Now since the number \( \varepsilon>0\) was arbitrary, this proves our claim.
(6) Our next claim, which proves in the assertion (1) in the theorem, is that any compact set \( K\subset X\) is measurable, and that we have:
In order to prove this claim, consider an arbitrary continuous function \( f:X\to[0,1]\) satisfying \( f(x)=1\) on \( K\) , then pick \( \alpha<1\) , and consider the following open set:
We have then \( K\subset U_\alpha\) , and so \( \alpha g\leq f\) , whenever a continuous function \( g:X\to[0,1]\) is supported by \( U_\alpha\) . By using this, we obtain the following estimate:
Now with \( \alpha\to 1\) , we obtain from this the following estimate:
Thus \( \mu(K)<\infty\) , as claimed, and the fact that \( K\) is measurable is clear too.
(7) Our next claim, which improves what we found in (6), is that for any compact set \( K\subset X\) we have in fact the following estimate:
In order to prove this, let us go back to the proof of (6). We know from there that for any continuous function \( f:X\to[0,1]\) satisfying \( f(x)=1\) on \( K\) , as above, we have:
Now pick \( \varepsilon>0\) , and choose an open set \( U\supset K\) satisfying:
By using Lemma 6.10, we can find a compactly supported function \( f\) such that:
But this gives the following estimate, using the above inequality \( \mu(K)\leq I(f)\) :
Now since the number \( \varepsilon>0\) was arbitrary, this proves our claim.
(8) Our claim now is that any open set \( U\subset X\) satisfies the following condition, that we used in (3) in order to define the measurable sets:
In order to prove this, pick an arbitrary number \( \alpha<\mu(U)\) . We can then find a continuous function \( f:X\to[0,1]\) supported on \( U\) , such that:
Consider now the compact set \( K=supp(f)\) . Then for any open set \( V\) containing \( K\) we have \( I(f)\leq\mu(V)\) , and we deduce from this that we have:
In other words, given \( \alpha<\mu(U)\) , we have found a compact set \( K\) such that:
Now since our number \( \alpha<\mu(U)\) was arbitrary, this proves our claim.
(9) Getting now towards the proof of the additivity of our measure \( \mu\) , let us first prove that for any two disjoint compact sets \( K_1,K_2\subset X\) , we have:
For this purpose, pick an arbitrary \( \varepsilon>0\) . By using Lemma 6.10, we can find a compactly supported continuous function \( f:X\to[0,1]\) satisfying:
On the other hand, by using (7) we can find a compactly supported continuous function \( g:X\to[0,1]\) satisfying \( g(x)=1\) on \( K_1\cup K_2\) , such that:
By using this, and the linearity of \( I\) , we have the following estimate:
Now since our number \( \varepsilon>0\) was arbitrary, this proves our claim, via (5).
(10) We are now in position of making a key verification, that of the additivity property of our measure \( \mu\) . To be more precise, our claim is that we have the following equality, valid for any family of pairwise disjoint subsets \( E_1,E_2,E_3,…\) of our space \( X\) :
In order to prove this, consider the union on the left, namely:
In the case \( \mu(E)=\infty\) our claim is proved by (5). So, assume \( \mu(E)<\infty\) , and pick an arbitrary number \( \varepsilon>0\) . We can then find compact sets \( K_i\subset E_i\) such that:
Now consider the following unions, which are compact sets too:
By using the additivity formula that we found in (9), recursively, we obtain:
Now since our number \( \varepsilon>0\) was arbitrary, this proves our claim.
(11) Thus, additivity property proved, and with this in hand, the other assertions are quite standard. In order to prove these, our first claim is that for any measurable set \( E\) and any \( \varepsilon>0\) , there is a compact set \( K\) , and an open set \( U\) , satisfying:
Indeed, in order to prove this claim, the first remark is that we can find indeed a compact set \( K\) and an open set \( U\) , satisfying the following conditions:
Since the set \( U-K\) is open, by using (8) and then (10) we obtain:
But with this, we can establish our inequality, as follows:
(12) Our next claim is that if \( A,B\) are measurable, then so are the following sets:
In order to prove this, pick an arbitrary number \( \varepsilon>0\) . By using (11) we can find certain compact sets \( K,H\) , and certain open sets \( U,V\) , satisfying:
Now observe that we have the following inclusion:
By using now (5), we obtain from this the following estimate:
Now since \( K-V\) is a compact subset of \( A-B\) , we conclude that \( A-B\) is measurable, as desired. Regarding now \( A\cup B\) , we can use here the following formula:
Indeed, by using (10) we obtain from this that \( A\cup B\) is measurable as well. Finally, regarding \( A\cap B\) , we can use here the following formula:
Thus \( A\cap B\) is measurable as well, and this finishes the proof of our claim.
(13) Our claim now is that the set \( M\) constructed in (3) is indeed an algebra, which contains all Borel sets. In order to prove this, consider an arbitrary compact set \( K\subset X\) . Given \( A\in M\) , we can write \( A^c\cap K\) as a difference of two sets in \( M\) , as follows:
Thus \( A^c\cap K\in M\) , and by using this, we conclude that we have:
Next, let us look at unions. Assume \( A_i\in M\) , and consider their union:
With \( K\subset X\) being an arbitrary compact set, as above, set \( B_1=A_1\cap K\) , and then define recursively the following sets:
In terms of these sets \( B_n\) , we have the following formula:
On the other hand, according to our definition of the sets \( B_n\) , and by using (12), we have \( B_n\in M\) . Thus, by using (10), we obtain \( A\in M\) . Thus, we have proved that:
Finally, if a subset \( C\subset X\) is closed, then \( C\cap K\) is compact, and so \( C\in M\) . Thus, as claimed above, \( M\) is indeed an algebra, which contains all the Borel sets.
(14) Our next claim, which together with what we found in (10) will prove that \( \mu\) is indeed a measure on \( M\) , is that, in the context of our constructions in (3), the family \( N\) constructed there consists precisely of the sets \( E\in M\) satisfying:
In order to prove this, let \( E\in N\) . By using (7) and (12) we deduce that we have \( E\cap K\in N\) for any compact set \( K\subset X\) , and we conclude from this that we have:
Conversely, assume that \( E\in M\) has finite measure, \( \mu(E)<\infty\) . In order to prove \( E\in N\) , we pick a number \( \varepsilon>0\) . We can then find an open set \( U\subset X\) such that:
Also, by (8) and (11) we can find a compact set \( K\subset X\) such that:
Now since \( E\cap K\in N\) , we can find a compact set \( H\subset X\) such that:
Observe now that we have the following inclusion:
By using this, we obtain the following inequality:
Thus \( E\in N\) , which finishes the proof of our claim, and proves that \( \mu\) is a measure.
(15) Time now to get into the final verification, that of the following formula:
As a first observation, by taking real and imaginary parts, it is enough to prove this for the real functions \( f\) . Moreover, and assuming now that \( f\) is real, by using \( f\to-f\) , in order to prove the above equality, it is enough to prove the following inequality:
(16) So, let us prove now this latter inequality. For this purpose, let \( K\subset X\) be the support of our function \( f\) , choose an interval containing the range, \( Im(f)\subset[a,b]\) , pick an arbitrary \( \varepsilon>0\) , and then numbers \( y_0,…,y_n\) such that \( y_i-y_{i-1}<\varepsilon\) , and:
With this done, consider now the following sets:
Since \( f\) is continuous, these are disjoint Borel sets, whose union is \( K\) . Thus, we can find open sets \( U_i\subset X\) , subject to the following conditions:
By using now Theorem 6.11, we can find certain continuous functions \( h_i:X\to[0,1]\) , with \( h_i\) supported on \( U_i\) , such that the following equality holds, on \( K\) :
By multiplying by \( f\) , we conclude that the following equality holds, on \( K\) :
On the other hand, by using (7), we have the following estimate:
Good news, with all these ingredients in hand, we can now finish. Since we have \( h_if\leq(y_i+\varepsilon)h_i\) , and since \( y_i-\varepsilon<f(x)\) on \( E\) , we have the following estimate:
Now since \( \varepsilon>0\) was arbitrary, this finishes the proof of our latest claim, that is, of the inequality in (15), and so finishes the proof of the present theorem.
We proved the Riesz theorem, and that was good learning. In practice now, in order to measure the various measurable sets \( E\subset X\) , the tools that we have come from (2) and (3) in that theorem. So, let us have a closer look at that two properties.
Let us start with a definition, inspired from what we have:
Definition 35
Let \( \mu\) be a Borel measure on a space \( X\) .
If all the Borel sets \( E\subset X\) are inner and outer regular, we say that \( \mu\) is regular.
We know from Theorem 6.12 that, under the assumptions there, all sets \( E\subset X\) are outer regular. However, in what regards inner regularity, we only have that for the sets \( E\subset X\) which are open, or of finite measure. So, there is certainly room for improvement here, and in relation with this, we first have the following result:
Theorem 86
Assume, in the context of the Riesz theorem, that \( X\) is locally compact, and is \( \sigma\) -compact too, in the sense that it is a countable union of compact sets.
Proof
This follows indeed by further building on Theorem 6.12 and its proof, under the assumptions in the statement, the idea being as follows:
(1) Let us write \( X=\cup_i K_i\) , with each \( K_i\) being compact. Given a measurable set \( E\subset X\) and a number \( \varepsilon>0\) , we can find then open sets \( U_i\supset(K_i\cap E)\) such that:
Now if we set \( U=\cup_iU_i\) , we have \( U-E\subset\cup_i(U_i-(K_i\cap E))\) , which gives:
We can apply this to \( E^c=X-E\) too, and we get an open set \( V\supset E^c\) such that:
But this gives the result, with \( U\) being as above, and with \( F=V^c\) .
(2) This follows indeed from (1), in the obvious way, applied with \( \varepsilon_i=1/i\) .
(3) This follows again from (1), which proves the needed inner regularity of \( E\) .
Along the same lines, we have as well the following result:
Theorem 87
Assume that \( X\) is locally compact, with all its open sets \( U\subset X\) being \( \sigma\) -compact. If \( \lambda\) is a Borel measure on \( X\) having the property that
for any compact subset \( K\subset X\) , then \( \lambda\) is regular.
Proof
Consider indeed the integration with respect to the measure \( \lambda\) :
By using our assumption, \( \lambda(K)<\infty\) for any \( K\subset X\) compact, we conclude that the Riesz theorem applies to this functional. Thus, we obtain a certain measure \( \mu\) , that we know in addition to be regular, as shown in Theorem 6.14, such that:
Thus, it remains to prove that we have \( \lambda=\mu\) , and this can be done as follows:
(1) Let us first prove that we have \( \lambda(U)=\mu(U)\) , for any open set \( U\subset X\) . For this purpose, we can use the \( \sigma\) -compactness assumption in the statement, which allows us to write our set \( U\) as follows, with the sets \( K_i\subset X\) being compact:
By using the Urysohn lemma, we can find continuous functions \( f_i:X\to[0,1]\) which are supported on \( U\) , and are equal to 1 on \( K_i\) . With this done, let us set:
The monotone convergence theorem applies then, and gives, as desired:
(2) Now let us prove that we have \( \lambda(E)=\mu(E)\) , for any Borel set \( E\subset X\) . For this purpose, pick \( \varepsilon>0\) . By using Theorem 6.14 (1), applied to the above measure \( \mu\) , coming from Riesz, we can find \( F\subset E\subset U\) , with \( F\) closed and \( U\) open, such that:
Now by using (1), applied to the open set \( U-F\) , we obtain from this \( |\lambda(E)-\mu(E)|<\varepsilon\) . But this being true for any \( \varepsilon>0\) , we obtain \( \lambda(E)=\mu(E)\) , as desired.
Good work that we did, in the above, and time now to enjoy what we learned. As a first consequence of the Riesz theorem, we can now formulate, following Lebesgue:
Theorem 88 (Lebesgue)
We can measure all the Borel sets
and the corresponding integration theory extends the usual Riemann integral.
Proof
This follows indeed from Theorem 6.12, by using the Riemann integral, that we know well from basic calculus, as the positive functional needed there:
Indeed, it follows from basic calculus that the Riemann integral satisfies indeed all the needed properties in Theorem 6.12, and this gives the result. We will leave the various verifications here, which are all elementary, as an instructive exercise.
Goind ahead with more theory, let us discuss now the case of \( \mathbb R^N\) . This is something quite tricky, and you would probably say here, why not simply generalizing Theorem 6.16 and its proof, by using this time the multivariable Riemann integral, that we know as well from basic calculus, as the positive functional needed there:
However, let me ask you here, for instance at \( N=2\) , do we really master well the above integral? For instance, what is its precise definition? And here, I bet that your answer would be that we can define the multivariable integral by iterating, as follows:
Which looks fine, at a first glance, but there is in fact a bug, with this. Indeed, assuming so, we would have by symmetry the following formula too:
Thus, and forgetting now about what we wanted to do, we can see that our method is based on the Fubini formula, stating that we must have:
But, you might already know from calculus that Fubini does not always work, with the counterexamples being not very difficult to construct, as follows:
Proposition 25
The Fubini formula, namely
can fail, for certain suitably chosen functions.
Proof
We have indeed the following computation:
On the other hand, by using this, and symmetry, we have as well:
Thus Fubini fails. Please pass this knowledge to your children, and grandchildren.
Summarizing, our method does not work. Which can be regarded as good news for us mathematicians, time now to get into some action, show our skills, and fix all this. Let us first formulate a general objective, inspired from the above, as follows:
Objective 1
We would like to measure all the Borel sets
with the corresponding integration theory extending and clarifying the Riemann integral, in several variables. Also, we would like to clarify the validity of the Fubini formula
in this formulation, in \( 2\) variables, and in general \( N\geq2\) variables too.
Getting to work now, it is pretty much obvious that, in order to deal with this problem, rigorously, it is better to forget everything that we know from multivariable calculus, regarding the integration in several variables, and recreate everything from scratch, with our tools being Theorem 6.12, complemented by Theorem 6.14 and Theorem 6.15.
So, here is our first result, using our abstract measure theory techniques:
Proposition 26
Given a compactly supported function \( f:\mathbb R^N\to\mathbb C\) , if we set
with \( \mathbb Z^N/2^k\subset\mathbb R^N\) being the points having as coordinates integer multiples of \( 1/2^k\) , then
converges, and \( f\to I(f)\) satisfies the assumptions of the Riesz theorem.
Proof
In order to prove the first assertion, we can assume that our function is real, \( f:\mathbb R^N\to\mathbb R\) . But then, we can use the fact that \( f\) is uniformly continuous, and this gives the convergence in the statement. As for the second assertion, the fact that \( f\to I(f)\) satisfies indeed the assumptions of the Riesz theorem is clear from definitions.
In view of the above, we can indeed apply the Riesz theorem. We obtain:
Theorem 89 (Lebesgue)
There is a unique regular, translation-invariant measure on \( \mathbb R^N\) , normalized as for the volume of the unit cube to be \( 1\) . This measure is called Lebesgue measure, and appears alternatively as the measure on the Borel sets
whose corresponding integration theory extends the usual Riemann integral, as constructed in Proposition 6.19, or equivalently, as in multivariable calculus.
Proof
Obviously, many things going on here, and as a first remark, this fulfills indeed our requirements from Objective 6.18, save for Fubini, that we will discuss later, in the next chapter. In what regards the rest, the idea is as follows:
(1) We want to prove that the Lebesgue measure exists, and is unique, with the various properties from the statement. But here, a bit of thinking shows that this measure can only be the one coming from Proposition 6.19, via the Riesz theorem.
(2) So, let us go back to Proposition 6.19, which allows us to apply the Riesz theorem, and apply indeed this Riesz theorem. We obtain in this way a certain regular Borel measure on \( \mathbb R^N\) , that we have to prove to have the properties in the statement.
(3) But here, all the assertions follow from what we have in the Riesz theorem, complemented where needed by Theorem 6.14 and Theorem 6.15, that we can use too. We will leave the various verifications here as an instructive exercise.
Summarizing, we know now how to integrate on \( \mathbb R^N\) . As a continuation of this, as a first task, let us go back to the Fubini formula problematics. We have:
Theorem 90 (Fubini, Tonelli)
Given a function \( f:\mathbb R^N\to\mathbb R\) which is measurable and integrable, in the sense that the following integral is finite,
we have the following equalities, for any decomposition \( N=N_1+N_2\) :
Moreover, the same holds when \( f:\mathbb R^N\to\mathbb R\) is assumed positive, and measurable.
Proof
In this statement, which is something quite compact, the first assertion is called Fubini theorem, and the second assertion is called Tonelli theorem. These two theorems are related, and can be also subject to various minor improvements. In what follows we will mostly focus on proving the Fubini theorem in its basic form, as stated.
(1) In order to prove the Fubini theorem, consider the vector space \( L\) consisting of the functions \( f:\mathbb R^N\to\mathbb R\) which are measurable, and integrable:
Now fix a decomposition \( N=N_1+N_2\) , and consider the family \( F\subset L\) of functions \( f:\mathbb R^N\to\mathbb R\) which satisfy the formula in the statement, namely:
With this conventions, we want to prove that the inclusion \( F\subset L\) is an equality.
(2) But this comes from the following sequence of observations, which all come from definitions, and from the Lebesgue dominated convergence theorem from chapter 5:
– \( F\) is stable under taking finite linear combinations.
– \( F\) is stable under taking pointwise limits of increasing sequences.
– \( 1_E\in F\) , for any set of finite measure of type \( E=E_1\times E_2\subset\mathbb R^N\) .
– \( 1_E\in F\) , for any open set of finite measure \( E\subset\mathbb R^N\) .
– \( 1_E\in F\) , for any countable intersection of open sets, of finite measure \( E\subset\mathbb R^N\) .
– \( 1_E\in F\) , for any set of null measure \( E\subset\mathbb R^N\) .
– \( 1_E\in F\) , for any set of finite measure \( E\subset\mathbb R^N\) .
Indeed, the proof of all these assertions, in the above precise order, is something quite straightforward, and once we have the last assertion, the result follows, because we can approximate any given function \( f\in L\) by step functions, and we obtain \( f\in F\) .
(3) Summarizing, Fubini proved, modulo some straightforward verifications left to you, and the same goes with Tonelli, whose proof is quite routine, based on Fubini.
As a conclusion, we know how to rigorously integrate on \( \mathbb R^N\) , and as a bonus, the theory that we developed seems to have many follow-ups. More on this later.
Tough chapter that we had here, with many things explained quite quickly, and as exercises on this, all in relation with better understanding what we did, we have:
Exercise 41
Clarify what has been said above about saturating with null sets.
Exercise 42
Clarify as well what we said about integration, modulo null sets.
Exercise 43
Learn more about almost everywhere equality, from probabilists.
Exercise 44
Meditate about Fubini, its victories, and its failures.
Exercise 45
Fill in the details for the properties of the Lebesgue measure.
Exercise 46
Learn as well other methods for constructing the Lebesgue measure.
Exercise 47
Learn about the Lusin theorem, and its applications.
Exercise 48
Learn also about the Vitali-Carathéodory theorem.
As bonus exercise, read some systematic measure theory, say from Rudin [3], with what we did here being an introduction to that, and from an alternative book too.
In this chapter and in the next one we discuss more advanced aspects of integration, on various explicit spaces \( X\subset\mathbb R^N\) , notably in relation with derivatives. To start with, we will be interested in the explicit computation of the integrals in \( \mathbb R^N\) .
Let us start with some basics. In what regards the area of the circle, sure you know that, but never too late to learn the truth about it, which is as follows:
Theorem 91
The following two definitions of \( \pi\) are equivalent:
Proof
In order to prove this theorem let us cut the unit disk as a pizza, into \( N\) slices, and forgetting about gastronomy, leave aside the rounded parts: 
The area to be eaten can be then computed as follows, where \( H\) is the height of the slices, \( S\) is the length of their sides, and \( P=NS\) is the total length of the sides:
Thus, with \( N\to\infty\) we obtain that we have \( A=L/2\) , as desired.
As a second computation, the area of an ellipse can be computed as follows:
Theorem 92
The area of an ellipse, given by the equation
with \( a,b>0\) being half the size of a box containing the ellipse, is \( A=\pi ab\) .
Proof
The idea is that of cutting the ellipse into vertical slices. First observe that, according to our equation \( (x/a)^2+(y/b)^2=1\) , the \( x\) coordinate can range as follows:
For any such \( x\) , the other coordinate \( y\) , satisfying \( (x/a)^2+(y/b)^2=1\) , is given by:
We conclude that the area of the ellipse is given by the following formula:
Finally, as a verification, for \( a=b=R\) we get \( A=\pi R^2\) , as we should.
In what regards now the length of the ellipse, the “pizza” argument from the proof of Theorem 7.1 does not work, and things here get fairly complicated, as follows:
Fact 8
The length of an ellipse, given by \( (x/a)^2+(y/b)^2=1\) , is
and with this integral being generically not computable.
To be more precise, the above formula can be deduced in the following way, and more on such things in chapter 8 below, when systematically discussing the curves:
As for the last assertion, when \( a=b=R\) we get of course \( L=2\pi R\) , as we should, but in general, when \( a\neq b\) , there is no trick for computing the above integral.
Moving now to 3D, as an obvious challenge here, we can try to compute the area and volume of the sphere, and more generally of the ellipsoids. We have here:
Theorem 93
The volume of the unit sphere in \( \mathbb R^3\) is given by:
More generally, the volume of an ellipsoid, given by \( (x/a)^2+(y/b)^2+(z/c)^2=1\) , is:
The area of the sphere is \( A=4\pi\) . For ellipsoids, the area is generically not computable.
Proof
There are several things going on here, as follows:
(1) Let us first compute the volume of the ellipsoid, which at \( a=b=c=1\) will give the volume of the unit sphere. The range of the first coordinate \( x\) is as follows:
Now when the first coordinate \( x\) is fixed, the other coordinates \( y,z\) vary on an ellipse, given by the equation \( (y/b)^2+(z/c)^2=1-(x/a)^2\) , which can be written as follows:
Thus, the vertical slice of our ellipsoid at \( x\) has area as follows:
We conclude that the volume of the ellipsoid is given, as claimed, by:
(2) At \( a=b=c=1\) we get \( V=4\pi/3\) , and this gives the area of the unit sphere too, because the “pizza” method from the proof of Theorem 7.1 applies, and gives:
(3) Finally, the last assertion is something quite informal, inspired from Fact 7.3.
In order to deal now with \( N\) dimensions, we will need the Wallis formula:
Theorem 94 (Wallis)
We have the following formulae,
where \( \varepsilon(n)=1\) if \( n\) is even, and \( \varepsilon(n)=0\) if \( n\) is odd, and where
with the product ending at \( 2\) if \( m\) is odd, and ending at \( 1\) if \( m\) is even.
Proof
Let us first compute the integral on the left in the statement:
We do this by partial integration. We have the following formula:
By integrating between \( 0\) and \( \pi/2\) , we obtain the following formula:
Thus we can compute \( I_n\) by recurrence, and we obtain:
But \( I_0=\frac{\pi}{2}\) and \( I_1=1\) , so we get the result. As for the second formula, this follows from the first one, with \( t=\frac{\pi}{2}-s\) . Thus, we have proved both formulae in the statement.
We can now compute the volumes of the \( N\) dimensional spheres, as follows:
Theorem 95
The volume of the unit sphere in \( \mathbb R^N\) is given by
with our usual convention \( N!!=(N-1)(N-3)(N-5)…\)
Proof
If we denote by \( V_N\) the volume of the unit sphere in \( \mathbb R^N\) , we have:
Now by recurrence, and by using the formula in Theorem 7.5, we obtain:
Thus, we are led to the formula in the statement.
As main particular cases of the above formula, we have:
Proposition 27
The volumes of the low-dimensional spheres are as follows:
Proof
Some of these results are well-known, but we can obtain all of them as particular cases of the general formula in Theorem 7.6, as follows:
(1) At \( N=1\) we obtain \( V=1\cdot\frac{2}{1}=2\) .
(2) At \( N=2\) we obtain \( V=\frac{\pi}{2}\cdot\frac{4}{2}=\pi\) .
(3) At \( N=3\) we obtain \( V=\frac{\pi}{2}\cdot\frac{8}{3}=\frac{4\pi}{3}\) .
(4) At \( N=4\) we obtain \( V=\frac{\pi^2}{4}\cdot\frac{16}{8}=\frac{\pi^2}{2}\) .
Next, we can compute as well the volumes of general ellipsoids, as follows:
Theorem 96
The volume of an arbitrary ellipsoid in \( \mathbb R^N\) , given by
is \( V=a_1… a_NV_0\) , with \( V_0\) being the volume of the unit sphere in \( \mathbb R^N\) .
Proof
We already know this at \( N=2,3\) , from Theorem 7.2 and Theorem 7.4, and the proof in general is similar, by suitably adapting the proof for the sphere. We will leave the computations here as an exercise, and we will come back to this in a moment, with a more conceptual argument, based on Theorem 7.6, and on a change of variables.
Finally, let us record as well the formula of the area of the sphere, as follows:
Theorem 97
The area of the unit sphere in \( \mathbb R^N\) is given by:
In particular, at \( N=2,3,4\) we obtain respectively \( A=2\pi,4\pi,2\pi^2\) .
Proof
As shown by the pizza argument from the proof of Theorem 7.1, which extends to arbitrary dimensions, the area and volume of the sphere in \( \mathbb R^N\) are related by:
Together with the formula in Theorem 7.6 for \( V\) , this gives the result.
Needless to say, in what regards the area of the general ellipsoids in \( \mathbb R^N\) , this is in general not computable, as we already know well at \( N=2,3\) , from the above.
We know from one-variable calculus that the change of variables for integrals is related to differentiation, and we can of course expect the same to happen in several variables. So, let us first recall the basics of differentiation, in several variables. We first have:
Theorem 98
The functions \( f:\mathbb R^N\to\mathbb R^M\) can be locally approximated as
with \( f'(x)\) being by definition the matrix of partial derivatives at \( x\) ,
acting on the vectors \( t\in\mathbb R^N\) by usual multiplication.
Proof
This is obviously something a bit informal, because the precise assumptions on \( f\) , for everything to work fine, are not mentioned in the statement. However, you surely know this, and the discussion below is just a matter of recalling the basics:
(1) First of all, at \( N=M=1\) what we have is a usual 1-variable function \( f:\mathbb R\to\mathbb R\) , and the formula in the statement is something that we know well, namely:
(2) Let us discuss now the case \( N=2,M=1\) . Here what we have is a function \( f:\mathbb R^2\to\mathbb R\) , and by using twice the basic approximation result from (1), we obtain:
(3) More generally, we can deal in this way with the general case \( M=1\) , with the formula here, obtained via a straightforward recurrence, being as follows:
(4) But this gives the result in the case where both \( N,M\in\mathbb N\) are arbitrary too. Indeed, we can apply (3) to each of the components \( f_i:\mathbb R^N\to\mathbb R\) , and we get:
But this collection of \( M\) formulae tells us precisely that the following happens, as an equality, or rather approximation, of vectors in \( \mathbb R^M\) :
Thus, we are led to the conclusion in the statement.
Generally speaking, Theorem 7.10 is all you need to know for upgrading from calculus to multivariable calculus. As a standard result here, we have:
Theorem 99
We have the chain derivative formula
as an equality of matrices.
Proof
Consider indeed a composition of functions, as follows:
According to Theorem 7.10, the derivatives of these functions are certain linear maps, corresponding to certain rectangular matrices, as follows:
Thus, our formula makes sense indeed. As for proof, this comes from:
Thus, we are led to the conclusion in the statement.
Getting now to integration matters, let us first recall that we have:
Proposition 28
We have the change of variable formula
where \( c=\varphi^{-1}(a)\) and \( d=\varphi^{-1}(b)\) .
Proof
This follows with \( f=F'\) , from the following differentiation rule:
Indeed, by integrating between \( c\) and \( d\) , we obtain the result.
In several variables now, things are quite similar, the result being as follows:
Theorem 100
Given a transformation \( \varphi=(\varphi_1,…,\varphi_N)\) , we have
with the \( J_\varphi\) quantity, called Jacobian, being given by
and with this generalizing the usual formula from one variable calculus.
Proof
This is something quite tricky, the idea being as follows:
(1) Observe first that this generalizes indeed the change of variable formula in 1 dimension, from Proposition 7.12, the point here being that the absolute value on the derivative appears as to compensate for the lack of explicit bounds for the integral.
(2) As a second observation, we can assume if we want, by linearity, that we are dealing with the constant function \( f=1\) . For this function, our formula reads:
In terms of \( D={\varphi^{-1}(E)}\) , this amounts in proving that we have:
Now since this latter formula is additive with respect to \( D\) , it is enough to prove it for small cubes \( D\) . And here, as a first remark, our formula is clear for the linear maps \( \varphi\) , by using the definition of the determinant of real matrices, as a signed volume.
(3) However, the extension of this to the case of non-linear maps \( \varphi\) is something which looks non-trivial, so we will not follow this path, in what follows. So, while the above \( f=1\) discussion is certainly something nice, our theorem is still in need of a proof.
(4) In order to prove the theorem, as stated, let us rather focus on the transformations used \( \varphi\) , instead of the functions to be integrated \( f\) . Our first claim is that the validity of the theorem is stable under taking compositions of such transformations \( \varphi\) .
(5) In order to prove this claim, consider a composition, as follows:
Assuming that the theorem holds for \( \varphi,\psi\) , we have the following computation:
Thus, our theorem holds as well for \( \varphi\circ\psi\) , and we have proved our claim.
(6) Next, as a key ingredient, let us examine the case where we are in \( N=2\) dimensions, and our transformation \( \varphi\) has one of the following special forms:
By symmetry, it is enough to deal with the first case. Here the Jacobian is \( d\psi/dx\) , and by replacing if needed \( \psi\to-\psi\) , we can assume that this Jacobian is positive, \( d\psi/dx>0\) . Now by assuming as before that \( D=\varphi^{-1}(E)\) is a rectangle, \( D=[a,b]\times[c,d]\) , we can prove our formula by using the change of variables in 1 dimension, as follows:
(7) But with this, we can now prove the theorem, in \( N=2\) dimensions. Indeed, given a transformation \( \varphi=(\varphi_1,\varphi_2)\) , consider the following two transformations:
We have then \( \varphi=\psi\circ\phi\) , and by using (6) for \( \psi,\phi\) , which are of the special form there, and then (3) for composing, we conclude that the theorem holds for \( \varphi\) , as desired.
(8) Thus, theorem proved in \( N=2\) dimensions, and the extension of the above proof to arbitrary \( N\) dimensions is straightforward, that we will leave this as an exercise.
As a basic application of this technology, we can recover Theorem 7.8, as follows:
Theorem 101
The volume of an arbitrary ellipsoid in \( \mathbb R^N\) , given by
is \( V=a_1… a_NV_0\) , with \( V_0\) being the volume of the unit sphere in \( \mathbb R^N\) .
Proof
This is indeed something clear, with the change of variables \( x_i=a_iy_i\) , whose Jacobian is constant, \( J=a_1… a_N\) . Thus, we are led to the formula in the statement.
In order to discuss now some other applications, in 2 dimensions, let us start with:
Theorem 102
We have polar coordinates in \( 2\) dimensions,
the corresponding Jacobian being \( J=r\) .
Proof
This is something elementary, with the Jacobian being as follows:
Thus, we have indeed the formula in the statement.
We can now compute the Gauss integral, which is the best calculus formula ever:
Theorem 103
We have the following formula,
called Gauss integral formula.
Proof
Let \( I\) be the above integral. By using polar coordinates, we obtain:
Thus, we are led to the formula in the statement.
Moving now to 3 dimensions, we have here the following result:
Theorem 104
We have spherical coordinates in \( 3\) dimensions,
the corresponding Jacobian being \( J(r,s,t)=r^2\sin s\) .
Proof
The fact that we have indeed spherical coordinates is clear. Regarding now the Jacobian, this is given by the following formula:
Thus, we have indeed the formula in the statement.
Let us work out now the general spherical coordinate formula, in arbitrary \( N\) dimensions. The formula here, which generalizes those at \( N=2,3\) , is as follows:
Theorem 105
We have spherical coordinates in \( N\) dimensions,
the corresponding Jacobian being given by the following formula,
and with this generalizing the known formulae at \( N=2,3\) .
Proof
As before, the fact that we have spherical coordinates is clear. Regarding now the Jacobian, also as before, by developing over the last column, we have:
Thus, we obtain the formula in the statement, by recurrence.
Let us discuss now the computation of the arbitrary integrals over the sphere. We will need a technical result extending Theorem 7.5, as follows:
Theorem 106 (Wallis)
We have the following formula,
where \( \varepsilon(p)=1\) if \( p\) is even, and \( \varepsilon(p)=0\) if \( p\) is odd, and where
with the product ending at \( 2\) if \( m\) is odd, and ending at \( 1\) if \( m\) is even.
Proof
We use the same idea as in Theorem 7.5. Let \( I_{pq}\) be the integral in the statement. In order to do the partial integration, observe that we have:
By integrating between \( 0\) and \( \pi/2\) , we obtain, for \( p,q>0\) :
Thus, we can compute \( I_{pq}\) by recurrence. When \( q\) is even we have:
But the last term comes from Theorem 7.5, and we obtain the result:
Observe that this gives the result for \( p\) even as well, by using \( I_{pq}=I_{qp}\) , coming via \( t=\frac{\pi}{2}-s\) . In the remaining case now, where both \( p,q\) are odd, we can use once again the formula \( pI_{p-1,q+1}=qI_{p+1,q-1}\) established above, and the recurrence goes as follows:
In order to compute the last term, observe that we have:
Thus, we can finish our computation in the case \( p,q\) odd, as follows:
Thus, we obtain the formula in the statement, the exponent of \( \pi/2\) appearing there being \( \varepsilon(p)\varepsilon(q)=0\cdot 0=0\) in the present case, and this finishes the proof.
We can now integrate over the spheres, as follows:
Theorem 107
The polynomial integrals over the unit sphere \( S^{N-1}_\mathbb R\subset\mathbb R^N\) , with respect to the normalized, mass \( 1\) measure, are given by the following formula,
valid when all exponents \( k_i\) are even. If an exponent is odd, the integral vanishes.
Proof
If one exponent \( k_i\) is odd, with \( x_i\to-x_i\) we can see that the integral in the statement vanishes. So, assume that all \( k_i\) are even. As a first observation, the result holds indeed at \( N=2\) , due to the formula from Theorem 7.19, which reads:
In the general case now, where the dimension \( N\in\mathbb N\) is arbitrary, the integral in the statement can be written in spherical coordinates, as follows:
Here \( A\) is the area of the sphere, \( J\) is the Jacobian, and the \( 2^N\) factor comes from the restriction to the \( 1/2^N\) part of the sphere where all the coordinates are positive. According to Theorem 7.9, the normalization constant in front of the integral is:
As for the unnormalized integral, this is given by:
By rearranging the terms, we obtain:
Now by using the above-mentioned formula at \( N=2\) , this gives:
Now let \( F\) be the part involving the double factorials, and \( P\) be the part involving the powers of \( \pi/2\) , so that \( I'=F\cdot P\) . Regarding \( F\) , by cancelling terms we have:
As in what regards \( P\) , by summing the exponents, we obtain \( P=\left(\frac{\pi}{2}\right)^{[N/2]}\) . We can now put everything together, and we obtain the formula in the statement.
In the case of the complex spheres, we have the following result:
Theorem 108
We have the following integration formula over the complex sphere \( S^{N-1}_\mathbb C\subset\mathbb C^N\) , with respect to the normalized uniform measure,
valid for any exponents \( k_i\in\mathbb N\) . As for the other polynomial integrals in \( z_1,…,z_N\) and their conjugates \( \bar{z}_1,…,\bar{z}_N\) , these all vanish.
Proof
The last assertion being clear by using \( z\to\lambda z\) , assume that we are in the non-vanishing case. By using the standard identification \( S^{N-1}_\mathbb C\simeq S^{2N-1}_\mathbb R\) , we obtain:
By using now our integration formula for the real spheres, we obtain:
Thus, we are led to the formula in the statement.
Good news, with our accumulated measure theory knowledge, we can now lay down some solid foundations for probability theory, both discrete and continuous. With the idea in mind of doing things a bit abstractly, our starting point will be:
Definition 36
Let \( X\) be a probability space, that is, a space with a probability measure, and with the corresponding integration denoted \( E\) , and called expectation.
Here the fact that \( \mu_f\) exists indeed is well-known. By linearity, we would like to have a real probability measure making hold the following formula, for any \( P\in\mathbb R[X]\) :
By using a standard continuity argument, it is enough to have this formula for the characteristic functions \( \chi_I\) of the measurable sets of real numbers \( I\subset\mathbb R\) :
But this latter formula, which reads \( P(f\in I)=\mu_f(I)\) , can serve as a definition for \( \mu_f\) , and we are done. Next in line, we need to talk about independence:
Definition 37
Two variables \( f,g\in L^\infty(X)\) are called independent when
happens, for any \( k,l\in\mathbb N\) .
Again, this definition hides some non-trivial things. Indeed, by linearity, we would like to have a formula as follows, valid for any polynomials \( P,Q\in\mathbb R[X]\) :
By using a continuity argument, it is enough to have this formula for characteristic functions \( \chi_I,\chi_J\) of the measurable sets of real numbers \( I,J\subset\mathbb R\) :
Thus, we are led to the usual definition of independence, namely:
All this might seem a bit abstract, but in practice, the idea is of course that \( f,g\) must be independent, in an intuitive, real-life sense. As a first result now, we have:
Proposition 29
Assuming that \( f,g\in L^\infty(X)\) are independent, we have
where \( *\) is the convolution of real probability measures.
Proof
We have the following formula, coming from the independence of \( f,g\) :
On the other hand, by using the Fubini theorem, we have as well:
Thus \( \mu_{f+g}\) and \( \mu_f*\mu_g\) have the same moments, so they coincide, as claimed.
Here is now a second result on independence, which is something more advanced:
Theorem 109
Assuming that \( f,g\in L^\infty(X)\) are independent, we have
where \( F_f(x)=E(e^{ixf})\) is the Fourier transform.
Proof
We have the following computation, using Proposition 7.24 and Fubini:
Thus, we are led to the conclusion in the statement.
But with this in hand, and with the Gauss formula helping too, we can now have a new, final look at the normal variables, discussed in chapter 4. Let us start with:
Definition 38
The normal law of parameter \( 1\) is the following measure:
More generally, the normal law of parameter \( t>0\) is the following measure:
These are also called Gaussian distributions, with “g” standing for Gauss.
Observe that these laws have indeed mass 1, as they should, due to:
Generally speaking, the normal laws appear as bit everywhere, in real life. The reasons for this come from the Central Limit Theorem (CLT), that we will explain in a moment, after developing some more general theory. As a first result, we have:
Theorem 110
We have the following formula, valid for any \( t>0\) :
In particular, the normal laws satisfy \( g_s*g_t=g_{s+t}\) , for any \( s,t>0\) .
Proof
The Fourier transform formula can be established as follows:
As for \( g_s*g_t=g_{s+t}\) , this follows via Theorem 7.25, \( \log F_{g_t}\) being linear in \( t\) .
We are now ready to state and prove the CLT, as follows:
Theorem 111 (CLT)
Given real variables \( f_1,f_2,f_3,…\in L^\infty(X)\) which are i.i.d., centered, and with common variance \( t>0\) , we have
with \( n\to\infty\) , in moments.
Proof
The Fourier transform of the variable in the statement is:
But this function being the Fourier transform of \( g_t\) , we obtain the result.
Finally, we will need the following result, regarding the moments:
Theorem 112
The even moments of the normal law are the numbers
where \( k!!=(k-1)(k-3)(k-5)…\,\) , and the odd moments vanish.
Proof
We have the following computation, valid for any integer \( k\in\mathbb N\) :
Thus, by recurrence, we are led to the formula in the statement.
In relation now with geometry, we first have the following result:
Theorem 113
The moments of the hyperspherical variables are
and the rescaled variables \( y_i=\sqrt{N}x_i\) become normal and independent with \( N\to\infty\) .
Proof
The moment formula in the statement follows from our general integration formula over the real spheres, which was as follows:
Indeed, as a consequence, with \( N\to\infty\) we have the following estimate:
Thus, the variables \( \sqrt{N}x_i\) become normal with \( N\to\infty\) , as claimed. As for the proof of the asymptotic independence, this is standard too, once again by using our general integration formula above. Indeed, the joint moments of \( x_1,…,x_N\) are given by:
By rescaling, the joint moments of the variables \( y_i=\sqrt{N}x_i\) are given by:
Thus, we have multiplicativity, and so independence with \( N\to\infty\) , as claimed.
In the case of the complex spheres, we have a similar result, as follows:
Theorem 114
The rescalings \( \sqrt{N}z_i\) of the unit complex sphere coordinates
become complex Gaussian and independent with \( N\to\infty\) .
Proof
As before, we have the following estimate, for a single coordinate, which shows that the rescaled variables \( \sqrt{N}z_i\) become normal with \( N\to\infty\) :
As for the asymptotic independence, this is standard too, again by using our general integration formulae. Indeed, the joint moments of \( z_1,…,z_N\) are given by:
Thus, we have multiplicativity, and so independence with \( N\to\infty\) , as claimed.
How many probability measures are there? In order to discuss this question, we need to compare abstract measures, and the main theorem here is a very useful decomposition result, due to Radon-Nikodym, that we would like to explain now.
Let us start with some abstract notions, in relation with this, as follows:
Definition 39
Let \( \lambda,\mu\) be two measures on a measurable space \( X\) .
As illustrations for this, consider the case where \( X=\mathbb R^N\) , and \( \mu\) is the Lebesgue measure. In this case it is easy to construct mesures \( \lambda<\mu\) , because any integrable function \( f:\mathbb R^N\to\mathbb R\) produces such a measure, according to the following formula:
As for the measures \( \lambda\perp\mu\) , there are many of them too, because any discrete measure \( \lambda\) , appearing by definition as follows, with \( \{x_i\}\subset\mathbb R^N\) being a sequence of points, and \( \{c_i\}\subset[0,\infty)\) being a sequence of positive numbers, has this orthogonality property:
Getting back now to the general case, where the measurable space \( X\) is arbitrary, we have the following remarkable decomposition result, due to Radon-Nikodym:
Theorem 115
Consider a measured space \( (X,\mu)\) , with \( \mu\) being \( \sigma\) -finite, and let \( \lambda\) be another measure on \( X\) . We can then find a decomposition as follows,
and in what regards the absolutely continuous part \( \nu\) , this appears as
for a certain function \( f\) , integrable with respect to \( \mu\) , called density of \( \nu\) .
Proof
This is something quite standard, based on the material that we know, but instead of explaining this, which is quite long, here is a quick proof due to von Neumann, using some functional analysis material that we will learn later in this book:
(1) Pick a function \( w:X\to\mathbb(0,1)\) which is integrable with respect to \( \mu\) , and set:
The integration with respect to \( \lambda\) being a bounded linear functional on the space of square-integrable functions with respect to \( \rho\) , we must have a formula as follows for this functional, for a certain function \( g\) , which is square-integrable with respect to \( \rho\) :
We have then \( g:X\to[0,1]\) , and the above formula can be rewritten as follows:
As already mentioned, we have used here some functional analysis material, that we will learn later in this book. So, don’t forget to come back later to review this proof.
(2) But this gives both the assertions. Indeed, consider first the following sets:
We can then construct the needed decomposition \( \lambda=\nu+\eta\) by setting:
Indeed, we have \( \lambda=\nu+\eta\) , and the conditions \( \nu<\mu\) and \( \eta\perp\mu\) are both clear.
(3) Moreover, with \( f=1+g+…+g^{n-1}\) in the formula from (1), we obtain:
But with this, we can set the density function \( f\) to be the limit of the integrands on the right, and the density function property is satisfied indeed, as desired.
Getting back now to \( \mathbb R^N\) , the Radon-Nikodym theorem has the following consequence:
Theorem 116
Any probability measure on \( \mathbb R^N\) is of the following form,
with \( f:\mathbb R^N\to\mathbb R_+\) being a certain density, and \( \eta\) being supported on a measure \( 0\) set.
Proof
This follows indeed from Theorem 7.33, when applied to the case of the probability measures, that is, of the positive measures of mass 1. Observe that, as basic examples, we have the measures of the following type, with \( f\geq0\) and \( c_i\geq0\) :
To be more precise, here \( f:\mathbb R^N\to\mathbb R\) is a certain density function, \( x_i\in\mathbb R^N\) are certain points, and \( c_i\in\mathbb R\) are certain numbers, subject to the following conditions:
However, these are just the basic examples, and this because exotic measures \( \eta\perp\mu\) do exist. For more on this, we refer to any specialized measure theory book.
We had a quite exciting chapter here, and as exercises on all this, we have:
Exercise 49
Meditate some more on the various definitions of \( \pi\) .
Exercise 50
Do some computations for the lengths of the ellipses.
Exercise 51
Do some computations for the areas of ellipsoids, too.
Exercise 52
Learn if needed the theory of partial derivatives, with full details.
Exercise 53
Fill in the details, in the proof of the change of the variable formula.
Exercise 54
Learn about the stereographic projection, and cartography in general.
Exercise 55
Compute the laws of hyperspherical variables at small values of \( N\) .
Exercise 56
Learn, with full details, the Radon-Nikodym theorem.
As bonus exercise, find and solve 100 difficult exercises on multivariable calculus.
Our goal in this chapter is to integrate over curves, surfaces, and other manifolds. But, as a question coming before that, what are the manifolds themselves?
In answer, things can be quite complicated, but everything comes from conics. We have indeed the following result, which is at the origin of modern mathematics:
Theorem 117
The conics, which are the algebraic curves of degree \( 2\) in the plane,
with \( \deg P\leq 2\) , appear modulo degeneration by cutting a \( 2\) -sided cone with a plane, and can be classified into ellipses, parabolas and hyperbolas.
Proof
This is something very standard, the idea being as follows:
(1) Let us first classify the conics up to non-degenerate linear transformations of the plane, which are by definition transformations as follows, with \( \det A\neq0\) :
Our claim is that as solutions we have the circles, parabolas, hyperbolas, along with some degenerate solutions, namely \( \emptyset\) , points, lines, pairs of lines, \( \mathbb R^2\) .
(2) As a first remark, it looks like we forgot precisely the ellipses, but via linear transformations these become circles, so things fine. As a second remark, all our claimed solutions can appear. Indeed, the circles, parabolas, hyperbolas can appear as follows:
As for \( \emptyset\) , points, lines, pairs of lines, \( \mathbb R^2\) , these can appear too, as follows, and with our polynomial \( P\) chosen, whenever possible, to be of degree exactly 2:
Observe here that, when dealing with these degenerate cases, assuming \( \deg P=2\) instead of \( \deg P\leq 2\) would only rule out \( \mathbb R^2\) itself, which is not worth it.
(3) Getting now to the proof of our claim in (1), classification up to linear transformations, consider an arbitrary conic, written as follows, with \( a,b,c,d,e,f\in\mathbb R\) :
Assume first \( a\neq0\) . By making a square out of \( ax^2\) , up to a linear transformation in \( (x,y)\) , we can get rid of the term \( cxy\) , and we are left with:
In the case \( b\neq0\) we can make two obvious squares, and again up to a linear transformation in \( (x,y)\) , we are left with an equation as follows:
In the case of positive sign, \( x^2+y^2=k\) , the solutions are the circle, when \( k\geq0\) , the point, when \( k=0\) , and \( \emptyset\) , when \( k<0\) . As for the case of negative sign, \( x^2-y^2=k\) , which reads \( (x-y)(x+y)=k\) , here once again by linearity our equation becomes \( xy=l\) , which is a hyperbola when \( l\neq0\) , and two lines when \( l=0\) .
(4) In the case \( b\neq0\) the study is similar, with the same solutions, so we are left with the case \( a=b=0\) . Here our conic is as follows, with \( c,d,e,f\in\mathbb R\) :
If \( c\neq 0\) , by linearity our equation becomes \( xy=l\) , which produces a hyperbola or two lines, as explained before. As for the remaining case, \( c=0\) , here our equation is:
But this is generically the equation of a line, unless we are in the case \( d=e=0\) , where our equation is \( f=0\) , having as solutions \( \emptyset\) when \( f\neq0\) , and \( \mathbb R^2\) when \( f=0\) .
(5) Thus, done with the classification, up to linear transformations as in (1). But this classification leads to the classification in general too, by applying now linear transformations to the solutions that we found. So, done with this, and very good.
(6) It remains to discuss the cone cutting. But here, what we have to do is to see how the cone equation \( x^2+y^2=kz^2\) changes, under a linear change of coordinates, and then set \( z=0\) , as to get the \( (x,y)\) equation of the intersection. But this leads, via some thinking or computations, to the conclusion that the cone equation \( x^2+y^2=kz^2\) becomes in this way a degree 2 equation in \( (x,y)\) , which can be arbitrary, as desired.
Ready for some physics? And here, good news, not only what we did in the above is relevant, but is actually at the origin of the whole modern physics, thanks to:
Theorem 118
Planets and other celestial bodies move around the Sun on conics,
with \( P\in\mathbb R[x,y]\) being of degree \( 2\) , which can be ellipses, parabolas or hyperbolas.
Proof
This is something quite long, due to Kepler and Newton, as follows:
(1) The force of attraction between two bodies of masses \( M,m\) is given by:
Here \( d\) is the distance between the two bodies, and \( G\simeq 6.674\times 10^{-11}\) is a constant. Now assuming that \( M\) is fixed at the origin \( 0\in\mathbb R^2\) , the force exterted on \( m\) positioned at \( z=(x,y)\in\mathbb R^2\) , regarded as a vector \( F\in\mathbb R^2\) , is given by the following formula:
But \( F=ma=m\ddot{z}\) , with \( a=\ddot{z}\) being the acceleration, so the equation of motion of \( m\) , assuming that \( M\) is fixed at \( 0\) , and with the notation \( K=GM\) , is:
(2) Let us write now our vector \( z=(x,y)\) in polar coordinates, as follows:
We have then \( ||z||=r\) , and our equation of motion becomes:
Let us differentiate now \( x,y\) . By using the standard calculus rules, we have:
Consider now the following two quantities, appearing as coefficients in the above:
In terms of these quantities, our second derivative formulae read:
(3) We can now solve the equation of motion. Indeed, with the formulae that we found above for \( \ddot{x},\ddot{y}\) , our equation of motion takes the following form:
But these two formulae can be written in the following way:
By making now the product, and assuming that we are in a non-degenerate case, where the angle \( \theta\) varies indeed, we obtain by positivity that we must have:
(4) Let us first examine the second equation, \( b=0\) . This can be solved as follows:
As for the first equation, namely \( a+K/r^2=0\) , with \( K=\lambda^2/c\) this becomes:
Now set \( f=1/r\) . With the convention that dots mean as usual derivatives with respect to \( t\) , and that primes denote derivatives with respect to \( \theta=\theta(t)\) , we have:
By differentiating one more time with respect to \( t\) , we obtain:
Thus, in terms of \( f=1/r\) as above, our equation for \( \ddot{r}\) simply reads:
But this latter equation is easy to solve, and by inverting, we obtain:
(5) But this leads to the conclusion that the trajectory is a conic. Indeed, in terms of the parameter \( \theta\) , the formulae of the coordinates are:
We conclude from this that our coordinates \( x,y\) satisfy the following equation:
But what we have here is an equation of a conic, and we are done.
All the above was very nice, all good old things, but the continuation of the story is more complicated, because beyond conics, things ramify. A first idea, in order to generalize the conics, is to look at the algebraic manifolds, in the following sense:
Definition 40
An algebraic manifold \( X\subset\mathbb R^N\) is a space of the form
with \( P_i\in\mathbb R[x_1,…,x_N]\) being certain polynomials.
Manhy things can be said about such manifolds, both of mathematical and physical nature, and as a fundamental result here, generalizing Theorem 8.1, let us record:
Theorem 119
The degree \( 2\) hypersurfaces \( S\subset\mathbb R^N\) , called quadrics, are up to degeneracy and to linear transformations the hypersurfaces of the following form,
and with the sphere being the only compact one.
Proof
We have two statements here, the idea being as follows:
(1) The equations for a quadric \( S\subset\mathbb R^N\) are best written as follows, with \( A\in M_N(\mathbb R)\) being a matrix, \( B\in M_{1\times N}(\mathbb R)\) being a row vector, and \( C\in\mathbb R\) being a constant:
(2) By doing the linear algebra, or by invoking the theorem of Sylvester on quadratic forms, we are left, modulo linear transformations, with signed sums of squares:
(3) To be more precise, with linear algebra, by evenly distributing the terms \( x_ix_j\) above and below the diagonal, we can assume that our matrix \( A\in M_N(\mathbb R)\) is symmetric. Thus \( A\) must be diagonalizable, and by changing the basis of \( \mathbb R^N\) , as to have it diagonal, our equation becomes as follows, with \( D\in M_N(\mathbb R)\) being now diagonal:
(4) But now, by making squares in the obvious way, which amounts in applying yet another linear transformation to our quadric, the equation takes the following form, with \( G\in M_N(-1,0,1)\) being diagonal, and with \( H\in\{0,1\}\) being a constant:
(5) Now barring the degenerate cases, we can further assume \( G\in M_N(-1,1)\) , and we are led in this way to the equation claimed in (2) above, namely:
(6) In particular we see that, up to some degenerate cases, namely emptyset and point, the only compact quadric, up to linear transformations, is the one given by:
(7) But this is the unit sphere, so are led to the conclusions in the statement.
Summarizing, we have some beginning of algebraic geometry theory going on. However, for our purposes here, which are integration, along with applications to physics, although many interesting algebraic manifolds appear at the advanced level, in what concerns the basics, here we are mostly in need of a different definition, namely:
Definition 41
A smooth manifold is a space \( X\) which is locally isomorphic to \( \mathbb R^N\) . To be more precise, this space \( X\) must be covered by charts, bijectively mapping open pieces of it to open pieces of \( \mathbb R^N\) , with the changes of charts being \( C^\infty\) functions.
As a basic example, we have \( \mathbb R^N\) itself, or any open subset \( X\subset\mathbb R^N\) . Another example is the circle, or curves like ellipses and so on, for obvious reasons. However, this might seem a bit vague, and here is a more precise statement in this sense:
Theorem 120
The following are smooth manifolds, in the plane:
Proof
All this is quite intuitive, the idea being as follows:
(1) Consider the unit circle, \( x^2+y^2=1\) . We can write then \( x=\cos t\) , \( y=\sin t\) , with \( t\in[0,2\pi)\) , and we seem to have here the solution to our problem, just using 1 chart. But this is of course wrong, because \( [0,2\pi)\) is not open, and we have a problem at \( 0\) . In practice we need to use 2 such charts, say with the first one being with \( t\in(0,3\pi/2)\) , and the second one being with \( t\in(\pi,5\pi/2)\) . As for the fact that the change of charts is indeed smooth, this comes by writing down the formulae, or just thinking a bit, and arguing that this change of chart being actually a translation, it is automatically linear.
(2) This follows from (1), by pulling the circle in both the \( Ox\) and \( Oy\) directions, and the formulae here, based on our various formulae above, are left to you reader.
(3) We already have the ellipses, and the case of the parabolas and hyperbolas is elementary as well, and in fact simpler than the case of the ellipses. Indeed, a parablola is clearly homeomorphic to \( \mathbb R\) , and a hyperbola, to two copies of \( \mathbb R\) .
(4) This is something which is clear too, depending of course on what exactly we mean by “smooth deformation”, and by using a bit of multivariable calculus if needed.
In higher dimensions now, as basic examples here, we have the unit sphere in \( \mathbb R^N\) , and smooth deformations of it, once again, somehow by obvious reasons, and with the formal proof of this fact coming for instance by using the spherical coordinate formulae from chapter 7. Alternatively, we can use the stereographic projection, which provides us with a proof that the sphere \( S^{N-1}_\mathbb R\) is indeed a smooth manifold, with just 2 charts.
We would like to discuss now the integration over curves, surfaces, and other smooth manifolds. Let us start with something very basic, regarding the curves, namely:
Theorem 121
The length of a curve \( \gamma:[a,b]\to\mathbb R^N\) is given by
with \( ||.||\) being the usual norm of \( \mathbb R^N\) .
Proof
This is something quite intuitive, that can even stand as a definition for the length of the curves, and that we will not prove in detail here, the idea being as follows:
(1) To start with, what is the length of a curve? Good question, and in answer, a physicist would say that this is the quantity obtained by integrating the magnitude of the velocity vector over the curve, with respect to time. But this velocity vector is \( \gamma'(t)\) , having magnitude \( ||\gamma'(t)||\) , so we are led to the formula in the statement.
(2) Regarding now mathematicians, these would say that the length of a curve is the following quantity, with \( (t_1=a,t_2,…,t_{n-1},t_n=b)\) being a uniform division of \( (a,b)\) :
But, by using the fundamental theorem of calculus, we can write this as follows:
And the point now is that, by doing some standard analysis, that we will leave here as an instructive exercise, we are led to the formula in the statement.
(3) So, we have our definition-theorem for the length of the curves, and for the discussion to be complete, we just need an illustration for this. But here, we have the following standard computation, for the length of the ellipses \( (x/a)^2+(y/b)^2=1\) :
There are of course many other applications of the formula in the statement, and we will leave some thinking and computations here as an instructive exercise.
Next, let us talk about the contour integrals over curves. We have here:
Theorem 122
Given a path \( \gamma\subset\mathbb R^3\) , we can talk about integrals of type
with \( f,g,h:\mathbb R^3\to\mathbb R\) , which are independent on the chosen parametrization of the path.
Proof
This is something quite straightforward, the idea being as follows:
(1) Regarding the statement itself, assume indeed that we have a path in \( \mathbb R^3\) , which can be best thought of as corresponding to a function as follows:
Observe that this function \( \gamma\) is not exactly the path itself, for instance because the following functions produce the same path, parametrized differently:
Our claim, however, is that we can talk about integrals as follows, with \( f,g,h:\mathbb R^3\to\mathbb R\) , which are independent on the chosen parametrization of our path:
(2) In order to prove this, let us choose a parametrization \( \gamma:[a,b]\to\mathbb R^3\) as above. This parametrization has as components three functions \( \gamma_1,\gamma_2,\gamma_3\) , given by:
In order to construct the integral \( I\) , it is quite clear, by suitably cutting our path into pieces, that we can restrict the attention to the case where all three components \( \gamma_1,\gamma_2,\gamma_3:[a,b]\to\mathbb R\) are increasing, or decreasing. Thus, we can assume that these three components are as follows, increasing or decreasing, and bijective on their images:
(3) Moreover, by using the obvious symmetry between the coordinates \( x_1,x_2,x_3\) , in order to construct \( I\) , we just need to construct integrals of the following type:
(4) So, let us construct this latter integral \( I_1\) , under the assumptions in (2). The simplest case is when the first path, \( \gamma_1:[a,b]\to[a_1,b_1]\) , is the identity:
In other words, the simplest case is when our path is of the following form, with \( \gamma_2,\gamma_3:[a,b]\to\mathbb R\) being certain functions, that should be increasing or decreasing, as per our conventions (2) above, but in what follows we will not need this assumption:
But now, with this convention made, we can define our contour integral, or rather its first component, as explained above, as a usual one-variable integral, as follows:
(5) With this understood, let us examine now the general case, where the first path, \( \gamma_1:[a,b]\to[a_1,b_1]\) , is arbitrary, increasing or decreasing, and bijective on its image. In this case we can reparametrize our curve, as to have it as in (4) above, as follows:
Now since we want our integral \( I_1=\int_\gamma f(x)dx_1\) to be independent of the parametrization, we are led to the following formula for it, coming from the formula in (4):
Here we have used at the end the change of variable formula, with \( x_1=\gamma_1(y_1)\) .
(6) Thus, job done, we have our definition for the contour integrals, with the formula being as follows, obtained by using (5) for all three coordinates \( x_1,x_2,x_3\) :
And with this, we are led to the conclusion in the statement.
Let us record as well the following more compact form of Theorem 8.8:
Theorem 123
The contour integrals over a curve \( \gamma:[a,b]\to\mathbb R^3\) are given by
valid for any \( F:\mathbb R^3\to\mathbb R^3\) , where on the right \( \gamma'(y)dy=(\gamma_i'(y_i)dy_i)_i\) .
Proof
This is a fancy reformulation of what we did in Theorem 8.8 and its proof. Indeed, with the notation \( F=(F_1,F_2,F_3)=(f,g,h)\) , the integral computed there is:
As for the value of this integral, according to the proof of Theorem 8.8, this is:
Now observe that we can write this in a more compact way, as follows:
And we can do even better. Indeed, we have only one integral here, \( \int_a^b\) , and in order to best express the integrand, consider the formal vector in the statement, namely:
Our integrand appears then as the scalar product of \( F(\gamma(y))\) with this latter vector \( \gamma'(y)dy\) , so our formula above for the contour integral takes the following form:
Thus, we are led to the conclusion in the statement.
More concretely now, let us temporarily forget about the paths \( \gamma\) , and have a look at the quantities which are to be integrated, namely:
Obviously, these are something rather mathematical, and many things can be said here. However, we can have some physical intuition on them. Assume indeed that we are given a function as follows, that you can think for instance as corresponding to an external force, with \( F(x)\in\mathbb R^3\) being the force vector applied at a given point \( x\in\mathbb R^3\) :
By writing \( F=(F_1,F_2,F_3)\) , we can then consider the following quantity, and when \( F:\mathbb R^3\to\mathbb R^3\) varies, we obtain exactly the abstract quantities \( \alpha\) considered above:
Thus, all in all, what we have done in the above with our construction of contour integrals, was to define quantities as follows, with \( \gamma\) being a path in \( \mathbb R^3\) , and with \( F:\mathbb R^3\to\mathbb R^3\) being a certain function, that we can think of, if we want, as being a force:
Which brings us into physics. Indeed, by assuming now that \( F:\mathbb R^3\to\mathbb R^3\) does correspond to a force, we can formulate the following definition:
Definition 42
The work done by a force \( F=F(x)\) for moving a particle from point \( p\in\mathbb R^3\) to point \( q\in\mathbb R^3\) via a given path \( \gamma:p\to q\) is the following quantity:
We say that \( F\) is conservative if this work quantity \( W(\gamma)\) does not depend on the chosen path \( \gamma:p\to q\) , and in this case we denote this quantity by \( W(p,q)\) .
This definition is something quite subtle, and as a first comment, assume that we have two paths \( \gamma:p\to q\) and \( \delta:p\to q\) . We can then consider the path \( \circ:p\to p\) obtained by going along \( \gamma:p\to q\) , and then along \( \delta\) reversed, \( \delta^{-1}:q\to p\) , and we have:
Thus \( F\) is conservative precisely when, for any loop \( \circ:p\to p\) , we have:
Intuitively, this means that \( F\) is some sort of “clean”, ideal force, with no dirty things like friction involved. As we will soon see, gravity is such a clean force, with a simple example coming from throwing a rock up in the sky. That rock will travel on a loop \( p\to q\to p\) , and will come back here to \( p\) unchanged, save for the fact that its speed vector is reversed. Thus, and assuming now that work has something to do with energy, which is intuitive, there has been no overall work of gravity on this loop, \( W(\circ)=0\) .
As a first result now, regarding the conservative forces, we have:
Theorem 124
The work done by a conservative force \( F\) on a mass \( m\) object is
with \( T=m||v||^2/2\) standing as usual for the kinetic energy of the object.
Proof
Assuming that \( F\) is conservative, and acts via the usual formula \( F=ma\) on our object of mass \( m\) , we have the following computation, as desired:
Here we have used in the middle the fact that the time derivative of a scalar product of functions \( <v,w>\) consists of two terms, which are equal when \( v=w\) .
In order to formulate our next result, observe that we have the following computation for the contour integrals of gradients, which is independent on the chosen path:
To be more precise, this computation certainly works when \( V\) is a function of just one variable, \( x_1\) , \( x_2\) or \( x_3\) , thanks to the fundamental theorem of calculus, and the general case follows from this, by using the chain rule for derivatives.
Now with this formula in hand, we can formulate the following result, which is quite conceptual, and which includes some basic gravitation physics too, at the end:
Theorem 125
A force \( F\) is conservative precisely when it is of the form
for a certain function \( V\) , and in this case the work done by it is given by:
Also, the gravitation force is conservative, coming from \( V=-C/||x||\) , with \( C>0\) .
Proof
This is something quite tricky, the idea being as follows:
(1) In one sense, assume that \( F\) is conservative. Since the work \( W(p,q)=W(\gamma)\) is independent of the chosen path \( \gamma:p\to q\) , we can find a function \( V\) such that:
Observe that this function \( V\) is well-defined up to an additive constant. Now with this formula in hand, we further obtain, as desired:
(2) In the other sense now, assuming \( F=-\nabla V\) , we have the following computation, valid for any loop \( \circ:p\to p\) , which shows that \( F\) is indeed conservative:
More generally, regarding the work done by such a force \( F=-\nabla V\) , along a path \( \gamma:p\to q\) , which is independent on this path \( \gamma\) , this is given by:
(3) Finally, regarding the last assertion, we will leave this as an exercise.
We can now put everything together, and we have the following result:
Theorem 126
Given a conservative force \( F\) , appearing as follows, with \( V\) being uniquely determined up to an additive constant,
the movements of a particle under \( F\) preserve the total energy, given by
with \( T=m||v||^2/2\) being the kinetic energy, and with \( V\) being called potential energy.
Proof
By using Theorem 8.11 and Theorem 8.12, we have:
Now observe that these equalities give the following formula:
Thus, the total energy \( E=T+V\) is conserved, as claimed.
At the level of mathematics now, in relation with the above, a useful result is:
Theorem 127 (Green)
Given a plane curve \( C\subset\mathbb R^2\) , we have the formula
where \( D\subset\mathbb R^2\) is the domain enclosed by \( C\) .
Proof
Assume indeed that we have a plane curve \( C\subset\mathbb R^2\) , without self-intersections, which is piecewise \( C^1\) , and is assumed to be counterclockwise oriented. In order to prove the formula regarding \( P\) , we can parametrize the enclosed domain \( D\) as follows:
We have then the following computation, which gives the result for \( P\) :
As for the result for the \( Q\) term, the computation here is similar.
Moving forward to 3D, let us start with a standard definition, as follows:
Definition 43
The vector product of two vectors in \( \mathbb R^3\) is given by
where \( n\in\mathbb R^3\) with \( n\perp x,y\) and \( ||n||=1\) is constructed using the right-hand rule:
Alternatively, in usual vertical linear algebra notation for all vectors,
the rule being that of computing \( 2\times2\) determinants, and adding a middle sign.
In practice now, in order to get familiar with the vector products, nothing better than doing some classical mechanics. We have here the following key result:
Theorem 128
In the gravitational \( 2\) -body problem, the angular momentum
with \( p=mv\) being the usual momentum, is conserved.
Proof
There are several things to be said here, the idea being as follows:
(1) First of all the usual momentum, \( p=mv\) , is not conserved, because the simplest solution is the circular motion, where the moment gets turned around. But this suggests precisely that, in order to fix the lack of conservation of the momentum \( p\) , what we have to do is to make a vector product with the position \( x\) . Leading to \( J\) , as above.
(2) Regarding now the proof, consider indeed a particle \( m\) moving under the gravitational force of a particle \( M\) , assumed, as usual, to be fixed at 0. By using the fact that for two proportional vectors, \( p\sim q\) , we have \( p\times q=0\) , we obtain:
Now since the derivative of \( J\) vanishes, this quantity is constant, as stated.
At the mathematical level now, we have the following key result:
Theorem 129
The area of a surface \( S\subset\mathbb R^3\) , parametrized as \( S=r(D)\) with \( r:D\to\mathbb R^3\) and \( D\subset\mathbb R^2\) , is given by
with \( r_x,r_y:D\to\mathbb R^3\) being the partial derivatives of \( r\) , and \( ||.||\) being the norm of \( \mathbb R^3\) .
Proof
Thus is something quite similar to Theorem 8.7, and many things can be said here, at the theoretical level, in analogy with those for the curves. Among others, let us mention that we can talk, more generally, about surface integrals, defined as follows:
As for the basic illustrations of the formula in the statement, consider a surface of type \( z=f(x,y)\) . Here we have \( r(x,y)=(x,y,f(x,y))\) , and we obtain:
There are of course many other applications of the formula in the statement, and we will leave some thinking and computations here as an instructive exercise.
Many other things can be said, as a continuation of the above, but let us not deviate too much, from what we wanted to do here. Next, comes the following definition:
Definition 44
We can talk about the divergence of \( \varphi:\mathbb R^3\to\mathbb R^3\) , as being
as well as about the curl of the same function \( \varphi:\mathbb R^3\to\mathbb R^3\) , as being
where \( u_x,u_y,u_z\) are the unit vectors along the coordinate directions \( x,y,z\) .
Getting back now to calculus tools, in 3 dimensions, we have the following result:
Theorem 130 (Stokes)
Given a smooth oriented surface \( S\subset\mathbb R^3\) , with boundary \( C\subset\mathbb R^3\) , and a vector field \( F\) , we have the following formula:
In other words, the line integral of a vector field \( F\) over a loop \( C\) equals the surface integral of the curl of the vector field \( \varphi\) , over the enclosed surface \( S\) .
Proof
This basically follows from the Green theorem, the idea being as follows:
(1) Let us first parametrize our surface \( S\) , and its boundary \( C\) . We can assume that we are in the situation where we have a closed oriented curve \( \gamma:[a,b]\to\mathbb R^2\) , with interior \( D\subset\mathbb R^2\) , and where the surface appears as \( S=\psi(D)\) , with \( \psi:D\to\mathbb R^3\) . In this case, the function \( \delta=\psi\circ\gamma\) parametrizes the boundary of our surface, \( C=\delta[a,b]\) .
(2) Let us first look at the integral on the right in the statement. We have the following formula, with \( J_y(\psi)\) standing for the Jacobian of \( \psi\) at the point \( y=\gamma(t)\) :
In order to further process this formula, let us introduce the following function:
In terms of this function, we have the following formula for our line integral:
(3) In order to compute now the other integral in the statement, we first have:
We conclude that the integral on the left in the statement is given by:
(4) But with this, we are done, because the integrals computed in (2) and (3) are indeed equal, due to the Green theorem. Thus, the Stokes formula holds indeed.
As a conclusion to what we did so far, we have the following statement:
Theorem 131
The following results hold, in \( 3\) dimensions:
Fundamental theorem for gradients, namely
Fundamental theorem for divergences, or Gauss or Green formula,
Fundamental theorem for curls, or Stokes formula,
where \( S\) is the boundary of the body \( B\) , and \( P\) is the boundary of the area \( A\) .
Proof
This follows indeed from the various formulae established above.
We would like to discuss now, following Gauss and others, some applications of the above to electrostatics, and why not to electrodynamics too. Let us start with:
Fact 9 (Coulomb law)
Any pair of charges \( q_1,q_2\in\mathbb R\) is subject to a force as follows, which is attractive if \( q_1q_2<0\) and repulsive if \( q_1q_2>0\) ,
where \( d>0\) is the distance between the charges, and \( K>0\) is a certain constant.
Observe the amazing similarity with the Newton law for gravity. However, as we will soon see, passed a few simple facts, things will be more complicated here.
In analogy with our study of gravity, let us start with:
Definition 45
Given charges \( q_1,…,q_k\in\mathbb R\) located at positions \( x_1,…,x_k\in\mathbb R^3\) , we define their electric field to be the vector function
so that their force applied to a charge \( Q\in\mathbb R\) positioned at \( x\in\mathbb R^3\) is given by \( F=QE\) .
Observe the analogy with gravity, save for the fact that instead of masses \( m>0\) we have now charges \( q\in\mathbb R\) , and that at the level of constants, \( G\) gets replaced by \( K\) .
More generally, we will be interested in electric fields of various non-discrete charge configurations, such as charged curves, surfaces and solid bodies. So, let us formulate:
Definition 46
The electric field of a charge configuration \( L\subset\mathbb R^3\) , with charge density function \( \rho:L\to\mathbb R\) , is the vector function
so that the force of \( L\) applied to a charge \( Q\) positioned at \( x\) is given by \( F=QE\) .
With the above definitions in hand, it is most convenient now to forget about the charges, and focus on the study of the corresponding electric fields \( E\) .
These fields are by definition vector functions \( E:\mathbb R^3\to\mathbb R^3\) , with the convention that they take \( \pm\infty\) values at the places where the charges are located, and intuitively, are best represented by their field lines, which are constructed as follows:
Definition 47
The field lines of an electric field \( E:\mathbb R^3\to\mathbb R^3\) are the oriented curves \( \gamma\subset\mathbb R^3\) pointing at every point \( x\in\mathbb R^3\) at the direction of the field, \( E(x)\in\mathbb R^3\) .
As a basic example here, for one charge the field lines are the half-lines emanating from its position, oriented according to the sign of the charge:
For two charges now, if these are of opposite signs, \( +\) and \( -\) , you get a picture that you are very familiar with, namely that of the field lines of a bar magnet:
If the charges are \( +,+\) or \( -,-\) , you get something of similar type, but repulsive this time, with the field lines emanating from the charges being no longer shared:
These pictures, and notably the last one, with \( +,+\) charges, are quite interesting, because the repulsion situation does not appear in the context of gravity. Thus, we can only expect our geometry here to be far more complicated than that of gravity.
In general now, the first thing that can be said about the field lines is that, by definition, they do not cross. Thus, what we have here is some sort of oriented 1D foliation of \( \mathbb R^3\) , in the sense that \( \mathbb R^3\) is smoothly decomposed into oriented curves \( \gamma\subset\mathbb R^3\) .
The field lines, as constructed in Definition 8.24, obviously do not encapsulate the whole information about the field, with the direction of each vector \( E(x)\in\mathbb R^3\) being there, but with the magnitude \( ||E(x)||\geq0\) of this vector missing. However, say when drawing, when picking up uniformly radially spaced field lines around each charge, and with the number of these lines proportional to the magnitude of the charge, and then completing the picture, the density of the field lines around each point \( x\in\mathbb R\) will give you the magnitude \( ||E(x)||\geq0\) of the field there, up to a scalar. So, let us formulate:
Proposition 30
Given an electric field \( E:\mathbb R^3\to\mathbb R^3\) , the knowledge of its field lines is the same as the knowledge of the composition
where \( S\subset\mathbb R^3\) is the unit sphere, and \( n:\mathbb R^3\to S\) is the rescaling map, namely:
However, in practice, when the field lines are accurately drawn, the density of the field lines gives you the magnitude of the field, up to a scalar.
Proof
We have two assertions here, the idea being as follows:
(1) The first assertion is clear from definitions, with of course our usual convention that the electric field and its problematics take place outside the locations of the charges, which makes everything in the statement to be indeed well-defined.
(2) Regarding now the last assertion, which is of course a bit informal, this follows from the above discussion. It is possible to be a bit more mathematical here, with a definition, formula and everything, but we will not need this, in what follows.
Let us introduce now a key definition, as follows:
Definition 48
The flux of an electric field \( E:\mathbb R^3\to\mathbb R^3\) through a surface \( S\subset\mathbb R^3\) , assumed to be oriented, is the quantity
with \( n(x)\) being unit vectors orthogonal to \( S\) , following the orientation of \( S\) . Intuitively, the flux measures the signed number of field lines crossing \( S\) .
Here by orientation of \( S\) we mean precisely the choice of unit vectors \( n(x)\) as above, orthogonal to \( S\) , which must vary continuously with \( x\) . For instance a sphere has two possible orientations, one with all these vectors \( n(x)\) pointing inside, and one with all these vectors \( n(x)\) pointing outside. More generally, any surface has locally two possible orientations, so if it is connected, it has two possible orientations. In what follows the convention is that the closed surfaces are oriented with each \( n(x)\) pointing outside.
Regarding the last sentence of Definition 8.26, this is of course something informal, meant to help, coming from the interpretation of the field lines from Proposition 8.25. However, we will see later that this simple interpretation can be of great use.
As a first illustration, let us do a basic computation, as follows:
Proposition 31
For a point charge \( q\in\mathbb R\) at the center of a sphere \( S\) ,
where the constant is \( \varepsilon_0=1/(4\pi K)\) , independently of the radius of \( S\) .
Proof
Assuming that \( S\) has radius \( r\) , we have the following computation:
Thus with \( \varepsilon_0=1/(4\pi K)\) as above, we obtain the result.
More generally now, we have the following key result, due to Gauss, which is the foundation of advanced electrostatics, and of everything following from it, namely electrodynamics, and then quantum mechanics, and particle physics:
Theorem 132 (Gauss law)
The flux of a field \( E\) through a surface \( S\) is given by
where \( Q_{enc}\) is the total charge enclosed by \( S\) , and \( \varepsilon_0=1/(4\pi K)\) .
Proof
This basically follows from Proposition 8.27, a bit modified, by adding to the computation there a number of standard ingredients. We refer here for instance to Feynman [6], but we will be back to this right next, with a more advanced proof.
In relation now with our previous mathematics, we have the following result:
Theorem 133
Given an electric potential \( E\) , its divergence is given by
where \( \rho\) denotes as usual the charge distribution. Also, we have
meaning that the curl of \( E\) vanishes.
Proof
We have several assertions here, the idea being as follows:
(1) The first formula, called Gauss law in differential form, follows from:
Now since this must hold for any \( B\) , this gives the formula in the statement.
(2) Regarding the curl, by discretizing and linearity we can assume that we are dealing with a single charge \( q\) , positioned at \( 0\) . We have, by using spherical coordinates \( r,s,t\) :
In particular the integral of \( E\) over any closed loop vanishes, and by using now the Stokes formula, we conclude that the curl of \( E\) vanishes, as stated.
So long for electrostatics, which provide a good motivation and illustration for our mathematics. When upgrading to electrodynamics, things become even more interesting, because our technology can be used in order to understand the Maxwell equations:
Theorem 134
Electrodynamics is governed by the formulae
called Maxwell equations.
Proof
This is something fundamental, coming as a tricky mixture of physics and mathematics. To be more precise, the first formula is the Gauss law, \( \rho\) being the charge, and \( \varepsilon_0\) being a constant, and with this Gauss law more or less replacing the Coulomb law from electrostatics. The second formula is something basic, and anonymous. The third formula is the Faraday law. As for the fourth formula, this is the Amp\‘ere law, as modified by Maxwell, with \( J\) being the volume current density, and \( \mu_0\) being a constant.
However, the above is not all. Quite remarkably, the constants \( \mu_0,\varepsilon_0\) are related by the following formula, due to Biot-Savart, with \( c\) being the speed of light:
So, what has light to do with all this? The idea is that accelerating or decelerating charges produce electromagnetic radiation, of various wavelengths, called light, of various colors, and with all this coming from the mathematics of the Maxwell equations.
We had an exciting geometric chapter here, and as exercises on this, we have:
Exercise 57
Clarify all the details in the classification of the conics.
Exercise 58
Learn about the stereographic projection, and cartography in general.
Exercise 59
Compute the lengths of some curves, of your choice.
Exercise 60
Learn more about conservative forces, and their properties.
Exercise 61
Compute the areas of some surfaces, of your choice.
Exercise 62
Clarify all the details in the proof of the Stokes formula.
Exercise 63
Try to compute the field lines, for simple charge configurations.
Exercise 64
Learn more about the Gauss law, and the Maxwell equations.
As bonus exercise, read some systematic differential geometry, and why not some algebraic geometry too, for curves, surfaces, and other manifolds.
Wanted man in California
Wanted man in Buffalo
Wanted man in Kansas City
Wanted man in Ohio
Welcome to function space theory, also known as functional analysis. Although, for being fully honest, the basics here, which are quite algebraic, rather deserve the name “functional algebra”. But do not worry, we will keep things as analytic as possible.
The idea is that various types of functions form various types of infinite dimensional complex spaces, that we can study with the usual methods from linear algebra, complemented for dealing with \( \infty\) by our favorite logic weapon, which is the Zorn lemma.
Let us start with something very general, as follows:
Definition 49
A normed space is a complex vector space \( V\) , which can be finite or infinite dimensional, together with a map
called norm, subject to the following conditions:
As a basic example here, which is finite dimensional, we have the space \( V=\mathbb C^N\) , with the norm on it being the usual length of the vectors, namely:
Indeed, for this space (1) is clear, (2) is clear too, and (3) is something well-known, which is equivalent to the triangle inequality in \( \mathbb C^N\) , and which can be deduced from the Cauchy-Schwarz inequality. More on this, with some generalizations, in a moment.
Getting back now to the general case, we have the following result:
Proposition 32
Any normed vector space \( V\) is a metric space, with
as distance. If this metric space is complete, we say that \( V\) is a Banach space.
Proof
This follows from the definition of the metric spaces, as follows:
(1) The first distance axiom, \( d(x,y)\geq0\) , and \( d(x,y)=0\) precisely when \( x=y\) , follows from the fact that the norm takes values in \( \mathbb R_+\) , and from \( ||x||=0\implies x=0\) .
(2) The second distance axiom, which is the symmetry one, \( d(x,y)=d(y,x)\) , follows from our condition \( ||\lambda x||=|\lambda|\cdot||x||\) , with \( \lambda=-1\) .
(3) As for the third distance axiom, which is the triangle inequality \( d(x,y)\leq d(x,z)+d(y,z)\) , this follows from our third norm axiom, namely \( ||x+y||\leq||x||+||y||\) .
Very nice all this, and it is possible to develop some general theory here, but before everything, however, we need more examples, besides \( \mathbb C^N\) with its usual norm.
However, these further examples are actually quite tricky to construct, needing some inequality know-how. Let us start with a very basic result, as follows:
Theorem 135 (Jensen)
Given a convex function \( f:\mathbb R\to\mathbb R\) , we have the following inequality, for any \( x_1,…,x_N\in\mathbb R\) , and any \( \lambda_1,…,\lambda_N>0\) summing up to \( 1\) ,
with equality when \( x_1=…=x_N\) . In particular, by taking the weights \( \lambda_i\) to be all equal, we obtain the following inequality, valid for any \( x_1,…,x_N\in\mathbb R\) ,
and once again with equality when \( x_1=…=x_N\) . We have a similar statement holds for the concave functions, with all the inequalities being reversed.
Proof
This is indeed something quite routine, the idea being as follows:
(1) First, we can talk about convex functions in a usual, intuitive way, with this meaning by definition that the following inequality must be satisfied:
(2) But this means, via a simple argument, by approximating numbers \( t\in[0,1]\) by sums of powers \( 2^{-k}\) , that for any \( t\in[0,1]\) we must have:
Alternatively, via yet another simple argument, this time by doing some geometry with triangles, this means that we must have:
But then, again alternatively, by combining the above two simple arguments, the following must happen, for any \( \lambda_1,…,\lambda_N>0\) summing up to \( 1\) :
(3) Summarizing, all our Jensen inequalities, at \( N=2\) and at \( N\in\mathbb N\) arbitrary, are equivalent. The point now is that, if we look at what the first Jensen inequality, that we took as definition for the convexity, means, this is simply equivalent to:
(4) Thus, we are led to the conclusions in the statement, regarding the convex functions. As for the concave functions, the proof here is similar. Alternatively, we can say that \( f\) is concave precisely when \( -f\) is convex, and get the results from what we have.
As a basic application of the Jensen inequality, we have:
Proposition 33
For \( p\in(1,\infty)\) we have the following inequality,
and for \( p\in(0,1)\) we have the following reverse inequality,
with in both cases equality precisely when \( |x_1|=…=|x_N|\) .
Proof
This follows indeed from Theorem 9.3, because we have:
Thus \( x^p\) is convex for \( p>1\) and concave for \( p<1\) , which gives the results.
Observe that, more generally, Theorem 9.3 applied with \( f(x)=x^p\) with \( p\in(1,\infty)\) shows that we have the following inequality, whenever \( \lambda_i>0\) sum up to 1:
For exponents \( p\in(0,1)\) we have the same inequality, reversed, namely:
Observe also that at \( p=2\) we obtain as particular case of the above inequality the Cauchy-Schwarz inequality, or rather something equivalent to it, namely:
We will be back to this later on in this book, when talking scalars products and Hilbert spaces, with some more conceptual proofs for such inequalities.
As another basic application of the Jensen inequality, we have:
Theorem 136 (Young)
We have the following inequality,
valid for any \( a,b\geq0\) , and any exponents \( p,q>1\) satisfying \( \frac{1}{p}+\frac{1}{q}=1\) .
Proof
We use the logarithm function, which is concave on \( (0,\infty)\) , due to:
Thus we can apply the Jensen inequality, and we obtain in this way:
Now by exponentiating, we obtain the Young inequality.
Observe that for the simplest exponents, namely \( p=q=2\) , the Young inequality gives something which is trivial, but is very useful and basic, namely:
In general, the Young inequality is something non-trivial, and the idea with it is that “when stuck with a problem, and with \( ab\leq\frac{a^2+b^2}{2}\) not working, try Young”. We will be back to this general principle, later in this book, with some illustrations.
Moving forward now, as a consequence of the Young inequality, we have:
Theorem 137 (Hölder)
Assuming that \( p,q\geq 1\) are conjugate, in the sense that
we have the following inequality, valid for any two vectors \( x,y\in\mathbb C^N\) ,
with the convention that an \( \infty\) exponent produces a \( \max|x_i|\) quantity.
Proof
This is something very standard, the idea being as follows:
(1) Assume first that we are dealing with finite exponents, \( p,q\in(1,\infty)\) . By linearity we can assume that \( x,y\) are normalized, in the following way:
In this case, we want to prove that the following inequality holds:
For this purpose, we use the Young inequality, which gives, for any \( i\) :
By summing now over \( i=1,…,N\) , we obtain from this, as desired:
(2) In the case \( p=1\) and \( q=\infty\) , or vice versa, the inequality holds too, trivially, with the convention that an \( \infty\) exponent produces a max quantity, according to:
Thus, we are led to the conclusion in the statement.
As a consequence now of the Hölder inequality, we have:
Theorem 138 (Minkowski)
Assuming \( p\in[1,\infty]\) , we have the inequality
for any two vectors \( x,y\in\mathbb C^N\) , with our usual conventions at \( p=\infty\) .
Proof
We have indeed the following estimate, using the Hölder inequality, and the conjugate exponent \( q\in[1,\infty]\) , given by \( 1/p+1/q=1\) :
Here we have used the following fact, at the end:
Now by dividing both sides by the last quantity at the end, we obtain:
Thus, we are led to the conclusion in the statement.
Good news, done with inequalities, and as a consequence of the above results, and more specifically of the Minkowski inequality obtained above, we can formulate:
Theorem 139
Given an exponent \( p\in[1,\infty]\) , the formula
with usual conventions at \( p=\infty\) , defines a norm on \( \mathbb C^N\) , making it a Banach space.
Proof
Here the normed space assertion follows from the Minkowski inequality, established above, and the Banach space assertion is trivial, because our space being finite dimensional, by standard linear algebra all the Cauchy sequences converge.
We will see later in this chapter that Theorem 9.8 has several generalizations, with some of them dealing with functions. Before that, however, you might have another question regarding Theorem 9.8, namely, what is the underlying geometry there.
So, let us discuss this latter question. We will actually see that, with a bit of geometric know-how, not only we will reach to a better understanding of the \( p\) -norm on \( \mathbb C^N\) , but we will also be able to recover the H
specialChar{34}older inequality for it, without any computation.
It is convenient to talk about \( \mathbb R^N\) instead of \( \mathbb C^N\) . Let us start with:
Proposition 34
For any exponent \( p>1\) , the following set
is a smooth submanifold of \( \mathbb R^N\) .
Proof
We know from chapter 8 that the unit sphere in \( \mathbb R^N\) is a smooth manifold. In our terms, this solves our problem at \( p=2\) , because this unit sphere is:
Now observe that we have a bijection \( S_p\simeq S_2\) , at least on the part where all the coordinates are positive, \( x_i>0\) , given by the following function:
Thus we obtain that \( S_p\) is indeed a manifold, as claimed.
We already know that the manifold \( S_p\) constructed above is the unit sphere, in the case \( p=2\) . In order to have a better geometric picture of what is going on, in general, observe that \( S_p\) can be constructed as well at \( p=1\) , as follows:
However, this is no longer a manifold, as we can see for instance at \( N=2\) , where we obtain a square. Now observe that we can talk as well about \( p=\infty\) , as follows:
This letter set is no longer a manifold either, as we can see for instance at \( N=2\) , where we obtain a again a square, containing the previous square, the one at \( p=1\) .
With these limiting constructions in hand, we can now have a better geometric picture of what is going on, in the general context of Proposition 9.9. Indeed, let us draw, at \( N=2\) for simplifying, our sets \( S_p\) at the values \( p=1,2,\infty\) of the exponent: 
We can see that what we have is a small square, at \( p=1\) , becoming smooth and inflating towards the circle, in the parameter range \( p\in(1,2]\) , and then further inflating, in the parameter range \( p\in[2,\infty)\) , towards the big square appearing at \( p=\infty\) .
Now let us get to our second objective, proving Hölder, without computations. We will use Lagrange multipliers, whose basic theory can be summarized as follows:
Fact 10
In order for a function \( f:X\to\mathbb R\) defined on a manifold \( X\) to have a local extremum at \( x\in X\) , we must have, as usual
but with this taking into account the fact that the equations defining the manifold count as well as “zero”, and so must be incorporated into the formula \( f'(x)=0\) .
To be more precise, we must have a formula as follows, with \( g_i\) being the constraint functions for \( X\) , and with \( \lambda_i\in\mathbb R\) being certain scalars, called Lagrange multipliers:
With these preliminaries in hand, we can formulate our result, as follows:
Theorem 140
The local extrema over \( S_p\) of the function
can be computed by using Lagrange multipliers, and this gives
with \( 1/p+1/q=1\) , that is, the Hölder inequality, with a purely geometric proof.
Proof
We can restrict the attention to the case where all the coordinates are positive, \( x_i>0\) and \( y_i>0\) . The derivative of the function in the statement is:
On the other hand, we know that the manifold \( S_p\) appears by definition as the set of zeroes of the function \( \varphi(x)=\sum_ix_i^p-1\) , having derivative as follows:
Thus, by using Lagrange multipliers, the critical points of \( f\) must satisfy:
In other words, the critical points must satisfy \( x_i=\lambda y_i^{1/(p-1)}\) , for some \( \lambda>0\) , and by using now \( \sum_ix_i^p=1\) we can compute the precise value of \( \lambda\) , and we get:
Now let us see what this means. Since the critical point is unique, this must be a maximum of our function, and we conclude that for any \( x\in S_p\) , we have:
Thus we have Höolder, and the general case follows from this, by rescaling.
Very nice all the above, but you might wonder at this point, what is the relation of all this with functions. In answer, Theorem 9.8 can be reformulated as follows:
Theorem 141
Given an exponent \( p\in[1,\infty]\) , the formula
with usual conventions at \( p=\infty\) , defines a norm on the space of functions
making it a Banach space.
Proof
This is a just fancy reformulation of Theorem 9.8, by using the fact that the space formed by the functions \( f:\{1,…,N\}\to\mathbb C\) is canonically isomorphic to \( \mathbb C^N\) , in the obvious way, and by replacing the sums from the \( \mathbb C^N\) context with integrals with respect to the counting measure on \( \{1,…,N\}\) , in the function context.
Moving now towards infinite dimensions and more standard analysis, the idea will be that of extending Theorem 9.12 to the arbitrary measured spaces. Let us start with:
Theorem 142
Given an exponent \( p\in[1,\infty]\) , the formula
with usual conventions at \( p=\infty\) , defines a norm on the space of sequences
making it a Banach space.
Proof
As before with the finite sequences, the normed space assertion follows from the Minkowski inequality, established above, which extends without problems to the case of the infinite sequences, and with the Banach space assertion being clear too.
We can unify and generalize what we have, in the following way:
Theorem 143
Given a discrete measured space \( X\) , and an exponent \( p\in[1,\infty]\) ,
with usual conventions at \( p=\infty\) , defines a norm on the space of functions
making it a Banach space.
Proof
This is just a fancy reformulation of what we have:
(1) The case where \( X\) is finite corresponds to Theorem 9.12.
(2) The case where \( X\) is countable corresponds to Theorem 9.13.
(3) Finally, the case where \( X\) is uncountable is easy to deal with too, by using the same arguments as in the countable case.
Before going further, let us mention that the \( l^p\) spaces above are not the only interesting Banach spaces of sequences. Indeed, we have as well the following result:
Theorem 144
The following space of complex sequences,
is a Banach space, with the sup norm, and we have \( c_0\subset l^\infty\) , proper inclusion.
Proof
We can define indeed \( c_0\) as above, as being the space of sequences \( x_i\in\mathbb C\) satisfying \( x_i\to0\) , with the sup norm. We have then an inclusion as follows:
It is then clear that \( c_0\) is closed, so it is indeed a Banach space, as claimed.
Now back to the \( l^p\) spaces, in order to further extend the above results, to the case of the arbitrary measured spaces \( X\) , which are not necessarily discrete, let us start with:
Theorem 145
Given two functions \( f,g:X\to\mathbb C\) and an exponent \( p\geq1\) , we have
called Minkowski inequality. Also, assuming that \( p,q\geq1\) satisfy \( 1/p+1/q=1\) , we have
called Hölder inequality. These inequalities hold as well for \( \infty\) values of the exponents.
Proof
This is very standard, exactly as in the case of sequences, finite or not, but since the above inequalities are really very general and final, here are the details:
(1) Let us first prove Hölder, in the case of finite exponents, \( p,q\in(1,\infty)\) . By linearity we can assume that \( f,g\) are normalized, in the following way:
In this case, we want to prove that the following inequality holds:
For this purpose, we use the Young inequality, which gives, for any \( x\in X\) :
By integrating now over \( x\in X\) , we obtain from this, as desired:
(2) Let us prove now Minkowski, again in the finite exponent case, \( p\in(1,\infty)\) . We have the following estimate, using the Hölder inequality, and the conjugate exponent:
Thus, we are led to the Minkowski inequality in the statement.
(3) Finally, in the infinite exponent cases we have similar results, with the convention that an \( \infty\) exponent produces an essential supremum, according to:
Thus, we are led to the conclusion in the statement.
We can now extend Theorem 9.14, into something very general, as follows:
Theorem 146
Given a measured space \( X\) , and \( p\in[1,\infty]\) , the following space, with the convention that functions are identified up to equality almost everywhere,
is a vector space, and the following quantity
is a norm on it, making it a Banach space.
Proof
This follows indeed from Theorem 9.16, with due attention to the null sets, and this because of the first normed space axiom, namely:
To be more precise, in order for this axiom to hold, we must identify the functions up to equality almost everywhere, as indicated in the statement.
Summarizing, very nice all this. So, we have our examples of Banach spaces, which look definitely interesting, and related to analysis. In the remainder of this chapter we will develop some general Banach space theory, and apply it to the above \( L^p\) spaces.
Getting now to work, as a first result about the abstract normed spaces, we would like to talk about the linear maps \( T:V\to W\) , which can be thought of as being some kind of infinite matrices, when \( V,W\) are infinite dimensional. We first have here:
Proposition 35
For a linear map \( T:V\to W\) , the following conditions are equivalent, and if they hold, we say that \( T\) is bounded:
Proof
This is something elementary, the idea being as follows:
\( (1)\iff(2)\) This is indeed clear from definitions, by using the linearity of \( T\) , and performing various rescalings.
\( (2)\iff(3)\) This is clear from definitions too, again by using the linearity of \( T\) , and performing various rescalings.
\( (3)\iff(4)\) This is clear from definitions too, with the number \( ||T||\) needed in (4) being the bound coming from (3).
With the above result in hand, we can now formulate:
Theorem 147
Given two Banach spaces \( V,W\) , the bounded linear maps
form a linear space \( B(V,W)\) , on which the following quantity is a norm,
making \( B(V,W)\) a Banach space. When \( V=W\) , we obtain a Banach algebra.
Proof
All this is very standard, and in the case \( V=W\) , the proof goes as follows:
(1) The fact that we have indeed an algebra, satisfying the product condition in the statement, follows from the following estimates, which are all elementary:
(2) Regarding now the last assertion, if \( \{T_n\}\subset B(V)\) is Cauchy then \( \{T_nx\}\) is Cauchy for any \( x\in V\) , so we can define the limit \( T=\lim_{n\to\infty}T_n\) by setting:
Let us first check that the application \( x\to Tx\) is linear. We have:
Similarly, we have as well the following computation:
Thus we have a linear map \( T:A\to A\) . It remains to prove that we have \( T\in B(V)\) , and that we have \( T_n\to T\) in norm. For this purpose, observe that we have:
As a first consequence, we obtain \( T\in B(V)\) , because we have:
As a second consequence, we obtain \( T_N\to T\) in norm, and we are done.
As a basic example for the above construction, in the case where both our spaces are finite dimensional, \( V=\mathbb C^N\) and \( W=\mathbb C^M\) , with \( N,M<\infty\) , we obtain a matrix space:
More on this later. On the other hand, of particular interest is as well the case \( W=\mathbb C\) of the above construction, which leads to the following result:
Theorem 148
Given a Banach space \( V\) , its dual space, constructed as
is a Banach space too, with norm given by:
When \( V\) is finite dimensional, we have \( V\simeq V^*\) .
Proof
This is clear indeed from Theorem 9.19, because we have:
Thus, we are led to the conclusions in the statement.
In order to better understand the linear forms, we will need:
Theorem 149 (Hahn-Banach)
Given a Banach space \( V\) , the following happen:
Proof
This is something quite tricky, the idea being as follows:
(1) As a first observation, the assertion (1) is a particular case of the assertion (2). Indeed, given a nonzero vector \( x\in V-\{0\}\) , we can consider the linear space \( W=\mathbb Cx\) , and construct a linear form \( f\in W^*\) by the following formula:
But with this done, any extension \( \tilde{f}\in V^*\) of this linear form \( f\in W^*\) , as provided by (2), does the job for (1), and this because we have \( \tilde{f}(x)=f(x)=1\) .
(2) As a comment now, you might wonder why keeping both (1,2) in the statement of the theorem. In answer, the Hahn-Banach theorem is (2), while (1) is the main consequence of this theorem, which is something very useful, in practice. So, with our theorem formulated as above, we have there all we need to know, about Hahn-Banach.
(3) As a second remark now, assuming that we are in finite dimensions, given a subspace \( W\subset V\) as in (2), we can always find a direct sum decomposition, as follows:
But once we have such a decomposition, given a linear form \( f\in W^*\) as in (2), we can define an extension \( \tilde{f}\in V^*\) by the following formula:
Thus we have proved a weak form of (2), without reference to the norm of the extension, which is still stronger than (1), in finite dimensions.
(4) Still staying in finite dimensions, it is possible to further build on this, as to have the equality \( ||\tilde{f}||=||f||\) happening too, as required by (2). For instance in the case of the space \( V=\mathbb C^N\) , with its usual norm, we can use the following decomposition, and we will leave the proof of the equality \( ||\tilde{f}||=||f||\) in this case as an easy exercise:
However, in the general finite dimensional case, that of the space \( V=\mathbb C^N\) , but with an arbitrary norm, this simple, geometric argument is no longer valid. We will be back to this in a moment, with explanations on how to do this, directly in a more general setting, that where the codimension of \( W\subset V\) is 1, or more generally, is finite.
(5) Getting now to the general case, where \( V\) can be infinite dimensional, we are facing two problems here, which can be summarized as follows:
– First, there is a problem coming from infinite dimensionality. Indeed, inspired by the above, we can try to extend linear forms by adding \( 1,2,3,…\) dimensions to our space. But if we want to add \( \infty\) dimensions, we will probably need the Zorn lemma.
– Second, at the technical level, when trying to compute norms, and arrange as for our extensions to satisfy \( ||\tilde{f}||=||f||\) , we have a problem with real vs complex. Indeed, and you will have to believe me here, the real problem is easier than the complex one.
In short, we are in need of a good plan here. In what follows we will first focus on the real problem, and solve it by adding 1 dimension, and then invoking the Zorn lemma. And then, we will use a trick, in order to convert our real result into a complex result.
(6) So, consider a real Banach space \( V\) , having a subspace \( W\subset V\) , and assume in addition that this subspace is of codimension one, in the sense that we have:
Our claim is that any real linear form \( f:W\to\mathbb R\) can be extended into a real linear form \( f:V\to\mathbb R\) , having the same norm. In order to prove this, let us set:
This linear form \( \tilde{f}:W\to\mathbb R\) extends then \( f:V\to\mathbb R\) , and the problem is that of finding the correct scalar \( \alpha\in\mathbb R\) , making the equality \( ||\tilde{f}||=||f||\) hold.
(7) In order to solve this latter question, we can assume, by linearity, that we have \( ||f||=1\) . In this case, we want to find \( \alpha\in\mathbb R\) such that the following holds:
By replacing \( w\to-\lambda w\) , we would like the following inequality to hold:
Equivalently, by dividing by \( \lambda\) , we would like the following inequality to hold:
But this is the same as asking for the following inequality to hold:
Thus, our scalar \( \alpha\in\mathbb R\) must be subject to the following inequalities:
(8) To summarize, we can find indeed our scalar \( \alpha\in\mathbb R\) , making our extension \( \tilde{f}\) to have norm 1 as desired, provided that the following inequality holds:
But, in order to establish this latter inequality, observe that our assumption \( ||f||=1\) , and then the triangle inequality give, for any two vectors \( v,w\in W\) :
The point now is that this estimate can be written in the following way:
Thus we can indeed pick \( \alpha\in\mathbb R\) to be between the quantities on the left, and those on the right, with \( v,w\in W\) varying, and with this choice, we have, as needed:
(9) Next, with this done, we can now prove Hahn-Banach in general, in the real case, by invoking the Zorn lemma. Assume indeed that we have a real Banach space \( V\) , a real subspace \( W\subset V\) , and a real linear form \( f:W\to\mathbb R\) . We can then consider the family of extensions \( \tilde{f}:\tilde{W}\to\mathbb R\) of our linear form \( f\) , having the same norm as it, \( ||\tilde{f}||=||f||\) . This family is then partially ordered, in the obvious way, and by using the Zorn lemma, we conclude that it must have a maximal element. But, in view of what we found in (8), this maximal element \( \tilde{f}:\tilde{W}\to\mathbb R\) must be such that \( \tilde{W}=V\) , and we are done.
(10) Finally, getting back now to the complex case, assume that we have a complex Banach space \( V\) , a complex subspace \( W\subset V\) , and a linear form \( f\in W^*\) . We can then apply the real result, established above, to the following real linear form:
We obtain in this way a certain extension \( \tilde{g}:V\to\mathbb R\) , and then we can set:
And with this, our claim is that we are done, with this being the needed extension.
(11) Indeed, as a first observation, \( \tilde{f}:V\to\mathbb C\) as constructed above is indeed a linear form, in the complex sense, with this coming from the following computation:
(12) Next, since we have \( z=Re(z)-iRe(iz)\) for any complex number \( z\in\mathbb C\) , by linearity we obtain that \( \tilde{f}\in V^*\) extends indeed the original linear form \( f\in W^*\) . As for the same norm assertion, this follows from the following equalities:
Thus, theorem proved, in its general complex form, the one in the statement.
With the above understood, we can now formulate a key result, as follows:
Theorem 150
Given a Banach space \( V\) , we have an embedding as follows,
which is an isomorphism in finite dimensions, and for the \( l^p\) and \( L^p\) spaces too.
Proof
There are several things going on here, the idea being as follows:
(1) The fact that we have indeed a vector space embedding \( V\subset V^{**}\) is clear from definitions, the formula of this embedding being as follows:
(2) However, the fact that this embedding \( V\subset V^{**}\) is isometric is something more subtle, which requires the use of the Hahn-Banach result from Theorem 9.21.
(3) Next, the fact that we have \( V=V^{**}\) in finite dimensions is clear.
(4) Regarding now the formula \( V=V^{**}\) for the various \( l^p\) and \( L^p\) spaces, this is something quite tricky. Let us start with the simplest case, that of the space \( V=l^2\) . We know that this space is given by definition by the following formula:
Now let us look for linear forms \( f:l^2\to\mathbb C\) . By linearity such a linear form must appear as follows, for certain scalars \( a_i\in\mathbb C\) , which must be such that \( f\) is well-defined:
But, what does the fact that \( f\) is well-defined mean? In answer, this means that the values of \( f\) must all converge, which in practice means that we must have:
Moreover, we would like our linear form \( f:l^2\to\mathbb C\) to be bounded, and by denoting by \( A=||f||<\infty\) the minimal bound, this means that we must have:
Now recall that the Cauchy-Schwarz inequality tells us that we have:
Thus, the linear form \( f:l^2\to\mathbb C\) associated to any \( a=(a_i)\in l^2\) will do. Moreover, conversely, by examining the proof of Cauchy-Schwarz, we conclude that this condition \( a=(a_i)\in l^2\) is in fact necessary. Thus, we have proved that we have:
But this gives the \( V=V^{**}\) result in the statement for our space \( V=l^2\) , because by dualizing one more time we obtain, as desired:
(5) Getting now to more complicated spaces, let us look, more generally, at \( L^2(X)\) . We know that this space is given by definition by the following formula:
As before, when looking for linear forms \( \varphi:L^2(X)\to\mathbb C\) , by linearity, and with some mesure theory helping, our forms must appear via a formula as follows:
Now in order for this integral to converge, as for our map \( \varphi:L^2(X)\to\mathbb C\) to be well-defined, and with the additional requirement that \( \varphi\) must be actually bounded, we must have an inequality as follows, for a certain positive constant \( A<\infty\) :
Now recall that the Cauchy-Schwarz inequality tells us that we have:
Thus, the linear form \( \varphi:L^2(X)\to\mathbb C\) associated to any \( \varphi\in L^2(X)\) will do. Moreover, conversely, by examining the proof of Cauchy-Schwarz, we conclude that this condition \( \varphi\in L^2(X)\) is in fact necessary. Thus, we have proved that we have:
But this gives the \( V=V^{**}\) result in the statement for our space \( V=L^2\) , because by dualizing one more time we obtain, as desired:
(6) Moving ahead now, let us go back to the \( l^p\) spaces, as in (4), but now with general exponents \( p\in[1,\infty]\) , instead of \( p=2\) . The space \( l^p\) is by definition given by:
Now by arguing as in (4), a linear form \( f:l^p\to\mathbb C\) must come as follows:
To be more precise, here \( a_i\in\mathbb C\) are certain scalars, which are subject to an inequality as follows, for a certain constant \( A<\infty\) , making \( f\) well-defined, and bounded:
Now recall that the H
specialChar{34}older inequality tells us that we have, with \( \frac{1}{p}+\frac{1}{q}=1\) :
Thus, the linear form \( f:l^2\to\mathbb C\) associated to any element \( a=(a_i)\in l^q\) will do. Moreover, conversely, by examining the proof of H
specialChar{34}older, we conclude that this condition \( a=(a_i)\in l^q\) is in fact necessary. Thus, we have proved that we have:
But this gives the \( V=V^{**}\) result in the statement for our space \( V=l^p\) , because by dualizing one more time we obtain, as desired:
(7) All this is very nice, and time now to generalize everything that we know, by looking at the general spaces \( L^p(X)\) , with \( p\in[1,\infty]\) . These spaces are given by:
As before in (5), when looking for linear forms \( \varphi:L^p(X)\to\mathbb C\) , by linearity, and with some mesure theory helping, our forms must appear via a formula as follows:
Now in order for this integral to converge, as for our map \( \varphi:L^p(X)\to\mathbb C\) to be well-defined, and with the additional requirement that \( \varphi\) must be actually bounded, we must have an inequality as follows, for a certain positive constant \( A<\infty\) :
Now recall that the H
specialChar{34}older inequality tells us that we have, with \( \frac{1}{p}+\frac{1}{q}=1\) :
Thus, the linear form \( \varphi:L^p(X)\to\mathbb C\) associated to any function \( \varphi\in L^q(X)\) will do. Moreover, conversely, by examining the proof of H
specialChar{34}older, we conclude that this condition \( \varphi\in L^q(X)\) is in fact necessary. Thus, we have proved that we have:
But this gives the \( V=V^{**}\) result in the statement for our space \( V=L^p\) , because by dualizing one more time we obtain, as desired:
(8) Finally, let us mention that not all Banach spaces satisfy \( V=V^{**}\) , with a basic counterexample here being the space \( c_0\) of sequences \( x_n\in\mathbb C\) satisfying \( x_n\to0\) , with the sup norm. Indeed, computations show that we have the following formulae:
Thus, in this case \( V\subset V^{**}\) is the embedding \( c_0\subset l^\infty\) , which is not an isomorphism.
(9) To be more precise here, let us define \( c_0\) as above, as being the space of sequences \( x_n\in\mathbb C\) satisfying \( x_n\to0\) , with the sup norm. We have then an inclusion as follows:
It is then clear that \( c_0\) is closed, so it is indeed a Banach space. Now regarding the duals, we can use here the standard fact that given an inclusion of Banach spaces \( V\subset W\) , by restricting the linear forms from \( W\) to \( V\) we obtain a surjection \( W^*\to V^*\) . By using this for the above inclusion \( c_0\subset l^\infty\) , we obtain a surjection as follows:
But this surjection is clearly injective, so we have indeed \( c_0^*=l^1\) , as claimed above.
This was a mixed algebraic and analytic chapter, and as exercises on this, we have:
Exercise 65
Review if needed the theory of convex functions, and Jensen.
Exercise 66
Learn about the Lagrange multipliers, that we used in the above.
Exercise 67
Find some other Banach space norms, in finite dimensions.
Exercise 68
Find some other examples of Banach spaces of sequences.
Exercise 69
Learn about bases of Banach spaces, theorems and questions.
Exercise 70
Learn about the tensor products of Banach spaces too.
Exercise 71
Learn more about the reflexive Banach spaces.
Exercise 72
Learn about Banach algebras, and what can be done with them.
As bonus exercise, find a good Banach space book, and start reading it.
We discuss in this chapter the Hilbert spaces, which are the most important types of Banach spaces. The idea is very simple, namely that the most basic Banach spaces should be those whose norm comes from a scalar product, according to the following formula:
In fact, we have already met such spaces in chapter 9, because the standard norm on \( \mathbb C^N\) comes as above, from the standard scalar product on \( \mathbb C^N\) , given by:
More generally, the norm on any \( l^2(I)\) comes as above, with the scalar product on \( l^2(I)\) being given the same formula. And even more generally, the norm on any space of type \( L^2(X)\) comes as above, with the scalar product on \( L^2(X)\) being given by:
Summarizing, we have some beginning of theory here, and it is tempting at this point to axiomatize the scalar products and the Hilbert spaces, and then develop the theory of these Hilbert spaces, as a continuation of what we did in chapter 9.
However, this is just one point of view on the Hilbert spaces. Alternatively, we can build everything from scratch, simply by starting with what we know about \( \mathbb C^N\) and its scalar product \( <\,,>\) , from linear algebra, and generalizing this to infinite dimensions. And with this being a quite nice approach, because everything that we learned in chapter 9 about the general Banach spaces heavily simplifies in the case of the Hilbert spaces.
Well, hope you get my point, we are here in front of a dillema, as follows:
Dillema 1
We have two ways of viewing the theory of Hilbert spaces, as a generalization of linear algebra, or as a particular case of Banach space theory.
So, what to do, in this kind of situation? Ask the cats of course, that we have not seen since chapter 4. And the cats, both more in shape than ever, declare:
Cats 1
We cats view the theory of Hilbert spaces as being part of quantum mechanics. But for a human introduction, better go with the linear algebra way.
Okay, thanks cats, I was sort of expecting that, but the advice at the end is really useful. So, we will go this way, and forgetting now everything that we learned in chapter 9 about Banach spaces, and no worries about this, we will manage to reprove all that things, in the Hilbert space case, let us start with a discussion in finite dimensions.
You certainly know from linear algebra and calculus many things about \( \mathbb C^N\) and its scalar product \( <\,,>\) , and we will review now this. Let us start with:
Definition 50
We endow \( \mathbb C^N\) with its usual scalar product, namely
so that the norm of the vectors is given by \( ||x||=\sqrt{<x,x>}\) .
Here the convention for linearity, which is at left, and the notation \( <\,,>\) for the scalar product are both taken from physics, and more specifically from cat quantum mechanics. Not to be confused with human quantum mechanics, where the conventions are usually different, with the scalar product being denoted \( <|>\) , and taken linear at right.
Getting now to what can be done with scalar products, as a basic result, we have:
Theorem 151
The linear maps \( T:\mathbb C^N\to\mathbb C^N\) are in correspondence with the square matrices \( A\in M_N(\mathbb C)\) , with the linear map associated to such a matrix being
and with the matrix associated to a linear map being \( A_{ij}=<Te_j,e_i>\) .
Proof
The first assertion is clear, because a linear map \( T:\mathbb C^N\to\mathbb C^N\) must send a vector \( x\in\mathbb C^N\) to a certain vector \( Tx\in\mathbb C^N\) , all whose components are linear combinations of the components of \( x\) . Thus, we can write, for certain complex numbers \( A_{ij}\in\mathbb C\) :
But this means \( Tx=Ax\) , as claimed. Regarding now the second assertion, with \( Tx=Ax\) , if we denote by \( e_1,…,e_N\) the standard basis of \( \mathbb C^N\) , we have:
But this gives the second formula, \( <Te_j,e_i>=A_{ij}\) , as desired.
Our claim now is that, no matter what we want to do with \( T\) or \( A\) , of advanced type, we will run at some point into their adjoints \( T^*\) and \( A^*\) , constructed as follows:
Proposition 36
The adjoint operator \( T^*:\mathbb C^N\to\mathbb C^N\) , which is given by
corresponds to the adjoint matrix \( A^*\in M_N(\mathbb C)\) , given by
via the correspondence between linear maps and matrices constructed above.
Proof
Given a linear map \( T:\mathbb C^N\to\mathbb C^N\) , fix \( y\in\mathbb C^N\) , and consider the linear form \( \varphi(x)=<Tx,y>\) . This form must be as follows, for a certain vector \( T^*y\in\mathbb C^N\) :
Thus, we have constructed a map \( y\to T^*y\) as in the statement, which is obviously linear, and that we can call \( T^*\) . Now by taking the vectors \( x,y\in\mathbb C^N\) to be elements of the standard basis of \( \mathbb C^N\) , our defining formula for \( T^*\) reads:
By reversing the scalar product on the right, this formula can be written as:
But this means that the matrix of \( T^*\) is given by \( (A^*)_{ij}=\bar{A}_{ji}\) , as desired.
Getting back to our claim, the adjoints \( *\) are indeed ubiquitous, as shown by:
Theorem 152
The following happen:
Proof
Let us first recall that the lengths, or norms, of the vectors \( x\in\mathbb C^N\) can be recovered from the knowledge of the scalar products, as follows:
Conversely, we can recover the scalar products out of norms, by using the following difficult to remember formula, called complex polarization identity:
Finally, we will use Proposition 10.5, and more specifically the following formula coming from there, valid for any matrix \( A\in M_N(\mathbb C)\) and any two vectors \( x,y\in\mathbb C^N\) :
(1) Given a matrix \( U\in M_N(\mathbb C)\) , we have indeed the following equivalences, with the first one coming from the polarization identity, and the other ones being clear:
(2) Given a matrix \( P\in M_N(\mathbb C)\) , in order for \( x\to Px\) to be an oblique projection, we must have \( P^2=P\) . Now observe that this projection is orthogonal when:
By conjugating, \( P^*P=P\) , so we must have \( P=P^*\) , which gives the result.
At a more advanced level now, diagonalization, we first have the following result:
Theorem 153
Any matrix \( A\in M_N(\mathbb C)\) which is self-adjoint, \( A=A^*\) , is diagonalizable, with the diagonalization being of the following type,
with \( U\in U_N\) , and with \( D\in M_N(\mathbb R)\) diagonal. The converse holds too.
Proof
As a first remark, the converse trivially holds, because if we take a matrix of the form \( A=UDU^*\) , with \( U\) unitary and \( D\) diagonal and real, then we have:
In the other sense now, assume that \( A\) is self-adjoint, \( A=A^*\) . Our first claim is that the eigenvalues are real. Indeed, assuming \( Av=\lambda v\) , we have:
Thus we obtain \( \lambda\in\mathbb R\) , as claimed. Our next claim now is that the eigenspaces corresponding to different eigenvalues are pairwise orthogonal. Assume indeed that we have \( Av=\lambda v\) , \( Aw=\mu w\) . We have then the following computation, using \( \lambda,\mu\in\mathbb R\) :
Thus \( \lambda\neq\mu\) implies \( v\perp w\) , as claimed. In order now to finish, it remains to prove that the eigenspaces of \( A\) span the whole \( \mathbb C^N\) . For this purpose, let us pick an eigenvector, \( Av=\lambda v\) . Assuming that we have a vector \( w\) orthogonal to it, \( v\perp w\) , we have:
Thus, if \( v\) is an eigenvector, then the vector space \( v^\perp\) is invariant under \( A\) . Moreover, since a matrix \( A\) is self-adjoint precisely when \( <Av,v>\in\mathbb R\) for any vector \( v\in\mathbb C^N\) , as one can see by expanding the scalar product, the restriction of \( A\) to the subspace \( v^\perp\) is self-adjoint. Thus, we can proceed by recurrence, and we obtain the result.
Let us discuss now the case of the unitary matrices. We have here:
Theorem 154
Any matrix \( U\in M_N(\mathbb C)\) which is unitary, \( U^*=U^{-1}\) , is diagonalizable, with the eigenvalues on \( \mathbb T\) . More precisely we have
with \( V\in U_N\) , and with \( D\in M_N(\mathbb T)\) diagonal. The converse holds too.
Proof
As a first remark, the converse trivially holds, because given a matrix of type \( U=VDV^*\) , with \( V\in U_N\) , and with \( D\in M_N(\mathbb T)\) being diagonal, we have:
Let us prove now the first assertion, stating that the eigenvalues of a unitary matrix \( U\in U_N\) belong to the unit circle \( \mathbb T\) . Assuming \( Uv=\lambda v\) , we have:
Thus \( \lambda\in\mathbb T\) , as claimed. Our next claim now is that the eigenspaces corresponding to different eigenvalues are pairwise orthogonal. Assume indeed that \( Uv=\lambda v\) , \( Uw=\mu w\) . We have then the following computation, using \( U^*=U^{-1}\) and \( \lambda,\mu\in\mathbb T\) :
Thus \( \lambda\neq\mu\) implies \( v\perp w\) , as claimed. In order now to finish, it remains to prove that the eigenspaces of \( U\) span the whole \( \mathbb C^N\) . For this purpose, let us pick an eigenvector, \( Uv=\lambda v\) . Assuming that we have a vector \( w\) orthogonal to it, \( v\perp w\) , we have:
Thus, if \( v\) is an eigenvector, then the vector space \( v^\perp\) is invariant under \( U\) . Now since \( U\) is an isometry, so is its restriction to this space \( v^\perp\) . Thus this restriction is a unitary, and so we can proceed by recurrence, and we obtain the result.
The self-adjoint matrices and the unitary matrices are particular cases of the general notion of a “normal matrix”, and we have here the following result:
Theorem 155
Any matrix \( A\in M_N(\mathbb C)\) which is normal, \( AA^*=A^*A\) , is diagonalizable, with the diagonalization being of the following type,
with \( U\in U_N\) , and with \( D\in M_N(\mathbb C)\) diagonal. The converse holds too.
Proof
As a first remark, the converse trivially holds, because if we take a matrix of the form \( A=UDU^*\) , with \( U\) unitary and \( D\) diagonal, then we have:
In the other sense now, this is something more technical. Our first claim is that a matrix \( A\) is normal precisely when the following happens, for any vector \( v\) :
Indeed, the above equality can be written as follows:
But this is equivalent to \( AA^*=A^*A\) , by expanding the scalar products. Our next claim is that \( A,A^*\) have the same eigenvectors, with conjugate eigenvalues:
Indeed, this follows from the following computation, and from the trivial fact that if a matrix \( A\) is normal, then so is any matrix of type \( A-\lambda 1_N\) :
Let us prove now that the eigenspaces of \( A\) are pairwise orthogonal. Assume that we have two eigenvectors, \( Av=\lambda v\) , \( Aw=\mu w\) , corresponding to different eigenvalues, \( \lambda\neq\mu\) . We have the following computation, which shows that \( \lambda\neq\mu\) implies \( v\perp w\) :
In order to finish, it remains to prove that the eigenspaces of \( A\) span the whole \( \mathbb C^N\) . For this purpose, let us pick two eigenvectors \( v,w\) of the matrix \( AA^*\) , corresponding to different eigenvalues, \( \lambda\neq\mu\) . The eigenvalue equations are then as follows:
We have the following computation, using the normality condition \( AA^*=A^*A\) , and the fact that the eigenvalues of \( AA^*\) , and in particular \( \mu\) , are real:
We conclude that we have \( <Av,w>=0\) . But this reformulates as follows:
Now since the eigenspaces of \( AA^*\) are pairwise orthogonal, and span the whole \( \mathbb C^N\) , we deduce from this that these eigenspaces are invariant under \( A\) :
But with this result in hand, we can finish. Indeed, we can decompose the problem, and the matrix \( A\) itself, following these eigenspaces of \( AA^*\) , which in practice amounts in saying that we can assume that we only have 1 eigenspace. Now by rescaling, this is the same as assuming that we have \( AA^*=1\) . But with this, we are now into the unitary case, that we know how to solve, as explained in Theorem 10.8, and so done.
Getting now to infinite dimensions, there are many things to be done here, in order to extend the above linear algebra material. We will do this slowly, as follows:
(1) In the remainder of this chapter we discuss the notion of Hilbert space, with axioms, general theory, examples, and a look into linear operators too.
(2) And in what regards diagonalization in infinite dimensions, which is something trickier than in finite dimensions, we will leave this for later, for chapter 12 below.
Getting started now, we have the following key definition, generalizing what we know from finite dimensions, about the space \( \mathbb C^N\) , and its scalar product \( <\,,>\) :
Definition 51
A Hilbert space is a complex vector space \( H\) with a scalar product \( <x,y>\) , which will be linear at left and antilinear at right,
which is complete with respect to corresponding norm
in the sense that any sequence which is Cauchy with respect to \( ||.||\) converges.
In other words, in terms of the Banach space theory from chapter 9, a Hilbert space is a Banach space whose norm comes from a scalar product, via \( ||x||=\sqrt{<x,x>}\) . However, for reasons discussed in the beginning of this chapter, we would rather prefer to develop the Hilbert space theory independently from what we know from chapter 9.
Getting now to Definition 10.10 as stated, this is something quite subtle, and there is some mathematics encapsulated there, needing some discussion. Indeed, we must prove that \( ||x||=\sqrt{<x,x>}\) is a norm. But this comes from the following result:
Theorem 156
Consider a complex vector space \( H\) with a scalar product \( <\,,>\) , and set \( ||x||=\sqrt{<x,x>}\) . The following happen:
Proof
This is something very standard, the idea being as follows:
(1) In order to prove Cauchy-Schwarz, consider the following quantity, depending on a real variable \( t\in\mathbb R\) , and on a variable on the unit circle, \( w\in\mathbb T\) :
By developing \( f\) , we see that this is a degree 2 polynomial in \( t\) :
Since \( f\) is obviously positive, its discriminant must be negative:
But this is equivalent to the following condition:
Now the point is that we can arrange for the number \( w\in\mathbb T\) to be such that the quantity \( w<x,y>\) is real. Thus, we obtain the following inequality, as desired:
(2) This follows indeed from the Cauchy-Schwarz inequality, as follows:
(3) This follows indeed from the Minkowski inequality, which corresponds to the triangle inequality, the other two axioms for a distance being trivially satisfied.
Summarizing, Definition 10.10 is fully justified. Before getting into basic examples and theory, let us record as well the following result, which further clarifies the relation between Hilbert and Banach spaces, and which is something very useful on its own:
Theorem 157
The scalar products can be recovered from distances, via the following formula, called complex polarization identity:
Thus, a Banach space is a Hilbert space precisely when this polarization formula defines, out of the norm \( ||.||\) , a scalar product \( <\,,>\) , satisfying \( <x,x>=||x||^2\) .
Proof
In what regards the polarization formula in the statement, this is certainly difficult to memorize, but easy to prove, as follows:
As for the last assertion, that is something rather philosophical, obtained by carefully examining Definition 10.10, with the complex polarization identity in mind.
At the level of examples now, we first have the following result:
Theorem 158
Given an index set \( I\) , the space of square-summable sequences
is a Hilbert space, with scalar product as follows:
When \( I\) is finite, \( I=\{1,…,N\}\) , we obtain in this way the usual space \( H=\mathbb C^N\) .
Proof
We have already met such things before, but let us quickly recall this:
(1) We know that \( l^2(I)\subset\mathbb C^I\) is the space of vectors satisfying \( ||x||<\infty\) . We want to prove that \( l^2(I)\) is a vector space, that \( <x,y>\) is a scalar product on it, that \( l^2(I)\) is complete with respect to \( ||.||\) , and finally that for \( |I|<\infty\) we have \( l^2(I)=\mathbb C^{|I|}\) .
(2) The last assertion, \( l^2(I)=\mathbb C^{|I|}\) for \( |I|<\infty\) , is clear, because in this case the sums are finite, so the condition \( ||x||<\infty\) is automatic. So, we know at least one thing.
(3) Regarding the other assertions, everything follows from Cauchy-Schwarz. We first obtain, by raising to the square and expanding, that for any \( x,y\in l^2(I)\) we have:
Thus \( l^2(I)\) is indeed a vector space, the other vector space conditions being trivial.
(4) Also, \( <x,y>\) is surely a scalar product on this vector space, because all the conditions for a scalar product are trivially satisfied.
(5) Finally, the fact that our space \( l^2(I)\) is indeed complete with respect to its norm \( ||.||\) follows in the obvious way, the limit of a Cauchy sequence \( \{x_n\}\) being the vector \( y=(y_i)\) given by \( y_i=\lim_{n\to\infty}x_{ni}\) , with all the verifications here being trivial.
Going now a bit abstract, we have, more generally, the following result:
Theorem 159
Given an arbitrary space \( X\) with a positive measure \( \mu\) on it, the space of square-summable complex functions on it, namely
is a Hilbert space, with scalar product as follows:
When \( X=I\) is discrete, meaning that the measure \( \mu\) on it is the counting measure, \( \mu(\{x\})=1\) for any \( x\in X\) , we obtain in this way the previous spaces \( l^2(I)\) .
Proof
This is something routine, remake of Theorem 10.13, as follows:
(1) The proof of the first, and main assertion is something perfectly similar to the proof of Theorem 10.13, by replacing everywhere the sums by integrals.
(2) With the remark that we forgot to say in the statement that the \( L^2\) functions are by definition taken up to equality almost everywhere, \( f=g\) when \( ||f-g||=0\) .
(3) As for the last assertion, when \( \mu\) is the counting measure all our integrals here become usual sums, and so we recover in this way Theorem 10.13.
As a conclusion to what we did so far, the Hilbert spaces are now axiomatized, and the main examples discussed. In order to do now some geometry on our spaces, in analogy with what we know from finite dimensions, let us start with the following definition:
Definition 52
Let \( H\) be a Hilbert space.
As a first comment here, in relation with (1), it follows from Cauchy-Schwarz that given any two vectors \( x,y\) we have \( <x,y>=||x||\cdot||y||\cdot\alpha\) , with \( \alpha\in[-1,1]\) . In the case of the real Hilbert spaces we have \( \alpha=\cos t\) , with \( t\) being the angle between \( x,y\) , but here we will only deal with complex Hilbert spaces, where this formula is not available.
In what regards (2), this is something very familiar too, and as an observation here, the subset \( S^\perp\subset H\) constructed above is a closed linear space. In finite dimensions a useful, well-known formula here is \( S^{\perp\perp}=S\) , in case \( S\subset H\) itself is a linear space. As explained below, this generalizes to the infinite dimensional setting as \( S^{\perp\perp}=\bar{S}\) .
Getting now to what can be done with orthogonality, we have here:
Theorem 160
Let \( H\) be a Hilbert space, and \( E\subset H\) be a closed subspace.
Proof
This is something very standard, the idea being as follows:
(1) Given \( x\in H\) and two vectors \( v,w\in E\) , we have the following estimate:
But this shows that any sequence in \( E\) realizing the inf in the definition of \( d(x,E)\) is Cauchy, so it converges to a vector \( y\) . Since \( E\) is closed we have \( y\in E\) , so \( y\) realizes the inf. Moreover, again from the above inequality, such a \( y\) realizing the inf is unique.
(2) In order to prove \( x-y\in E^\perp\) , let \( v\in E\) and choose \( w\in\mathbb T\) such that \( w<x-y,v>\) is a real number. For any \( t\in\mathbb R\) we have the following equality:
By construction of the vector \( y\) we know that this function has a minimum at \( t=0\) . But this function is a degree 2 polynomial, so the middle term must vanish:
Now since this must hold for any \( v\in E\) , we must have \( x-y\in E^\perp\) , as desired.
(3) This is consequence of what we found in (1,2).
(4) This is also a consequence of what we found in (1,2).
Many other things can be said, as a continuation of the above, as for instance with the formula \( E^{\perp\perp}=E\) , when \( E\subset H\) is a closed subspace, which must be adjusted to \( E^{\perp\perp}=\bar{E}\) , in the case of the arbitrary subspaces \( E\subset H\) . Also, importantly, we have:
Theorem 161
Given a Hilbert space \( H\) and a closed subspace \( E\subset H\) , any linear form \( f:E\to\mathbb C\) can be extended into a linear form \( \tilde{f}:H\to\mathbb C\) , having the same norm, and this simply by using \( H=E\oplus E^\perp\) , and setting \( \tilde{f}=0\) on \( E^\perp\) .
Proof
This is indeed something self-explanatory. Observe that what we have here is the Hahn-Banach theorem, for the Hilbert spaces, coming with a trivial proof.
Still talking abstract functional analysis, the few Banach space findings left from chapter 9 trivialize in the case of Hilbert spaces, as shown by the following result:
Theorem 162
Let \( H\) be a Hilbert space.
Proof
This is something that we already know from chapter 9, coming from our general discussion there, but we have an elementary proof for this, as follows:
(1) Consider a linear form \( f:H\to\mathbb C\) . Choose \( v\in H\) such that \( f(v)\neq 0\) . By linearity we may assume \( f(v)=1\) . Then each \( z\in H\) decomposes in the following way:
This shows that we have a direct sum decomposition of \( H\) , as follows:
Now pick \( z\in\ker(f)^\perp\) and consider the kernel of the linear form \( f_z(y)=<z,y>\) :
The linear forms \( f_z\) and \( f\) are then given by the following formulae:
It follows that we have \( f=\mu f_z\) , with \( \mu=f_z(v)^{-1}\) , and so that we have, as desired:
(2) This is just an abstract reformulation of what we found in (1).
(3) This follows from (2), because we have \( H^{**}=\bar{H}^*=\bar{\bar{H}}=H\) .
As a conclusion to all this, the various general Banach space results form chapter 9 are all clear in the Hilbert space setting. However, do not worry, the Hilbert spaces have their own amount of mystery, that we will explore in what follows.
At a more advanced level now, we can talk about orthonormal bases, and the related notion of dimension of a Hilbert space. We have here the following result:
Theorem 163
Any Hilbert space \( H\) has an orthonormal basis \( \{e_i\}_{i\in I}\) , which is by definition a set of vectors whose span is dense in \( H\) , and which satisfy
with \( \delta\) being a Kronecker symbol. The cardinality \( |I|\) of the index set, which can be finite, countable, or uncountable, depends only on \( H\) , and is called dimension of \( H\) . We have
in the obvious way, mapping \( \sum\lambda_ie_i\to(\lambda_i)\) . The Hilbert spaces with \( \dim H=|I|\) being countable, such as \( l^2(\mathbb N)\) , are all isomorphic, and are called separable.
Proof
We have many assertions here, the idea being as follows:
(1) In finite dimensions an orthonormal basis \( \{e_i\}_{i\in I}\) can be constructed by starting with any vector space basis \( \{f_i\}_{i\in I}\) , and using the Gram-Schmidt procedure. As for the other assertions, these are all clear, from basic linear algebra.
(2) In general, the same method works, namely Gram-Schmidt, with a subtlety coming from the fact that the basis \( \{e_i\}_{i\in I}\) will not span in general the whole \( H\) , but just a dense subspace of it, as it is in fact obvious by looking at the standard basis of \( l^2(\mathbb N)\) .
(3) And there is a second subtlety as well, coming from the fact that the recurrence procedure needed for Gram-Schmidt must be replaced by some sort of “transfinite recurrence”, using standard tools from logic, and more specifically the Zorn lemma.
(4) Finally, everything at the end, regarding our notion of separability for the Hilbert spaces, is clear from definitions, and from our various results above.
According to Theorem 10.19, there is only one separable Hilbert space, up to isomorphism. There are many interesting things that can be said, about this magic and unique Hilbert space. As a first result such result, which is something theoretical, we have:
Theorem 164
The following happen, in relation with separability:
Proof
Many things can be said here, the idea being as follows:
(1) The fact that \( H=L^2[-1,1]\) is separable is clear indeed from the Weierstrass density theorem, which provides us with the algebraic basis \( g_k=x^k\) , which can be orthogonalized by using the Gram-Schmidt procedure, as explained in Theorem 10.19.
(2) Regarding now more general spaces, of type \( H=L^2(\mathbb R,\mu)\) , we can use here the same argument, after modifying if needed our measure \( \mu\) , in order for the functions \( g_k=x^k\) to be indeed square-summable. As for higher dimensions, the situation here is similar, because we can use here the multivariable polynomials \( g_k(x)=x_1^{k_1}… x_N^{k_N}\) .
(3) Finally, the last assertion, regarding the general spaces of type \( H=L^2(X)\) , which generalizes all this, comes as a consequence of the general measure theory developed in Part II, and we will leave working out the details here as an instructive exercise.
As a conclusion to all this, which is a bit philosophical, we have:
Conclusion 6
We are interested in one space, namely the unique separable Hilbert space \( H\) , but due to various technical reasons, it is often better to forget that we have \( H=l^2(\mathbb N)\) , and say instead that we have \( H=L^2(X)\) , with \( X\) being a separable measured space, or simply say that \( H\) is an abstract separable Hilbert space.
It is also possible to make some physics comments here, with the unique separable Hilbert space \( H\) from Conclusion 10.21, that we will be presumably obsessed with, in what follows, being, and no surprise here, the space that we live in.
Let us go back now to Theorem 10.20 and its proof, which is something quite subtle. That material leads us into orthogonal polynomials, which are defined as follows:
Definition 53
The orthogonal polynomials with respect to \( d\mu(x)=f(x)dx\) are polynomials \( P_k\in\mathbb R[x]\) of degree \( k\in\mathbb N\) , which are orthogonal inside \( H=L^2(\mathbb R,\mu)\) :
Equivalently, these orthogonal polynomials \( \{P_k\}_{k\in\mathbb N}\) , which are each unique modulo scalars, appear from the Weierstrass basis \( \{x^k\}_{k\in\mathbb N}\) , by doing Gram-Schmidt.
As a first observation, the orthogonal polynomials exist indeed for any real measure \( d\mu(x)=f(x)dx\) , because we can obtain them from the monomials \( x^k\) via Gram-Schmidt, as indicated above. It is possible to be a bit more explicit here, as follows:
Theorem 165
The orthogonal polynomials with respect to \( \mu\) are given by
where \( M_k=\int_\mathbb Rx^kd\mu(x)\) are the moments of \( \mu\) , and \( c_k\in\mathbb R^*\) can be any numbers.
Proof
Let us first see what happens at small values of \( k\in\mathbb N\) . At \( k=0\) our formula is as follows, stating that the first polynomial \( P_0\) must be a constant, as it should:
At \( k=1\) now, again by using \( M_0=1\) , the formula is as follows:
But this is again the good formula, because the degree is 1, and we have:
At \( k=2\) now, things get more complicated, with the formula being as follows:
However, no need for big computations here, in order to check the orthogonality, because by using the fact that \( x^k\) integrates up to \( M_k\) , we obtain:
Similarly, again by using the fact that \( x^k\) integrates up to \( M_k\) , we have as well:
Thus, result proved at \( k=0,1,2\) , and the proof in general is similar.
In practice now, all this leads us to a lot of interesting combinatorics, and countless things can be said. For the simplest measured space \( X\subset\mathbb R\) , which is the interval \( [-1,1]\) , with its uniform measure, the orthogonal basis problem can be solved as follows:
Theorem 166
The orthonormal polynomials for \( L^2[-1,1]\) , subject to
and called Legendre polynomials, satisfy the equation
which is the Legendre equation from physics. Moreover, we have the formula
called Rodrigues formula for the Legendre polynomials.
Proof
The idea here is that thinking at what Gram-Schmidt does, this is certainly something by recurrence. And examining the recurrence leads to the Legendre equation. As for the Rodrigues formula, we have two choices here, either by verifying that \( \{P_k\}\) is orthonormal, or by verifying the Legendre equation. And both methods work.
The above result is just the tip of the iceberg, and as a continuation, we have:
Theorem 167
The orthogonal polynomials for \( L^2[-1,1]\) , with measure
called Jacobi polynomials, satisfy as well a degree \( 2\) equation, and are given by:
At \( \alpha=\beta=0\) we recover the Legendre polynomials, and at \( \alpha=\beta=\pm\frac{1}{2}\) we recover the Chebycheff polynomials of the first and second kind, from trigonometry.
Proof
Obviously, many things going on here, and much more can be added, but the idea is quite simple, namely that this appears as a generalization of Theorem 10.24. We will leave learning more about all this as an interesting exercise.
Getting now to other spaces \( X\subset\mathbb R\) , of particular interest here is the following result, which complements well Theorem 10.24, for the needs of basic quantum mechanics:
Theorem 168
The orthogonal polynomials for \( L^2[0,\infty)\) , with scalar product
are the Laguerre polynomials \( \{P_k\}\) , given by the following formula,
called Rodrigues formula for the Laguerre polynomials.
Proof
The story here is very similar to that of the Legendre polynomials, and many further things can be said here, with exercise for you to learn a bit about all this.
Finally, regarding the space \( X=\mathbb R\) itself, we have here the following result:
Theorem 169
The orthogonal polynomials for \( L^2(\mathbb R)\) , with scalar product
are the Hermite polynomials \( \{P_k\}\) , given by the following formula,
called Rodrigues formula for the Hermite polynomials.
Proof
As before, the story here is quite similar to that of the Legendre and other orthogonal polynomials, and exercise for you to learn a bit about all this.
And with this, good news, end of the story with the orthogonal polynomials, at least at the very introductory level, and this due to the following fact, which is something quite technical, and that we will not attempt to prove, or even explain in detail here:
Fact 11
From an abstract point of view, coming from degree \( 2\) equations, and Rodrigues formulae for the solutions, there are only three types of “classical” orthogonal polynomials, namely the Jacobi, Laguerre and Hermite ones, discussed above.
Finally, as already mentioned, the above results are very useful in the context of basic quantum mechanics, and more specifically, for solving the hydrogen atom, following Heisenberg and Schrödinger. Again, exercise for you to learn a bit about this.
Time now to slow down, after the above exciting material regarding the orthogonal polynomials. Remember indeed from the beginning of this chapter that our main goal is that of generalizing linear algebra, to the case of infinite dimensions. But for generalizing the linear algebra basics, what we have in Conclusion 10.21 will basically do.
Let us start with a basic result regarding the linear operators, as follows:
Theorem 170
Given a Hilbert space \( H\) , consider the linear operators \( T:H\to H\) , and for each such operator define its norm by the following formula:
The operators which are bounded, \( ||T||<\infty\) , form then a complex algebra \( B(H)\) , which is complete with respect to \( ||.||\) . When \( H\) comes with a basis \( \{e_i\}_{i\in I}\) , we have
where \( \mathcal L(H)\) is the algebra of all linear operators \( T:H\to H\) , and \( \mathcal L(H)\subset M_I(\mathbb C)\) is the correspondence \( T\to M\) obtained via the usual linear algebra formulae, namely:
In infinite dimensions, none of the above two inclusions is an equality.
Proof
In what regards the first assertion, we already know this from chapter 9. Regarding now the second assertion, given a bounded operator \( T:H\to H\) , let us associate to it a matrix \( M\in M_I(\mathbb C)\) as in the statement, by the following formula:
The correspondence \( T\to M\) is then linear, having kernel is \( \{0\}\) , so we have an embedding \( B(H)\subset M_I(\mathbb C)\) . Our claim now is that this embedding is multiplicative. But this is clear too, because if we denote by \( T\to M_T\) our correspondence, we have:
Next, we must prove that the original operator \( T:H\to H\) can be recovered from its matrix \( M\in M_I(\mathbb C)\) via the formula in the statement, namely \( Tx=Mx\) . But it is enough to check this on the vectors of the basis, \( x=e_j\) , and here we have, indeed:
Finally, in finite dimensions the inclusions \( B(H)\subset\mathcal L(H)\subset M_I(\mathbb C)\) are of course both isomorphisms, but in general this does not hold. Indeed, over the space \( H=L^2(\mathbb N)\) , a basic example of matrix not coming from a linear operator is the all-one matrix:
As for the examples of linear operators which are not bounded, these are more complicated, coming from logic, and we will not really need them in what follows.
As in the finite dimensional case, we can talk about adjoint operators, in this setting, the definition and main properties of the construction \( T\to T^*\) being as follows:
Theorem 171
Each operator \( T\in B(H)\) has an adjoint \( T^*\in B(H)\) , given by:
The operation \( T\to T^*\) is antilinear, antimultiplicative, involutive, and satisfies:
When \( H\) comes with a basis \( \{e_i\}_{i\in I}\) , the operation \( T\to T^*\) corresponds to
at the level of the associated matrices \( M\in M_I(\mathbb C)\) .
Proof
There are several things to be proved here, the idea being as follows:
(1) Given \( y\in H\) , the formula \( \varphi(x)=<Tx,y>\) defines a linear map \( H\to\mathbb C\) . Thus, we must have a formula as follows, for a certain vector \( T^*y\in H\) :
Moreover, this vector \( T^*y\in H\) is unique with this property, and we conclude from this that the formula \( y\to T^*y\) defines a certain map \( T^*:H\to H\) , which is unique.
(2) Let us prove that we have \( T^*\in B(H)\) . By using once again the uniqueness of \( T^*\) , we conclude that we have the following formulae, which show that \( T^*\) is linear:
Observe also that \( T^*\) is bounded as well, because we have:
(3) The fact that the correspondence \( T\to T^*\) is antilinear, antimultiplicative, and is an involution comes from the following formulae, coming from uniqueness:
As for \( ||T||=||T^*||\) , this is something that we already know, from the proof of (2).
(4) In order to prove now the formula \( ||TT^*||=||T||^2\) , observe first that we have:
On the other hand, we have as well the following estimate:
By replacing \( T\to T^*\) we obtain from this \( ||T||^2\leq||TT^*||\) , and we are done.
(5) Regarding the last assertion, we can compute the matrix \( M^*\in M_I(\mathbb C)\) associated to the operator \( T^*\in B(H)\) , by using \( <Tx,y>=<x,T^*y>\) , in the following way:
Thus, we have reached to the usual formula for the adjoints of matrices, as desired.
As in finite dimensions, the operators \( T,T^*\) can be thought of as being “twin brothers”, and there is a lot more mathematics connecting them, as for instance:
Theorem 172
Given a bounded operator \( T\in B(H)\) , the following happen:
Proof
Both these assertions are elementary, as follows:
(1) Let us first prove “\( \subset\) ”. Assuming \( T^*x=0\) , we have indeed \( x\perp ImT\) , because:
As for “\( \supset\) ”, assuming \( <x,Ty>=0\) for any \( y\) , we have \( T^*x=0\) , because:
(2) This can be deduced from (1), applied to the operator \( T^*\) , as follows:
Here we have used the formula \( K^{\perp\perp}=\bar{K}\) , valid for any linear subspace \( K\subset H\) of a Hilbert space, which for \( K\) closed reads \( K^{\perp\perp}=K\) , and comes from \( H=K\oplus K^\perp\) , and which in general follows from \( K^{\perp\perp}\subset\bar{K}^{\perp\perp}=\bar{K}\) , the reverse inclusion being clear.
At the level of examples now, let us begin with the rotations. Things here are quite tricky in arbitrary dimensions. We first have the following result:
Theorem 173
For a linear operator \( U\in B(H)\) the following conditions are equivalent, and if they are satisfied, we say that \( U\) is an isometry:
In finite dimensions, we recover in this way the usual unitary transformations.
Proof
The proofs are similar to those in finite dimensions, as follows:
\( (1)\iff(2)\) This follows indeed from the formula of the distances, namely:
\( (2)\iff(3)\) This is again standard, because we can pass from scalar products to distances, and vice versa, by using \( ||x||=\sqrt{<x,x>}\) , and the polarization formula:
\( (3)\iff(4)\) We have indeed the following equivalences, by using the standard formula \( <Tx,y>=<x,T^*y>\) , which defines the adjoint operator:
Thus, we are led to the conclusions in the statement.
The point now is that the condition \( U^*U=1\) does not imply in general \( UU^*=1\) , the simplest counterexample here being the shift operator on \( l^2(\mathbb N)\) :
Proposition 37
The shift operator on the space \( l^2(\mathbb N)\) , given by
is an isometry, \( S^*S=1\) . However, we have \( SS^*\neq1\) .
Proof
The adjoint of the shift is given by the following formula:
When composing \( S,S^*\) , in one sense we obtain the following formula:
In other other sense now, we obtain the following formula:
Summarizing, the compositions are given by the following formulae:
Thus, we are led to the conclusions in the statement.
As a conclusion, the notion of isometry is not the correct infinite dimensional analogue of the notion of unitary. However, the unitary operators can be introduced as follows:
Theorem 174
For a linear operator \( U\in B(H)\) the following conditions are equivalent, and if they are satisfied, we say that \( U\) is a unitary:
Moreover, the unitary operators from a group \( U(H)\subset B(H)\) .
Proof
There are several statements here, the idea being as follows:
(1) The various equivalences in the statement are all clear from definitions, and from Theorem 10.32 in what regards the various possible notions of isometries which can be used, by using the formula \( (ST)^*=T^*S^*\) for the adjoints of the products of operators.
(2) The fact that the products and inverses of unitaries are unitaries is also clear, and we conclude that the unitary operators from a group \( U(H)\subset B(H)\) , as stated.
Let us discuss now the projections. We have here the following result:
Theorem 175
For a linear operator \( P\in B(H)\) the following conditions are equivalent, and if they are satisfied, we say that \( P\) is a projection:
Proof
As in finite dimensions, \( P\) is an abstract projection, not necessarily orthogonal, when it is an idempotent, algebrically speaking, in the sense that we have:
The point now is that this projection is orthogonal when:
Now observe that by conjugating, we obtain \( P^*P=P\) . Thus, we must have \( P=P^*\) , and so we have shown that any orthogonal projection must satisfy, as claimed:
Conversely, if this condition is satisfied, \( P^2=P\) shows that \( P\) is a projection, and \( P=P^*\) shows via the above computation that \( P\) is indeed orthogonal.
There is a relation between the projections and the general isometries, such as the shift \( S\) that we met before, and we have the following result:
Proposition 38
Given an isometry \( U\in B(H)\) , the operator
is a projection, namely the orthogonal projection on \( Im(U)\) .
Proof
Assume indeed that we have an isometry, \( U^*U=1\) . The fact that \( P=UU^*\) is indeed a projection can be checked abstractly, as follows:
As for the last assertion, this is something that we already met, for the shift, and the situation in general is similar, with the result itself being clear.
More generally now, along the same lines, and clarifying the whole situation with the unitaries and isometries, we have the following result:
Theorem 176
An operator \( U\in B(H)\) is a partial isometry, in the usual geometric sense, when the following two operators are projections:
Moreover, the isometries, adjoints of isometries and unitaries are respectively characterized by the conditions \( Q=1\) , \( P=1\) , \( P=Q=1\) .
Proof
The first assertion is a straightforward extension of Proposition 10.36, and the second assertion follows from various results regarding isometries established above.
So long for basic operator theory, fundamentals and examples. We will be back to this in chapter 12 below, with suitable diagonalization results for the self-adjoint operators, \( T=T^*\) , and then for the general normal operators too, \( TT^*=T^*T\) , in analogy with what we learned in the beginning of this chapter, in finite dimensions.
We had an exciting chapter here, and as exercises on this, we have:
Exercise 73
What is the formula of the rank one projection on \( x\in\mathbb C^N\) ?
Exercise 74
Prove that the diagonalizable matrices in \( M_N(\mathbb C)\) are dense.
Exercise 75
Work out the equality case in Cauchy-Schwarz, and Minkowski.
Exercise 76
Review the full basic Banach space theory, for the Hilbert spaces.
Exercise 77
Learn if needed the full proof of the Zorn lemma.
Exercise 78
Learn more about various families of orthogonal polynomials.
Exercise 79
What should symmetry mean, in the operator theory setting?
Exercise 80
Try to axiomatize the Banach \( *\) -algebras, with \( B(H)\) in mind.
As bonus exercise, and no surprise here, learn some quantum mechanics.
Welcome to advanced functional analysis. We have seen in chapters 9-10 the basic theory of Banach spaces and Hilbert spaces, and normally, with the knowledge of the \( L^p\) spaces, and in particular of the \( L^2\) spaces, we can do many things. However, there are far more things that must be known, and this even at the beginner level, and this chapter and the next one will be an introduction to this. Our plan will be as follows:
(1) In this chapter, which will be basically a continuation of chapter 9, we discuss a number of fundamental functional analysis topics, namely convolution, distributions, complex analysis, Fourier analysis, and some basic applications to physics.
(2) And in the next chapter, which will be basically a continuation of chapter 10, we discuss a fundamental question that we still have left from there, namely the diagonalization of the self-adjoint operators, and of the normal operators.
Getting started, let us first talk about convolution. We have already talked in Part I about the convolution of Dirac masses, according to the formula \( \delta_x*\delta_y=\delta_{x+y}\) , and of more general discrete measures, according to this formula, and linearity. In the continuous measure setting, at the level of densities, the relevant definition is as follows:
Definition 54
The convolution of two functions \( f,g:\mathbb R\to\mathbb C\) is the function
provided that the function \( y\to f(x-y)g(y)\) is indeed integrable, for any \( x\) .
There are many reasons for introducing this operation, that we will gradually discover, in what follows. As a basic example, let us take \( g=\chi_{[0,1]}\) . We have then:
Thus, with this choice of \( g\) , the operation \( f\to f*g\) has some sort of “regularizing effect”, that can be useful for many purposes. We will be back to this, later.
Goinh ahead with more theory, let us try to understand when the convolution operation is well-defined. We have here the following basic result:
Proposition 39
The convolution operation is well-defined on the space
of continuous functions \( f:\mathbb R\to\mathbb C\) having compact support.
Proof
We have several things to be proved, the idea being as follows:
(1) First we must show that given two functions \( f,g\in C_c(\mathbb R)\) , their convolution \( f*g\) is well-defined, as a function \( f*g:\mathbb R\to\mathbb C\) . But this follows from the following estimate, where \( l\) denotes the length of the compact subsets of \( \mathbb R\) :
(2) Next, we must show that the function \( f*g:\mathbb R\to\mathbb C\) that we constructed is indeed continuous. But this follows from the following estimate, where \( K_f\) is the constant of uniform continuity for the function \( f\in C_c(\mathbb R)\) :
(3) Finally, we must show that the function \( f*g\in C(\mathbb R)\) that we constructed has indeed compact support. For this purpose, our claim is that we have:
In order to prove this claim, observe that we have, by definition of \( f*g\) :
But this latter quantity being 0 for \( x\notin supp(f)+supp(g)\) , this gives the result.
Here are now a few remarkable properties of the convolution operation:
Proposition 40
The following hold, for the functions in \( C_c(\mathbb R)\) :
Proof
These formulae are all elementary, the idea being as follows:
(1) This follows from the following computation, with \( y=x-t\) :
(2) This is clear from definitions.
(3) Once again, this is clear from definitions.
In relation with derivatives, and with the “regularizing effect” of the convolution operation mentioned after Definition 11.1, we have the following result:
Theorem 177
Given two functions \( f,g\in C_c(\mathbb R)\) , assuming that \( g\) is differentiable, then so is \( f*g\) , with derivative given by the following formula:
More generally, given \( f,g\in C_c(\mathbb R)\) , and assuming that \( g\) is \( k\) times differentiable, then so is \( f*g\) , with \( k\) -th derivative given by \( (f*g)^{(k)}=f*g^{(k)}\) .
Proof
In what regards the first assertion, with \( y=x-t\) , then \( t=x-y\) , we get:
As for the second assertion, this follows form the first one, by recurrence.
Finally, getting beyond the compactly supported continuous functions, we have the following result, which is of particular theoretical importance:
Theorem 178
The convolution operation is well-defined on \( L^1(\mathbb R)\) , and we have:
Thus, if \( f\in L^1(\mathbb R)\) and \( g\in C_c^k(\mathbb R)\) , then \( f*g\) is well-defined, and \( f*g\in C_c^k(\mathbb R)\) .
Proof
In what regards the first assertion, this follows from:
As for the second assertion, this follows from this, and from Theorem 11.4.
Summarizing, we have some theory for the convolution of functions, in analogy with what we knew from Part I about the convolution of Dirac masses, \( \delta_x*\delta_y=\delta_{x+y}\) .
Let us discuss now a related topic, the mathematical distributions. These are something quite smart, and as an advertisement for what we will be doing, we have:
Advertisement 1
With a suitable theory of distributions, covering both the functions and the Dirac masses, the basic step function, namely
is differentiable when viewed as distribution, with derivative \( H'=\delta_0\) .
And isn’t this crazy, hope you agree with me. Getting started now, there is a price to pay for doing such things, namely formulating a technical definition, as follows:
Definition 55
A distribution on an open interval \( I\subset\mathbb R\) is a functional
such that for any \( K\subset I\) compact, there exist \( n\in\mathbb N\) and \( c>0\) such that
for any \( f\in C_c^\infty(I)\) having support in \( K\) , where \( ||f||_{C^n(K)}=\sup_{x\in K}\sum_{i=0}^n|f^{(i)}(x)|\) .
Obviously, this is something quite technical, but believe me, every little thing in the above is there for a reason, in order to the whole theory to work fine.
At the level of main examples of distributions, we have the integration functionals associated to the measures, and in particular to the measures having a density. In view of this, we can consider any function \( f\in L^1(I)\) , viewed as density, as being a distribution. Other basic examples include the Dirac masses \( \delta_x\) at the points \( x\in I\) .
Regarding the general theory of distributions, that is quite similar to the theory of functions. Algebrically, the distributions form a vector space, and are subject to a number of supplementary operations too, such as dilations, translations and so on, and multiplication by functions too. Analytically, we can talk about convergence of distributions, \( \varphi_n\to\varphi\) , and about their support too, \( supp(\varphi)\subset I\) , in a quite straightforward way.
Getting now to what we wanted to do, derivatives, we have here:
Theorem 179
We can talk about the derivatives of distributions, given by
and with this notion in hand, the following happen:
Proof
The first assertion, which by the way explains the need for the \( -\) sign, follows from the Leibnitz rule for derivatives. Regarding the second assertion, this follows from:
As for the third assertion, which generalizes (1,2), we will leave this as an exercise.
Summarizing, quite interesting all this. Many other things can be said about distributions, notably with a convolution theory for them too, generalizing what we know about functions and Dirac masses, and we will leave some thinking or learning here as an exercise. Importantly, all this generalizes to several dimensions too, by making a use of the Jacobian where needed, and again, we will leave some learning here as an exercise.
Time now for some complex analysis. You certainly know about holomorphic functions, and perhaps about harmonic functions too, and time now to review all this. At the beginning, we have the Cauchy formula for complex functions, as follows:
Theorem 180
The Cauchy formula, namely
holds for any holomorphic function \( f:X\to\mathbb C\) .
Proof
This is something quite tricky, that you surely know, and as a matter of reminding this, let us verify it for polynomials \( P\in\mathbb C[X]\) . By linearity we can assume \( P(x)=x^n\) . Also, by a cut-and-paste argument, we can assume that we are on a circle:
The point now is that for a monomial \( P(x)=x^n\) we have:
Here we have used the following key identity, valid for any exponent \( m\in\mathbb Z\) :
By using now the above computation, we obtain the following formula:
Thus, the Cauchy formula holds indeed for polynomials \( P\in\mathbb C[X]\) . In general, the proof is substantially more complicated, but you surely know it, say from Rudin [3].
As a main application of the Cauchy formula, we have:
Theorem 181
The following conditions are equivalent, for a function \( f:X\to\mathbb C\) :
Proof
This is routine from what we have, the idea being as follows:
\( (1)\implies(4)\) is non-trivial, but we know this from Theorem 11.9.
\( (4)\implies(3)\) is trivial, by expanding the series in the Cauchy formula.
\( (3)\implies(2)\implies(1)\) are both elementary, and good exercises for you.
As another application of the Cauchy formula, we have:
Theorem 182
Any holomorphic function \( f:X\to\mathbb C\) satisfies the maximum principle, in the sense that given a domain \( D\) , with boundary \( \gamma\) , we have:
That is, the maximum of \( |f|\) over a domain is attained on its boundary.
Proof
This follows indeed from the Cauchy formula. Observe that the converse is not true, for instance because functions like \( \bar{x}\) satisfy too the maximum principle. We will be back to this in a moment, when talking about harmonic functions.
Conceptually, a good explanation for the fact that the maximum principle holds could be that the values of our function inside a disk can be recovered from its values on the boundary. And fortunately, this is indeed the case, and we have:
Theorem 183
Given an holomorphic function \( f:X\to\mathbb C\) , and a disk \( D\) , with boundary \( \gamma\) , the following happen:
Proof
As usual, this follows from the Cauchy formula, which is something truly powerful, and with working out the details being a good exercise for you.
Finally, as yet another application of the Cauchy formula, which is something nice-looking and conceptual, we have the following statement, called Liouville theorem:
Theorem 184
An entire, bounded holomorphic function
must be constant. In particular, knowing \( f\to0\) with \( z\to\infty\) gives \( f=0\) .
Proof
This follows as usual from the Cauchy formula, namely:
Alternatively, we can view this as a consequence of Theorem 11.12, because given two points \( x\neq y\) , we can view the values of \( f\) at these points as averages over big disks centered at these points, say \( D=D_x(R)\) and \( E=D_y(R)\) , with \( R>>0\) :
Indeed, the point is that when the radius goes to \( \infty\) , these averages tend to be equal, and so we have \( f(x)\simeq f(y)\) , which gives \( f(x)=f(y)\) in the limit.
In order to reach to a continuation of this, let us formulate the following definition:
Definition 56
The Laplace operator in \( 2\) dimensions is:
A function \( f:\mathbb R^2\to\mathbb C\) satisfying \( \Delta f=0\) will be called harmonic.
Here the formula of \( \Delta\) is the one coming from physics, and more on this later. As for the notion of harmonic function, this is something quite natural. Indeed, we can think of \( \Delta\) as being a linear operator on the space of functions \( f:\mathbb R^2\to\mathbb C\) , and previous experience with linear operators, and linear algebra in general, suggests looking first into the eigenvectors of \( \Delta\) . But the simplest such eigenvectors are those corresponding to the eigenvalue \( \lambda=0\) , and these are exactly our harmonic functions, satisfying:
Getting now to more concrete things, and to some mathematics that we can do, using our knowledge, let us try to find the functions \( f:\mathbb R^2\to\mathbb C\) which are harmonic. And here, as a good surprise, we have an interesting link with the holomorphic functions:
Theorem 185
Any holomorphic function \( f:\mathbb C\to\mathbb C\) , when regarded as real function
is harmonic. Moreover, the conjugates \( \bar{f}\) of holomorphic functions are harmonic too.
Proof
The first assertion follows from the following computation, for the power functions \( f(z)=z^n\) , with the usual notation \( z=x+iy\) :
As for the second assertion, this follows from \( \Delta\bar{f}=\overline{\Delta f}\) , which is clear from definitions, and which shows that if \( f\) is harmonic, then so is its conjugate \( \bar{f}\) .
In view of the above, we can expect the harmonic functions to share many interesting properties with the holomorphic functions. We will be back to this, which holds indeed, in a moment. Moving now to arbitrary \( N\in\mathbb N\) dimensions, let us first formulate:
Definition 57
The Laplace operator in \( N\) dimensions is:
A function \( f:\mathbb R^N\to\mathbb C\) satisfying \( \Delta f=0\) will be called harmonic.
Again, these are notions coming from physics, and more on this later, at the end of the present chapter, where we will discuss the wave equation, involving \( \Delta\) .
In view of Theorem 11.15, and in analogy with the theory of holomorphic functions, that we know well from the above, we have the following result:
Theorem 186
The harmonic functions in \( N\) dimensions obey to the same general principles as the holomorphic functions, namely:
Proof
This is something quite tricky, the idea being as follows:
(1) Regarding the plain mean value formula, here the statement is that given an harmonic function \( f:X\to\mathbb C\) , and a ball \( B\) , the following happens:
In order to prove this, we can assume that \( B\) is centered at \( 0\) , of radius \( r>0\) . If we denote by \( \chi_r\) the characteristic function of this ball, nomalized as to integrate up to 1, in terms of the usual convolution operation for functions, we want to prove that we have:
For doing so, let us pick a number \( 0<s<r\) , and a solution \( w\) of the following equation, supported on \( B\) , which can be constructed explicitly:
By using the standard properties of the convolution operation \( *\) , we have:
Thus \( f*\chi_r=f*\chi_s\) , and by letting now \( s\to0\) , we get \( f*\chi_r=f\) , as desired.
(2) Regarding the boundary mean value formula, here the statement is that given an harmonic function \( f:X\to\mathbb C\) , and a ball \( B\) , with boundary \( \gamma\) , the following happens:
But this follows as a consequence of the plain mean value formula in (1), with our two mean value formulae, the one there and the one here, being in fact equivalent, by using annuli and radial integration for the proof of the equivalence, in the obvious way.
(3) Regarding the maximum modulus principle, the statement here is that any holomorphic function \( f:X\to\mathbb C\) has the property that the maximum of \( |f|\) over a domain is attained on its boundary. That is, given a domain \( D\) , with boundary \( \gamma\) , we have:
But this is something which follows again from the mean value formula in (1), first for the balls, and then in general, by using a standard division argument.
(4) Finally, regarding the Liouville theorem, we can view this as a consequence of (1), because given two points \( x\neq y\) , we can view the values of \( f\) at these points as averages over big balls centered at these points, say \( B=B_x(R)\) and \( C=B_y(R)\) , with \( R>>0\) :
Indeed, the point is that when the radius goes to \( \infty\) , these averages tend to be equal, and so we have \( f(x)\simeq f(y)\) , which gives \( f(x)=f(y)\) in the limit, as desired.
In addition to the above, let us mention too that at \( N=2\) , locally, the real harmonic functions are known to be the real parts of holomorphic functions. Also, there is a interesting relation between harmonic functions and soap fims. Exercise of course for you, to learn more about this, with all this being first class mathematics and physics.
Let us discuss now the construction and main properties of the Fourier transform, which is the main tool in analysis, and mathematics in general. We first have:
Definition 58
Given \( f\in L^1(\mathbb R)\) , we define a function \( \widehat{f}:\mathbb R\to\mathbb C\) by
and call it Fourier transform of \( f\) .
As a first observation, even if \( f\) is a real function, \( \widehat{f}\) is a complex function, which is not necessarily real. Also, \( \widehat{f}\) is obviously well-defined, because \( f\in L^1(\mathbb R)\) and \( |e^{ix\xi}|=1\) . Also, the condition \( f\in L^1(\mathbb R)\) is basically needed for construcing \( \widehat{f}\) , because:
Generally speaking, the Fourier transform is there for helping with various computations, with the above formula \( \widehat{f}(0)=\int f\) being something quite illustrating. Here are some basic properties of the Fourier transform, all providing some good motivations:
Proposition 41
The Fourier transform has the following properties:
Proof
These results are all elementary, as follows:
(1) The linearity formulae are indeed both clear from definitions.
(2) The continuity of \( \widehat{f}\) follows indeed from:
As for the boundedness of \( \widehat{f}\) , this is clear as well.
(3) This follows from the following computation, assuming that \( f\) is even:
(4) The proof here is similar to the proof of (3), by changing some signs.
We will be back to more theory in a moment, but let us explore now the examples. Here are some basic computations of Fourier transforms:
Proposition 42
We have the following Fourier transform formulae,
valid for any number \( a>0\) .
Proof
All this follows from some calculus, as follows:
(1) In what regards first formula, assuming \( f=\chi_{[-a,a]}\) , we have, by using the fact that \( \sin(x\xi)\) is an odd function, whose integral vanishes on centered intervals:
(2) With \( f(x)=e^{-ax}\chi_{[0,\infty]}(x)\) , the computation goes as follows:
(3) Regarding the third formula, this follows from the second one, by using the following fact, generalizing the parity computations from Proposition 11.19:
(4) The last 2 formulae follow from what we have, by making sums and differences, and the linearity properties of the Fourier transform, from Proposition 11.19.
We will see many other examples, in what follows. Getting back now to theory, we have the following result, adding to the various general properties in Proposition 11.19, and providing more motivations for the Fourier transform:
Proposition 43
Given \( f,g\in L^1(\mathbb R)\) we have \( \widehat{f}g,f\widehat{g}\in L^1(\mathbb R)\) and
called “exchange of hat” formula.
Proof
Regarding the fact that we have indeed \( \widehat{f}g,f\widehat{g}\in L^1(\mathbb R)\) , this is actually a bit non-trivial, but we will be back to this later. Assuming this, we have:
On the other hand, we have as well the following formula:
Thus, with \( x\leftrightarrow\xi\) , we are led to the formula in the statement.
As a key result now, showing the power of the Fourier transform, we have:
Theorem 187
Given \( f:\mathbb R\to\mathbb C\) such that \( f,f'\in L^1(\mathbb R)\) , we have:
More generally, assuming \( f,f',f'',…,f^{(n)}\in L^1(\mathbb R)\) , we have
for any \( k=1,2,…,n\) .
Proof
These results follow by doing a partial integration, as follows:
(1) Assuming that \( f:\mathbb R\to\mathbb C\) has compact support, we have indeed:
(2) Regarding the higher derivatives, the formula here follows by recurrence.
Importantly, we have a converse statement as well, as follows:
Theorem 188
Assuming that \( f\in L^1(\mathbb R)\) is such that \( F(x)=xf(x)\) belongs to \( L^1(\mathbb R)\) too, the function \( \widehat{f}\) is differentiable, with derivative given by:
More generally, if \( F_k(x)=x^kf(x)\) belongs to \( L^1(\mathbb R)\) , for \( k=0,1,…,n\) , we have
for any \( k=1,2,…,n\) .
Proof
These results are both elementary, as follows:
(1) Regarding the first assertion, the computation here is as follows:
(2) As for the second assertion, this follows from the first one, by recurrence.
As a conclusion to all this, we are on a good way with our theory, and we have:
Conclusion 7
Modulo normalization factors, the Fourier transform converts the derivatives into multiplications by the variable, and vice versa.
And isn’t this interesting, because isn’t computing derivatives a difficult task. Here is now another useful result, of the same type, this time regarding convolutions:
Theorem 189
Assuming \( f,g\in L^1(\mathbb R)\) , the following happens:
Moreover, under suitable assumptions, the formula \( \widehat{fg}=\widehat{f}*\widehat{g}\) holds too.
Proof
This is something quite subtle, the idea being as follows:
(1) Regarding the first assertion, this is something elementary, as follows:
(2) As for the second assertion, this is something more tricky, and we will be back to this later. In the meantime, here is however some sort of proof, not very honest:
To be more precise, the point here is that we can pass from the triple to the single integral by arguing that “we must have \( x=y\) ”. We will be back to this later.
As an updated conclusion to all this, we have, modulo a few bugs, to be fixed:
Conclusion 8
The Fourier transform converts the derivatives into multiplications by the variable, and convolutions into products, and vice versa.
We will see applications of this later, after developing some more general theory. So, let us develop now more theory for the Fourier transform. We first have:
Theorem 190
Given \( f\in L^1(\mathbb R)\) , its Fourier transform satisfies
called Riemann-Lebesgue property of \( \widehat{f}\) .
Proof
This is something quite technical, as follows:
(1) Given a function \( f:\mathbb R\to\mathbb C\) and a number \( y\in\mathbb R\) , let us set:
Our claim is then is that if \( f\in L^p(\mathbb R)\) , then the following function is uniformly continuous, with respect to the usual \( p\) -norm on the right:
(2) In order to prove this, fix \( \varepsilon>0\) . Since \( f\in L^p(\mathbb R)\) , we can find a function of type \( g:[-K,K]\to\mathbb C\) which is continuous, such that:
Now since \( g\) is uniformly continuous, we can find \( \delta\in(0,K)\) such that:
But this shows that we have the following estimate:
By using now the formula \( ||f||_p=||f_s||_p\) , which is clear, we obtain:
But this being true for any \( |s-t|<\delta\) , we have proved our claim.
(3) Let us prove now the Riemann-Lebesgue property of \( \widehat{f}\) , as formulated in the statement. By using \( e^{\pi i}=-1\) , and the change of variables \( x\to x-\pi/\xi\) , we have:
On the other hand, we have as well the following formula:
Thus by summing, we obtain the following formula:
But this gives the following estimate, with notations from (1):
Since by (1) this goes to \( 0\) with \( \xi\to\pm\infty\) , this gives the result.
Quite remarkably, and as a main result now regarding Fourier transforms, a function \( f:\mathbb R\to\mathbb C\) can be recovered from its Fourier transform \( \widehat{f}:\mathbb R\to\mathbb C\) , as follows:
Theorem 191
Assuming \( f,\widehat{f}\in L^1(\mathbb R)\) , we have
almost everywhere, called Fourier inversion formula.
Proof
This is something quite tricky, due to the fact that a direct attempt by double integration fails. Consider the following function, depending on a parameter \( \lambda>0\) :
We have then the following computation:
By letting now \( \lambda\to0\) , we obtain from this the following formula:
On the other hand, by using Theorem 11.27 we obtain that, almost everywhere:
Thus, we are led to the conclusion in the statement.
There are many more things that can be said about Fourier transforms, a key result being the Plancherel formula, over \( L^2(\mathbb R)\) . Also, we can talk about the Fourier transform over the Schwartz space \( \mathcal S\) . Also, we can talk about other types of Fourier transforms. As usual, if very interested in mathematical analysis, have a look at all this.
We have now enough accumulated material for talking about waves. Hang on, tough mathematics and physics to come next. To start with, waves appear as follows:
Theorem 192
The wave equation in \( \mathbb R^N\) is
where \( v>0\) is the propagation speed.
Proof
The equation in the statement is of course what comes out of experiments. However, allowing us a bit of imagination, and trust in this imagination, we can mathematically “prove” this equation, by discretizing, as follows:
(1) Let us first consider the 1D case. In order to understand the propagation of waves, we will model \( \mathbb R\) as a network of balls, with springs between them, as follows:
Now let us send an impulse, and see how the balls will be moving. For this purpose, we zoom on one ball. The situation here is as follows, \( l\) being the spring length:
We have two forces acting at \( x\) . First is the Newton motion force, mass times acceleration, which is as follows, with \( m\) being the mass of each ball:
And second is the Hooke force, displacement of the spring, times spring constant. Since we have two springs at \( x\) , this is as follows, \( k\) being the spring constant:
We conclude that the equation of motion, in our model, is as follows:
(2) Now let us take the limit of our model, as to reach to continuum. For this purpose we will assume that our system consists of \( B>>0\) balls, having a total mass \( M\) , and spanning a total distance \( L\) . Thus, our previous infinitesimal parameters are as follows, with \( K\) being the spring constant of the total system, which is of course lower than \( k\) :
With these changes, our equation of motion found in (1) reads:
Now observe that this equation can be written, more conveniently, as follows:
With \( N\to\infty\) , and therefore \( l\to0\) , we obtain in this way:
We are therefore led to the wave equation in the statement, which is \( \ddot{\varphi}=v^2\varphi''\) in our present \( N=1\) dimensional case, the propagation speed being \( v=\sqrt{K/M}\cdot L\) .
(3) In \( 2\) dimensions now, the same argument carries on. Indeed, we can use here a lattice model as follows, with all the edges standing for small springs:

As before in one dimension, we send an impulse, and we zoom on one ball. The situation here is as follows, with \( l\) being the spring length:

We have two forces acting at \( (x,y)\) . First is the Newton motion force, mass times acceleration, which is as follows, with \( m\) being the mass of each ball:
And second is the Hooke force, displacement of the spring, times spring constant. Since we have four springs at \( (x,y)\) , this is as follows, \( k\) being the spring constant:
We conclude that the equation of motion, in our model, is as follows:
(4) Now let us take the limit of our model, as to reach to continuum. For this purpose we will assume that our system consists of \( B^2>>0\) balls, having a total mass \( M\) , and spanning a total area \( L^2\) . Thus, our previous infinitesimal parameters are as follows, with \( K\) being the spring constant of the total system, taken to be equal to \( k\) :
With these changes, our equation of motion found in (3) reads:
Now observe that this equation can be written, more conveniently, as follows:
With \( N\to\infty\) , and therefore \( l\to0\) , we obtain in this way:
Thus, we are led in this way to the following wave equation in two dimensions, with \( v=\sqrt{K/M}\cdot L\) being the propagation speed of our wave:
But we recognize at right the Laplace operator, and we are done. As before in 1D, there is of course some discussion to be made here, arguing that our spring model in (3) is indeed the correct one. But do not worry, experiments confirm our findings.
(5) In 3 dimensions now, which is the case of the main interest, corresponding to our real-life world, the same argument carries over, and the wave equation is as follows:
(6) Finally, the same argument, namely a lattice model, carries on in arbitrary \( N\) dimensions, and the wave equation here is as follows:
Thus, we are led to the conclusion in the statement.
Getting now to mathematics, we will need a standard calculus result, as follows:
Proposition 44
The derivative of a function of type
is given by the formula \( \varphi'(x)=f(h(x))h'(x)-f(g(x))g'(x)\) .
Proof
Consider a primitive of the function that we integrate, \( F'=f\) . We have:
By using now the chain rule for derivatives, we obtain from this:
Thus, we are led to the formula in the statement.
Now back to the waves, the general result in 1D, due to d’Alembert, along with a little more, in relation with our lattice models above, is as follows:
Theorem 193
The solution of the 1D wave equation with initial value conditions \( \varphi(x,0)=f(x)\) and \( \dot{\varphi}(x,0)=g(x)\) is given by the d’Alembert formula, namely:
In the context of our lattice model discretizations, what happens is more or less that the above d’Alembert integral gets computed via Riemann sums.
Proof
There are several things going on here, the idea being as follows:
(1) Let us first check that the d’Alembert solution is indeed a solution of the wave equation \( \ddot{\varphi}=v^2\varphi''\) . The first time derivative is computed as follows:
The second time derivative is computed as follows:
Regarding now space derivatives, the first one is computed as follows:
As for the second space derivative, this is computed as follows:
Thus we have indeed \( \ddot{\varphi}=v^2\varphi''\) . As for the initial conditions, \( \varphi(x,0)=f(x)\) is clear from our definition of \( \varphi\) , and \( \dot{\varphi}(x,0)=g(x)\) is clear from our above formula of \( \dot{\varphi}\) .
(2) Conversely now, we must show that our solution is unique, but instead of going here into abstract arguments, we will simply solve our equation, which among others will doublecheck out computations in (1). Let us make the following change of variables:
With this change of variables, which is quite tricky, mixing space and time variables, our wave equation \( \ddot{\varphi}=v^2\varphi''\) reformulates in a very simple way, as follows:
But this latter equation tells us that our new \( \xi,\eta\) variables get separated, and we conclude from this that the solution must be of the following special form:
Now by taking into account the intial conditions \( \varphi(x,0)=f(x)\) and \( \dot{\varphi}(x,0)=g(x)\) , and then integrating, we are led to the d’Alembert formula in the statement.
(3) In regards now with our discretization questions, by using a 1D lattice model with balls and springs as before, what happens to all the above is more or less that the above d’Alembert integral gets computed via Riemann sums, in our model, as stated.
At \( N\geq2\) and higher things are more complicated. In the case \( N=3\) , which is the one that we are mainly interested in, the situation is as follows:
Fact 12
The Laplace operator in spherical coordinates is given by the following formula, with our standard convention for the \( 3D\) spherical coordinates,
and this allows to reformulate the 3D wave equation in spherical coordinates, and say a few things about it, say in relation with waves which propagate from a source.
And we will end our study here. Many interesting things that we have learned, in this chapter, and if interested in all this, a lot of exciting literature is waiting for you.
We had a quite broad chapter here, and as exercises on this, we have:
Exercise 81
Learn more about the theory of distributions, on \( \mathbb R\) .
Exercise 82
Learn as well about distributions in higher dimensions.
Exercise 83
Review if needed the full proof of the Cauchy formula.
Exercise 84
Learn more about harmonic functions, in \( 2\) or more dimensions.
Exercise 85
Learn the Plancherel formula, over \( L^2(\mathbb R)\) , and its applications.
Exercise 86
Learn about the Fourier transform over the Schwartz space \( \mathcal S\) .
Exercise 87
Learn about the other possible types of Fourier transforms.
Exercise 88
Read some physics, on the wave equation in \( 2D\) , and \( 3D\) .
As bonus exercise, now that we know about waves, learn about the heat equation too.
We discuss in this chapter the diagonalization of the linear operators \( T:H\to H\) over a Hilbert space \( H\) , usually taken to be separable. Our motivation comes from both mathematics, because what we will be doing here will be a natural infinite dimensional extension of basic linear algebra, and physics, more specifically quantum mechanics.
Let us begin our study with a number of general results regarding the operators that we are targeting. Regarding the self-adjoint operators, we have the following result:
Theorem 194
For an operator \( T\in B(H)\) , the following conditions are equivalent, and if they are satisfied, we call \( T\) self-adjoint:
In finite dimensions, we recover in this way the usual self-adjointness notion.
Proof
There are several assertions here, the idea being as follows:
\( (1)\implies(2)\) This implication is clear, because we have:
\( (2)\implies(1)\) In order to prove this, observe that the beginning of the above computation shows that, when assuming \( <Tx,x>\in\mathbb R\) , the following happens:
Thus, in terms of the operator \( S=T-T^*\) , we have:
In order to finish, we use a polarization trick. We have the following formula:
Since the first 3 terms vanish, the sum of the 2 last terms vanishes too. But, by using \( S^*=-S\) , coming from \( S=T-T^*\) , we can process this latter vanishing as follows:
Thus we must have \( <Sx,y>\in\mathbb R\) , and with \( y\to iy\) we obtain \( <Sx,y>\in i\mathbb R\) too, and so \( <Sx,y>=0\) . Thus \( S=0\) , which gives \( T=T^*\) , as desired.
(3) Finally, in what regards finite dimensions, here the condition \( T=T^*\) corresponds to the usual self-adjointness condition \( M=M^*\) at the level of the associated matrices.
Regarding the normal operators, which are more general, we have:
Theorem 195
For an operator \( T\in B(H)\) , the following conditions are equivalent, and if they are satisfied, we call \( T\) normal:
In finite dimensions, we recover in this way the usual normality notion.
Proof
There are several assertions here, the idea being as follows:
\( (1)\implies(2)\) This is clear, due to the following computation:
\( (2)\implies(1)\) This is clear as well, because the above computation shows that, when assuming \( ||Tx||=||T^*x||\) , the following happens:
Thus, in terms of the operator \( S=TT^*-T^*T\) , we have:
In order to finish, we use a polarization trick. We have the following formula:
Since the first 3 terms vanish, the sum of the 2 last terms vanishes too. But, by using \( S=S^*\) , coming from \( S=TT^*-T^*T\) , we can process this latter vanishing as follows:
Thus we must have \( <Sx,y>\in i\mathbb R\) , and with \( y\to iy\) we obtain \( <Sx,y>\in \mathbb R\) too, and so \( <Sx,y>=0\) . Thus \( S=0\) , which gives \( TT^*=T^*T\) , as desired.
(3) Finally, in what regards finite dimensions, or more generally the case where our Hilbert space comes with a basis, \( H=l^2(I)\) , here the condition \( TT^*=T^*T\) corresponds to the usual normality condition \( MM^*=M^*M\) at the level of the associated matrices.
Getting now to diagonalization work for the linear operators, as a first observation, we can talk about eigenvalues and eigenvectors of such operators, as follows:
Definition 59
Given an operator \( T\in B(H)\) , assuming that we have
we say that \( x\in H\) is an eigenvector of \( T\) , with eigenvalue \( \lambda\in\mathbb C\) .
We know many things about eigenvalues and eigenvectors, in the finite dimensional case. However, most of these will not extend to the infinite dimensional case, or at least not extend in a straightforward way, due to a number of reasons:
Summarizing, we are in trouble. Forgetting about (2), which obviously leads nowhere, let us focus on the difficulties in (1). In order to cut short the discussion there, regarding the various properties of \( T-\lambda\) , we can just say that \( T-\lambda\) is either invertible with bounded inverse, the “good case”, or not. We are led in this way to the following definition:
Definition 60
The spectrum of an operator \( T\in B(H)\) is the set
where \( B(H)^{-1}\subset B(H)\) is the set of invertible operators.
As a basic example, in the finite dimensional case, \( H=\mathbb C^N\) , the spectrum of a usual matrix \( A\in M_N(\mathbb C)\) is the collection of its eigenvalues, taken without multiplicities. We will see many other examples. In general, the spectrum has the following properties:
Proposition 45
The spectrum of \( T\in B(H)\) contains the eigenvalue set
and \( \varepsilon(T)\subset\sigma(T)\) is an equality in finite dimensions, but not in infinite dimensions.
Proof
We have several assertions here, the idea being as follows:
(1) First of all, the eigenvalue set is indeed the one in the statement, because \( Tx=\lambda x\) tells us precisely that \( T-\lambda\) must be not injective. The fact that we have \( \varepsilon(T)\subset\sigma(T)\) is clear as well, because if \( T-\lambda\) is not injective, it is not bijective.
(2) In finite dimensions we have \( \varepsilon(T)=\sigma(T)\) , because \( T-\lambda\) is injective if and only if it is bijective, with the boundedness of the inverse being automatic.
(3) In infinite dimensions we can assume \( H=l^2(\mathbb N)\) , and the shift operator \( S(e_i)=e_{i+1}\) is injective but not surjective. Thus \( 0\in\sigma(T)-\varepsilon(T)\) .
Philosophically, the best way of thinking at this is as follows: the numbers \( \lambda\notin\sigma(T)\) are good, because we can invert \( T-\lambda\) , the numbers \( \lambda\in\sigma(T)-\varepsilon(T)\) are bad, because so they are, and the eigenvalues \( \lambda\in\varepsilon(T)\) are evil. Welcome to operator theory.
Let us develop now some general theory. As a first result, we would like to prove that the spectra are non-empty. This is something tricky, and we will need:
Proposition 46
The following happen:
Proof
All these assertions are elementary, as follows:
(1) This follows as in the scalar case, the computation being as follows, provided that everything converges under the norm, which amounts in saying that \( ||T||<1\) :
(2) Assuming \( T\in B(H)^{-1}\) , let us pick \( S\in B(H)\) such that:
With this choice, we have then the following estimate:
Thus we have \( T^{-1}S\in B(H)^{-1}\) , and so \( S\in B(H)^{-1}\) , as desired.
(3) In the scalar case, the derivative of \( f(t)=t^{-1}\) is \( f'(t)=-t^{-2}\) . In the present normed space setting the derivative is no longer a number, but rather a linear transformation, which can be found by developing \( f(T)=T^{-1}\) at order 1, as follows:
Thus \( f(T)=T^{-1}\) is indeed differentiable, with derivative \( f'(T)S=-T^{-1}ST^{-1}\) .
We can now formulate our first theorem about spectra, as follows:
Theorem 196
The spectrum of a bounded operator \( T\in B(H)\) is:
Proof
This can be proved by using Proposition 12.6, as follows:
(1) In view of (2) below, it is enough to prove that \( \sigma(T)\) is closed. But this follows from the following computation, with \( |\varepsilon|\) being small:
(2) This follows from the following computation:
(3) Assume by contradiction \( \sigma(T)=\emptyset\) . Given a linear form \( f\in B(H)^*\) , consider the following map, which is well-defined, due to our assumption \( \sigma(T)=\emptyset\) :
By using the fact that \( T\to T^{-1}\) is differentiable, that we know from Proposition 12.6, we conclude that this map is differentiable, and so holomorphic. Also, we have:
Thus by the Liouville theorem we obtain \( \varphi=0\) . But, in view of the definition of \( \varphi\) , this gives \( (T-\lambda)^{-1}=0\) , which is a contradiction, as desired.
Here is now a second basic result regarding the spectra, inspired from what happens in finite dimensions, for the usual complex matrices, and which shows that things do not necessarily extend without troubles to the infinite dimensional setting:
Theorem 197
We have the following formula, valid for any operators \( S,T\) :
In finite dimensions we have \( \sigma(ST)=\sigma(TS)\) , but this fails in infinite dimensions.
Proof
There are several assertions here, the idea being as follows:
(1) This is something that we know in finite dimensions, coming from the fact that the characteristic polynomials of the associated matrices \( A,B\) coincide:
Thus we obtain \( \sigma(ST)=\sigma(TS)\) in this case, as claimed. Observe that this improves twice the general formula in the statement, first because we have no issues at 0, and second because what we obtain is actually an equality of sets with mutiplicities.
(2) In general now, let us first prove the main assertion, stating that \( \sigma(ST),\sigma(TS)\) coincide outside 0. We first prove that we have the following implication:
Assume indeed that \( 1-ST\) is invertible, with inverse denoted \( R\) :
We have then the following formulae, relating our variables \( R,S,T\) :
By using \( RST=R-1\) , we have the following computation:
A similar computation, using \( STR=R-1\) , shows that we have:
Thus \( 1-TS\) is invertible, with inverse \( 1+TRS\) , which proves our claim. Now by multiplying by scalars, we deduce from this that for any \( \lambda\in\mathbb C-\{0\}\) we have:
But this leads to the conclusion in the statement.
(3) Regarding now the counterexample to the formula \( \sigma(ST)=\sigma(TS)\) , in general, let us take \( S\) to be the shift on \( H=L^2(\mathbb N)\) , given by the following formula:
As for \( T\) , we can take it to be the adjoint of \( S\) , which is the following operator:
Let us compose now these two operators. In one sense, we have:
In the other sense, however, the situation is different, as follows:
Thus, the spectra do not match on \( 0\) , and we have our counterexample, as desired.
Let us develop now some systematic theory for the computation of the spectra, based on what we know about the eigenvalues of the usual complex matrices. As a first result, which is well-known for the usual matrices, and extends well, we have:
Theorem 198
We have the “polynomial functional calculus” formula
valid for any polynomial \( P\in\mathbb C[X]\) , and any operator \( T\in B(H)\) .
Proof
We pick a scalar \( \lambda\in\mathbb C\) , and we decompose the polynomial \( P-\lambda\) :
We have then the following equivalences:
Thus, we are led to the formula in the statement.
The above result is something very useful, and generalizing it will be our next task. As a first ingredient here, assuming that \( A\in M_N(\mathbb C)\) is invertible, we have:
It is possible to extend this formula to the arbitrary operators, and we will do this in a moment. Before starting, however, we have to find a class of functions generalizing both the polynomials \( P\in\mathbb C[X]\) and the inverse function \( x\to x^{-1}\) . The answer to this question is provided by the rational functions, which are as follows:
Definition 61
A rational function \( f\in\mathbb C(X)\) is a quotient of polynomials:
Assuming that \( P,Q\) are prime to each other, we can regard \( f\) as a usual function,
with \( X\) being the set of zeros of \( Q\) , also called poles of \( f\) .
Now that we have our class of functions, the next step consists in applying them to operators. Here we cannot expect \( f(T)\) to make sense for any \( f\) and any \( T\) , for instance because \( T^{-1}\) is defined only when \( T\) is invertible. We are led in this way to:
Definition 62
Given an operator \( T\in B(H)\) , and a rational function \( f=P/Q\) having poles outside \( \sigma(T)\) , we can construct the following operator,
that we can denote as a usual fraction, as follows,
due to the fact that \( P(T),Q(T)\) commute, so that the order is irrelevant.
To be more precise, \( f(T)\) is indeed well-defined, and the fraction notation is justified too. In more formal terms, we can say that we have a morphism of complex algebras as follows, with \( \mathbb C(X)^T\) standing for the rational functions having poles outside \( \sigma(T)\) :
Summarizing, we have now a good class of functions, generalizing both the polynomials and the inverse map \( x\to x^{-1}\) . We can now extend Theorem 12.9, as follows:
Theorem 199
We have the “rational functional calculus” formula
valid for any rational function \( f\in\mathbb C(X)\) having poles outside \( \sigma(T)\) .
Proof
We pick a scalar \( \lambda\in\mathbb C\) , we write \( f=P/Q\) , and we set:
By using now Theorem 12.9, for this polynomial, we obtain:
Thus, we are led to the formula in the statement.
As an application of the above methods, we can investigate certain special classes of operators, such as the self-adjoint ones, and the unitary ones. Let us start with:
Proposition 47
The following happen:
Proof
We have several assertions here, the idea being as follows:
(1) The spectrum of the adjoint operator \( T^*\) can be computed as follows:
(2) This is clear indeed from (1).
(3) For a unitary operator, \( U^*=U^{-1}\) , Theorem 12.12 and (1) give:
Thus, we are led to the conclusion in the statement.
In analogy with what happens for the usual matrices, we would like to improve now (2,3) above, with results stating that the spectrum \( X=\sigma(T)\) satisfies \( X\subset\mathbb R\) for self-adjoints, and \( X\subset\mathbb T\) for unitaries. This will be tricky. Let us start with:
Theorem 200
The spectrum of a unitary operator
is on the unit circle, \( \sigma(U)\subset\mathbb T\) .
Proof
Assuming \( U^*=U^{-1}\) , we have the following norm computation:
Now if we denote by \( D\) the unit disk, we obtain from this:
On the other hand, once again by using \( U^*=U^{-1}\) , we have as well:
Thus, as before with \( D\) being the unit disk in the complex plane, we have:
Now by using Theorem 12.12, we obtain \( \sigma(U) \subset D\cap D^{-1} =\mathbb T\) , as desired.
We have as well a similar result for self-adjoints, as follows:
Theorem 201
The spectrum of a self-adjoint operator
consists of real numbers, \( \sigma(T)\subset\mathbb R\) .
Proof
The idea is that we can deduce the result from Theorem 12.14, by using the following remarkable rational function, depending on a parameter \( r\in\mathbb R\) :
Indeed, for \( r>>0\) the operator \( f(T)\) is well-defined, and we have:
Thus \( f(T)\) is unitary, and by using Theorem 12.14 we obtain:
Thus, we are led to the conclusion in the statement.
Moving on, one key thing that we know about matrices, which is clear for the diagonalizable matrices, and then in general follows by density, is the following formula:
We would like to have such formulae for the general operators \( T\in B(H)\) , but this is something quite technical. Consider the rational calculus morphism from Definition 12.11, which is as follows, with the exponent standing for “having poles outside \( \sigma(T)\) ”:
As mentioned before, the rational functions are holomorphic outside their poles, and this raises the question of extending this morphism, as follows:
Normally this can be done in several steps. Let us start with:
Proposition 48
We can exponentiate any operator \( T\in B(H)\) , by setting:
Similarly, we can define \( f(T)\) , for any holomorphic function \( f:\mathbb C\to\mathbb C\) .
Proof
We must prove that the series defining \( e^T\) converges, and this follows from:
The case of the arbitrary holomorphic functions \( f:\mathbb C\to\mathbb C\) is similar.
In general, the holomorphic functions are not entire, and the above method won’t cover the rational functions \( f\in\mathbb C(X)^T\) that we want to generalize. Thus, we must use something else. And the answer here comes from the Cauchy formula:
Indeed, given a rational function \( f\in\mathbb C(X)^T\) , the operator \( f(T)\in B(H)\) , constructed in Definition 12.9, can be recaptured in an analytic way, as follows:
Now given an arbitrary function \( f\in Hol(\sigma(T))\) , we can define \( f(T)\in B(H)\) by the exactly same formula, and we obtain in this way the desired correspondence:
This was for the plan. In practice now, all this needs a bit of care, with many verifications needed, and with the technical remark that a winding number must be added to the above Cauchy formulae, for things to be correct. Let us start with:
Definition 63
If \( \gamma\) is a loop in \( \mathbb C\) the number of times \( \gamma\) goes around a point \( z\in\mathbb C-\gamma\) is computed by the following integral, called winding number:
We say that \( \gamma\) turns around \( z\) if \( Ind(\gamma,z)=1\) , and that it does not turn if \( Ind(\gamma,z)=0\) . Otherwise, we say that \( \gamma\) turns around \( z\) many times, or in the bad sense, or both.
Let \( f:U\to\mathbb C\) be an holomorphic function defined on an open subset of \( \mathbb C\) , and \( \gamma\) be a loop in \( U\) . If \( Ind(\gamma,z)\neq 0\) for \( z\in\mathbb C-U\) then \( f(z)\) is given by the Cauchy formula:
Also, if \( Ind(\gamma,z)=0\) for \( z\in\mathbb C-U\) then the integral of \( f\) on \( \gamma\) is zero:
It is convenient to use formal combinations of loops, called cycles:
The winding number for \( \Sigma\) is by definition the corresponding linear combination of winding numbers of its loop components, and the Cauchy formula holds for arbitrary cycles. Now by getting back to the linear operators, we can formulate:
Definition 64
Let \( T\in B(H)\) and let \( f:U\to\mathbb C\) be an holomorphic function defined on an open set containing \( \sigma(T)\) . Define an element \( f(T)\) by the formula
where \( \Sigma\) is a cycle in \( U-\sigma(T)\) which turns around \( \sigma(T)\) and doesn’t turn around \( \mathbb C-U\) .
The formula makes sense because \( \Sigma\) is in \( U-\sigma(T)\) . Also, \( f(T)\) is independent of the choice of \( \Sigma\) . Indeed, let \( \Sigma_1\) and \( \Sigma_2\) be two cycles. Their difference \( \Sigma_1-\Sigma_2\) is a cycle which doesn’t turn around \( \sigma(a)\) , neither around \( \mathbb C-U\) . The function \( z\to f(z)/(z-T)\) being holomorphic \( U-\sigma(T)\to B(H)\) , its integral on \( \Sigma_1-\Sigma_2\) must be zero:
Thus \( f(T)\) is the same with respect to \( \Sigma_1\) and to \( \Sigma_2\) , and so Definition 12.18 is fully justified. Now with this definition in hand, we have the following result:
Theorem 202
Given \( T\in B(H)\) , we have a morphism of algebras as follows, where \( Hol(\sigma(T))\) is the algebra of functions which are holomorphic around \( \sigma(T)\) ,
which extends the previous rational functional calculus \( f\to f(T)\) . We have:
Moreover, if \( \sigma(T)\) is contained in an open set \( U\) and \( f_n,f:U\to\mathbb C\) are holomorphic functions such that \( f_n\to f\) uniformly on compact subsets of \( U\) then \( f_n(T)\to f(T)\) .
Proof
There are several things to be proved here, as follows:
(1) Consider indeed the algebra \( Hol(\sigma(T))\) , with the convention that two functions are identified if they coincide on an open set containing \( \sigma(T)\) . We have then a construction \( f\to f(T)\) as in the statement, provided by Definition 12.18.
(2) Let us prove now that our construction extends the one for rational functions. Since \( 1,z\) generate \( \mathbb C(X)\) , it is enough to show that \( f(z)=1\) implies \( f(T)=1\) , and that \( f(z)=z\) implies \( f(T)=T\) . For this purpose, we prove that \( f(z)=z^n\) implies \( f(T)=T^n\) for any \( n\) . But this follows by integrating over a circle \( \gamma\) of big radius, as follows:
(3) Regarding \( \sigma(f(T))=f(\sigma(T))\) , it is enough to prove that this equality holds on the point \( 0\) , and we can do this by double inclusion, as follows:
“\( \supset\) ”. Assume that \( f(\sigma(T))\) contains \( 0\) , and let \( z_0\in\sigma(T)\) be such that \( f(z_0)=0\) . Consider the function \( g(z)=f(z)/(z-z_0)\) . We have \( g(T)(T-z_0)=f(T)\) by using the morphism property. Since \( T-z_0\) is not invertible, \( f(T)\) is not invertible either.
“\( \subset\) ”. Assume now that \( f(\sigma(T))\) does not contain \( 0\) . With the holomorphic function \( g(z)=1/f(z)\) we get \( g(T)=f(T)^{-1}\) , so \( f(T)\) is invertible, and we are done.
(4) Finally, regarding the last assertion, this is clear from definitions. And with the remark that this can be applied to holomorphic functions written as series:
Indeed, if this is the expansion of \( f\) around \( z_0\) , with convergence radius \( r\) , and if \( \sigma(T)\) is contained in the disc centered at \( z_0\) of radius \( r\) , then \( f(T)\) is given by:
Summarizing, we have proved the result, and fully extended Theorem 12.12.
In order to formulate now our next result, we will need the following notion:
Definition 65
Given an operator \( T\in B(H)\) , its spectral radius
is the radius of the smallest disk centered at \( 0\) containing \( \sigma(T)\) .
Now with this notion in hand, we have the following key result, improving our key theoretical result so far about spectra, namely \( \sigma(T)\neq\emptyset\) , from Theorem 12.7:
Theorem 203
The spectral radius of an operator \( T\in B(H)\) is given by
and in this formula, we can replace the limit by an inf.
Proof
We have several things to be proved, the idea being as follows:
(1) Our first claim is that the numbers \( u_n=||T^n||^{1/n}\) satisfy:
Indeed, we have the following estimate, using the Young inequality \( ab\leq a^p/p+b^q/q\) , with exponents \( p=(n+m)/n\) and \( q=(n+m)/m\) :
(2) Our second claim is that the second assertion holds, namely:
For this purpose, we just need the inequality found in (1). Indeed, fix \( m\geq1\) , let \( n\geq1\) , and write \( n=lm+r\) with \( 0\leq r\leq m-1\) . By using twice \( u_{ab}\leq u_b\) , we get:
It follows that we have \( \lim\sup_nu_n\leq u_m\) , which proves our claim.
(3) Summarizing, we are left with proving the main formula, which is as follows, and with the remark that we already know that the sequence on the right converges:
In one sense, we can use the polynomial calculus formula \( \sigma(T^n)=\sigma(T)^n\) . Indeed, this gives the following estimate, valid for any \( n\) , as desired:
(4) For the reverse inequality, we fix a number \( \rho>\rho(T)\) , and we want to prove that we have \( \rho\geq\lim_{n\to\infty}||T^n||^{1/n}\) . By using the Cauchy formula, we have:
By applying the norm we obtain from this formula:
Since the sup does not depend on \( n\) , by taking \( n\) -th roots, we obtain in the limit:
Now recall that \( \rho\) was by definition an arbitrary number satisfying \( \rho>\rho(T)\) . Thus, we have obtained the following estimate, valid for any \( T\in B(H)\) :
Thus, we are led to the conclusion in the statement.
In the case of the normal elements, we have the following finer result:
Theorem 204
The spectral radius of a normal element,
is equal to its norm.
Proof
We can proceed in two steps, as follows:
Step 1. In the case \( T=T^*\) we have \( ||T^n||=||T||^n\) for any exponent of the form \( n=2^k\) , by using the formula \( ||TT^*||=||T||^2\) , and by taking \( n\) -th roots we get:
Thus, we are done with the self-adjoint case, with the result \( \rho(T)=||T||\) .
Step 2. In the general normal case \( TT^*=T^*T\) we have \( T^n(T^n)^*=(TT^*)^n\) , and by using this, along with the result from Step 1, applied to \( TT^*\) , we obtain:
Thus, we are led to the conclusion in the statement.
By using Theorem 12.22 we can say a number of non-trivial things about the self-adjoint and normal operators, commonly known as “spectral theorems”. As a first result here, we can improve the polynomial functional calculus formula, as follows:
Theorem 205
Given \( T\in B(H)\) normal, we have a morphism of algebras
having the properties \( ||P(T)||=||P_{|\sigma(T)}||\) , and \( \sigma(P(T))=P(\sigma(T))\) .
Proof
This is an improvement of Theorem 12.9 in the normal case, with the extra assertion being the norm estimate. But the element \( P(T)\) being normal, we can apply to it the spectral radius formula for normal elements, and we obtain:
Thus, we are led to the conclusions in the statement.
We can improve as well the rational calculus formula, and the holomorphic calculus formula, in the same way. Importantly now, at a more advanced level, we have:
Theorem 206
Given \( T\in B(H)\) normal, we have a morphism of algebras
which is isometric, \( ||f(T)||=||f||\) , and has the property \( \sigma(f(T))=f(\sigma(T))\) .
Proof
The idea here is to “complete” the morphism in Theorem 12.23, namely:
Indeed, we know from Theorem 12.23 that this morphism is continuous, and is in fact isometric, when regarding the polynomials \( P\in\mathbb C[X]\) as functions on \( \sigma(T)\) :
Thus, by Stone-Weierstrass, we have a unique isometric extension, as follows:
It remains to prove \( \sigma(f(T))=f(\sigma(T))\) , and we can do this by double inclusion:
“\( \subset\) ” Given a continuous function \( f\in C(\sigma(T))\) , we must prove that we have:
For this purpose, consider the following function, which is well-defined:
We can therefore apply this function to \( T\) , and we obtain:
In particular \( f(T)-\lambda\) is invertible, so \( \lambda\notin\sigma(f(T))\) , as desired.
“\( \supset\) ” Given a continuous function \( f\in C(\sigma(T))\) , we must prove that we have:
But this is the same as proving that we have:
For this purpose, we approximate our function by polynomials, \( P_n\to f\) , and we examine the following convergence, which follows from \( P_n\to f\) :
We know from polynomial functional calculus that we have:
Thus, the operators \( P_n(T)-P_n(\mu)\) are not invertible. On the other hand, we know that the set formed by the invertible operators is open, so its complement is closed. Thus the limit \( f(T)-f(\mu)\) is not invertible either, and so \( f(\mu)\in\sigma(f(T))\) , as desired.
As an important comment, Theorem 12.24 is not exactly in final form, because it misses an important point, namely that our correspondence maps:
However, this is something non-trivial, and we will be back to this later. Observe however that Theorem 12.24 is fully powerful for the self-adjoint operators, \( T=T^*\) , where the spectrum is real, so where \( z=\bar{z}\) on the spectrum. We will be back to this.
As a second result now, along the same lines, we can further extend Theorem 12.24 into a measurable functional calculus theorem, as follows:
Theorem 207
Given \( T\in B(H)\) normal, we have a morphism of algebras as follows, with \( L^\infty\) standing for abstract measurable functions, or Borel functions,
which is isometric, \( ||f(T)||=||f||\) , and has the property \( \sigma(f(T))=f(\sigma(T))\) .
Proof
As before, the idea will be that of “completing” what we have. To be more precise, we can use the Riesz theorem and a polarization trick, as follows:
(1) Given a vector \( x\in H\) , consider the following functional:
By the Riesz theorem, this functional must be the integration with respect to a certain measure \( \mu\) on the space \( \sigma(T)\) . Thus, we have a formula as follows:
Now given an arbitrary Borel function \( f\in L^\infty(\sigma(T))\) , as in the statement, we can define a number \( <f(T)x,x>\in\mathbb C\) , by using exactly the same formula, namely:
Thus, we have managed to define numbers \( <f(T)x,x>\in\mathbb C\) , for all vectors \( x\in H\) , and in addition we can recover these numbers as follows, with \( g_n\in C(\sigma(T))\) :
(2) In order to define now numbers \( <f(T)x,y>\in\mathbb C\) , for all vectors \( x,y\in H\) , we can use a polarization trick. Indeed, for any operator \( S\in B(H)\) we have:
By replacing \( y\to iy\) , we have as well the following formula:
By multiplying this formula by \( i\) , and summing with the first one, we obtain:
(3) But with this, we can now finish. Indeed, by combining (1,2), given a Borel function \( f\in L^\infty(\sigma(T))\) , we can define numbers \( <f(T)x,y>\in\mathbb C\) for any \( x,y\in H\) , and it is routine to check, by using approximation by continuous functions \( g_n\to f\) as in (1), that we obtain in this way an operator \( f(T)\in B(H)\) , having all the desired properties.
As a comment here, the above result and its proof provide us with more than a Borel functional calculus, because what we got is a certain measure on the spectrum \( \sigma(T)\) , along with a functional calculus for the \( L^\infty\) functions with respect to this measure. We will be back to this later, and for the moment we will only need Theorem 12.25 as formulated, with \( L^\infty(\sigma(T))\) standing, a bit abusively, for the Borel functions on \( \sigma(T)\) .
We can now diagonalize the normal operators. We will do this in 3 steps, first for the self-adjoint operators, then for the families of commuting self-adjoint operators, and finally for the general normal operators, by using a trick of the following type:
The diagonalization in infinite dimensions is more tricky than in finite dimensions, and instead of writing a formula of type \( T=UDU^*\) , with \( U,D\in B(H)\) being respectively unitary and diagonal, we will express our operator as \( T=U^*MU\) , with \( U:H\to K\) being a certain unitary, and \( M\in B(K)\) being a certain diagonal operator. This is how the spectral theorem is best formulated, in view of applications.
In practice, the explicit construction of \( U,M\) , which will be actually part of the proof, is also needed. For the self-adjoint operators, the statement and proof are as follows:
Theorem 208
Any self-adjoint operator \( T\in B(H)\) can be diagonalized,
with \( U:H\to L^2(X)\) being a unitary operator from \( H\) to a certain \( L^2\) space associated to \( T\) , with \( f:X\to\mathbb R\) being a certain function, once again associated to \( T\) , and with
being the usual multiplication operator by \( f\) , on the Hilbert space \( L^2(X)\) .
Proof
The construction of \( U,f\) can be done in several steps, as follows:
(1) We first prove the result in the special case where our operator \( T\) has a cyclic vector \( x\in H\) , with this meaning that the following holds:
For this purpose, let us go back to the proof of Theorem 12.25. We will use the following formula from there, with \( \mu\) being the measure on \( X=\sigma(T)\) associated to \( x\) :
Our claim is that we can define a unitary \( U:H\to L^2(X)\) , first on the dense part spanned by the vectors \( T^kx\) , by the following formula, and then by continuity:
Indeed, the following computation shows that \( U\) is well-defined, and isometric:
We can then extend \( U\) by continuity into a unitary \( U:H\to L^2(X)\) , as claimed. Now observe that we have the following formula:
Thus our result is proved in the present case, with \( U\) as above, and with \( f(z)=z\) .
(2) We discuss now the general case. Our first claim is that \( H\) has a decomposition as follows, with each \( H_i\) being invariant under \( T\) , and admitting a cyclic vector \( x_i\) :
Indeed, this is something elementary, the construction being by recurrence in finite dimensions, in the obvious way, and by using the Zorn lemma in general. Now with this decomposition in hand, we can make a direct sum of the diagonalizations obtained in (1), for each of the restrictions \( T_{|H_i}\) , and we obtain the formula in the statement.
We have the following technical generalization of the above result:
Theorem 209
Any family of commuting self-adjoint operators \( T_i\in B(H)\) can be jointly diagonalized,
with \( U:H\to L^2(X)\) being a unitary operator from \( H\) to a certain \( L^2\) space associated to \( \{T_i\}\) , with \( f_i:X\to\mathbb R\) being certain functions, once again associated to \( T_i\) , and with
being the usual multiplication operator by \( f_i\) , on the Hilbert space \( L^2(X)\) .
Proof
This is similar to the proof of Theorem 12.26, by suitably modifying the measurable calculus formula, and the measure \( \mu\) itself, as to have this formula working for all the operators \( T_i\) . With this modification done, everything extends.
In order to discuss now the case of the arbitrary normal operators, we will need:
Proposition 49
Any operator \( T\in B(H)\) can be written as
with \( Re(T),Im(T)\in B(H)\) being self-adjoint, and this decomposition is unique.
Proof
This is something elementary, the idea being as follows:
(1) We can use indeed the same formulae for the real and imaginary part as in the complex number case, the decomposition formula being as follows:
To be more precise, both the operators on the right are self-adjoint, and the summing formula holds indeed, and so we have our decomposition result, as desired.
(2) Regarding now the uniqueness, by linearity it is enough to show that \( R+iS=0\) with \( R,S\) both self-adjoint implies \( R=S=0\) . But this follows by applying the adjoint to \( R+iS=0\) , which gives \( R-iS=0\) , and so \( R=S=0\) , as desired.
We can now discuss the case of arbitrary normal operators, as follows:
Theorem 210
Any normal operator \( T\in B(H)\) can be diagonalized,
with \( U:H\to L^2(X)\) being a unitary operator from \( H\) to a certain \( L^2\) space associated to \( T\) , with \( f:X\to\mathbb C\) being a certain function, once again associated to \( T\) , and with
being the usual multiplication operator by \( f\) , on the Hilbert space \( L^2(X)\) .
Proof
This is our main diagonalization theorem, the idea being as follows:
(1) Consider the decomposition of \( T\) into its real and imaginary parts, as constructed in the proof of Proposition 12.28, namely:
We know that the real and imaginary parts are self-adjoint operators. Now since \( T\) was assumed to be normal, \( TT^*=T^*T\) , these real and imaginary parts commute:
Thus Theorem 12.27 applies to these real and imaginary parts, and gives the result.
This was for our series of diagonalization theorems. There is of course one more result here, regarding the families of commuting normal operators, as follows:
Theorem 211
Any family of commuting normal operators \( T_i\in B(H)\) can be jointly diagonalized,
with \( U:H\to L^2(X)\) being a unitary operator from \( H\) to a certain \( L^2\) space associated to \( \{T_i\}\) , with \( f_i:X\to\mathbb C\) being certain functions, once again associated to \( T_i\) , and with
being the usual multiplication operator by \( f_i\) , on the Hilbert space \( L^2(X)\) .
Proof
This is similar to the proof of Theorem 12.27 and Theorem 12.29, by combining the arguments there. To be more precise, this follows as Theorem 12.27, by using the decomposition trick from the proof of Theorem 12.29.
With the above diagonalization results in hand, we can now “fix” the continuous and measurable functional calculus theorems, with a key complement, as follows:
Theorem 212
Given a normal operator \( T\in B(H)\) , the following hold, for both the functional calculus and the measurable calculus morphisms:
Proof
These assertions are more or less equivalent, with (1) being the main one, which obviously implies everything else. But this assertion (1) follows from the diagonalization result for normal operators, from Theorem 12.29.
This was a quite fundamental chapter, and as exercises on this, we have:
Exercise 89
Develop the theory of positive operators, satisfying \( <Tx,x>\geq0\) .
Exercise 90
Discuss as well the strict positivity, \( <Tx,x>>0\) for \( x\neq0\) .
Exercise 91
Check the various details of the holomorphic calculus theorem.
Exercise 92
Check as well the details of the measurable calculus theorem.
Exercise 93
Learn the various alternative proofs for the spectral theorem.
Exercise 94
Work out a polar decomposition result, for the linear operators.
Exercise 95
Learn about the compact operators, and their various properties.
Exercise 96
Learn also about unbounded operators, and quantum mechanics.
As bonus exercise, and no surprise here, learn some systematic operator theory.
Down the road I look and there runs Mary
Hair of gold and lips like cherries
It’s good to touch
The green, green grass of home
We have learned the basics of measure theory and probability, and time now to see if this knowledge can be of any help, in relation with concrete questions. The first question that we would like to discuss, which is something very basic, is as follows:
Question 10
Given a graph \( X\) , with a distinguished vertex \( *\) :
To be more precise, we are mainly interested in the first question, counting loops on graphs, with this being notoriously related to many applied mathematics questions, of discrete type. As for the second question, this is a technical, useful probabilistic reformulation of the first question, that we will usually prefer, in what follows.
Actually, in relation with this, the fact that a real measure \( \mu\) as above exists indeed is not exactly obvious. But comes from the following result, which is something quite elementary, and which can be very helpful for explicit computations:
Theorem 213
Given a graph \( X\) , with adjacency matrix \( d\in M_N(0,1)\) , we have:
When writing \( d=UDU^t\) with \( U\in O_N\) and \( D=diag(\lambda_1,…,\lambda_N)\) with \( \lambda_i\in\mathbb R\) , we have
and the real probability measure \( \mu\) having these numbers as moments is given by
with the delta symbols standing as usual for Dirac masses.
Proof
There are several things going on here, the idea being as follows:
(1) According to the usual rule of matrix multiplication, the formula for the powers of the adjacency matrix \( d\in M_N(0,1)\) is as follows:
In particular, with \( i_0=i_k=*\) , we obtain the following formula, as claimed:
(2) Now since \( d\in M_N(0,1)\) is symmetric, this matrix is diagonalizable, with the diagonalization being as follows, with \( U\in O_N\) , and \( D=diag(\lambda_1,…,\lambda_N)\) with \( \lambda_i\in\mathbb R\) :
By using this formula, we obtain the second formula in the statement:
(3) Finally, the last assertion is clear from this, because the moments of the measure in the statement, \( \mu=\sum_iU_{*i}^2\delta_{\lambda_i}\) , are the following numbers:
Observe also that \( \mu\) is indeed of mass 1, because all rows of \( U\in O_N\) must be of norm 1, and so \( \sum_iU_{*i}^2=1\) . Thus, we are led to the conclusions in the statement.
At the level of examples, the finite graphs can be quite complicated, and we will discuss them later. So, let us look instead at the infinite graphs, and try for instance to count the length \( k\) paths on \( \mathbb Z\) , based at \( 0\) . At \( k=1\) we have \( 2\) such paths, ending at \( -1\) and \( 1\) , and the count results can be pictured as follows, in a self-explanatory way:

At \( k=2\) now, we have 4 paths, one of which ends at \( -2\) , two of which end at 0, and one of which ends at 2. The results can be pictured as follows:

At \( k=3\) now, we have 8 paths, the distribution of the endpoints being as follows:

As for \( k=4\) , here we have 16 paths, the distribution of the endpoints being as follows:

And good news, we can see in the above the Pascal triangle. Thus, getting back now to Question 13.1, we can answer it for the graph \( \mathbb Z\) , the result being as follows:
Theorem 214
The paths on \( \mathbb Z\) are counted by the binomial coefficients. In particular, the \( 2k\) -paths based at \( 0\) are counted by the central binomial coefficients,
and \( \mu\) is the centered measure having these numbers as even moments.
Proof
This basically follows from the above discussion, as follows:
(1) In what regards the count, we certainly have the Pascal triangle, as discovered above, and the rest is just a matter of finishing. There are many possible ways here, a straightforward one being that of arguing that the number \( C_k^l\) of length \( k\) loops \( 0\to l\) is subject, due to the binary choice at the end, to the following recurrence relation:
But this is exactly the recurrence for the Pascal triangle, so done with the count.
(2) As for the second assertion, the first part, regarding \( L_{2k}\) , is clear from this, and the second part is more of an empty statement, with \( \mu\) still remaining to be computed.
As a second illustration, let us try to count the loops of \( \mathbb N\) , based at 0. This is something less obvious, and at the experimental level, the result is as follows:
Proposition 50
The Catalan numbers \( C_k\) , counting the loops on \( \mathbb N\) based at \( 0\) ,
are numerically \( 1,2,5,14,42,132,429,1430,4862,16796,58786,…\)
Proof
To start with, we have indeed \( C_1=1\) , the only loop here being \( 0-1-0\) . Then we have \( C_2=2\) , due to two possible loops, namely:
Then we have \( C_3=5\) , the possible loops here being as follows:
In general, the same method works, with \( C_4=14\) being left to you, as an exercise, and with \( C_5\) and higher to me, and I will be back with the solution, in due time.
Obviously, computing the numbers \( C_k\) is no easy task, and finding the formula of \( C_k\) , out of the data that we have, does not look as an easy task either. So, we will do what combinatorists do, let me teach you. The first step is to relax, then to look around, not with the aim of computing your numbers \( C_k\) , but rather with the aim of finding other objects counted by the same numbers \( C_k\) . With a bit of luck, among these objects some will be easier to count than the others, and this will eventually compute \( C_k\) .
This was for the strategy. In practice now, we first have the following result:
Theorem 215
The Catalan numbers \( C_k\) count:
Proof
All this is standard combinatorics, the idea being as follows:
(1) To start with, in what regards the various objects involved, the length \( 2k\) loops on \( \mathbb N\) are the length \( 2k\) loops on \( \mathbb N\) that we know, and the same goes for the noncrossing pairings of \( 1,…,2k\) , and for the noncrossing partitions of \( 1,…,k\) , the idea here being that you must be able to draw the pairing or partition in a noncrossing way.
(2) Regarding now the length \( 2k\) Dyck paths in the plane, these are by definition the paths from \( (0,0)\) to \( (k,k)\) , marching North-East over the integer lattice \( \mathbb Z^2\subset\mathbb R^2\) , by staying inside the square \( [0,k]\times[0,k]\) , and staying as well under the diagonal of this square. As an example, here are the 5 possible Dyck paths at \( n=3\) :

(3) Thus, we have definitions for all the objects involved, and in each case, if you start counting them, as we did in Proposition 13.4 with the loops on \( \mathbb N\) , you always end up with the same sequence of numbers, namely those found in Proposition 13.4:
(4) In order to prove now that (1-4) produce indeed the same numbers, many things can be said. The idea is that, leaving aside mathematical brevity, and more specifically abstract reasonings of type \( a=b,b=c\implies a=c\) , what we have to do, in order to fully understand what is going on, is to etablish \( \binom{4}{2}=6\) equalities, via bijective proofs.
(5) But this can be done, indeed. As an example here, the noncrossing pairings of \( 1,…,2k\) from (2) are in bijection with the noncrossing partitions of \( 1,…,k\) from (3), via fattening the pairings and shrinking the partitions. We will leave the details here as an instructive exercise, and exercise as well, to add (1) and (4) to the picture.
(6) However, matter of having our theorem formally proved, I mean by me professor and not by you student, here is a less elegant argument, which is however very quick, and does the job. The point is that, in each of the cases (1-4) under consideration, the numbers \( C_k\) that we get are easily seen to be subject to the following recurrence:
The initial data being the same, namely \( C_1=1\) and \( C_2=2\) , in each of the cases (1-4) under consideration, we get indeed the same numbers.
Now we can pass to the second step, namely selecting in the above list the objects that we find the most convenient to count, and count them. This leads to:
Theorem 216
The Catalan numbers are given by the formula
with this being best seen by counting the length \( 2k\) Dyck paths in the plane.
Proof
This is something quite tricky, the idea being as follows:
(1) Let us count indeed the Dyck paths in the plane. For this purpose, we use a trick. Indeed, if we ignore the assumption that our path must stay under the diagonal of the square, we have \( \binom{2k}{k}\) such paths. And among these, we have the “good” ones, those that we want to count, and then the “bad” ones, those that we want to ignore.
(2) So, let us count the bad paths, those crossing the diagonal of the square, and reaching the higher diagonal next to it, the one joining \( (0,1)\) and \( (k,k+1)\) . In order to count these, the trick is to “flip” their bad part over that higher diagonal, as follows: 
(3) Now observe that, as it is obvious on the above picture, due to the flipping, the flipped bad path will no longer end in \( (k,k)\) , but rather in \( (k-1,k+1)\) . Moreover, more is true, in the sense that, by thinking a bit, we see that the flipped bad paths are precisely those ending in \( (k-1,k+1)\) . Thus, we can count these flipped bad paths, and so the bad paths, and so the good paths too, and so good news, we are done.
(4) To finish now, by putting everything together, we have:
Thus, we are led to the formula in the statement.
We have as well another approach to all this, computation of the Catalan numbers, this time based on rock-solid standard calculus, as follows:
Theorem 217
The Catalan numbers have the following properties:
Proof
This is best viewed by using noncrossing pairings, as follows:
(1) Let us count the noncrossing pairings of \( \{1,…,2k+2\}\) . Such a pairing appears by pairing 1 to an odd number, \( 2a+1\) , and then inserting a noncrossing pairing of \( \{2,…,2a\}\) , and a noncrossing pairing of \( \{2a+2,…,2k+2\}\) . Thus we have, as claimed:
(2) Consider now the generating series of the Catalan numbers, \( f(z)=\sum_{k\geq0}C_kz^k\) . In terms of this generating series, the above recurrence gives, as desired:
(3) By solving the equation \( zf^2-f+1=0\) found above, and choosing the solution which is bounded at \( z=0\) , we obtain the following formula, as claimed:
(4) In order to compute this function, we use the generalized binomial formula, which is as follows, with \( p\in\mathbb R\) being an arbitrary exponent, and with \( |t|<1\) :
To be more precise, this formula, which generalizes the usual binomial formula, holds indeed due to the Taylor formula, with the binomial coefficients being given by:
(5) For the exponent \( p=1/2\) , the generalized binomial coefficients are:
(6) Thus the generalized binomial formula at exponent \( p=1/2\) reads:
With \( t=-4z\) we obtain from this the following formula:
(7) Now back to our series \( f\) , we obtain the following formula for it:
(8) Thus the Catalan numbers are given by the formula the statement, namely:
So done, and note in passing that I kept my promise, from the proof of Proposition 13.4. Indeed, with the above final formula, the numerics are easily worked out.
Many other things can be said about the Catalan numbers, as a continuation of the above, and about the central binomial coefficients too. We will be back to this.
According to the above, the problem is now, how to recover a probability measure out of its moments. And the answer here, which is something non-trivial, is as follows:
Theorem 218
The density of a real probability measure \( \mu\) can be recaptured from the sequence of moments \( \{M_k\}_{k\geq0}\) via the Stieltjes inversion formula
where the function on the right, given in terms of moments by
is the Cauchy transform of the measure \( \mu\) .
Proof
The Cauchy transform of our measure \( \mu\) is given by:
Now with \( \xi=x+it\) , we obtain the following formula:
By integrating over \( [a,b]\) we obtain, with the change of variables \( x=y+tz\) :
Now observe that with \( t\searrow0\) we have:
We therefore obtain the following formula:
Thus, we are led to the conclusion in the statement.
Before getting further, let us mention that the above result does not fully solve the moment problem, because we still have the question of understanding when a sequence of numbers \( M_1,M_2,M_3,…\) can be the moments of a measure \( \mu\) . We have here:
Theorem 219
A sequence of numbers \( M_0,M_1,M_2,M_3,…\in\mathbb R\) , with \( M_0=1\) , is the series of moments of a real probability measure \( \mu\) precisely when:
That is, the associated Hankel determinants must be all positive.
Proof
This is something a bit more advanced, the idea being as follows:
(1) As a first observation, the positivity conditions in the statement tell us that the following associated linear forms must be positive:
(2) But this is something very classical, in one sense the result being elementary, coming from the following computation, which shows that we have positivity indeed:
(3) As for the other sense, here the result comes once again from the above formula, this time via some standard functional analysis.
As a basic application of the Stieltjes formula, let us solve the moment problem for the Catalan numbers \( C_k\) , and for the central binomial coefficients \( D_k\) . We first have:
Theorem 220
The real measure having as even moments the Catalan numbers, \( C_k=\frac{1}{k+1}\binom{2k}{k}\) , and having all odd moments \( 0\) is the measure
called Wigner semicircle law on \( [-2,2]\) .
Proof
In order to apply the inversion formula, our starting point will be the formula from Theorem 13.7 for the generating series of the Catalan numbers, namely:
By using this formula with \( z=\xi^{-2}\) , we obtain the following formula:
Now let us apply Theorem 13.8. The study here goes as follows:
(1) According to the general philosophy of the Stieltjes formula, the first term, namely \( \xi/2\) , which is “trivial”, will not contribute to the density.
(2) As for the second term, which is something non-trivial, this will contribute to the density, the rule here being that the square root \( \sqrt{\xi^2-4}\) will be replaced by the “dual” square root \( \sqrt{4-x^2}\,dx\) , and that we have to multiply everything by \( -1/\pi\) .
(3) As a conclusion, by Stieltjes inversion we obtain the following density:
Thus, we have obtained the mesure in the statement, and we are done.
We have the following version of the above result:
Theorem 221
The real measure having as sequence of moments the Catalan numbers, \( C_k=\frac{1}{k+1}\binom{2k}{k}\) , is the measure
called Marchenko-Pastur law on \( [0,4]\) .
Proof
As before, we use the standard formula for the generating series of the Catalan numbers. With \( z=\xi^{-1}\) in that formula, we obtain the following formula:
With this in hand, let us apply now the Stieltjes inversion formula, from Theorem 13.8. We obtain, a bit as before in Theorem 13.10, the following density:
Thus, we are led to the conclusion in the statement.
Regarding now the central binomial coefficients, we have here:
Theorem 222
The real probability measure having as moments the central binomial coefficients, \( D_k=\binom{2k}{k}\) , is the measure
called arcsine law on \( [0,4]\) .
Proof
We have the following computation, using some standard formulae:
But this gives the density in the statement, via Theorem 13.8.
Finally, we have the following version of the above result:
Theorem 223
The real probability measure having as moments the middle binomial coefficients, \( E_k=\binom{k}{[k/2]}\) , is the following law on \( [-2,2]\) ,
called modified arcsine law on \( [-2,2]\) .
Proof
In terms of the central binomial coefficients \( D_k\) , we have:
Standard calculus based on the Taylor formula for \( (1+t)^{-1/2}\) gives:
With \( x=\xi^{-1}\) we obtain the following formula for the Cauchy transform:
By Stieltjes inversion we obtain the density in the statement.
All this is very nice, and we are obviously building here, as this book goes by, some solid knowledge in classical probability. We will be back to all this later.
With the above done, we can come back now to walks on finite graphs, that we know from the above to be related to the eigenvalues of the adjacency matrix \( d\in M_N(0,1)\) . But here, we are led to the following philosophical question, to start with:
Question 11
What are the most important finite graphs, that we should do our computations for?
Not an easy question, you have to agree with me, with the answer to this obviously depending on your previous experience with mathematics, or physics, or chemistry, or computer science, or other branch of science that you are interested in, and also, on the specific problems that you are the most in love with, in that part of science.
So, we have to be subjective here. And with me writing this book, and doing some sort of complicated quantum physics, as daytime job, I will choose the ADE graphs. It is beyond our scope here to explain where these ADE graphs exactly come from, and what they are good for, but as a piece of advertisement for them, we have:
Fact 13
The ADE graphs classify the following:
Which sounds exciting, doesn’t it. So, have a look at this, and with the comment that some heavy learning work is needed, in order to understand how all this works. And with the extra comment that, in view of (7), tough physics, no one really understands how all this works. A nice introduction to all this is the book of Jones [17].
Getting to work now, we first need to know what the ADE graphs are. The A graphs, which are the simplest, are as follows, with the distinguished vertex being denoted \( \bullet\) , and with \( A_n\) having \( n\geq2\) vertices, and \( \tilde{A}_{2n}\) having \( 2n\geq2\) vertices:
You might probably say, why not stopping here, and doing our unfinished business for the segment and the circle, with whatever new ideas that we might have. Good point, but in answer, these ideas will apply as well, with minimal changes, to the D graphs, which are as follows, with \( D_n\) having \( n\geq3\) vertices, and \( \tilde{D}_n\) having \( n+1\geq5\) vertices:
As a comment here, the labeling conventions for the AD graphs, while very standard, can be a bit confusing. The first graph in each series is by definition as follows:
Finally, there are also a number of exceptional ADE graphs. First we have:
Then, we have extended versions of the above exceptional graphs, as follows:
And good news, according to the general theory, that is all. Getting now to work, we have plenty of graphs here, and the problem that we would like to solve is:
Problem 1
How to count loops on the ADE graphs?
In answer, we know that \( A_\infty\) and \( \tilde{A}_\infty\) are respectively the graphs that we previously called \( \mathbb N\) and \( \mathbb Z\) . So, based on our previous work for these graphs, where the combinatorics naturally led us into generating series, let us formulate the following definition:
Definition 66
The Poincar
specialChar{39}e series of a rooted bipartite graph \( X\) is
where \( L_{2k}\) is the number of \( 2k\) -loops based at the root.
To be more precise, observe that all the above ADE graphs are indeed bipartite. Now the point is that, for a bipartite graph, the loops based at any point must have even length. Thus, in order to study the loops on the ADE graphs, based at the root, we just have to count the above numbers \( L_{2k}\) . And then, considering the generating series \( f(z)\) of these numbers, and calling this Poincar
specialChar{39}e series, is something very standard.
Before getting into computations, let us introduce as well:
Definition 67
The positive spectral measure \( \mu\) of a rooted bipartite graph \( X\) is the real probability measure having the numbers \( L_{2k}\) as moments:
Equivalently, we must have the Stieltjes transform formula
where \( f\) is the Poincaré series of \( X\) .
Here the existence of \( \mu\) , and the fact that this is indeed a positive measure, meaning a measure supported on \( [0,\infty)\) , comes from the following simple fact:
Theorem 224
The positive spectral measure of a rooted bipartite graph \( X\) is given by the following formula, with \( d\) being the adjacency matrix of the graph,
and with the probabilistic computation being with respect to the expectation
with \( <A>\) being the \( (*,*)\) -entry of a matrix \( A\) , where \( *\) is the root.
Proof
With the above conventions, we have the following computation:
But this shows that we have \( \mu=law(d^2)\) , as desired.
The above result shows that computing \( \mu\) might be actually a simpler problem than computing \( f\) , and in practice, this is indeed the case. So, in what follows we will rather forget about loops and Definition 13.17, and use Definition 13.18 instead, with our computations to follow being based on the concrete interpretation from Theorem 13.19.
However, even with this probabilistic trick in our bag, things are not exactly trivial. Let us introduce as well the following notion:
Definition 68
The circular measure \( \varepsilon\) of a rooted bipartite graph \( X\) is given by
where \( \mu\) is the associated positive spectral measure.
To be more precise, we know from Theorem 13.19 that the positive measure \( \mu\) is the spectral measure of a certain positive matrix, \( d^2\geq0\) , and it follows from this, and from basic spectral theory, that this measure is supported by the positive reals:
But then, with this observation in hand, we can define indeed the circular measure \( \varepsilon\) as above, as being the pullback of \( \mu\) via the following map:
As a basic example for this, to start with, assume that \( \mu\) is a discrete measure, supported by \( n\) positive numbers \( x_1<…<x_n\) , with corresponding densities \( p_1,…,p_n\) :
For each \( i\in\{1,…,n\}\) the equation \( (q+q^{-1})^2=x_i\) has then four solutions, that we can denote \( q_i,q_i^{-1},-q_i,-q_i^{-1}\) . And with this notation, we have:
In general, the basic properties of \( \varepsilon\) can be summarized as follows:
Theorem 225
The circular measure has the following properties:
Proof
These results can be deduced from definitions, the idea being that (1-5) are trivial, and that (6) follows from the formula of \( \mu\) from Theorem 13.19.
Getting now to computations, we first have the following result:
Theorem 226
The circular measure of the basic index \( 4\) graph, namely
is the uniform measure on the \( 2n\) -roots of unity.
Proof
Let us identify the vertices of \( X=\tilde{A}_{2n}\) with the group \( \{w^k\}\) formed by the \( 2n\) -th roots of unity in the complex plane, where \( w=e^{\pi i/n}\) . The adjacency matrix of \( X\) acts then on the functions \( f\in C(X)\) in the following way:
But this shows that we have \( d=K+K^{-1}\) , where \( K\) is given by:
Thus we can use Theorem 13.19 and Theorem 13.21 (6), and we get:
But this is the uniform measure on the \( 2n\) -roots of unity, as claimed.
All this is nice, but before going ahead with more computations, let us have a look into subfactor theory, and explain what is behind this. Following Jones [17], we have:
Definition 69
The theta series of a rooted bipartite graph \( X\) is
where \( f\) is the Poincaré series.
The theta series can be written as \( \Theta(q)=\sum a_rq^r\) , and it follows from the above formula, via some simple manipulations, that its coefficients are integers:
When \( X\) is the principal graph of a subfactor of index \( N>4\) , it is known that the numbers \( a_r\) are certain multiplicities associated to the planar algebra inclusion \( TL_N\subset P\) . In particular, the coefficients of the theta series are in this case positive integers:
In relation now with the circular measure, the result here, which is quite similar to the Stieltjes transform formula from Definition 13.18, is as follows:
Theorem 227
We have the Stieltjes transform type formula
where the \( T\) series of a rooted bipartite graph \( X\) is by definition given by
with \( \Theta\) being the associated theta series.
Proof
This follows indeed by applying the change of variables \( q\to (q+q^{-1})^2\) to the fact that \( f\) is the Stieltjes transform of \( \mu\) .
In order to discuss all this more systematically, let us introduce as well:
Definition 70
The series of the form
with \( n_i,m_i\in\mathbb N\) are called cyclotomic.
It is technically convenient to allow as well \( 1+q^n\) factors, to be designated by \( n^+\) symbols in the above writing. For instance we have, by definition:
Also, it is convenient in what follows to use the following notations:
The Poincaré series of the ADE graphs are given by quite complicated formulae. However, the corresponding \( T\) series are all cyclotomic, as follows:
Theorem 228
The \( T\) series of the ADE graphs are as follows:
Proof
These formulae can be obtained by counting loops, and then by making the following change of variables, and factorizing the resulting series:
An alternative proof can be obtained by using planar algebra methods.
Our purpose now will be that of converting the above technical results, regarding the \( T\) series, into some final results, regarding the corresponding circular measures \( \varepsilon\) . In order to formulate our results, we will need some more theory. First, we have:
Definition 71
A cyclotomic measure is a probability measure \( \varepsilon\) on the unit circle, having the following properties:
As a first observation, it follows from Theorem 13.21 and from Theorem 13.26 that the circular measures of the finite ADE graphs are supported by certain roots of unity, hence are cyclotomic. We will be back to this in a moment, with details, and computations.
At the general level now, let us introduce as well the following notion:
Definition 72
The \( T\) series of a cyclotomic measure \( \varepsilon\) is given by
with \( \varepsilon\) being as usual the circular spectral measure.
Observe that this formula is nothing but the one in Theorem 13.24, written now in the other sense. In other words, if the cyclotomic measure \( \varepsilon\) happens to be the circular measure of a rooted bipartite graph, then the \( T\) series as defined above coincides with the \( T\) series as defined before. This is useful for explicit computations.
Good news, with this technology in hand, and with a computation already done, in Theorem 13.22, we are now ready to discuss the circular measures of all ADE graphs.
The idea will be that these measures are all cyclotomic, of level \( \leq 3\) , and can be expressed in terms of the basic polynomial densities of degree \( \leq 6\) , namely:
To be more precise, we have the following final result on the subject, with \( \alpha,\beta,\gamma\) being as above, with \( d_n\) being the uniform measure on the \( 2n\) -th roots of unity, and with \( d_n'=2d_{2n}-d_n\) being the uniform measure on the odd \( 4n\) -roots of unity:
Theorem 229
The circular measures of the ADE graphs are given by:
Proof
This is something which can be proved in three steps, as follows:
(1) For the simplest graph, namely the circle \( \tilde{A}_{2n}\) , we already have the result, from Theorem 13.22, with the proof there being something elementary.
(2) For the other non-exceptional graphs, that is, of type A and D, the same method works, namely direct loop counting, with some matrix tricks.
(3) In general, this follows from the \( T\) series formulae in Theorem 13.26, via some manipulations based on the general conversion formulae given above.
We refer to [16] and the subfactor literature for more on all this. Also, let us point out that all this leads to a more conceptual understanding of what we did before, for the graphs \( \mathbb N\) and \( \mathbb Z\) . Indeed, even for these very basic graphs, using the unit circle and circular measures as above leads to a better understanding of the combinatorics.
This was a rather elementary chapter, and as exercises on this, we have:
Exercise 97
Work out loop count results for various products of graphs.
Exercise 98
Learn more about Catalan numbers, the more, the better.
Exercise 99
Clarify what happens to the atoms, in the Stieltjes formula.
Exercise 100
Learn more about Hankel determinants, and the moment problem.
Exercise 101
Learn about the Wigner law, and its various remarkable properties.
Exercise 102
Learn about the Marchenko-Pastur law, and its various properties.
Exercise 103
Learn as well about the arcsine law, and its various properties.
Exercise 104
Learn about various ADE classifications, in math and physics.
As bonus exercise, learn about subfactors, say from the book of Jones [17].
We have seen so far the foundations and basic results of measure theory and probability. Before stepping into more complicated things, such as random matrices and free probability, we would like to clarify one important question which appeared several times, namely the computation of integrals over the compact groups of unitary matrices \( G\subset U_N\) , and its probabilistic consequences. The precise question that we have in mind is:
Question 12
Given a compact group \( G\subset U_N\) , how to compute the integrals
and then, how to use their formula in order to compute the laws of variables of type
depending on a polynomial \( P\) ? What about the \( N\to\infty\) asymptotics of such laws?
All this is quite subtle, and as a basic illustration for this, we have a fundamental result from chapter 3, stating that for \( G=S_N\) the law of the variable \( \chi=\sum_ig_{ii}\) can be explicitly computed, and becomes Poisson (1) with \( N\to\infty\) . This is something truly remarkable, and it is this kind of result that we would like to systematically have.
We will discuss this in this whole chapter, and later on too. This might seem of course quite long, but believe me, it is worth the effort, because it is quite hard to do any type of advanced probability theory without knowing the answer to Question 14.1. But probably enough advertisement, let us get to work. Following Weyl, we first have:
Definition 73
A unitary representation of a compact group \( G\) is a continuous group morphism into a unitary group
which can be faithful or not. The character of such a representation is the function
where \( Tr\) is the usual, unnormalized trace of the \( N\times N\) matrices.
At the level of examples, most of the compact groups that we met so far, finite or continuous, naturally appear as closed subgroups \( G\subset U_N\) . In this case, the embedding \( G\subset U_N\) is of course a representation, called fundamental representation. In general now, let us first discuss the various operations on the representations. We have here:
Proposition 51
The representations of a compact group \( G\) are subject to:
Proof
The fact that the operations in the statement are indeed well-defined, among morphisms from \( G\) to unitary groups, is indeed clear from definitions.
In relation now with characters, we have the following result:
Proposition 52
We have the following formulae, regarding characters
in relation with the basic operations for the representations.
Proof
All these assertions are elementary, by using the following well-known trace formulae, valid for any square matrices \( V,W\) , and any unitary \( U\) :
Thus, we are led to the formulae in the statement.
Assume now that we are given a closed subgroup \( G\subset U_N\) . By using the above operations, we can construct a whole family of representations of \( G\) , as follows:
Definition 74
Given a closed subgroup \( G\subset U_N\) , its Peter-Weyl representations are the various tensor products between the fundamental representation and its conjugate:
We denote these tensor products \( v^{\otimes k}\) , with \( k=\circ\bullet\bullet\circ…\) being a colored integer, with the colored tensor powers being defined according to the rules
and with the convention that \( v^{\otimes\emptyset}\) is the trivial representation \( 1:G\to U_1\) .
Here are a few examples of such representations, namely those coming from the colored integers of length 2, which will often appear in what follows:
In relation now with characters, we have the following result:
Proposition 53
The characters of the Peter-Weyl representations are given by
with the colored powers being given by \( \chi^\circ=\chi\) , \( \chi^\bullet=\bar{\chi}\) and multiplicativity.
Proof
This follows indeed from the additivity, multiplicativity and conjugation formulae from Proposition 14.4, via the conventions in Definition 14.5.
Getting back now to our motivations, we can see the interest in the above constructions. Indeed, the joint moments of the main character \( \chi=\chi_v\) and its adjoint \( \bar{\chi}=\chi_{\bar{v}}\) are the expectations of the characters of various Peter-Weyl representations:
In order to advance, we must develop some general theory. Let us start with:
Definition 75
Given a compact group \( G\) , and two of its representations,
we define the space of intertwiners between these representations as being
and we use the following conventions:
The terminology here is standard, with Fix, Hom, End standing for fixed points, homomorphisms and endomorphisms. We will see later that irreducible means indecomposable, in a suitable sense. Here are now a few basic results, regarding these spaces:
Proposition 54
The spaces of intertwiners have the following properties:
In abstract terms, we say that the Hom spaces form a tensor \( *\) -category.
Proof
All the formulae in the statement are indeed clear from definitions, via elementary computations. As for the last assertion, this is something coming from (1,2,3). We will be back to tensor categories later on, with more details on this latter fact.
As a main consequence of the above result, we have:
Proposition 55
Given a representation \( v:G\to U_N\) , the linear space
is a \( *\) -algebra, with respect to the usual involution of the matrices.
Proof
By definition, \( End(v)\) is a linear subspace of \( M_N(\mathbb C)\) . We know from Proposition 14.8 (1) that this subspace \( End(v)\) is a subalgebra of \( M_N(\mathbb C)\) , and then we know as well from Proposition 14.8 (3) that this subalgebra is stable under the involution \( *\) . Thus, what we have here is a \( *\) -subalgebra of \( M_N(\mathbb C)\) , as claimed.
In order to exploit the above fact, we will need a basic result from linear algebra, stating that any \( *\) -algebra \( A\subset M_N(\mathbb C)\) decomposes as a direct sum, as follows:
Indeed, let us write the unit \( 1\in A\) as \( 1=p_1+…+p_k\) , with \( p_i\in A\) being central minimal projections. Then each of the spaces \( A_i=p_iAp_i\) is a subalgebra of \( A\) , and we have a decomposition \( A=A_1\oplus…\oplus A_k\) . But since each central projection \( p_i\in A\) was chosen minimal, we have \( A_i\simeq M_{N_i}(\mathbb C)\) , with \( N_i=rank(p_i)\) , as desired.
We can now formulate our first Peter-Weyl type theorem, as follows:
Theorem 230 (Peter-Weyl 1)
Let \( v:G\to U_N\) be a representation, consider the algebra \( A=End(v)\) , and write its unit \( 1=p_1+…+p_k\) as above. We have then
with each \( v_i\) being an irreducible representation, obtained by restricting \( v\) to \( Im(p_i)\) .
Proof
This basically follows from Proposition 14.9, as follows:
(1) We first associate to our representation \( v:G\to U_N\) the corresponding action map on \( \mathbb C^N\) . If a linear subspace \( W\subset\mathbb C^N\) is invariant, the restriction of the action map to \( W\) is an action map too, which must come from a subrepresentation \( w\subset v\) .
(2) Consider now a projection \( p\in End(v)\) . From \( pv=vp\) we obtain that the linear space \( W=Im(p)\) is invariant under \( v\) , and so this space must come from a subrepresentation \( w\subset v\) . It is routine to check that the operation \( p\to w\) maps subprojections to subrepresentations, and minimal projections to irreducible representations.
(3) With these preliminaries in hand, let us decompose the algebra \( End(v)\) as above, by using the decomposition \( 1=p_1+…+p_k\) into central minimal projections. If we denote by \( v_i\subset v\) the subrepresentation coming from the vector space \( V_i=Im(p_i)\) , then we obtain in this way a decomposition \( v=v_1+…+v_k\) , as in the statement.
Here is now our second Peter-Weyl theorem, complementing Theorem 14.10:
Theorem 231 (Peter-Weyl 2)
Given a closed subgroup \( G\subset_vU_N\) , any of its irreducible smooth representations
appears inside a tensor product of the fundamental representation \( v\) and its adjoint \( \bar{v}\) .
Proof
Given a representation \( w:G\to U_M\) , we define the space of coefficients \( C_w\subset C(G)\) of this representation as being the following linear space:
With this notion in hand, the result can be deduced as follows:
(1) The construction \( w\to C_w\) is functorial, in the sense that it maps subrepresentations into linear subspaces. This is indeed something which is routine to check.
(2) A closed subgroup \( G\subset_vU_N\) is a Lie group, and a representation \( w:G\to U_M\) is smooth when we have an inclusion \( C_w\subset<C_v>\) . This is indeed well-known.
(3) By definition of the Peter-Weyl representations, as arbitrary tensor products between the fundamental representation \( v\) and its conjugate \( \bar{v}\) , we have:
(4) Now by putting together the above observations (2,3) we conclude that we must have an inclusion as follows, for certain exponents \( k_1,…,k_p\) :
(5) By using now (1), we deduce that we have an inclusion \( w\subset v^{\otimes k_1}\oplus…\oplus v^{\otimes k_p}\) , and by applying Theorem 14.10, this leads to the conclusion in the statement.
In order to further advance with Peter-Weyl theory, we need to talk about integration over \( G\) . In the finite group case the situation is trivial, as follows:
Proposition 56
Any finite group \( G\) has a unique probability measure which is invariant under left and right translations,
and this is the normalized counting measure on \( G\) , given by \( \mu(E)=|E|/|G|\) .
Proof
This is indeed something trivial, which follows from definitions.
In the general, continuous case, let us begin with the following key result:
Proposition 57
Given a unital positive linear form \( \psi:C(G)\to\mathbb C\) , the limit
exists, and for a coefficient of a representation \( f=(\tau\otimes id)w\) we have
where \( P\) is the orthogonal projection onto the \( 1\) -eigenspace of \( (id\otimes\psi)w\) .
Proof
By linearity it is enough to prove the first assertion for functions of the following type, where \( w\) is a Peter-Weyl representation, and \( \tau\) is a linear form:
Thus we are led into the second assertion, and more precisely we can have the whole result proved if we can establish the following formula, with \( f=(\tau\otimes id)w\) :
In order to prove this latter formula, observe that we have:
Let us set \( M=(id\otimes\psi)w\) . In terms of this matrix, we have:
Thus we have the following formula, valid for any \( k\in\mathbb N\) :
It follows that our Ces\‘aro limit is given by the following formula:
Now since \( w\) is unitary we have \( ||w||=1\) , and so \( ||M||\leq1\) . Thus the last Cesàro limit converges, and equals the orthogonal projection onto the \( 1\) -eigenspace of \( M\) :
Thus our initial Ces\‘aro limit converges as well, to \( \tau(P)\) , as desired.
When the linear form \( \psi\in C(G)^*\) is faithful, we have the following finer result:
Proposition 58
Given a faithful unital linear form \( \psi\in C(G)^*\) , the limit
exists, and is independent of \( \psi\) , given on coefficients of representations by
where \( P\) is the orthogonal projection onto the space \( Fix(w)=\left\{\xi\in\mathbb C^n\big|w\xi=\xi\right\}\) .
Proof
In view of Proposition 14.13, it remains to prove that when \( \psi\) is faithful, the \( 1\) -eigenspace of the matrix \( M=(id\otimes\psi)w\) equals the space \( Fix(w)\) .
“\( \supset\) ” This is clear, and for any \( \psi\) , because we have the following implication:
“\( \subset\) ” Here we must prove that, when \( \psi\) is faithful, we have:
For this purpose, assume that we have \( M\xi=\xi\) , and consider the following function:
We must prove that we have \( f=0\) . Since \( v\) is unitary, we have:
By using now our assumption \( M\xi=\xi\) , we obtain from this:
Now since \( \psi\) is faithful, this gives \( f=0\) , and so \( w\xi=\xi\) , as claimed.
We can now formulate a main result, as follows:
Theorem 232
Any compact group \( G\) has a unique Haar integration, which can be constructed by starting with any faithful positive unital form \( \psi\in C(G)^*\) , and setting:
Moreover, for any representation \( w\) we have the formula
where \( P\) is the orthogonal projection onto \( Fix(w)=\left\{\xi\in\mathbb C^n\big|w\xi=\xi\right\}\) .
Proof
Let us first go back to the general context of Proposition 14.13. Since convolving one more time with \( \psi\) will not change the Ces\‘aro limit appearing there, the functional \( \int_\psi\in C(G)^*\) constructed there has the following invariance property:
In the case where \( \psi\) is assumed to be faithful, as in Proposition 14.14, our claim is that we have the following formula, valid this time for any \( \varphi\in C(G)^*\) :
Indeed, it is enough to prove this formula on a coefficient of a corepresentation:
In order to do so, consider the following two matrices:
We have then the following formulae, which all follow from definitions:
Thus, in order to prove our claim, it is enough to establish the following formula:
But this follows from the fact, from Proposition 14.14, that \( P=(id\otimes\int_\psi)w\) is the orthogonal projection onto \( Fix(w)\) . Thus, we proved our claim. Now observe that, with \( \Delta f(g\otimes h)=f(gh)\) , this formula that we proved can be written as follows:
This formula being true for any \( \varphi\in C(G)^*\) , we can simply delete \( \varphi\) , and we conclude that \( \int_G=\int_\psi\) has the required left and right invariance property, namely:
Finally, the uniqueness is clear as well, because if we have two invariant integrals \( \int_G,\int_G'\) , then their convolution equals on one hand \( \int_G\) , and on the other hand, \( \int_G'\) .
Summarizing, we know how to integrate over \( G\) . Before getting into probabilistic applications, let us develop however more Peter-Weyl theory. We will need:
Proposition 59
We have a Frobenius type isomorphism
valid for any two representations \( v,w\) .
Proof
According to definitions, we have the following equivalences:
On the other hand, we have as well the following equivalences:
With these formulae in hand, both inclusions follow from the unitarity of \( v,w\) .
We can now formulate a third Peter-Weyl theorem, as follows:
Theorem 233 (Peter-Weyl 3)
The dense subalgebra \( \mathcal C(G)\subset C(G)\) generated by the coefficients of the fundamental representation decomposes as a direct sum
with the summands being pairwise orthogonal with respect to the scalar product
where \( \int_G\) is the Haar integration over \( G\) .
Proof
By combining the previous two Peter-Weyl results, Theorems 14.10 and 14.11, we deduce that we have a linear space decomposition as follows:
Thus, in order to conclude, it is enough to prove that for any two irreducible representations \( v,w\in Irr(G)\) , the corresponding spaces of coefficients are orthogonal:
But this follows from Theorem 14.15, via Proposition 14.16. Let us set indeed:
Then \( P\) is the orthogonal projection onto the following vector space:
Thus we have \( P=0\) , and this gives the result.
Finally, we have the following result, completing the Peter-Weyl theory:
Theorem 234 (Peter-Weyl 4)
The characters of irreducible representations belong to the algebra
called algebra of central functions on \( G\) , and form an orthonormal basis of it.
Proof
Observe first that \( \mathcal C(G)_{central}\) is indeed an algebra, which contains all the characters. Conversely, consider a function \( f\in\mathcal C(G)\) , written as follows:
The condition \( f\in\mathcal C(G)_{central}\) states then that for any \( w\in Irr(G)\) , we must have:
But this means that \( f_w\) must be a scalar multiple of \( \chi_w\) , so the characters form a basis of \( \mathcal C(G)_{central}\) , as stated. Also, the fact that we have an orthogonal basis follows from Theorem 14.17. As for the fact that the characters have norm 1, this follows from:
Here we have used the fact, coming from Theorem 14.15 and Proposition 14.16, that the integrals \( \int_Gw_{ij}\bar{w}_{kl}\) form the orthogonal projection onto the following vector space:
Thus, the proof of our theorem is now complete.
In view of the above results, no matter on what we want to do with our group, we must compute the spaces \( Fix(v^{\otimes k})\) . It is technically convenient to slightly enlarge the class of spaces to be computed, by talking about Tannakian categories, as follows:
Definition 76
The Tannakian category associated to a closed subgroup \( G\subset_vU_N\) is the collection \( C_G=(C_G(k,l))\) of vector spaces
where the representations \( v^{\otimes k}\) with \( k=\circ\bullet\bullet\circ…\) colored integer, defined by
and multiplicativity, \( v^{\otimes kl}=v^{\otimes k}\otimes v^{\otimes l}\) , are the Peter-Weyl representations.
Let us make a summary of what we have so far, regarding these spaces \( C_G(k,l)\) . In order to formulate our result, let us start with the following definition:
Definition 77
Let \( H\) be a finite dimensional Hilbert space. A tensor category over \( H\) is a collection \( C=(C(k,l))\) of linear spaces
satisfying the following conditions:
Here the tensor power Hilbert spaces \( H^{\otimes k}\) , with \( k=\circ\bullet\bullet\circ…\) being a colored integer, are defined by the following formulae, and multiplicativity:
With these conventions, we have the following result, summarizing our knowledge on the subject, coming from the results established in the above:
Theorem 235
For a closed subgroup \( G\subset_vU_N\) , the associated Tannakian category
is a tensor category over the Hilbert space \( H=\mathbb C^N\) .
Proof
We know that the fundamental representation \( v\) acts on the Hilbert space \( H=\mathbb C^N\) , and that its conjugate \( \bar{v}\) acts on the Hilbert space \( \bar{H}=\mathbb C^N\) . Now by multiplicativity we conclude that any Peter-Weyl representation \( v^{\otimes k}\) acts on the Hilbert space \( H^{\otimes k}\) , and so that we have embeddings as in Definition 14.20, as follows:
Regarding now the fact that the axioms (1-6) in Definition 14.20 are indeed satisfied, this is something that we basically already know. To be more precise, (1-4) are clear, and (5) follows from the fact that each element \( g\in G\) is a unitary, which gives:
As for (6), this is something trivial, coming from the fact that the matrix coefficients \( g\to g_{ij}\) and their complex conjugates \( g\to\bar{g}_{ij}\) commute with each other.
Our purpose now will be that of showing that any closed subgroup \( G\subset U_N\) is uniquely determined by its Tannakian category \( C_G=(C_G(k,l))\) . This result, known as Tannakian duality, is something quite deep, and extremely useful. Indeed, the idea is that what we would have here is a “linearization” of \( G\) , allowing us to do combinatorics, and to ultimately reach to concrete and powerful results, regarding \( G\) itself. We first have:
Theorem 236
Given a tensor category \( C=(C(k,l))\) over a finite dimensional Hilbert space \( H\simeq\mathbb C^N\) , the following construction,
produces a closed subgroup \( G_C\subset U_N\) .
Proof
This is something elementary, with the fact that the closed subset \( G_C\subset U_N\) constructed in the statement is indeed stable under the multiplication, unit and inversion operation for the unitary matrices \( g\in U_N\) being clear from definitions.
We can now formulate the Tannakian duality result, as follows:
Theorem 237
The above Tannakian constructions
are bijective, and inverse to each other.
Proof
This is something quite technical, obtained by doing some abstract algebra, and for details here, we refer to the Tannakian duality literature. The whole subject is actually, in modern times, for the most part of quantum algebra, and you can consult here various quantum group papers and books, such as [16], for details on the above.
In order to reach now to more concrete things, following Brauer, we have:
Definition 78
Let \( P(k,l)\) be the set of partitions between an upper colored integer \( k\) , and a lower colored integer \( l\) . A collection of subsets
with \( D(k,l)\subset P(k,l)\) is called a category of partitions when it has the following properties:
There are many examples of such categories, as for instance the category of all pairings \( P_2\) , or of all matching pairings \( \mathcal P_2\) . We will be back to examples in a moment.
Let us formulate as well the following definition:
Definition 79
Given a partition \( \pi\in P(k,l)\) and an integer \( N\in\mathbb N\) , we can construct a linear map between tensor powers of \( \mathbb C^N\) ,
by the following formula, with \( e_1,…,e_N\) being the standard basis of \( \mathbb C^N\) ,
and with the coefficients on the right being Kronecker type symbols,
whose values depend on whether the indices fit or not.
To be more precise, we put the indices of \( i,j\) on the legs of \( \pi\) , in the obvious way. In case all the blocks of \( \pi\) contain equal indices of \( i,j\) , we set \( \delta_\pi(^i_j)=1\) . Otherwise, we set \( \delta_\pi(^i_j)=0\) . The relation with the Tannakian categories comes from:
Proposition 60
The assignement \( \pi\to T_\pi\) is categorical, in the sense that
where \( c(\pi,\nu)\) are certain integers, coming from the erased components in the middle.
Proof
This is something elementary, the computations being as follows:
(1) The concatenation axiom can be checked as follows:
(2) The composition axiom can be checked as follows:
(3) Finally, the involution axiom can be checked as follows:
Summarizing, our correspondence is indeed categorical.
In relation now with the groups, we have the following result:
Theorem 238
Each category of partitions \( D=(D(k,l))\) produces a family of compact groups \( G=(G_N)\) , with \( G_N\subset_vU_N\) , via the formula
and the Tannakian duality correspondence.
Proof
Given an integer \( N\in\mathbb N\) , consider the correspondence \( \pi\to T_\pi\) constructed in Definition 14.25, and then the collection of linear spaces in the statement, namely:
According to Proposition 14.26, and to our axioms for the categories of partitions, from Definition 14.24, this collection of spaces \( C=(C(k,l))\) satisfies the axioms for the Tannakian categories, from Definition 14.20. Thus the Tannakian duality result, Theorem 14.23, applies, and provides us with a closed subgroup \( G_N\subset_vU_N\) such that:
Thus, we are led to the conclusion in the statement.
We can now formulate a key definition, as follows:
Definition 80
A closed subgroup \( G\subset_vU_N\) is called easy when we have
for any colored integers \( k,l\) , for a certain category of partitions \( D\subset P\) .
The notion of easiness goes back to the results of Brauer regarding the orthogonal group \( O_N\) , and the unitary group \( U_N\) , which can be formulated as follows:
Theorem 239
We have the following results:
Proof
This is something very standard, the idea being as follows:
(1) The group \( U_N\) being defined via the relations \( v^*=v^{-1}\) , \( v^t=\bar{v}^{-1}\) , the associated Tannakian category is \( C=span(T_\pi|\pi\in D)\) , with:
(2) The group \( O_N\subset U_N\) being defined by imposing the relations \( v_{ij}=\bar{v}_{ij}\) , the associated Tannakian category is \( C=span(T_\pi|\pi\in D)\) , with:
Thus, we are led to the conclusion in the statement.
Beyond this, a first natural question is that of computing the easy group associated to the category \( P\) itself, and we have here the following Brauer type theorem:
Theorem 240
The symmetric group \( S_N\) , regarded as group of unitary matrices,
via the permutation matrices, is easy, coming from the category of all partitions \( P\) .
Proof
Consider the easy group \( G\subset O_N\) coming from the category of all partitions \( P\) . Since \( P\) is generated by the one-block partition \( Y\in P(2,1)\) , we have:
The linear map associated to \( Y\) is given by the following formula:
Thus, the relation defining the above group \( G\subset O_N\) reformulates as follows:
In other words, the elements \( v_{ij}\) must be projections, and these projections must be pairwise orthogonal on the rows of \( v=(v_{ij})\) . We conclude that \( G\subset O_N\) is the subgroup of matrices \( g\in O_N\) having the property \( g_{ij}\in\{0,1\}\) . Thus we have \( G=S_N\) , as claimed.
In fact, we have the following general easiness result, regarding the series of complex reflection groups \( H_N^s\subset U_N\) , that we introduced in chapter 3:
Theorem 241
The group \( H_N^s=\mathbb Z_s\wr S_N\) is easy, the corresponding category \( P^s\) consisting of the partitions satisfying \( \#\circ=\#\bullet(s)\) in each block. In particular:
Proof
This is something that we already know at \( s=1\) , from Theorem 14.30. In general, the proof is similar, based on Tannakian duality. To be more precise, in what regards the main assertion, the idea here is that the one-block partition \( \pi\in P(s)\) , which generates the category \( P^s\) , implements the relations producing the subgroup \( H_N^s\subset U_N\) . As for the last assertions, these follow from the following observations:
(1) At \( s=1\) we know that we have \( H_N^1=S_N\) . Regarding now the corresponding category, here the condition \( \#\circ=\#\bullet(1)\) is automatic, and so \( P^1=P\) .
(2) At \( s=2\) we know that we have \( H_N^2=H_N\) . Regarding now the corresponding category, here the condition \( \#\circ=\#\bullet(2)\) reformulates as follows:
Thus each block must have even size, and we obtain, as claimed, \( P^2=P_{even}\) .
(3) At \( s=\infty\) we know that we have \( H_N^\infty=K_N\) . Regarding now the corresponding category, here the condition \( \#\circ=\#\bullet(\infty)\) reads:
But this is the condition defining \( \mathcal P_{even}\) , and so \( P^\infty=\mathcal P_{even}\) , as claimed.
Let us go back now to probability questions, with the aim of applying the above abstract theory, to questions regarding characters. The situation here is as follows:
(1) Given a closed subgroup \( G\subset_vU_N\) , we know from Peter-Weyl that the moments of the main character count the fixed points of the representations \( v^{\otimes k}\) .
(2) On the other hand, assuming that our group \( G\subset_vU_N\) is easy, coming from a category of partitions \( D=(D(k,l))\) , the space formed by these fixed points is spanned by the following vectors, indexed by partitions \( \pi\) belonging to the set \( D(k)=D(0,k)\) :
(3) Thus, we are left with investigating linear independence questions for the vectors \( \xi_\pi\) , and once these questions solved, to compute the moments of \( \chi\) .
In order to investigate linear independence questions for the vectors \( \xi_\pi\) , we will use the Gram matrix of these vectors. Let us begin with some standard definitions:
Definition 81
Let \( P(k)\) be the set of partitions of \( \{1,…,k\}\) , and let \( \pi,\nu\in P(k)\) .
As an illustration here, at \( k=2\) we have \( P(2)=\{||,\sqcap\}\) , and the order is:
At \( k=3\) we have \( P(3)=\{|||,\sqcap|,\sqcap-3.2mm{ }_|\,,|\sqcap,\sqcap-0.7mm\sqcap\}\) , and the order relation is as follows:
Observe also that we have \( \pi,\nu\leq\pi\vee\nu\) . In fact, \( \pi\vee\nu\) is the smallest partition with this property, called supremum of \( \pi,\nu\) . Now back to the easy groups, we have:
Proposition 61
The Gram matrix \( G_{kN}(\pi,\nu)=<\xi_\pi,\xi_\nu>\) is given by
where \( |.|\) is the number of blocks.
Proof
According to our formula of the vectors \( \xi_\pi\) , we have:
Thus, we have obtained the formula in the statement.
In order to study the Gram matrix, and more specifically to compute its determinant, we will need several standard facts about the partitions. We first have:
Definition 82
The M
specialChar{34}obius function of any lattice, and so of \( P\) , is given by
with the construction being performed by recurrence.
As an illustration here, let us go back to the set of 2-point partitions, \( P(2)=\{||,\sqcap\}\) . Here we have by definition:
Also, we know that we have \( ||<\sqcap\) , with no intermediate partition in between, and so the above recurrence procedure gives the following formular:
Finally, we have \( \sqcap\not\leq||\) , which gives \( \mu(\sqcap,||)=0\) . Thus, as a conclusion, the M
specialChar{34}obius matrix \( M_{\pi\nu}=\mu(\pi,\nu)\) of the lattice \( P(2)=\{||,\sqcap\}\) is as follows:
The interest in the Möbius function comes from the Möbius inversion formula:
In linear algebra terms, the statement and proof of this formula are as follows:
Theorem 242
The inverse of the adjacency matrix of \( P\) , given by
is the M
specialChar{34}obius matrix of \( P\) , given by \( M_{\pi\nu}=\mu(\pi,\nu)\) .
Proof
This is well-known, coming for instance from the fact that \( A\) is upper triangular. Thus, when inverting, we are led into the recurrence from Definition 14.34.
As an illustration here, for \( P(2)\) the formula \( M=A^{-1}\) appears as follows:
Now back to our Gram matrix considerations, we have the following result:
Proposition 62
The Gram matrix is given by \( G_{kN}=AL\) , where
and where \( A=M^{-1}\) is the adjacency matrix of \( P(k)\) .
Proof
We have the following computation:
According to Proposition 14.33 and to the definition of \( A,L\) , this formula reads:
Thus, we obtain the formula in the statement.
With the above result in hand, we can now investigate the linear independence properties of the vectors \( \xi_\pi\) . To be more precise, we have the following result:
Theorem 243
The determinant of the Gram matrix \( G_{kN}\) is given by
and in particular, for \( N\geq k\) , the vectors \( \{\xi_\pi|\pi\in P(k)\}\) are linearly independent.
Proof
According to the formula in Proposition 14.36, we have:
Now if we order \( P(k)\) as usual, with respect to the number of blocks, and then lexicographically, we see that \( A\) is upper triangular, and that \( L\) is lower triangular. Thus \( \det(A)\) can be computed simply by making the product on the diagonal, and we obtain \( 1\) . As for \( \det(L)\) , this can computed as well by making the product on the diagonal, and we obtain the number in the statement, with the technical remark that in the case \( N<k\) the convention is that we obtain a vanishing determinant.
Now back to the laws of characters, we can formulate:
Proposition 63
For an easy group \( G=(G_N)\) , coming from a category of partitions \( D=(D(k,l))\) , the asymptotic moments of the main character are given by
where \( D(k)=D(\emptyset,k)\) , with the limiting sequence on the left consisting of certain integers, and being stationary at least starting from the \( k\) -th term.
Proof
This follows indeed from the Peter-Weyl theory, by using the linear independence result for the vectors \( \xi_\pi\) coming from Theorem 14.37.
With these preliminaries in hand, we can now state and prove:
Theorem 244
In the \( N\to\infty\) limit, the laws of the main character for the main easy groups, real and complex, and discrete and continuous, are as follows,

with these laws, namely the real and complex Gaussian and Bessel laws, being the main limiting laws in real and complex, and discrete and continuous probability.
Proof
This follows from the above results. To be more precise, we know that the above groups are all easy, the corresponding categories of partitions being as follows:

Thus, we can use Proposition 14.38, are we are led into counting partitions, and then recovering the measures via their moments, and this leads to the result.
Our aim now is to go beyond what we have, with results regarding the truncated characters. Let us start with a general formula coming from Peter-Weyl, namely:
Theorem 245
The Haar integration over a closed subgroup \( G\subset_vU_N\) is given on the dense subalgebra of smooth functions by the Weingarten type formula
valid for any colored integer \( k=e_1… e_k\) and any multi-indices \( i,j\) , where \( D(k)\) is a linear basis of \( Fix(v^{\otimes k})\) , the associated generalized Kronecker symbols are given by
and \( W_k=G_k^{-1}\) is the inverse of the Gram matrix, \( G_k(\pi,\nu)=<\pi,\nu>\) .
Proof
This is something very standard, coming from the fact that the above integrals form altogether the orthogonal projection \( P^k\) onto the following space:
Consider now the following linear map, with \( D(k)=\{\xi_k\}\) being as in the statement:
By a standard linear algebra computation, it follows that we have \( P=WE\) , where \( W\) is the inverse of the restriction of \( E\) to the following space:
But this restriction is the linear map given by the matrix \( G_k\) , and so \( W\) is the linear map given by the inverse matrix \( W_k=G_k^{-1}\) , and this gives the result.
In the easy case, we have the following more concrete result:
Theorem 246
For an easy group \( G\subset U_N\) , coming from a category of partitions \( D=(D(k,l))\) , we have the Weingarten formula
for any \( k=e_1… e_k\) and any \( i,j\) , where \( D(k)=D(\emptyset,k)\) , \( \delta\) are usual Kronecker type symbols, checking whether the indices match, and \( W_{kN}=G_{kN}^{-1}\) , with
where \( |.|\) is the number of blocks.
Proof
We use the abstract Weingarten formula, from Theorem 14.40. Indeed, the Kronecker type symbols there are then the usual ones, as shown by:
The Gram matrix being as well the correct one, we obtain the result.
Let us go back now to the general easy groups \( G\subset U_N\) , with the idea in mind of computing the laws of truncated characters. First, we have the following formula:
Proposition 64
The moments of truncated characters are given by the formula
where \( G_{kN}\) and \( W_{kN}=G_{kN}^{-1}\) are the associated Gram and Weingarten matrices.
Proof
We have indeed the following computation:
Thus, we have reached to the formula in the statement.
In order to process now the above formula, and reach to concrete results, we must impose on our group a uniformity condition. Let us start with:
Proposition 65
For an easy group \( G=(G_N)\) , coming from a category of partitions \( D\subset P\) , the following conditions are equivalent:
If these conditions are satisfied, we say that \( G=(G_N)\) is uniform.
Proof
The equivalence \( (1)\iff(2)\) comes from the inclusion \( S_N\subset G_N\) , which makes everything \( S_N\) -invariant. Regarding \( (1)\iff(3)\) , given a subgroup \( K\subset_vU_{N-1}\) , consider the matrix \( u=diag(v,1)\) . Our claim is that for any \( \pi\in P(k)\) we have:
In order to prove this claim, we must study the condition on the left. We have:
Now let us recall that our representation has the special form \( u=diag(v,1)\) . We conclude from this that for any index \( a\in\{1,…,k\}\) , we have:
With this observation in hand, if we denote by \( i',j'\) the multi-indices obtained from \( i,j\) obtained by erasing all the above \( i_a=j_a=N\) values, and by \( k'\leq k\) the common length of these new multi-indices, our condition becomes:
Here the index \( j\) is by definition obtained from the index \( j'\) by filling with \( N\) values. In order to finish now, we have two cases, depending on \( i\) , as follows:
Case 1. Assume that the index set \( \{a|i_a=N\}\) corresponds to a certain subpartition \( \pi'\subset\pi\) . In this case, the \( N\) values will not matter, and our formula becomes:
Case 2. Assume now the opposite, namely that the set \( \{a|i_a=N\}\) does not correspond to a subpartition \( \pi'\subset\pi\) . In this case the indices mix, and our formula reads \( 0=0\) . Thus we have \( \xi_{\pi'}\in Fix(u^{\otimes k'})\) in both cases, for any subpartition \( \pi'\subset\pi\) , as desired.
Now back to the laws of truncated characters, we have the following result:
Theorem 247
For a uniform easy group \( G=(G_N)\) , we have the formula
with \( D\subset P\) being the associated category of partitions.
Proof
We use Proposition 14.42. With \( s=[tN]\) , the formula there becomes:
The point now is that in the uniform case the Gram matrix, and so the Weingarten matrix too, is asymptotically diagonal. Thus, we obtain the following estimate:
Thus, we are led to the formula in the statement.
We can now enlarge our collection of truncated character results, and we have:
Theorem 248
With \( N\to\infty\) , the laws of truncated characters are as follows:
Proof
We already know these results at \( t=1\) . In the general case, \( t>0\) , these follow via some standard combinatorics, from the formula in Theorem 14.44.
This was a technical and exciting chapter, and as exercises on this, we have:
Exercise 105
Learn the decomposition result for the \( *\) -algebras \( A\subset M_N(\mathbb C)\) .
Exercise 106
Learn more about central functions, on finite and compact groups.
Exercise 107
Find and read one of the available proofs of Tannakian duality.
Exercise 108
Learn as well about the theorems of Deligne, and Doplicher-Roberts.
Exercise 109
Work out all the details for the easiness property of \( H_N^s\) .
Exercise 110
Compute the Gram determinants for other easy groups.
Exercise 111
Work out all the details for our main character computations.
Exercise 112
Work out too the details for our truncated character computations.
As bonus exercise, learn as well about Haar integration over locally compact groups.
Welcome to the random matrices, which are first class mathematics and physics. In order to talk about such matrices and their spectral measures, we need to do some more linear algebra in infinite dimensions, as a continuation of our operator theory discussion from chapters 10 and 12. It is convenient to upgrade our formalism, as follows:
Definition 83
An abstract operator algebra, or \( C^*\) -algebra, is a complex algebra \( A\) having a norm \( ||.||\) and an involution \( *\) , subject to the following conditions:
In other words, what we did here is to axiomatize the abstract properties of the operator algebras \( A\subset B(H)\) , coming from the various general results about linear operators from chapter 10, without any reference to the ambient Hilbert space \( H\) .
As basic examples here, we have the usual matrix algebras \( M_N(\mathbb C)\) , with the norm and the involution being the usual matrix norm and involution, given by:
Some other basic examples are the algebras \( L^\infty(X)\) of essentially bounded functions \( f:X\to\mathbb C\) on a measured space \( X\) , with the usual norm and involution, namely:
We can put these two basic classes of examples together, as follows:
Proposition 66
The random matrix algebras \( A=M_N(L^\infty(X))\) are \( C^*\) -algebras, with their usual norm and involution, given by:
These algebras generalize both the algebras \( M_N(\mathbb C)\) , and the algebras \( L^\infty(X)\) .
Proof
The fact that the \( C^*\) -algebra axioms are satisfied is clear from definitions. As for the last assertion, this follows by taking \( X=\{.\}\) and \( N=1\) , respectively.
We can in fact say more about the above algebras, as follows:
Theorem 249
Any algebra of type \( L^\infty(X)\) is an operator algebra, as follows:
More generally, any random matrix algebra is an operator algebra, as follows,
with the embedding being the above one, tensored with the identity.
Proof
We have two assertions to be proved, the idea being as follows:
(1) Given \( f\in L^\infty(X)\) , consider the following operator, acting on \( H=L^2(X)\) :
Observe that \( T_f\) is indeed well-defined, and bounded as well, because:
The application \( f\to T_f\) being linear, involutive, continuous, and injective as well, we obtain in this way a \( C^*\) -algebra embedding \( L^\infty(X)\subset B(H)\) , as desired.
(2) Regarding the second assertion, this is best viewed in the following way:
Here we have used (1), and some standard tensor product identifications.
Our purpose in what follows is to develop the spectral theory of the \( C^*\) -algebras, and in particular that of the random matrix algebras \( A=M_N(L^\infty(X))\) that we are interested in, one of our objectives being that of talking about spectral measures, in the normal case, in analogy with what we know about the usual matrices. Let us start with:
Theorem 250
Given an element \( a\in A\) of a \( C^*\) -algebra, define its spectrum as:
The following spectral theory results hold, exactly as in the \( A=B(H)\) case:
In addition, assuming \( a\in A\subset B\) , the spectra of \( a\) with respect to \( A\) and to \( B\) coincide.
Proof
Here the assertions (1-5), which are of course formulated a bit informally, are well-known for the full operator algebra \( A=B(H)\) , and the proof in general is similar:
(1) Assuming that \( 1-ab\) is invertible, with inverse \( c\) , we have \( abc=cab=c-1\) , and it follows that \( 1-ba\) is invertible too, with inverse \( 1+bca\) . Thus \( \sigma(ab),\sigma(ba)\) agree on \( 1\in\mathbb C\) , and by linearity, it follows that \( \sigma(ab),\sigma(ba)\) agree on any point \( \lambda\in\mathbb C^*\) .
(2) The formula \( \sigma(f(a))=f(\sigma(a))\) is clear for polynomials, \( f\in\mathbb C[X]\) , by factorizing \( f-\lambda\) , with \( \lambda\in\mathbb C\) . Then, the extension to the rational functions is straightforward, because \( P(a)/Q(a)-\lambda\) is invertible precisely when \( P(a)-\lambda Q(a)\) is.
(3) By using \( 1/(1-b)=1+b+b^2+…\) for \( ||b||<1\) we obtain that \( a-\lambda\) is invertible for \( |\lambda|>||a||\) , and so \( \sigma(a)\subset D_0(||a||)\) . It is also clear that \( \sigma(a)\) is closed, so what we have is a compact set. Finally, assuming \( \sigma(a)=\emptyset\) the function \( f(\lambda)=\varphi((a-\lambda)^{-1})\) is well-defined, for any \( \varphi\in A^*\) , and by Liouville we get \( f=0\) , contradiction.
(4) Assuming \( u^*=u^{-1}\) we have \( ||u||=1\) , and so \( \sigma(u)\subset D_0(1)\) . But with \( f(z)=z^{-1}\) we obtain via (2) that we have as well \( \sigma(u)\subset f(D_0(1))\) , and this gives \( \sigma(u)\subset\mathbb T\) . As for the result regarding the self-adjoints, this can be obtained from the result for the unitaries, by using (2) with functions of type \( f(z)=(z+it)/(z-it)\) , with \( t\in\mathbb R\) .
(5) It is routine to check, by integrating quantities of type \( z^n/(z-a)\) over circles centered at the origin, and estimating, that the spectral radius is given by \( \rho(a)=\lim||a^n||^{1/n}\) . But in the self-adjoint case, \( a=a^*\) , this gives \( \rho(a)=||a||\) , by using exponents of type \( n=2^k\) , and then the extension to the general normal case is straightforward.
(6) Regarding now the last assertion, the inclusion \( \sigma_B(a)\subset\sigma_A(a)\) is clear. For the converse, assume \( a-\lambda\in B^{-1}\) , and set \( b=(a-\lambda )^*(a-\lambda )\) . We have then:
Thus this difference in an open subset of \( \mathbb C\) . On the other hand \( b\) being self-adjoint, its two spectra are both real, and so is their difference. Thus the two spectra of \( b\) are equal, and in particular \( b\) is invertible in \( A\) , and so \( a-\lambda\in A^{-1}\) , as desired.
We can now a prove a key result, as follows:
Theorem 251 (Gelfand)
If \( X\) is a compact space, the algebra \( C(X)\) of continuous functions on it \( f:X\to\mathbb C\) is a \( C^*\) -algebra, with usual norm and involution, namely:
Conversely, any commutative \( C^*\) -algebra is of this form, \( A=C(X)\) , with
with topology making continuous the evaluation maps \( ev_a:\chi\to\chi(a)\) .
Proof
There are several things going on here, the idea being as follows:
(1) The first assertion is clear from definitions. Observe that we have indeed:
Observe also that the algebra \( C(X)\) is commutative, because \( fg=gf\) .
(2) Conversely, given a commutative \( C^*\) -algebra \( A\) , let us define \( X\) as in the statement. Then \( X\) is compact, and \( a\to ev_a\) is a morphism of algebras, as follows:
(3) We first prove that \( ev\) is involutive. We use the following formula, which is similar to the \( z=Re(z)+iIm(z)\) decomposition formula for usual complex numbers:
Thus it is enough to prove \( ev_{a^*}=ev_a^*\) for the self-adjoint elements \( a\) . But this is the same as proving that \( a=a^*\) implies that \( ev_a\) is a real function, which is in turn true, by Theorem 15.4, because \( ev_a(\chi)=\chi(a)\) is an element of \( \sigma(a)\) , contained in \( \mathbb R\) .
(4) Since \( A\) is commutative, each element is normal, so \( ev\) is isometric:
It remains to prove that \( ev\) is surjective. But this follows from the Stone-Weierstrass theorem, because \( ev(A)\) is a closed subalgebra of \( C(X)\) , which separates the points.
As a main consequence of the Gelfand theorem, we have:
Theorem 252
For any normal element \( a\in A\) we have an identification as follows:
In addition, given a function \( f\in C(\sigma(a))\) , we can apply it to \( a\) , and we have
which generalizes the previous rational calculus formula, in the normal case.
Proof
Since \( a\) is normal, the \( C^*\) -algebra \( <a>\) that is generates is commutative, so if we denote by \( X\) the space of the characters \( \chi:<a>\to\mathbb C\) , we have:
Now since the map \( X\to\sigma(a)\) given by evaluation at \( a\) is bijective, we obtain:
Thus, we are dealing here with usual functions, and this gives all the assertions.
In order to get now towards noncommutative probability, we first have to develop the theory of positive elements, and linear forms. First, we have the following result:
Proposition 67
For an element \( a\in A\) , the following are equivalent:
Proof
This is something very standard, as follows:
\( (1)\implies(2)\) Observe first that \( \sigma(a)\subset\mathbb R\) implies \( a=a^*\) . Thus the algebra \( <a>\) is commutative, and by using Theorem 15.6, we can set \( b=\sqrt{a}\) .
\( (2)\implies(3)\) This is trivial, because we can simply set \( c=b\) .
\( (2)\implies(1)\) This is clear too, because we have:
\( (3)\implies(1)\) We proceed by contradiction. By multiplying \( c\) by a suitable element of \( <cc^*>\) , we are led to the existence of an element \( d\neq0\) satisfying:
By writing now \( d=x+iy\) with \( x=x^*,y=y^*\) we have:
Thus \( d^*d\geq0\) , which is easily seen to contradict the condition \( -dd^*\geq0\) .
We can talk as well about positive linear forms, as follows:
Definition 84
Consider a linear map \( \varphi:A\to\mathbb C\) .
In the commutative case, \( A=C(X)\) , the positive linear forms appear as follows, with \( \mu\) being positive, and strictly positive if we want \( \varphi\) to be faithful and positive:
In general, the positive linear forms can be thought of as being integration functionals with respect to some underlying “positive measures”. We have:
Definition 85
Let \( A\) be a \( C^*\) -algebra, given with a positive trace \( tr:A\to\mathbb C\) .
Here the exponent \( k=\circ\bullet\bullet\circ…\) is by definition a colored integer, and the powers \( a^k\) are defined by the following formulae, and multiplicativity:
As for the polynomial \( P\) , this is a noncommuting \( *\) -polynomial in one variable:
Observe that the law is uniquely determined by the moments, because we have:
At the level of the general theory, we have the following key result, extending the various results that we have, regarding the self-adjoint and normal matrices:
Theorem 253
Let \( A\) be a \( C^*\) -algebra, with a trace \( tr\) , and consider an element \( a\in A\) which is normal, in the sense that \( aa^*=a^*a\) .
Proof
This is something very standard, that we already know for the usual complex matrices, and whose proof in general is quite similar, as follows:
(1) In the normal case, \( aa^*=a^*a\) , the Gelfand theorem, or rather the subsequent continuous functional calculus theorem, tells us that we have:
Thus the functional \( f(a)\to tr(f(a))\) can be regarded as an integration functional on the algebra \( C(\sigma(a))\) , and by the Riesz theorem, this latter functional must come from a probability measure \( \mu\) on the spectrum \( \sigma(a)\) , in the sense that we must have:
We are therefore led to the conclusions in the statement, with the uniqueness assertion coming from the fact that the elements \( a^k\) , taken as usual with respect to colored integer exponents, \( k=\circ\bullet\bullet\circ…\) , generate the whole \( C^*\) -algebra \( C(\sigma(a))\) .
(2) This is something which is clear from definitions.
(3) Once again, this is something which is clear from definitions.
As a first concrete application now, by getting back to the random matrices, and to the various questions raised in the beginning of this chapter, we have:
Theorem 254
Given a random matrix \( Z\in M_N(L^\infty(X))\) which is normal,
its law, which is by definition the following abstract functional,
when restricted to the usual polynomials in two variables,
must come from a probability measure on the spectrum \( \sigma(Z)\subset\mathbb C\) , as follows:
We agree to use the symbol \( \mu\) for all these notions.
Proof
This follows indeed from what we know from Theorem 15.10, applied to the normal element \( a=Z\) , belonging to the \( C^*\) -algebra \( A=M_N(L^\infty(X))\) .
We have now all the needed ingredients for launching some explicit random matrix computations. Our goal will be that of computing the asymptotic moments, and then the asymptotic laws, with \( N\to\infty\) , for the main classes of large random matrices.
Let us begin by specifying the precise classes of matrices that we are interested in. First we have the complex Gaussian matrices, which are constructed as follows:
Definition 86
A complex Gaussian matrix is a random matrix of type
which has i.i.d. centered complex normal entries.
To be more precise, the assumption in this definition is that all the matrix entries \( Z_{ij}\) are independent, and follow the same complex normal law \( G_t\) , for a fixed value of \( t>0\) . We will see that the above matrices have an interesting, and “central” combinatorics, among all kinds of random matrices, with the study of the other random matrices being usually obtained as a modification of the study of the Gaussian matrices.
As a somewhat surprising remark, using real normal variables in Definition 15.12, instead of the complex ones appearing there, leads nowhere. The correct real versions of the Gaussian matrices are the Wigner random matrices, constructed as follows:
Definition 87
A Wigner matrix is a random matrix of type
which has i.i.d. centered complex normal entries, up to the constraint \( Z=Z^*\) .
This definition is something a bit compacted, and to be more precise, a Wigner matrix is by definition a random matrix as follows, with the diagonal entries being real normal variables, \( a_i\sim g_t\) , for some \( t>0\) , the upper diagonal entries being complex normal variables, \( b_{ij}\sim G_t\) , the lower diagonal entries being the conjugates of the upper diagonal entries, as indicated, and with all the variables \( a_i,b_{ij}\) being independent:
As a comment here, for many concrete applications the Wigner matrices are in fact the central objects in random matrix theory, and in particular, they are often more important than the Gaussian matrices. In fact, these are the random matrices which were first considered and investigated, a long time ago, by Wigner himself.
However, as we will soon discover, the Gaussian matrices are somehow more fundamental than the Wigner matrices, at least from an abstract point of view, and this will be the point of view that we will follow here, with the Gaussian matrices coming first.
Finally, we will be interested as well in the complex Wishart matrices, which are the positive versions of the above random matrices, constructed as follows:
Definition 88
A complex Wishart matrix is a random matrix of type
with \( Y\) being a complex Gaussian matrix.
As before with the Gaussian and Wigner matrices, there are many possible comments that can be made here, of technical or historical nature. As a first key fact, using real Gaussian variables instead of complex ones leads to a less interesting combinatorics. Also, these matrices were introduced and studied by Marchenko and Pastur not long after Wigner, and so historically came second. Finally, in what regards their combinatorics and applications, these matrices quite often come first, before both the Gaussian and the Wigner ones, with all this being of course a matter of knowledge and taste.
Summarizing, we have three main types of random matrices, which can be thought of as being “complex”, “real” and “positive”, and that we will study in what follows, in this precise order, with this order being the one that fits us best here. Let us also mention that there are many other interesting classes of random matrices, which are more specialized, usually appearing as modifications of the above. More on these later.
In order to compute the asymptotic laws of the Gaussian, Wigner and Wishart matrices, we use the moment method. We first have the following result:
Theorem 255
Given a sequence of Gaussian random matrices
having independent \( G_t\) variables as entries, for some fixed \( t>0\) , we have
for any colored integer \( k=\circ\bullet\bullet\circ…\,\) , in the \( N\to\infty\) limit.
Proof
This is something standard, which can be done as follows:
(1) We fix \( N\in\mathbb N\) , and we let \( Z=Z_N\) . Let us first compute the trace of \( Z^k\) . With \( k=k_1… k_s\) , and with the convention \( (ij)^\circ=ij,(ij)^\bullet=ji\) , we have:
(2) Next, we rescale our variable \( Z\) by a \( \sqrt{N}\) factor, as in the statement, and we also replace the usual trace by its normalized version, \( tr=Tr/N\) . Our formula becomes:
Thus, the moment that we are interested in is given by:
(3) Let us apply now the Wick formula, that we know since chapter 4. We conclude that the moment that we are interested in is given by the following formula:
(4) Our claim now is that in the \( N\to\infty\) limit the combinatorics of the above sum simplifies, with only the noncrossing partitions contributing to the sum, and with each of them contributing precisely with a 1 factor, so that we will have, as desired:
(5) In order to prove this, the first observation is that when \( k\) is not uniform, in the sense that it contains a different number of \( \circ\) , \( \bullet\) symbols, we have \( \mathcal P_2(k)=\emptyset\) , and so:
(6) Thus, we are left with the case where \( k\) is uniform. Let us examine first the case where \( k\) consists of an alternating sequence of \( \circ\) and \( \bullet\) symbols, as follows:
In this case it is convenient to relabel our multi-index \( i=(i_1,…,i_s)\) , with \( s=2p\) , in the form \( (j_1,l_1,j_2,l_2,…,j_p,l_p)\) . With this done, our moment formula becomes:
Now observe that, with \( k\) being as above, we have an identification \( \mathcal P_2(k)\simeq S_p\) , obtained in the obvious way. With this done too, our moment formula becomes:
(7) We are now ready to do our asymptotic study, and prove the claim in (4). Let indeed \( \gamma\in S_p\) be the full cycle, which is by definition the following permutation:
In terms of \( \gamma\) , the conditions \( j_r=j_{\pi(r)+1}\) and \( l_r=l_{\pi(r)}\) found above read:
Counting the number of free parameters in our moment formula, we obtain:
(8) The point now is that the last exponent is well-known to be \( \leq 0\) , with equality precisely when the permutation \( \pi\in S_p\) is geodesic, which in practice means that \( \pi\) must come from a noncrossing partition. Thus we obtain, in the \( N\to\infty\) limit, as desired:
This finishes the proof in the case of the exponents \( k\) which are alternating, and the case where \( k\) is an arbitrary uniform exponent is similar, by permuting everything.
The above result is very nice, but the resulting asymptotic measure is still in need to be interpreted. For more on all this, we refer to free probability theory [18].
Regarding now the Wigner matrices, we have here the following result, coming as a consequence of Theorem 15.15, via some simple algebraic manipulations:
Theorem 256
Given a sequence of Wigner random matrices
having independent \( G_t\) variables as entries, with \( t>0\) , up to \( Z_N=Z_N^*\) , we have
for any integer \( k\in\mathbb N\) , in the \( N\to\infty\) limit.
Proof
This can be deduced from a direct computation based on the Wick formula, similar to that from the proof of Theorem 15.15, but the best is to deduce this result from Theorem 15.15 itself. Indeed, we know from there that for Gaussian matrices \( Y_N\in M_N(L^\infty(X))\) we have the following formula, valid for any colored integer \( K=\circ\bullet\bullet\circ…\,\) , in the \( N\to\infty\) limit, with \( \mathcal{NC}_2\) standing for noncrossing matching pairings:
By doing some combinatorics, we deduce from this that we have the following formula for the moments of the matrices \( Re(Y_N)\) , with respect to usual exponents, \( k\in\mathbb N\) :
Now since the matrices \( Z_N=\sqrt{2}Re(Y_N)\) are of Wigner type, this gives the result.
Now by putting everything together, we obtain the Wigner theorem, as follows:
Theorem 257
Given a sequence of Wigner random matrices
which by definition have i.i.d. complex normal entries, up to \( Z_N=Z_N^*\) , we have
in the \( N\to\infty\) limit, where \( \gamma_t=\frac{1}{2\pi t}\sqrt{4t-x^2}dx\) is the Wigner semicircle law.
Proof
This follows indeed from Theorem 15.16, via some combinatorics, that we know from chapter 13, in order to recover the Wigner law, out of the Catalan numbers.
Let us discuss now the Wishart matrices, which are the positive analogues of the Wigner matrices. Quite surprisingly, the computation here leads to the Catalan numbers, but not in the same way as for the Wigner matrices, the result being as follows:
Theorem 258
Given a sequence of complex Wishart matrices
with \( Y_N\) being \( N\times N\) complex Gaussian of parameter \( t>0\) , we have
for any exponent \( k\in\mathbb N\) , in the \( N\to\infty\) limit.
Proof
There are several possible proofs for this result, as follows:
(1) A first method is by using the formula that we have in Theorem 15.15, for the Gaussian matrices \( Y_N\) . Indeed, we know from there that we have the following formula, valid for any colored integer \( K=\circ\bullet\bullet\circ…\,\) , in the \( N\to\infty\) limit:
With \( K=\circ\bullet\circ\bullet…\,\) , alternating word of lenght \( 2k\) , with \( k\in\mathbb N\) , this gives:
Thus, in terms of the Wishart matrix \( W_N=Y_NY_N^*\) we have, for any \( k\in\mathbb N\) :
The point now is that, by doing some combinatorics, we have:
Thus, we are led to the formula in the statement.
(2) A second method, that we will explain now as well, is by proving the result directly, starting from definitions. The matrix entries of our matrix \( W=W_N\) are given by:
Thus, the normalized traces of powers of \( W\) are given by the following formula:
By rescaling now \( W\) by a \( 1/N\) factor, as in the statement, we obtain:
By using now the Wick rule, we obtain the following formula for the moments, with \( K=\circ\bullet\circ\bullet…\,\) , alternating word of lenght \( 2k\) , and with \( I=(i_1r_1,i_2r_1,…,i_kr_k,i_1r_k)\) :
In order to compute this quantity, we use the standard bijection \( \mathcal P_2(K)\simeq S_k\) . By identifying the pairings \( \pi\in\mathcal P_2(K)\) with their counterparts \( \pi\in S_k\) , we obtain:
Now let \( \gamma\in S_k\) be the full cycle, which is by definition the following permutation:
The general factor in the product computed above is then 1 precisely when following two conditions are simultaneously satisfied:
Counting the number of free parameters in our moment formula, we obtain:
The point now is that the last exponent is well-known to be \( \leq 0\) , with equality precisely when the permutation \( \pi\in S_k\) is geodesic, which in practice means that \( \pi\) must come from a noncrossing partition. Thus we obtain, in the \( N\to\infty\) limit:
Thus, we are led to the conclusion in the statement.
We are led in this way to the following result:
Theorem 259
Given a sequence of complex Wishart matrices
with \( Y_N\) being \( N\times N\) complex Gaussian of parameter \( t>0\) , we have
with \( N\to\infty\) , with the limiting measure being the Marchenko-Pastur law \( \pi_1\) .
Proof
This follows indeed from Theorem 15.18, via the combinatorics from chapter 13, in order to recover the Marchenko-Pastur law, out of the Catalan numbers.
Let us discuss now a generalization of the above results, motivated by a whole array of concrete questions, and bringing into the picture a “true” parameter \( t>0\) , which is different from the parameter \( t>0\) used above, which is something quite trivial.
For this purpose, let us go back to the definition of the Wishart matrices. There were as follows, with \( Y\) being a \( N\times N\) matrix with i.i.d. entries, each following the law \( G_t\) :
The point now is that, more generally, we can use in this construction a \( N\times M\) matrix \( Y\) with i.i.d. entries, each following the law \( G_t\) , with \( M\in\mathbb N\) being arbitrary. Thus, we have a new parameter, and by ditching the old parameter \( t>0\) , we are led to the following definition, which is the “true” definition of the Wishart matrices:
Definition 89
A complex Wishart matrix is a \( N\times N\) matrix of the form
where \( Y\) is a \( N\times M\) matrix with i.i.d. entries, each following the law \( G_1\) .
In order to see now what is going on, combinatorially, let us compute moments. The result here is substantially more interesting than that for the previous Wishart matrices, with the new revelant numeric parameter being now the number \( t=M/N\) , as follows:
Theorem 260
Given a sequence of complex Wishart matrices
with \( Y_N\) being \( N\times M\) complex Gaussian of parameter \( 1\) , we have
for any exponent \( k\in\mathbb N\) , in the \( M=tN\to\infty\) limit.
Proof
This is something which is very standard, as follows:
(1) Before starting, let us clarify the relation with our previous Wishart matrix results. In the case \( M=N\) we have \( t=1\) , and the formula in the statement reads:
Thus, what we have here is the previous Wishart matrix formula, in full generality, at the value \( t=1\) of our old parameter \( t>0\) .
(2) Observe also that by rescaling, we can obtain if we want from this the previous Wishart matrix formula, in full generality, at any value \( t>0\) of our old parameter. Thus, things fine, we are indeed generalizing what we did before.
(3) In order to prove now the formula in the statement, we proceed as usual, by using the Wick formula. The matrix entries of our Wishart matrix \( W=W_N\) are given by:
Thus, the normalized traces of powers of \( W\) are given by the following formula:
By rescaling now \( W\) by a \( 1/N\) factor, as in the statement, we obtain:
(4) By using now the Wick rule, we obtain the following formula for the moments, with \( K=\circ\bullet\circ\bullet…\,\) , alternating word of lenght \( 2k\) , and \( I=(i_1r_1,i_2r_1,…,i_kr_k,i_1r_k)\) :
(5) In order to compute this quantity, we use the standard bijection \( \mathcal P_2(K)\simeq S_k\) . By identifying the pairings \( \pi\in\mathcal P_2(K)\) with their counterparts \( \pi\in S_k\) , we obtain:
Now let \( \gamma\in S_k\) be the full cycle, which is by definition the following permutation:
The general factor in the product computed above is then 1 precisely when following two conditions are simultaneously satisfied:
Counting the number of free parameters in our expectation formula, we obtain:
(6) Now by using the same arguments as in the case \( M=N\) , from the proof of Theorem 15.18, we conclude that in the \( M=tN\to\infty\) limit the permutations \( \pi\in S_k\) which matter are those coming from noncrossing partitions, and so that we have:
We are therefore led to the conclusion in the statement.
In order to recapture now the density out of the moments, we can of course use the Stieltjes inversion formula, but the computations here are a bit opaque. So, inspired from what happens at \( t=1\) , let us cheat a bit, and formulate things as follows:
Definition 90
The Marchenko-Pastur law \( \pi_t\) of parameter \( t>0\) is given by:
That is, \( \pi_t\) the law of the square of a variable following the law \( \gamma_t\) .
This is certainly nice and simple, and we know that at \( t=1\) we obtain indeed the Marchenko-Pastur law \( \pi_1\) , as constructed above. In general, we have:
Proposition 68
The Marchenko-Pastur law of parameter \( t>0\) is
the support being \( [0,4t^2]\) , and the moments of this measure are
exactly as for the asymptotic moments of the complex Wishart matrices.
Proof
This follows as usual, by doing some computations, either combinatorics, or calculus. To be more precise, we have three formulae for \( \pi_t\) to be connected, namely the one in Definition 15.22, and the two ones from the present statement, and the connections between them can be established exactly as we did before, at \( t=1\) .
Now back to the complex Wishart matrices that we are interested in, we can now formulate a final result regarding them, as follows:
Theorem 261
Given a sequence of complex Wishart matrices
with \( Y_N\) being \( N\times M\) complex Gaussian of parameter \( 1\) , we have
with \( M=tN\to\infty\) , with the limiting measure being the Marchenko-Pastur law \( \pi_t\) .
Proof
This follows indeed from Theorem 15.21 and Proposition 15.23.
Many other things can be said, along these lines, and for more on all this, we refer to free probability theory [18], which has answers to nearly all potential questions that can be asked, regarding the various classes of random matrices investigated above.
Our goal now will be that of explaining a surprising result, stating that when suitably block-transposing the entries of a complex Wishart matrix, we obtain as asymptotic distribution a shifted version of Wigner’s semicircle law. Let us start with:
Definition 91
The partial transpose of a complex Wishart matrix \( W\) of parameters \( (dn,dm)\) is the matrix
where \( id\) is the identity of \( M_d(\mathbb C)\) , and \( t\) is the transposition of \( M_n(\mathbb C)\) .
In more familiar terms of bases and indices, the standard decomposition \( \mathbb C^{dn}=\mathbb C^d\otimes\mathbb C^n\) induces an algebra decomposition \( M_{dn}(\mathbb C)=M_d(\mathbb C)\otimes M_n(\mathbb C)\) , and with this convention made, the partial transpose matrix \( \tilde{W}\) constructed above has entries as follows:
Our goal in what follows will be that of computing the law of \( \tilde{W}\) , first when \( d,n,m\) are fixed, and then in the \( d\to\infty\) regime. For this purpose, we will need a number of standard facts regarding the noncrossing partitions. Let us start with:
Proposition 69
For a permutation \( \sigma\in S_p\) , we have the formula
where \( |\sigma|\) is the number of cycles of \( \sigma\) , and \( \#\sigma\) is the minimal \( k\in\mathbb N\) such that \( \sigma\) is a product of \( k\) transpositions. Also, the following formula defines a distance on \( S_p\) ,
and the set of permutations \( \sigma\in S_p\) which saturate the triangular inequality
where \( \gamma\in S_p\) is a full cycle, is in bijection with the set \( NC(p)\) .
Proof
All this is standard combinatorics, that we will leave here as an exercise.
We will need as well the following well-known result:
Proposition 70
The number \( ||\pi||\) of blocks having even size is given by
for every noncrossing partition \( \pi \in NC(p)\) .
Proof
We use a recurrence over the number of blocks of \( \pi\) . If \( \pi\) has just one block, its associated geodesic permutation is \( \gamma\) and we have:
For partitions \( \pi\) with more than one block, we can assume without loss of generality that \( \pi = \hat 1_k \sqcup \pi'\) , where \( \hat 1_k\) is a contiguous block of size \( k\) . Recall that the number of blocks of the permutation \( \pi\gamma\) is given by the following formula, where \( \rho_{14} \in P_2(2p)\) is the pair partition which pairs an element \( i\) with \( i+(-1)^{i+1}3\) :
If \( k\) is an even number, \( k=2r\) , consider the following partition, which contains the block \( (1 \, 4 \, 5 \, 8 \, … 4r-3 \, 4r)\) , along with the blocks coming from elements of the form \( 4i+2, 4i+3\) from \( \{1, …, 4r\}\) and from \( \pi'\) :
We can count the blocks of the join of two partitions by drawing them one beneath the other and counting the number of connected components of the curve, without taking into account the possible crossings. We conclude that we have the following formula, where \( \rho'_{14}\) is \( \rho_{14}\) restricted to the set \( \{2k+1, 2k+2 …, 2p\}\) :
If \( k\) is odd, \( k=2r+1\) , there is no extra block appearing, so we have:
Thus, we are led to the conclusion in the statement.
We can now investigate the block-transposed Wishart matrices, and we have:
Theorem 262
For any \( p\geq 1\) we have the formula
where \( |.|\) and \( ||.||\) are the number of blocks, and the number of blocks of even size.
Proof
The matrix elements of the partial transpose matrix are given by:
This gives the following formula:
The average of the general term can be computed by the Wick rule, namely:
Let \( \gamma\in S_p\) be the full cycle \( \gamma=(1 \, 2 \, … \, p)^{-1}\) . The general factor in the above product is 1 if and only if the following four conditions are simultaneously satisfied:
Counting the number of free parameters in the above equation, we obtain:
The exponent of \( d\) in the last expression on the right is:
As explained in the beginning of this section, this quantity is known to be \( \leq 0\) , with equality iff \( \pi\) is geodesic, hence associated to a noncrossing partition. Thus:
Together with \( |\pi\gamma|=||\pi||+1\) , this gives the result.
We would like now to find an equation for the moment generating function of the asymptotic law of \( m\tilde{W}\) . This moment generating function is defined by:
We have the following result, regarding this moment generating function:
Theorem 263
The moment generating function of \( m\tilde{W}\) satisfies the equation
in the \( d\to\infty\) limit.
Proof
We use the formula in Theorem 15.28. If we denote by \( N(p,b,e)\) the number of partitions in \( NC(p)\) having \( b\) blocks and \( e\) even blocks, we have:
Let us try to find a recurrence formula for the numbers \( N(p,b,e)\) . If we look at the block containing \( 1\) , this block must have \( r\geq 0\) other legs, and we get:
Here \( p_1,…,p_{r+1}\) are the number of points between the legs of the block containing 1, so that we have \( p=(p_1+…+p_{r+1})+r+1\) , and the whole sum is split over two cases, \( r\) even or odd, because the parity of \( r\) affects the number of even blocks of our partition. Now by multiplying everything by a \( z^pm^bn^e\) factor, and by carefully distributing the various powers of \( z,m,b\) on the right, we obtain the following formula:
Let us sum now all these equalities, over all \( p\geq 1\) and over all \( b,e\geq 0\) . According to the definition of \( F\) , at left we obtain \( F-1\) . As for the two sums appearing on the right, that is, at right of the two \( z^{r+1}\) factors, when summing them over all \( p\geq 1\) and over all \( b,e\geq 0\) , we obtain in both cases \( F^{r+1}\) . So, we have the following formula:
But this gives the formula in the statement, and we are done.
We can reformulate Theorem 15.29 as follows:
Theorem 264
The Cauchy transform of \( m\tilde{W}\) satisfies the equation
in the \( d\to\infty\) limit. Moreover, this equation simply reads
with the substitutions \( G\to z\) and \( \xi\to R+z^{-1}\) .
Proof
We have two assertions to be proved, the idea being as follows:
(1) Consider the equation of \( F\) , found in Theorem 15.29, namely:
With \( z\to\xi^{-1}\) and \( F\to\xi G\) , so that \( zF\to G\) , we obtain, as desired:
(2) Thus, we have our equation for the Cauchy transform, and with this in hand, we can try to go ahead, and use somehow the Stieltjes inversion formula, in order to reach to a formula for the density. This is certainly possible, but our claim is that we can do better, by performing first some clever manipulations on the Cauchy transform.
(3) To be more precise, with \( \xi \to K\) and \( G\to z\) , this equation becomes:
The point now is that with \( K\to R+z^{-1}\) this latter equation becomes:
But the solution of this latter equation is trivial to compute, given by:
Thus, we are led to the conclusion in the statement.
All the above suggests the following definition:
Definition 92
Given a real probability measure \( \mu\) , define its \( R\) -transform by:
That is, the \( R\) -transform is the inverse of the Cauchy transform, up to a \( \xi^{-1}\) factor.
Getting back now to our questions, we would like to find the probability measure having as \( R\) -transform the function in Theorem 15.30. But here, we can only expect to find some kind of modification of the Marchenko-Pastur law, so as a first piece of work, let us just compute the \( R\) -transform of the Marchenko-Pastur law. We have here:
Proposition 71
The \( R\) -transform of the Marchenko-Pastur law \( \pi_t\) is
for any \( t>0\) .
Proof
This can be done in two steps, as follows:
(1) At \( t=1\) , we know that the moments of \( \pi_1\) are the Catalan numbers, \( M_k=C_k\) , and we obtain that the Cauchy transform is given by the following formula:
Now with \( R(\xi)=\frac{1}{1-\xi}\) being the function in the statement, at \( t=1\) , we have:
Thus, the function \( R(\xi)=\frac{1}{1-\xi}\) is indeed the \( R\) -transform of \( \pi_1\) , in the above sense.
(2) In the general case, \( t>0\) , the proof is similar, by using the moment formula for \( \pi_t\) , that we know from the above, and we will leave this as an exercise.
All this is very nice, and we can now further build on Theorem 15.30, as follows:
Theorem 265
The \( R\) -transform of \( m\tilde{W}\) is given by
in the \( d\to\infty\) limit, where \( s=m(n+1)/2\) and \( t=m(n-1)/2\) .
Proof
We know from Theorem 15.30 that the \( R\) -transform of \( m\tilde{W}\) is given by:
By using now the formula in Proposition 15.32, this gives the result.
We can now formulate a final result, due to Aubrun, as follows:
Theorem 266
For a block-transposed Wishart matrix \( \tilde{W}=(id\otimes t)W\) we have, in the \( n=\beta m\to\infty\) limit, with \( \beta>0\) fixed, the formula
with \( \gamma_\beta^1\) being the shifted version of the semicircle law \( \gamma_\beta\) , with support centered at \( 1\) .
Proof
This follows from Theorem 15.33. Indeed, in the \( n=\beta m\to\infty\) limit, with \( \beta>0\) fixed, we are led to the following formula for the Stieltjes transform:
But this is the density of the shifted semicircle law having support as follows:
Thus, we are led to the conclusion in the statement.
This was a quite exciting chapter, and as exercises on this, we have:
Exercise 113
Compute the spectral measures of various matrices, of your choice.
Exercise 114
Learn more about \( C^*\) -algebras, including the GNS theorem.
Exercise 115
Importantly, learn as well about the von Neumann algebras.
Exercise 116
Learn as well some other approaches to the spectral measures.
Exercise 117
Is the asymptotic law of Gaussian matrices a “circular law”?
Exercise 118
Is the Wigner semicircle law some sort of “free Gaussian law”?
Exercise 119
Is the Marchenko-Pastur law some sort of “free Poisson law”?
Exercise 120
Read more, from Aubrun and others, about block modifications.
As bonus exercise, learn some free probability theory, from [18].
Welcome to freeness. We will be interested here in developing free geometry and analysis, with the hope that all this might be related to physics, at very small scales, quarks and below. The idea being very simple, based on old findings of Heisenberg and others, if it is true indeed that the more you zoom down, the more commutativity dissapears, then, logically, if you zoom hard enough, things will become free.
So, what is free? The simplest free object in mathematics is the free group \( F_N\) :
Definition 93
The free group \( F_N\) is the infinite group
generated by \( N\) variables \( g_1,…,g_N\) , with no relations between them.
This might look a bit abstract, but no worries, \( F_N\) has some interesting mathematics, coming right away, if you have some knowledge in discrete groups, and know how to look for interesting questions. For instance if you want to draw the Cayley graph of \( F_N\) , whose vertices are the elements of \( F_N\) , with edges \( h-k\) drawn when \( h=g_i^{\pm1}k\) for some \( i\) , you will end up with an interesting picture, which at \( N=2\) looks like this: 
And this type of graph certainly has interesting mathematics. One good question for instance is that of computing the number of length \( 2k\) loops based at the root. Another question, which is in fact equivalent, via moments, is that of computing the Kesten measure of \( F_N\) , which is that of the following variable in the group algebra of \( F_N\) :
All this looks very good, we most likely have here our first object of free geometry. In order now to formally understand this, let us recall the following formula, with \( \mathbb T_N=\mathbb T^N\) being the usual torus, and with \( \mathbb Z^N\) being the free abelian group:
Thus, getting back now to our free group \( F_N\) , which is the free analogue of \( \mathbb Z^N\) , it is in fact its dual \( \widehat{F_N}\) which is a free manifold, and more specifically the free analogue of \( \mathbb T^N\) . Which is a nice finding, so let us formulate our conclusions as follows:
Definition 94
The free torus \( \mathbb T_N^+\) is the dual of the free group \( F_N\) ,
in analogy with the fact that the usual torus \( \mathbb T_N=\mathbb T^N\) appears as
with on the right the group \( \mathbb Z^N\) being the free abelian group.
It is of course possible to formulate things more precisely, and we will be back to this in a moment, but before that, isn’t this a bit too abstract? But the point here is that no, at the level of questions to be solved, these remain the same, as for instance the computation of the Kesten measure, which is now a “function” on the free torus:
In fact, this function is the main character of \( \mathbb T_N^+\) , regarded as a compact quantum group, and so our Kesten problem suddenly becomes something very conceptual, namely the computation of the law of the main character of \( \mathbb T_N^+\) . Which is very nice.
Before getting into details regarding all this, recall that \( \mathbb R^N\) is as interesting as \( \mathbb C^N\) . So, let us formulate as well the real version of Definition 16.2, as follows:
Definition 95
The free real torus, or free cube, \( T_N^+\) is the dual
of the group \( L_N=F_N/<g_i^2=1>\) , in analogy with the fact that the usual cube is
with on the right the group \( \mathbb Z_2^N\) being the free real abelian group.
Here the “real” at the end stands for the fact that the generators must satisfy the real reflection condition \( g^2=1\) . As for the fact that “real torus = cube”, as stated, this needs some thinking, and in the hope that, after such thinking, you will agree with me that there is indeed a standard torus inside \( \mathbb R^N\) , and that is the unit cube.
Summarizing, all this sounds good, we have a beginning of free geometry, both real and complex, worth developing, by knowing at least what the torus of each theory is. In practice now, at the level of details, in order to talk about \( \mathbb T_N^+=\widehat{F_N}\) and \( T_N^+=\widehat{L_N}\) we need an extension of the usual Pontrjagin duality theory for the abelian groups, and this is best done via operator algebras, and the related notion of compact quantum group.
In order to understand all this, let us start with operator algebras. As explained in chapter 15, in relation with our random matrix needs there, the starting point is:
Definition 96
A \( C^*\) -algebra is a complex algebra \( A\) , having:
As basic examples, we have \( B(H)\) itself, as well as any norm closed \( *\) -subalgebra \( A\subset B(H)\) . It is possible to prove that any \( C^*\) -algebra appears in this way, but we will not need in what follows this deep result, called GNS theorem after Gelfand, Naimark, Segal. So, let us simply agree that, by definition, the \( C^*\) -algebras \( A\) are some sort of “generalized operator algebras”, and their elements \( a\in A\) can be thought of as being some kind of “generalized operators”, on some Hilbert space which is not present.
In practice, this vague idea is all that we need. Indeed, by taking some inspiration from linear algebra, we can emulate spectral theory in our setting, as follows:
Proposition 72
Given \( a\in A\) , define its spectrum as being the set
and its spectral radius \( \rho(a)\) as the radius of the smallest centered disk containing \( \sigma(a)\) .
Proof
This is something that we know well from chapter 12 in the case \( A\subset B(H)\) , and from chapter 15 in general, and we refer to the material there for details.
Generally speaking, Proposition 16.5 is all you need for doing some further operator algebras. As a main application, that we already met in chapter 15, we have:
Theorem 267 (Gelfand)
If \( X\) is a compact space, the algebra \( C(X)\) of continuous functions \( f:X\to\mathbb C\) is a commutative \( C^*\) -algebra, with structure as follows:
Conversely, any commutative \( C^*\) -algebra is of the form \( C(X)\) , with its “spectrum” \( X=Spec(A)\) appearing as the space of characters \( \chi :A\to\mathbb C\) .
Proof
Given a commutative \( C^*\) -algebra \( A\) , we can define indeed \( X\) to be the set of characters \( \chi :A\to\mathbb C\) , with the topology making continuous all the evaluation maps \( ev_a:\chi\to\chi(a)\) . Then \( X\) is a compact space, and \( a\to ev_a\) is a morphism of algebras:
We first prove that \( ev\) is involutive. We use the following formula:
Thus it is enough to prove the equality \( ev_{a^*}=ev_a^*\) for self-adjoint elements \( a\) . But this is the same as proving that \( a=a^*\) implies that \( ev_a\) is a real function, which is in turn true, because \( ev_a(\chi)=\chi(a)\) is an element of \( \sigma(a)\) , contained in \( \mathbb R\) . So, claim proved. Also, since \( A\) is commutative, each element is normal, so \( ev\) is isometric:
It remains to prove that \( ev\) is surjective. But this follows from the Stone-Weierstrass theorem, because \( ev(A)\) is a closed subalgebra of \( C(X)\) , which separates the points.
The Gelfand theorem suggests formulating the following definition:
Definition 97
Given a \( C^*\) -algebra \( A\) , not necessarily commutative, we write
and call the abstract object \( X\) a “compact quantum space”.
This might look quite revolutionary, but in practice, this definition changes nothing to what we have been doing so far, namely studying the \( C^*\) -algebras. So, we will keep studying the \( C^*\) -algebras, but by using the above fancy quantum space terminology. For instance whenever we have a morphism \( \Phi:A\to B\) , we will write \( A=C(X),B=C(Y)\) , and rather speak of the corresponding morphism \( \phi:Y\to X\) . And so on.
Let us also mention that, at a closer look, there are in fact some problems with the above definition, coming from amenability and others. More on this later.
Now that we have our notion of quantum spaces, good time to get back towards Definitions 16.2 and 16.3. In order to understand what that free tori are, we will need:
Theorem 268
Let \( \Gamma\) be a discrete group, and consider the complex group algebra \( \mathbb C[\Gamma]\) , with involution given by the fact that all group elements are unitaries, \( g^*=g^{-1}\) .
Proof
All this is very standard, the idea being as follows:
(1) In order to prove the result, we must find a \( *\) -algebra embedding \( \mathbb C[\Gamma]\subset B(H)\) , with \( H\) being a Hilbert space. For this purpose, consider the space \( H=l^2(\Gamma)\) , having \( \{h\}_{h\in\Gamma}\) as orthonormal basis. Our claim is that we have an embedding, as follows:
Indeed, since \( \pi(g)\) maps the basis \( \{h\}_{h\in\Gamma}\) into itself, this operator is well-defined, bounded, and is an isometry. It is also clear from the formula \( \pi(g)(h)=gh\) that \( g\to\pi(g)\) is a morphism of algebras, and since this morphism maps the unitaries \( g\in\Gamma\) into isometries, this is a morphism of \( *\) -algebras. Finally, the faithfulness of \( \pi\) is clear.
(2) Since \( \Gamma\) is abelian, the corresponding group algebra \( A=C^*(\Gamma)\) is commutative. Thus, we can apply the Gelfand theorem, and we obtain \( A=C(X)\) , with:
But the spectrum \( X=Spec(A)\) , consisting of the characters \( \chi:C^*(\Gamma)\to\mathbb C\) , can be identified with the Pontrjagin dual \( G=\widehat{\Gamma}\) , and this gives the result.
The above result suggests the following definition:
Definition 98
Given a discrete group \( \Gamma\) , the compact quantum space \( G\) given by
is called abstract dual of \( \Gamma\) , and is denoted \( G=\widehat{\Gamma}\) .
Good news, this definition is exactly what we need, in order to understand the meaning of Definitions 16.2 and 16.3. To be more precise, we have the following result:
Proposition 73
The basic tori are all group duals, as follows,

where \( F_N=\mathbb Z^{*N}\) is the free group on \( N\) generators, and \( L_N=\mathbb Z_2^{*N}\) is its real version.
Proof
The basic tori appear indeed as group duals, and together with the Fourier transform identifications from Theorem 16.8 (2), this gives the result.
Moving ahead, now that we have our formalism, we can start developing free geometry. As a first objective, we would like to better understand the relation between the classical and free tori. In order to discuss this, let us introduce the following notion:
Definition 99
Given a compact quantum space \( X\) , its classical version is the usual compact space \( X_{class}\subset X\) obtained by dividing \( C(X)\) by its commutator ideal:
In this situation, we also say that \( X\) appears as a “liberation” of \( X\) .
In other words, the space \( X_{class}\) appears as the Gelfand spectrum of the commutative \( C^*\) -algebra \( C(X)/I\) . Observe in particular that \( X_{class}\) is indeed a classical space.
In relation now with our tori, we have the following result:
Theorem 269
We have inclusions between the various tori, as follows,

and the free tori on top appear as liberations of the tori on the bottom.
Proof
This is indeed clear from definitions, because commutativity of a group algebra means precisely that the group in question is abelian.
In order to extend now the free geometries that we have, real and complex, let us begin with the spheres. We have the following notions:
Definition 100
We have free real and complex spheres, defined via
where the symbol \( C^*\) stands for universal enveloping \( C^*\) -algebra.
Here the fact that these algebras are indeed well-defined comes from the following estimate, which shows that the biggest \( C^*\) -norms on these \( *\) -algebras are bounded:
As a first result now, regarding the above free spheres, we have:
Theorem 270
We have embeddings of compact quantum spaces, as follows,

and the spaces on top appear as liberations of the spaces on the bottom.
Proof
The first assertion, regarding the inclusions, comes from the fact that at the level of the associated \( C^*\) -algebras, we have surjective maps, as follows:

For the second assertion, we must establish the following isomorphisms, where the symbol \( C^*_{comm}\) stands for “universal commutative \( C^*\) -algebra generated by”:
It is enough to establish the second isomorphism. So, consider the second universal commutative \( C^*\) -algebra \( A\) constructed above. Since the standard coordinates on \( S^{N-1}_\mathbb C\) satisfy the defining relations for \( A\) , we have a quotient map of as follows:
Conversely, let us write \( A=C(S)\) , by using the Gelfand theorem. Then \( x_1,…,x_N\) become in this way true coordinates, providing us with an embedding as follows:
Also, the quadratic relations become \( \sum_i|x_i|^2=1\) , so we have \( S\subset S^{N-1}_\mathbb C\) . Thus, we have a quotient map \( C(S^{N-1}_\mathbb C)\to A\) , as desired, and this gives all the results.
By using the free spheres constructed above, we can now formulate:
Definition 101
A real algebraic manifold \( X\subset S^{N-1}_{\mathbb C,+}\) is a closed quantum subspace defined, at the level of the corresponding \( C^*\) -algebra, by a formula of type
for certain family of noncommutative polynomials, as follows:
We denote by \( \mathcal C(X)\) the \( *\) -subalgebra of \( C(X)\) generated by the coordinates \( x_1,…,x_N\) .
As a basic example here, we have the free real sphere \( S^{N-1}_{\mathbb R,+}\) . The classical spheres \( S^{N-1}_\mathbb C,S^{N-1}_\mathbb R\) , and their real submanifolds, are covered as well by this formalism. At the level of the general theory, we have the following version of the Gelfand theorem:
Theorem 271
If \( X\subset S^{N-1}_{\mathbb C,+}\) is an algebraic manifold, as above, we have
and \( X\) appears as a liberation of \( X_{class}\) .
Proof
This is something that we already met, in the context of the free spheres. In general, the proof is similar, by using the Gelfand theorem. Indeed, if we denote by \( X_{class}'\) the manifold constructed in the statement, then we have a quotient map of \( C^*\) -algebras as follows, mapping standard coordinates to standard coordinates:
Conversely now, from \( X\subset S^{N-1}_{\mathbb C,+}\) we obtain \( X_{class}\subset S^{N-1}_\mathbb C\) . Now since the relations defining \( X_{class}'\) are satisfied by \( X_{class}\) , we obtain an inclusion \( X_{class}\subset X_{class}'\) . Thus, at the level of algebras of continuous functions, we have a quotient map of \( C^*\) -algebras as follows, mapping standard coordinates to standard coordinates:
Thus, we have constructed a pair of inverse morphisms, and we are done.
Finally, once again at the level of the general theory, we have:
Definition 102
We agree to identify two real algebraic submanifolds \( X,Y\subset S^{N-1}_{\mathbb C,+}\) when we have a \( *\) -algebra isomorphism between \( *\) -algebras of coordinates
mapping standard coordinates to standard coordinates.
We will see later the reasons for making this convention, coming from amenability. Now back to the tori, as constructed before, we can see that these are examples of algebraic manifolds, in the sense of Definition 16.15. In fact, we have the following result:
Theorem 272
The four main quantum spheres produce the main quantum tori

via the formula \( T=S\cap\mathbb T_N^+\) , with the intersection being taken inside \( S^{N-1}_{\mathbb C,+}\) .
Proof
This comes from the above results, the situation being as follows:
(1) Free complex case. Here the formula in the statement reads:
But this is something trivial, because we have \( \mathbb T_N^+\subset S^{N-1}_{\mathbb C,+}\) .
(2) Free real case. Here the formula in the statement reads:
But this is clear as well, the real version of \( \mathbb T_N^+\) being \( T_N^+\) .
(3) Classical complex case. Here the formula in the statement reads:
But this is clear as well, the classical version of \( \mathbb T_N^+\) being \( \mathbb T_N\) .
(4) Classical real case. Here the formula in the statement reads:
But this follows by intersecting the formulae from the proof of (2) and (3).
In order to better understand the structure of the free spheres \( S^{N-1}_{\mathbb R,+},S^{N-1}_{\mathbb C,+}\) , we need to talk about free rotations. Following Woronowicz, let us start with:
Definition 103
A Woronowicz algebra is a \( C^*\) -algebra \( A\) , given with a unitary matrix \( u\in M_N(A)\) whose coefficients generate \( A\) , such that the formulae
define morphisms of \( C^*\) -algebras \( \Delta:A\to A\otimes A\) , \( \varepsilon:A\to\mathbb C\) , \( S:A\to A^{opp}\) .
We say that \( A\) is cocommutative when \( \Sigma\Delta=\Delta\) , where \( \Sigma(a\otimes b)=b\otimes a\) is the flip. We have the following result, which justifies the terminology and axioms:
Theorem 273
The following are Woronowicz algebras:
\( C(G)\) , with \( G\subset U_N\) compact Lie group. Here the structural maps are:
\( C^*(\Gamma)\) , with \( F_N\to\Gamma\) finitely generated group. Here the structural maps are:
Moreover, we obtain in this way all the commutative/cocommutative algebras.
Proof
This is something very standard, the idea being as follows:
(1) Given \( G\subset U_N\) , we can set \( A=C(G)\) , which is a Woronowicz algebra, together with the matrix \( u=(u_{ij})\) formed by coordinates of \( G\) , given by:
Conversely, if \( (A,u)\) is a commutative Woronowicz algebra, by using the Gelfand theorem we can write \( A=C(X)\) , with \( X\) being a certain compact space. The coordinates \( u_{ij}\) give then an embedding \( X\subset M_N(\mathbb C)\) , and since the matrix \( u=(u_{ij})\) is unitary we actually obtain an embedding \( X\subset U_N\) , and finally by using the maps \( \Delta,\varepsilon,S\) we conclude that our compact subspace \( X\subset U_N\) is in fact a compact Lie group, as desired.
(2) Consider a finitely generated group \( F_N\to\Gamma\) . We can set \( A=C^*(\Gamma)\) , which is by definition the completion of the complex group algebra \( \mathbb C[\Gamma]\) , with involution given by \( g^*=g^{-1}\) , for any \( g\in\Gamma\) , with respect to the biggest \( C^*\) -norm, and we obtain a Woronowicz algebra, together with the diagonal matrix formed by the generators of \( \Gamma\) :
Conversely, if \( (A,u)\) is a cocommutative Woronowicz algebra, the Peter-Weyl theory of Woronowicz, to be explained below, shows that the irreducible corepresentations of \( A\) are all 1-dimensional, and form a group \( \Gamma\) , and so we have \( A=C^*(\Gamma)\) , as desired.
The above result makes it quite clear that what we have in Definition 16.19 is some sort of joint definition for the compact and discrete quantum groups. In order to further comment on this, let us go back to the formulae in Definition 16.19, namely:
The morphisms \( \Delta,\varepsilon,S\) are called comultiplication, counit and antipode, and they have the following properties, which are something very familiar in abstract algebra:
Theorem 274
Let \( (A,u)\) be a Woronowicz algebra.
\( \Delta,\varepsilon\) satisfy the usual axioms for a comultiplication and a counit, namely:
\( S\) satisfies the antipode axiom, on the \( *\) -subalgebra generated by entries of \( u\) :
Proof
Observe first that the result holds indeed in the case where the algebra \( A\) is commutative. Indeed, by using Theorem 16.20 we can write:
The above 3 conditions come then by transposition from the basic 3 group theory conditions satisfied by \( m,u,i\) , which are as follows, with \( \delta(g)=(g,g)\) :
Observe that \( S^2=id\) is satisfied as well, coming from \( i^2=id\) , which is a consequence of the group axioms. Finally, observe that the result holds also, trivially, when \( A\) is cocommutative, again by Theorem 16.20. In general now, the proof goes as follows:
(1) We have indeed the following computation:
On the other hand, we have as well the following computation:
The proof for the counit axiom is quite similar. We first have:
On the other hand, we have as well the following computation:
(2) By using the fact that the matrix \( u=(u_{ij})\) is unitary, we obtain:
Similarly, we have the following computation, which gives the antipode axiom:
(3) Finally, the formula \( S^2=id\) holds as well on the generators, and we are done.
Summarizing, the Woronowicz algebras appear to have nice properties. In view of Theorem 16.20 and Theorem 16.21, we can formulate the following definition:
Definition 104
Given a Woronowicz algebra \( A\) , we formally write
and call \( G\) compact quantum group, and \( \Gamma\) discrete quantum group.
In relation with this, there are actually some analytic subtleties, coming from amenability, so our objects must be divided by a certain equivalence relation, for everything to work fine. To be more precise, we agree to write \( (A,u)=(B,v)\) when there is a \( *\) -algebra isomorphism as follows, mapping standard coordinates to standard coordinates:
Moving ahead now, let us call now corepresentation of \( A\) any unitary matrix \( v\in M_n(A)\) satisfying the same conditions as those satisfied by \( u\) , namely:
These corepresentations can be thought of as corresponding representations of the underlying compact quantum group \( G\) . Following Woronowicz, we have:
Theorem 275
Any Woronowicz algebra has a unique Haar integration functional,
which can be constructed by starting with any faithful positive form \( \varphi\in A^*\) , and setting
where \( \phi*\psi=(\phi\otimes\psi)\Delta\) . Moreover, for any corepresentation \( v\in M_n(\mathbb C)\otimes A\) we have
where \( P\) is the orthogonal projection onto \( Fix(v)=\{\xi\in\mathbb C^n|v\xi=\xi\}\) .
Proof
This can be done in 3 steps, as follows:
(1) Given \( \varphi\in A^*\) , our claim is that the following limit converges, for any \( a\in A\) :
Indeed, by linearity we can assume that \( a\) is the coefficient of corepresentation, \( a=(\tau\otimes id)v\) . But in this case, an elementary computation shows that we have the following formula, where \( P_\varphi\) is the orthogonal projection onto the \( 1\) -eigenspace of \( (id\otimes\varphi)v\) :
(2) Since \( v\xi=\xi\) implies \( [(id\otimes\varphi)v]\xi=\xi\) , we have \( P_\varphi\geq P\) , where \( P\) is the orthogonal projection onto the space \( Fix(v)=\{\xi\in\mathbb C^n|v\xi=\xi\}\) . The point now is that when \( \varphi\in A^*\) is faithful, by using a positivity trick, one can prove that we have \( P_\varphi=P\) . Thus our linear form \( \int_\varphi\) is independent of \( \varphi\) , and is given on coefficients \( a=(\tau\otimes id)v\) by:
(3) With the above formula in hand, the left and right invariance of \( \int_G=\int_\varphi\) is clear on coefficients, and so in general, and this gives all the assertions.
Consider now the dense \( *\) -subalgebra \( \mathcal A\subset A\) generated by the coefficients of \( u\) , that we met after Definition 16.22, and endow it with the following scalar product:
Still following Woronowicz, we have the following key result:
Theorem 276
We have the following Peter-Weyl type results:
Proof
All these results are very standard, the idea being as follows:
(1) Given \( v\in M_n(A)\) , its intertwiner algebra \( End(v)=\{T\in M_n(\mathbb C)|Tv=vT\}\) is a finite dimensional \( C^*\) -algebra, and so decomposes as \( End(v)=M_{n_1}(\mathbb C)\oplus…\oplus M_{n_r}(\mathbb C)\) . But this gives a decomposition of type \( v=v_1+…+v_r\) , as desired.
(2) Consider indeed the Peter-Weyl corepresentations, \( u^{\otimes k}\) with \( k\) colored integer, defined by \( u^{\otimes\emptyset}=1\) , \( u^{\otimes\circ}=u\) , \( u^{\otimes\bullet}=\bar{u}\) and multiplicativity. The coefficients of these corepresentations span the dense algebra \( \mathcal A\) , and by using (1), this gives the result.
(3) Here the direct sum decomposition, which is technically a \( *\) -coalgebra isomorphism, follows from (2). As for the second assertion, this follows from the fact that \( (id\otimes\int_G)v\) is the orthogonal projection \( P_v\) onto the space \( Fix(v)\) , for any corepresentation \( v\) .
(4) Let us define indeed the character of \( v\in M_n(A)\) to be the matrix trace, \( \chi_v=Tr(v)\) . Since this character is a coefficient of \( v\) , the orthogonality assertion follows from (3). As for the norm 1 claim, this follows once again from \( (id\otimes\int_G)v=P_v\) .
Good news, we can now talk about free rotations. Following Wang, we have:
Theorem 277
The following universal algebras are Woronowicz algebras,
so the underlying spaces \( O_N^+,U_N^+\) are compact quantum groups.
Proof
This follows from the elementary fact that if a matrix \( u=(u_{ij})\) is orthogonal or biunitary, then so must be the following matrices:
Thus, we can indeed define morphisms \( \Delta,\varepsilon,S\) as in Definition 16.19, by using the universal properties of \( C(O_N^+)\) , \( C(U_N^+)\) , and this gives the result.
In order to discuss now the relation with the spheres, which can only come via some sort of “isometric actions”, let us start with the following standard fact:
Proposition 74
Given a closed subset \( X\subset S^{N-1}_\mathbb C\) , the formula
defines a compact group of unitary matrices, or isometries, called affine isometry group of \( X\) . For the spheres \( S^{N-1}_\mathbb R,S^{N-1}_\mathbb C\) we obtain in this way the groups \( O_N,U_N\) .
Proof
The fact that \( G(X)\) as defined above is indeed a group is clear, its compactness is clear as well, and finally the last assertion is clear as well. In fact, all this works for any closed subset \( X\subset\mathbb C^N\) , but we are not interested here in such general spaces.
Observe that in the case of the real and complex spheres, the affine isometry group \( G(X)\) leaves invariant the Riemannian metric, because this metric is equivalent to the one inherited from \( \mathbb C^N\) , which is preserved by our isometries \( U\in U_N\) .
Thus, we could have constructed as well \( G(X)\) as being the group of metric isometries of \( X\) , with of course some extra care in relation with the complex structure, as for the complex sphere \( X=S^{N-1}_\mathbb C\) to produce \( G(X)=U_N\) instead of \( G(X)=O_{2N}\) . But, such things won’t really work for the free spheres, and so are to be avoided.
The point now is that we have the following quantum analogue of Proposition 16.26, which is a perfect analogue, save for the fact that \( X\) is now assumed to be algebraic, for some technical reasons, which allows us to talk about quantum isometry groups:
Theorem 278
Given an algebraic manifold \( X\subset S^{N-1}_{\mathbb C,+}\) , the category of the closed subgroups \( G\subset U_N^+\) acting affinely on \( X\) , in the sense that the formula
defines a morphism of \( C^*\) -algebras \( \Phi:C(X)\to C(X)\otimes C(G)\) , has a universal object, denoted \( G^+(X)\) , and called affine quantum isometry group of \( X\) .
Proof
Assume indeed that our manifold \( X\subset S^{N-1}_{\mathbb C,+}\) comes as follows:
In order to prove the result, consider the following variables:
Our claim is that the quantum group in the statement \( G=G^+(X)\) appears as:
In order to prove this, pick one of the defining polynomials, and write it as follows:
With \( X_i=\sum_jx_j\otimes u_{ji}\) as above, we have the following formula:
Since the variables on the right span a certain finite dimensional space, the relations \( f_\alpha(X_1,…,X_N)=0\) correspond to certain relations between the variables \( u_{ij}\) . Thus, we have indeed a closed subspace \( G\subset U_N^+\) , with a universal map, as follows:
In order to show now that \( G\) is a quantum group, consider the following elements:
Consider as well the following elements, with \( \gamma\in\{\Delta,\varepsilon,S\}\) :
From the relations \( f_\alpha(X_1,…,X_N)=0\) we deduce that we have:
Thus we can map \( u_{ij}\to u_{ij}^\gamma\) for any \( \gamma\in\{\Delta,\varepsilon,S\}\) , and we are done.
We can now formulate a result about spheres and rotations, as follows:
Theorem 279
The quantum isometry groups of the basic spheres are

modulo identifying, as usual, the various \( C^*\) -algebraic completions.
Proof
We have 4 results to be proved, the idea being as follows:
\( S^{N-1}_{\mathbb C,+}\) . Let us first construct an action \( U_N^+\curvearrowright S^{N-1}_{\mathbb C,+}\) . We must prove here that the variables \( X_i=\sum_jx_j\otimes u_{ji}\) satisfy the defining relations for \( S^{N-1}_{\mathbb C,+}\) , namely:
By using the biunitarity of \( u\) , we have the following computation:
Once again by using the biunitarity of \( u\) , we have as well:
Thus we have an action \( U_N^+\curvearrowright S^{N-1}_{\mathbb C,+}\) , which gives \( G^+(S^{N-1}_{\mathbb C,+})=U_N^+\) , as desired.
\( S^{N-1}_{\mathbb R,+}\) . Let us first construct an action \( O_N^+\curvearrowright S^{N-1}_{\mathbb R,+}\) . We already know that the variables \( X_i=\sum_jx_j\otimes u_{ji}\) satisfy the defining relations for \( S^{N-1}_{\mathbb C,+}\) , so we just have to check that these variables are self-adjoint. But this is clear from \( u=\bar{u}\) , as follows:
Conversely, assume that we have an action \( G\curvearrowright S^{N-1}_{\mathbb R,+}\) , with \( G\subset U_N^+\) . The variables \( X_i=\sum_jx_j\otimes u_{ji}\) must be then self-adjoint, and the above computation shows that we must have \( u=\bar{u}\) . Thus our quantum group must satisfy \( G\subset O_N^+\) , as desired.
\( S^{N-1}_\mathbb C\) . The fact that we have an action \( U_N\curvearrowright S^{N-1}_\mathbb C\) is clear. Conversely, assume that we have an action \( G\curvearrowright S^{N-1}_\mathbb C\) , with \( G\subset U_N^+\) . We must prove that this implies \( G\subset U_N\) , and we will use a standard trick of Bhowmick-Goswami. We have:
By multiplying this formula with itself we obtain:
Since the variables \( x_i\) commute, these formulae can be written as:
Since the tensors at left are linearly independent, we must have:
By applying the antipode to this formula, then applying the involution, and then relabelling the indices, we succesively obtain:
Now by comparing with the original formula, we obtain from this:
In order to finish, it remains to prove that the coordinates \( u_{ij}\) commute as well with their adjoints. For this purpose, we use a similar method. We have:
Since the variables on the left are equal, we deduce from this that we have:
Thus we have \( u_{ji}u_{lk}^*=u_{lk}^*u_{ji}\) , and so \( G\subset U_N\) , as claimed.
\( S^{N-1}_\mathbb R\) . The fact that we have an action \( O_N\curvearrowright S^{N-1}_\mathbb R\) is clear. In what regards the converse, this follows by combining the results that we already have, as follows:
Thus, we conclude that we have \( G^+(S^{N-1}_\mathbb R)=O_N\) , as desired.
Let us discuss now the correspondence \( U\to S\) . In the classical case the situation is very simple, because the sphere \( S=S^{N-1}\) appears by rotating the point \( x=(1,0,…,0)\) by the isometries in \( U=U_N\) . Moreover, the stabilizer of this action is the subgroup \( U_{N-1}\subset U_N\) acting on the last \( N-1\) coordinates, and so the sphere \( S=S^{N-1}\) appears from the corresponding rotation group \( U=U_N\) as an homogeneous space, as follows:
In functional analytic terms, all this becomes even simpler, the correspondence \( U\to S\) being obtained, at the level of algebras of functions, as follows:
In general now, the straightforward homogeneous space interpretation of \( S\) as above fails. However, we can have some theory going by using the functional analytic viewpoint, with an embedding \( x_i\to u_{1i}\) as above. Let us start with the following result:
Proposition 75
For the basic spheres, we have a diagram as follows,

where on top \( \Phi(x_i)=\sum_jx_j\otimes u_{ji}\) , and on the left \( \alpha(x_i)=u_{1i}\) .
Proof
The diagram in the statement commutes indeed on the standard coordinates, the corresponding arrows being as follows, on these coordinates: 
Thus by linearity and multiplicativity, the whole the diagram commutes.
As a consequence of the above result, we can now formulate:
Proposition 76
We have a quotient map and an inclusion as follows,
with \( S_U\) being the first row space of \( U\) , given by
at the level of the corresponding algebras of functions.
Proof
At the algebra level, we have an inclusion and a quotient map as follows:
Thus, we obtain the result, by transposing.
The above result is all that we need, for getting started with our study, and we will prove in what follows that the inclusion \( S_U\subset S\) constructed above is an isomorphism. This will produce the correspondence \( U\to S\) that we are currently looking for.
In order to do so, we will use the uniform integration over \( S\) , which can be introduced, in analogy with what happens in the classical case, in the following way:
Definition 105
We endow each of the algebras \( C(S)\) with its integration functional
obtained by composing the morphism \( x_i\to u_{1i}\) with the Haar integration of \( C(U)\) .
With this done, in order now to efficiently integrate over our various spheres \( S\) , and in the lack of some trick like spherical coordinates, we need to know how to efficiently integrate over the corresponding quantum isometry groups \( U\) .
Now regarding this latter question, we already have some good experience with it, in the classical case, from chapter 14. And the point is that the material there extends without much difficulties in the free case, and we have the following result:
Theorem 280
Assuming that a compact quantum group \( G\subset U_N^+\) is easy, coming from a category of partitions \( D\subset P\) , we have the Weingarten formula
for any indices \( i_r,j_r\in\{1,…,N\}\) and any exponents \( e_r\in\{\emptyset,*\}\) , where \( \delta\) are the usual Kronecker type symbols, and where
is the inverse of the matrix \( G_{kN}(\pi,\sigma)=N^{|\pi\vee\sigma|}\) .
Proof
Let us arrange indeed all the integrals to be computed, at a fixed value of the exponent \( k=(e_1… e_k)\) , into a single matrix, of size \( N^k\times N^k\) , as follows:
According to the construction of the Haar measure of Woronowicz, explained in the above, this matrix \( P\) is the orthogonal projection onto the following space:
In order to compute this projection, consider the following linear map:
Consider as well the inverse \( W\) of the restriction of \( E\) to the following space:
By a standard linear algebra computation, it follows that we have:
But the restriction of \( E\) is the linear map corresponding to \( G_{kN}\) , so \( W\) is the linear map corresponding to \( W_{kN}\) , and this gives the result.
With this in hand, we can now integrate over the spheres \( S\) , as follows:
Theorem 281
The integration over the basic spheres is given by
with \( \pi,\sigma\in D(k)\) , where \( W_{kN}=G_{kN}^{-1}\) is the inverse of \( G_{kN}(\pi,\sigma)=N^{|\pi\vee\sigma|}\) .
Proof
According to our conventions, the integration over \( S\) is a particular case of the integration over \( U\) , via \( x_i=u_{1i}\) . By using now Theorem 16.32, we obtain:
Thus, we are led to the formula in the statement.
Again with some inspiration from the classical case, we have the following key result:
Theorem 282
The integration functional of \( S\) has the ergodicity property
where \( \Phi:C(S)\to C(S)\otimes C(U)\) is the universal affine coaction map.
Proof
In the real case, \( x_i=x_i^*\) , it is enough to check the equality in the statement on an arbitrary product of coordinates, \( x_{i_1}… x_{i_k}\) . The left term is as follows:
Let us look now at the last sum on the right. The situation is as follows:
– In the free case we have to sum quantities of type \( x_{j_1}… x_{j_k}\) , over all choices of multi-indices \( j=(j_1,…,j_k)\) which fit into our given noncrossing pairing \( \pi\) , and just by using the condition \( \sum_ix_i^2=1\) , we conclude that the sum is 1.
– The same happens in the classical case. Indeed, our pairing \( \pi\) can now be crossing, but we can use the commutation relations \( x_ix_j=x_jx_i\) , and the sum is again 1.
Thus the sum on the right is 1, in all cases, and we obtain:
On the other hand, another application of the Weingarten formula gives:
Thus, we are done with the proof of the result, in the real case. In the complex case the proof is similar, by adding exponents everywhere.
We can now deduce a useful characterization of the integration, as follows:
Theorem 283
There is a unique positive unital trace \( tr:C(S)\to\mathbb C\) satisfying
where \( \Phi\) is the coaction map of the corresponding quantum isometry group,
and this is the canonical integration, as constructed in Definition 16.31.
Proof
First of all, it follows from the Haar integral invariance condition for \( U\) that the canonical integration has indeed the invariance property in the statement, namely:
In order to prove now the uniqueness, let \( tr\) be as in the statement. We have:
On the other hand, according to Theorem 16.34, we have as well:
We therefore conclude that \( tr\) equals the standard integration, as claimed.
Getting back now to our axiomatization questions, we have:
Theorem 284
The operation \( S\to S_U\) produces a correspondence as follows,

between basic unitary groups and the basic noncommutative spheres.
Proof
We use the ergodicity formula from Theorem 16.35, namely:
We know that \( \int_U\) is faithful on \( \mathcal C(U)\) , and that we have:
The coaction map \( \Phi\) follows to be faithful as well. Thus for any \( x\in\mathcal C(S)\) we have:
Thus \( \int_S\) is faithful on \( \mathcal C(S)\) . But this shows that we have:
Thus, we are led to the conclusion in the statement.
Very nice all this, all the above is quite encouraging, we definitely have a beginning of free geometry theory here. It is possible to go much further, along these lines, and certain mathematicians have done indeed so. However, before explaining all this, we should perhaps doublecheck with the cats that all this is indeed relevant to quantum physics.
Unfortunately the cats are gone, so I will leave it to you, reader, clarifying such questions. And in case you believe it is so, do not hesitate to have a look at the free geometry, analysis, probability and so on literature, that might help you, in your quest.
Congratulations for having read this book, and no exercises for this final chapter. But, of course, with the hope to hear from you sometimes soon, with some heavy integration theorems of your own. The one who knows how to integrate wins.
[1] T. Banica, Calculus and applications (2024).
[2] W. Rudin, Principles of mathematical analysis, McGraw-Hill (1964).
[3] W. Rudin, Real and complex analysis, McGraw-Hill (1966).
[4] P. Lax and M.S. Terrell, Calculus with applications, Springer (2013).
[5] P. Lax and M.S. Terrell, Multivariable calculus with applications, Springer (2018).
[6] R.P. Feynman, R.B. Leighton and M. Sands, The Feynman lectures on physics I: mainly mechanics, radiation and heat, Caltech (1963).
[7] R.P. Feynman, R.B. Leighton and M. Sands, The Feynman lectures on physics II: mainly electromagnetism and matter, Caltech (1964).
[8] R.P. Feynman, R.B. Leighton and M. Sands, The Feynman lectures on physics III: quantum mechanics, Caltech (1966).
[9] D.J. Griffiths, Introduction to electrodynamics, Cambridge Univ. Press (2017).
[10] D.J. Griffiths and D.F. Schroeter, Introduction to quantum mechanics, Cambridge Univ. Press (2018).
[11] D.J. Griffiths, Introduction to elementary particles, Wiley (2020).
[12] S. Weinberg, Foundations of modern physics, Cambridge Univ. Press (2011).
[13] S. Weinberg, Lectures on quantum mechanics, Cambridge Univ. Press (2012).
[14] S. Weinberg, Lectures on astrophysics, Cambridge Univ. Press (2019).
[15] S. Lang, Algebra, Addison-Wesley (1993).
[16] T. Banica, Principles of operator algebras (2024).
[17] V.F.R. Jones, Subfactors and knots, AMS (1991).
[18] D.V. Voiculescu, K.J. Dykema and A. Nica, Free random variables, AMS (1992).
[19] V.I. Arnold, Ordinary differential equations, Springer (1973).
[20] V.I. Arnold, Mathematical methods of classical mechanics, Springer (1974).
[21] V.I. Arnold, Catastrophe theory, Springer (1974).
[22] V.I. Arnold, Lectures on partial differential equations, Springer (1997).
[23] V.I. Arnold and B.A. Khesin, Topological methods in hydrodynamics, Springer (1998).
[24] M.F. Atiyah, K-theory, CRC Press (1964).
[25] M.F. Atiyah, The geometry and physics of knots, Cambridge Univ. Press (1990).
[26] T. Banica, Introduction to modern physics (2025).
[27] R.J. Baxter, Exactly solved models in statistical mechanics, Academic Press (1982).
[28] I. Bengtsson and K. \.Zyczkowski, Geometry of quantum states, Cambridge Univ. Press (2006).
[29] N. Berline, E. Getzler and M. Vergne, Heat kernels and Dirac operators, Springer (2004).
[30] S.J. Blundell and K.M. Blundell, Concepts in thermal physics, Oxford Univ. Press (2006).
[31] S.M. Carroll, Spacetime and geometry, Cambridge Univ. Press (2004).
[32] A.R. Choudhuri, Astrophysics for physicists, Cambridge Univ. Press (2012).
[33] A. Connes, Noncommutative geometry, Academic Press (1994).
[34] J.B. Conway, A course in functional analysis, Springer (1985).
[35] W.N. Cottingham and D.A. Greenwood, An introduction to the standard model of particle physics, Cambridge Univ. Press (2012).
[36] P.A. Davidson, Introduction to magnetohydrodynamics, Cambridge Univ. Press (2001).
[37] P.A.M. Dirac, Principles of quantum mechanics, Oxford Univ. Press (1930).
[38] M.P. do Carmo, Differential geometry of curves and surfaces, Dover (1976).
[39] M.P. do Carmo, Riemannian geometry, Birkh\"auser (1992).
[40] S. Dodelson, Modern cosmology, Academic Press (2003).
[41] R. Durrett, Probability: theory and examples, Cambridge Univ. Press (1990).
[42] A. Einstein, Relativity: the special and the general theory, Dover (1916).
[43] L.C. Evans, Partial differential equations, AMS (1998).
[44] W. Feller, An introduction to probability theory and its applications, Wiley (1950).
[45] E. Fermi, Thermodynamics, Dover (1937).
[46] R.P. Feynman and A.R. Hibbs, Quantum mechanics and path integrals, Dover (1965).
[47] P. Flajolet and R. Sedgewick, Analytic combinatorics, Cambridge Univ. Press (2009).
[48] A.P. French, Special relativity, Taylor and Francis (1968).
[49] W. Fulton, Algebraic topology, Springer (1995).
[50] W. Fulton and J. Harris, Representation theory, Springer (1991).
[51] C. Godsil and G. Royle, Algebraic graph theory, Springer (2001).
[52] H. Goldstein, C. Safko and J. Poole, Classical mechanics, Addison-Wesley (1980).
[53] M.B. Green, J.H. Schwarz and E. Witten, Superstring theory, Cambridge Univ. Press (2012).
[54] G.H. Hardy and E.M. Wright, An introduction to the theory of numbers, Oxford Univ. Press (1938).
[55] J. Harris, Algebraic geometry, Springer (1992).
[56] R. Hartshorne, Algebraic geometry, Springer (1977).
[57] A. Hatcher, Algebraic topology, Cambridge Univ. Press (2002).
[58] H. Hofer and E. Zehnder, Symplectic invariants and Hamiltonian dynamics, Birkh\"auser (1994).
[59] L. H\"ormander, The analysis of linear partial differential operators, Springer (1983).
[60] R.A. Horn and C.R. Johnson, Matrix analysis, Cambridge Univ. Press (1985).
[61] K. Huang, Introduction to statistical physics, CRC Press (2001).
[62] K. Huang, Quantum field theory, Wiley (1998).
[63] K. Huang, Quarks, leptons and gauge fields, World Scientific (1982).
[64] J.E. Humphreys, Introduction to Lie algebras and representation theory, Springer (1972).
[65] K. Ireland and M. Rosen, A classical introduction to modern number theory, Springer (1982).
[66] L.P. Kadanoff, Statistical physics: statics, dynamics and renormalization, World Scientific (2000).
[67] T. Kibble and F.H. Berkshire, Classical mechanics, Imperial College Press (1966).
[68] M. Kumar, Quantum: Einstein, Bohr, and the great debate about the nature of reality, Norton (2009).
[69] T. Lancaster and K.M. Blundell, Quantum field theory for the gifted amateur, Oxford Univ. Press (2014).
[70] L.D. Landau and E.M. Lifshitz, Mechanics, Pergamon Press (1960).
[71] L.D. Landau and E.M. Lifshitz, The classical theory of fields, Addison-Wesley (1951).
[72] L.D. Landau and E.M. Lifshitz, Quantum mechanics: non-relativistic theory, Pergamon Press (1959).
[73] P. Lax, Linear algebra and its applications, Wiley (2007).
[74] P. Lax, Functional analysis, Wiley (2002).
[75] D. McDuff and D. Salamon, Introduction to symplectic topology, Oxford Univ. Press (2017).
[76] M.L. Mehta, Random matrices, Elsevier (2004).
[77] M.A. Nielsen and I.L. Chuang, Quantum computation and quantum information, Cambridge Univ. Press (2000).
[78] R.K. Pathria and and P.D. Beale, Statistical mechanics, Elsevier (1972).
[79] A. Peres, Quantum theory: concepts and methods, Kluwer (1993).
[80] P. Petersen, Linear algebra, Springer (2012).
[81] W. Rudin, Functional analysis, McGraw-Hill (1973).
[82] W. Rudin, Fourier analysis on groups, Dover (1974).
[83] B. Ryden, Introduction to cosmology, Cambridge Univ. Press (2002).
[84] D.V. Schroeder, An introduction to thermal physics, Oxford Univ. Press (1999).
[85] J. Schwinger and B.H. Englert, Quantum mechanics: symbolism of atomic measurements, Springer (2001).
[86] J.P. Serre, A course in arithmetic, Springer (1973).
[87] J.P. Serre, Linear representations of finite groups, Springer (1977).
[88] I.R. Shafarevich, Basic algebraic geometry, Springer (1974).
[89] R. Shankar, Fundamentals of physics, Yale Univ. Press (2014).
[90] A.M. Steane, Thermodynamics, Oxford Univ. Press (2016).
[91] S. Sternberg, Dynamical systems, Dover (2010).
[92] J.R. Taylor, Classical mechanics, Univ. Science Books (2003).
[93] J. von Neumann, Mathematical foundations of quantum mechanics, Princeton Univ. Press (1955).
[94] J. von Neumann and O. Morgenstern, Theory of games and economic behavior, Princeton Univ. Press (1944).
[95] J. Watrous, The theory of quantum information, Cambridge Univ. Press (2018).
[96] S. Weinberg, Cosmology, Oxford Univ. Press (2008).
[97] H. Weyl, The theory of groups and quantum mechanics, Princeton Univ. Press (1931).
[98] H. Weyl, The classical groups: their invariants and representations, Princeton Univ. Press (1939).
[99] H. Weyl, Space, time, matter, Princeton Univ. Press (1918).
[100] B. Zwiebach, A first course in string theory, Cambridge Univ. Press (2004).
I am normally hidden by the status bar