Sunday, November 30, 2008

Day 14: CM, Page 1

Day 14: The first page of Concrete Mathematics. "This chapter explores three sample problems that give a feel for what's to come... their solutions all use the idea of recurrence" (CM p.1)

The first page explains the Tower of Hanoi problem. "We are give a tower of eight disks, initially stacked in decreasing size on one of three pegs. The objective is to transfer the entire tower to one of the other pegs, moving only one disk at a time and never moving a larger one onto a smaller" (CM p.1). This is a famous problem in computer science. Most CS students are exposed to it at some point. The picture below shows a solution with four disks.


Here is a brief description of the legend behind the problem:
There is a legend about a Vietnamese or Indian temple which contains a large room with three time-worn posts in it surrounded by 64 golden disks. The priests of Brahma, acting out the command of an ancient prophecy, have been moving these disks, in accordance with the rules of the puzzle, since that time.... According to the legend, when the last move of the puzzle is completed, the world will end.
- wikipedia
There is no discussion of a solution on the first page, only a description of the problem. We'll save discussion of the solution for tomorrow.

Saturday, November 29, 2008

Day 13: CM, Page xii

Day 13: The last page of the table of contents and the last page before the main part of the book. The second half of the book covers binomial coefficients, "special numbers", generating functions, discrete probability, and asymptotics. I also just realized that there are 100 pages of answers and 30 pages of bibliography, and 20 pages of index. I hope that 150 days isn't too boring.

We've already looked at binomial coefficients a little, and "special numbers" (Stirling Numbers and Fibonacci Numbers), but what are generating functions and asymptotics?

"In mathematics a generating function is a formal power series whose coefficients encode information about a sequence an that is indexed by the natural numbers" (wikipedia).

An ordinary generating function is defined by:

The summation produces a sequence like:


The coefficients on the power series are 1, 1, 1, 1.... So this is the generating function for the sequence containing all ones. The sum's equality to 1/(1-x) holds for x in (-1,1). Not all generating functions will converge or have closed forms and some generating functions (like the one above) will only have closed forms for a range of numbers. For 1/(1-x) and x > 1, the sum clearly does not converge and does not match the closed form. However, for x < 1, the sum does converge. You can see that the equality holds under these conditions by multiplying both sides of the equation by (1-x):


Where m is a number approaching infinity. As m approaches infinity for x > 1, x^m approaches a giant number, so the closed form does not hold for |x| > 1. We'll get to learn much more about generating functions in a few hundred days.

Friday, November 28, 2008

Day 12: CM, Page xii

Day 12: The first page of table of contents. In the next 200 days or so we'll cover recurrent problems, sums, integer functions, number theory, and part of a chapter on binomial coefficients. A couple of sections jump out due to their fancy names.

Independent Residues: Not even wikipedia knows what this is.

Phi and Mu: Nor this. Unless it's talking about "the second oldest female fraternal organization established in the United States".

Thursday, November 27, 2008

Day 11: CM, Page xi

Day 11: A second page of notation. I'm going to practice my LaTeX on this page.

The link below is useful for LaTeX notation:
http://en.wikibooks.org/wiki/LaTeX/Mathematics

This is the notation for a Stirling cycle number (the "first kind"). "The notation of using brackets and braces, in analogy to the binomial coefficients, was introduced in 1935 by Jovan Karamata and promoted later by Donald Knuth; it is referred to as Karamata notation" (wikipedia). "Stirling numbers of the first kind (without the qualifying adjective unsigned) are the coefficients in the expansion [to the right] where (x)(n) is the falling factorial." They "count the number of permutations of n elements with k disjoint cycles" (wikipedia). In this context, "cycle" refers to a cyclic permutation. For example, the picture below shows a cyclic permutation on 4 objects with 2 cycles. So, in this case s(4,2) = 11.
How is this related to the formula above? The formula says that the falling factorial is equal to the coefficients on a formula that looks like ax^4 + bx^3 + cx^2 + dx^1 (for n=4). So... what are a, b, c, and d? For n=4, they are:


Notice that "11" appears as the coefficient of x^2. This matches the solution for s(4,2) above. So for s(4,1) and s(4,3), we should find six distinct cycles. And for s(4,4), we should find one. Let's verify assuming we have nodes a, b, c, d.

For s(4,1) we have:
abcd
abdc
acbd
acdb
adbc
adcb

And for s(4,3) we have:
a b cd
a bc d
ab c d
cd b a
d bc a
d c ab

And for s(4,4) we only have:
a b c d

So the cyclic permutations match the coefficients on the expansion of the falling fractorial. Just to confirm that the equation above holds, let's consider it for x=5. Sure enough, it works:

The "first kind" of Sterling numbers let us expand rising and falling factorials and count cyclic permutations. Pretty neat. We'll learn about these more in 260 days or so.

Wednesday, November 26, 2008

Day 10: CM, Page x

Day 10. This section is called "A Note on Notation". It has a discussion of some of the notation used later in the book. Let's look at some of the interesting notation.

is a subfactorial: n!/0! - n!/1! + ... + (-1)^n * n!/n!. A subfactorial "gives the number of permutations of a sequence of n distinct values in which none of the elements occur in their original place.... In practical terms, subfactorial is the number of ways in which n persons can each give one present to one other person so that everyone receives a present" (wikipedia).

ln x: natural logarithm: log base e of x. "The mathematical constant e is the unique real number such that the function ex has the same value as the slope of the tangent line, for all values of x." I got distracted reading about e and stumbled across Cantor's diagonal argument. I had forgotten about that from college. It's a great little observation about why the set of all sequences of 1s and 0s are not countable (i.e. there are more sequences of 1s and 0s than there are natural numbers).
s1 = (0, 0, 0, 0, 0, 0, 0, ...)
s2 = (1, 1, 1, 1, 1, 1, 1, ...)
s3 = (0, 1, 0, 1, 0, 1, 0, ...)
s4 = (1, 0, 1, 0, 1, 0, 1, ...)
s5 = (1, 1, 0, 1, 0, 1, 1, ...)
s6 = (0, 0, 1, 1, 0, 1, 1, ...)
s7 = (1, 0, 0, 0, 1, 0, 0, ...)
...
s0 = (1, 0, 1, 1, 1, 0, 1, ...)
Each sequences s1, s2, etc, above is some unique sequence of 1s and 0s. Each sequence is numbered from 1 out to infinity. s zero is constructed so that it is different from every other sequence in it's nth position. This suggests a mapping from the sequences of ones and zeros to the natural numbers cannot be constructed. For any sequence we can create and put in our list, we can create a sequence that is not in the list (s zero).

We'll get to the rest of the notation in more gory detail later. Here's a picture of Cantor:

Tuesday, November 25, 2008

Day 9: CM, Page ix

We have tried to produce a perfect book, but we are imperfect authors. Therefore we solicit help in correcting any mistakes that we've made. A reward of $2.56 will gratefully be paid to the first finder of any error, whether it is mathematical, historical, or typographical.
- Concrete Mathematics, p.ix

Day 9: The last page of the preface. We're almost to the real content of the book! The authors discuss the new font they developed for the book and they thank the various people who helped with the book.

Let's see how many of the people they thanked we can find on wikipedia.
  1. Andrei Broder: Now a VP at Yahoo.
  2. Ernst Mayr: Not on wikipedia. There is a biologist on wikipedia, but I don't think it is the same person.
  3. Andrew Yao: A prominent computer scientist. Won the Knuth Prize and the Turing Award.
  4. Frances Yao: Not on wikipedia.
  5. Donald Zeilberger: Mathematician who won the Euler Medal (Ron Graham also won one). Here are a couple of funny quotes: "People who believe that applied math is bad math are bad mathematicians" and "Guess what? Programming is even more fun than proving, and, more importantly it gives as much, if not more, insight and understanding". And if you follow the links, the opinion pieces are funny too: "I was just kidding. They are not bad mathematicians, because they are not mathematicians at all. A true mathematician has respect for all parts of mathematics, and does not believe in arbitrary divisions into `Pure', `Applied', `Theoretical',`Practical', `Conceptual', `Computational'. Mathematics is a Web, an infinite dimensional tapestry with everything intertwined.". And "Now, Hear Yea, all you self-righteous defenders of insight and understanding. A good proof may give the appearance of understanding, but IN ORDER TO UNDERSTAND SOMETHING REALLY DEEPLY, YOU SHOULD PROGRAM IT."
That's it for the preface. Here's a picture of Donald Zeilberger.

Monday, November 24, 2008

Day 8: CM, Page viii

Research Problems may or may not be humanly solvable, but the ones presented here seem to be worth a try (without time pressure).
- Concrete Mathematics, p.viii
Day 8: Another easy preface page. On this page, the authors describe the different types of problems. The problems range in difficulty from Warmup, to Basic, to Homework, to Exam, to Bonus, to Research. It sounds like the first four categories are solvable and the last two are less solvable, with the research problems possibly not "humanly solvable". My goal is to do the Warmup, Basic, Homework, and Exam problems. I'll probably need to spend more than a day on each of those pages.

Sunday, November 23, 2008

Day 7: CM, Page vii

... a concrete life preserver thrown to students sinking in a sea of abstraction.
- W. Gottschalk
Day 7: The third page of the preface. Today is another easy day. The authors describe the comments in the margins of the book. The margins contain a mix of quotations from prominent mathematicians and the students in the class. "Students always know better than their teachers, so we have asked the first students of this material to contribute their frank opinions, as "graffiti" in the margins" (CM p.vii). For example, "I have only a marginal interest in this subject". Many of the comments in the margins are humorous or corny, but many are also pragmatic and provide hints or things to watch out for.

Six more pages of preface, notation, and table of contents. Then we'll actually begin with the book!

On the side, I'm trying to get LaTeX working on my laptop. I'm going for a transparent background so I can get the blog converted back to black. Here's a test:


Hmm... the editor has a white background... I guess I'll stick to the white background.

Saturday, November 22, 2008

Day 6: CM, Page vi

But what exactly is Concrete Mathematics? It is a blend of CONtinuous and disCRETE mathematics. More concretely, it is the controlled manipulation of mathematical formulas, using a collection of techniques for solving problems.
- Concrete Mathematics, page vi
Day 6: The second page of the preface. The authors explain that although Concrete Mathematics was partially a negative reaction to the new math movement, "the main reasons for its existence were positive instead of negative" (CM p.vi). One of the main goals of Concrete Mathematics is to give students a set of tools for solving real world problems.

So what do we have to look forward to? "The major topics treated in this book include sums, recurences, elementary number theory, binomial coefficients, generating functions, discrete probability, and asymptotic methods" (CM p.vi). Hey, that sounds good! But... what does it mean?

Summation

Summation is adding numbers together. Summation uses everybody's favorite mathematical symbol, the symbol on the cover of CM:


For the programmers out there, you can think of this roughly as a for loop summing numbers in an array:

for (int i = m; i <= n; i++) {
sum += x[i];
}

Here's a "concrete" example:


Often summation is more similar to summing over a function than a pre-calculated sequence of numbers in an array. This example is similar to a for loop like:

for (int k = 2; k <= 6; k++) {
sum += square(k);
}

And of course, there are many variations on the syntax above. For example, the loop boundaries may be omitted if they are clear from the context, or the loop may be over a set rather than a sequence.

Recurrences

Recurrences are recursive functions. Recursive functions include themself in their own definition.The Fibonacci Sequence is probably the most famous example:
With seeds values of 1 and 1 for n=0 and n=1, the Fibonacci Sequence is 1, 1, 2, 3, 5, 8, 13, 21, 34,... Fractals are also often based on recurrences. The Mandlebrot fractal is one of the most famous examples.

Elementary Number Theory

According to wikipedia:
In elementary number theory, integers are studied without use of techniques from other mathematical fields. Questions of divisibility, use of the Euclidean algorithm to compute greatest common divisors, integer factorizations into prime numbers, investigation of perfect numbers and congruences belong here. Several important discoveries of this field are Fermat's little theorem, Euler's theorem, the Chinese remainder theorem and the law of quadratic reciprocity. The properties of multiplicative functions such as the Möbius function and Euler's φ function, integer sequences, factorials, and Fibonacci numbers all also fall into this area.
That all sounds familiar... but it's been a while.

Binomial Coefficients

Binomial Coefficients are so named for the binomial theorem (which we will probably return to later). They are read as "n choose k" and intuitively denote the number of ways k subsets can be chosen from a set of size n. Binomial coefficients are defined as:

To understand this formula, first start my imagining how to build unique k-sized sequences from a set of size n. You have n choices for the first element, n-1 choices for the second element, and so on down to (n - k + 1). So the probability of choosing a particular k-sized sequence is n * (n - 1) * ... * (n - k + 1). However, we are actually interested in distinct sets of elements. The same k elements in our particular sequence could also be picked in a different order and still form the same set. So how many different sequences could those k elements be picked in? k! - the number of permutations of a sequence of size k. Each of those k! sequences has an equal probability of being picked. So that particular set has a probability of being selected of k!/(n * (n - 1) * ... (n - k + 1)). Since each set has an equal opportunity of being selected, the total number of sets is the inverse of the probability of selecting a particular set, or (n * (n - 1) * (n - k + 1))/k!.

Binomial Coefficients produce many interesting patterns. For example, Pascal's Triangle is a way to calculate the binomial coefficient for any combination of n and k.

0:







1







1:






1
1






2:





1
2
1





3:




1
3
3
1




4:



1
4
6
4
1



5:


1
5
10
10
5
1


6:

1
6
15
20
15
6
1

7:
1
7
21
35
35
21
7
1
8: 1
8
28
56
70
56
28
8
1

"Row number n contains the numbers  \tbinom n k for k = 0,…,n. It is constructed by starting with ones at the outside and then always adding two adjacent numbers and writing the sum directly underneath" (wikipedia).

Within Pascal's Triangle there are also interesting patterns. For k=2, the numbers are triangular. In other words they can be drawn as a triangle. For example:

* = 1

** = 3
*

*** = 6
**
*

**** = 10
***
**
*

In the example above, the first four triangular numbers are shown: 1, 3, 6, 10. The triangular numbers can be calculated with the formula n * (n + 1) / 2. In fact, there is famous story about the mathematician, Gauss, who discovered this as a young child:

... Gauss, mathematician extrodanaire, had a lazy teacher. The “educator” wanted to keep the kids busy so he could take a nap — he asked the class to add the numbers 1 to 100 - no small feat when most kids are still learning how to carry.

Gauss walked up and showed the answer on his blackboard: 5050. The teacher thought he had cheated, but no, Gauss had figured it out by sidestepping the problem. Manual addition is for suckers, Gauss had a better way:

Sum from 1 to n = n*(n+1)/2

You can understand the formula intuitively by thinking of it as a rectangle of size n and n+1 cut in half to form a triangle. This triangle, in turn, is formed of rows of size 1, 2, 3, ... n. Adding up the rows of the triangle is the same as summing from 1 to n which is the same as n(n+1)/2.

Interestingly, there is also a Triangular pattern within Pascal's Triangle for k=3. It is the sum of triangular numbers from 1 to n-2. For example, consider n=6, k=3. The binomial coefficient is 6!/3!(6-3)! = 6!/3!*3! = 6 * 5 * 4 / 3 * 2 * 1 = 120 / 6 = 20. This is also the same as the sum of the first four triangular numbers: 1 + 3 + 6 + 10 = 20. So the list of numbers in Pascal's Trinalge for k=3 and n>2 can be defined as:
Similarly, for k=4 and n>3, we have a nested summation of triangular numbers:
And for k=5 and n>4:
A pattern is starting to emerge. I'm not going to generalize on k or try to prove it for now. It's amazing that such a complicated summation can be represented by such a compact function. The results aren't something you would guess immediately, but it makes some sense if you imagine building k sized sets by iterating over an array. For k=2, you hold one pointer constant (point it at the first position) and iterate a second pointer over the rest of the elements. This forms the first row in the triangle. Next you move your first pointer to the second position. Then you iterate your second pointer over the remaining elements. This forms the second row of the triangle - it will be one element smaller than the previous row. If you repeat, it will form an entire triangle. For k=3, the pattern is similar. You hold two pointers constant and iterate a third pointer. That forms one row of a triangle. You increment your second pointer and iterate your third pointer again. This forms a second row of a triangle. You repeat until both the second and third pointers hit the end of the array. At this point, you've formed one triangle. You bump the first pointer and repeat. This begins forming a new triangle that will be one unit smaller than the previous triangle. A similar triangular pattern will be true for k=4, k=5, and so on. As you form sets by iterating over the array, each iteration creates a triangle. So all Binary Coefficients with k > 1 and k < onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://mathforum.org/dr.math/faq/pascalfib.gif">
You can even create a Sierpinski Gasket by removing odd numbers from Pascal's Triangle. For more information about Sierpinski Gaskets and Cantor Dust, check out this link.



That's all I have time for today. We'll have to save generating functions, discrete probability, and asymptotic methods for later.

Day 5: CM, Page v

The course title "Concrete Mathematics" was originally intended as an antidote to "Abstract Mathematics," since concrete classical results were rapidly being swept out of the modern mathematical curiculum by a new wave of abstract ideas popularly called the "New Math".
- Concrete Mathematics, page v
Day 5: The first page of the preface of CM. From here on out there won't be much more biographical information. We'll be getting into the meat of the book. But before that, we have a bit more honeymoon with the preface.

The first page of the preface discusses some of the changes that were happening in the educational system when the book was written in the 60s and 70s. The quote above claims that "Concrete Mathematics" was an "antidote" to "abstract mathematics" or "new math". So what's the difference between them?

According to wikipedia, pure mathematics is "motivated entirely for reasons other than application. It is distinguished by its rigour, abstraction and beauty." New math was an educational movement in the 1960s, introduced after the Spunik crisis, that attempted to provide students with a better mathematical foundation. It focused on abstract and pure mathematical concepts:
New Math emphasized mathematical structure through abstract concepts like set theory and number bases other than 10. Beginning in the early 1960s the new educational doctrine was installed, not only in the USA, but all over the developed world.... For example, in some cases pupils were taught axiomatic set theory at an early age. The idea behind this was that if the axiomatic foundations of mathematics were introduced to children, they could easily cope with the theorems of the mathematical system later.
- wikipedia
Not surprisingly, there was some backlash against new math. Parents and teachers were concerned that the material was too abstract and that children weren't learning the basics. For example, "Professor George F. Simmons wrote that the New Math produced students who had 'heard of the commutative law, but did not know the multiplication table'" (wikipedia), and Morris Kline wrote a book called "Why Johnny Can't Add: the Failure of the New Math".

Concrete Mathematics was also a part of the backlash. It eschews abstract mathematics and attempts to provide a pragmatic foundation for computer scientists and programmers.
The end-users of mathematics studies were at that time mostly in the physical sciences and engineering; and they expected manipulative skill in calculus, rather than more abstract ideas. Some compromises have since been required, given that discrete mathematics is the basic language of computing.
- wikipedia
New math is even mocked in popular culture. It was mentioned in a Simpson episode (Dog of Death) when when "Principal Skinner describes the new textbooks he'd like to use: 'History books that know how the Korean War came out… math books that don't have that base six crap in them…'" (wikipedia).



Tom Lehrer wrote a song entitle "New Math" that captures the frustration people felt. It "centered around the process of subtracting 173 from 342 in base 10 and in base 8. The song is in the style of a lecture about the general concept of subtraction in arbitrary number systems, illustrated by two simple calculations, and highlights the emphasis on insight and abstract concepts of the New Math approach."

Excerpt:

So you've got thirteen 10s and you take away seven
And that leaves five ... well, six, actually.
But the idea is the important thing.

Chorus:

Hooray for new math,
New-hoo-hoo-math,
It won't do you a bit of good to review math.
It's so simple,
So very simple,
That only a child can do it!
Concrete Mathematics was developed as an "antitode" to more abstract mathematics and the new math movement. So we can look forward to lots of pragmatic, non-abstract math! But first the rest of the preface...

Wednesday, November 19, 2008

Day 4: CM, Page iv

Richard M. Stallman, Linus Torvalds, and Donald E. Knuth engage in a discussion on whose impact on the computerized world was the greatest.
Stallman: "God told me I have programmed the best editor in the world!"
Torvalds: "Well, God told *me* that I have programmed
the best operating system in the world!"
Knuth: "Wait, wait - I never said that."

From rec.humor.funny.

submitted by ermel@gmx.de (Erik Meltzer)

Day 4. Another easy page. The bibliographic information. Copyright '94, and '89. Nothing else too interesting, so let's take some time and get to know Knuth a little better. Here's a good link with information about Knuth. I will pull out some of the highlights, but go read the link for more. http://www.softpanorama.org/People/Knuth/index.shtml

Here is one of the earlier anecdotes about Knuth. Even when he was young, he was a problem solver:
One episode, repeated in most biographies of Knuth but still worth repeating in this one, concerns "Ziegler's Giant Bar." He entered a competition by the confectionary manufacturer Ziegler to make as many words with the letters from the phrase "Ziegler's Giant Bar" as possible. Donald spent two weeks during which he pretended to be ill and came up with 4500 words. The judges for the competition had only found 2500.
Not surprisingly, Knuth was a strong student, but had doubts about his mathematical abilities:
According to www.digitalcentury.com encyclopedia article about him: Although Knuth accomplished the highest grade point average in the history of his high school, he and his advisors had doubts as to his ability to succeed at college math. Knuth reports having something of an inferiority complex during his high school and early college years, a problem to which he attributes his over-achiever status. As a college freshman he would spend hours working extra problems in math out of fear that he would fail, only to find after a few months that his abilities far exceeded any of his colleagues.
He proved to be a strong mathematician:
Here is a relevant quote from www.digitalcentury.com encyclopedia: Knuth’s plan to become a music major changed when he was offered a physics scholarship at Case Institute of Technology (now Case Western Reserve). His turn toward math came during his sophomore year when a particularly difficult professor assigned a special problem, offering an immediate "A" in the class to any student who could solve it. Although like most of the other students, Knuth considered the problem unsolvable, he made an attempt at it one day when he found himself with some free time, having missed the bus that was taking the marching band to a performance. By what he states was "a stroke of luck," he solved the problem in short order, earned his "A," and skipped class for the rest of the semester. Although Knuth reports that he felt guilty about skipping class, he was obviously able to overcome the lost instruction because the following year he earned an "A" in abstract mathematics and was given the job of grading papers for the very course he had failed to attend.
And eventually got his PhD - he solved his thesis problem in an hour:
I got a listing from a guy at Princeton who had just computed 32 solutions to a problem that I had been looking at for a homework problem in my combinatorics class. I was riding up on the elevator with Olga Todd, one of our professors, and I said, "Mrs. Todd, I think I'm going to have a theorem in an hour. I am going to psyche out the rule that explains why there happen to be 32 of each kind." Sure enough, an hour later I had seen how to get from each solution on the first page to the solution on the second page. I showed this to Marshall Hall. He said, "Don, that's your thesis. Don't worry about this block design with =2 business. Write this up instead and get out of here." So that became my thesis. And it is a good thing, because since then only one more design with =2 has been discovered in the history of the world. I might still be working on my thesis if I had stuck to that problem. But I felt a little guilty that I had solved my Ph.D. problem in one hour, so I dressed it up with a few other chapters of stuff.
With Knuth's ability, he had a lot of options. He eventually decided he wasn't going to "optimize [his] income":
I had learned about the Burroughs 205 machine language, and it was kind of appealing to me. So I made my own proposal to Burroughs. I said, "I'll write you an ALGOL compiler for $5,000. But I can't implement all of ALGOL for this; I am just one guy. Let's leave out procedures." Well, this is a big hole in the language! Burroughs said, "No, you've got to put in procedures." I said, "Okay, I will put in procedures, but you've got to pay me $5,500." That's what happened. They paid me $5,500, which was a fairly good salary in those days. So between graduating from Case and going to Caltech, I worked on this compiler. Heading out to California, I drove 100 miles each day and then sat in a motel and wrote code. Then a startup company came to me and said, "Don, write compilers for us and we will take care of finding computers to debug them. Name your price." I said, "Oh, okay, $100,000," assuming that this was [outrageous]. The guy didn't blink. He agreed. I didn't blink either. I said, "I'm not going to do it. I just thought that was an impossible number." At that point I made the decision in my life that I wasn't going to optimize my income.
His work on compilers eventually morphed into TAOCP:
In 1962 Addison-Wesley proposed that he write a book on compiler construction. That was a much larger and much more important field that it is now. Indeed, all computer science research in those days was pretty much carved up into three parts: numerical analysis, artificial intelligence, programming languages and their compilers. The offer was accepted and lead to now famous "The Art of Computer Programming" which is definitely much more then the book on computer construction and related algorithms. He began that project in the summer of 1962 one year before graduation and being married to his wife just four months.
Here's what Knuth had to say about it:
A man from Addison-Wesley came to visit me and said "Don, we would like you to write a book about how to write compilers." I thought about it and decided "Yes, I've got this book inside of me." That day I sketched out—I still have that sheet of tablet paper—12 chapters that I thought should be in such a book. I told my new wife, Jill, "I think I'm going to write a book." Well, we had just four months of bliss, because the rest of our marriage has all been devoted to this book. We still have had happiness, but really, I wake up every morning and I still haven't finished the book. So I try to organize the rest of my life around this, as one main unifying theme.
He wrote the first volume of TAOCP by the age of 28:
[Knuth] wrote one of the most compact Algol compilers at the age of 22 and published the first volume of The Art of Computer Programming at the age of 28. His three volumes of The Art of Computer Programming played an important role in establishing and defining Computer Science as a rigorous, intellectual discipline.
After getting his PhD and finishing the first volume, he became a professor at Stanford. He already had a reputation:
Even among computer pioneers in Stanford, Donald Knuth was considered to be almost legendary figure. The following old anecdote from Alan Kay, one of the principal designers of the Smalltalk language, illustrates the point: When I was at Stanford with the AI project [in the late 1960s] one of the things we used to do every Thanksgiving is have a computer programming contest with people on research projects in the Bay area. The prize I think was a turkey. [John] McCarthy used to make up the problems. The one year that Knuth entered this, he won both the fastest time getting the program running and he also won the fastest execution of the algorithm. He did it on the worst system with remote batch called the Wilbur system. And he basically beat the shit out of everyone. And they asked him, "How could you possibly do this?" And he answered, "When I learned to program, you were lucky if you got five minutes with the machine a day. If you wanted to get the program going, it just had to be written right. So people just learned to program like it was carving stone. You sort of have to sidle up to it. That's how I learned to program."
Knuth finished the second and third volume of TAOCP as well as creating the TeX typesetting language. And of course he has done many other things:
He really has managed to make contributions to a very wide area of computer science topics. Among them:
- Suggested the name "Backus-Naur Form"
- Wrote one of first and one of the most compact Algol compilers;
- Designed the SOL simulation language;
- Designed and wrote the TeX and Metafont systems of documentation (written in WEB for Pascal);
- Introduced and refined the LR parsing algorithm (see Knuth, D.E. 1965. "On the Translation of Languages from Left to Right", Information and Control, Vol. 8, pp. 607-639. The first paper on LR parsing.)
- Invented several important algorithms including Knuth-Morris-Pratt string-searching algorithm;
- Made the crucial contribution to the "Structured programming without GOTO" programming debate, which was a decisive blow to the structured programming fundamentalists led by E. Dijkstra;
- Published his groundbreaking paper "An empirical study of FORTRAN programs." ( Software---Practice and Experience, vol 1, pages 105-133, 1971) which laid the cornerstone of an empirical study of programming languages.
- Made the crucial contribution to dispelling "correctness proof" mythology.
But the TAOCP is his masterpiece:
Still no other book in CS can be compared with his encyclopedic The Art of Computer Programming (TAoCP). It is one of the few books that remain largely relevant 30 years after the initial publication. I am convinced that every person who suspects that he/she has a programming abilities (and BTW an early start is a good indication of abilities in any field, including programming) should buy or borrow the first volume of TAoCP and try to solve exercises after each chapter. The ability to solve difficult exercises form the first volume is a sure sign of people with substantial talent and can be a good self-test if you it make sense to dedicate yourself to carrier in computer science or not.
He is continuing to work on the forth volume. Hopefully, he will finish it soon (there are rumors of 2008). But he admits he may not be able to finish his original ambitious plan:
His primary agenda item is, after all, Volume 4, although every few months he’ll browse journals for items to squirrel away for Volumes 5 through 7. People inquire delicately about how he expects to complete a seven-volume project at his current pace, and Knuth concedes he’ll be content to finish the first five.
How do your gain expertise in something?
By studying the masters and not their pupils. Abel, Niels H. (1802 - 1829)
So let's get started studying the master!

Day 3: CM, Page iii

If you optimize everything, you will always be unhappy.
Donald Knuth

It would be very discouraging if somewhere down the line you could ask a computer if the Riemann hypothesis is correct and it said, 'Yes, it is true, but you won't be able to understand the proof.'
Ronald Graham
% This program is copyright (C) 1985 by Oren Patashnik; all rights reserved.
% Copying of this file is authorized only if (1) you are Oren Patashnik, or if
% (2) you make absolutely no changes to your copy.
Oren Patashnik

Day 3: The third page of the introduction to Concrete Mathematics.
Another title page, but this time with the authors: Ronald L. Graham,
Donald E. Knuth, and Oren Patashnik. Let's get to know the authors.

Ronald L. Graham

I bet you would't have guessed that Ronald Graham is the Ex President of the International Jugglers Association? There is a long history of computer scientists and juggling. No.. really. Look here, and here, and here, and you have to watch this one here. That last one is good.

But wait, there's more:
In college days, Ron was part of a circus act, called the Bouncing Bears. He was on stage with Cirque du Soleil and in an issue of Discover magazine about the Science of the Circus. He was a qualified judge for international trampoline competitions and has a unique bungee trampoline for daily exercise.
- Ron Graham's Special Page

I'm not kidding. Check this out:


Oh, and he did some computer science stuff too:
Several mathematical areas were started by Ron's work, such as worst case analysis in scheduling theory, on-line algorithms and amortized analysis in the Graham's scan in Computational Geometry, and of course, his favorite topics on Ramsey Theory, and the recent work on quasi-randomness. Ron's mathematics was highlighted in the nomination article written by Gian-Carlo Rota for the first contested election of AMS President. (He won ). He received the Steele Prize for Lifetime Achievement in 2003.
- Ron Graham's Special Page
And of course helped write the wonderful book we are about to read.

Donald E. Knuth
So who is this Donald Knuth that the blog is named after?

Author of the seminal multi-volume work The Art of Computer Programming[3] Knuth has been called the "father" of the analysis of algorithms, contributing to the development of, and systematizing formal mathematical techniques for, the rigorous analysis of the computational complexity of algorithms, and in the process popularizing asymptotic notation. ("TAOCP"),

In addition to fundamental contributions in several branches of theoretical computer science, Knuth is the creator of the TeX computer typesetting system, the related METAFONT font definition language and rendering system, and the Computer Modern family of typefaces.

-wikipedia

Not too shabby. And here's a picture:


Donald Knuth has a reputation for having a good sense of humor.

He used to pay a finder’s fee of $2.56 for any typographical errors or mistakes discovered in his books, because “256 pennies is one hexadecimal dollar”.
-wikipedia

He unfortunately had to stop due to bank fraud. Here' s a good anecdote about Concrete Mathematics from its preface:

“When Knuth taught Concrete Mathematics at Stanford for the first time, he explained the somewhat strange title by saying that it was his attempt to teach a math course that was hard instead of soft. He announced that, contrary to the expectations of some of his colleagues, he was notTheory of Aggregates, nor Stone’s Embedding Theorem, nor even the Stone-Čech compactification. (Several students from the civil engineering going to teach the department got up and quietly left the room.)”

Donald Knuth is one of the fathers of computer science. Unfortunately, not many people make it through all of the books in TAOCP. I have personally only made it about 3 pages in. The books are well written, but the material can be difficult.

If you think you're a really good programmer. . . read [Knuth's] Art of Computer Programming.... You should definitely send me a resume if you can read the whole thing. -Bill Gates

Oren Patashnik

Oren seems to be the least famous of the three. He was a graduate student of Knuth's and assisted Graham and Knuth in the writing. According to wikipedia:

Oren Patashnik (born 1954) is a computer scientist. He is notable for co-creating BibTeX, and co-writing Concrete Mathematics: A Foundation for Computer Science. He is a researcher at the Center for Communications Research, La Jolla.

And I thought this was a nice highlight:
In 1980, he used 1500 hours of computer time to prove that Qubic (a sort of 3-Dtic-tac-toe) can always be won by the first player.

So those are our three authors. And finally, the book is published by Addison-Wesley:
Addison-Wesley is a book publishing imprint of Pearson PLC, best known for computer books. As well as publishing books, Addison-Wesley distributes its technical titles through the Safari Books Online e-reference service.
-wikipedia

Tuesday, November 18, 2008

Day 2: CM, Page ii

To put it bluntly, I seem to have a whole superstructure with no foundation. But I'm working on the foundation.
-
Marilyn Monroe

Computer science is no more about computers than astronomy is about telescopes.
-
Edsger Dijkstra
Day 2. page ii. Another easy page. The subtitle. Only five words: A Foundation for Computer Science.

We all know what a foundation is:
foundation (plural foundations): That upon which anything is founded; that on which anything stands, and by which it is supported; the lowest and supporting layer of a superstructure; groundwork; basis.
- wiktionary
And we all know what computer science is:
computer science: The study of computers and their architecture, languages, and applications, in all aspects, as well as the mathematical structures that relate to computers and computation.
- wiktionary
But did you know:
Despite its name, a significant amount of computer science does not involve the study of computers themselves. Because of this, several alternative names have been proposed. Danish scientist Peter Naur suggested the term datalogy, to reflect the fact that the scientific discipline revolves around data and data treatment, while not necessarily involving computers. The first scientific institution to use the term was the Department of Datalogy at the University of Copenhagen, founded in 1969, with Peter Naur being the first professor in datalogy. The term is used mainly in the Scandinavian countries. Also, in the early days of computing, a number of terms for the practitioners of the field of computing were suggested in the Communications of the ACMturingineer, turologist, flow-charts-man, applied meta-mathematician, and applied epistemologist.[17] Three months later in the same journal, comptologisthypologist.[18] The term computics has also been suggested.[19] Informatik was a term used in Europe with more frequency.
- wikipedia
And I'm sure you know:
The relationship between computer science and software engineering is a contentious issue, which is further muddied by disputes over what the term "software engineering" means, and how computer science is defined. David Parnas, taking a cue from the relationship between other engineering and science disciplines, has claimed that the principal focus of computer science is studying the properties of computation in general, while the principal focus of software engineering is the design of specific computations to achieve practical goals, making the two separate but complementary disciplines.[21]
- wikipedia
So what do you do for a living? I'm a applied epistemologist. Oh wait.. I guess I'm just a software engineer.


Monday, November 17, 2008

Day 1: CM, Page i

Sing to me of the man, Muse, the man of twists and turns
driven time and again off course, once he had plundered
the hallowed heights of Troy.
Many cities of men he saw and learned their minds,
many pains he suffered, heartsick on the open sea,
fighting to save his life and bring his comrades home.
But he could not save them from disaster, hard as he strove -
the recklessness of their own ways destroyed them all,
the blind fools, they devoured the cattle of the Sun
and the Sungod blotted out the day of their return.
Launch out on his story, Muse, daughter of Zeus,
start from where you will - sing from our time too.
- The Odyssey


Day 1. Page i. Concrete Mathematics: A Foundation For Computer Science. Graham, Knuth, Patashnik. The prologue to the old testament of computer science. Its cover is imprinted with a giant Σ. It's ominous and absolute. It's mysterious. It's like the moon. I have a plan. I'm going to read all the Knuth books starting with the Concrete Mathematics (CM). One page a day. For 2907 days. To be completed on Nov 3, 2016. Brilliant!

Page i is easy. The title page: CONCRETE MATHEMATICS Second Editition. Dedicated to Leonhard Euler (1707 - 1783). Done! No sweat! 2906 pages to go.

Wait a minute. What is concrete mathematics? According to wikipedia, it's... it's this book. That's not very helpful. What else? It "'is a blend of CONtinuous and disCRETE mathematics' and calculus is frequently used in the explanations and exercises" (uh oh. I don't remember much calculus). And "the term is also used to denote the opposite of abstract mathematics" (that sounds good). And of, course, the cover is a picture of concrete with a mathematical symbol. Get it? Concrete mathematics! Second edition? The first edition was published in 1988. The second edition was published in 1994. There is no third edition. Dedicated to Euler? Why Euler?
Euler made important discoveries in fields as diverse as calculus and graph theory. He also introduced much of the modern mathematical terminology and notation, particularly for mathematical analysis, such as the notion of a mathematical function.[3] He is also renowned for his work in mechanics, optics, and astronomy.
Ok. But what else?
Euler is considered to be the preeminent mathematician of the 18th century and one of the greatest of all time. He is also one of the most prolific; his collected works fill 60–80 quarto volumes.[4] A statement attributed to Pierre-Simon Laplace expresses Euler's influence on mathematics: "Read Euler, read Euler, he is the master [i.e., teacher] of us all."[5]
Ok! Maybe when I'm done with Knuth. This is Euler. Maybe we'll have a picture of Knuth tomorrow.