Mardia, Kent, Bibby. Multivariate Analysis

Question 1.6.1 and Solution.
Pretty easy, but worth hands on practice.








Solutions to Question 2.2.1 to 2.2.5

Subject to Moment Constraints

Simple Exponential Families has the maximum entropy.

A proof can be seen at

http://www.math.nyu.edu/faculty/kleeman/infolect6.pdf

Matrix Calculus

http://www.ee.ic.ac.uk/hp/staff/dmb/matrix/intro.html

A couple of troubles

OpenCV doesn't work well on Vista.
Solved, don't link to highguid.lib.

Function icvCalcOpticalFlowLK_8u32fR doesn't support window size greater or equal to 16. Optimization of code is so difficult to read and modify, so just abandon it.

LK optical flow algorithms in OpenCV and UCF's matlab code produce different results. I have to further investigate what's going on here.

Problem of illumination alignment

Windows Live photo album provide free photo stitching function, it works well, but it seems that uneven exposure across the images, especially when the sun is in the image would cause problem.
I doubt that sun suppress the image dynamic range, so it's pretty hard to align the illumination.

Worth Reading

Bruce D. Lucas. An Iterative Image Registration Technique with an Application to Stereo Vision. 1981
comment: Expand the error function using Taylor's equation. Simple yet effective method.

Worth Reading

David A. Forsyth et al., Computational Studies of Human Motion: Part1, Tracking and Motion Synthesis.
comment: a good book to review the works in this area.

See P108, this book gives an excellent example of a simple non-linear system dynamic demonstrating multi-model behavior .I managed to verify this model by the following MATLAB script:

X0=randn(10000,1);
k=100
epsilon=1e-3;
for ii=1:k
X0=X0+epsilon*sin(X0);
end
hist(X0);

As k is small (=100), the system is still uni-model.

k=10000, there's two models

A solution to Question 5.1.8

John E. Hopecroft, Introduction to Automata Theory, Languages, and Computation. 2nd Edition.

Consider the CFG G defined by productions

S->aSbS|bSaS|e

Prove that L(G) is the set of all strings with an equal number of a's and b's.

The point of inductive proof is that for arbitrary string s with equal a's and b's longer than k,k>=1, we can always find substrings x,y such that s=axby or s=bxay and x,y has equal number of a's and b's respectively.

You can click on the following image to see a zoomed version if it's too small. (Please don't copy it for your homework)


My proof here is clumsy, hope someone would show me an elegant one.

I found another solution by accident today (March 21, 2008) at http://www.cs.iit.edu/~cs532/HW/HW_04_Solution.pdf

The idea is almost the same, and it's also wordy.

Office Hour

Know, that I am one among the heretical Jann and I sinned against Sulayman, David son (on the twain be peace!) I together with the famous Sakhr al Jinni;"[FN#71] whereupon the Prophet sent his minister, Asaf son of Barkhiya, to seize me; and this Wazir brought me against my will and led me in bonds to him (I being downcast despite my nose) and he placed me standing before him like a suppliant. When Sulayman saw me, he took refuge with Allah and bade me embrace the True Faith and obey his behests; but I refused, so sending for this cucurbit[FN#72] he shut me up therein, and stopped it over with lead whereon he impressed the Most High Name, and gave his orders to the Jann who carried me off, and cast me into the midmost of the ocean. There I abode an hundred years, during which I said in my heart,

"Whoso shall release me, him will I enrich for ever and ever." But the full century went by and, when no one set me free, I entered upon the second five score saying,

"Whoso shall release me, for him I will open the hoards of the earth." Still no one set me free and thus four hundred years passed away. Then quoth I,

"Whoso shall release me, for him will I fulfil three wishes." Yet no one set me free. Thereupon I waxed wroth with exceeding wrath and said to myself,

Topology and Borel Sets

Borel Sets is the smallest cosi-algebra of a topology, so it contains more than the 'open sets' of topology.
In Rudin, Real and Complex Analysis, proof of a lot of theorems start with the 'open sets' and then use the bounding open sets and compact sets to prove for the other sets.
Say, for arbitrary Borel set E, find a compact set KE, s.t. u(V-K)=0. So K and V are actually very tight bondings to the measure of E.
Then this bounding can be used in inequalities. See Riesz Representation Theorem and theorems 2.17, 2.18.

Rudin, Real and Complex Analysis

p44

By Urysohn's lemma, there exists f belongs to Cc(X) s.t. f(x)=1 on K1, f(x)=0 on K2....

comment:

One should noted a property of Hausdorff space, disjointed compact sets of a Hausdorff space is always separated, which means for K1 and K2, there are open sets V1 and V2 that K1 included by V1, K2 included by V2 and V1 and V2 has no common points. Then Urysohn's lemma can be applied and the preceding conclusion can be drawn.

further comment:

This book should be read in parallel with Halmos' book and an introductory topology textbook.

Rudin, Real and Complex Analysis

p35

Moreover, it turns out that the euclidean properties of Rn play no role in the proof; in fact, if one thinks of them too much they just get in the way....

Ye, right, that's why they are constantly creating new concepts and words to keep things simple

Worth Reading

Lena Gorelick et atl. Shape Representation and Classification Using the Poisson Equation.

comment: A new transform of silhouette other than Distant Transform.
How can these guys find so many ideas from Physics, Maths and other disciplines?
Am I suppose to be a know-it-all to finish my PH.D.?
To dig out some fundamental theories is more fun, but it's a daunting task.

Moshe Blank, Lena Gorelick et al. Actions as Space-Time Shapes.
comment: Lena Gorelick

Eli Shechtman, Matching Local Self-Similarities across Images and Videos
comment: A quite interesting way to take self-repeating pattern as a feature. So texture information is ruled out.