Jens Zimmermann, Forschungszentrum Jülich, Astroteilchenschule 10/041 Jens Zimmermann Max-Planck-Institut für Physik, München MPI.

Präsentation zum Thema: "Jens Zimmermann, Forschungszentrum Jülich, Astroteilchenschule 10/041 Jens Zimmermann Max-Planck-Institut für Physik, München MPI."—  Präsentation transkript:

1 Jens Zimmermann, Forschungszentrum Jülich, Astroteilchenschule 10/041 Jens Zimmermann zimmerm@mppmu.mpg.de Max-Planck-Institut für Physik, München MPI für extraterrestrische Physik, München Forschungszentrum Jülich GmbH Statistical Learning? Three Classes of Learning Methods Applications in Physics Analysis Training the Learning Methods Examples Conclusion Statistical Learning in Astrophysics

2 Jens Zimmermann, Forschungszentrum Jülich, Astroteilchenschule 10/042 Some Events 0 1 2 3 4 5 6 x10 # formulas # slides 0 1 2 3 4 5 6 x10 # formulas# slides 4221 288 7119 6431 2936 1534 4844 5651 2555 1216 Experimentalists Theorists

3 Jens Zimmermann, Forschungszentrum Jülich, Astroteilchenschule 10/043 First Analysis 0 2 4 6 x10 # formulas 0 2 4 6 x10 # slides Experimentalists Theorists 0 1 2 3 4 5 6 x10 # formulas # slides 0 1 2 3 4 5 6 x10

4 Jens Zimmermann, Forschungszentrum Jülich, Astroteilchenschule 10/044 Decision Trees 0 2 4 6 x10 # formulas #formulas < 20 exp #formulas > 60th 0 2 4 6 x10 # slides 20 < #formulas < 60? #slides > 40exp #slides < 40th #slides < 40#slides > 40 expth #formulas < 20 #formulas > 60 rest exp th all events subset 20 < #formulas < 60

5 Jens Zimmermann, Forschungszentrum Jülich, Astroteilchenschule 10/045 Local Density Estimators Search for similar events that are already classified and count the members of the two classes. (e.g. k-Nearest-Neighbour) 0 1 2 3 4 5 6 x10 # formulas # slides 0 1 2 3 4 5 6 x10 # formulas # slides 0 1 2 3 4 5 6 x10

6 Jens Zimmermann, Forschungszentrum Jülich, Astroteilchenschule 10/046 Methods Based on Linear Separation Divide the input space into regions separated by one or more hyperplanes. Extrapolation is done! 0 1 2 3 4 5 6 x10 # formulas # slides 0 1 2 3 4 5 6 x10 # formulas # slides 0 1 2 3 4 5 6 x10

7 Jens Zimmermann, Forschungszentrum Jülich, Astroteilchenschule 10/047 Neural Networks 0 1 2 3 4 5 6 x10 -50 +0.1 +1.1-1.1 +20 +0.2 +3.6 -1.8 # formulas# slides 0 1 Construct NN with two separating hyperplanes:Train NN with two hidden neurons (gradient descent):

8 Jens Zimmermann, Forschungszentrum Jülich, Astroteilchenschule 10/048 NN Training 8 hidden neurons = 8 separating lines Test-Error Train-Error signal background Training Epochs

9 Jens Zimmermann, Forschungszentrum Jülich, Astroteilchenschule 10/049 Training of Statistical Learning Methods Statistical Learning Method: From N examplesinfer a rule Important: Generalisation vs. Overtraining Without noise and separable by complicated boundary? 0 1 2 3 4 5 6 x10 # formulas # slides 0 1 2 3 4 5 6 x10 Easily separable but with noise? Too high degree of polynomial results in interpolation but too low degree means bad approximation

10 Jens Zimmermann, Forschungszentrum Jülich, Astroteilchenschule 10/0410 Applications in Physics Analysis Classification Online „Trigger“ H1L2NNCharged Current Classification Offline „Purification“ Gamma vs. Hadron MAGIC Regression ~300µm ~10µm XEUSX-ray CCD

11 Jens Zimmermann, Forschungszentrum Jülich, Astroteilchenschule 10/0411 Features Choose raw quantities PCA FFT SymmetrieFit

12 Jens Zimmermann, Forschungszentrum Jülich, Astroteilchenschule 10/0412 Regression of the Incident Position of X-ray Photons  of reconstruction in µm: Neural Networks 3.6classical methods k-Nearest-Neighbour 3.7 ETA 3.9 CCOM 4.0 XEUS ~300µm ~10µm electron potential transfer direction

13 Jens Zimmermann, Forschungszentrum Jülich, Astroteilchenschule 10/0413 Pileup Recognition – Setup Pileup vs. Single photon classical algorithm „XMM“ ? photon efficiency [%] pileup rejection [%] 99/52 99/67

14 Jens Zimmermann, Forschungszentrum Jülich, Astroteilchenschule 10/0414 MAGIC - Gamma/Hadron Separation Observation Mkn421 22.04.04 SuperCuts 39.0  SuperCuts + Neural Network 46.8  

15 Jens Zimmermann, Forschungszentrum Jülich, Astroteilchenschule 10/0415 Conclusion Three classes of statistical learning methods Decision Trees (Bagging) Local Density Estimators Linear Separation Many applications in current astrophysics experiments and analysis Compared to classical methods usually at least small improvements C4.5 CART Random Forest Maximum Likelihood k-Nearest-Neighbour Linear Discriminant Analysis Neural Networks Support Vector Machines

16 Jens Zimmermann, Forschungszentrum Jülich, Astroteilchenschule 10/0416 Theory of Communication: Minimum Description Length Principle Bayes Hypothesis H and Data D Our hypothesis should have the maximum probability given the data: Shannon MDLP Rissanen 18 th century 1948 1990

17 Jens Zimmermann, Forschungszentrum Jülich, Astroteilchenschule 10/0417 Statistical Learning Theory: Structural Risk Minimization We have N training events with input x i and correct output y i empirical riskactual risk Relationship is uniform convergence: h 1 <h 2 <h3h3 Create nested subsets of function spaces with rising complexity Upper bound for the actual risk (Vapnik): h : VC Dimension of learning method (complexity) 1996

18 Jens Zimmermann, Forschungszentrum Jülich, Astroteilchenschule 10/0418 Support Vector Machines Separating hyperplane with maximum distance to each datapoint: Maximum margin classifier Found by setting up condition for correct classfication and minimizing which leads to the Lagrangian Necessary condition for a minimum is So the output becomes Only linear separation? The mapping to feature space is hidden in a kernel No! Replace dot products: KKT: only SV have Non-separable case:

19 Jens Zimmermann, Forschungszentrum Jülich, Astroteilchenschule 10/0419 Finally 0 1 2 3 4 5 6 x10 # formulas # slides 0 1 2 3 4 5 6 x10 Include Statistical Learning Theory: ~25 formulas on 19 slides 0 1 2 3 4 5 6 x10 # formulas # slides 0 1 2 3 4 5 6 x10 Skip Theory: ~7 formulas on 16 slides