Intelligente Datenanalysen und Vorhersagen für ihr Business

Präsentation zum Thema: "Intelligente Datenanalysen und Vorhersagen für ihr Business"—  Präsentation transkript:

1 Intelligente Datenanalysen und Vorhersagen für ihr Business
wie leiten sie gezielt und effizient Wissen aus ihren gespeicherten Daten ab? IUG-Workshop (M) 9. Nov 2005

2 Agenda Daten -> Informationen -> Wissen (Status Quo)
Intelligente Datenanalysen (Predictive Analytics, Unterschiede zu klassischen BI-Lösungen, Enterprise Performance Management) KXEN Life-Demo F & A (Proof-of-Value für IUG-Mitglieder )

3 We (take) care about your
INFORMATION! Software Engineering (> 15 Jahre) Daten-Management (> 10 Jahre) Intelligente Datenanalysen (> 7 Jahre) (Data Mining) Produkte seit 1998 im deutschsprachigen Raum eingeführt (Deutsche Telekom, AOL Deutschland, ABB, Fraunhofer Institut, Gruner + Jahr ...) Legacy data web Data Warehouse RDBMS Mart

4 Firmensitz in San Francisco, R&D in Frankreich
Gründung im Juli 1998 Firmensitz in San Francisco, R&D in Frankreich Erfahrenes Management Team R. Haddad (CEO), E. Marcade (CTO), M. Bera (CSO), J. Gerault (CFO) Aktiver wissenschaftlicher Beirat Vladimir Vapnik, Gregory Piatetsky-Shapiro, Gilbert Saporta, Yann Le Cun, Bernhard Schölkopf u.a. KXEN Analytic Framework in 3. Version 400+ Kunden setzen KXEN produktiv ein Partner: SAP, Cognos, BO, Teradata, SPSS u.a. Quick story of how KXEN was founded: M. Haddad, the CEO was the creator of several successful corporations, one of them a leading international HW and SW distribution company called Metrologie with 4500 employees and $900M in revenues. M. Bera, had the idea,- after locking himself up for several months with Vapnik’s books he translated the theories into algorithms; He has 20 years of experience with algorithms while working at Matra/ Aerospace and in the financial industry. As CSO he heads the scientific committee, shaping the future direction of KXEN solutions M. Marcade, has been responsible for turning the ideas into what has been acknowledged to be some of the most efficient and best quality analytic code of the industry; He developed his expertise working for Cadence (real time software) and Atos, consulting with the largest French automobile manufacturer 1°) US Cie with RD Fr 2°) 250 Site prod 3°) 3ème release major : Mature Tool 4°) Committee scientifique Strong to support and drive our technology and business dvpmt

5 3 komplementäre Bereiche eines modernen analytischen Unternehmens
Predictive Analytics Cumulative ROI Positive Returns Business Intelligence ROI (%) Break Even Operational systems like ERP…people have to have them has without that the enterprise does not exit BI is the next step Predictive Analytics allows to go one step further Operational Systems Negative Returns Time (Years) Source: Jack Noonan, 2002

6 Predictive Analytics Recommendations Scoring Data Mining OPTIMIZE l ROI Analysis l Customer Retention l Promotions l Demand Planning l Quality Improvement ENABLE l  Customer Interaction l  Inventory Control l  Supply Chain Mgmt l  Quality Measurement l  Employee Self Service ERP CRM ERM UNDERSTAND l  Customer Satisfaction l Product Revenue l  Cost of Goods Sold l  HR Turnover Business Intelligence OLAP Data Quality Query/Report Warehouse WEB Operational Systems Source: Jack Noonan, 2002 Ein modernes analytisches Unternehmen entsteht aus 3 komplementären Bereichen

7 Intelligente Datenanalysen (1)
The early days of computer-based data analysis started with mainframe computers and COBOL to generate lists. Enterprise SW still relies heavily on it – SAP ships with over 4000 “standard” Reports Problem: The IT department became the bottleneck for analysis, because each change in a report needed to go through IT OLAP (online Analytical Processing) gave analytic capabilities to the end-user Provided that they had access to the appropriate data cubes they could drill down into the data and slice and dice it to perform root-cause analysis There are two problems with this approach: Manual data exploration - too many dimensions make it impossible to navigate all the available data Analyses are always limited to historic data – if you have the time you can find out where your business ran into a problem yesterday, (after it occurred) Descriptive Modeling provides AUTOMATIC analysis of large amounts of data Predictive Analytics lets users anticipate and plan for future events based on historic data

8 Intelligente Datenanalysen (2)
The early days of computer-based data analysis started with mainframe computers and COBOL to generate lists. Enterprise SW still relies heavily on it – SAP ships with over 4000 “standard” Reports Problem: The IT department became the bottleneck for analysis, because each change in a report needed to go through IT OLAP (online Analytical Processing) gave analytic capabilities to the end-user Provided that they had access to the appropriate data cubes they could drill down into the data and slice and dice it to perform root-cause analysis There are two problems with this approach: Manual data exploration - too many dimensions make it impossible to navigate all the available data Analyses are always limited to historic data – if you have the time you can find out where your business ran into a problem yesterday, (after it occurred) Descriptive Modeling provides AUTOMATIC analysis of large amounts of data Predictive Analytics lets users anticipate and plan for future events based on historic data

9 Intelligente Datenanalysen (3)
The early days of computer-based data analysis started with mainframe computers and COBOL to generate lists. Enterprise SW still relies heavily on it – SAP ships with over 4000 “standard” Reports Problem: The IT department became the bottleneck for analysis, because each change in a report needed to go through IT OLAP (online Analytical Processing) gave analytic capabilities to the end-user Provided that they had access to the appropriate data cubes they could drill down into the data and slice and dice it to perform root-cause analysis There are two problems with this approach: Manual data exploration - too many dimensions make it impossible to navigate all the available data Analyses are always limited to historic data – if you have the time you can find out where your business ran into a problem yesterday, (after it occurred) Descriptive Modeling provides AUTOMATIC analysis of large amounts of data Predictive Analytics lets users anticipate and plan for future events based on historic data

10 Intelligente Datenanalysen (4)
Kündiger-Indikatoren (nach Relevanz) The early days of computer-based data analysis started with mainframe computers and COBOL to generate lists. Enterprise SW still relies heavily on it – SAP ships with over 4000 “standard” Reports Problem: The IT department became the bottleneck for analysis, because each change in a report needed to go through IT OLAP (online Analytical Processing) gave analytic capabilities to the end-user Provided that they had access to the appropriate data cubes they could drill down into the data and slice and dice it to perform root-cause analysis There are two problems with this approach: Manual data exploration - too many dimensions make it impossible to navigate all the available data Analyses are always limited to historic data – if you have the time you can find out where your business ran into a problem yesterday, (after it occurred) Descriptive Modeling provides AUTOMATIC analysis of large amounts of data Predictive Analytics lets users anticipate and plan for future events based on historic data

11 Intelligente Datenanalysen (5)

12 Intelligente Datenanalysen (6)

13 Intelligente Datenanalysen (7)

14 Was sind Daten-Analysen?
Data Prediction Classification, Regression Kündigungs- Wahrscheinlichkeit für jeden einzelnen Kunden und eine Erklärung “warum” Predictive Modeling Automated Data Exploration Clustering, Associations Charakteristika von “Kündigern” und “Nicht-Kündigern” Descriptive Modeling Manual Data Exploration Data Cubes Kündiger-Historie per Quartal und Region OLAP The early days of computer-based data analysis started with mainframe computers and COBOL to generate lists. Enterprise SW still relies heavily on it – SAP ships with over 4000 “standard” Reports Problem: The IT department became the bottleneck for analysis, because each change in a report needed to go through IT OLAP (online Analytical Processing) gave analytic capabilities to the end-user Provided that they had access to the appropriate data cubes they could drill down into the data and slice and dice it to perform root-cause analysis There are two problems with this approach: Manual data exploration - too many dimensions make it impossible to navigate all the available data Analyses are always limited to historic data – if you have the time you can find out where your business ran into a problem yesterday, (after it occurred) Descriptive Modeling provides AUTOMATIC analysis of large amounts of data Predictive Analytics lets users anticipate and plan for future events based on historic data Data Aggregation and Summarization SQL, Spreadsheets Kündiger-Historie quartalsweise Query und Reporting Konzept Technologie Beispiel

15 Klassifikation von Daten-Analysen
Analytics Reporting OLAP Statistics Predictive Analytics SAS, SPSS u.a. Cognos, BO, Hyperion Production Modeling Embedded Exploratory KXEN Anzahl Variablen 1 10 100 1000

16 Beispiel einer Cross-selling Analyse
Transaktionen Q2-> Q3 2004 Q4 Kunden Informationen Analyze Marketing Beispiel einer Cross-selling Analyse Mitarbeiter APPLY How does it work? The key is to bring it into operations to get it our from statistical Ghetto and start using it in operations!

17 Predictive Analytics für jedermann?
Komplexität (Guru Faktor) Traditionelle Statistik-/Data Mining Applikationen erfordern sehr hohen Skill-Level (Vor-) Auswahl der Kriterien/Variablen ist sehr aufwendig und nicht trivial Zuverlässigkeit der Ergebnisse bzgl. zukünftiger Daten/Vorgänge ist entscheidend Zeit und Kosten Ergebnisse erzeugen mit traditionellen Verfahren/Tools beansprucht Wochen Experten verwenden 60-90% ihrer Zeit für die Datenvorbereitung Web logs (als Datengeneratoren) müssen evaluiert werden - real time Geschwindigkeit (Umsetzung in Geschäftsprozesse) In kurzen Zyklen sich verändernde Daten erfordern zeitgerechte Umsetzung Analytische Systeme duplizieren Daten, sind Resourcen-intensiv und schwer integrierbar Now why are people not using it…simply because there is/were large hurdels that were preventing these systems from becoming operational Nur wenige Projekte rechtfertigten Einsatz/Kosten von traditionellen Verfahren/Tools Oft wurden (wenige) Modelle umgesetzt und Jahre lang (unverändert) genutzt

18 Was ist neu? Traditionell
Traditioneller Data Mining Ansatz 3 Weeks Select Variables Understand Business Question Apply Prepare Data Build Model Test Model KXEN Ansatz 3 Hours Business Question Build Model Understand Apply mit KXEN Now days things are changing as business users can start building models themselves (under certain assumptions) and therefore make the bring it into operations Our company builds hundreds of predictive models in the same time we used to build one. KXEN allows us to save millions of dollars with more effective campaigns Financial Industry Customer

19 Structured Risk Minimization (SRM)
Qualität: Wie gut beschreibt ein Modell ihre existierenden Daten? Zielerreichung durch Minimierung des Fehlers. Zuverlässigkeit/Robustheit: Wie gut wird ein Modell zukünftige Daten vorhersagen? Zielerreichung durch Minimierung der Unzuverlässigkeit. Risiko Gesamtrisiko Bestes Modell Unzuverlässigkeit Fehler Modell-Komplexität Structured Risk Minimization (SRM)

20 SRM = Qualität + Zuverlässigkeit + Geschwindigkeit
Select Variables Prepare Data Build Model Test Model SRM erspart 3 entscheidende Schritte Nutzung aller verfügbaren Variablen Automatisierte Datenvorbereitung Variablen-Decodierung – Nominal, Ordinal, Continuous Handling von Fehlwerten Erkennen von Ausreisserwerten Zieloptimierte Gruppierung – Sinnvolle Wertebereiche Ereignisbezogene Aggregation – Zeit, Wert, Sequenz Automatisierter Modell-Test KXEN changes the game by including in the modeling process 4 steps that are usually distinct: - Variable selection: due to the SRM methodology, algorithms can cope with larger number of variables…no need to get rid of some of the information (i.e variable) to build a model even if there is 1000s of them. - Encoding: KXEN automatically encode the data (see 3rd bullet point)

21 Manual Data Exploration Data Aggregation and Summarization
Die analytische Lücke Modeling Data Prediction 5 Statisticians 10 Questions OLAP Manual Data Exploration Query & Reporting have come a long way from COBOL Thanks to BI companies power today is in the hands of the end user Today enterprises have too much information Manual data exploration leads to an overload on the business user side There is a highly trained group within the company that can help to make sense of the data and find information But they are busy solving 10 business questions per year: a new bottleneck There is a problem for companies to close the analytic gap – it’s too hard to use the data mining tools There is no natural evolution from OLAP to data Mining Query & Reporting Data Aggregation and Summarization 10,000’s Business Users 10,000’s Questions

22 KXEN Enterprise Modeling
Production Modeling Data Prediction 5 50 Exploratory Modeling Automatic Data Exploration 100’s Users 1,000’s Questions Our Enterprise modeling bridges that gap. It drives production modeling as well as OLAP & Reporting and also increases the productivity of the modeling department. Q&R – How are my sales doing – e.g. churn by region OLAP – Drill into churn by type of customer & month – manual trial & error, EM – turn the question around - what is causing customers to leave Modeling – will this customer leave now? Embedded Modeling Analytics are part of Enterprise Applications 10,000’s Business Users 10,000’s Questions

23 KXEN-Integration ins gesamte Unternehmen
Customer Touch- Points Kiosk Web Phone POS Call Center Mail Feedback Order Churn Prevention Fraud Detection Risk Analysis Demand Forecast Channel Optimizat. Inbound Outbound Product Recom- mendation Cross-sell Up-sell Campaign Managem. Production Models Data Warehouse External Offer Lifestyle Life-stage Segmen- tation Lifetime Valuation Exploratory Models This is useful for customers, as well as OEM and SI partners to show where in the business process models can be used. In this example it is for implementing models as part of a CRM environment. Example: CRM KXEN-Integration ins gesamte Unternehmen

24 KXEN Analytic Framework
23 13 235 38 REPORT Exploration Event Log KEL Consistent Coder K2C Robust Regression K2R Model Export KMX RDBMS Oracle, Teradata, DB/2, MS, IBM Informix ... Text Files csv, txt, tab … Native SAP BW, SAS, SPSS, Excel … C, JAVA, VB, SAS, HTML, XML … Smart Segmenter K2S Scoring Time Series KTS Sequence Coder KSC SQL UDF PMML JAVA … Association Rules KAR Data Manipulation Preparation Modeling Model Deployment Production Access

25 LIFE DEMO - American Census
Business Case: Who in my dB is earning more than $50K and what are the important criteria? Available Information: UCI University International Benchmark Dataset: Records 15 attributes 24% of responders Financial data contains: How many credit cards, Debt / Income ratios.. Demographic data: Lifestyle, life-stage Customer data: Address, past purchases

26 KXEN-Referenzen Telekommunikation Finanzwesen Handel u.a. AT&T
Bell Canada Cingular China Telecom MCI Cox Com. PCS France Telecom Orange Swisscom Belgacom Vodafone D2 Wanadoo Finanzwesen E*Trade Washington Mutual JP Morgan Chase Fidelity Barclays Lloyds TSB Société Générale Socredo Bank Austria Disbank Crédit Lyonnais Great Eastern Life Insurance Handel u.a. Sears Wal-Mart Bottega Verde AC Nielsen Microsoft Europe ID Analytics Earthlink La Poste PowerGen Pfizer Veolia Waters ebay NFO

27 F & A Proof of Value (PoV) für IUG-Mitglieder Aufgabenstellung definieren Termin vereinbaren PoV durchführen kostenfrei Aufwand: Std. Anruf ( ) oder genügt