DATA WAREHOUSE Oracle Data Warehouse Mit Big Data neue Horizonte für das Data Warehouse ermöglichen Alfred Schlaucher, Detlef Schroeder DATA WAREHOUSE

Themen Big Data Buzz Word oder eine neue Dimension und Möglichkeiten Oracles Technologie zu Speichern von unstrukturierten und teilstrukturierten Massendaten Cloudera Framwork „Connectors“ in die neue Welt Oracle Loader for Hadoop und HDFS Big Data Appliance Mit Oracle R Enterprise neue Analyse-Horizonte entdecken Big Data Analysen mit Endeca

Oracle Endeca | Steckbrief Firmenhauptsitz: Cambridge, Massachusetts 600+ Kunden, 33% aus den “Fortune 100” 2 Produktlinien: Endeca Information Discovery Customer Experience Management Geführte Suche - integriert in Web-Sites = Endeca Kernkompetenz Kombination aus Textbasierte Suche + Business Intelligence Copyright 2012 Oracle and it's affiliates. All rights reserved.

Oracle Endeca | Guided Search Dynam. Angabe der Treffermenge, sehr schnelle Aktualisierung Graphisch festlegbare Filterkriterien Zutreffende Attribut-gruppen: thematisch sortiert  mit dynam. Häufigkeits-angabe Copyright 2012 Oracle and it's affiliates. All rights reserved.

Oracle Endeca | Information Discovery (OEID) Copyright 2012 Oracle and it's affiliates. All rights reserved.

Oracle Endeca | Information Discovery (OEID) OEID kombiniert Einfachheit der Suche mit Business Intelligence Analyse-Funktionen/-Power Basiert auf 10 Jahren Design-Erfahrung im E-Commerce (B2C und B2B) Suche + Faceted Navigation (Facetten Suche) + Visuelle Analyse Suche und Attributauswahl  ähnlich wie auf einer Web-Site User Interface Konzept ist teilw. vergleichbar mit dem Tool Infozoom (ist aber ein Fat Client) Ergebnisse mit Karten, Tag Clouds, etc. visualisierbar Schnell reagierender Endeca Server erlaubt interaktive Analysen bzw. den Aufbau agiler BI Anwendungen Copyright 2012 Oracle and it's affiliates. All rights reserved.

Interaktiv Daten untersuchen und neue Zusammenhänge entdecken Oracle Endeca | Information Discovery (OEID) Interaktiv Daten untersuchen und neue Zusammenhänge entdecken + + Advanced Search Search look-ahead Spell-correction Data-driven filtering Faceted Navigation Select attributes, like a web site Visual Analysis Charting & crosstabs Geographic visualization Tag clouds Copyright 2012 Oracle and it's affiliates. All rights reserved.

Oracle Endeca | Einsatzgebiete E-Commerce  “Customer Experience Mgmt.” Intuitive, “easy-to-use” Benutzeroberflächen für Konsumenten Schnell Unternehmen  Suche / Information Discovery verteilt, komplex, veränderte Daten und Inhalte Öffentlicher Dienst  Information Discovery High Performance, Skalierbarkeit, Zugriffssicherheit Copyright 2012 Oracle and it's affiliates. All rights reserved.

OEID Information Integration OEID Architektur | Server OEID Studio Hybride Technologie: Suchmaschine und analytische Datenbank in einem Umfassende Suchfunktionen, Navigation und Analytik über unterschiedliche und sich ändernde Daten(-quellen) Columnar Storage Model / In-Memory Verarbeitung Datenspeicherung  auf Festplatte Überführung in RAM, sobald Daten referenziert werden Embedded Index-Trees: nur benötigte Daten werden gescannt Faceted Data Model Parallelisierbar Endeca Server OEID Information Integration Copyright 2012 Oracle and it's affiliates. All rights reserved.

Medikamentbeschreibung (Text) OEID Architektur | Server – Faceted Data Model Original System (z.B. Datenbank) Care Team ID Gender OHIP Number Patient ID Patient City Disease Gender OHIP Number Endeca Index Care Team ID Metformin Drug Description Disease Medikamentbeschreibung (Text) Shaun Mahal | June 20, 2010 Metformin was approved for use in the U.S. for treatment of type 2 diabetes in December, 1994. It is sold under the brand name Glucophage and is also available generically. Metformin is approved for treatment with sulfonylureas, or with insulin, or as monotherapy (by itself). Glucophage XR Extended Release tablets, a once daily version of metformin, is available. Also, metformin is available… Metformin Sulfonylureas http://en.wikipedia.org/wiki/Faceted_search Massachusetts Institute of Technology - verschiedene Projekte Faceted search, also called faceted navigation or faceted browsing, is a technique for accessing information organized according to a faceted classification system, allowing users to explore a collection of information by applying multiple filters. A faceted classification system classifies each information element along multiple explicit dimensions, enabling the classifications to be accessed and ordered in multiple ways rather than in a single, pre-determined, taxonomic order. Projects Within the academic community, faceted search has attracted interest primarily among library and information science researchers, and to some extent among computer science researchers specializing in information retrieval. ----- With records, each field is a facet. We maintain the relationships that tie the value(s) of each facet to that facet; the relationship of that facet-value pair to the record it belongs to. With documents, some facets are already explicit, like tags or other meta-data. Other facets are implicit in the text and can be made explicit through tagging, entity extraction, noun phrase extraction or other techniques. And some facets – some characteristics of what the document is about – remain implicit. This is recorded in the facet that holds the full-text index of that document. Suche innerhalb von Polygonnetzen http://de.wikipedia.org/wiki/Polygonnetz Shaun Mahal Copyright 2012 Oracle and it's affiliates. All rights reserved.

OEID Architektur | Server – Faceted Data Model Merkmale & Beispiel  Endeca Record TxnID = 12324 ProductID = 506 Category = Mountain Bike Amount = $499.99 Suspension = Fox 32 F-Series FrameType = Aluminium Saddle = Bontrager SSR Mountain Accessories = Fork and shock sag meter Mountain Accessories = Water Bottle Review = A great bike for off road. Smooth ride over the bumps ReviewSentiment = Positive ReviewTerm = Great ReviewTerm = Off Road ReviewTerm = Smooth ReviewTerm = Bumps beherbergt: quasi kein Schema, jeder Record beschreibt sich bzw. repräsentiert sich selbst und kann prinzipiell ein eigenes Schema haben Multi-value Datenfelder sind möglich Unstrukturierte Datenfelder sind möglich Modell ist eine Art Key Value store Modell besteht aus Records / Attributen Facetten (= Zeiger auf Kantenlisten in Polygonnetzen) Jeder Record ist eine Sammlung von Attribute-Werte-Paaren Keine Aufteilung der Daten(-speicherung) in Tabellen TxnID = 12325 ProductID = 507 Category = Road Bike Amount = $1399.49 Weight = 20lb. FrameType = Composite Saddle = Bontrager Race Review = Disappointing for the price. The frame feels heavier than I expected. ReviewSentiment = Negative ReviewTerm = Disappointing ReviewTerm = Price ReviewTerm = Heavier Copyright 2012 Oracle and it's affiliates. All rights reserved.

OEID Architektur | Server – Faceted Data Model Integration strukturierter Daten TxnID = 12324 ProductID = 506 Category = Mountain Bike Amount = $499.99 Suspension = Fox 32 F-Series FrameType = Aluminium Saddle = Bontrager SSR Mountain Accessories = Fork and shock sag meter Mountain Accessories = Water Bottle Review = A great bike for off road. Smooth ride over the bumps ReviewSentiment = Positive ReviewTerm = Great ReviewTerm = Off Road ReviewTerm = Smooth ReviewTerm = Bumps Strukturierte Daten können direkt via ETL in ein Faceted Data Model geladen und gespeichert werden Jedes Tupel wird zu einem Record Jede Spalte wird zu einem Attribut ETL TxnID = 12325 ProductID = 507 Category = Road Bike Amount = $1399.49 Weight = 20lb. FrameType = Composite Saddle = Bontrager Race Review = Disappointing for the price. The frame feels heavier than I expected. ReviewSentiment = Negative ReviewTerm = Disappointing ReviewTerm = Price ReviewTerm = Heavier Transaction TxnID ProductID Category Amount 12324 506 Mountain Bike 499 12325 507 Road Bike 1399 Relationale Tabelle Copyright 2012 Oracle and it's affiliates. All rights reserved.

OEID Architektur | Server – Faceted Data Model Integration semi-strukturierter Daten TxnID = 12324 ProductID = 506 Category = Mountain Bike Amount = $499.99 Suspension = Fox 32 F-Series FrameType = Aluminium Saddle = Bontrager SSR Mountain Accessories = Fork and shock sag meter Mountain Accessories = Water Bottle Review = A great bike for off road. Smooth ride over the bumps ReviewSentiment = Positive ReviewTerm = Great ReviewTerm = Off Road ReviewTerm = Smooth ReviewTerm = Bumps Semi-strukturierte Daten können als Key-Value-Paare aus XML Quellen, Feeds,Unternehmens-Applikationen,, etc. geladen werden Typische Datenstruktur, die auch im Polizeiumfeld vielfach verwendet wird <ProductData ProductID="506"> <Attribute Key="Suspension">Fox 32 F-Series</Attribute> <Attribute Key="FrameType">Aluminium</Attribute> <Attribute Key="Saddle">Bontrager SSR</Attribute> <Attribute Key="Mountain Accessories"> Fork and shock sag meter</Attribute> Water Bottle</Attribute> </ProductData> <ProductData ProductID="507"> <Attribute Key="Weight">20lb.</Attribute> <Attribute Key="FrameType">Composite</Attribute> <Attribute Key="Saddle">Bontrager Race</Attribute> ETL TxnID = 12325 ProductID = 507 Category = Road Bike Amount = $1399.49 Weight = 20lb. FrameType = Composite Saddle = Bontrager Race Review = Disappointing for the price. The frame feels heavier than I expected. ReviewSentiment = Negative ReviewTerm = Disappointing ReviewTerm = Price ReviewTerm = Heavier XML Quelle Copyright 2012 Oracle and it's affiliates. All rights reserved.

OEID Architektur | Server – Faceted Data Model Integration unstrukturierter Daten TxnID = 12324 ProductID = 506 Category = Mountain Bike Amount = $499.99 Suspension = Fox 32 F-Series FrameType = Aluminium Saddle = Bontrager SSR Mountain Accessories = Fork and shock sag meter Mountain Accessories = Water Bottle Review = A great bike for off road. Smooth ride over the bumps ReviewSentiment = Positive ReviewTerm = Great ReviewTerm = Off Road ReviewTerm = Smooth ReviewTerm = Bumps Unstrukturierte Daten können mit anderen Records über einen beliebigen Schlüssel verbunden werden Unstrukturierte Elemente können separat als eigene Records für “side by side” Analysen gespeichert werden Endeca Content Acquisition System (CAS) lädt Dokumente, RSS-Feeds und kann Twitter, Facebook, Web-Foren crawlen CAS+ETL TxnID = 12325 ProductID = 507 Category = Road Bike Amount = $1399.49 Weight = 20lb. FrameType = Composite Saddle = Bontrager Race Review = Disappointing for the price. The frame feels heavier than I expected. ReviewSentiment = Negative ReviewTerm = Disappointing ReviewTerm = Price ReviewTerm = Heavier Review: #1301 Product: 506 A great bike for off road. Smooth ride over the bumps Review: #1327 Product: 507 Disappointing for the price. The frame feels heavier than I expected. Copyright 2012 Oracle and it's affiliates. All rights reserved.

OEID Architektur | Server – Faceted Data Model Daten anreichern TxnID = 12324 ProductID = 506 Category = Mountain Bike Amount = $499.99 Suspension = Fox 32 F-Series FrameType = Aluminium Saddle = Bontrager SSR Mountain Accessories = Fork and shock sag meter Mountain Accessories = Water Bottle Review = A great bike for off road. Smooth ride over the bumps ReviewSentiment = Positive ReviewTerm = Great ReviewTerm = Off Road ReviewTerm = Smooth ReviewTerm = Bumps Jedes unstrukturierte Attribut kann prinzipiell um weitere Informationen angereichert werden, z.B. durch Text Analytics zur Erweiterung Datensatzstruktur Gängige Techniken: Automatic Tagging Named Entity Extraction Sentiment Analysis Term Extraction Geospatial Matching TxnID = 12325 ProductID = 507 Category = Road Bike Amount = $1399.49 Weight = 20lb. FrameType = Composite Saddle = Bontrager Race Review = Disappointing for the price. The frame feels heavier than I expected. ReviewSentiment = Negative ReviewTerm = Disappointing ReviewTerm = Price ReviewTerm = Heavier Copyright 2012 Oracle and it's affiliates. All rights reserved.

Supports a breadth of structured and unstructured search capabilities: OEID Architektur | Server – Volltextsuche Search index storage and analysis build on same column storage core as structured data store / indexes Structured Data Column value, id Acme Corp, 375 Search Index Column word, id, position Corp, 375 (89,72) Supports a breadth of structured and unstructured search capabilities: Guided Navigation Keyword search Boolean search Parametric search Wildcard search Dimension search Dimension filters Dimension precedence rules Numeric range filters Geospatial filters Date/Time filters Security filters Spell correction/suggestion, DYM Find similar 1- and 2-way synonyms Stemming and lemmatization Keyword-in-context snippeting Results clustering Relevance ranking Sorting and paging Language support Copyright 2012 Oracle and it's affiliates. All rights reserved.

OEID Information Integration OEID Architektur | Information Integration Endeca Server OEID Studio OEID Information Integration Endeca Workbench CloverETL Copyright 2012 Oracle and it's affiliates. All rights reserved.

OEID Architektur | Information Integration Endeca Server Open API Structured and Semi-Structured Data Unstructured Data Enterprise Structured Data Data Integrator (aktuell: CloverETL) Content Aquisition System (CAS) Erweiterbares Framework für die Anbindung und Behandlung unstrukturierter Datenquellen Crawler für Dateiserver und das Web Adapter für Content-Management Systeme Text und Metadaten Extraktion Text Enrichment Fähigkeiten Flexible und agile ETL Umgebung Adapters für JDBC und übliche Dateitypen (XML, delimited, fixed-width, etc.) Java SDK Framework zum Erstellen eigener Adapter und Module zur Daten Manipulation Copyright 2012 Oracle and it's affiliates. All rights reserved.

OEID Information Integration OEID Architektur | OEID Studio Interaktive, Komponenten-basierte Benutzeroberfläche Komplette Bibliothek mit fertigen BI-Komponenten enthalten Realisiert mit Best Practice “Design Pattern” für die User Interface Entwicklung AJAX Interaktion Setzt auf Industriestandards Enterprise-class manageability OEID Studio Endeca Server OEID Information Integration Copyright 2012 Oracle and it's affiliates. All rights reserved.

OEID Architektur | OEID Studio OEID Studio Komponenten – Out-of-the-Box Advanced Visualization Bookmarks Breadcrumbs Chart Data Sources Guided Navigation® Performance Metrics Range Filters Record Details Results Table Search Box Metrics Bar Cross Tab Find Similar Copyright 2012 Oracle and it's affiliates. All rights reserved.

OEID Architektur | OEID Studio Jede Komponente hat eigene Kontrolleinstellungen und einen Editor Copyright 2012 Oracle and it's affiliates. All rights reserved.

Demonstration 22

OEID | Zusammenfassung Oracle Endeca Information Discovery kombiniert eine leistungsfähige Suchmaschine mit einer Analytischen Datenbank zu einer agilen Business Intelligence Lösung Structured Semi-Structured Unstructured Endeca Server (MDEX-Engine): spaltenorientierte Datenhaltung, In-Memory Technologie, Faceted Datenmodell Offene API / zusätzliche Komponenten, z.B. für Entitäten Extraktion, Sentiment Analyse Interaktive und Geführte Suche über dynamische Filter, Drill-down Datenvisualsierung, z.B. mit Tag Clouds, Geodaten, Master-Detail-Diagramme Power-User Funktionen (vgl. mit infoZoom) Integration strukturierter, semi-strukturierter und unstrukturierter Daten erfolgt via Endeca Content Aquisition System (CAS) und ETL

OEID | Zusammenfassung

Kombinierter Analyse-Ansatz mit Oracle Big Data / endeca Copyright 2012 Oracle and it's affiliates. All rights reserved.

Oracle’s kombinierter Analyse-Ansatz Enterprise Systems & Content Stores Un-/Semi- structured Data Sources File Systems Databases Internet / Social Networks Content Mgt Systems SOA, ESB, Web Service Data Snapshots Oracle OLTP Database Oracle NoSQL Database Hadoop Distributed File System (HDFS) Oracle Big Data Appliance Information Integration ETL/ELT-Systems (Warehouse Builder. Data Integrator) Oracle Endeca Integration Suite Oracle Loader für HADOOP Hadoop MapReduce (Framework) Datenstrom | Erfassen | Organisieren | Analysieren | Entscheiden Unstructured Data Transformation Data Warehouse & Data Marts Oracle Data Warehouse Database Endeca In-Memory DB Data Marts, Analysis Sandpits OLAP Cubes In-Database Analytics (“R”, Data Mining, etc.) Information Discovery & Search Information Delivery Oracle Business Intelligence Analytical Applications Oracle Endeca Studio Reports, Visualisierung, . .. Embedded Analytics / Search …. Multidim. Analysis & Search Copyright 2012 Oracle and it's affiliates. All rights reserved.

Kontakt und mehr Informationen Oracle Data Warehouse Community Mitglied werden Viele kostenlose Seminare und Events Download – Server: www.ORACLEdwh.de Nächste deutschsprachige Oracle DWH Konferenz: 19. + 20. März 2013 Kassel