Die Präsentation wird geladen. Bitte warten

Die Präsentation wird geladen. Bitte warten

Use this title slide only with an image Towards a Web-scale Data Management Ecosystem Demonstrated by SAP HANA Stefan Bäuerle, Jonathan Dees, Franz Faerber,

Ähnliche Präsentationen


Präsentation zum Thema: "Use this title slide only with an image Towards a Web-scale Data Management Ecosystem Demonstrated by SAP HANA Stefan Bäuerle, Jonathan Dees, Franz Faerber,"—  Präsentation transkript:

1 Use this title slide only with an image Towards a Web-scale Data Management Ecosystem Demonstrated by SAP HANA Stefan Bäuerle, Jonathan Dees, Franz Faerber, Wolfgang Lehner

2 ©2015 SAP SE or an SAP affiliate company. All rights reserved.2 Public Motivation & Requirements Different Processing Engines and Integration Scale out edition engine Agenda

3 ©2015 SAP SE or an SAP affiliate company. All rights reserved.3 Public Application requirements for a modern DBMS  data types  consumption models  data models  notions of consistency  application and query language  levels of scaling  hardware capabilities Different:

4 ©2015 SAP SE or an SAP affiliate company. All rights reserved.4 Public HANA Platform

5 ©2015 SAP SE or an SAP affiliate company. All rights reserved.5 Public HANA System

6 ©2015 SAP SE or an SAP affiliate company. All rights reserved.6 Public Beyond relational data processing (1/3) Bringing OLAP and OLTP together Proven: works in thousands of customer systems Simplicity: get rid of extracts, loads and redundancy, one system OLAP dominates OLTP in real world systems: optimize accordingly Data mining and prediction Examples: Basked analysis, different forecasting algorithms… Easy interaction with R and SAS Unstructured data Support text search > 30 languages including: Stemming, speech tagging, noun extractions, … Classification, clustering, named entity recognition, sentinel analysis Planning extensions Planning: Define and align business figures for foreseeable future Data heavy operators like disaggregation or logical snapshots Integrate as deep as possible into the engine

7 ©2015 SAP SE or an SAP affiliate company. All rights reserved.7 Public Beyond relational data processing (2/3) Graph processing Real world business data often resembles graphs Model as graph: More explicit and more efficient operators Distance, siblings, shortest path, reachability, transitive closure, … Hierarchy processing Special type of general graphs Used by almost every business application Support for time dependent and versioned hierarchies Extended graph operators: level, neighbor, is_ancestor, … Geospatial processing & Time series Native relational data types Existing compression techniques + powerful specializations for sensor data Spatial: WithinDistance, Contains, Area, … Time series: Group by time interval, Interpolate Missing Values, …

8 ©2015 SAP SE or an SAP affiliate company. All rights reserved.8 Public Beyond relational data processing (3/3) Scientific processing Bring prominent operators into the engine Simplifies and speeds up operations in scientific and financial area Matrix operators: Eigenvalue, Multiply, … Financial operators: Interest Rates, GarmanKohlagenProcess, … No SQL processing Document based models, XML, JSON, … Key value stores Flexible Schema, in HANA via specific flexible table type Massive scale out Conventional business applications fit on single box, but: there is a new kind of applications requiring massive scale out Deep and seamless integration with the Hadoop system Scale out and single box application act as one system

9 ©2015 SAP SE or an SAP affiliate company. All rights reserved.9 Public Application integration ( examples )  Currency conversion  Hierarchy handling  Aging / dynamic tiering  Dictionary maintenance  Graph optimizations

10 ©2015 SAP SE or an SAP affiliate company. All rights reserved.10 Public HANA Data Platform Dynamic Tiering HANA Dynamic Tiering  Declare table to use disk storage  Cost efficient for big data  Optimized disk based processing powered by IQ New warm option beside  Hot (in-memory)  Cold (Near Linear Storage) CREATE TABLE „demo“.“SalesOrders_WARM“ ( IDInteger NOT NULL, CustomerIDInteger NOT NULL, OrderDatedate NOT NULL, …, PRIMARY KEY (id) ) USING EXTENDED STORAGE; INSERT INTO „demo“.“SalesOrders_WARM“ VALUES ( … );

11 ©2015 SAP SE or an SAP affiliate company. All rights reserved.11 Public HANA Data Platform BigData | Vision HANA native BigData  Dynamic Tiering  Smart Data Streaming  NoSQL | Graph | Geo | TimeSeries HANA & Hadoop  SDA  Hive | Spark  MapReduce | HDFS  Admin & Monitoring  User Mgmt / Security Hadoop Extension  Velocity Engine  Integrated with HANA and Hadoop HANA Data Management Platform Instant Results SAP HANA In-Memory Warm Data HANA Dynamic Tiering 0.1sec ∞ Infinite Storage Raw Data HADOOP HANA Scale Out Information Management | Text | Search | Graph | Geospatial | Predictive Smart Data Streaming Administration | Monitoring | Operations | User Management | Security

12 ©2015 SAP SE or an SAP affiliate company. All rights reserved.12 Public SAP HANA Massive Scale Out Edition (Velocity) Motivation: Engine for massive scale out and big data Key Features: Scale to thousands of nodes Different data freshness and consistency levels Efficient fail safety design First class citizen within Hadoop (Spark) Support variety of hardware and operating systems Extreme query performance by compiling SQL to native code

13 ©2015 SAP SE or an SAP affiliate company. All rights reserved.13 Public SAP HANA SOE (Velocity) and Hadoop (1/2) Ambari Cluster Management Hadoop Ecosystem Zookeeper Coordination Pig Scripting MLib Machine Learning Hive SQL SparkSQL SQL Yarn Processing HDFS Distributed File System HBase Database Spark Processing

14 ©2015 SAP SE or an SAP affiliate company. All rights reserved.14 Public SAP HANA SOE (Velocity) and Hadoop (2/2) Steps  Stage 1: Integration with Spark (  2015)  Stage 2: Independent execution cluster Benefits  Integration of SAP data with data lakes  HANA features add Value into Hadoop (e.g. SQL extensions like time series, hierarchies, …)  Performance  Holistic data platform

15 ©2015 SAP SE or an SAP affiliate company. All rights reserved.15 Public Architecture to Support Different Data Freshness Levels DTX Query Engine 1 Transaction Broker Version Table A, B, C Query Engine 2 Query Engine 3 R Storage 1 Storage n Storage 2 Distributed Log R … … … R R R A, D A, C, D DQP Storage (checkpoints) Connection n Connection 1 (Session data) Options read your own writes up-to-date data vs. certain age Separate component for Transactions

16 ©2015 SAP SE or an SAP affiliate company. All rights reserved.16 Public SAP HANA scale out integration

17 ©2015 SAP SE or an SAP affiliate company. All rights reserved.17 Public Conclusion Today’s applications have multidimensional set of specialized requirements Gains from moving these requirements into a (single) DBMS: Simplified and more explicit data modeling and processing for applications Increased performance No complicated data transfer between specialized engines Powerful orchestration required Web-scale processing is key to support new applications SAP HANA strives to answer all these requirements in a single data management platform.

18 ©2015 SAP SE or an SAP affiliate company. All rights reserved.18 Public SAP HANA Massive scale out edition (Project Velocity)  Scales to thousands of nodes –Support of massive distribution and failure tolerance –ACID properties on large landscape  Can run on small devices –Low footprint allows to run on small commodity hardware and small devices  Integration into Hadoop infrastructure ( Spark ) –Access via standard Hadoop mechanisms ( i.e. map & reduce) –Deep integration into Spark execution framework  Extreme performance with SQL compilation –Compile SQL into C code and realtime compilation into executable  Support for IoT and semi structured data –Special data types for IoT ( time series data) –Support of document style data in a massive scale environment

19 ©2015 SAP SE or an SAP affiliate company. All rights reserved.19 Public SAP HANA SOE (Velocity) and Hadoop (2/2) General: Embrace Hadoop as technology Goal: Get our own Engine on Hadoop  Velocity  HANA Scale-Out Extension Steps  First step: Integrated with Spark (  Q3 2015)  Mid Term: independent execution cluster Benefits  Holistic data platform  Integration of SAP data with data lakes  HANA features on Hadoop (e.g. time series)  Value added abilities on Hadoop data  Performance

20 ©2015 SAP SE or an SAP affiliate company. All rights reserved.20 Public Architecture to Support Different Data Freshness Levels Distributed log Distributed filesystem (for checkpoints …) Distributed transaction manager Distributed query processor Workers Storage Document Velocity (OLTP) Velocity (OLAP) Text Graph Time series

21 ©2015 SAP SE or an SAP affiliate company. All rights reserved. Thank you

22 ©2015 SAP SE or an SAP affiliate company. All rights reserved.22 Public © 2015 SAP SE or an SAP affiliate company. All rights reserved. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company. SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company) in Germany and other countries. Please see for additional trademark information and notices.http://global12.sap.com/corporate-en/legal/copyright/index.epx Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors. National product specifications may vary. These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP SE or its affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP SE or SAP affiliate company products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty. In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release any functionality mentioned therein. This document, or any related presentation, and SAP SE’s or its affiliated companies’ strategy and possible future developments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward- looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.

23 ©2015 SAP SE or an SAP affiliate company. All rights reserved.23 Public © 2015 SAP SE oder ein SAP-Konzernunternehmen. Alle Rechte vorbehalten. Weitergabe und Vervielfältigung dieser Publikation oder von Teilen daraus sind, zu welchem Zweck und in welcher Form auch immer, ohne die ausdrückliche schriftliche Genehmigung durch SAP SE oder ein SAP-Konzernunternehmen nicht gestattet. SAP und andere in diesem Dokument erwähnte Produkte und Dienstleistungen von SAP sowie die dazugehörigen Logos sind Marken oder eingetragene Marken der SAP SE (oder von einem SAP-Konzernunternehmen) in Deutschland und verschiedenen anderen Ländern weltweit. Weitere Hinweise und Informationen zum Markenrecht finden Sie unter Die von SAP SE oder deren Vertriebsfirmen angebotenen Softwareprodukte können Softwarekomponenten auch anderer Softwarehersteller enthalten. Produkte können länderspezifische Unterschiede aufweisen. Die vorliegenden Unterlagen werden von der SAP SE oder einem SAP-Konzernunternehmen bereitgestellt und dienen ausschließlich zu Informationszwecken. Die SAP SE oder ihre Konzernunternehmen übernehmen keinerlei Haftung oder Gewährleistung für Fehler oder Unvollständigkeiten in dieser Publikation. Die SAP SE oder ein SAP-Konzernunternehmen steht lediglich für Produkte und Dienstleistungen nach der Maßgabe ein, die in der Vereinbarung über die jeweiligen Produkte und Dienstleistungen ausdrücklich geregelt ist. Keine der hierin enthaltenen Informationen ist als zusätzliche Garantie zu interpretieren. Insbesondere sind die SAP SE oder ihre Konzernunternehmen in keiner Weise verpflichtet, in dieser Publikation oder einer zugehörigen Präsentation dargestellte Geschäftsabläufe zu verfolgen oder hierin wiedergegebene Funktionen zu entwickeln oder zu veröffentlichen. Diese Publikation oder eine zugehörige Präsentation, die Strategie und etwaige künftige Entwicklungen, Produkte und/oder Plattformen der SAP SE oder ihrer Konzernunternehmen können von der SAP SE oder ihren Konzernunternehmen jederzeit und ohne Angabe von Gründen unangekündigt geändert werden. Die in dieser Publikation enthaltenen Informationen stellen keine Zusage, kein Versprechen und keine rechtliche Verpflichtung zur Lieferung von Material, Code oder Funktionen dar. Sämtliche vorausschauenden Aussagen unterliegen unterschiedlichen Risiken und Unsicherheiten, durch die die tatsächlichen Ergebnisse von den Erwartungen abweichen können. Die vorausschauenden Aussagen geben die Sicht zu dem Zeitpunkt wieder, zu dem sie getätigt wurden. Dem Leser wird empfohlen, diesen Aussagen kein übertriebenes Vertrauen zu schenken und sich bei Kaufentscheidungen nicht auf sie zu stützen.


Herunterladen ppt "Use this title slide only with an image Towards a Web-scale Data Management Ecosystem Demonstrated by SAP HANA Stefan Bäuerle, Jonathan Dees, Franz Faerber,"

Ähnliche Präsentationen


Google-Anzeigen