Die Präsentation wird geladen. Bitte warten

Die Präsentation wird geladen. Bitte warten

Data SAP Axel Herbst Performance, Data Management & Scalability SAP AG.

Ähnliche Präsentationen


Präsentation zum Thema: "Data SAP Axel Herbst Performance, Data Management & Scalability SAP AG."—  Präsentation transkript:

1 Data SAP Axel Herbst Performance, Data Management & Scalability SAP AG

2  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 2 The Life Cycle of SAP Data t t DATABASEFILE SYSTEM S T O R A G E S Y S T E M Creation Non changeable Business complete Residence time Data access Deletion Importance Audits

3  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 3 Legal Compliance - Record Retention Periods Pharmac./ Life Sci. (21 CFR Part 11) Healthcare (HIPAA) Financial Services (SEC 17a-4) OSHA Sarbanes Oxley Minimum Retention Period on Compliant Media (Years) years after completion - Original correspondences from financial audits or publicly-traded corporations 30 years after completion of audit - Employee and medical records of individuals exposed to toxic substances End of account +6 years - Trading account records 2 years after patient’s death - Medical records 21 years+ (perhaps for life) - Medical records for minors from birth to 21 End-of-life of enterprise - Member registration for broker/dealers 5 years - All hospitals must retain records in originally or legally produced form 5 years after end of manufacturing or product - Records relating to the manufacturing of biological products 3 years after distribution - Records relating to the manufacturing, processing, packing of drugs and pharmaceuticals 2 years after commercial release - Records relating to the manufacturing, processing, packing of food 3 years - Financial statements Source: Enterprise Storage Group, May 2003

4  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 4 Shared Memory and Buffers Dispatcher SPO WP BTC WP DIA WP Shared Memory and Buffers Dispatcher DIA WP ENQ WP Enqueue Table SAP GUI DBWP Gateway Front End Application (e.g. Word) DBMS Database Web Server Architecture of an ABAP-Based SAP Component SAP GUI Gateway

5  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 5 Size of Database < 300 GB GB GB GB TB >1.5 TB Survey Results from the American SAP User Group ASUG (Archiving Track)

6  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 6 Today's requirements to RDBMS in SAP Systems Requirements concerning business objects Several TB net data in RDBMS Size of a business object between some KB and some 100 MB Typically at least 3 search criteria  Business partner  Material/product/service/...  Date Need to access business objects for several (even 10+) years Requirements concerning OLTP 1000 database transactions per second 50 MB/s write access to database files Database transactions change hundreds of tables (dozens of business objects) Recovery in less than 15 minutes 7*24 hours (2 hours planned down time a month)

7  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 7 System Availability - Downtime Costs Per Hour Downtime Losses by Application Typical Loss Per Hour of Unplanned Downtime (US$) Financial/Trading$2,400,000 Supply Chain$600,000 ERP$600,000 CRM$480,000 E-Commerce$480,000 E-Business$480,000 Business Application$300,000 Database$300,000 Messaging$60,000 Infrastructure$42,000 Source: Storage Magazine

8  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 8 Use of Resources - Storage Savings Data Records 10 GB Total savings in this scenario: 97.5% Mirrors 20 GB Archiving Compression to 20 % 2 GB2 GB Backup to Tape System Copies 80 GB

9  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 9 Long-Term Issues Costs Efficient archive format: compact, indexable Inexpensive storage system/media Easy administration (low TCO) Long-term reuse Appropriate search/query capability to find archived data Readability of archive  Replaceable/maintainable/migrateable hardware and software components Interpretability (self-contained and self-describing format, schema) Application connectivity (open interface/protocol) Application integration  Enough semantics/context for future use  Archival of referenced documents Security  Write once guarantee (-> for legal, tax and audit reasons)  Authorization, authentication, nonrepudiation, privacy, encryption Operating (archiving in production environment) Mass data transfer from DB to archive Automation, Monitoring,...

10  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 10 Zusammenfassung: Datenarchivierung – Warum? Kontrolliertes Wachstum der Datenbank  Kosteneinsparungen geringere Hardwarekosten geringere Verwaltungskosten  Bessere Performance schnelleres Sichern und Wiederherstellen schnelleres Erstellen der CBO-Statistiken schnellere Release-Wechsel  Kürzere Antwortzeiten für Endbenutzer bei Geschäftsvorfällen mit in DB verbleibenden Daten Langfristige Speicherung von Daten in “nutzbarer” Form  … wegen neuer rechtlicher Anforderungen (GDPdU, SOA) … trotz mehr “Bewegung” in der IT-Landschaft (End-of-life System vs. Data) DB-Platz Zeit

11  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 11 Application-oriented DB Archiving Architectural classification w.r.t. provider of archiving service DBMS-based DBMS-integrated DBMS DB Application with archiving functionality DB Archive DBMS with archiving functionality Application Archive

12  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 12 SAP Data Archiving For SAP, only the DBMS-based approach is an option. However, SAP applications use Basis Archiving Functionality. DBMS DB SAP basis archiving functionality („ADK“) Archive SAP application with archive integration DBMS independence (DB2/..., Informix, MS SQL Server, Oracle, MaxDB)  Availability: DBMS-integrated approach only in DB2/390 RAM  Even „Standard“ SQL needs to be unified (SAP „Open SQL“) Tertiary storage integration  Needs to be vendor-independent as well: dedicated archiving/CM systems connected via certified interface to store archive files Application „awareness“ of archiving  DB schema hardly contains application semantics (almost no integrity constraints on DB level, business context for archive access,...)

13  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 13 Datenarchivierung aus SAP-Kundensicht Archivdateien Auswertungen/Zugriff archivierte Daten lesen Datenobjekte in Datenbank löschen Datenbank Anwendungsdaten in Archivdateien schreiben

14  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 14 Anwendungsbezug durch Archivierungsobjekte Archivierungsobjekt Struktur: Definition der logisch (aus betriebswirtschaftlicher Sicht) zusammenhängenden, gemeinsam zu archivierenden Daten – einschl. Kontext = “Schema” (allerdings nicht Archivschema) Verhalten: Programme, Anwendungsspezifische Prüfungen Technische Einstellungen Daten Customizing Programme für CO- Belege für FI-Belege Archivierungs- objekt

15  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 15 Programme eines Archivierungsobjekts … sichtbar als Aktionen für den DA-Administrator

16  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 16 Kernfunktionen in Archivierungsprogrammen I Schreibprogramm Berechtigungsprüfungen Einlesen des anwendungsspezifischen Customizings Lesen von der Datenbank: SELECTs aus allen beteiligten Tabellen/Views Archivierbarkeitsprüfung Erzeugen eines neuen Archivierungslaufs Aufbau komplexer (Daten-)Objekte aus gelesenen Sätzen Sichern der Datenobjekte in Archivdatei Ausgabe eines Protokolls Abschluß des Archivierungslaufs ggf. Anstoßen einer Folgeaktion (Löschen oder Ablegen)

17  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 17 Beispiel für Archivierbarkeitsprüfungen Erreicht ÄnderungsdatumErfassungsdatumArchivierung Zeit Verkaufsbeleg Residenzzeit Beginn 2 Beginn 1 Faktura SD_VBRK Lieferung RV_LIKP Verkaufsbeleg SD_VBAK... Abhängigkeiten im Belegfluß Berücksichtigung der Residenzzeit

18  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 18 Kernfunktionen in Archivierungsprogrammen II Löschprogramm Berechtigungsprüfungen Einlesen des technischen objektspezifischen Customizings (z.B. Transaktionsgrenzen) An die aktuelle Plattform und das aktuelle Release (DB-Schema!) angepaßte Lesen der Datenobjekte aus der Archivdatei Löschen der korrespondierenden Daten(sätze) in der Datenbank (SAP Business Information Warehouse: DROP PARTITION) Ausgabe eines Protokolls Abschlußarbeiten ggf. Anstoßen einer Folgeaktion (Ablegen oder Nachlauf)

19  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 19 Archivzugriff Sichtbarkeit von archivierten Daten eigenständiges Archiv integriertes Archiv … je nach SAP-Anwendung! Bewußte Unterscheidung DB Archiv oder uniformer Zugriff bei Anzeige aus Anwendungstransaktion heraus Generische Tools  Archivinformationssystem  Document Relationship Browser

20  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 20 Das Archive Development Kit – ADK ADK = Generische Entwicklungs- und Laufzeitumgebung für ABAP- Archivierungsprogramme Von SAP für alle Archivierungsobjekte eingesetzt  Schreiben, Löschen, Lesen, Zurückladen, Umsetzen  vom Archivinformationssystem benutzt Für Kunden zur Entwicklung eigener Archivierungsobjekte freigegeben ADK-Komponenten Laufzeitsystem  Kapselung von anwendungsunabhängigen Basisfunktionen, vor allem in „Richtung Archiv“ Repository  Verwaltungs- und Metadaten Administration  Benutzerschnittstelle für DA-Administrator

21  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 21 ADK als Laufzeitumgebung ADK- Laufzeitsystem SAP-System Dateisystem Archivdateien HSM- System Ablagesystem Datenfluß (beim Schreiben) Steuerfluß Datenobjektdienste Konvertierungen Dateiverwaltung Statusverwaltung Klassenaufrufe Statistikmodul CMS-Anbindung AS-Aufrufe Monitoring Jobsteuerung Archivadministration Datenbank Anwendungsdaten Verwaltungs- und Metadaten Archivierungs- programme AS DA- Monitor Hinter- grund- verarbei- tung Archive- Link/ Content Mgmt.- Service

22  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 22 Temporäre Konvertierungen bei lesenden Zugriffen Plattformanpassung Dateipfad entsprechend aktuellem Betriebssystem Zeichensatz (Codepage) und Zahlenformat  für ADK-Format-Metadaten (Header, Tags)  für zeichenartige und numerische Anwendungsdaten  Erfordert „Bootstrapping“ beim Header-Lesen Strukturanpassung -> Umgehen mit Schemaevolution Schema in Archivdatei wird mit aktuellem Schema (aus DDIC) verglichen Bei namensgleichen Strukturen (Tabellen)  Namensgleiche Felder müssen zuweisungskompatibel sein  Initialwert bei noch nicht zum Archivierungszeitpunkt vorhandenen Feldern  Ausblenden nicht mehr vorhandener Felder

23  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 23 Transaktionsaspekte Konsistenz Datenbank Dateisystem Physische Datei erst dann logisch sichtbar, wenn „fclose“ okay; d.h. vom Schreibprogramm übergebene Datenobjekte nicht logisch synchron archiviert „orphan files“ unkritisch Konsistenz Datenbank-Löschen Archiv-Schreiben Pragmatische effektive Lösung durch 2-Phasigkeit Ausschluß von Änderungen durch Anwendung zwischen Schreib- und Löschphase  Read-only-Zugriff und Archivierbarkeit über Status gewährleisten oder  unkritische Änderungen zulassen oder  Anwendungssperren verwalten

24  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 24 Repräsentation der Phasen in der Archivverwaltung Write Phase Delete Phase File 1 Delete Phase File 2

25  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 25 Auswahl eines geeigneten Archivierungsobjekts

26  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 26 Auswertung von DA-Statistiken

27  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 27 Free Space Reserved for Update Extension Overhead Area Data Rows Data Removed During Archiving Session Additional Session Extends to Minimum Level Database Page Concepts

28  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 28 Free Space Overhead Area Reserved for Update Extension Fragmentation

29  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / Index... Table Effect: Index range scans: More index leaf blocks read More tree levels -> Index Reorganization/Rebuild! After Delete Processes Index Fragmentation

30  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 30 Customer Example Starting point Approx. 290GB DB size and approx. 15GB DB growth per month Aim Reduction of DB growth rate to: Reduce hardware costs Maintain stable system performance  Response times and system administration  Faster implementation of support packages and upgrade projects  Local currency conversion Archiving 19 archiving objects from FI, CO, MM, SD, and HR Result (after 15 months) > 200GB archived

31  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 31 Customer Example Mar Feb May Jun Jul Aug Sep Oct NovDec Jan Feb Mar Apr May Jun Jul AugSep DB growth: ~15GB/month 'Without' Archiving Initial Archiving Reduction: ~60GB With regular archiving DB growth: ~7GB/month Allocated DB content Expected size without Archiving Allocated DB size

32  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 32 ADK-Based Data Archiving... proved to be a valued concept and implementation for ABAP-based SAP components. And will continue to do so. However, let‘s review the long-term issues... and discuss the pros & cons of new ideas.

33  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 33 Drivers for XML-Based Archiving Standards for long-term use of archived objects XML http(s) WebDAV Java J2EE Archive access Independent of “home“ system Potentially cross-system For Java applications as well Central archiving service Reduced redundancy in distributed scenarios Central administration

34  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 34 XML-Based Archiving – A Service Approach Services Database XML DAS Connector mySAP X XML DAS Connector mySAP X Archive Browser XML App. System Database Open and inexpensive storage system Business data archive XML Data Archiving Service (XML DAS) Services

35  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 35 ADK- vs. XML Archiving HTTP WebDAV File system File Resource Storage system XML Archiving Programs SAP Web AS XML Archive API Local Archive Admin. XML DAS Connector XML DAS Adminis- tration SAP J2EE Engine XML Data Archiving Service (XML DAS) ADK Archiving Programs SAP Web AS ADK Local Archive Admin. ArchiveLinkFile system FileDocument Storage system ArchiveLink

36  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 36 WebDAV-Like Archive Hierarchy Archive Store /b4t/ 000 /b4t/000/bc_sbook_x/ 2003 Resource Archive path Name URI = Archive path + Name XSDXML Collection Archive path Name /b4t/000/ bc_sbook_x / b4t /b4t/000/bs_sbook_x/2003/ order_schema.xsd /b4t/000/bs_sbook_x/2003/ order_4711.xml /b4t/000/bs_sbook_x/2003/ order_4712.xml System ID x Client x Arch. Store1 Arch. Store2 Arch. Store3 XML archiving object y

37  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 37 Properties of Resources WebDAV concept for attributing and finding resources A set of properties can be defined independently of resources. Such a property set is called property index. The property index is used to attach property values to a resource either when a resource is archived or later on. Properties are typed:  VARCHAR(n)  … Properties are used in value-based queries XML OrderNumber: 4711 OrderDate: 2005/01/10 ShipTo: Eppingen-Rohrbach

38  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 38 Storage System Support Storage system protocols service interface: WebDAV-like HTTP methods WebDAV URL Phys path Resource Collection File Directory PUT XML document/stream MKCOLURI WebDAV client File system I/O internal storage abstraction resources (XML doc n) Resources (XML doc 1) Application layer Storage layer Global service layer

39  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 39 Open Archive File Format A wrapper format ensuring… Efficiency (many data objects, byte-addressable for random access) Long-term interpretability (XML schema, XML, easy to construct) XML resource Standard Decompression Prefix Compressed Resource 1 Prefix Compressed Resource 2 Prefix Compressed Resource n... Offset 1 Offset 2 Offset n Header

40  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 40 XML DAS Functionality Beyond WebDAV Write once, resources cannot be modified  Insertion of resources into collections can be disabled  Deletions possible (occurs with logging) Identification of resources  Via unique URI  Long-term and stable Archive queries  Hierarchical search (using the path)  Value-based search (with property indexes)  Future: Content search using other engines No WebDAV spec ambiguities  No case sensitivity anywhere  Within one collection, resources and collections cannot have same name  More status codes

41  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 41 XML DAS Functionality Beyond WebDAV XML awareness  TYPE = { XML | XSD | XSL | BIN | COL | RES | ALL |... }  Check XML for well-formedness, and validity against schema  Keep meta data (XSD, XSL) “close“ to business data Automatic naming  Unique name for a resource within a collection Integration of different storage systems  Independent of the logical XML DAS hierarchy  Supports data life cycle management Archiving application integration  For example, support of safe delete from DB, even in one phase Security  Authorization, HTTPS, check sum Pack resources  Optional  Asynchronous

42  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 42 ABAP XML Archiving Object BC_SBOOK_X Transaction SARA also used in XML-based archiving

43  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 43 JAVA Archiving: A System Deployment Scenario SAP J2EE Engine XML Data Archiving Service 1 …n instances Archiving programs run here XML DAS Connector for JAVA SAP J2EE Engine Java Application

44  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 44 Java Archiving: GUI Prototype

45  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 45 Positioning ADK and XML ADK: primarily for reducing size of database; reliable, stable, secure; for ABAP only XML: advantages when end-of-life of data longer than end-of-life of system and in multi system environments; for Java as well Striking Differences for Users Storage in the form of resources in standardized XML format instead of ADK files  Archived resources can be read by your tool  Easier interpretation in the long term  Compression optional through pack function Application-specific searches with help of property indexes Direct archiving, no separate store phase (WebDAV or file system) Schedule as many delete jobs as reasonable

46  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 46 About SAP Data Archiving Focus SAP R/3 Enterprise Further components SAP BW SAP CRM Detailled information about Technology and administration (ADK) Data storage and data access Implementation of archiving projects Authors Archiving experts at SAP (also mal nicht Küspert, Schaarschmidt, Zeller, Langguth ;-)

47  SAP AG 2005, UniJena05.ppt, A. Herbst, 10 Jan 05 / 47 No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice. Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors. Microsoft®, WINDOWS®, NT®, EXCEL®, Word®, PowerPoint® and SQL Server® are registered trademarks of Microsoft Corporation. IBM®, DB2®, DB2 Universal Database, OS/2®, Parallel Sysplex®, MVS/ESA, AIX®, S/390®, AS/400®, OS/390®, OS/400®, iSeries, pSeries, xSeries, zSeries, z/OS, AFP, Intelligent Miner, WebSphere®, Netfinity®, Tivoli®, Informix and Informix® Dynamic ServerTM are trademarks of IBM Corporation in USA and/or other countries. ORACLE® is a registered trademark of ORACLE Corporation. UNIX®, X/Open®, OSF/1®, and Motif® are registered trademarks of the Open Group. Citrix®, the Citrix logo, ICA®, Program Neighborhood®, MetaFrame®, WinFrame®, VideoFrame®, MultiWin® and other Citrix product names referenced herein are trademarks of Citrix Systems, Inc. HTML, DHTML, XML, XHTML are trademarks or registered trademarks of W3C®, World Wide Web Consortium, Massachusetts Institute of Technology. JAVA® is a registered trademark of Sun Microsystems, Inc. JAVASCRIPT® is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by Netscape. MarketSet and Enterprise Buyer are jointly owned trademarks of SAP AG and Commerce One. SAP, SAP Logo, R/2, R/3, mySAP, mySAP.com and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned are trademarks of their respective companies. Copyright 2005 SAP AG. All Rights Reserved


Herunterladen ppt "Data SAP Axel Herbst Performance, Data Management & Scalability SAP AG."

Ähnliche Präsentationen


Google-Anzeigen