Deutsche DWHs 2007 - Analyse und Trends Kai Fischer Strategisch Technische Unterstützung (STU ) Terabyte Club 2007 13. Februar Oracle Frankfurt.

Präsentation zum Thema: "Deutsche DWHs 2007 - Analyse und Trends Kai Fischer Strategisch Technische Unterstützung (STU ) Terabyte Club 2007 13. Februar Oracle Frankfurt."—  Präsentation transkript:

1 Deutsche DWHs 2007 - Analyse und Trends Kai Fischer Strategisch Technische Unterstützung (STU ) Terabyte Club 2007 13. Februar Oracle Frankfurt

2 Ziele: -Eine Oracle Community in D etablieren -Maßnahmen aus Trends ableiten 1. Terabyte Club Treffen 2005 10 von ( 21 ) Firmen, 20 Teilnehmer Keine Betreiber anwesend Vorträge: O2, EDS Clearing House, DKRZ 2. Terabyte Club Treffen 2007 21 (von 38 ) Firmen, 37 Teilnehmer Erste Betreiber sind dabei ( Atos Origin, Postbank Systems, T-Systems ) Vorträge: Karstadt Quelle, Postbank Systems, Jomo Erste Analysen / Trends sind möglich Deutscher Oracle Terabyte Club – Quo Vadis ?

3 -T-Mobile -E-Plus -Debitel -O2 -T-Com -Talkline -Freenet / Mobilcom -Tengelmann -Bofrost -Jomo -Otto Versand -Karstadt Quelle -Globus SB Warenhaus -Deutsche Bank -Comdirect Bank -Postbank -HVB -Allianz Deutscher Oracle Terabyte Club – Wer ? -Volkswagen -Ferdinand Bielstein -Continental Werke -KKH -AOK Berlin -Insight Health -Deutsche Post -Landesvermessungsamt BaWü -DKRZ -GfK Informatik -Deutsches Apotheken Prüfungsinstitut -Thyssen Krupp Stahl -ENBW -Schenker -Rhenus AG -BASF -... In Summe: 411 Terabytes Tablespace Used ( incl. Kompression )

4 Betrieb des Data Warehouses

5 Aktuelle Architektur

6 CPU-Typ Verteilung

7 Betriebssystem Verteilung

8 Storage Vendor Verteilung

9 Storage Anschluß SAN oder direkt ?

10 Oracle Release

11 Oracle Datenhaltung

12 Durchsätze in MB/Sekunde 761 MB/Sek

13 Anzahl physikalischer Disks 170 Disks

14 Physikalische Disks/Terabyte 85 Disks / TB 45 Disks / TB

15 -Der Großteil der Kunden betreibt sein DWH selber -Betriebssysteme/CPUs gut verteilt, SPARC/Solaris am stärksten, Linux bzw. x86 beginnt zu wachsen -RAC wird zu 29% verwendet ! -Storage ist zu 85% über SAN angeschlossen -ASM und RAW Devices werden öfter als Filesystem verwendet -Bei 10g DWH fast nur ASM -Sehr unterschiedliche DWH Leistungsfähigkeit im IO-Bereich -> Disks/Terabyte -DWH erben operationale Anforderungen ( Online Backup, Mixed Workload, HA, Human Error etc. ) Status und Trends - Zusammenfassung

16 Oracle Information Appliance Initiative Joint solutions of Oracle with hardware vendors

17 Agenda Data warehousing market trends Oracle Information Appliance Initiative Definition Configuration methodology Q&A

18 Emerging Trends for DW Data warehouses have become ubiquitous parts of the IT infrastructure Problem: Data warehouse system configurations are easy to get wrong Goal: Make it easy to deliver A scalable system With reduced implementation times Eliminated deployment risks Lower cost

19 DW Configuration Problem: I/O DatabaseCPUsMemoryActuatorsLUNsDisksRaid DatabaseCPUsMemoryActuatorsLUNsDisksRaid An unbalanced configuration A balanced configuration 100% Possible Efficiency 100% Possible Efficiency 100% Achieved Efficiency < 50% Achieved Efficiency

20 Sample Customer Configuration FC-Switch1FC-Switch2 HBA1HBA2 HBA1HBA2HBA1HBA2 HBA1HBA2 FC-Switch1 FC-Switch2 P690 32 CPUs 24x 2Gb HBA 4x EMC DMX 2000 P2 frames 6x Max IO throughput: 25GB/sec Max IO throughput: 18GB/sec 72% capacity, 25%degradation Max IO throughput: 5 – 6GB/sec 20 -24% capacity, 76 – 80% degradation

21 Agenda Data warehousing market trends Oracle Information Appliance Initiative Definition Configuration methodology Q&A

22 Oracle Information Appliance Initiative Appliance Foundations Documented best-practice configurations for data warehousing For customers requiring flexibility and choice Benefits: High performance Simple to scale: modular building blocks Built on Oracle database and standard hardware Available today with HP, IBM, and EMC Information Appliances Scalable systems pre-installed and pre-configured: ready to run out-of- the-box For customers looking for the simplest, fastest solutions Benefits: High performance Simple to buy Fast to implement Built on Oracle database and standard hardware Available soon with Panta Systems Further announcements in coming months

23 Oracle Information Appliance Foundations

24 Oracle Information Appliance Foundation What is it? Documented balanced system configurations for pre-defined DWBI environments Starting point for sizing a system Balanced system consists of CPU, memory, I/O, and cabling Sizing factors are raw data, avg. concurrent users and workload complexity Leverages scalable, modular components Enables incremental growth (scale-in, scale-out) Mitigates implementation risks

25 Different OS and HW configurations Choice of Linux and proprietary Unix Small, medium, and large nodes Classical SMP and clustered solutions Different workload configurations Price/Performance versus I/O intensive Guided Choices Various alternative HW vendors and HW components Sweet spot of price, performance, and fit into existing environment Oracle Information Appliance Foundation What is it?

26 Varying workload Varying system size Spreadsheet-like guidance First entry point, not all available configurations Choice of workload and data volume Choice of hardware and operating system Sizing and pricing starting point (option 1) Sizing and pricing starting point (option 2) Oracle Information Appliance Foundation Sample OIA Foundation

27 Balanced system configurations are built on modular balanced building blocks Add building blocks as you grow Each building block is a balanced unit Disks DiskDisk Disks DisksDisks Disks Disks DiskDisk Disk Disk Disks Different building blocks provide guided choices Many small nodes versus few large nodes Oracle Information Appliance Foundation What is it?

28 What is a balanced unit ? The weakest link defines the throughput Each building block is a balanced unit Components to consider: CPU: Quantity and speed HBA (Host Bus Adapter): Quantity and speed Switch speed Controller: Quantity and speed Disk: Quantity and speed FC-Switch1FC-Switch2 Disk Array 1 Disk Array 2 Disk Array 3 Disk Array 4 Disk Array 5 Disk Array 6 Disk Array 7 Disk Array 8 HBA1HBA2 HBA1HBA2HBA1HBA2HBA1HBA2

29 Throughput Performance Componenttheory (Bit/s)maximal Byte/s HBA1/2Gbit/s100/200 Mbytes/s 16 Port Switch8 x 2Gbit/s1600 Mbytes/s Fibre Channel2Gbit/s200 Mbytes/s Disk Controller 2Gbit/s200 Mbytes/s GigE NIC 1Gbit/s80 Mbytes/s Infiniband10Gbit/s890 Mbytes/s CPU100-200MB/s Grid Components Rough Sizing numbers * 2Gbit based

30 Sample Balanced Unit The weakest link defines the throughput Each building block is a balanced unit Components to consider: CPU: Quantity and speed HBA (Host Bus Adapter): Quantity and speed Switch speed Controller: Quantity and speed Disk: Quantity and speed FC-Switch1FC-Switch2 Disk Array 1 Disk Array 2 Disk Array 3 Disk Array 4 Disk Array 5 Disk Array 6 Disk Array 7 Disk Array 8 Each machine has 4 CPUs All four servers drive about 4 * 100MB/s * 4 = 1600 MB/s HBA1HBA2 HBA1HBA2HBA1HBA2HBA1HBA2

31 Sample Balanced Unit The weakest link defines the throughput Each building block is a balanced unit Components to consider: CPU: Quantity and speed HBA (Host Bus Adapter): Quantity and speed Switch speed Controller: Quantity and speed Disk: Quantity and speed HBA1HBA2 HBA1HBA2HBA1HBA2HBA1HBA2 FC-Switch1FC-Switch2 Disk Array 1 Disk Array 2 Disk Array 3 Disk Array 4 Disk Array 5 Disk Array 6 Disk Array 7 Disk Array 8 Each machine has 2 Gb HBAs All 8 HBAs can sustain 8 * 200MB/s = 1600 MB/s Each machine has 4 CPUs All four servers drive about 4 * 100MB/s * 4 = 1600 MB/s

32 Sample Balanced Unit The weakest link defines the throughput Each building block is a balanced unit Components to consider: CPU: Quantity and speed HBA (Host Bus Adapter): Quantity and speed Switch speed Controller: Quantity and speed Disk: Quantity and speed FC-Switch1FC-Switch2 Disk Array 1 Disk Array 2 Disk Array 3 Disk Array 4 Disk Array 5 Disk Array 6 Disk Array 7 Disk Array 8 HBA1HBA2 HBA1HBA2HBA1HBA2HBA1HBA2 Each machine has 2 Gb HBAs All 8 HBAs can sustain 8 * 200MB/s = 1600 MB/s Each switch needs to support 800MB/s to guarantee a total system throughput of 1600 MB/s Each machine has 4 CPUs All four servers drive about 4 * 100MB/s * 4 = 1600 MB/s

33 Sample Balanced Unit The weakest link defines the throughput Each building block is a balanced unit Components to consider: CPU: Quantity and speed HBA (Host Bus Adapter): Quantity and speed Switch speed Controller: Quantity and speed Disk: Quantity and speed FC-Switch1FC-Switch2 Disk Array 1 Disk Array 2 Disk Array 3 Disk Array 4 Disk Array 5 Disk Array 6 Disk Array 7 Disk Array 8 HBA1HBA2 HBA1HBA2HBA1HBA2HBA1HBA2 Each machine has 2 Gb HBAs All 8 HBAs can sustain 8 * 200MB/s = 1600 MB/s Each switch needs to support 800MB/s to guarantee a total system throughput of 1600 MB/s Each disk array has one 2Gbit controller All 8 disk arrays can sustain 8 * 200MB/s = 1600 MB/s Each machine has 4 CPUs All four servers drive about 4 * 100MB/s * 4 = 1600 MB/s

34 High performance Offers proven and well-known configurations Simple to scale: modular building blocks Shorten the decision and implementation cycle Built on Oracle database and standard hardware Optimal integration into existing hardware infrastructure Oracle Information Appliance Foundation Benefits

35 Oracle Information Appliance Initiative Appliance Foundations Documented best-practice configurations for data warehousing For customers requiring flexibility and choice Benefits: High performance Simple to scale: modular building blocks Built on Oracle database and standard hardware Available today with HP, IBM, and EMC Information Appliances Scalable systems pre-installed and pre-configured: ready to run out-of-the-box For customers looking for the simplest, fastest solutions Benefits: High performance Simple to buy Fast to implement Built on Oracle database and standard hardware Available soon with Panta Systems Further announcements in coming months

36 Oracle Information Appliance Initiative

37 Information Appliance – An Example Panta 2700 Data Warehouse Appliance 8 Blade RAC cluster 4 AMD 2.2Ghz dual core processors per blade 8 GB memory per processor Total of 64 cores and 256 GB memory 96 TB database storage 532 disk drives 250GB SATA 7200 RPM / drive 2 Silverstorm 9024 Infiniband Switches Up to 12 GB / sec I/O throughput observed in database processing Oracle Database 10g with RAC and Partitioning Red Hat Enterprise Linux Advanced Server 4 Update 3

38 Information Appliance: New 1TB TPC-H in October 2006 The system: Example data warehouse appliance shown on the previous slide Pre-defined hardware and software stack The results: Performance: 59,353.9 QphH@1000GB Price-performance: $24.94 $/QphH@1000GB Compelling combination of performance and price-performance: #2 in absolute performance #2 in price-performance As of October 23, 2006: PANTA Systems PANTAmatrix, 59,353.9QphH@1000GB, $24.94/QphH@1000GB, available 4/15/07. Source: Transaction Processing Performance Council (TPC), www.tpc.orgwww.tpc.org

39 Agenda Data warehousing market trends Oracle Information Appliance Initiative Definition Configuration methodology Q&A

40 A Q &