We have had so much interest in the new Guinness World record, set by SAP Sybase IQ 16 for the fastest loading of Big Data, that I thought I would share some further details with you all...
This test of SAP Sybase IQ 16 was run on a standard HP ProLiant DL980 G7 server in an 80 core system (8 processors, 10 cores per processor) with 1TB RAM running Red Hat RHEL 6.2. The EDMT analytics archive from SAP partner BMMsoft, certified with SAP Sybase IQ 16, was used to ingest and query the following data:
- 6.5 million Documents: Records containing a document and metadata about the document. The document portions of the records were made of images, video clips, audio clips and office type documents.
- 500 million transactions: Records produced as a result of conventional transaction processing. These records are made of traditional data types such as numbers and character strings.
- 5 million short messages: Records produced by capturing short text messages. These records were made of a combination of email messages and SMS type messages.
The complete load process included 50 batches of data. Each batch was made of 16 files containing raw data representing the various data types outlined and totaled above.
The test recorded a load speed of 34.3TB/hour (loading and indexing 33.13TB of raw data in 57.91 min). Queries were executed throughout the load process to verify the accessibility of the newly ingested data.
This record is significantly faster than published claims by Oracle for Exadata (12 TB/hr- June 2012) and EMC for their Greenplum-based appliance (10TB/hr- Jan 2013) and SAP holds the only independently verified test for Big Data loading. :grin:
Once again – congratulations team, way to go!
