Network Computing is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Warehouse Data Earns Its Keep: Page 3 of 8

Getting transaction and internal financial data to the data warehouse quickly and efficiently is paramount. Alkhalaf says MasterCard is gunning for a real-time, load-as-you-go operation for loading, mining and analyzing data. Most loading is still done in batches, though that's beginning to change. MasterCard has started running IBM's WebsphereMQ (formerly MQSeries) and BEA Systems' Tuxedo middleware to post bank transactions in near real time instead of relying on batch data transfers.

Squeaky Clean Data

Transactions between retailers and banks include everything from bulk grocery purchases at Sam's Club to lunch at Sam's Diner. The company then loads the data into the data warehouse.

Since MasterCard doesn't have any control over how the retailer's and financial institution's transaction systems record Joe Smith's charges for bulk foods at the discount store or for a steak and mashed potatoes at the diner, the records in the data warehouse can get awfully messy, awfully fast. A bank couldn't just sample credit-card charges to "Sam's," for instance, since there's more than one retailer by that name.

Maintaining and measuring the quality of the data that lands in the data warehouse is a major challenge for MasterCard. There aren't any mature tools that measure the quality of the data, Alkhalaf says. "When you bring in multiple information from multiple transactional sources, there's no way to reconcile the data correctly," he says. "Your data warehouse isn't much use if the results are invalid because your underlying data is inaccurate and inconsistent."