As part of our vision to help enterprises gather business insights and key metrics, we assisted the design and development of a big data warehouse to store and query their customer actions and interactions. We used Amazon’s Web Services – S3 for log storage, Elastic Map Reduce for log analysis, EC2 with an EBS backed up MySQL for the database and Pentaho as the reporting front end. The following picture depicts the architecture of the system, and the data flow.
With this architecture in place, it was pretty easy to provide business analysis and day to day reporting to all the users spread across various geographies. And that too in a scalable, fault tolerant way with pay as you go.
