Project: Big Data Solution Boosts Performance of Video Commerce Platform
Client: Industry Leader in Video Commerce
For many retailers and brands who are feeling the pressure to compete in an overly crowded online marketplace, videos have become one of the most powerful ways to differentiate themselves and connect with their consumers. The challenge for many companies is not being able to effectively turn their video content into sales. One company is leading the way to solve this challenge.
The company considers itself the most comprehensive shoppable video platform which empowers brands and retailers to source, merchandise and publish Video Commerce Experiences at scale.
Challenge: Platform Infrastructure Not Able to Support Large Quantities of Data
When Dunn Solutions met its client, their cloud based platform was not able to populate their reporting dashboards for their larger customers, some of whom were major retailers. Their infrastructure did not support analysis on the big data being generated. The company was using a PHP application to get real time data and feed it to their MySQL database. This was a complex process because of the many transformations taking place within their PHP code which prevented it from operating in real time.
Because of the volume of data their analytics dashboard could not return results for more than a month's worth of data for many of their enterprise level customers.
Solution: From Complex PHP Code and MySQL to AWS Services and Snowflake
Dunn Solutions re-architected the solution by capturing events data through AWS Kinesis and AWS Kinesis Firehose. Using Kinesis Firehose, Dunn Solutions transformed the JSON events data in real time and staged it in an AWS S3 bucket. The next step was to create an ingestion process within Snowflake to get the data from the S3 bucket in real time. Finally, AWS Lambda Functions were used to process the real time data within a Snowflake data lake and populate their resulting analytics tables.
Moving away from complex PHP code to using AWS services and Snowflake yielded a significant improvement in our client's platform infrastructure. It made it easier for them to maintain the process as well. Dunn Solutions also developed a real time dimensional model which was easy to query and resulted in both our client and their customers understanding their data better.
Result: Critical Performance Improvements Support Business Growth
The client is now able to populate their dashboards for their largest customers within less than a second – which was not possible before. They are now able to provide their customers with accurate and up to the minute data which is critical to retaining and growing their business.
