Collect Analytics for Large Video Platform

posted on   July 25, 2012

Project: Springboard Video Platform (author Milos Todorovic)

Request:  Track analytics for 500 partners and for half a million videos

Tech: JavaScript, Flash, PHP, MySQL,Key-Value DB

Problem: Collect and insert large amount of data into system to match reading speed, scalability, and none-zero bounced requests.

Solution: Flash Media and HTML5 video players are sending analytics data as they fire request with query string parameters to the load-balanced chain of servers. Each quarter of the video content is tracked (start, 25%, 50%, 75%, end).

Servers are receiving query string values into fast key-value database that is stored into RAM memory. This way we managed to temporarily store data very fast, before it’s processed.
Every 15 minutes all raw data from key-value database will be flushed from RAM and inserted into intermediary MySQL table that is partitioned by time. MySQL stored procedures will trigger periodically and crunch data from intermediary table into appropriate Site database. All tables in Site database are constructed such that MySQL queries that should serve Front-End will hit a minimum number of rows.

posted in   Uncategorized | No Replies

Leave a Reply

Your email address will not be published. Required fields are marked *