Problem Description:
PriceSpider provides a commerce platform for brands to drive conversion, track performance, monitor retailers, monitor digital commerce, ensure guideline compliance, and take action on all these inputs unifying brand marketing and performance marketing. Their clients, including many of the world’s top brands, rely on them for reliable, timely data.
PriceSpider processes tens of millions of business events per day. Their Engineering and Business Ops teams make use of Kibana based dashboards to ensure these events are flowing correctly and customer changes are visible quickly.
In order to onboard new customers and fulfill customer feedback critical events must appear in these dashboards as soon as possible. They were seeing cases where some (but not all) events were severely delayed impacting multiple customers each day. They found nothing in common about these events or any root cause.
Solutions Highlights:
AVM worked with PriceSpider’s Platform Engineering team on understanding the event pipeline that included AWS Lambda, Amazon Kinesis Data Firehose, and Amazon ElasticSearch Service and its Kibana as well as establishing the KPIs important to all stakeholders. Based on that, AMV Streaming experts then performed an end to end analysis of the component configurations and historic performance using CloudWatch to pinpoint the problem areas and inefficiencies.
Within weeks AVM distilled many separate issues in the pipeline into new resiliency strategies in Firehose and ElasticSearch that exceed the performance SLAs 99.99% of the time, improved observability and alerting of the complex pipeline, and proposed developing additional Lambda based tools to evolve the pipeline to a self-healing system. In addition, AVM found cost savings of over 50% with a more resilient architecture.
