Problem Description:
PriceSpider provides a commerce platform for brands to drive conversion, track performance, monitor retailers, monitor digital commerce, ensure guideline compliance, and take action on all these inputs unifying brand marketing and performance marketing. Their clients, including many of the world’s top brands, rely on them for reliable, timely data.
PriceSpider has a suite of products that are logically separated and deployed at different both AWS and GCP clouds. Crawler services continuously produce millions of data every day and these are consumed by multiple applications.
Due to the highly distributed nature of the platform. It’s very challenging to monitor the life cycle of messages that flow through different services.
Solutions Highlights:
AVM worked with PriceSpider team to build a shared data platform that uses Kafka as the core messaging hub and Datadog as the observability platform. All the products within the company were analyzed to understand observability requirements concerning the shared data model.
The engineering team at AVM proposed a correlation model for distributed components along with a proof of concept. Furthermore, custom metric exporters were introduced to monitor managed services that do not provide built-in support for Datadog monitoring.
The proposed distributed observability model helped PriceSpider team to simplify the platform complexity and use Datadog as a single pane glass for their multi-cloud platform.
