Ads Candidate Generation using Behavioral Sequence Modeling

Pinterest's Ads team developed transformer-based behavioral sequence models to improve ad candidate generation using users' offsite activity history.

• •A two-tower model with a bidirectional transformer user tower and MLP advertiser tower predicts which advertisers a user is most likely to convert with next.
• •Training uses sampled softmax loss with log-Q bias correction and in-batch negatives, with positives defined as checkout/add-to-cart/signup events from a K-day future window.
• •An offline batch workflow precomputes top-100 advertiser candidates per user, stored in a feature store for low-latency online serving via an L1 ranker.
• •A second item-level model extends this to predict specific product interactions from a 1B+ item corpus, using internal Pin embeddings and catalog metadata.
• •The item model showed up to 45% improvement in user checkout recall offline and significant CPA reduction with conversion lift in online A/B experiments.