Skip to main content

Crowd signals: co-play and popularity

Tags describe what games are. Crowd signals describe what customers do: games that the same people like together, and games many people really play. Both are computed in one pass by CrowdAccumulator (src/Kaleido.Gre.Recommender/Crowd.cs) during a crowd build.

Co-play: item-item cosine over customers​

Think of a matrix with one row per customer and one column per game, holding affinities. Two games are co-played when their columns point the same way.

cos(i, j) = sum over customers u of a(u,i) x a(u,j)
-----------------------------------------
sqrt(sum_u a(u,i)^2) x sqrt(sum_u a(u,j)^2)

The accumulator builds this incrementally, one customer at a time, without materialising the matrix:

GuardValueWhy
Games per customertop 30 (20 from Redis)Bounds pair work at about n squared / 2 per customer
Minimum support3 customersOne coincidence must not become a trend
Neighbours kept30 per gameEnough for candidate generation, small enough to hold in memory
Pair key(i << 32) or j in a longOne dictionary entry per unordered pair

The result is stored as Kind = 1 rows in recommender.neighbour with Score (cosine) and Support (shared customers).

Popularity​

A customer counts as a player of a game when their affinity for it is at least 0.3. Popularity is the number of players divided by the number for the most-played game, so the top game scores 1. It is stored in recommender.popularity with the raw customer count. The 300 most popular games are always added to the candidate set, which is what makes cold-start lists possible.

Where the affinities come from​

RedisCrowdFeeds walks the live profiles with SCAN. When there are more customers than Crowd:SampleCustomers (250,000), it keeps a stable sample: an FNV hash of the key mapped to 0 to 1 decides membership, so the same customers are sampled on every page and every run. Co-play is a statistical signal, so a large random sample gives the same links as everyone without holding over a million customers' pairs in memory. The catalogue part of the feed still comes from the Taxonomy API.

With Builds:CrowdEveryMinutes set (compose uses 0.17, about every 10 seconds), streamed play improves recommendations without anyone pressing a button. Builds never overlap; a slow build stretches the gap.

Limits​

The accumulator runs in memory, which is fine up to a few hundred thousand customers. Beyond that, the code comment names the next step: a batch job with ALS or sharded counting.