Hello, I’ve been playing with the Pinsage example in examples/pytorch/pinsage and just have a question about the HITS@10 being .01–I understand that some modifications had to be made to the algorithm so that it would converge on an open source dataset, as it would not be very useful as an example with no data, but I’m curious whether a HITS@10 of .01 (I actually got .09) is as low as it seems at first glance for a “working” recommender? If this is indeed the probability of a correct cointeraction for a user given the top-10 predictions, the napkin math I did (1M ratings/6000 users ≈ 167 reviews per user, divided by 4000 movies = 1 in 24 random chance of correct answer, * 10 chances) would suggest random chance is around .4–actually slightly higher because you’re choosing 10 without replacement, but probably the same neighborhood.
I feel like I’m probably making an error in my assumptions as I’m not as familiar with recommender systems as classification, just want to know what it is.