SIHG4SR — Side Info Heterogeneous Graph for Session Recommender

…the video.

Okay, hi, this is Chendi Xue. I'm from Intel, and this is our team. We use SIHG4SR, which stands for Side Info Heterogeneous Graph for Session Recommender System Problem.

So since we only have 10 minutes, so I just include the key contributions from our solution in this slide. I will talk about the graph, the side info heterogeneous graph representation. Then there are four main contributions, which are feature engineering, the training strategy, the hyperparameter tuning, and the multi-level model ensemble.

Okay, so for the model, we are simply using a GNN-based model. So there are three phases in our model. The first phase is to convert the tabular data to the heterogeneous graph. I think you already know that there are multiple sessions in this dataset. Actually, there are one million sessions in this dataset. So each session will be built into a graph, like the one at the side.

And so there are three types of nodes in the graph. The first type of node is item, which means when the user views one item, then the next item, and the next item, it will just link as the item sequence. Then we are using a second type of node, which is a feature. Since the feature data in the dataset is anonymous, but we are trying to think of the features as meaning something. So we can think like the feature maybe means like the color is white, the length is long, the discount is 70%, something like that. Then we can think of like we are to link the item to the corresponding feature. Then there is a third type of node, which is category, which means what is the feature category of this feature. And then we do this heterogeneous graph.

Then we're feeding this heterogeneous graph into our GNN. And then we will have three types of embedding table because we have three types of nodes. Then the embedding tables are item, feature, and the category. So we didn't aggregate these three types of nodes immediately, but we use two channels to propagate these two types of information. The first channel… The first channel is only for the items, like this channel, and the second channel is for feature and the category aggregation. So it's very straightforward to think of why we use the first channel only for the item, because this is the main information. And then this is something about the side information, which is the feature and the category.

And to here, you can see there is a problem, because for the feature, the number of nodes is seven, but the item is only five. So how can I aggregate these two pieces of information together? This is how we are doing different types of reshape to the original embedding table. For feature, we will, we will try to do the shuffling, which means like where white belongs to item one, item two, and item five, we will just add feature one to item one, item two, and item five. Then we will, we will have a new reshaped embedding table, which shares the same size as the item. Then for the category, we try to just do a mean and the broadcast, because the category is more general information. Then we add these two reshaped embedding tables together and decode as the second logit.

Then we fuse these two channels of logits together to get our final logit, and use a weighted loss. Weighted loss here means — because this is a question about fashion, which means the most recent data will tell us more than the maybe one-year-ago data. So this is why we're using a weight to tell us that the data is recent or the data is years ago. So this is how we build our model. So we can predict the candidate items with the highest scores on the right side. [garbled in the recording: “that kind of document with a really great course”]

And then there are, as I mentioned, there are four key contributions to improve our score. The first one is the feature engineering work. So you already know that there are three types of nodes and three types of edges in our graph. Then we are thinking that putting which kind of node into the graph maybe will either contaminate or contribute to the final score. This is how we are trying to just exclude one category or feature each time, to see if there are some categories maybe just contributing some conflicting information into our GNN, which brought the score down. This is why we exclude the categories 29 and 44.

Then we also think about if there is any chance that we can build new features into the graph, which is because we have the timestamp in the dataset, and we are thinking this is a very good source for new features. So there are three new types of feature we are creating. One is we just do statistics on the history item click count, to say, oh okay, this item is hot, it was clicked multiple times, and this item was only clicked one or two times. And the second feature is the elapsed time to the session end. Why are we adding this type of information? Because you can think that if one user opened a session maybe two hours ago, then she or he does the purchase at the last one minute. So the items she or he viewed in the last five minutes contribute more to her or his decision than the item from two hours ago. This is why we are using the elapsed time as a new feature. Then the feature item publish time, this is another type of new feature.

So besides the new features we are creating, we are also thinking about whether there is any chance we should drop the edges. This is what we draw in the bottom part of this slide. So firstly we keep all the edges, which are the item sequence edges, the item-to-feature edges, and the feature-to-category edges. Then we try to just remove the item sequence edges, because we already have the elapsed time, so we don't need to use the sequence. Then we also try to do a random drop of the edges with a probability of 0.1, which helps us to fix the overfitting issue.

Okay, then the second key contribution is the training strategy. So we are using two types of training strategies. The first one is just to load in the dataset, create a graph for each record, and then train using the purchase item as a label. Then we're thinking that since the next click also reveals the user's intention, right, so how about we use the next click to pre-train the model, then we use the real purchase item to train the model. It's something like borrowing from transfer learning: we can train a model with a bigger dataset, and then we train on the real problem.

So what we do here is: the first session is something like this. Then we augment it into two sessions. The first one is the first click, with the next click as its label. Then the second one is the whole session with the purchase item. And then, because the dataset is too big, we found that only using the last six months of the dataset is already enough. And this is how we use the training strategy to improve the score.