DeepCTR Documentation

Easy-to-use,Modular and Extendible package of deep-learning based CTR models .

8,049 stars Python Python 76 files · ~29,753 tokens #autoint#click-through-rate#ctr#deep-learning#deepcross#deepfm#deepinterestevolutionnetwork#deepinterestnetwork
RAW Doc

Technical Documentation: shenweichen/DeepCTR

ℹ️ Provenance: Multi-Tier Ingestion · CodeWiki Reference

Overview (`README.md`)

DeepCTR





GitHub Issues







DeepCTR is a Easy-to-use, Modular and Extendible package of deep-learning based CTR models along with lots of
core components layers which can be used to easily build custom models.You can use any complex model with model.fit()
，and model.predict() .

  • Provide tf.keras.Model like interfaces for quick experiment. example
  • Provide tensorflow estimator interface for large scale data and distributed training. example
  • It is compatible with both tf 1.15 and tf 2.x.

Installation and compatibility

DeepCTR does not pin or install TensorFlow for you. Install a TensorFlow build that matches your Python, NumPy, CPU/GPU, and operating system first, then install DeepCTR:

bash
pip install tensorflow
pip install deepctr

For Python >=3.9, DeepCTR allows modern h5py releases with h5py>=3.7.0. If TensorFlow reports a NumPy conflict, follow the TensorFlow requirement for your selected TensorFlow release, for example using numpy<2 when required by TensorFlow.

Use public tensorflow.keras APIs in your own code and examples. Avoid mixing tensorflow.python.keras with tensorflow.keras, because tensorflow.python.* is private TensorFlow API and can break model serialization or optimizer/metric loading across TensorFlow versions.

Some related projects:

Let's Get Started!(Chinese
Introduction) and welcome to join us!

Models List

Model Paper
Convolutional Click Prediction Model [CIKM 2015]A Convolutional Click Prediction Model
Factorization-supported Neural Network [ECIR 2016]Deep Learning over Multi-field Categorical Data: A Case Study on User Response Prediction
Product-based Neural Network [ICDM 2016]Product-based neural networks for user response prediction
Wide & Deep [DLRS 2016]Wide & Deep Learning for Recommender Systems
DeepFM [IJCAI 2017]DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
Piece-wise Linear Model [arxiv 2017]Learning Piece-wise Linear Models from Large Scale Data for Ad Click Prediction
Deep & Cross Network [ADKDD 2017]Deep & Cross Network for Ad Click Predictions
Attentional Factorization Machine [IJCAI 2017]Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks
Neural Factorization Machine [SIGIR 2017]Neural Factorization Machines for Sparse Predictive Analytics
xDeepFM [KDD 2018]xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems
Deep Interest Network [KDD 2018]Deep Interest Network for Click-Through Rate Prediction
AutoInt [CIKM 2019]AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks
Deep Interest Evolution Network [AAAI 2019]Deep Interest Evolution Network for Click-Through Rate Prediction
FwFM [WWW 2018]Field-weighted Factorization Machines for Click-Through Rate Prediction in Display Advertising
ONN [arxiv 2019]Operation-aware Neural Networks for User Response Prediction
FGCNN [WWW 2019]Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction
Deep Session Interest Network [IJCAI 2019]Deep Session Interest Network for Click-Through Rate Prediction
FiBiNET [RecSys 2019]FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Prediction
FLEN [arxiv 2019]FLEN: Leveraging Field for Scalable CTR Prediction
BST [DLP-KDD 2019]Behavior sequence transformer for e-commerce recommendation in Alibaba
IFM [IJCAI 2019]An Input-aware Factorization Machine for Sparse Prediction
DCN V2 [arxiv 2020]DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems
DIFM [IJCAI 2020]A Dual Input-aware Factorization Machine for CTR Prediction
FEFM and DeepFEFM [arxiv 2020]Field-Embedded Factorization Machines for Click-through rate prediction
SharedBottom [arxiv 2017]An Overview of Multi-Task Learning in Deep Neural Networks
ESMM [SIGIR 2018]Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate
MMOE [KDD 2018]Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts
PLE [RecSys 2020]Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations
EDCN [KDD 2021]Enhancing Explicit and Implicit Feature Interactions via Information Sharing for Parallel Deep CTR Models

Citation

If you find this code useful in your research, please cite it using the following BibTeX:

bibtex
@misc{shen2017deepctr,
  author = {Weichen Shen},
  title = {DeepCTR: Easy-to-use,Modular and Extendible package of deep-learning based CTR models},
  year = {2017},
  publisher = {GitHub},
  journal = {GitHub Repository},
  howpublished = {\url{https://github.com/shenweichen/deepctr}},
}

DisscussionGroup

公众号：浅梦学习笔记 微信：deepctrbot

Main contributors([welcome to join us!](./CONTRIBUTING.md))

| ​
​ Shen Weichen ​

text
Alibaba Group  ​ | [](https://github.com/zanshuxun)
     [Zan Shuxun](https://github.com/zanshuxun) ​
    Alibaba Group  ​ | ​ [](https://github.com/pandeconscious)
    ​ [Harshit Pande](https://github.com/pandeconscious)
     Amazon   ​ | ​ [](https://github.com/morningsky)
    ​ [Lai Mincai](https://github.com/morningsky)
     ByteDance ​ | ​ [](https://github.com/codewithzichao)
    ​ [Li Zichao](https://github.com/codewithzichao)
     ByteDance   ​ | ​ [](https://github.com/TanTingyi)
     [Tan Tingyi](https://github.com/TanTingyi)
       Chongqing University  of  Posts and  Telecommunications   ​ |

| --- | --- | --- | --- | --- | --- |

Requirements.Readthedocs (`docs/requirements.readthedocs.txt`)

numpy<2
Jinja2<3.1
docutils<0.18
sphinx==4.5.0
sphinx-rtd-theme==0.5.2
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==2.0.0
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.5
recommonmark==0.7.1
tensorflow==2.15.0

Source Estimators (`docs/source/Estimators.rst`)

DeepCTR Estimators API

.. toctree::
CCPM<deepctr.estimator.models.ccpm>
FNN<deepctr.estimator.models.fnn>
PNN<deepctr.estimator.models.pnn>
WDL<deepctr.estimator.models.wdl>
DeepFM<deepctr.estimator.models.deepfm>
NFM<deepctr.estimator.models.nfm>
AFM<deepctr.estimator.models.afm>
DCN<deepctr.estimator.models.dcn>
xDeepFM<deepctr.estimator.models.xdeepfm>
AutoInt<deepctr.estimator.models.autoint>
FiBiNET<deepctr.estimator.models.fibinet>

Source Examples (`docs/source/Examples.md`)

Examples

Classification: Criteo

The Criteo Display Ads dataset is for the purpose of predicting ads click-through rate. It has 13 integer features and
26 categorical features where each category has a high cardinality.

In this example,we simply normailize the dense feature between 0 and 1,you can try other transformation technique like
log normalization or discretization.Then we use SparseFeat
and DenseFeat to generate feature columns for sparse features and dense features.

This example shows how to use DeepFM to solve a simple binary classification task. You can get the demo
data criteo_sample.txt
and run the following codes.

text
/* Detailed source-code truncated for AI context efficiency. */

Classification: Criteo with feature hashing on the fly

This example shows how to use DeepFM to solve a simple binary classification task using feature hashing. You can get
the demo data criteo_sample.txt
and run the following codes.

text
/* Detailed source-code truncated for AI context efficiency. */

Regression: Movielens

The MovieLens data has been used for personalized tag recommendation,which contains 668, 953 tag applications of users
on movies. Here is a small fraction of data include only sparse field.

This example shows how to use DeepFM to solve a simple binary regression task. You can get the demo data
movielens_sample.txt and run the
following codes.

text
/* Detailed source-code truncated for AI context efficiency. */

Multi-value Input : Movielens

The MovieLens data has been used for personalized tag recommendation,which contains 668, 953 tag applications of users
on movies. Here is a small fraction of data include sparse fields and a multivalent field.

There are 2 additional steps to use DeepCTR with sequence feature input.

  1. Generate the paded and encoded sequence feature of sequence input feature(value 0 is for padding).
  2. Generate config of sequence feature with VarLenSparseFeat

For more details about padding, maxlen, embedding sharing, DIN history fields, DSIN session fields, and TFRecord inputs, see the Sequence Feature Cookbook.

This example shows how to use DeepFM with sequence(multi-value) feature. You can get the demo data
movielens_sample.txt and run the
following codes.

text
/* Detailed source-code truncated for AI context efficiency. */

Multi-value Input : Movielens with feature hashing on the fly

text
/* Detailed source-code truncated for AI context efficiency. */

Hash Layer with pre-defined key-value vocabulary

This examples how to use pre-defined key-value vocabulary in Hash Layer.movielens_age_vocabulary.csv stores the
key-value mapping for age feature.

text
/* Detailed source-code truncated for AI context efficiency. */

Estimator with TFRecord: Classification Criteo

This example shows how to use DeepFMEstimator to solve a simple binary classification task. You can get the demo
data criteo_sample.tr.tfrecords
and criteo_sample.te.tfrecords
and run the following codes.

text
/* Detailed source-code truncated for AI context efficiency. */

Estimator with Pandas DataFrame: Classification Criteo

This example shows how to use DeepFMEstimator to solve a simple binary classification task. You can get the demo
data criteo_sample.txt
and run the following codes.

text
/* Detailed source-code truncated for AI context efficiency. */

MultiTask Learning:MMOE

The UCI census-income dataset is extracted from the 1994 census database. It contains 299,285 instances of demographic
information of American adults. There are 40 features in total. We construct a multi-task learning problem from this
dataset by setting some of the features as prediction targets :

  • Task 1: Predict whether the income exceeds $50K;
  • Task 2: Predict whether this person’s marital status is never married.

This example shows how to use MMOE to solve a multi task learning problem. You can get the demo
data census-income.sample and run
the following codes.

text
/* Detailed source-code truncated for AI context efficiency. */

Source FAQ (`docs/source/FAQ.md`)

FAQ

1. Save or load weights/models

To save/load weights,you can write codes just like any other keras models.

python
model = DeepFM()
model.save_weights('DeepFM_w.h5')
model.load_weights('DeepFM_w.h5')

To save/load models,just a little different.

python
from tensorflow.keras.models import save_model, load_model
model = DeepFM()
save_model(model, 'DeepFM.h5')# save_model, same as before

from deepctr.layers import custom_objects
model = load_model('DeepFM.h5',custom_objects)# load_model,just add a parameter

2. Set learning rate and use earlystopping

You can use any models in DeepCTR like a keras model object.
Here is a example of how to set learning rate and earlystopping:

python
import deepctr
from tensorflow.keras.optimizers import Adam, Adagrad
from tensorflow.keras.callbacks import EarlyStopping

model = deepctr.models.DeepFM(linear_feature_columns,dnn_feature_columns)
model.compile(Adagrad(0.1024),'binary_crossentropy',metrics=['binary_crossentropy'])

es = EarlyStopping(monitor='val_binary_crossentropy')
history = model.fit(model_input, data[target].values,batch_size=256, epochs=10, verbose=2, validation_split=0.2,callbacks=[es] )

If you are using Estimator models, you can set learning rate like:

python
from deepctr.estimator import DeepFMEstimator
import tensorflow as tf

model = DeepFMEstimator(linear_feature_columns, dnn_feature_columns, task='binary', 
                            linear_optimizer=tf.train.FtrlOptimizer(0.05), dnn_optimizer=tf.train.AdagradOptimizer(0.1)
                            )

3. Get the attentional weights of feature interactions in AFM

First,make sure that you have install the latest version of deepctr.

Then,use the following code,the attentional_weights[:,i,0] is the feature_interactions[i]'s attentional weight of all samples.

python
import itertools
import deepctr
from deepctr.models import AFM
from deepctr.feature_column import get_feature_names
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Lambda

model = AFM(linear_feature_columns,dnn_feature_columns)
model.fit(model_input,target)

afmlayer = model.layers[-3]
afm_weight_model = Model(model.input,outputs=Lambda(lambda x:afmlayer.normalized_att_score)(model.input))
attentional_weights = afm_weight_model.predict(model_input,batch_size=4096)

feature_names = get_feature_names(dnn_feature_columns)
feature_interactions = list(itertools.combinations(feature_names ,2))

4. How to extract the embedding vectors in deepfm?

python
feature_columns = [SparseFeat('user_id',120,),SparseFeat('item_id',60,),SparseFeat('cate_id',60,)]

def get_embedding_weights(dnn_feature_columns,model):
    embedding_dict = {}
    for fc in dnn_feature_columns:
        if hasattr(fc,'embedding_name'):
            if fc.embedding_name is not None:
                name = fc.embedding_name
            else:
                name = fc.name
            embedding_dict[name] = model.get_layer("sparse_emb_"+name).get_weights()[0]
    return embedding_dict
    
embedding_dict = get_embedding_weights(feature_columns,model)

user_id_emb = embedding_dict['user_id']
item_id_emb = embedding_dict['item_id']

5. How to add a long dense feature vector as a input to the model?

python
from deepctr.models import DeepFM
from deepctr.feature_column import SparseFeat, DenseFeat,get_feature_names
import numpy as np

feature_columns = [SparseFeat('user_id',120,),SparseFeat('item_id',60,),DenseFeat("pic_vec",5)]
fixlen_feature_names = get_feature_names(feature_columns)

user_id = np.array([[1],[0],[1]])
item_id = np.array([[30],[20],[10]])
pic_vec = np.array([[0.1,0.5,0.4,0.3,0.2],[0.1,0.5,0.4,0.3,0.2],[0.1,0.5,0.4,0.3,0.2]])
label = np.array([1,0,1])

model_input = {'user_id':user_id,'item_id':item_id,'pic_vec':pic_vec}

model = DeepFM(feature_columns,feature_columns)
model.compile('adagrad','binary_crossentropy')
model.fit(model_input,label)

6. How to use pretrained weights to initialize embedding weights and frozen embedding weights?

Use tf.initializers.identity() to set the embeddings_initializer of SparseFeat,and set trainable=False to frozen embedding weights.

python
import numpy as np
import tensorflow as tf
from deepctr.models import DeepFM
from deepctr.feature_column import SparseFeat,get_feature_names

pretrained_item_weights = np.random.randn(60,4)
pretrained_weights_initializer = tf.initializers.constant(pretrained_item_weights)

feature_columns = [SparseFeat('user_id',120,),SparseFeat('item_id',60,embedding_dim=4,embeddings_initializer=pretrained_weights_initializer,trainable=False)]
fixlen_feature_names = get_feature_names(feature_columns)

user_id = np.array([[1],[0],[1]])
item_id = np.array([[30],[20],[10]])
label = np.array([1,0,1])

model_input = {'user_id':user_id,'item_id':item_id,}

model = DeepFM(feature_columns,feature_columns)
model.compile('adagrad','binary_crossentropy')
model.fit(model_input,label)

7. How to run the demo with GPU ?

Install the TensorFlow build recommended for your CUDA, cuDNN, and platform combination, then install deepctr.

8. How to avoid TensorFlow, Keras, h5py, or NumPy compatibility errors?

Install TensorFlow separately before installing DeepCTR. Pick the TensorFlow release according to your Python version, CPU/GPU environment, and platform.

bash
$ pip install tensorflow
$ pip install deepctr

For Python >=3.9, DeepCTR uses h5py>=3.7.0, so newer h5py releases such as 3.9+ and 3.12+ are allowed. If TensorFlow reports a NumPy conflict, follow the TensorFlow requirement for the TensorFlow release you installed, for example using numpy<2 when required by TensorFlow.

Use public tensorflow.keras imports in your own code:

python
from tensorflow.keras.models import load_model
from tensorflow.keras.optimizers import Adam

Avoid mixing tensorflow.python.keras with tensorflow.keras. tensorflow.python.* is private TensorFlow API and can break serialization, optimizer loading, or metric loading across TensorFlow versions.

9. How to run the demo with multiple GPUs

you can use multiple gpus with tensorflow version higher than 1.4,see run_classification_criteo_multi_gpu.py

Source Features (`docs/source/Features.md`)

Features

Overview

With the great success of deep learning,DNN-based techniques have been widely used in CTR prediction task.

DNN based CTR prediction models usually have following 4 modules:
Input,Embedding,Low-order&High-order Feature Extractor,Prediction

  • Input&Embedding

The data in CTR estimation task usually includes high sparse,high cardinality categorical features and some dense numerical features.

Since DNN are good at handling dense numerical features,we usually map the sparse categorical features to dense numerical through embedding technique.

For numerical features,we usually apply discretization or normalization on them.

  • Feature Extractor

Low-order Extractor learns feature interaction through product between vectors.Factorization-Machine and it's variants are widely used to learn the low-order feature interaction.

High-order Extractor learns feature combination through complex neural network functions like MLP,Cross Net,etc.

Feature Columns

SparseFeat

SparseFeat is a namedtuple with
signature SparseFeat(name, vocabulary_size, embedding_dim, use_hash, vocabulary_path, dtype, embeddings_initializer, embedding_name, group_name, trainable)

  • name : feature name
  • vocabulary_size : number of unique feature values for sparse feature or hashing space when use_hash=True
  • embedding_dim : embedding dimension
  • use_hash : default False.If True the input will be hashed to space of size vocabulary_size.
  • vocabulary_path : default None. The CSV text file path of the vocabulary table used
    by tf.lookup.TextFileInitializer, which assigns one entry in the table for each line in the file. One entry contains
    two columns separated by comma, the first is the value column, the second is the key column. The 0 value is reserved
    to use if a key is missing in the table, so hash value need start from 1.
  • dtype : default int32.dtype of input tensor.
  • embeddings_initializer : initializer for the embeddings matrix.
  • embedding_name : default None. If None, the embedding_name will be same as name.
  • group_name : feature group of this feature.
  • trainable: default True.Whether or not the embedding is trainable.

DenseFeat

DenseFeat is a namedtuple with signature DenseFeat(name, dimension, dtype, transform_fn)

  • name : feature name
  • dimension : dimension of dense feature vector.
  • dtype : default float32.dtype of input tensor.
  • transform_fn : If not None , a function that can be used to transform values of the feature. the function takes the
    input Tensor as its argument, and returns the output Tensor.
    (e.g. lambda x: (x - 3.0) / 4.2).

VarLenSparseFeat

VarLenSparseFeat is a namedtuple with
signature VarLenSparseFeat(sparsefeat, maxlen, combiner, length_name, weight_name,weight_norm)

  • sparsefeat : a instance of SparseFeat
  • maxlen : maximum length of this feature for all samples
  • combiner : pooling method,can be sum,mean or max
  • length_name : feature length name,if None, value 0 in feature is for padding.
  • weight_name : default None. If not None, the sequence feature will be multiplyed by the feature whose name
    is weight_name.
  • weight_norm : default True. Whether normalize the weight score or not.

Models

CCPM (Convolutional Click Prediction Model)

CCPM can extract local-global key features from an input instance with varied elements, which can be implemented for not
only single ad impression but also sequential ad impression.

CCPM Model API
CCPM Estimator API

Liu Q, Yu F, Wu S, et al. A convolutional click prediction model[C]//Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. ACM, 2015: 1743-1746.

FNN (Factorization-supported Neural Network)

According to the paper,FNN learn embedding vectors of categorical data via pre-trained FM. It use FM's latent vector to
initialiaze the embedding vectors.During the training stage,it concatenates the embedding vectors and feeds them into a
MLP(MultiLayer Perceptron).

FNN Model API
FNN Estimator API

Zhang W, Du T, Wang J. Deep learning over multi-field categorical data[C]//European conference on information retrieval. Springer, Cham, 2016: 45-57.

PNN (Product-based Neural Network)

PNN concatenates sparse feature embeddings and the product between embedding vectors as the input of MLP.

PNN Model API
PNN Estimator API

Qu Y, Cai H, Ren K, et al. Product-based neural networks for user response prediction[C]//Data Mining (ICDM), 2016 IEEE 16th International Conference on. IEEE, 2016: 1149-1154.

Wide & Deep

WDL's deep part concatenates sparse feature embeddings as the input of MLP,the wide part use handcrafted feature as
input. The logits of deep part and wide part are added to get the prediction probability.

WDL Model API
WDL Estimator API

Cheng H T, Koc L, Harmsen J, et al. Wide & deep learning for recommender systems[C]//Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. ACM, 2016: 7-10.

DeepFM

DeepFM can be seen as an improvement of WDL and FNN.Compared with WDL,DeepFM use FM instead of LR in the wide part and
use concatenation of embedding vectors as the input of MLP in the deep part. Compared with FNN,the embedding vector of
FM and input to MLP are same. And they do not need a FM pretrained vector to initialiaze,they are learned end2end.

DeepFM Model API
DeepFM Estimator API

Guo H, Tang R, Ye Y, et al. Deepfm: a factorization-machine based neural network for ctr prediction[J]. arXiv preprint arXiv:1703.04247, 2017.

MLR(Mixed Logistic Regression/Piece-wise Linear Model)

MLR can be viewed as a combination of $2m$ LR model, $m$ is the piece(region) number. $m$ LR model learns the weight
that the sample belong to each region,another m LR model learn sample's click probability in the region. Finally,the
sample's CTR is a weighted sum of each region's click probability.Notice the weight is normalized weight.

MLR Model API

Gai K, Zhu X, Li H, et al. Learning Piece-wise Linear Models from Large Scale Data for Ad Click Prediction[J]. arXiv preprint arXiv:1704.05194, 2017.

NFM (Neural Factorization Machine)

NFM use a bi-interaction pooling layer to learn feature interaction between embedding vectors and compress the result
into a singe vector which has the same size as a single embedding vector. And then fed it into a MLP.The output logit of
MLP and the output logit of linear part are added to get the prediction probability.

NFM Model API
NFM Estimator API

He X, Chua T S. Neural factorization machines for sparse predictive analytics[C]//Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval. ACM, 2017: 355-364.

AFM (Attentional Factorization Machine)

AFM is a variant of FM,tradional FM sums the inner product of embedding vector uniformly. AFM can be seen as weighted
sum of feature interactions.The weight is learned by a small MLP.

AFM Model API
AFM Estimator API

Xiao J, Ye H, He X, et al. Attentional factorization machines: Learning the weight of feature interactions via attention networks[J]. arXiv preprint arXiv:1708.04617, 2017.

DCN (Deep & Cross Network)

DCN use a Cross Net to learn both low and high order feature interaction explicitly,and use a MLP to learn feature
interaction implicitly. The output of Cross Net and MLP are concatenated.The concatenated vector are feed into one fully
connected layer to get the prediction probability.

DCN Model API
DCN Estimator API

Wang R, Fu B, Fu G, et al. Deep & cross network for ad click predictions[C]//Proceedings of the ADKDD'17. ACM, 2017: 12.

DCN-Mix (Improved Deep & Cross Network with mix of experts and matrix kernel)

DCN-Mix uses a matrix kernel instead of vector kernel in CrossNet compared with DCN,and it uses mixture of experts to
learn feature interactions.

DCN-Mix Model API

Wang R, Shivanna R, Cheng D Z, et al. DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems[J]. arXiv preprint arXiv:2008.13535, 2020.

xDeepFM

xDeepFM use a Compressed Interaction Network (CIN) to learn both low and high order feature interaction explicitly,and
use a MLP to learn feature interaction implicitly. In each layer of CIN,first compute outer products between $x^k$ and
$x_0$ to get a tensor $Z_{k+1}$,then use a 1DConv to learn feature maps $H_{k+1}$ on this tensor. Finally,apply sum
pooling on all the feature maps $H_k$ to get one vector.The vector is used to compute the logit that CIN contributes.

xDeepFM Model API
xDeepFM Estimator API

Lian J, Zhou X, Zhang F, et al. xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems[J]. arXiv preprint arXiv:1803.05170, 2018.

AutoInt(Automatic Feature Interaction)

AutoInt use a interacting layer to model the interactions between different features. Within each interacting layer,
each feature is allowed to interact with all the other features and is able to automatically identify relevant features
to form meaningful higher-order features via the multi-head attention mechanism. By stacking multiple interacting
layers,AutoInt is able to model different orders of feature interactions.

AutoInt Model API
AutoInt Estimator API

Song W, Shi C, Xiao Z, et al. Autoint: Automatic feature interaction learning via self-attentive neural networks[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 2019: 1161-1170.

ONN(Operation-aware Neural Networks for User Response Prediction)

ONN models second order feature interactions like like FFM and preserves second-order interaction information as much as
possible.Further more,deep neural network is used to learn higher-ordered feature interactions.

ONN Model API

Yang Y, Xu B, Shen F, et al. Operation-aware Neural Networks for User Response Prediction[J]. arXiv preprint arXiv:1904.12579, 2019.

FGCNN(Feature Generation by Convolutional Neural Network)

FGCNN models with two components: Feature Generation and Deep Classifier. Feature Generation leverages the strength of
CNN to generate local patterns and recombine them to generate new features. Deep Classifier adopts the structure of IPNN
to learn interactions from the augmented feature space.

FGCNN Model API

Liu B, Tang R, Chen Y, et al. Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction[J]. arXiv preprint arXiv:1904.04447, 2019.

FiBiNET(Feature Importance and Bilinear feature Interaction NETwork)

Feature Importance and Bilinear feature Interaction NETwork is proposed to dynamically learn the feature importance and
fine-grained feature interactions. On the one hand, the FiBiNET can dynamically learn the importance of fea- tures via
the Squeeze-Excitation network (SENET) mechanism; on the other hand, it is able to effectively learn the feature
interactions via bilinear function.

FiBiNET Model API
FiBiNET Estimator API

Huang T, Zhang Z, Zhang J. FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Prediction[J]. arXiv preprint arXiv:1905.09433, 2019.

FLEN(Field-Leveraged Embedding Network)

A large-scale CTR prediction model with efficient usage of field information to alleviate gradient coupling problem.

FLEN Model API

FLEN example

Chen W, Zhan L, Ci Y, Lin C. FLEN: Leveraging Field for Scalable CTR Prediction[J]. arXiv preprint arXiv:1911.04690, 2019.

IFM(Input-aware Factorization Machine)

IFM improves FMs by explicitly considering the impact of each individual input upon the representation of features,
which learns a unique input-aware factor for the same feature in different instances via a neural network.

IFM Model API

Yu Y, Wang Z, Yuan B. An Input-aware Factorization Machine for Sparse Prediction[C]//IJCAI. 2019: 1466-1472.

DIFM(Dual Input-aware Factorization Machine)

Dual Input-aware Factorization Machines (DIFMs) can adaptively reweight the original feature representations at the
bit-wise and vector-wise levels simultaneously.
DIFM Model API

Lu W, Yu Y, Chang Y, et al. A Dual Input-aware Factorization Machine for CTR Prediction[C]//IJCAI. 2020: 3139-3145.

DeepFEFM(Deep Field-Embedded Factorization Machine)

FEFM learns symmetric matrix embeddings for each field pair along with the usual single vector embeddings for each
feature. FEFM has significantly lower model complexity than FFM and roughly the same complexity as FwFM.
DeepFEFM Model API

Pande H. Field-Embedded Factorization Machines for Click-through rate prediction[J]. arXiv preprint arXiv:2009.09931, 2020.

EDCN(Enhancing Explicit and Implicit Feature Interactions DCN)

EDCN introduces two advanced modules, namelybridge moduleandregulation module, which work collaboratively tocapture the layer-wise interactive signals and learn discriminativefeature distributions for each hidden layer of the parallel networks.

EDCN Model API

Chen B, Wang Y, Liu Z, et al. Enhancing explicit and implicit feature interactions via information sharing for parallel deep ctr models[C]//Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 2021: 3757-3766.

Sequence Models

DIN (Deep Interest Network)

DIN introduce a attention method to learn from sequence(multi-valued) feature. Tradional method usually use sum/mean
pooling on sequence feature. DIN use a local activation unit to get the activation score between candidate item and
history items. User's interest are represented by weighted sum of user behaviors. user's interest vector and other
embedding vectors are concatenated and fed into a MLP to get the prediction.

DIN Model API

DIN example

Zhou G, Zhu X, Song C, et al. Deep interest network for click-through rate prediction[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 2018: 1059-1068.

DIEN (Deep Interest Evolution Network)

Deep Interest Evolution Network (DIEN) uses interest extractor layer to capture temporal interests from history behavior
sequence. At this layer, an auxiliary loss is proposed to supervise interest extracting at each step. As user interests
are diverse, especially in the e-commerce system, interest evolving layer is proposed to capture interest evolving
process that is relative to the target item. At interest evolving layer, attention mechanism is embedded into the
sequential structure novelly, and the effects of relative interests are strengthened during interest evolution.

DIEN Model API

DIEN example

Zhou G, Mou N, Fan Y, et al. Deep Interest Evolution Network for Click-Through Rate Prediction[J]. arXiv preprint arXiv:1809.03672, 2018.

DSIN(Deep Session Interest Network)

Deep Session Interest Network (DSIN) extracts users' multiple historical sessions in their behavior sequences. First it
uses self-attention mechanism with bias encoding to extract users' interests in each session. Then apply Bi-LSTM to
model how users' interests evolve and interact among sessions. Finally, local activation unit is used to adaptively
learn the influences of various session interests on the target item.

DSIN Model API

DSIN example

Feng Y, Lv F, Shen W, et al. Deep Session Interest Network for Click-Through Rate Prediction[J]. arXiv preprint arXiv:1905.06482, 2019.

BST(Behavior Sequence Transformer)

BST use the powerful Transformer model to capture the sequential signals underlying users’ behavior sequences .

BST Model API

BST example

Qiwei Chen, Huan Zhao, Wei Li, Pipei Huang, and Wenwu Ou. 2019. Behavior sequence transformer for e-commerce recommendation in Alibaba. In Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data (DLP-KDD '19). Association for Computing Machinery, New York, NY, USA, Article 12, 1–4. DOI:)

MultiTask Models

SharedBottom

Hard parameter sharing is the most commonly used approach to MTL in neural networks. It is generally applied by sharing the hidden layers between all tasks, while keeping several task-specific output layers.

SharedBottom Model API

Ruder S. An overview of multi-task learning in deep neural networks[J]. arXiv preprint arXiv:1706.05098, 2017.

ESMM(Entire Space Multi-task Model)

ESMM models CVR in a brand-new perspective by making good use of sequential pattern of user actions, i.e., impression →
click → conversion. The proposed Entire Space Multi-task Model (ESMM) can eliminate the two problems simultaneously by
i) modeling CVR directly over the entire space, ii) employing a feature representation transfer learning strategy.

ESMM Model API

Ma X, Zhao L, Huang G, et al. Entire space multi-task model: An effective approach for estimating post-click conversion rate[C]//The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 2018.

MMOE(Multi-gate Mixture-of-Experts)

Multi-gate Mixture-of-Experts (MMoE) explicitly learns to model task relationships from data. We adapt the Mixture-of-
Experts (MoE) structure to multi-task learning by sharing the expert submodels across all tasks, while also having a
gating network trained to optimize each task.

MMOE Model API

Ma J, Zhao Z, Yi X, et al. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018.

PLE(Progressive Layered Extraction)

PLE separates shared components and task-specific components explicitly and adopts a progressive rout- ing mechanism to
extract and separate deeper semantic knowledge gradually, improving efficiency of joint representation learning and
information routing across tasks in a general setup.

PLE Model API

Tang H, Liu J, Zhao M, et al. Progressive layered extraction (ple): A novel multi-task learning (mtl) model for personalized recommendations[C]//Fourteenth ACM Conference on Recommender Systems. 2020.

Layers

The models of deepctr are modular, so you can use different modules to build your own models.

The module is a class that inherits from tf.keras.layers.Layer,it has the same attributes and methods as keras Layers
like tf.keras.layers.Dense() etc

You can see layers API in Layers

Source History (`docs/source/History.md`)

History

  • 04/16/2026 : v0.9.4 released.Support higher tensorflow version.
  • 11/10/2022 : v0.9.3 released.Add EDCN.
  • 10/15/2022 : v0.9.2 released.Support python 3.9,3.10.
  • 06/11/2022 : v0.9.1 released.Improve compatibility with tensorflow 2.x.
  • 09/03/2021 : v0.9.0 released.Add multitask learning models:SharedBottom,ESMM,MMOE and PLE. running example
  • 07/18/2021 : v0.8.7 released.Support pre-defined key-value vocabulary in Hash Layer. example
  • 06/14/2021 : v0.8.6 released.Add IFM DIFM, FEFM and DeepFEFM model.
  • 03/13/2021 : v0.8.5 released.Add BST model.
  • 02/12/2021 : v0.8.4 released.Fix bug in DCN-Mix.
  • 01/06/2021 : v0.8.3 released.Add DCN-Mix model.Support transform_fn in DenseFeat.
  • 10/11/2020 : v0.8.2 released.Refactor DNN Layer.
  • 09/12/2020 : v0.8.1 released.Improve the reproducibility & fix some bugs.
  • 06/27/2020 : v0.8.0 released.
    • Support Tensorflow Estimator for large scale data and distributed training. example: Estimator with TFRecord
    • Support different initializers for different embedding weights and loading pretrained embeddings. example
    • Add new model FwFM.
  • 05/17/2020 : v0.7.5 released.Fix numerical instability in LayerNormalization.
  • 03/15/2020 : v0.7.4 released.Add FLEN and FieldWiseBiInteraction.
  • 03/04/2020 : v0.7.3 released.Fix the inconsistency of prediction results when the model is loaded with trained weights.
  • 02/08/2020 : v0.7.2 released.Fix some bugs.
  • 01/28/2020 : v0.7.1 released.Simplify VarLenSparseFeat,support setting weight_normalization.Fix problem of embedding size of SparseFeat in linear_feature_columns.
  • 11/24/2019 : v0.7.0 released.Refactor feature columns.Different features can use different embedding_dim and group-wise interaction is available by setting group_name.
  • 11/06/2019 : v0.6.3 released.Add WeightedSequenceLayer and support weighted sequence feature input.
  • 10/03/2019 : v0.6.2 released.Simplify the input logic.
  • 09/08/2019 : v0.6.1 released.Fix bugs in CCPM and DynamicGRU.
  • 08/02/2019 : v0.6.0 released.Now DeepCTR is compatible with tensorflow 1.14 and 2.0.0.
  • 07/21/2019 : v0.5.2 released.Refactor Linear Layer.
  • 07/10/2019 : v0.5.1 released.Add FiBiNET.
  • 06/30/2019 : v0.5.0 released.Refactor inputs module.
  • 05/19/2019 : v0.4.1 released.Add DSIN.
  • 05/04/2019 : v0.4.0 released.Support feature hashing on the fly and python2.7.
  • 04/27/2019 : v0.3.4 released.Add FGCNN and FGCNNLayer.
  • 04/21/2019 : v0.3.3 released.Add CCPM.
  • 03/30/2019 : v0.3.2 released.Add DIEN and ONN Model.
  • 02/17/2019 : v0.3.1 released.Refactor layers ,add BiLSTM and Transformer.
  • 01/24/2019 : v0.2.3 released.Use a new feature config generation method and fix bugs.
  • 01/01/2019 : v0.2.2 released.Add sequence(multi-value) input support for AFM,AutoInt,DCN,DeepFM,FNN,NFM,PNN,xDeepFM models.
  • 12/27/2018 : v0.2.1 released.Add AutoInt Model.
  • 12/22/2018 : v0.2.0 released.Add xDeepFM and automatic check for new version.
  • 12/19/2018 : v0.1.6 released.Now DeepCTR is compatible with tensorflow from 1.4-1.12 except for 1.7 and 1.8.
  • 11/29/2018 : v0.1.4 released.Add FAQ in docs
  • 11/24/2018 : DeepCTR first version v0.1.0 is released on PyPi

Source Layers (`docs/source/Layers.rst`)

DeepCTR Layers API

.. toctree::
:maxdepth: 3
:caption: API:

Core Layers<deepctr.layers.core>
Interaction Layers<deepctr.layers.interaction>
Activation Layers<deepctr.layers.activation>
Normalization Layers<deepctr.layers.normalization>
Sequence Layers<deepctr.layers.sequence>

Source Model Methods (`docs/source/Model_Methods.md`)

Methods

compile

python
compile(optimizer, loss=None, metrics=None, loss_weights=None, sample_weight_mode=None, weighted_metrics=None, target_tensors=None)

Configures the model for training.

Arguments

  • optimizer: String (name of optimizer) or optimizer instance. See optimizers.
  • loss: String (name of objective function) or objective function. See losses. If the model has multiple outputs, you can use a different loss on each output by passing a dictionary or a list of losses. The loss value that will be minimized by the model will then be the sum of all individual losses.
  • metrics: List of metrics to be evaluated by the model during training and testing. Typically you will use metrics=['accuracy']. To specify different metrics for different outputs of a multi-output model, you could also pass a dictionary, such as metrics={'output_a': 'accuracy'}.
  • loss_weights: Optional list or dictionary specifying scalar coefficients (Python floats) to weight the loss contributions of different model outputs. The loss value that will be minimized by the model will then be the weighted sum of all individual losses, weighted by the loss_weights coefficients. If a list, it is expected to have a 1:1 mapping to the model's outputs. If a tensor, it is expected to map output names (strings) to scalar coefficients.
  • sample_weight_mode: If you need to do timestep-wise sample weighting (2D weights), set this to "temporal". None defaults to sample-wise weights (1D). If the model has multiple outputs, you can use a different sample_weight_mode on each output by passing a dictionary or a list of modes.
  • weighted_metrics: List of metrics to be evaluated and weighted by sample_weight or class_weight during training and testing.
  • target_tensors: By default, Keras will create placeholders for the model's target, which will be fed with the target data during training. If instead you would like to use your own target tensors (in turn, Keras will not expect external Numpy data for these targets at training time), you can specify them via the target_tensors argument. It can be a single tensor (for a single-output model), a list of tensors, or a dict mapping output names to target tensors.

Raises

  • ValueError: In case of invalid arguments for optimizer, loss, metrics or sample_weight_mode.

fit

python
fit(x=None, y=None, batch_size=None, epochs=1, verbose=1, callbacks=None, validation_split=0.0, validation_data=None, shuffle=True, class_weight=None, sample_weight=None, initial_epoch=0, steps_per_epoch=None, validation_steps=None, validation_freq=1)

Trains the model for a given number of epochs (iterations on a dataset).

Arguments

  • x: Numpy array of training data (if the model has a single input), or list of Numpy arrays (if the model has multiple inputs). If input layers in the model are named, you can also pass a dictionary mapping input names to Numpy arrays. x can be None (default) if feeding from framework-native tensors (e.g. TensorFlow data tensors).
  • y: Numpy array of target (label) data (if the model has a single output), or list of Numpy arrays (if the model has multiple outputs). If output layers in the model are named, you can also pass a dictionary mapping output names to Numpy arrays. y can be None (default) if feeding from framework-native tensors (e.g. TensorFlow data tensors).
  • batch_size: Integer or None. Number of samples per gradient update. If unspecified, batch_size will default to 32.
  • epochs: Integer. Number of epochs to train the model. An epoch is an iteration over the entire x and y data provided. Note that in conjunction with initial_epoch, epochs is to be understood as "final epoch". The model is not trained for a number of iterations given by epochs, but merely until the epoch of index epochs is reached.
  • verbose: Integer. 0, 1, or 2. Verbosity mode. 0 = silent, 1 = progress bar, 2 = one line per epoch.
  • callbacks: List of tf.keras.callbacks.Callback instances. List of callbacks to apply during training and validation (if ). See callbacks.
  • validation_split: Float between 0 and 1. Fraction of the training data to be used as validation data. The model will set apart this fraction of the training data, will not train on it, and will evaluate the loss and any model metrics on this data at the end of each epoch. The validation data is selected from the last samples in the x and y data provided, before shuffling.
  • validation_data: tuple (x_val, y_val) or tuple (x_val, y_val, val_sample_weights) on which to evaluate the loss and any model metrics at the end of each epoch. The model will not be trained on this data. validation_data will override validation_split.
  • shuffle: Boolean (whether to shuffle the training data before each epoch) or str (for 'batch'). 'batch' is a special option for dealing with the limitations of HDF5 data; it shuffles in batch-sized chunks. Has no effect when steps_per_epoch is not None.
  • class_weight: Optional dictionary mapping class indices (integers) to a weight (float) value, used for weighting the loss function (during training only). This can be useful to tell the model to "pay more attention" to samples from an under-represented class.
  • sample_weight: Optional Numpy array of weights for the training samples, used for weighting the loss function (during training only). You can either pass a flat (1D) Numpy array with the same length as the input samples (1:1 mapping between weights and samples), or in the case of temporal data, you can pass a 2D array with shape (samples, sequence_length), to apply a different weight to every timestep of every sample. In this case you should make sure to specify sample_weight_mode="temporal" in compile().
  • initial_epoch: Integer. Epoch at which to start training (useful for resuming a previous training run).
  • steps_per_epoch: Integer or None. Total number of steps (batches of samples) before declaring one epoch finished and starting the next epoch. When training with input tensors such as TensorFlow data tensors, the default None is equal to the number of samples in your dataset divided by the batch size, or 1 if that cannot be determined.
    validation_steps: Only relevant if steps_per_epoch is specified. Total number of steps (batches of samples) to validate before stopping.
  • validation_freq: Only relevant if validation data is provided. Integer or list/tuple/set. If an integer, specifies how many training epochs to run before a new validation run is performed, e.g. validation_freq=2 runs validation every 2 epochs. If a list, tuple, or set, specifies the epochs on which to run validation, e.g. validation_freq=[1, 2, 10] runs validation at the end of the 1st, 2nd, and 10th epochs.

Returns

  • A History object. Its History.history attribute is a record of training loss values and metrics values at successive epochs, as well as validation loss values and validation metrics values (if applicable).

Raises

  • RuntimeError: If the model was never compiled.
    ValueError: In case of mismatch between the provided input data and what the model expects.

evaluate

python
evaluate(x=None, y=None, batch_size=None, verbose=1, sample_weight=None, steps=None, callbacks=None)

Returns the loss value & metrics values for the model in test mode.
Computation is done in batches.

Arguments

  • x: Numpy array of test data (if the model has a single input), or list of Numpy arrays (if the model has multiple inputs). If input layers in the model are named, you can also pass a dictionary mapping input names to Numpy arrays. x can be None (default) if feeding from framework-native tensors (e.g. TensorFlow data tensors).
  • y: Numpy array of target (label) data (if the model has a single output), or list of Numpy arrays (if the model has multiple outputs). If output layers in the model are named, you can also pass a dictionary mapping output names to Numpy arrays. y can be None (default) if feeding from framework-native tensors (e.g. TensorFlow data tensors).
  • batch_size: Integer or None. Number of samples per evaluation step. If unspecified, batch_size will default to 32.
  • verbose: 0 or 1. Verbosity mode. 0 = silent, 1 = progress bar.
  • sample_weight: Optional Numpy array of weights for the test samples, used for weighting the loss function. You can either pass a flat (1D) Numpy array with the same length as the input samples (1:1 mapping between weights and samples), or in the case of temporal data, you can pass a 2D array with shape (samples, sequence_length), to apply a different weight to every timestep of every sample. In this case you should make sure to specify sample_weight_mode="temporal" in compile().
  • steps: Integer or None. Total number of steps (batches of samples) before declaring the evaluation round finished. Ignored with the default value of None.
  • callbacks: List of tf.keras.callbacks.Callback instances. List of callbacks to apply during evaluation. See callbacks.

Returns

  • Scalar test loss (if the model has a single output and no metrics) or list of scalars (if the model has multiple outputs and/or metrics). The attribute model.metrics_names will give you the display labels for the scalar outputs.

predict

python
predict(x, batch_size=None, verbose=0, steps=None, callbacks=None)

Generates output predictions for the input samples.

Computation is done in batches.

Arguments

  • x: The input data, as a Numpy array (or list of Numpy arrays if the model has multiple inputs).
    batch_size: Integer. If unspecified, it will default to 32.
  • verbose: Verbosity mode, 0 or 1.
  • steps: Total number of steps (batches of samples) before declaring the prediction round finished. Ignored with the default value of None.
  • callbacks: List of tf.keras.callbacks.Callback instances. List of callbacks to apply during prediction. See callbacks.

Returns

  • Numpy array(s) of predictions.

Raises

  • ValueError: In case of mismatch between the provided input data and the model's expectations, or in case a stateful model receives a number of samples that is not a multiple of the batch size.

train_on_batch

python
train_on_batch(x, y, sample_weight=None, class_weight=None)

Runs a single gradient update on a single batch of data.

Arguments

  • x: Numpy array of training data, or list of Numpy arrays if the model has multiple inputs. If all inputs in the model are named, you can also pass a dictionary mapping input names to Numpy arrays.
  • y: Numpy array of target data, or list of Numpy arrays if the model has multiple outputs. If all outputs in the model are named, you can also pass a dictionary mapping output names to Numpy arrays.
  • sample_weight: Optional array of the same length as x, containing weights to apply to the model's loss for each sample. In the case of temporal data, you can pass a 2D array with shape (samples, sequence_length), to apply a different weight to every timestep of every sample. In this case you should make sure to specify sample_weight_mode="temporal" in compile().
  • class_weight: Optional dictionary mapping class indices (integers) to a weight (float) to apply to the model's loss for the samples from this class during training. This can be useful to tell the model to "pay more attention" to samples from an under-represented class.

Returns

  • Scalar training loss (if the model has a single output and no metrics) or list of scalars (if the model has multiple outputs and/or metrics). The attribute model.metrics_names will give you the display labels for the scalar outputs.

test_on_batch

python
test_on_batch(x, y, sample_weight=None)

Test the model on a single batch of samples.

Arguments

  • x: Numpy array of test data, or list of Numpy arrays if the model has multiple inputs. If all inputs in the model are named, you can also pass a dictionary mapping input names to Numpy arrays.
  • y: Numpy array of target data, or list of Numpy arrays if the model has multiple outputs. If all outputs in the model are named, you can also pass a dictionary mapping output names to Numpy arrays.
  • sample_weight: Optional array of the same length as x, containing weights to apply to the model's loss for each sample. In the case of temporal data, you can pass a 2D array with shape (samples, sequence_length), to apply a different weight to every timestep of every sample. In this case you should make sure to specify sample_weight_mode="temporal" in compile().

Returns

  • Scalar test loss (if the model has a single output and no metrics) or list of scalars (if the model has multiple outputs and/or metrics). The attribute model.metrics_names will give you the display labels for the scalar outputs.

predict_on_batch

python
predict_on_batch(x)

Returns predictions for a single batch of samples.

Arguments

  • x: Input samples, as a Numpy array.

Returns

  • Numpy array(s) of predictions.

fit_generator

python
fit_generator(generator, steps_per_epoch=None, epochs=1, verbose=1, callbacks=None, validation_data=None, validation_steps=None, validation_freq=1, class_weight=None, max_queue_size=10, workers=1, use_multiprocessing=False, shuffle=True, initial_epoch=0)

Trains the model on data generated batch-by-batch by a Python generator (or an instance of Sequence).
The generator is run in parallel to the model, for efficiency. For instance, this allows you to do real-time data augmentation on images on CPU in parallel to training your model on GPU.
The use of tf.keras.utils.Sequence guarantees the ordering and guarantees the single use of every input per epoch when using use_multiprocessing=True.

Arguments

  • generator: A generator or an instance of Sequence (tf.keras.utils.Sequence) object in order to avoid duplicate data when using multiprocessing. The output of the generator must be either

    a tuple (inputs, targets) or
    a tuple (inputs, targets, sample_weights).
    This tuple (a single output of the generator) makes a single batch. Therefore, all arrays in this tuple must have the same length (equal to the size of this batch). Different batches may have different sizes. For example, the last batch of the epoch is commonly smaller than the others, if the size of the dataset is not divisible by the batch size. The generator is expected to loop over its data indefinitely. An epoch finishes when steps_per_epoch batches have been seen by the model.

  • steps_per_epoch: Integer. Total number of steps (batches of samples) to yield from generator before declaring one epoch finished and starting the next epoch. It should typically be equal to ceil(num_samples / batch_size) Optional for Sequence: if unspecified, will use the len(generator) as a number of steps.

  • epochs: Integer. Number of epochs to train the model. An epoch is an iteration over the entire data provided, as defined by steps_per_epoch. Note that in conjunction with initial_epoch, epochs is to be understood as "final epoch". The model is not trained for a number of iterations given by epochs, but merely until the epoch of index epochs is reached.

  • verbose: Integer. 0, 1, or 2. Verbosity mode. 0 = silent, 1 = progress bar, 2 = one line per epoch.

  • callbacks: List of tf.keras.callbacks.Callback instances. List of callbacks to apply during training. See callbacks.

  • validation_data: This can be either
    a generator or a Sequence object for the validation data
    tuple (x_val, y_val)
    tuple (x_val, y_val, val_sample_weights)
    on which to evaluate the loss and any model metrics at the end of each epoch. The model will not be trained on this data.

  • validation_steps: Only relevant if validation_data is a generator. Total number of steps (batches of samples) to yield from validation_data generator before stopping at the end of every epoch. It should typically be equal to the number of samples of your validation dataset divided by the batch size. Optional for Sequence: if unspecified, will use the len(validation_data) as a number of steps.

  • validation_freq: Only relevant if validation data is provided. Integer or collections.Container instance (e.g. list, tuple, etc.). If an integer, specifies how many training epochs to run before a new validation run is performed, e.g. validation_freq=2 runs validation every 2 epochs. If a Container, specifies the epochs on which to run validation, e.g. validation_freq=[1, 2, 10] runs validation at the end of the 1st, 2nd, and 10th epochs.

  • class_weight: Optional dictionary mapping class indices (integers) to a weight (float) value, used for weighting the loss function (during training only). This can be useful to tell the model to "pay more attention" to samples from an under-represented class.

  • max_queue_size: Integer. Maximum size for the generator queue. If unspecified, max_queue_size will default to 10.

  • workers: Integer. Maximum number of processes to spin up when using process-based threading. If unspecified, workers will default to 1. If 0, will execute the generator on the main thread.

  • use_multiprocessing: Boolean. If True, use process-based threading. If unspecified, use_multiprocessing will default to False. Note that because this implementation relies on multiprocessing, you should not pass non-picklable arguments to the generator as they can't be passed easily to children processes.

  • shuffle: Boolean. Whether to shuffle the order of the batches at the beginning of each epoch. Only used with instances of Sequence (tf.keras.utils.Sequence). Has no effect when steps_per_epoch is not None.
    initial_epoch: Integer. Epoch at which to start training (useful for resuming a previous training run).

Returns

  • A History object. Its History.history attribute is a record of training loss values and metrics values at successive epochs, as well as validation loss values and validation metrics values (if applicable).

Raises

  • ValueError: In case the generator yields data in an invalid format.

Example

python
def generate_arrays_from_file(path):
    while True:
        with open(path) as f:
            for line in f:
                # create numpy arrays of input data
                # and labels, from each line in the file
                x1, x2, y = process_line(line)
                yield ({'input_1': x1, 'input_2': x2}, {'output': y})

model.fit_generator(generate_arrays_from_file('/my_file.txt'),
                    steps_per_epoch=10000, epochs=10)

evaluate_generator

python
evaluate_generator(generator, steps=None, callbacks=None, max_queue_size=10, workers=1, use_multiprocessing=False, verbose=0)

Evaluates the model on a data generator.
The generator should return the same kind of data as accepted by test_on_batch.

Arguments

  • generator: Generator yielding tuples (inputs, targets) or (inputs, targets, sample_weights) or an instance of Sequence (tf.keras.utils.Sequence) object in order to avoid duplicate data when using multiprocessing.
  • steps: Total number of steps (batches of samples) to yield from generator before stopping. Optional for Sequence: if unspecified, will use the len(generator) as a number of steps.
  • callbacks: List of tf.keras.callbacks.Callback instances. List of callbacks to apply during training. See callbacks.
  • max_queue_size: maximum size for the generator queue
  • workers: Integer. Maximum number of processes to spin up when using process based threading. If unspecified, workers will default to 1. If 0, will execute the generator on the main thread.
  • use_multiprocessing: if True, use process based threading. Note that because this implementation relies on multiprocessing, you should not pass non picklable arguments to the generator as they can't be passed easily to children processes.
  • verbose: verbosity mode, 0 or 1.

Returns

  • Scalar test loss (if the model has a single output and no metrics) or list of scalars (if the model has multiple outputs and/or metrics). The attribute model.metrics_names will give you the display labels for the scalar outputs.

Raises

  • ValueError: In case the generator yields data in an invalid format.

predict_generator

python
predict_generator(generator, steps=None, callbacks=None, max_queue_size=10, workers=1, use_multiprocessing=False, verbose=0)

Generates predictions for the input samples from a data generator.
The generator should return the same kind of data as accepted by predict_on_batch.

Arguments

  • generator: Generator yielding batches of input samples or an instance of Sequence (tf.keras.utils.Sequence) object in order to avoid duplicate data when using multiprocessing.
  • steps: Total number of steps (batches of samples) to yield from generator before stopping. Optional for Sequence: if unspecified, will use the len(generator) as a number of steps.
  • callbacks: List of tf.keras.callbacks.Callback instances. List of callbacks to apply during training. See callbacks.
  • max_queue_size: Maximum size for the generator queue.
  • workers: Integer. Maximum number of processes to spin up when using process based threading. If unspecified, workers will default to 1. If 0, will execute the generator on the main thread.
  • use_multiprocessing: If True, use process based threading. Note that because this implementation relies on multiprocessing, you should not pass non picklable arguments to the generator as they can't be passed easily to children processes.
  • verbose: verbosity mode, 0 or 1.

Returns

  • Numpy array(s) of predictions.

Raises

  • ValueError: In case the generator yields data in an invalid format.

get_layer

python
get_layer(name=None, index=None)

Retrieves a layer based on either its name (unique) or index.
If name and index are both provided, index will take precedence.
Indices are based on order of horizontal graph traversal (bottom-up).

Arguments

  • name: String, name of layer.
  • index: Integer, index of layer.

Returns

  • A layer instance.

Raises

  • ValueError: In case of invalid layer name or index.

Source Models (`docs/source/Models.rst`)

DeepCTR Models API

.. toctree::
Model Methods<Model_Methods.md>
CCPM<deepctr.models.ccpm>
FNN<deepctr.models.fnn>
PNN<deepctr.models.pnn>
WDL<deepctr.models.wdl>
DeepFM<deepctr.models.deepfm>
MLR<deepctr.models.mlr>
NFM<deepctr.models.nfm>
AFM<deepctr.models.afm>
DCN<deepctr.models.dcn>
DCNMix<deepctr.models.dcnmix>
DIN<deepctr.models.sequence.din>
DIEN<deepctr.models.sequence.dien>
DSIN<deepctr.models.sequence.dsin>
BST<deepctr.models.sequence.bst>
xDeepFM<deepctr.models.xdeepfm>
AutoInt<deepctr.models.autoint>
ONN<deepctr.models.onn>
FGCNN<deepctr.models.fgcnn>
FiBiNET<deepctr.models.fibinet>
FLEN<deepctr.models.flen>
IFM<deepctr.models.ifm>
DIFM<deepctr.models.difm>
DeepFEFM<deepctr.models.deepfefm>
SharedBottom<deepctr.models.multitask.sharedbottom>
ESMM<deepctr.models.multitask.esmm>
MMOE<deepctr.models.multitask.mmoe>
PLE<deepctr.models.multitask.ple>
EDCN<deepctr.models.edcn>

Source Quick Start (`docs/source/Quick-Start.md`)

Quick-Start

Installation Guide

Now deepctr supports Python >=3.7 and is tested with TensorFlow 1.15 and TensorFlow 2.x.

DeepCTR does not pin or install TensorFlow for you. Install a TensorFlow build that matches your Python, NumPy, CPU/GPU, and operating system first, then install DeepCTR:

bash
$ pip install tensorflow
$ pip install deepctr

For GPU environments, install the TensorFlow package recommended for your CUDA, cuDNN, and platform combination, then install deepctr.

For Python >=3.9, DeepCTR allows modern h5py releases with h5py>=3.7.0. If TensorFlow reports a NumPy conflict, follow the TensorFlow requirement for your selected TensorFlow release, for example using numpy<2 when required by TensorFlow.

Use public tensorflow.keras APIs in your own code. Avoid mixing tensorflow.python.keras with tensorflow.keras, because tensorflow.python.* is private TensorFlow API and can break model serialization or optimizer/metric loading across TensorFlow versions.

Install from source

bash
$ git clone https://github.com/shenweichen/DeepCTR.git
$ cd DeepCTR
$ pip install .

Getting started: 4 steps to DeepCTR

Step 1: Import model

python
import pandas as pd
from sklearn.preprocessing import LabelEncoder, MinMaxScaler
from sklearn.model_selection import train_test_split
from deepctr.models import DeepFM
from deepctr.feature_column import SparseFeat, DenseFeat,get_feature_names

data = pd.read_csv('./criteo_sample.txt')

sparse_features = ['C' + str(i) for i in range(1, 27)]
dense_features = ['I'+str(i) for i in range(1, 14)]

data[sparse_features] = data[sparse_features].fillna('-1', )
data[dense_features] = data[dense_features].fillna(0,)
target = ['label']

Step 2: Simple preprocessing

Usually we have two methods to encode the sparse categorical feature for embedding

  • Label Encoding: map the features to integer value from 0 ~ len(#unique) - 1
    python
    for feat in sparse_features:
    lbe = LabelEncoder()
    data[feat] = lbe.fit_transform(data[feat])
  • Hash Encoding: map the features to a fix range,like 0 ~ 9999.We have 2 methods to do that:

    • Do feature hashing before training

      python
      for feat in sparse_features:
    lbe = HashEncoder()
    data[feat] = lbe.transform(data[feat])

    • Do feature hashing on the fly in training process

      We can do feature hashing by setting use_hash=True in SparseFeat or VarlenSparseFeat in Step3.

And for dense numerical features,they are usually discretized to buckets,here we use normalization.

python
mms = MinMaxScaler(feature_range=(0,1))
data[dense_features] = mms.fit_transform(data[dense_features])

Step 3: Generate feature columns

For sparse features, we transform them into dense vectors by embedding techniques.
For dense numerical features, we concatenate them to the input tensors of fully connected layer.

And for varlen(multi-valued) sparse features,you can use VarlenSparseFeat. Visit examples of using VarlenSparseFeat and the Sequence Feature Cookbook for sequence model input conventions.

  • Label Encoding
python
fixlen_feature_columns = [SparseFeat(feat, vocabulary_size=data[feat].max() + 1,embedding_dim=4)
                       for i,feat in enumerate(sparse_features)] + [DenseFeat(feat, 1,)
                      for feat in dense_features]
  • Feature Hashing on the fly
python
fixlen_feature_columns = [SparseFeat(feat, vocabulary_size=1e6,embedding_dim=4, use_hash=True, dtype='string')  # the input is string
                              for feat in sparse_features] + [DenseFeat(feat, 1, )
                          for feat in dense_features]
  • generate feature columns
python
dnn_feature_columns = fixlen_feature_columns
linear_feature_columns = fixlen_feature_columns

feature_names = get_feature_names(linear_feature_columns + dnn_feature_columns)

Step 4: Generate the training samples and train the model

python
train, test = train_test_split(data, test_size=0.2)

train_model_input = {name:train[name].values for name in feature_names}
test_model_input = {name:test[name].values for name in feature_names}


model = DeepFM(linear_feature_columns,dnn_feature_columns,task='binary')
model.compile("adam", "binary_crossentropy",
              metrics=['binary_crossentropy'], )

history = model.fit(train_model_input, train[target].values,
                    batch_size=256, epochs=10, verbose=2, validation_split=0.2, )
pred_ans = model.predict(test_model_input, batch_size=256)

You can check the full code here.

You also can run a distributed training job with the keras model on Kubernetes using ElasticDL.

Getting started: 4 steps to DeepCTR Estimator with TFRecord

Step 1: Import model

python
import tensorflow as tf

from deepctr.estimator.inputs import input_fn_tfrecord
from deepctr.estimator.models import DeepFMEstimator

Step 2: Generate feature columns for linear part and dnn part

python
sparse_features = ['C' + str(i) for i in range(1, 27)]
dense_features = ['I' + str(i) for i in range(1, 14)]

dnn_feature_columns = []
linear_feature_columns = []

for i, feat in enumerate(sparse_features):
    dnn_feature_columns.append(tf.feature_column.embedding_column(
        tf.feature_column.categorical_column_with_identity(feat, 1000), 4))
    linear_feature_columns.append(tf.feature_column.categorical_column_with_identity(feat, 1000))
for feat in dense_features:
    dnn_feature_columns.append(tf.feature_column.numeric_column(feat))
    linear_feature_columns.append(tf.feature_column.numeric_column(feat))

Step 3: Generate the training samples with TFRecord format

python
feature_description = {k: tf.io.FixedLenFeature(dtype=tf.int64, shape=1) for k in sparse_features}
feature_description.update(
    {k: tf.io.FixedLenFeature(dtype=tf.float32, shape=1) for k in dense_features})
feature_description['label'] = tf.io.FixedLenFeature(dtype=tf.float32, shape=1)

train_model_input = input_fn_tfrecord('./criteo_sample.tr.tfrecords', feature_description, 'label', batch_size=256,
                                      num_epochs=1, shuffle_factor=10)
test_model_input = input_fn_tfrecord('./criteo_sample.te.tfrecords', feature_description, 'label',
                                     batch_size=2 ** 14, num_epochs=1, shuffle_factor=0)

Step 4: Train and evaluate the model

python
model = DeepFMEstimator(linear_feature_columns, dnn_feature_columns, task='binary')

model.train(train_model_input)
eval_result = model.evaluate(test_model_input)

print(eval_result)

You can check the full code here.

You also can run a distributed training job with the estimator model on Kubernetes using ElasticDL.

Source Sequence Cookbook (`docs/source/Sequence-Cookbook.md`)

Sequence Feature Cookbook

This cookbook collects the input conventions for multi-value features and sequence models such as DIN, BST, DIEN, and DSIN.

One Row Means One Prediction Target

DeepCTR models expect each training sample to be one row. The row contains:

  • the candidate feature to score, such as item_id or cate_id
  • user/context features, such as user, gender, or pay_score
  • optional history or session features that describe the user's past behavior
  • one label for this candidate row

For example, if the same user has two candidate interactions, build two rows. The history fields can be repeated or updated per row.

text
user  item_id  cate_id  hist_item_id      hist_cate_id      label
0     1        1        [1, 2, 3, 0]      [1, 2, 2, 0]      1
0     2        2        [1, 2, 3, 1]      [1, 2, 2, 1]      0

History fields are context for the row. The label belongs to the candidate row, not to every element inside the history.

Feature Column Shapes

Use SparseFeat for one categorical id per row.

python
SparseFeat("item_id", vocabulary_size=item_count + 1, embedding_dim=8)

Its input shape is usually (batch_size,) or (batch_size, 1).

Use DenseFeat for numerical values or dense vectors.

python
DenseFeat("pay_score", 1)
DenseFeat("article_vector", 128)

Use VarLenSparseFeat for a list of categorical ids per row.

python
VarLenSparseFeat(
    SparseFeat("genres", vocabulary_size=genre_count + 1, embedding_dim=4),
    maxlen=max_genre_len,
    combiner="mean",
)

Its input shape is (batch_size, maxlen). Values must be padded to the same maxlen before being passed to the model.

Padding and Length

For VarLenSparseFeat, value 0 is the default padding value. Do not use 0 as a valid category id when length_name is not set.

python
genres = np.array([
    [1, 3, 0, 0],
    [2, 5, 8, 0],
])

If you pass length_name, DeepCTR uses that length feature to build the pooling mask.

python
VarLenSparseFeat(
    SparseFeat("hist_item_id", vocabulary_size=item_count + 1, embedding_dim=8),
    maxlen=4,
    length_name="seq_length",
)

Then add seq_length to the model input:

python
model_input = {
    "hist_item_id": hist_item_id,
    "seq_length": np.array([3, 2, 4]),
}

Padding is still useful because tensors in one batch need the same shape.

Multi-Value Feature or Multi-Hot Vector

VarLenSparseFeat stores categorical ids and then applies embedding lookup plus pooling. maxlen is the maximum number of values in one sample. It is not the vocabulary size.

For a genres feature with 18 possible genres and at most 5 genres per movie:

python
VarLenSparseFeat(
    SparseFeat("genres", vocabulary_size=18 + 1, embedding_dim=4),
    maxlen=5,
    combiner="mean",
)

A multi-hot vector is also possible, but then it is a dense vector with length equal to the vocabulary size:

python
DenseFeat("genres_multihot", 18)

The two representations are different. VarLenSparseFeat learns an embedding for every genre id and pools only the present genres. A multi-hot vector feeds the raw indicator vector into the dense part of the model. For large vocabularies, VarLenSparseFeat is usually more memory efficient and easier to share with other categorical fields.

Multiple VarLenSparseFeat Fields

Add multiple variable-length fields to the same feature column list. Each field can have its own vocabulary and maxlen.

python
feature_columns = [
    SparseFeat("user", user_count + 1, embedding_dim=8),
    SparseFeat("item_id", item_count + 1, embedding_dim=8),
    VarLenSparseFeat(
        SparseFeat("genres", genre_count + 1, embedding_dim=4),
        maxlen=max_genre_len,
        combiner="mean",
    ),
    VarLenSparseFeat(
        SparseFeat("tags", tag_count + 1, embedding_dim=4),
        maxlen=max_tag_len,
        combiner="mean",
    ),
]

The model input must include one padded array for each field:

python
model_input = {
    "user": user,
    "item_id": item_id,
    "genres": genres,
    "tags": tags,
}

Sharing Embeddings

Use the same embedding_name when two features represent ids from the same dictionary and should share one embedding table.

python
feature_columns = [
    SparseFeat("item_id", item_count + 1, embedding_dim=8),
    VarLenSparseFeat(
        SparseFeat(
            "hist_item_id",
            item_count + 1,
            embedding_dim=8,
            embedding_name="item_id",
        ),
        maxlen=4,
    ),
]

Use different embedding_name values when the fields are semantically different, even if they have the same value range.

DIN and BST History Feature Names

DIN and BST use history_feature_list to find the history sequence that should be matched with the current candidate feature.

python
behavior_feature_list = ["item_id", "cate_id"]

For every name in behavior_feature_list, the history sequence feature must be named with the hist_ prefix:

python
SparseFeat("item_id", item_count + 1, embedding_dim=8)
SparseFeat("cate_id", cate_count + 1, embedding_dim=4)

VarLenSparseFeat(
    SparseFeat("hist_item_id", item_count + 1, embedding_dim=8, embedding_name="item_id"),
    maxlen=4,
    length_name="seq_length",
)
VarLenSparseFeat(
    SparseFeat("hist_cate_id", cate_count + 1, embedding_dim=4, embedding_name="cate_id"),
    maxlen=4,
    length_name="seq_length",
)

If behavior_feature_list = ["item"], the expected history feature name is hist_item. If behavior_feature_list = ["item_id"], the expected history feature name is hist_item_id.

Inside DIN:

  • varlen_sparse_feature_columns means all VarLenSparseFeat fields passed to the model.
  • history_feature_columns means the VarLenSparseFeat fields whose names match hist_ + a behavior feature name. These fields are used as keys in attention.
  • sparse_varlen_feature_columns means other VarLenSparseFeat fields. They are pooled and appended to the DNN input, but they are not used as the DIN attention history.

For example, genres is a non-history multi-value field:

python
VarLenSparseFeat(
    SparseFeat("genres", genre_count + 1, embedding_dim=4),
    maxlen=max_genre_len,
    combiner="mean",
)

It belongs to sparse_varlen_feature_columns, not to history_feature_columns.

DIN Does Not Support VarLen of VarLen

DIN expects the candidate behavior feature to be one sparse id per row, and the history behavior feature to be one padded sequence per row.

Supported:

text
item_id:       3
hist_item_id:  [1, 2, 4, 0]

Not supported directly:

text
item_categories:       [1, 2, 0]
hist_item_categories:  [[1, 2, 0], [3, 4, 0], [5, 0, 0]]

The second case is a "sequence of multi-value behaviors", or a 3D tensor. DeepCTR's current DIN implementation does not provide a VarLenSparseFeat nested inside another VarLenSparseFeat.

Common workarounds are:

  • choose one representative category for the candidate and each history item
  • map a category set to one categorical id before training
  • pre-pool the category set outside the model and build a custom DIN variant
  • customize the model to handle a 3D behavior tensor

Dense History Features in DIN

Current DIN attention is built from sparse embedding sequences. Dense features such as pay_score are appended to the DNN input, but they are not part of the attention keys by default.

If you need dense history values inside attention, common choices are:

  • discretize the dense value into buckets and use it as a sparse sequence feature
  • concatenate or combine dense history representations before passing them to a custom attention layer
  • build a custom model based on deepctr.models.sequence.din.DIN

DeepCTR does not provide a VarLenDenseFeat class.

DSIN Input Format

DeepCTR's DSIN implementation expects sessions to be prepared before training. It does not split a raw event stream into sessions inside the model.

For behavior_feature_list = ["item", "cate_id"] and sess_max_count=2, prepare fields like:

python
feature_columns = [
    SparseFeat("item", item_count + 1, embedding_dim=4),
    SparseFeat("cate_id", cate_count + 1, embedding_dim=4),
    VarLenSparseFeat(
        SparseFeat("sess_0_item", item_count + 1, embedding_dim=4, embedding_name="item"),
        maxlen=4,
    ),
    VarLenSparseFeat(
        SparseFeat("sess_0_cate_id", cate_count + 1, embedding_dim=4, embedding_name="cate_id"),
        maxlen=4,
    ),
    VarLenSparseFeat(
        SparseFeat("sess_1_item", item_count + 1, embedding_dim=4, embedding_name="item"),
        maxlen=4,
    ),
    VarLenSparseFeat(
        SparseFeat("sess_1_cate_id", cate_count + 1, embedding_dim=4, embedding_name="cate_id"),
        maxlen=4,
    ),
]

The model input should contain one padded sequence per session field and sess_length, the number of valid sessions for each row:

python
model_input = {
    "item": item,
    "cate_id": cate_id,
    "sess_0_item": sess_0_item,
    "sess_0_cate_id": sess_0_cate_id,
    "sess_1_item": sess_1_item,
    "sess_1_cate_id": sess_1_cate_id,
    "sess_length": np.array([2, 1, 0]),
}

Usually you keep only the most recent sess_max_count sessions and the most recent maxlen events in each session, then pad the rest with 0.

DSIN Labels and Sessions

Each row still has one label for the candidate item in that row.

text
user  item  cate_id  sess_0_item     sess_1_item     sess_length  label
0     1     1        [4, 5, 0, 0]    [2, 3, 0, 0]    2            1
0     2     2        [4, 5, 1, 0]    [2, 3, 0, 0]    2            0

The same user can appear in multiple rows. Sessions are historical context for the candidate row. They do not have their own labels inside the DSIN input.

If your raw data contains timestamps or time deltas, use them during preprocessing to split events into sessions. If you want time gaps as model features, bucket them into categorical ids or build a custom model. DeepCTR does not provide VarLenDenseFeat for raw variable-length dense time values.

Estimator and TFRecord Inputs

The Keras-style DeepCTR models use SparseFeat, DenseFeat, and VarLenSparseFeat. The Estimator models use TensorFlow tf.feature_column objects directly.

For TFRecord vector features, make the dtype and feature column type match:

  • categorical id fields should be integer features and use categorical_column_with_identity, categorical_column_with_hash_bucket, or another categorical column
  • dense float vectors should use numeric_column(..., shape=(dim,)), not embedding_column

Example:

python
feature_description = {
    "article_id": tf.io.FixedLenFeature(shape=(1,), dtype=tf.int64),
    "article_vector": tf.io.FixedLenFeature(shape=(128,), dtype=tf.float32),
    "clicked": tf.io.FixedLenFeature(shape=(1,), dtype=tf.float32),
}

dnn_feature_columns = [
    tf.feature_column.embedding_column(
        tf.feature_column.categorical_column_with_identity("article_id", num_buckets=100000),
        dimension=8,
    ),
    tf.feature_column.numeric_column("article_vector", shape=(128,)),
]
linear_feature_columns = [
    tf.feature_column.categorical_column_with_identity("article_id", num_buckets=100000),
    tf.feature_column.numeric_column("article_vector", shape=(128,)),
]

For padded integer sequence fields in TFRecord, store the sequence as a fixed-length integer feature:

python
feature_description = {
    "hist_item_id": tf.io.FixedLenFeature(shape=(maxlen,), dtype=tf.int64),
}

Then use TensorFlow's categorical feature columns according to your Estimator model. The VarLenSparseFeat(maxlen=...) argument is part of the Keras-style API and is not used by tf.feature_column.

Quick Checklist

  • Use 0 for padding and start valid categorical ids from 1 for sequence fields.
  • Set vocabulary_size to at least max_id + 1.
  • Pad every VarLenSparseFeat input to (batch_size, maxlen).
  • Use length_name when you want explicit sequence lengths.
  • Use embedding_name to share embeddings between candidate and history fields.
  • For DIN/BST, history names must be hist_ + the names in behavior_feature_list.
  • For DSIN, split sessions offline and pass sess_0_*, sess_1_*, and sess_length.
  • Use DenseFeat(name, dimension) or numeric_column(name, shape=(dimension,)) for dense vectors.

Source Deepctr.Contrib.Rnn (`docs/source/deepctr.contrib.rnn.rst`)

deepctr.contrib.rnn module

.. automodule:: deepctr.contrib.rnn
:members:
:undoc-members:
:show-inheritance:

Source Deepctr.Contrib (`docs/source/deepctr.contrib.rst`)

deepctr.contrib package

Submodules

.. toctree::

deepctr.contrib.rnn
deepctr.contrib.utils

Module contents

.. automodule:: deepctr.contrib
:members:
:undoc-members:
:show-inheritance:

Source Deepctr.Contrib.Utils (`docs/source/deepctr.contrib.utils.rst`)

deepctr.contrib.utils module

.. automodule:: deepctr.contrib.utils
:members:
:undoc-members:
:show-inheritance:

Source Deepctr.Estimator.Feature Column (`docs/source/deepctr.estimator.feature_column.rst`)

deepctr.estimator.feature\_column module

.. automodule:: deepctr.estimator.feature_column
:members:
:undoc-members:
:show-inheritance:

Source Deepctr.Estimator.Inputs (`docs/source/deepctr.estimator.inputs.rst`)

deepctr.estimator.inputs module

.. automodule:: deepctr.estimator.inputs
:members:
:undoc-members:
:show-inheritance:

Source Deepctr.Estimator.Models.Afm (`docs/source/deepctr.estimator.models.afm.rst`)

deepctr.estimator.models.afm module

.. automodule:: deepctr.estimator.models.afm
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Estimator.Models.Autoint (`docs/source/deepctr.estimator.models.autoint.rst`)

deepctr.estimator.models.autoint module

.. automodule:: deepctr.estimator.models.autoint
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Estimator.Models.Ccpm (`docs/source/deepctr.estimator.models.ccpm.rst`)

deepctr.estimator.models.ccpm module

.. automodule:: deepctr.estimator.models.ccpm
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Estimator.Models.Dcn (`docs/source/deepctr.estimator.models.dcn.rst`)

deepctr.estimator.models.dcn module

.. automodule:: deepctr.estimator.models.dcn
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Estimator.Models.Deepfefm (`docs/source/deepctr.estimator.models.deepfefm.rst`)

deepctr.estimator.models.deepfefm module

.. automodule:: deepctr.estimator.models.deepfefm
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Estimator.Models.Deepfm (`docs/source/deepctr.estimator.models.deepfm.rst`)

deepctr.estimator.models.deepfm module

.. automodule:: deepctr.estimator.models.deepfm
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Estimator.Models.Fibinet (`docs/source/deepctr.estimator.models.fibinet.rst`)

deepctr.estimator.models.fibinet module

.. automodule:: deepctr.estimator.models.fibinet
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Estimator.Models.Fnn (`docs/source/deepctr.estimator.models.fnn.rst`)

deepctr.estimator.models.fnn module

.. automodule:: deepctr.estimator.models.fnn
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Estimator.Models.Fwfm (`docs/source/deepctr.estimator.models.fwfm.rst`)

deepctr.estimator.models.fwfm module

.. automodule:: deepctr.estimator.models.fwfm
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Estimator.Models.Nfm (`docs/source/deepctr.estimator.models.nfm.rst`)

deepctr.estimator.models.nfm module

.. automodule:: deepctr.estimator.models.nfm
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Estimator.Models.Pnn (`docs/source/deepctr.estimator.models.pnn.rst`)

deepctr.estimator.models.pnn module

.. automodule:: deepctr.estimator.models.pnn
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Estimator.Models (`docs/source/deepctr.estimator.models.rst`)

deepctr.estimator.models package

Submodules

.. toctree::

deepctr.estimator.models.afm
deepctr.estimator.models.autoint
deepctr.estimator.models.ccpm
deepctr.estimator.models.dcn
deepctr.estimator.models.deepfm
deepctr.estimator.models.fwfm
deepctr.estimator.models.fibinet
deepctr.estimator.models.fnn
deepctr.estimator.models.nfm
deepctr.estimator.models.pnn
deepctr.estimator.models.wdl
deepctr.estimator.models.xdeepfm

Module contents

.. automodule:: deepctr.estimator.models
:members:
:undoc-members:
:show-inheritance:

Source Deepctr.Estimator.Models.Wdl (`docs/source/deepctr.estimator.models.wdl.rst`)

deepctr.estimator.models.wdl module

.. automodule:: deepctr.estimator.models.wdl
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Estimator.Models.Xdeepfm (`docs/source/deepctr.estimator.models.xdeepfm.rst`)

deepctr.estimator.models.xdeepfm module

.. automodule:: deepctr.estimator.models.xdeepfm
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Estimator (`docs/source/deepctr.estimator.rst`)

deepctr.estimator package

Subpackages

.. toctree::

text
deepctr.estimator.models

Submodules

.. toctree::

deepctr.estimator.feature_column
deepctr.estimator.inputs
deepctr.estimator.utils

Module contents

.. automodule:: deepctr.estimator
:members:
:undoc-members:
:show-inheritance:

Source Deepctr.Estimator.Utils (`docs/source/deepctr.estimator.utils.rst`)

deepctr.estimator.utils module

.. automodule:: deepctr.estimator.utils
:members:
:undoc-members:
:show-inheritance:

Source Deepctr.Feature Column (`docs/source/deepctr.feature_column.rst`)

deepctr.feature\_column module

.. automodule:: deepctr.feature_column
:members:
:undoc-members:
:show-inheritance:

Source Deepctr.Inputs (`docs/source/deepctr.inputs.rst`)

deepctr.inputs module

.. automodule:: deepctr.inputs
:members:
:undoc-members:
:show-inheritance:

Source Deepctr.Layers.Activation (`docs/source/deepctr.layers.activation.rst`)

deepctr.layers.activation module

.. automodule:: deepctr.layers.activation
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Layers.Core (`docs/source/deepctr.layers.core.rst`)

deepctr.layers.core module

.. automodule:: deepctr.layers.core
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Layers.Interaction (`docs/source/deepctr.layers.interaction.rst`)

deepctr.layers.interaction module

.. automodule:: deepctr.layers.interaction
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Layers.Normalization (`docs/source/deepctr.layers.normalization.rst`)

deepctr.layers.normalization module

.. automodule:: deepctr.layers.normalization
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Layers (`docs/source/deepctr.layers.rst`)

deepctr.layers package

Submodules

.. toctree::

deepctr.layers.activation
deepctr.layers.core
deepctr.layers.interaction
deepctr.layers.normalization
deepctr.layers.sequence
deepctr.layers.utils

Module contents

.. automodule:: deepctr.layers
:members:
:undoc-members:
:show-inheritance:

Source Deepctr.Layers.Sequence (`docs/source/deepctr.layers.sequence.rst`)

deepctr.layers.sequence module

.. automodule:: deepctr.layers.sequence
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Layers.Utils (`docs/source/deepctr.layers.utils.rst`)

deepctr.layers.utils module

.. automodule:: deepctr.layers.utils
:members:
:undoc-members:
:show-inheritance:

Source Deepctr.Models.Afm (`docs/source/deepctr.models.afm.rst`)

deepctr.models.afm module

.. automodule:: deepctr.models.afm
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Autoint (`docs/source/deepctr.models.autoint.rst`)

deepctr.models.autoint module

.. automodule:: deepctr.models.autoint
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Ccpm (`docs/source/deepctr.models.ccpm.rst`)

deepctr.models.ccpm module

.. automodule:: deepctr.models.ccpm
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Dcn (`docs/source/deepctr.models.dcn.rst`)

deepctr.models.dcn module

.. automodule:: deepctr.models.dcn
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Dcnmix (`docs/source/deepctr.models.dcnmix.rst`)

deepctr.models.dcnmix module

.. automodule:: deepctr.models.dcnmix
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Deepfefm (`docs/source/deepctr.models.deepfefm.rst`)

deepctr.models.deepfefm module

.. automodule:: deepctr.models.deepfefm
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Deepfm (`docs/source/deepctr.models.deepfm.rst`)

deepctr.models.deepfm module

.. automodule:: deepctr.models.deepfm
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Difm (`docs/source/deepctr.models.difm.rst`)

deepctr.models.difm module

.. automodule:: deepctr.models.difm
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Edcn (`docs/source/deepctr.models.edcn.rst`)

deepctr.models.edcn module

.. automodule:: deepctr.models.edcn
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Fgcnn (`docs/source/deepctr.models.fgcnn.rst`)

deepctr.models.fgcnn module

.. automodule:: deepctr.models.fgcnn
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Fibinet (`docs/source/deepctr.models.fibinet.rst`)

deepctr.models.fibinet module

.. automodule:: deepctr.models.fibinet
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Flen (`docs/source/deepctr.models.flen.rst`)

deepctr.models.flen module

.. automodule:: deepctr.models.flen
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Fnn (`docs/source/deepctr.models.fnn.rst`)

deepctr.models.fnn module

.. automodule:: deepctr.models.fnn
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Fwfm (`docs/source/deepctr.models.fwfm.rst`)

deepctr.models.fwfm module

.. automodule:: deepctr.models.fwfm
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Ifm (`docs/source/deepctr.models.ifm.rst`)

deepctr.models.ifm module

.. automodule:: deepctr.models.ifm
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Mlr (`docs/source/deepctr.models.mlr.rst`)

deepctr.models.mlr module

.. automodule:: deepctr.models.mlr
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Multitask.Esmm (`docs/source/deepctr.models.multitask.esmm.rst`)

deepctr.models.multitask.esmm module

.. automodule:: deepctr.models.multitask.esmm
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Multitask.Mmoe (`docs/source/deepctr.models.multitask.mmoe.rst`)

deepctr.models.multitask.mmoe module

.. automodule:: deepctr.models.multitask.mmoe
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Multitask.Ple (`docs/source/deepctr.models.multitask.ple.rst`)

deepctr.models.multitask.ple module

.. automodule:: deepctr.models.multitask.ple
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Multitask.Sharedbottom (`docs/source/deepctr.models.multitask.sharedbottom.rst`)

deepctr.models.multitask.sharedbottom module

.. automodule:: deepctr.models.multitask.sharedbottom
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Nfm (`docs/source/deepctr.models.nfm.rst`)

deepctr.models.nfm module

.. automodule:: deepctr.models.nfm
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Onn (`docs/source/deepctr.models.onn.rst`)

deepctr.models.onn module

.. automodule:: deepctr.models.onn
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Pnn (`docs/source/deepctr.models.pnn.rst`)

deepctr.models.pnn module

.. automodule:: deepctr.models.pnn
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models (`docs/source/deepctr.models.rst`)

deepctr.models package

Submodules

.. toctree::

deepctr.models.afm
deepctr.models.autoint
deepctr.models.ccpm
deepctr.models.dcn
deepctr.models.dcnmix
deepctr.models.edcn
deepctr.models.deepfm
deepctr.models.dien
deepctr.models.din
deepctr.models.dsin
deepctr.models.fgcnn
deepctr.models.fibinet
deepctr.models.fnn
deepctr.models.fwfm
deepctr.models.mlr
deepctr.models.onn
deepctr.models.nfm
deepctr.models.pnn
deepctr.models.wdl
deepctr.models.xdeepfm
deepctr.models.flen
deepctr.models.ifm
deepctr.models.difm
deepctr.models.deepfefm
deepctr.models.multitask.sharedbottom
deepctr.models.multitask.esmm
deepctr.models.multitask.mmoe
deepctr.models.multitask.ple

Module contents

.. automodule:: deepctr.models
:members:
:undoc-members:
:show-inheritance:

Source Deepctr.Models.Sequence.Bst (`docs/source/deepctr.models.sequence.bst.rst`)

deepctr.models.sequence.bst module

.. automodule:: deepctr.models.sequence.bst
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Sequence.Dien (`docs/source/deepctr.models.sequence.dien.rst`)

deepctr.models.sequence.dien module

.. automodule:: deepctr.models.sequence.dien
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Sequence.Din (`docs/source/deepctr.models.sequence.din.rst`)

deepctr.models.sequence.din module

.. automodule:: deepctr.models.sequence.din
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Sequence.Dsin (`docs/source/deepctr.models.sequence.dsin.rst`)

deepctr.models.sequence.dsin module

.. automodule:: deepctr.models.sequence.dsin
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Wdl (`docs/source/deepctr.models.wdl.rst`)

deepctr.models.wdl module

.. automodule:: deepctr.models.wdl
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr.Models.Xdeepfm (`docs/source/deepctr.models.xdeepfm.rst`)

deepctr.models.xdeepfm module

.. automodule:: deepctr.models.xdeepfm
:members:
:no-undoc-members:
:no-show-inheritance:

Source Deepctr (`docs/source/deepctr.rst`)

deepctr package

Subpackages

.. toctree::

text
deepctr.contrib
deepctr.layers
deepctr.models

Submodules

.. toctree::

deepctr.inputs
deepctr.utils

Module contents

.. automodule:: deepctr
:members:
:undoc-members:
:show-inheritance:

Source Deepctr.Utils (`docs/source/deepctr.utils.rst`)

deepctr.utils module

.. automodule:: deepctr.utils
:members:
:undoc-members:
:show-inheritance:

Source Index (`docs/source/index.rst`)

.. DeepCTR documentation master file, created by
sphinx-quickstart on Fri Nov 23 21:08:54 2018.
You can adapt this file completely to your liking, but it should at least
contain the root toctree directive.

Welcome to DeepCTR's documentation!

|Downloads|_ |Stars|_ |Forks|_ |PyPii|_ |Issues|_ |Chat|_

.. |Downloads| image:: https://pepy.tech/badge/deepctr
.. _Downloads: https://pepy.tech/project/deepctr

.. |Stars| image:: https://img.shields.io/github/stars/shenweichen/deepctr.svg
.. _Stars: https://github.com/shenweichen/DeepCTR

.. |Forks| image:: https://img.shields.io/github/forks/shenweichen/deepctr.svg
.. _Forks: https://github.com/shenweichen/DeepCTR/fork

.. |PyPii| image:: https://img.shields.io/pypi/v/deepctr.svg
.. _PyPii: https://pypi.org/project/deepctr

.. |Issues| image:: https://img.shields.io/github/issues/shenweichen/deepctr.svg
.. _Issues: https://github.com/shenweichen/deepctr/issues

.. |Chat| image:: https://img.shields.io/badge/chat-wechat-brightgreen?style=flat
.. _Chat: ./#disscussiongroup

DeepCTR is a Easy-to-use , Modular and Extendible package of deep-learning based CTR models along with lots of core components layer which can be used to easily build custom models.You can use any complex model with model.fit() and model.predict().

  • Provide tf.keras.Model like interface for quick experiment. example <https://deepctr-doc.readthedocs.io/en/latest/Quick-Start.html#getting-started-4-steps-to-deepctr>_
  • Provide tensorflow estimator interface for large scale data and distributed training. example <https://deepctr-doc.readthedocs.io/en/latest/Quick-Start.html#getting-started-4-steps-to-deepctr-estimator-with-tfrecord>_
  • It is compatible with both tf 1.15 and tf 2.x.

Install a TensorFlow build that matches your Python, NumPy, CPU/GPU, and operating system first, then install DeepCTR. Use public tensorflow.keras APIs in your own code and avoid mixing them with private tensorflow.python.keras imports.

Let's Get Started! <./Quick-Start.html>_ (Chinese Introduction <https://zhuanlan.zhihu.com/p/53231955>_)

You can read the latest code and related projects

News

04/16/2026 : Support higher tensorflow version . Changelog <https://github.com/shenweichen/DeepCTR/releases/tag/v0.9.4>_

11/10/2022 : Add EDCN . Changelog <https://github.com/shenweichen/DeepCTR/releases/tag/v0.9.3>_

10/15/2022 : Support python 3.9 , 3.10 . Changelog <https://github.com/shenweichen/DeepCTR/releases/tag/v0.9.2>_

DisscussionGroup

公众号：浅梦学习笔记 wechat ID: deepctrbot

Discussions <https://github.com/shenweichen/DeepCTR/discussions>_

.. image:: ../pics/code.png

.. toctree::
:maxdepth: 2
:caption: Home:

Quick-Start<Quick-Start.md>
Features<Features.md>
Examples<Examples.md>
FAQ<FAQ.md>
History<History.md>

.. toctree::
:maxdepth: 2
:caption: Cookbook:

Sequence Feature Cookbook<Sequence-Cookbook.md>

.. toctree::
:maxdepth: 3
:caption: API:

Models
Estimators
Layers

Indices and tables

  • :ref:genindex
  • :ref:modindex
  • :ref:search

Source Modules (`docs/source/modules.rst`)

deepctr

.. toctree::
:maxdepth: 4

deepctr

--- METRICS ---

Explore Related Technologies & Topics

View All 9,111+ Repositories →

Discover similar high-velocity repositories, agent skills, and OpenAPI specifications across the ecosystem.

#autoint Browse autoint catalog → #click-through-rate Browse click-through-rate catalog → #ctr Browse ctr catalog → #deep-learning Browse deep-learning catalog → #deepcross Browse deepcross catalog → #deepfm Browse deepfm catalog → ⚡ MCP Servers Model Context Protocol → 🧠 Agent Skills Prompt & rule files → 🔌 OpenAPI Specs REST & Swagger schemas →