# Technical Documentation: shenweichen/DeepCTR > ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/shenweichen/DeepCTR) ### Overview (`README.md`) # DeepCTR [](https://pypi.org/project/deepctr) [](https://pypi.org/project/deepctr) [](https://pepy.tech/project/deepctr) [](https://pypi.org/project/deepctr) [![GitHub Issues](https://img.shields.io/github/issues/shenweichen/deepctr.svg )](https://github.com/shenweichen/deepctr/issues) [](https://deepctr-doc.readthedocs.io/) [](https://github.com/shenweichen/DeepCTR/actions/workflows/ci.yml) [](https://github.com/shenweichen/DeepCTR/actions/workflows/ci2.yml) [](https://codecov.io/gh/shenweichen/DeepCTR) [](https://www.codacy.com/gh/shenweichen/DeepCTR?utm_source=github.com&utm_medium=referral&utm_content=shenweichen/DeepCTR&utm_campaign=Badge_Grade) [](./README.md#DisscussionGroup) [](https://github.com/shenweichen/deepctr/blob/master/LICENSE) DeepCTR is a **Easy-to-use**, **Modular** and **Extendible** package of deep-learning based CTR models along with lots of core components layers which can be used to easily build custom models.You can use any complex model with `model.fit()` ，and `model.predict()` . - Provide `tf.keras.Model` like interfaces for **quick experiment**. [example](https://deepctr-doc.readthedocs.io/en/latest/Quick-Start.html#getting-started-4-steps-to-deepctr) - Provide `tensorflow estimator` interface for **large scale data** and **distributed training**. [example](https://deepctr-doc.readthedocs.io/en/latest/Quick-Start.html#getting-started-4-steps-to-deepctr-estimator-with-tfrecord) - It is compatible with both `tf 1.15` and `tf 2.x`. ## Installation and compatibility DeepCTR does not pin or install TensorFlow for you. Install a TensorFlow build that matches your Python, NumPy, CPU/GPU, and operating system first, then install DeepCTR: ```bash pip install tensorflow pip install deepctr ``` For Python `>=3.9`, DeepCTR allows modern `h5py` releases with `h5py>=3.7.0`. If TensorFlow reports a NumPy conflict, follow the TensorFlow requirement for your selected TensorFlow release, for example using `numpy<2` when required by TensorFlow. Use public `tensorflow.keras` APIs in your own code and examples. Avoid mixing `tensorflow.python.keras` with `tensorflow.keras`, because `tensorflow.python.*` is private TensorFlow API and can break model serialization or optimizer/metric loading across TensorFlow versions. Some related projects: - DeepMatch: https://github.com/shenweichen/DeepMatch - DeepCTR-Torch: https://github.com/shenweichen/DeepCTR-Torch Let's [**Get Started!**](https://deepctr-doc.readthedocs.io/en/latest/Quick-Start.html)([Chinese Introduction](https://zhuanlan.zhihu.com/p/53231955)) and [welcome to join us!](./CONTRIBUTING.md) ## Models List | Model | Paper | | :------------------------------------: | :-------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Convolutional Click Prediction Model | [CIKM 2015][A Convolutional Click Prediction Model](http://ir.ia.ac.cn/bitstream/173211/12337/1/A%20Convolutional%20Click%20Prediction%20Model.pdf) | | Factorization-supported Neural Network | [ECIR 2016][Deep Learning over Multi-field Categorical Data: A Case Study on User Response Prediction](https://arxiv.org/pdf/1601.02376.pdf) | | Product-based Neural Network | [ICDM 2016][Product-based neural networks for user response prediction](https://arxiv.org/pdf/1611.00144.pdf) | | Wide & Deep | [DLRS 2016][Wide & Deep Learning for Recommender Systems](https://arxiv.org/pdf/1606.07792.pdf) | | DeepFM | [IJCAI 2017][DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](http://www.ijcai.org/proceedings/2017/0239.pdf) | | Piece-wise Linear Model | [arxiv 2017][Learning Piece-wise Linear Models from Large Scale Data for Ad Click Prediction](https://arxiv.org/abs/1704.05194) | | Deep & Cross Network | [ADKDD 2017][Deep & Cross Network for Ad Click Predictions](https://arxiv.org/abs/1708.05123) | | Attentional Factorization Machine | [IJCAI 2017][Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks](http://www.ijcai.org/proceedings/2017/435) | | Neural Factorization Machine | [SIGIR 2017][Neural Factorization Machines for Sparse Predictive Analytics](https://arxiv.org/pdf/1708.05027.pdf) | | xDeepFM | [KDD 2018][xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems](https://arxiv.org/pdf/1803.05170.pdf) | | Deep Interest Network | [KDD 2018][Deep Interest Network for Click-Through Rate Prediction](https://arxiv.org/pdf/1706.06978.pdf) | | AutoInt | [CIKM 2019][AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://arxiv.org/abs/1810.11921) | | Deep Interest Evolution Network | [AAAI 2019][Deep Interest Evolution Network for Click-Through Rate Prediction](https://arxiv.org/pdf/1809.03672.pdf) | | FwFM | [WWW 2018][Field-weighted Factorization Machines for Click-Through Rate Prediction in Display Advertising](https://arxiv.org/pdf/1806.03514.pdf) | | ONN | [arxiv 2019][Operation-aware Neural Networks for User Response Prediction](https://arxiv.org/pdf/1904.12579.pdf) | | FGCNN | [WWW 2019][Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction ](https://arxiv.org/pdf/1904.04447) | | Deep Session Interest Network | [IJCAI 2019][Deep Session Interest Network for Click-Through Rate Prediction ](https://arxiv.org/abs/1905.06482) | | FiBiNET | [RecSys 2019][FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Prediction](https://arxiv.org/pdf/1905.09433.pdf) | | FLEN | [arxiv 2019][FLEN: Leveraging Field for Scalable CTR Prediction](https://arxiv.org/pdf/1911.04690.pdf) | | BST | [DLP-KDD 2019][Behavior sequence transformer for e-commerce recommendation in Alibaba](https://arxiv.org/pdf/1905.06874.pdf) | | IFM | [IJCAI 2019][An Input-aware Factorization Machine for Sparse Prediction](https://www.ijcai.org/Proceedings/2019/0203.pdf) | | DCN V2 | [arxiv 2020][DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://arxiv.org/abs/2008.13535) | | DIFM | [IJCAI 2020][A Dual Input-aware Factorization Machine for CTR Prediction](https://www.ijcai.org/Proceedings/2020/0434.pdf) | | FEFM and DeepFEFM | [arxiv 2020][Field-Embedded Factorization Machines for Click-through rate prediction](https://arxiv.org/abs/2009.09931) | | SharedBottom | [arxiv 2017][An Overview of Multi-Task Learning in Deep Neural Networks](https://arxiv.org/pdf/1706.05098.pdf) | | ESMM | [SIGIR 2018][Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate](https://arxiv.org/abs/1804.07931) | | MMOE | [KDD 2018][Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts](https://dl.acm.org/doi/abs/10.1145/3219819.3220007) | | PLE | [RecSys 2020][Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations](https://dl.acm.org/doi/10.1145/3383313.3412236) | | EDCN | [KDD 2021][Enhancing Explicit and Implicit Feature Interactions via Information Sharing for Parallel Deep CTR Models](https://dlp-kdd.github.io/assets/pdf/DLP-KDD_2021_paper_12.pdf) | ## Citation - Weichen Shen. (2017). DeepCTR: Easy-to-use,Modular and Extendible package of deep-learning based CTR models. https://github.com/shenweichen/deepctr. If you find this code useful in your research, please cite it using the following BibTeX: ```bibtex @misc{shen2017deepctr, author = {Weichen Shen}, title = {DeepCTR: Easy-to-use,Modular and Extendible package of deep-learning based CTR models}, year = {2017}, publisher = {GitHub}, journal = {GitHub Repository}, howpublished = {\url{https://github.com/shenweichen/deepctr}}, } ``` ## DisscussionGroup - [Github Discussions](https://github.com/shenweichen/DeepCTR/discussions) - Wechat Discussions |公众号：浅梦学习笔记|微信：deepctrbot| |:--:|:--:| | [](https://github.com/shenweichen/AlgoNotes)| [](https://github.com/shenweichen/AlgoNotes)| ## Main contributors([welcome to join us!](./CONTRIBUTING.md))
​ pic
​ Shen Weichen ​

Alibaba Group

​ 		pic
Zan Shuxun ​

Alibaba Group

​ 		​ pic
​ Harshit Pande

Amazon

​ 		​ pic
​ Lai Mincai

ByteDance

​ 		​ pic
​ Li Zichao

ByteDance

​ 		​ pic
Tan Tingyi

Chongqing University
of Posts and
Telecommunications

​
--- ### Requirements.Readthedocs (`docs/requirements.readthedocs.txt`) numpy<2 Jinja2<3.1 docutils<0.18 sphinx==4.5.0 sphinx-rtd-theme==0.5.2 sphinxcontrib-applehelp==1.0.2 sphinxcontrib-devhelp==1.0.2 sphinxcontrib-htmlhelp==2.0.0 sphinxcontrib-qthelp==1.0.3 sphinxcontrib-serializinghtml==1.1.5 recommonmark==0.7.1 tensorflow==2.15.0 --- ### Source Estimators (`docs/source/Estimators.rst`) DeepCTR Estimators API ====================== .. toctree:: CCPM FNN PNN WDL DeepFM NFM AFM DCN xDeepFM AutoInt FiBiNET --- ### Source Examples (`docs/source/Examples.md`) # Examples ## Classification: Criteo The Criteo Display Ads dataset is for the purpose of predicting ads click-through rate. It has 13 integer features and 26 categorical features where each category has a high cardinality. In this example,we simply normailize the dense feature between 0 and 1,you can try other transformation technique like log normalization or discretization.Then we use [SparseFeat](./Features.html#sparsefeat) and [DenseFeat](./Features.html#densefeat) to generate feature columns for sparse features and dense features. This example shows how to use ``DeepFM`` to solve a simple binary classification task. You can get the demo data [criteo_sample.txt](https://github.com/shenweichen/DeepCTR/tree/master/examples/criteo_sample.txt) and run the following codes. ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Classification: Criteo with feature hashing on the fly This example shows how to use ``DeepFM`` to solve a simple binary classification task using feature hashing. You can get the demo data [criteo_sample.txt](https://github.com/shenweichen/DeepCTR/tree/master/examples/criteo_sample.txt) and run the following codes. ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Regression: Movielens The MovieLens data has been used for personalized tag recommendation,which contains 668, 953 tag applications of users on movies. Here is a small fraction of data include only sparse field. This example shows how to use ``DeepFM`` to solve a simple binary regression task. You can get the demo data [movielens_sample.txt](https://github.com/shenweichen/DeepCTR/tree/master/examples/movielens_sample.txt) and run the following codes. ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Multi-value Input : Movielens The MovieLens data has been used for personalized tag recommendation,which contains 668, 953 tag applications of users on movies. Here is a small fraction of data include sparse fields and a multivalent field. There are 2 additional steps to use DeepCTR with sequence feature input. 1. Generate the paded and encoded sequence feature of sequence input feature(**value 0 is for padding**). 2. Generate config of sequence feature with [VarLenSparseFeat](./Features.html#varlensparsefeat) For more details about padding, `maxlen`, embedding sharing, DIN history fields, DSIN session fields, and TFRecord inputs, see the Sequence Feature Cookbook. This example shows how to use ``DeepFM`` with sequence(multi-value) feature. You can get the demo data [movielens_sample.txt](https://github.com/shenweichen/DeepCTR/tree/master/examples/movielens_sample.txt) and run the following codes. ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Multi-value Input : Movielens with feature hashing on the fly ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Hash Layer with pre-defined key-value vocabulary This examples how to use pre-defined key-value vocabulary in `Hash` Layer.`movielens_age_vocabulary.csv` stores the key-value mapping for `age` feature. ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Estimator with TFRecord: Classification Criteo This example shows how to use ``DeepFMEstimator`` to solve a simple binary classification task. You can get the demo data [criteo_sample.tr.tfrecords](https://github.com/shenweichen/DeepCTR/tree/master/examples/criteo_sample.tr.tfrecords) and [criteo_sample.te.tfrecords](https://github.com/shenweichen/DeepCTR/tree/master/examples/criteo_sample.te.tfrecords) and run the following codes. ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Estimator with Pandas DataFrame: Classification Criteo This example shows how to use ``DeepFMEstimator`` to solve a simple binary classification task. You can get the demo data [criteo_sample.txt](https://github.com/shenweichen/DeepCTR/tree/master/examples/criteo_sample.txt) and run the following codes. ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## MultiTask Learning:MMOE The UCI census-income dataset is extracted from the 1994 census database. It contains 299,285 instances of demographic information of American adults. There are 40 features in total. We construct a multi-task learning problem from this dataset by setting some of the features as prediction targets : - Task 1: Predict whether the income exceeds $50K; - Task 2: Predict whether this person’s marital status is never married. This example shows how to use ``MMOE`` to solve a multi task learning problem. You can get the demo data [census-income.sample](https://github.com/shenweichen/DeepCTR/tree/master/examples/census-income.sample) and run the following codes. ``` /* Detailed source-code truncated for AI context efficiency. */ ``` --- ### Source FAQ (`docs/source/FAQ.md`) # FAQ ## 1. Save or load weights/models ---------------------------------------- To save/load weights,you can write codes just like any other keras models. ```python model = DeepFM() model.save_weights('DeepFM_w.h5') model.load_weights('DeepFM_w.h5') ``` To save/load models,just a little different. ```python from tensorflow.keras.models import save_model, load_model model = DeepFM() save_model(model, 'DeepFM.h5')# save_model, same as before from deepctr.layers import custom_objects model = load_model('DeepFM.h5',custom_objects)# load_model,just add a parameter ``` ## 2. Set learning rate and use earlystopping --------------------------------------------------- You can use any models in DeepCTR like a keras model object. Here is a example of how to set learning rate and earlystopping: ```python import deepctr from tensorflow.keras.optimizers import Adam, Adagrad from tensorflow.keras.callbacks import EarlyStopping model = deepctr.models.DeepFM(linear_feature_columns,dnn_feature_columns) model.compile(Adagrad(0.1024),'binary_crossentropy',metrics=['binary_crossentropy']) es = EarlyStopping(monitor='val_binary_crossentropy') history = model.fit(model_input, data[target].values,batch_size=256, epochs=10, verbose=2, validation_split=0.2,callbacks=[es] ) ``` If you are using Estimator models, you can set learning rate like: ```python from deepctr.estimator import DeepFMEstimator import tensorflow as tf model = DeepFMEstimator(linear_feature_columns, dnn_feature_columns, task='binary', linear_optimizer=tf.train.FtrlOptimizer(0.05), dnn_optimizer=tf.train.AdagradOptimizer(0.1) ) ``` ## 3. Get the attentional weights of feature interactions in AFM -------------------------------------------------------------------------- First,make sure that you have install the latest version of deepctr. Then,use the following code,the `attentional_weights[:,i,0]` is the `feature_interactions[i]`'s attentional weight of all samples. ```python import itertools import deepctr from deepctr.models import AFM from deepctr.feature_column import get_feature_names from tensorflow.keras.models import Model from tensorflow.keras.layers import Lambda model = AFM(linear_feature_columns,dnn_feature_columns) model.fit(model_input,target) afmlayer = model.layers[-3] afm_weight_model = Model(model.input,outputs=Lambda(lambda x:afmlayer.normalized_att_score)(model.input)) attentional_weights = afm_weight_model.predict(model_input,batch_size=4096) feature_names = get_feature_names(dnn_feature_columns) feature_interactions = list(itertools.combinations(feature_names ,2)) ``` ## 4. How to extract the embedding vectors in deepfm? ```python feature_columns = [SparseFeat('user_id',120,),SparseFeat('item_id',60,),SparseFeat('cate_id',60,)] def get_embedding_weights(dnn_feature_columns,model): embedding_dict = {} for fc in dnn_feature_columns: if hasattr(fc,'embedding_name'): if fc.embedding_name is not None: name = fc.embedding_name else: name = fc.name embedding_dict[name] = model.get_layer("sparse_emb_"+name).get_weights()[0] return embedding_dict embedding_dict = get_embedding_weights(feature_columns,model) user_id_emb = embedding_dict['user_id'] item_id_emb = embedding_dict['item_id'] ``` ## 5. How to add a long dense feature vector as a input to the model? ```python from deepctr.models import DeepFM from deepctr.feature_column import SparseFeat, DenseFeat,get_feature_names import numpy as np feature_columns = [SparseFeat('user_id',120,),SparseFeat('item_id',60,),DenseFeat("pic_vec",5)] fixlen_feature_names = get_feature_names(feature_columns) user_id = np.array([[1],[0],[1]]) item_id = np.array([[30],[20],[10]]) pic_vec = np.array([[0.1,0.5,0.4,0.3,0.2],[0.1,0.5,0.4,0.3,0.2],[0.1,0.5,0.4,0.3,0.2]]) label = np.array([1,0,1]) model_input = {'user_id':user_id,'item_id':item_id,'pic_vec':pic_vec} model = DeepFM(feature_columns,feature_columns) model.compile('adagrad','binary_crossentropy') model.fit(model_input,label) ``` ## 6. How to use pretrained weights to initialize embedding weights and frozen embedding weights? ----------------------------------------------------------------------------------------------------- Use `tf.initializers.identity()` to set the `embeddings_initializer` of `SparseFeat`,and set `trainable=False` to frozen embedding weights. ```python import numpy as np import tensorflow as tf from deepctr.models import DeepFM from deepctr.feature_column import SparseFeat,get_feature_names pretrained_item_weights = np.random.randn(60,4) pretrained_weights_initializer = tf.initializers.constant(pretrained_item_weights) feature_columns = [SparseFeat('user_id',120,),SparseFeat('item_id',60,embedding_dim=4,embeddings_initializer=pretrained_weights_initializer,trainable=False)] fixlen_feature_names = get_feature_names(feature_columns) user_id = np.array([[1],[0],[1]]) item_id = np.array([[30],[20],[10]]) label = np.array([1,0,1]) model_input = {'user_id':user_id,'item_id':item_id,} model = DeepFM(feature_columns,feature_columns) model.compile('adagrad','binary_crossentropy') model.fit(model_input,label) ``` ## 7. How to run the demo with GPU ? Install the TensorFlow build recommended for your CUDA, cuDNN, and platform combination, then install `deepctr`. ## 8. How to avoid TensorFlow, Keras, h5py, or NumPy compatibility errors? Install TensorFlow separately before installing DeepCTR. Pick the TensorFlow release according to your Python version, CPU/GPU environment, and platform. ```bash $ pip install tensorflow $ pip install deepctr ``` For Python `>=3.9`, DeepCTR uses `h5py>=3.7.0`, so newer `h5py` releases such as `3.9+` and `3.12+` are allowed. If TensorFlow reports a NumPy conflict, follow the TensorFlow requirement for the TensorFlow release you installed, for example using `numpy<2` when required by TensorFlow. Use public `tensorflow.keras` imports in your own code: ```python from tensorflow.keras.models import load_model from tensorflow.keras.optimizers import Adam ``` Avoid mixing `tensorflow.python.keras` with `tensorflow.keras`. `tensorflow.python.*` is private TensorFlow API and can break serialization, optimizer loading, or metric loading across TensorFlow versions. ## 9. How to run the demo with multiple GPUs you can use multiple gpus with tensorflow version higher than ``1.4``,see [run_classification_criteo_multi_gpu.py](https://github.com/shenweichen/DeepCTR/blob/master/examples/run_classification_criteo_multi_gpu.py) --- ### Source Features (`docs/source/Features.md`) # Features ## Overview With the great success of deep learning,DNN-based techniques have been widely used in CTR prediction task. DNN based CTR prediction models usually have following 4 modules: `Input,Embedding,Low-order&High-order Feature Extractor,Prediction` - Input&Embedding > The data in CTR estimation task usually includes high sparse,high cardinality categorical features and some dense numerical features. > Since DNN are good at handling dense numerical features,we usually map the sparse categorical features to dense numerical through `embedding technique`. > For numerical features,we usually apply `discretization` or `normalization` on them. - Feature Extractor > Low-order Extractor learns feature interaction through product between vectors.Factorization-Machine and it's variants are widely used to learn the low-order feature interaction. > High-order Extractor learns feature combination through complex neural network functions like MLP,Cross Net,etc. ## Feature Columns ### SparseFeat ``SparseFeat`` is a namedtuple with signature ``SparseFeat(name, vocabulary_size, embedding_dim, use_hash, vocabulary_path, dtype, embeddings_initializer, embedding_name, group_name, trainable)`` - name : feature name - vocabulary_size : number of unique feature values for sparse feature or hashing space when `use_hash=True` - embedding_dim : embedding dimension - use_hash : default `False`.If `True` the input will be hashed to space of size `vocabulary_size`. - vocabulary_path : default `None`. The `CSV` text file path of the vocabulary table used by `tf.lookup.TextFileInitializer`, which assigns one entry in the table for each line in the file. One entry contains two columns separated by comma, the first is the value column, the second is the key column. The `0` value is reserved to use if a key is missing in the table, so hash value need start from `1`. - dtype : default `int32`.dtype of input tensor. - embeddings_initializer : initializer for the `embeddings` matrix. - embedding_name : default `None`. If None, the embedding_name will be same as `name`. - group_name : feature group of this feature. - trainable: default `True`.Whether or not the embedding is trainable. ### DenseFeat ``DenseFeat`` is a namedtuple with signature ``DenseFeat(name, dimension, dtype, transform_fn)`` - name : feature name - dimension : dimension of dense feature vector. - dtype : default `float32`.dtype of input tensor. - transform_fn : If not `None` , a function that can be used to transform values of the feature. the function takes the input Tensor as its argument, and returns the output Tensor. (e.g. `lambda x: (x - 3.0) / 4.2)`. ### VarLenSparseFeat ``VarLenSparseFeat`` is a namedtuple with signature ``VarLenSparseFeat(sparsefeat, maxlen, combiner, length_name, weight_name,weight_norm)`` - sparsefeat : a instance of `SparseFeat` - maxlen : maximum length of this feature for all samples - combiner : pooling method,can be ``sum``,``mean`` or ``max`` - length_name : feature length name,if `None`, value 0 in feature is for padding. - weight_name : default `None`. If not None, the sequence feature will be multiplyed by the feature whose name is `weight_name`. - weight_norm : default `True`. Whether normalize the weight score or not. ## Models ### CCPM (Convolutional Click Prediction Model) CCPM can extract local-global key features from an input instance with varied elements, which can be implemented for not only single ad impression but also sequential ad impression. [**CCPM Model API**](./deepctr.models.ccpm.html) [**CCPM Estimator API**](./deepctr.estimator.models.ccpm.html) [Liu Q, Yu F, Wu S, et al. A convolutional click prediction model[C]//Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. ACM, 2015: 1743-1746.](http://ir.ia.ac.cn/bitstream/173211/12337/1/A%20Convolutional%20Click%20Prediction%20Model.pdf) ### FNN (Factorization-supported Neural Network) According to the paper,FNN learn embedding vectors of categorical data via pre-trained FM. It use FM's latent vector to initialiaze the embedding vectors.During the training stage,it concatenates the embedding vectors and feeds them into a MLP(MultiLayer Perceptron). [**FNN Model API**](./deepctr.models.fnn.html) [**FNN Estimator API**](./deepctr.estimator.models.fnn.html) [Zhang W, Du T, Wang J. Deep learning over multi-field categorical data[C]//European conference on information retrieval. Springer, Cham, 2016: 45-57.](https://arxiv.org/pdf/1601.02376.pdf) ### PNN (Product-based Neural Network) PNN concatenates sparse feature embeddings and the product between embedding vectors as the input of MLP. [**PNN Model API**](./deepctr.models.pnn.html) [**PNN Estimator API**](./deepctr.estimator.models.pnn.html) [Qu Y, Cai H, Ren K, et al. Product-based neural networks for user response prediction[C]//Data Mining (ICDM), 2016 IEEE 16th International Conference on. IEEE, 2016: 1149-1154.](https://arxiv.org/pdf/1611.00144.pdf) ### Wide & Deep WDL's deep part concatenates sparse feature embeddings as the input of MLP,the wide part use handcrafted feature as input. The logits of deep part and wide part are added to get the prediction probability. [**WDL Model API**](./deepctr.models.wdl.html) [**WDL Estimator API**](./deepctr.estimator.models.wdl.html) [Cheng H T, Koc L, Harmsen J, et al. Wide & deep learning for recommender systems[C]//Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. ACM, 2016: 7-10.](https://arxiv.org/pdf/1606.07792.pdf) ### DeepFM DeepFM can be seen as an improvement of WDL and FNN.Compared with WDL,DeepFM use FM instead of LR in the wide part and use concatenation of embedding vectors as the input of MLP in the deep part. Compared with FNN,the embedding vector of FM and input to MLP are same. And they do not need a FM pretrained vector to initialiaze,they are learned end2end. [**DeepFM Model API**](./deepctr.models.deepfm.html) [**DeepFM Estimator API**](./deepctr.estimator.models.deepfm.html) [Guo H, Tang R, Ye Y, et al. Deepfm: a factorization-machine based neural network for ctr prediction[J]. arXiv preprint arXiv:1703.04247, 2017.](http://www.ijcai.org/proceedings/2017/0239.pdf) ### MLR(Mixed Logistic Regression/Piece-wise Linear Model) MLR can be viewed as a combination of $2m$ LR model, $m$ is the piece(region) number. $m$ LR model learns the weight that the sample belong to each region,another m LR model learn sample's click probability in the region. Finally,the sample's CTR is a weighted sum of each region's click probability.Notice the weight is normalized weight. [**MLR Model API**](./deepctr.models.mlr.html) [Gai K, Zhu X, Li H, et al. Learning Piece-wise Linear Models from Large Scale Data for Ad Click Prediction[J]. arXiv preprint arXiv:1704.05194, 2017.](http://arxiv.org/abs/1704.05194) ### NFM (Neural Factorization Machine) NFM use a bi-interaction pooling layer to learn feature interaction between embedding vectors and compress the result into a singe vector which has the same size as a single embedding vector. And then fed it into a MLP.The output logit of MLP and the output logit of linear part are added to get the prediction probability. [**NFM Model API**](./deepctr.models.nfm.html) [**NFM Estimator API**](./deepctr.estimator.models.nfm.html) [He X, Chua T S. Neural factorization machines for sparse predictive analytics[C]//Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval. ACM, 2017: 355-364.](https://arxiv.org/pdf/1708.05027.pdf) ### AFM (Attentional Factorization Machine) AFM is a variant of FM,tradional FM sums the inner product of embedding vector uniformly. AFM can be seen as weighted sum of feature interactions.The weight is learned by a small MLP. [**AFM Model API**](./deepctr.models.afm.html) [**AFM Estimator API**](./deepctr.estimator.models.afm.html) [Xiao J, Ye H, He X, et al. Attentional factorization machines: Learning the weight of feature interactions via attention networks[J]. arXiv preprint arXiv:1708.04617, 2017.](http://www.ijcai.org/proceedings/2017/435) ### DCN (Deep & Cross Network) DCN use a Cross Net to learn both low and high order feature interaction explicitly,and use a MLP to learn feature interaction implicitly. The output of Cross Net and MLP are concatenated.The concatenated vector are feed into one fully connected layer to get the prediction probability. [**DCN Model API**](./deepctr.models.dcn.html) [**DCN Estimator API**](./deepctr.estimator.models.dcn.html) [Wang R, Fu B, Fu G, et al. Deep & cross network for ad click predictions[C]//Proceedings of the ADKDD'17. ACM, 2017: 12.](https://arxiv.org/abs/1708.05123) ### DCN-Mix (Improved Deep & Cross Network with mix of experts and matrix kernel) DCN-Mix uses a matrix kernel instead of vector kernel in CrossNet compared with DCN,and it uses mixture of experts to learn feature interactions. [**DCN-Mix Model API**](./deepctr.models.dcnmix.html) [Wang R, Shivanna R, Cheng D Z, et al. DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems[J]. arXiv preprint arXiv:2008.13535, 2020. ](https://arxiv.org/abs/2008.13535) ### xDeepFM xDeepFM use a Compressed Interaction Network (CIN) to learn both low and high order feature interaction explicitly,and use a MLP to learn feature interaction implicitly. In each layer of CIN,first compute outer products between $x^k$ and $x_0$ to get a tensor $Z_{k+1}$,then use a 1DConv to learn feature maps $H_{k+1}$ on this tensor. Finally,apply sum pooling on all the feature maps $H_k$ to get one vector.The vector is used to compute the logit that CIN contributes. [**xDeepFM Model API**](./deepctr.models.xdeepfm.html) [**xDeepFM Estimator API**](./deepctr.estimator.models.xdeepfn.html) [Lian J, Zhou X, Zhang F, et al. xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems[J]. arXiv preprint arXiv:1803.05170, 2018.](https://arxiv.org/pdf/1803.05170.pdf) ### AutoInt(Automatic Feature Interaction) AutoInt use a interacting layer to model the interactions between different features. Within each interacting layer, each feature is allowed to interact with all the other features and is able to automatically identify relevant features to form meaningful higher-order features via the multi-head attention mechanism. By stacking multiple interacting layers,AutoInt is able to model different orders of feature interactions. [**AutoInt Model API**](./deepctr.models.autoint.html) [**AutoInt Estimator API**](./deepctr.estimator.models.autoint.html) [Song W, Shi C, Xiao Z, et al. Autoint: Automatic feature interaction learning via self-attentive neural networks[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 2019: 1161-1170. ](https://arxiv.org/abs/1810.11921) ### ONN(Operation-aware Neural Networks for User Response Prediction) ONN models second order feature interactions like like FFM and preserves second-order interaction information as much as possible.Further more,deep neural network is used to learn higher-ordered feature interactions. [**ONN Model API**](./deepctr.models.onn.html) [Yang Y, Xu B, Shen F, et al. Operation-aware Neural Networks for User Response Prediction[J]. arXiv preprint arXiv:1904.12579, 2019.](https://arxiv.org/pdf/1904.12579.pdf) ### FGCNN(Feature Generation by Convolutional Neural Network) FGCNN models with two components: Feature Generation and Deep Classifier. Feature Generation leverages the strength of CNN to generate local patterns and recombine them to generate new features. Deep Classifier adopts the structure of IPNN to learn interactions from the augmented feature space. [**FGCNN Model API**](./deepctr.models.fgcnn.html) [Liu B, Tang R, Chen Y, et al. Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction[J]. arXiv preprint arXiv:1904.04447, 2019.](https://arxiv.org/pdf/1904.04447) ### FiBiNET(Feature Importance and Bilinear feature Interaction NETwork) Feature Importance and Bilinear feature Interaction NETwork is proposed to dynamically learn the feature importance and fine-grained feature interactions. On the one hand, the FiBiNET can dynamically learn the importance of fea- tures via the Squeeze-Excitation network (SENET) mechanism; on the other hand, it is able to effectively learn the feature interactions via bilinear function. [**FiBiNET Model API**](./deepctr.models.fibinet.html) [**FiBiNET Estimator API**](./deepctr.estimator.models.fibinet.html) [Huang T, Zhang Z, Zhang J. FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Prediction[J]. arXiv preprint arXiv:1905.09433, 2019.](https://arxiv.org/pdf/1905.09433.pdf) ### FLEN(Field-Leveraged Embedding Network) A large-scale CTR prediction model with efficient usage of field information to alleviate gradient coupling problem. [**FLEN Model API**](./deepctr.models.flen.html) [FLEN example](https://github.com/shenweichen/DeepCTR/tree/master/examples/run_flen.py) [Chen W, Zhan L, Ci Y, Lin C. FLEN: Leveraging Field for Scalable CTR Prediction[J]. arXiv preprint arXiv:1911.04690, 2019.](https://arxiv.org/pdf/1911.04690.pdf) ### IFM(Input-aware Factorization Machine) IFM improves FMs by explicitly considering the impact of each individual input upon the representation of features, which learns a unique input-aware factor for the same feature in different instances via a neural network. [**IFM Model API**](./deepctr.models.ifm.html) [Yu Y, Wang Z, Yuan B. An Input-aware Factorization Machine for Sparse Prediction[C]//IJCAI. 2019: 1466-1472.](https://www.ijcai.org/Proceedings/2019/0203.pdf) ### DIFM(Dual Input-aware Factorization Machine) Dual Input-aware Factorization Machines (DIFMs) can adaptively reweight the original feature representations at the bit-wise and vector-wise levels simultaneously. [**DIFM Model API**](./deepctr.models.difm.html) [Lu W, Yu Y, Chang Y, et al. A Dual Input-aware Factorization Machine for CTR Prediction[C]//IJCAI. 2020: 3139-3145.](https://www.ijcai.org/Proceedings/2020/0434.pdf) ### DeepFEFM(Deep Field-Embedded Factorization Machine) FEFM learns symmetric matrix embeddings for each field pair along with the usual single vector embeddings for each feature. FEFM has significantly lower model complexity than FFM and roughly the same complexity as FwFM. [**DeepFEFM Model API**](./deepctr.models.deepfefm.html) [Pande H. Field-Embedded Factorization Machines for Click-through rate prediction[J]. arXiv preprint arXiv:2009.09931, 2020.](https://arxiv.org/pdf/2009.09931) ### EDCN(Enhancing Explicit and Implicit Feature Interactions DCN) EDCN introduces two advanced modules, namelybridge moduleandregulation module, which work collaboratively tocapture the layer-wise interactive signals and learn discriminativefeature distributions for each hidden layer of the parallel networks. [**EDCN Model API**](./deepctr.models.edcn.html) [Chen B, Wang Y, Liu Z, et al. Enhancing explicit and implicit feature interactions via information sharing for parallel deep ctr models[C]//Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 2021: 3757-3766.](https://dlp-kdd.github.io/assets/pdf/DLP-KDD_2021_paper_12.pdf) ## Sequence Models ### DIN (Deep Interest Network) DIN introduce a attention method to learn from sequence(multi-valued) feature. Tradional method usually use sum/mean pooling on sequence feature. DIN use a local activation unit to get the activation score between candidate item and history items. User's interest are represented by weighted sum of user behaviors. user's interest vector and other embedding vectors are concatenated and fed into a MLP to get the prediction. [**DIN Model API**](./deepctr.models.sequence.din.html) [DIN example](https://github.com/shenweichen/DeepCTR/tree/master/examples/run_din.py) [Zhou G, Zhu X, Song C, et al. Deep interest network for click-through rate prediction[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 2018: 1059-1068.](https://arxiv.org/pdf/1706.06978.pdf) ### DIEN (Deep Interest Evolution Network) Deep Interest Evolution Network (DIEN) uses interest extractor layer to capture temporal interests from history behavior sequence. At this layer, an auxiliary loss is proposed to supervise interest extracting at each step. As user interests are diverse, especially in the e-commerce system, interest evolving layer is proposed to capture interest evolving process that is relative to the target item. At interest evolving layer, attention mechanism is embedded into the sequential structure novelly, and the effects of relative interests are strengthened during interest evolution. [**DIEN Model API**](./deepctr.models.sequence.dien.html) [DIEN example](https://github.com/shenweichen/DeepCTR/tree/master/examples/run_dien.py) [Zhou G, Mou N, Fan Y, et al. Deep Interest Evolution Network for Click-Through Rate Prediction[J]. arXiv preprint arXiv:1809.03672, 2018.](https://arxiv.org/pdf/1809.03672.pdf) ### DSIN(Deep Session Interest Network) Deep Session Interest Network (DSIN) extracts users' multiple historical sessions in their behavior sequences. First it uses self-attention mechanism with bias encoding to extract users' interests in each session. Then apply Bi-LSTM to model how users' interests evolve and interact among sessions. Finally, local activation unit is used to adaptively learn the influences of various session interests on the target item. [**DSIN Model API**](./deepctr.models.sequence.dsin.html) [DSIN example](https://github.com/shenweichen/DeepCTR/tree/master/examples/run_dsin.py) [Feng Y, Lv F, Shen W, et al. Deep Session Interest Network for Click-Through Rate Prediction[J]. arXiv preprint arXiv:1905.06482, 2019.](https://arxiv.org/abs/1905.06482) ### BST(Behavior Sequence Transformer) BST use the powerful Transformer model to capture the sequential signals underlying users’ behavior sequences . [**BST Model API**](./deepctr.models.sequence.bst.html) [BST example](https://github.com/shenweichen/DeepCTR/tree/master/examples/run_din.py) [Qiwei Chen, Huan Zhao, Wei Li, Pipei Huang, and Wenwu Ou. 2019. Behavior sequence transformer for e-commerce recommendation in Alibaba. In Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data (DLP-KDD '19). Association for Computing Machinery, New York, NY, USA, Article 12, 1–4. DOI:)](https://arxiv.org/pdf/1905.06874.pdf) ## MultiTask Models ### SharedBottom Hard parameter sharing is the most commonly used approach to MTL in neural networks. It is generally applied by sharing the hidden layers between all tasks, while keeping several task-specific output layers. [**SharedBottom Model API**](./deepctr.models.multitask.sharedbottom.html) [Ruder S. An overview of multi-task learning in deep neural networks[J]. arXiv preprint arXiv:1706.05098, 2017.](https://arxiv.org/pdf/1706.05098.pdf) ### ESMM(Entire Space Multi-task Model) ESMM models CVR in a brand-new perspective by making good use of sequential pattern of user actions, i.e., impression → click → conversion. The proposed Entire Space Multi-task Model (ESMM) can eliminate the two problems simultaneously by i) modeling CVR directly over the entire space, ii) employing a feature representation transfer learning strategy. [**ESMM Model API**](./deepctr.models.multitask.esmm.html) [Ma X, Zhao L, Huang G, et al. Entire space multi-task model: An effective approach for estimating post-click conversion rate[C]//The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 2018.](https://arxiv.org/abs/1804.07931) ### MMOE(Multi-gate Mixture-of-Experts) Multi-gate Mixture-of-Experts (MMoE) explicitly learns to model task relationships from data. We adapt the Mixture-of- Experts (MoE) structure to multi-task learning by sharing the expert submodels across all tasks, while also having a gating network trained to optimize each task. [**MMOE Model API**](./deepctr.models.multitask.mmoe.html) [Ma J, Zhao Z, Yi X, et al. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018.](https://dl.acm.org/doi/abs/10.1145/3219819.3220007) ### PLE(Progressive Layered Extraction) PLE separates shared components and task-specific components explicitly and adopts a progressive rout- ing mechanism to extract and separate deeper semantic knowledge gradually, improving efficiency of joint representation learning and information routing across tasks in a general setup. [**PLE Model API**](./deepctr.models.multitask.ple.html) [Tang H, Liu J, Zhao M, et al. Progressive layered extraction (ple): A novel multi-task learning (mtl) model for personalized recommendations[C]//Fourteenth ACM Conference on Recommender Systems. 2020.](https://dl.acm.org/doi/10.1145/3383313.3412236) ## Layers The models of deepctr are modular, so you can use different modules to build your own models. The module is a class that inherits from `tf.keras.layers.Layer`,it has the same attributes and methods as keras Layers like `tf.keras.layers.Dense()` etc You can see layers API in [Layers](./Layers.html) --- ### Source History (`docs/source/History.md`) # History - 04/16/2026 : [v0.9.4](https://github.com/shenweichen/DeepCTR/releases/tag/v0.9.4) released.Support higher tensorflow version. - 11/10/2022 : [v0.9.3](https://github.com/shenweichen/DeepCTR/releases/tag/v0.9.3) released.Add [EDCN](./Features.html#edcn-enhancing-explicit-and-implicit-feature-interactions-dcn). - 10/15/2022 : [v0.9.2](https://github.com/shenweichen/DeepCTR/releases/tag/v0.9.2) released.Support python `3.9`,`3.10`. - 06/11/2022 : [v0.9.1](https://github.com/shenweichen/DeepCTR/releases/tag/v0.9.1) released.Improve compatibility with tensorflow `2.x`. - 09/03/2021 : [v0.9.0](https://github.com/shenweichen/DeepCTR/releases/tag/v0.9.0) released.Add multitask learning models:[SharedBottom](./Features.html#sharedbottom),[ESMM](./Features.html#esmm-entire-space-multi-task-model),[MMOE](./Features.html#mmoe-multi-gate-mixture-of-experts) and [PLE](./Features.html#ple-progressive-layered-extraction). [running example](./Examples.html#multitask-learning-mmoe) - 07/18/2021 : [v0.8.7](https://github.com/shenweichen/DeepCTR/releases/tag/v0.8.7) released.Support pre-defined key-value vocabulary in `Hash` Layer. [example](./Examples.html#hash-layer-with-pre-defined-key-value-vocabulary) - 06/14/2021 : [v0.8.6](https://github.com/shenweichen/DeepCTR/releases/tag/v0.8.6) released.Add [IFM](./Features.html#ifm-input-aware-factorization-machine) [DIFM](./Features.html#difm-dual-input-aware-factorization-machine), [FEFM and DeepFEFM](./Features.html#deepfefm-deep-field-embedded-factorization-machine) model. - 03/13/2021 : [v0.8.5](https://github.com/shenweichen/DeepCTR/releases/tag/v0.8.5) released.Add [BST](./Features.html#bst-behavior-sequence-transformer) model. - 02/12/2021 : [v0.8.4](https://github.com/shenweichen/DeepCTR/releases/tag/v0.8.4) released.Fix bug in DCN-Mix. - 01/06/2021 : [v0.8.3](https://github.com/shenweichen/DeepCTR/releases/tag/v0.8.3) released.Add [DCN-Mix](./Features.html#dcn-mix-improved-deep-cross-network-with-mix-of-experts-and-matrix-kernel) model.Support `transform_fn` in `DenseFeat`. - 10/11/2020 : [v0.8.2](https://github.com/shenweichen/DeepCTR/releases/tag/v0.8.2) released.Refactor `DNN` Layer. - 09/12/2020 : [v0.8.1](https://github.com/shenweichen/DeepCTR/releases/tag/v0.8.1) released.Improve the reproducibility & fix some bugs. - 06/27/2020 : [v0.8.0](https://github.com/shenweichen/DeepCTR/releases/tag/v0.8.0) released. - Support `Tensorflow Estimator` for large scale data and distributed training. [example: Estimator with TFRecord](./Examples.html#estimator-with-tfrecord-classification-criteo) - Support different initializers for different embedding weights and loading pretrained embeddings. [example](./FAQ.html#how-to-use-pretrained-weights-to-initialize-embedding-weights-and-frozen-embedding-weights) - Add new model `FwFM`. - 05/17/2020 : [v0.7.5](https://github.com/shenweichen/DeepCTR/releases/tag/v0.7.5) released.Fix numerical instability in `LayerNormalization`. - 03/15/2020 : [v0.7.4](https://github.com/shenweichen/DeepCTR/releases/tag/v0.7.4) released.Add [FLEN](./Features.html#flen-field-leveraged-embedding-network) and `FieldWiseBiInteraction`. - 03/04/2020 : [v0.7.3](https://github.com/shenweichen/DeepCTR/releases/tag/v0.7.3) released.Fix the inconsistency of prediction results when the model is loaded with trained weights. - 02/08/2020 : [v0.7.2](https://github.com/shenweichen/DeepCTR/releases/tag/v0.7.2) released.Fix some bugs. - 01/28/2020 : [v0.7.1](https://github.com/shenweichen/DeepCTR/releases/tag/v0.7.1) released.Simplify [VarLenSparseFeat](./Features.html#varlensparsefeat),support setting weight_normalization.Fix problem of embedding size of `SparseFeat` in `linear_feature_columns`. - 11/24/2019 : [v0.7.0](https://github.com/shenweichen/DeepCTR/releases/tag/v0.7.0) released.Refactor [feature columns](./Features.html#feature-columns).Different features can use different `embedding_dim` and group-wise interaction is available by setting `group_name`. - 11/06/2019 : [v0.6.3](https://github.com/shenweichen/DeepCTR/releases/tag/v0.6.3) released.Add `WeightedSequenceLayer` and support [weighted sequence feature input](./Examples.html#multi-value-input-movielens). - 10/03/2019 : [v0.6.2](https://github.com/shenweichen/DeepCTR/releases/tag/v0.6.2) released.Simplify the input logic. - 09/08/2019 : [v0.6.1](https://github.com/shenweichen/DeepCTR/releases/tag/v0.6.1) released.Fix bugs in `CCPM` and `DynamicGRU`. - 08/02/2019 : [v0.6.0](https://github.com/shenweichen/DeepCTR/releases/tag/v0.6.0) released.Now DeepCTR is compatible with tensorflow `1.14` and `2.0.0`. - 07/21/2019 : [v0.5.2](https://github.com/shenweichen/DeepCTR/releases/tag/v0.5.2) released.Refactor `Linear` Layer. - 07/10/2019 : [v0.5.1](https://github.com/shenweichen/DeepCTR/releases/tag/v0.5.1) released.Add [FiBiNET](./Features.html#fibinet-feature-importance-and-bilinear-feature-interaction-network). - 06/30/2019 : [v0.5.0](https://github.com/shenweichen/DeepCTR/releases/tag/v0.5.0) released.Refactor inputs module. - 05/19/2019 : [v0.4.1](https://github.com/shenweichen/DeepCTR/releases/tag/v0.4.1) released.Add [DSIN](./Features.html#dsin-deep-session-interest-network). - 05/04/2019 : [v0.4.0](https://github.com/shenweichen/DeepCTR/releases/tag/v0.4.0) released.Support [feature hashing on the fly](./Examples.html#classification-criteo-with-feature-hashing-on-the-fly) and python2.7. - 04/27/2019 : [v0.3.4](https://github.com/shenweichen/DeepCTR/releases/tag/v0.3.4) released.Add [FGCNN](./Features.html#fgcnn-feature-generation-by-convolutional-neural-network) and `FGCNNLayer`. - 04/21/2019 : [v0.3.3](https://github.com/shenweichen/DeepCTR/releases/tag/v0.3.3) released.Add [CCPM](./Features.html#ccpm-convolutional-click-prediction-model). - 03/30/2019 : [v0.3.2](https://github.com/shenweichen/DeepCTR/releases/tag/v0.3.2) released.Add [DIEN](./Features.html#dien-deep-interest-evolution-network) and [ONN](./Features.html#onn-operation-aware-neural-networks-for-user-response-prediction) Model. - 02/17/2019 : [v0.3.1](https://github.com/shenweichen/DeepCTR/releases/tag/v0.3.1) released.Refactor layers ,add `BiLSTM` and `Transformer`. - 01/24/2019 : [v0.2.3](https://github.com/shenweichen/DeepCTR/releases/tag/v0.2.3) released.Use a new feature config generation method and fix bugs. - 01/01/2019 : [v0.2.2](https://github.com/shenweichen/DeepCTR/releases/tag/v0.2.2) released.Add [sequence(multi-value) input support](./Examples.html#multi-value-input-movielens) for `AFM,AutoInt,DCN,DeepFM,FNN,NFM,PNN,xDeepFM` models. - 12/27/2018 : [v0.2.1](https://github.com/shenweichen/DeepCTR/releases/tag/v0.2.1) released.Add [AutoInt](./Features.html#autoint-automatic-feature-interaction) Model. - 12/22/2018 : [v0.2.0](https://github.com/shenweichen/DeepCTR/releases/tag/v0.2.0) released.Add [xDeepFM](./Features.html#xdeepfm) and automatic check for new version. - 12/19/2018 : [v0.1.6](https://github.com/shenweichen/DeepCTR/releases/tag/v0.1.6) released.Now DeepCTR is compatible with tensorflow from `1.4-1.12` except for `1.7` and `1.8`. - 11/29/2018 : [v0.1.4](https://github.com/shenweichen/DeepCTR/releases/tag/v0.1.4) released.Add [FAQ](./FAQ.html) in docs - 11/24/2018 : DeepCTR first version v0.1.0 is released on [PyPi](https://pypi.org/project/deepctr/) --- ### Source Layers (`docs/source/Layers.rst`) DeepCTR Layers API ====================== .. toctree:: :maxdepth: 3 :caption: API: Core Layers Interaction Layers Activation Layers Normalization Layers Sequence Layers --- ### Source Model Methods (`docs/source/Model_Methods.md`) # Methods ## compile ```python compile(optimizer, loss=None, metrics=None, loss_weights=None, sample_weight_mode=None, weighted_metrics=None, target_tensors=None) ``` Configures the model for training. **Arguments** - **optimizer**: String (name of optimizer) or optimizer instance. See [optimizers](https://www.tensorflow.org/versions/r1.12/api_docs/python/tf/keras/optimizers/). - **loss**: String (name of objective function) or objective function. See [losses](https://www.tensorflow.org/versions/r1.12/api_docs/python/tf/keras/losses). If the model has multiple outputs, you can use a different loss on each output by passing a dictionary or a list of losses. The loss value that will be minimized by the model will then be the sum of all individual losses. - **metrics**: List of metrics to be evaluated by the model during training and testing. Typically you will use `metrics=['accuracy']`. To specify different metrics for different outputs of a multi-output model, you could also pass a dictionary, such as `metrics={'output_a': 'accuracy'}`. - **loss_weights**: Optional list or dictionary specifying scalar coefficients (Python floats) to weight the loss contributions of different model outputs. The loss value that will be minimized by the model will then be the weighted sum of all individual losses, weighted by the `loss_weights` coefficients. If a list, it is expected to have a 1:1 mapping to the model's outputs. If a tensor, it is expected to map output names (strings) to scalar coefficients. - **sample_weight_mode**: If you need to do timestep-wise sample weighting (2D weights), set this to `"temporal"`. `None` defaults to sample-wise weights (1D). If the model has multiple outputs, you can use a different `sample_weight_mode` on each output by passing a dictionary or a list of modes. - **weighted_metrics**: List of metrics to be evaluated and weighted by sample_weight or class_weight during training and testing. - **target_tensors**: By default, Keras will create placeholders for the model's target, which will be fed with the target data during training. If instead you would like to use your own target tensors (in turn, Keras will not expect external Numpy data for these targets at training time), you can specify them via the `target_tensors` argument. It can be a single tensor (for a single-output model), a list of tensors, or a dict mapping output names to target tensors. **Raises** - **ValueError**: In case of invalid arguments for `optimizer`, `loss`, `metrics` or `sample_weight_mode`. ## fit ```python fit(x=None, y=None, batch_size=None, epochs=1, verbose=1, callbacks=None, validation_split=0.0, validation_data=None, shuffle=True, class_weight=None, sample_weight=None, initial_epoch=0, steps_per_epoch=None, validation_steps=None, validation_freq=1) ``` Trains the model for a given number of epochs (iterations on a dataset). **Arguments** - **x**: Numpy array of training data (if the model has a single input), or list of Numpy arrays (if the model has multiple inputs). If input layers in the model are named, you can also pass a dictionary mapping input names to Numpy arrays. `x` can be `None` (default) if feeding from framework-native tensors (e.g. TensorFlow data tensors). - **y**: Numpy array of target (label) data (if the model has a single output), or list of Numpy arrays (if the model has multiple outputs). If output layers in the model are named, you can also pass a dictionary mapping output names to Numpy arrays. `y` can be `None` (default) if feeding from framework-native tensors (e.g. TensorFlow data tensors). - **batch_size**: Integer or `None`. Number of samples per gradient update. If unspecified, `batch_size` will default to 32. - **epochs**: Integer. Number of epochs to train the model. An epoch is an iteration over the entire `x` and `y` data provided. Note that in conjunction with `initial_epoch`, `epochs` is to be understood as "final epoch". The model is not trained for a number of iterations given by `epochs`, but merely until the epoch of index `epochs` is reached. - **verbose**: Integer. 0, 1, or 2. Verbosity mode. 0 = silent, 1 = progress bar, 2 = one line per epoch. - **callbacks**: List of `tf.keras.callbacks.Callback` instances. List of callbacks to apply during training and validation (if ). See [callbacks](https://www.tensorflow.org/versions/r1.12/api_docs/python/tf/keras/callbacks). - **validation_split**: Float between 0 and 1. Fraction of the training data to be used as validation data. The model will set apart this fraction of the training data, will not train on it, and will evaluate the loss and any model metrics on this data at the end of each epoch. The validation data is selected from the last samples in the `x` and `y` data provided, before shuffling. - **validation_data**: tuple `(x_val, y_val)` or tuple `(x_val, y_val, val_sample_weights)` on which to evaluate the loss and any model metrics at the end of each epoch. The model will not be trained on this data. `validation_data` will override `validation_split`. - **shuffle**: Boolean (whether to shuffle the training data before each epoch) or str (for 'batch'). 'batch' is a special option for dealing with the limitations of HDF5 data; it shuffles in batch-sized chunks. Has no effect when `steps_per_epoch` is not `None`. - **class_weight**: Optional dictionary mapping class indices (integers) to a weight (float) value, used for weighting the loss function (during training only). This can be useful to tell the model to "pay more attention" to samples from an under-represented class. - **sample_weight**: Optional Numpy array of weights for the training samples, used for weighting the loss function (during training only). You can either pass a flat (1D) Numpy array with the same length as the input samples (1:1 mapping between weights and samples), or in the case of temporal data, you can pass a 2D array with shape `(samples, sequence_length)`, to apply a different weight to every timestep of every sample. In this case you should make sure to specify `sample_weight_mode="temporal"` in `compile()`. - **initial_epoch**: Integer. Epoch at which to start training (useful for resuming a previous training run). - **steps_per_epoch**: Integer or `None`. Total number of steps (batches of samples) before declaring one epoch finished and starting the next epoch. When training with input tensors such as TensorFlow data tensors, the default `None` is equal to the number of samples in your dataset divided by the batch size, or 1 if that cannot be determined. validation_steps: Only relevant if `steps_per_epoch` is specified. Total number of steps (batches of samples) to validate before stopping. - **validation_freq**: Only relevant if validation data is provided. Integer or list/tuple/set. If an integer, specifies how many training epochs to run before a new validation run is performed, e.g. `validation_freq=2` runs validation every 2 epochs. If a list, tuple, or set, specifies the epochs on which to run validation, e.g. `validation_freq=[1, 2, 10]` runs validation at the end of the 1st, 2nd, and 10th epochs. **Returns** - A `History` object. Its `History.history` attribute is a record of training loss values and metrics values at successive epochs, as well as validation loss values and validation metrics values (if applicable). **Raises** - **RuntimeError**: If the model was never compiled. ValueError: In case of mismatch between the provided input data and what the model expects. ## evaluate ```python evaluate(x=None, y=None, batch_size=None, verbose=1, sample_weight=None, steps=None, callbacks=None) ``` Returns the loss value & metrics values for the model in test mode. Computation is done in batches. **Arguments** - **x**: Numpy array of test data (if the model has a single input), or list of Numpy arrays (if the model has multiple inputs). If input layers in the model are named, you can also pass a dictionary mapping input names to Numpy arrays. `x` can be `None` (default) if feeding from framework-native tensors (e.g. TensorFlow data tensors). - **y**: Numpy array of target (label) data (if the model has a single output), or list of Numpy arrays (if the model has multiple outputs). If output layers in the model are named, you can also pass a dictionary mapping output names to Numpy arrays. `y` can be `None` (default) if feeding from framework-native tensors (e.g. TensorFlow data tensors). - **batch_size**: Integer or `None`. Number of samples per evaluation step. If unspecified, `batch_size` will default to 32. - **verbose**: 0 or 1. Verbosity mode. 0 = silent, 1 = progress bar. - **sample_weight**: Optional Numpy array of weights for the test samples, used for weighting the loss function. You can either pass a flat (1D) Numpy array with the same length as the input samples (1:1 mapping between weights and samples), or in the case of temporal data, you can pass a 2D array with shape `(samples, sequence_length)`, to apply a different weight to every timestep of every sample. In this case you should make sure to specify `sample_weight_mode="temporal"` in `compile()`. - **steps**: Integer or `None`. Total number of steps (batches of samples) before declaring the evaluation round finished. Ignored with the default value of `None`. - **callbacks**: List of `tf.keras.callbacks.Callback` instances. List of callbacks to apply during evaluation. See [callbacks](https://www.tensorflow.org/versions/r1.12/api_docs/python/tf/keras/callbacks). **Returns** - Scalar test loss (if the model has a single output and no metrics) or list of scalars (if the model has multiple outputs and/or metrics). The attribute `model.metrics_names` will give you the display labels for the scalar outputs. ## predict ```python predict(x, batch_size=None, verbose=0, steps=None, callbacks=None) ``` Generates output predictions for the input samples. Computation is done in batches. **Arguments** - **x**: The input data, as a Numpy array (or list of Numpy arrays if the model has multiple inputs). batch_size: Integer. If unspecified, it will default to 32. - **verbose**: Verbosity mode, 0 or 1. - **steps**: Total number of steps (batches of samples) before declaring the prediction round finished. Ignored with the default value of None. - **callbacks**: List of `tf.keras.callbacks.Callback` instances. List of callbacks to apply during prediction. See [callbacks](https://www.tensorflow.org/versions/r1.12/api_docs/python/tf/keras/callbacks). **Returns** - Numpy array(s) of predictions. **Raises** - **ValueError**: In case of mismatch between the provided input data and the model's expectations, or in case a stateful model receives a number of samples that is not a multiple of the batch size. ## train_on_batch ```python train_on_batch(x, y, sample_weight=None, class_weight=None) ``` Runs a single gradient update on a single batch of data. **Arguments** - **x**: Numpy array of training data, or list of Numpy arrays if the model has multiple inputs. If all inputs in the model are named, you can also pass a dictionary mapping input names to Numpy arrays. - **y**: Numpy array of target data, or list of Numpy arrays if the model has multiple outputs. If all outputs in the model are named, you can also pass a dictionary mapping output names to Numpy arrays. - **sample_weight**: Optional array of the same length as x, containing weights to apply to the model's loss for each sample. In the case of temporal data, you can pass a 2D array with shape (samples, sequence_length), to apply a different weight to every timestep of every sample. In this case you should make sure to specify sample_weight_mode="temporal" in compile(). - **class_weight**: Optional dictionary mapping class indices (integers) to a weight (float) to apply to the model's loss for the samples from this class during training. This can be useful to tell the model to "pay more attention" to samples from an under-represented class. **Returns** - Scalar training loss (if the model has a single output and no metrics) or list of scalars (if the model has multiple outputs and/or metrics). The attribute `model.metrics_names` will give you the display labels for the scalar outputs. ## test_on_batch ```python test_on_batch(x, y, sample_weight=None) ``` Test the model on a single batch of samples. **Arguments** - **x**: Numpy array of test data, or list of Numpy arrays if the model has multiple inputs. If all inputs in the model are named, you can also pass a dictionary mapping input names to Numpy arrays. - **y**: Numpy array of target data, or list of Numpy arrays if the model has multiple outputs. If all outputs in the model are named, you can also pass a dictionary mapping output names to Numpy arrays. - **sample_weight**: Optional array of the same length as x, containing weights to apply to the model's loss for each sample. In the case of temporal data, you can pass a 2D array with shape (samples, sequence_length), to apply a different weight to every timestep of every sample. In this case you should make sure to specify `sample_weight_mode="temporal"` in `compile()`. **Returns** - Scalar test loss (if the model has a single output and no metrics) or list of scalars (if the model has multiple outputs and/or metrics). The attribute `model.metrics_names` will give you the display labels for the scalar outputs. ## predict_on_batch ```python predict_on_batch(x) ``` Returns predictions for a single batch of samples. **Arguments** - **x**: Input samples, as a Numpy array. **Returns** - Numpy array(s) of predictions. ## fit_generator ```python fit_generator(generator, steps_per_epoch=None, epochs=1, verbose=1, callbacks=None, validation_data=None, validation_steps=None, validation_freq=1, class_weight=None, max_queue_size=10, workers=1, use_multiprocessing=False, shuffle=True, initial_epoch=0) ``` Trains the model on data generated batch-by-batch by a Python generator (or an instance of `Sequence`). The generator is run in parallel to the model, for efficiency. For instance, this allows you to do real-time data augmentation on images on CPU in parallel to training your model on GPU. The use of `tf.keras.utils.Sequence` guarantees the ordering and guarantees the single use of every input per epoch when using `use_multiprocessing=True`. **Arguments** - **generator**: A generator or an instance of `Sequence` (`tf.keras.utils.Sequence`) object in order to avoid duplicate data when using multiprocessing. The output of the generator must be either a tuple `(inputs, targets)` or a tuple `(inputs, targets, sample_weights)`. This tuple (a single output of the generator) makes a single batch. Therefore, all arrays in this tuple must have the same length (equal to the size of this batch). Different batches may have different sizes. For example, the last batch of the epoch is commonly smaller than the others, if the size of the dataset is not divisible by the batch size. The generator is expected to loop over its data indefinitely. An epoch finishes when `steps_per_epoch` batches have been seen by the model. - **steps_per_epoch**: Integer. Total number of steps (batches of samples) to yield from `generator` before declaring one epoch finished and starting the next epoch. It should typically be equal to `ceil(num_samples / batch_size)` Optional for `Sequence`: if unspecified, will use the `len(generator)` as a number of steps. - **epochs**: Integer. Number of epochs to train the model. An epoch is an iteration over the entire data provided, as defined by `steps_per_epoch`. Note that in conjunction with `initial_epoch`, `epochs` is to be understood as "final epoch". The model is not trained for a number of iterations given by `epochs`, but merely until the epoch of index `epochs` is reached. - **verbose**: Integer. 0, 1, or 2. Verbosity mode. 0 = silent, 1 = progress bar, 2 = one line per epoch. - **callbacks**: List of `tf.keras.callbacks.Callback` instances. List of callbacks to apply during training. See [callbacks](https://www.tensorflow.org/versions/r1.12/api_docs/python/tf/keras/callbacks). - **validation_data**: This can be either a generator or a `Sequence` object for the validation data tuple `(x_val, y_val)` tuple `(x_val, y_val, val_sample_weights)` on which to evaluate the loss and any model metrics at the end of each epoch. The model will not be trained on this data. - **validation_steps**: Only relevant if `validation_data` is a generator. Total number of steps (batches of samples) to yield from `validation_data` generator before stopping at the end of every epoch. It should typically be equal to the number of samples of your validation dataset divided by the batch size. Optional for `Sequence`: if unspecified, will use the `len(validation_data)` as a number of steps. - **validation_freq**: Only relevant if validation data is provided. Integer or `collections.Container` instance (e.g. list, tuple, etc.). If an integer, specifies how many training epochs to run before a new validation run is performed, e.g. `validation_freq=2` runs validation every 2 epochs. If a Container, specifies the epochs on which to run validation, e.g. `validation_freq=[1, 2, 10]` runs validation at the end of the 1st, 2nd, and 10th epochs. - **class_weight**: Optional dictionary mapping class indices (integers) to a weight (float) value, used for weighting the loss function (during training only). This can be useful to tell the model to "pay more attention" to samples from an under-represented class. - **max_queue_size**: Integer. Maximum size for the generator queue. If unspecified, `max_queue_size` will default to 10. - **workers**: Integer. Maximum number of processes to spin up when using process-based threading. If unspecified, `workers` will default to 1. If 0, will execute the generator on the main thread. - **use_multiprocessing**: Boolean. If `True`, use process-based threading. If unspecified, `use_multiprocessing` will default to `False`. Note that because this implementation relies on multiprocessing, you should not pass non-picklable arguments to the generator as they can't be passed easily to children processes. - **shuffle**: Boolean. Whether to shuffle the order of the batches at the beginning of each epoch. Only used with instances of `Sequence` (`tf.keras.utils.Sequence`). Has no effect when `steps_per_epoch` is not `None`. initial_epoch: Integer. Epoch at which to start training (useful for resuming a previous training run). **Returns** - A `History` object. Its `History.history` attribute is a record of training loss values and metrics values at successive epochs, as well as validation loss values and validation metrics values (if applicable). **Raises** - **ValueError**: In case the generator yields data in an invalid format. **Example** ```python def generate_arrays_from_file(path): while True: with open(path) as f: for line in f: # create numpy arrays of input data # and labels, from each line in the file x1, x2, y = process_line(line) yield ({'input_1': x1, 'input_2': x2}, {'output': y}) model.fit_generator(generate_arrays_from_file('/my_file.txt'), steps_per_epoch=10000, epochs=10) ``` ## evaluate_generator ```python evaluate_generator(generator, steps=None, callbacks=None, max_queue_size=10, workers=1, use_multiprocessing=False, verbose=0) ``` Evaluates the model on a data generator. The generator should return the same kind of data as accepted by `test_on_batch`. **Arguments** - **generator**: Generator yielding tuples (inputs, targets) or (inputs, targets, sample_weights) or an instance of Sequence (tf.keras.utils.Sequence) object in order to avoid duplicate data when using multiprocessing. - **steps**: Total number of steps (batches of samples) to yield from `generator` before stopping. Optional for `Sequence`: if unspecified, will use the `len(generator)` as a number of steps. - **callbacks**: List of `tf.keras.callbacks.Callback` instances. List of callbacks to apply during training. See [callbacks](https://www.tensorflow.org/versions/r1.12/api_docs/python/tf/keras/callbacks). - **max_queue_size**: maximum size for the generator queue - **workers**: Integer. Maximum number of processes to spin up when using process based threading. If unspecified, `workers` will default to 1. If 0, will execute the generator on the main thread. - **use_multiprocessing**: if True, use process based threading. Note that because this implementation relies on multiprocessing, you should not pass non picklable arguments to the generator as they can't be passed easily to children processes. - **verbose**: verbosity mode, 0 or 1. **Returns** - Scalar test loss (if the model has a single output and no metrics) or list of scalars (if the model has multiple outputs and/or metrics). The attribute `model.metrics_names` will give you the display labels for the scalar outputs. **Raises** - **ValueError**: In case the generator yields data in an invalid format. ## predict_generator ```python predict_generator(generator, steps=None, callbacks=None, max_queue_size=10, workers=1, use_multiprocessing=False, verbose=0) ``` Generates predictions for the input samples from a data generator. The generator should return the same kind of data as accepted by `predict_on_batch`. **Arguments** - **generator**: Generator yielding batches of input samples or an instance of Sequence (`tf.keras.utils.Sequence`) object in order to avoid duplicate data when using multiprocessing. - **steps**: Total number of steps (batches of samples) to yield from `generator` before stopping. Optional for `Sequence`: if unspecified, will use the `len(generator)` as a number of steps. - **callbacks**: List of `tf.keras.callbacks.Callback` instances. List of callbacks to apply during training. See [callbacks](https://www.tensorflow.org/versions/r1.12/api_docs/python/tf/keras/callbacks). - **max_queue_size**: Maximum size for the generator queue. - **workers**: Integer. Maximum number of processes to spin up when using process based threading. If unspecified, `workers` will default to 1. If 0, will execute the generator on the main thread. - **use_multiprocessing**: If `True`, use process based threading. Note that because this implementation relies on multiprocessing, you should not pass non picklable arguments to the generator as they can't be passed easily to children processes. - **verbose**: verbosity mode, 0 or 1. **Returns** - Numpy array(s) of predictions. **Raises** - **ValueError**: In case the generator yields data in an invalid format. ## get_layer ```python get_layer(name=None, index=None) ``` Retrieves a layer based on either its name (unique) or index. If `name` and `index` are both provided, `index` will take precedence. Indices are based on order of horizontal graph traversal (bottom-up). **Arguments** - **name**: String, name of layer. - **index**: Integer, index of layer. **Returns** - A layer instance. **Raises** - **ValueError**: In case of invalid layer name or index. --- ### Source Models (`docs/source/Models.rst`) DeepCTR Models API ====================== .. toctree:: Model Methods CCPM FNN PNN WDL DeepFM MLR NFM AFM DCN DCNMix DIN DIEN DSIN BST xDeepFM AutoInt ONN FGCNN FiBiNET FLEN IFM DIFM DeepFEFM SharedBottom ESMM MMOE PLE EDCN --- ### Source Quick Start (`docs/source/Quick-Start.md`) # Quick-Start [](https://dsw-dev.data.aliyun.com/#/?fileUrl=https://pai-public-data.oss-cn-beijing.aliyuncs.com/deep-ctr/Getting-started-4-steps-to-DeepCTR.ipynb&fileName=Getting-started-4-steps-to-DeepCTR.ipynb) ## Installation Guide Now `deepctr` supports Python `>=3.7` and is tested with TensorFlow `1.15` and TensorFlow `2.x`. DeepCTR does not pin or install TensorFlow for you. Install a TensorFlow build that matches your Python, NumPy, CPU/GPU, and operating system first, then install DeepCTR: ```bash $ pip install tensorflow $ pip install deepctr ``` For GPU environments, install the TensorFlow package recommended for your CUDA, cuDNN, and platform combination, then install `deepctr`. For Python `>=3.9`, DeepCTR allows modern `h5py` releases with `h5py>=3.7.0`. If TensorFlow reports a NumPy conflict, follow the TensorFlow requirement for your selected TensorFlow release, for example using `numpy<2` when required by TensorFlow. Use public `tensorflow.keras` APIs in your own code. Avoid mixing `tensorflow.python.keras` with `tensorflow.keras`, because `tensorflow.python.*` is private TensorFlow API and can break model serialization or optimizer/metric loading across TensorFlow versions. ### Install from source ```bash $ git clone https://github.com/shenweichen/DeepCTR.git $ cd DeepCTR $ pip install . ``` ## Getting started: 4 steps to DeepCTR ### Step 1: Import model ```python import pandas as pd from sklearn.preprocessing import LabelEncoder, MinMaxScaler from sklearn.model_selection import train_test_split from deepctr.models import DeepFM from deepctr.feature_column import SparseFeat, DenseFeat,get_feature_names data = pd.read_csv('./criteo_sample.txt') sparse_features = ['C' + str(i) for i in range(1, 27)] dense_features = ['I'+str(i) for i in range(1, 14)] data[sparse_features] = data[sparse_features].fillna('-1', ) data[dense_features] = data[dense_features].fillna(0,) target = ['label'] ``` ### Step 2: Simple preprocessing Usually we have two methods to encode the sparse categorical feature for embedding - Label Encoding: map the features to integer value from 0 ~ len(#unique) - 1 ```python for feat in sparse_features: lbe = LabelEncoder() data[feat] = lbe.fit_transform(data[feat]) ``` - Hash Encoding: map the features to a fix range,like 0 ~ 9999.We have 2 methods to do that: - Do feature hashing before training ```python for feat in sparse_features: lbe = HashEncoder() data[feat] = lbe.transform(data[feat]) ``` - Do feature hashing on the fly in training process We can do feature hashing by setting `use_hash=True` in `SparseFeat` or `VarlenSparseFeat` in Step3. And for dense numerical features,they are usually discretized to buckets,here we use normalization. ```python mms = MinMaxScaler(feature_range=(0,1)) data[dense_features] = mms.fit_transform(data[dense_features]) ``` ### Step 3: Generate feature columns For sparse features, we transform them into dense vectors by embedding techniques. For dense numerical features, we concatenate them to the input tensors of fully connected layer. And for varlen(multi-valued) sparse features,you can use [VarlenSparseFeat](./Features.html#varlensparsefeat). Visit [examples](./Examples.html#multi-value-input-movielens) of using `VarlenSparseFeat` and the Sequence Feature Cookbook for sequence model input conventions. - Label Encoding ```python fixlen_feature_columns = [SparseFeat(feat, vocabulary_size=data[feat].max() + 1,embedding_dim=4) for i,feat in enumerate(sparse_features)] + [DenseFeat(feat, 1,) for feat in dense_features] ``` - Feature Hashing on the fly ```python fixlen_feature_columns = [SparseFeat(feat, vocabulary_size=1e6,embedding_dim=4, use_hash=True, dtype='string') # the input is string for feat in sparse_features] + [DenseFeat(feat, 1, ) for feat in dense_features] ``` - generate feature columns ```python dnn_feature_columns = fixlen_feature_columns linear_feature_columns = fixlen_feature_columns feature_names = get_feature_names(linear_feature_columns + dnn_feature_columns) ``` ### Step 4: Generate the training samples and train the model ```python train, test = train_test_split(data, test_size=0.2) train_model_input = {name:train[name].values for name in feature_names} test_model_input = {name:test[name].values for name in feature_names} model = DeepFM(linear_feature_columns,dnn_feature_columns,task='binary') model.compile("adam", "binary_crossentropy", metrics=['binary_crossentropy'], ) history = model.fit(train_model_input, train[target].values, batch_size=256, epochs=10, verbose=2, validation_split=0.2, ) pred_ans = model.predict(test_model_input, batch_size=256) ``` You can check the full code [here](./Examples.html#classification-criteo). You also can run a distributed training job with the keras model on Kubernetes using [ElasticDL](https://github.com/sql-machine-learning/elasticdl/blob/develop/docs/tutorials/elasticdl_deepctr_keras.md). ## Getting started: 4 steps to DeepCTR Estimator with TFRecord ### Step 1: Import model ```python import tensorflow as tf from deepctr.estimator.inputs import input_fn_tfrecord from deepctr.estimator.models import DeepFMEstimator ``` ### Step 2: Generate feature columns for linear part and dnn part ```python sparse_features = ['C' + str(i) for i in range(1, 27)] dense_features = ['I' + str(i) for i in range(1, 14)] dnn_feature_columns = [] linear_feature_columns = [] for i, feat in enumerate(sparse_features): dnn_feature_columns.append(tf.feature_column.embedding_column( tf.feature_column.categorical_column_with_identity(feat, 1000), 4)) linear_feature_columns.append(tf.feature_column.categorical_column_with_identity(feat, 1000)) for feat in dense_features: dnn_feature_columns.append(tf.feature_column.numeric_column(feat)) linear_feature_columns.append(tf.feature_column.numeric_column(feat)) ``` ### Step 3: Generate the training samples with TFRecord format ```python feature_description = {k: tf.io.FixedLenFeature(dtype=tf.int64, shape=1) for k in sparse_features} feature_description.update( {k: tf.io.FixedLenFeature(dtype=tf.float32, shape=1) for k in dense_features}) feature_description['label'] = tf.io.FixedLenFeature(dtype=tf.float32, shape=1) train_model_input = input_fn_tfrecord('./criteo_sample.tr.tfrecords', feature_description, 'label', batch_size=256, num_epochs=1, shuffle_factor=10) test_model_input = input_fn_tfrecord('./criteo_sample.te.tfrecords', feature_description, 'label', batch_size=2 ** 14, num_epochs=1, shuffle_factor=0) ``` ### Step 4: Train and evaluate the model ```python model = DeepFMEstimator(linear_feature_columns, dnn_feature_columns, task='binary') model.train(train_model_input) eval_result = model.evaluate(test_model_input) print(eval_result) ``` You can check the full code [here](./Examples.html#estimator-with-tfrecord-classification-criteo). You also can run a distributed training job with the estimator model on Kubernetes using [ElasticDL](https://github.com/sql-machine-learning/elasticdl/blob/develop/docs/tutorials/elasticdl_deepctr_estimator.md). --- ### Source Sequence Cookbook (`docs/source/Sequence-Cookbook.md`) # Sequence Feature Cookbook This cookbook collects the input conventions for multi-value features and sequence models such as DIN, BST, DIEN, and DSIN. ## One Row Means One Prediction Target DeepCTR models expect each training sample to be one row. The row contains: - the candidate feature to score, such as `item_id` or `cate_id` - user/context features, such as `user`, `gender`, or `pay_score` - optional history or session features that describe the user's past behavior - one label for this candidate row For example, if the same user has two candidate interactions, build two rows. The history fields can be repeated or updated per row. ```text user item_id cate_id hist_item_id hist_cate_id label 0 1 1 [1, 2, 3, 0] [1, 2, 2, 0] 1 0 2 2 [1, 2, 3, 1] [1, 2, 2, 1] 0 ``` History fields are context for the row. The label belongs to the candidate row, not to every element inside the history. ## Feature Column Shapes Use `SparseFeat` for one categorical id per row. ```python SparseFeat("item_id", vocabulary_size=item_count + 1, embedding_dim=8) ``` Its input shape is usually `(batch_size,)` or `(batch_size, 1)`. Use `DenseFeat` for numerical values or dense vectors. ```python DenseFeat("pay_score", 1) DenseFeat("article_vector", 128) ``` Use `VarLenSparseFeat` for a list of categorical ids per row. ```python VarLenSparseFeat( SparseFeat("genres", vocabulary_size=genre_count + 1, embedding_dim=4), maxlen=max_genre_len, combiner="mean", ) ``` Its input shape is `(batch_size, maxlen)`. Values must be padded to the same `maxlen` before being passed to the model. ## Padding and Length For `VarLenSparseFeat`, value `0` is the default padding value. Do not use `0` as a valid category id when `length_name` is not set. ```python genres = np.array([ [1, 3, 0, 0], [2, 5, 8, 0], ]) ``` If you pass `length_name`, DeepCTR uses that length feature to build the pooling mask. ```python VarLenSparseFeat( SparseFeat("hist_item_id", vocabulary_size=item_count + 1, embedding_dim=8), maxlen=4, length_name="seq_length", ) ``` Then add `seq_length` to the model input: ```python model_input = { "hist_item_id": hist_item_id, "seq_length": np.array([3, 2, 4]), } ``` Padding is still useful because tensors in one batch need the same shape. ## Multi-Value Feature or Multi-Hot Vector `VarLenSparseFeat` stores categorical ids and then applies embedding lookup plus pooling. `maxlen` is the maximum number of values in one sample. It is not the vocabulary size. For a `genres` feature with 18 possible genres and at most 5 genres per movie: ```python VarLenSparseFeat( SparseFeat("genres", vocabulary_size=18 + 1, embedding_dim=4), maxlen=5, combiner="mean", ) ``` A multi-hot vector is also possible, but then it is a dense vector with length equal to the vocabulary size: ```python DenseFeat("genres_multihot", 18) ``` The two representations are different. `VarLenSparseFeat` learns an embedding for every genre id and pools only the present genres. A multi-hot vector feeds the raw indicator vector into the dense part of the model. For large vocabularies, `VarLenSparseFeat` is usually more memory efficient and easier to share with other categorical fields. ## Multiple VarLenSparseFeat Fields Add multiple variable-length fields to the same feature column list. Each field can have its own vocabulary and `maxlen`. ```python feature_columns = [ SparseFeat("user", user_count + 1, embedding_dim=8), SparseFeat("item_id", item_count + 1, embedding_dim=8), VarLenSparseFeat( SparseFeat("genres", genre_count + 1, embedding_dim=4), maxlen=max_genre_len, combiner="mean", ), VarLenSparseFeat( SparseFeat("tags", tag_count + 1, embedding_dim=4), maxlen=max_tag_len, combiner="mean", ), ] ``` The model input must include one padded array for each field: ```python model_input = { "user": user, "item_id": item_id, "genres": genres, "tags": tags, } ``` ## Sharing Embeddings Use the same `embedding_name` when two features represent ids from the same dictionary and should share one embedding table. ```python feature_columns = [ SparseFeat("item_id", item_count + 1, embedding_dim=8), VarLenSparseFeat( SparseFeat( "hist_item_id", item_count + 1, embedding_dim=8, embedding_name="item_id", ), maxlen=4, ), ] ``` Use different `embedding_name` values when the fields are semantically different, even if they have the same value range. ## DIN and BST History Feature Names DIN and BST use `history_feature_list` to find the history sequence that should be matched with the current candidate feature. ```python behavior_feature_list = ["item_id", "cate_id"] ``` For every name in `behavior_feature_list`, the history sequence feature must be named with the `hist_` prefix: ```python SparseFeat("item_id", item_count + 1, embedding_dim=8) SparseFeat("cate_id", cate_count + 1, embedding_dim=4) VarLenSparseFeat( SparseFeat("hist_item_id", item_count + 1, embedding_dim=8, embedding_name="item_id"), maxlen=4, length_name="seq_length", ) VarLenSparseFeat( SparseFeat("hist_cate_id", cate_count + 1, embedding_dim=4, embedding_name="cate_id"), maxlen=4, length_name="seq_length", ) ``` If `behavior_feature_list = ["item"]`, the expected history feature name is `hist_item`. If `behavior_feature_list = ["item_id"]`, the expected history feature name is `hist_item_id`. Inside DIN: - `varlen_sparse_feature_columns` means all `VarLenSparseFeat` fields passed to the model. - `history_feature_columns` means the `VarLenSparseFeat` fields whose names match `hist_` + a behavior feature name. These fields are used as keys in attention. - `sparse_varlen_feature_columns` means other `VarLenSparseFeat` fields. They are pooled and appended to the DNN input, but they are not used as the DIN attention history. For example, `genres` is a non-history multi-value field: ```python VarLenSparseFeat( SparseFeat("genres", genre_count + 1, embedding_dim=4), maxlen=max_genre_len, combiner="mean", ) ``` It belongs to `sparse_varlen_feature_columns`, not to `history_feature_columns`. ## DIN Does Not Support VarLen of VarLen DIN expects the candidate behavior feature to be one sparse id per row, and the history behavior feature to be one padded sequence per row. Supported: ```text item_id: 3 hist_item_id: [1, 2, 4, 0] ``` Not supported directly: ```text item_categories: [1, 2, 0] hist_item_categories: [[1, 2, 0], [3, 4, 0], [5, 0, 0]] ``` The second case is a "sequence of multi-value behaviors", or a 3D tensor. DeepCTR's current `DIN` implementation does not provide a `VarLenSparseFeat` nested inside another `VarLenSparseFeat`. Common workarounds are: - choose one representative category for the candidate and each history item - map a category set to one categorical id before training - pre-pool the category set outside the model and build a custom DIN variant - customize the model to handle a 3D behavior tensor ## Dense History Features in DIN Current DIN attention is built from sparse embedding sequences. Dense features such as `pay_score` are appended to the DNN input, but they are not part of the attention keys by default. If you need dense history values inside attention, common choices are: - discretize the dense value into buckets and use it as a sparse sequence feature - concatenate or combine dense history representations before passing them to a custom attention layer - build a custom model based on `deepctr.models.sequence.din.DIN` DeepCTR does not provide a `VarLenDenseFeat` class. ## DSIN Input Format DeepCTR's DSIN implementation expects sessions to be prepared before training. It does not split a raw event stream into sessions inside the model. For `behavior_feature_list = ["item", "cate_id"]` and `sess_max_count=2`, prepare fields like: ```python feature_columns = [ SparseFeat("item", item_count + 1, embedding_dim=4), SparseFeat("cate_id", cate_count + 1, embedding_dim=4), VarLenSparseFeat( SparseFeat("sess_0_item", item_count + 1, embedding_dim=4, embedding_name="item"), maxlen=4, ), VarLenSparseFeat( SparseFeat("sess_0_cate_id", cate_count + 1, embedding_dim=4, embedding_name="cate_id"), maxlen=4, ), VarLenSparseFeat( SparseFeat("sess_1_item", item_count + 1, embedding_dim=4, embedding_name="item"), maxlen=4, ), VarLenSparseFeat( SparseFeat("sess_1_cate_id", cate_count + 1, embedding_dim=4, embedding_name="cate_id"), maxlen=4, ), ] ``` The model input should contain one padded sequence per session field and `sess_length`, the number of valid sessions for each row: ```python model_input = { "item": item, "cate_id": cate_id, "sess_0_item": sess_0_item, "sess_0_cate_id": sess_0_cate_id, "sess_1_item": sess_1_item, "sess_1_cate_id": sess_1_cate_id, "sess_length": np.array([2, 1, 0]), } ``` Usually you keep only the most recent `sess_max_count` sessions and the most recent `maxlen` events in each session, then pad the rest with `0`. ## DSIN Labels and Sessions Each row still has one label for the candidate item in that row. ```text user item cate_id sess_0_item sess_1_item sess_length label 0 1 1 [4, 5, 0, 0] [2, 3, 0, 0] 2 1 0 2 2 [4, 5, 1, 0] [2, 3, 0, 0] 2 0 ``` The same user can appear in multiple rows. Sessions are historical context for the candidate row. They do not have their own labels inside the DSIN input. If your raw data contains timestamps or time deltas, use them during preprocessing to split events into sessions. If you want time gaps as model features, bucket them into categorical ids or build a custom model. DeepCTR does not provide `VarLenDenseFeat` for raw variable-length dense time values. ## Estimator and TFRecord Inputs The Keras-style DeepCTR models use `SparseFeat`, `DenseFeat`, and `VarLenSparseFeat`. The Estimator models use TensorFlow `tf.feature_column` objects directly. For TFRecord vector features, make the dtype and feature column type match: - categorical id fields should be integer features and use `categorical_column_with_identity`, `categorical_column_with_hash_bucket`, or another categorical column - dense float vectors should use `numeric_column(..., shape=(dim,))`, not `embedding_column` Example: ```python feature_description = { "article_id": tf.io.FixedLenFeature(shape=(1,), dtype=tf.int64), "article_vector": tf.io.FixedLenFeature(shape=(128,), dtype=tf.float32), "clicked": tf.io.FixedLenFeature(shape=(1,), dtype=tf.float32), } dnn_feature_columns = [ tf.feature_column.embedding_column( tf.feature_column.categorical_column_with_identity("article_id", num_buckets=100000), dimension=8, ), tf.feature_column.numeric_column("article_vector", shape=(128,)), ] linear_feature_columns = [ tf.feature_column.categorical_column_with_identity("article_id", num_buckets=100000), tf.feature_column.numeric_column("article_vector", shape=(128,)), ] ``` For padded integer sequence fields in TFRecord, store the sequence as a fixed-length integer feature: ```python feature_description = { "hist_item_id": tf.io.FixedLenFeature(shape=(maxlen,), dtype=tf.int64), } ``` Then use TensorFlow's categorical feature columns according to your Estimator model. The `VarLenSparseFeat(maxlen=...)` argument is part of the Keras-style API and is not used by `tf.feature_column`. ## Quick Checklist - Use `0` for padding and start valid categorical ids from `1` for sequence fields. - Set `vocabulary_size` to at least `max_id + 1`. - Pad every `VarLenSparseFeat` input to `(batch_size, maxlen)`. - Use `length_name` when you want explicit sequence lengths. - Use `embedding_name` to share embeddings between candidate and history fields. - For DIN/BST, history names must be `hist_` + the names in `behavior_feature_list`. - For DSIN, split sessions offline and pass `sess_0_*`, `sess_1_*`, and `sess_length`. - Use `DenseFeat(name, dimension)` or `numeric_column(name, shape=(dimension,))` for dense vectors. --- ### Source Deepctr.Contrib.Rnn (`docs/source/deepctr.contrib.rnn.rst`) deepctr.contrib.rnn module ========================== .. automodule:: deepctr.contrib.rnn :members: :undoc-members: :show-inheritance: --- ### Source Deepctr.Contrib (`docs/source/deepctr.contrib.rst`) deepctr.contrib package ======================= Submodules ---------- .. toctree:: deepctr.contrib.rnn deepctr.contrib.utils Module contents --------------- .. automodule:: deepctr.contrib :members: :undoc-members: :show-inheritance: --- ### Source Deepctr.Contrib.Utils (`docs/source/deepctr.contrib.utils.rst`) deepctr.contrib.utils module ============================ .. automodule:: deepctr.contrib.utils :members: :undoc-members: :show-inheritance: --- ### Source Deepctr.Estimator.Feature Column (`docs/source/deepctr.estimator.feature_column.rst`) deepctr.estimator.feature\_column module ======================================== .. automodule:: deepctr.estimator.feature_column :members: :undoc-members: :show-inheritance: --- ### Source Deepctr.Estimator.Inputs (`docs/source/deepctr.estimator.inputs.rst`) deepctr.estimator.inputs module =============================== .. automodule:: deepctr.estimator.inputs :members: :undoc-members: :show-inheritance: --- ### Source Deepctr.Estimator.Models.Afm (`docs/source/deepctr.estimator.models.afm.rst`) deepctr.estimator.models.afm module =================================== .. automodule:: deepctr.estimator.models.afm :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Estimator.Models.Autoint (`docs/source/deepctr.estimator.models.autoint.rst`) deepctr.estimator.models.autoint module ======================================= .. automodule:: deepctr.estimator.models.autoint :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Estimator.Models.Ccpm (`docs/source/deepctr.estimator.models.ccpm.rst`) deepctr.estimator.models.ccpm module ==================================== .. automodule:: deepctr.estimator.models.ccpm :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Estimator.Models.Dcn (`docs/source/deepctr.estimator.models.dcn.rst`) deepctr.estimator.models.dcn module =================================== .. automodule:: deepctr.estimator.models.dcn :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Estimator.Models.Deepfefm (`docs/source/deepctr.estimator.models.deepfefm.rst`) deepctr.estimator.models.deepfefm module ====================================== .. automodule:: deepctr.estimator.models.deepfefm :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Estimator.Models.Deepfm (`docs/source/deepctr.estimator.models.deepfm.rst`) deepctr.estimator.models.deepfm module ====================================== .. automodule:: deepctr.estimator.models.deepfm :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Estimator.Models.Fibinet (`docs/source/deepctr.estimator.models.fibinet.rst`) deepctr.estimator.models.fibinet module ======================================= .. automodule:: deepctr.estimator.models.fibinet :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Estimator.Models.Fnn (`docs/source/deepctr.estimator.models.fnn.rst`) deepctr.estimator.models.fnn module =================================== .. automodule:: deepctr.estimator.models.fnn :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Estimator.Models.Fwfm (`docs/source/deepctr.estimator.models.fwfm.rst`) deepctr.estimator.models.fwfm module ======================================== .. automodule:: deepctr.estimator.models.fwfm :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Estimator.Models.Nfm (`docs/source/deepctr.estimator.models.nfm.rst`) deepctr.estimator.models.nfm module =================================== .. automodule:: deepctr.estimator.models.nfm :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Estimator.Models.Pnn (`docs/source/deepctr.estimator.models.pnn.rst`) deepctr.estimator.models.pnn module =================================== .. automodule:: deepctr.estimator.models.pnn :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Estimator.Models (`docs/source/deepctr.estimator.models.rst`) deepctr.estimator.models package ================================ Submodules ---------- .. toctree:: deepctr.estimator.models.afm deepctr.estimator.models.autoint deepctr.estimator.models.ccpm deepctr.estimator.models.dcn deepctr.estimator.models.deepfm deepctr.estimator.models.fwfm deepctr.estimator.models.fibinet deepctr.estimator.models.fnn deepctr.estimator.models.nfm deepctr.estimator.models.pnn deepctr.estimator.models.wdl deepctr.estimator.models.xdeepfm Module contents --------------- .. automodule:: deepctr.estimator.models :members: :undoc-members: :show-inheritance: --- ### Source Deepctr.Estimator.Models.Wdl (`docs/source/deepctr.estimator.models.wdl.rst`) deepctr.estimator.models.wdl module =================================== .. automodule:: deepctr.estimator.models.wdl :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Estimator.Models.Xdeepfm (`docs/source/deepctr.estimator.models.xdeepfm.rst`) deepctr.estimator.models.xdeepfm module ======================================= .. automodule:: deepctr.estimator.models.xdeepfm :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Estimator (`docs/source/deepctr.estimator.rst`) deepctr.estimator package ========================= Subpackages ----------- .. toctree:: deepctr.estimator.models Submodules ---------- .. toctree:: deepctr.estimator.feature_column deepctr.estimator.inputs deepctr.estimator.utils Module contents --------------- .. automodule:: deepctr.estimator :members: :undoc-members: :show-inheritance: --- ### Source Deepctr.Estimator.Utils (`docs/source/deepctr.estimator.utils.rst`) deepctr.estimator.utils module ============================== .. automodule:: deepctr.estimator.utils :members: :undoc-members: :show-inheritance: --- ### Source Deepctr.Feature Column (`docs/source/deepctr.feature_column.rst`) deepctr.feature\_column module ============================== .. automodule:: deepctr.feature_column :members: :undoc-members: :show-inheritance: --- ### Source Deepctr.Inputs (`docs/source/deepctr.inputs.rst`) deepctr.inputs module ===================== .. automodule:: deepctr.inputs :members: :undoc-members: :show-inheritance: --- ### Source Deepctr.Layers.Activation (`docs/source/deepctr.layers.activation.rst`) deepctr.layers.activation module ================================ .. automodule:: deepctr.layers.activation :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Layers.Core (`docs/source/deepctr.layers.core.rst`) deepctr.layers.core module ========================== .. automodule:: deepctr.layers.core :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Layers.Interaction (`docs/source/deepctr.layers.interaction.rst`) deepctr.layers.interaction module ================================= .. automodule:: deepctr.layers.interaction :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Layers.Normalization (`docs/source/deepctr.layers.normalization.rst`) deepctr.layers.normalization module =================================== .. automodule:: deepctr.layers.normalization :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Layers (`docs/source/deepctr.layers.rst`) deepctr.layers package ====================== Submodules ---------- .. toctree:: deepctr.layers.activation deepctr.layers.core deepctr.layers.interaction deepctr.layers.normalization deepctr.layers.sequence deepctr.layers.utils Module contents --------------- .. automodule:: deepctr.layers :members: :undoc-members: :show-inheritance: --- ### Source Deepctr.Layers.Sequence (`docs/source/deepctr.layers.sequence.rst`) deepctr.layers.sequence module ============================== .. automodule:: deepctr.layers.sequence :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Layers.Utils (`docs/source/deepctr.layers.utils.rst`) deepctr.layers.utils module =========================== .. automodule:: deepctr.layers.utils :members: :undoc-members: :show-inheritance: --- ### Source Deepctr.Models.Afm (`docs/source/deepctr.models.afm.rst`) deepctr.models.afm module ========================= .. automodule:: deepctr.models.afm :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Autoint (`docs/source/deepctr.models.autoint.rst`) deepctr.models.autoint module ============================= .. automodule:: deepctr.models.autoint :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Ccpm (`docs/source/deepctr.models.ccpm.rst`) deepctr.models.ccpm module ========================== .. automodule:: deepctr.models.ccpm :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Dcn (`docs/source/deepctr.models.dcn.rst`) deepctr.models.dcn module ========================= .. automodule:: deepctr.models.dcn :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Dcnmix (`docs/source/deepctr.models.dcnmix.rst`) deepctr.models.dcnmix module ========================= .. automodule:: deepctr.models.dcnmix :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Deepfefm (`docs/source/deepctr.models.deepfefm.rst`) deepctr.models.deepfefm module ============================== .. automodule:: deepctr.models.deepfefm :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Deepfm (`docs/source/deepctr.models.deepfm.rst`) deepctr.models.deepfm module ============================ .. automodule:: deepctr.models.deepfm :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Difm (`docs/source/deepctr.models.difm.rst`) deepctr.models.difm module ============================= .. automodule:: deepctr.models.difm :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Edcn (`docs/source/deepctr.models.edcn.rst`) deepctr.models.edcn module ========================= .. automodule:: deepctr.models.edcn :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Fgcnn (`docs/source/deepctr.models.fgcnn.rst`) deepctr.models.fgcnn module =========================== .. automodule:: deepctr.models.fgcnn :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Fibinet (`docs/source/deepctr.models.fibinet.rst`) deepctr.models.fibinet module ============================= .. automodule:: deepctr.models.fibinet :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Flen (`docs/source/deepctr.models.flen.rst`) deepctr.models.flen module ============================= .. automodule:: deepctr.models.flen :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Fnn (`docs/source/deepctr.models.fnn.rst`) deepctr.models.fnn module ========================= .. automodule:: deepctr.models.fnn :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Fwfm (`docs/source/deepctr.models.fwfm.rst`) deepctr.models.fwfm module ========================== .. automodule:: deepctr.models.fwfm :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Ifm (`docs/source/deepctr.models.ifm.rst`) deepctr.models.ifm module ============================= .. automodule:: deepctr.models.ifm :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Mlr (`docs/source/deepctr.models.mlr.rst`) deepctr.models.mlr module ========================= .. automodule:: deepctr.models.mlr :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Multitask.Esmm (`docs/source/deepctr.models.multitask.esmm.rst`) deepctr.models.multitask.esmm module ============================= .. automodule:: deepctr.models.multitask.esmm :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Multitask.Mmoe (`docs/source/deepctr.models.multitask.mmoe.rst`) deepctr.models.multitask.mmoe module ============================= .. automodule:: deepctr.models.multitask.mmoe :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Multitask.Ple (`docs/source/deepctr.models.multitask.ple.rst`) deepctr.models.multitask.ple module ============================= .. automodule:: deepctr.models.multitask.ple :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Multitask.Sharedbottom (`docs/source/deepctr.models.multitask.sharedbottom.rst`) deepctr.models.multitask.sharedbottom module ============================= .. automodule:: deepctr.models.multitask.sharedbottom :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Nfm (`docs/source/deepctr.models.nfm.rst`) deepctr.models.nfm module ========================= .. automodule:: deepctr.models.nfm :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Onn (`docs/source/deepctr.models.onn.rst`) deepctr.models.onn module ========================== .. automodule:: deepctr.models.onn :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Pnn (`docs/source/deepctr.models.pnn.rst`) deepctr.models.pnn module ========================= .. automodule:: deepctr.models.pnn :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models (`docs/source/deepctr.models.rst`) deepctr.models package ====================== Submodules ---------- .. toctree:: deepctr.models.afm deepctr.models.autoint deepctr.models.ccpm deepctr.models.dcn deepctr.models.dcnmix deepctr.models.edcn deepctr.models.deepfm deepctr.models.dien deepctr.models.din deepctr.models.dsin deepctr.models.fgcnn deepctr.models.fibinet deepctr.models.fnn deepctr.models.fwfm deepctr.models.mlr deepctr.models.onn deepctr.models.nfm deepctr.models.pnn deepctr.models.wdl deepctr.models.xdeepfm deepctr.models.flen deepctr.models.ifm deepctr.models.difm deepctr.models.deepfefm deepctr.models.multitask.sharedbottom deepctr.models.multitask.esmm deepctr.models.multitask.mmoe deepctr.models.multitask.ple Module contents --------------- .. automodule:: deepctr.models :members: :undoc-members: :show-inheritance: --- ### Source Deepctr.Models.Sequence.Bst (`docs/source/deepctr.models.sequence.bst.rst`) deepctr.models.sequence.bst module ========================= .. automodule:: deepctr.models.sequence.bst :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Sequence.Dien (`docs/source/deepctr.models.sequence.dien.rst`) deepctr.models.sequence.dien module ========================== .. automodule:: deepctr.models.sequence.dien :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Sequence.Din (`docs/source/deepctr.models.sequence.din.rst`) deepctr.models.sequence.din module ========================= .. automodule:: deepctr.models.sequence.din :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Sequence.Dsin (`docs/source/deepctr.models.sequence.dsin.rst`) deepctr.models.sequence.dsin module ========================== .. automodule:: deepctr.models.sequence.dsin :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Wdl (`docs/source/deepctr.models.wdl.rst`) deepctr.models.wdl module ========================= .. automodule:: deepctr.models.wdl :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr.Models.Xdeepfm (`docs/source/deepctr.models.xdeepfm.rst`) deepctr.models.xdeepfm module ============================= .. automodule:: deepctr.models.xdeepfm :members: :no-undoc-members: :no-show-inheritance: --- ### Source Deepctr (`docs/source/deepctr.rst`) deepctr package =============== Subpackages ----------- .. toctree:: deepctr.contrib deepctr.layers deepctr.models Submodules ---------- .. toctree:: deepctr.inputs deepctr.utils Module contents --------------- .. automodule:: deepctr :members: :undoc-members: :show-inheritance: --- ### Source Deepctr.Utils (`docs/source/deepctr.utils.rst`) deepctr.utils module ==================== .. automodule:: deepctr.utils :members: :undoc-members: :show-inheritance: --- ### Source Index (`docs/source/index.rst`) .. DeepCTR documentation master file, created by sphinx-quickstart on Fri Nov 23 21:08:54 2018. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. Welcome to DeepCTR's documentation! =================================== |Downloads|_ |Stars|_ |Forks|_ |PyPii|_ |Issues|_ |Chat|_ .. |Downloads| image:: https://pepy.tech/badge/deepctr .. _Downloads: https://pepy.tech/project/deepctr .. |Stars| image:: https://img.shields.io/github/stars/shenweichen/deepctr.svg .. _Stars: https://github.com/shenweichen/DeepCTR .. |Forks| image:: https://img.shields.io/github/forks/shenweichen/deepctr.svg .. _Forks: https://github.com/shenweichen/DeepCTR/fork .. |PyPii| image:: https://img.shields.io/pypi/v/deepctr.svg .. _PyPii: https://pypi.org/project/deepctr .. |Issues| image:: https://img.shields.io/github/issues/shenweichen/deepctr.svg .. _Issues: https://github.com/shenweichen/deepctr/issues .. |Chat| image:: https://img.shields.io/badge/chat-wechat-brightgreen?style=flat .. _Chat: ./#disscussiongroup DeepCTR is a **Easy-to-use** , **Modular** and **Extendible** package of deep-learning based CTR models along with lots of core components layer which can be used to easily build custom models.You can use any complex model with ``model.fit()`` and ``model.predict()``. - Provide ``tf.keras.Model`` like interface for **quick experiment**. `example `_ - Provide ``tensorflow estimator`` interface for **large scale data** and **distributed training**. `example `_ - It is compatible with both ``tf 1.15`` and ``tf 2.x``. Install a TensorFlow build that matches your Python, NumPy, CPU/GPU, and operating system first, then install DeepCTR. Use public ``tensorflow.keras`` APIs in your own code and avoid mixing them with private ``tensorflow.python.keras`` imports. Let's `Get Started! <./Quick-Start.html>`_ (`Chinese Introduction `_) You can read the latest code and related projects - DeepCTR: https://github.com/shenweichen/DeepCTR - DeepMatch: https://github.com/shenweichen/DeepMatch - DeepCTR-Torch: https://github.com/shenweichen/DeepCTR-Torch News ----- 04/16/2026 : Support higher tensorflow version . `Changelog `_ 11/10/2022 : Add `EDCN` . `Changelog `_ 10/15/2022 : Support python `3.9` , `3.10` . `Changelog `_ DisscussionGroup ----------------------- 公众号：**浅梦学习笔记** wechat ID: **deepctrbot** `Discussions `_ .. image:: ../pics/code.png .. toctree:: :maxdepth: 2 :caption: Home: Quick-Start Features Examples FAQ History .. toctree:: :maxdepth: 2 :caption: Cookbook: Sequence Feature Cookbook .. toctree:: :maxdepth: 3 :caption: API: Models Estimators Layers Indices and tables ================== * :ref:`genindex` * :ref:`modindex` * :ref:`search` --- ### Source Modules (`docs/source/modules.rst`) deepctr ======= .. toctree:: :maxdepth: 4 deepctr --- --- METRICS --- - Files Extracted: 76 - Estimated Token Budget: ~29671 tokens - Canonical Reference: https://codewiki.google/github.com/shenweichen/DeepCTR