Predicting Sequence Specificities of Transcription Factors with Self-Attention Sequence Modeling

Abstract

Transcription factor plays crucial role in gene expression via regulating transcription process. To predict the sequence specificity of each transcription factor I propose a deep learning model, the AttendBind which employs k-mer embedding and self-attention sequence modeling approaches. The experimental results on real biophysical data show that the proposed method outperforms other deep learning methods, indicating that the self-attention sequence modeling is highly effective on this task. In addition to the given prediction task, the visualization of self-attention maps and top-3 frequency based analyses can provide useful information for interpreting the deep learning model and discovering scientific knowledge.