TransactionGPT as a Payments Foundation Model
Jun 22, 2026
This episode explores TransactionGPT, a Visa Research paper that argues for a foundation-model approach to consumer transaction data spanning generation, anomaly detection, and representation learning. It explains why payment histories are fundamentally different from text or simple time series: each event mixes merchant IDs, amounts, timestamps, and engineered risk signals, creating a multi-modal, temporal, tabular structure that demands more schema-aware modeling. The discussion walks through the paper’s 1D, 2D, and 3D architecture progression, highlighting how separate transformers for transaction metadata, downstream features, and behavioral sequences aim to avoid the pitfalls of flattening everything into a single embedding space. Listeners would find it interesting for its clear debate over whether TransactionGPT is a genuine reusable backbone for payments or mainly a strong engineering response to real-world constraints like heterogeneity, scale, regulation, and low-latency fraud decisioning.
Interactive Visualization: TransactionGPT as a Payments Foundation Model
Sources:
1. TransactionGPT — Yingtong Dou, Zhimeng Jiang, Tianyi Zhang, Mingzhi Hu, Zhichao Xu, Shubham Jain, Uday Singh Saini, Xiran Fan, Jiarui Sun, Menghai Pan, Junpeng Wang, Xin Dai, Liang Wang, Chin-Chia Michael Yeh, Yujie Fan, Yan Zheng, Vineeth Rakesh, Huiyuan Chen, Guanchu Wang, Mangesh Bendre, Zhongfang Zhuang, Xiaoting Li, Prince Aboagye, Vivian Lai, Minghua Xu, Hao Yang, Yiwei Cai, Mahashweta Das, Yuzhong Chen, 2025
http://arxiv.org/abs/2511.08939
2. TabDPT: Scaling Tabular Foundation Models on Real Data — Junwei Ma, Valentin Thomas, Rasa Hosseinzadeh, Alex Labach, Hamidreza Kamkari, Jesse C. Cresswell, Keyvan Golestan, Guangwei Yu, Anthony L. Caterini, Maksims Volkovs, 2024
https://arxiv.org/abs/2410.18164
3. Towards a Foundation Purchasing Model: Pretrained Generative Autoregression on Transaction Sequences — Piotr Skalski, David Sutton, Stuart Burrell, Iker Perez, Jason Wong, 2024
https://arxiv.org/abs/2401.01641
4. Open Banking Foundational Model: Learning Language Representations from Few Financial Transactions — Gustavo Polleti, Marlesson Santana, Eduardo Fontes, 2025
https://arxiv.org/abs/2511.12154
5. TransactionGPT — Yingtong Dou, Zhimeng Jiang, Tianyi Zhang, Mingzhi Hu, Zhichao Xu, Shubham Jain, Uday Singh Saini, Xiran Fan, Jiarui Sun, Menghai Pan, Junpeng Wang, Xin Dai, Liang Wang, Chin-Chia Michael Yeh, Yujie Fan, Vineeth Rakesh, Huiyuan Chen, Mangesh Bendre, Zhongfang Zhuang, Xiaoting Li, Prince Aboagye, Vivian Lai, Minghua Xu, Hao Yang, Yiwei Cai, Mahashweta Das, Yuzhong Chen, 2025
https://arxiv.org/abs/2511.08939
6. TabTransformer: Tabular Data Modeling Using Contextual Embeddings — Xin Huang, Ashish Khetan, Milan Cvitkovic, Zohar Karnin, 2020
https://arxiv.org/abs/2012.06678
7. Tabular Transformers for Modeling Multivariate Time Series — Inkit Padhi, Yair Schiff, Igor Melnyk, Mattia Rigotti, Youssef Mroueh, Pierre Dognin, Jerret Ross, Ravi Nair, Erik Altman, 2021
https://arxiv.org/abs/2011.01843
8. FATA-Trans: Field And Time-Aware Transformer for Sequential Tabular Data — Dongyu Zhang, Liang Wang, Xin Dai, Shubham Jain, Junpeng Wang, Yujie Fan, Chin-Chia Michael Yeh, Yan Zheng, Zhongfang Zhuang, Wei Zhang, 2023
https://arxiv.org/abs/2310.13818
9. Multi-modal Time Series Analysis: A Tutorial and Survey — Yushan Jiang, Kanghui Ning, Zijie Pan, Xuyang Shen, Jingchao Ni, Wenchao Yu, Anderson Schneider, Haifeng Chen, Yuriy Nevmyvaka, Dongjin Song, 2025
https://arxiv.org/abs/2503.13709
10. Credit card fraud detection using machine learning: A survey — Yvan Lucas, Johannes Jurgovsky, 2020
https://arxiv.org/abs/2010.06479
11. Interleaved Sequence RNNs for Fraud Detection — Bernardo Branco, Pedro Abreu, Ana Sofia Gomes, Mariana S. C. Almeida, João Tiago Ascensão, Pedro Bizarro, 2020
https://arxiv.org/abs/2002.05988
12. FraudTransformer: Time-Aware GPT for Transaction Fraud Detection — Gholamali Aminian, Andrew Elliott, Tiger Li, Timothy Cheuk Hin Wong, Victor Claude Dehon, Lukasz Szpruch, Carsten Maple, Christopher Read, Martin Brown, Gesine Reinert, Mo Mamouei, 2025
https://arxiv.org/abs/2509.23712
13. Enhancing Foundation Models in Transaction Understanding with LLM-based Sentence Embeddings — Xiran Fan, Zhimeng Jiang, Chin-Chia Michael Yeh, Yuzhong Chen, Yingtong Dou, Menghai Pan, Yan Zheng, 2025
https://scholar.google.com/scholar?q=Enhancing+Foundation+Models+in+Transaction+Understanding+with+LLM-based+Sentence+Embeddings
14. Self-Attentive Sequential Recommendation — Wang-Cheng Kang, Julian McAuley, 2018
https://scholar.google.com/scholar?q=Self-Attentive+Sequential+Recommendation
15. TabLLM: Few-shot Classification of Tabular Data with Large Language Models — Stefan Hegselmann, Alejandro Buendia, Hunter Lang, Monica Agrawal, Xiaoyi Jiang, David Sontag, 2022
https://scholar.google.com/scholar?q=TabLLM%3A+Few-shot+Classification+of+Tabular+Data+with+Large+Language+Models
16. TimeGPT-1 — Azul Garza, Cristian Challu, Max Mergenthaler-Canseco, 2023
https://scholar.google.com/scholar?q=TimeGPT-1
17. ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learning — Junguang Jiang et al., 2023
https://scholar.google.com/scholar?q=ForkMerge%3A+Mitigating+Negative+Transfer+in+Auxiliary-Task+Learning
18. Identification of Negative Transfers in Multitask Learning Using Surrogate Models — Dongyue Li, Huy L. Nguyen, Hongyang R. Zhang, 2023
https://scholar.google.com/scholar?q=Identification+of+Negative+Transfers+in+Multitask+Learning+Using+Surrogate+Models
19. Enriching Tabular Data with Contextual LLM Embeddings: A Comprehensive Ablation Study for Ensemble Classifiers — Gjergji Kasneci, Enkelejda Kasneci, 2024
https://scholar.google.com/scholar?q=Enriching+Tabular+Data+with+Contextual+LLM+Embeddings%3A+A+Comprehensive+Ablation+Study+for+Ensemble+Classifiers
20. LLM Embeddings Improve Test-time Adaptation to Tabular Y|X-Shifts — Yibo Zeng et al., 2024
https://scholar.google.com/scholar?q=LLM+Embeddings+Improve+Test-time+Adaptation+to+Tabular+Y%7CX-Shifts
21. LLM Embeddings for Deep Learning on Tabular Data — Boshko Koloski et al., 2025
https://scholar.google.com/scholar?q=LLM+Embeddings+for+Deep+Learning+on+Tabular+Data
22. TableGPT2: A Large Multimodal Model with Tabular Data Integration — Aofeng Su et al., 2024
https://scholar.google.com/scholar?q=TableGPT2%3A+A+Large+Multimodal+Model+with+Tabular+Data+Integration
23. AI Post Transformers: KumoRFM for In-Context Relational Learning — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-06-11-kumorfm-for-in-context-relational-learni-520d2b.mp3
24. AI Post Transformers: Relational Graph Transformer for Multi-Table Learning — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-06-11-relational-graph-transformer-for-multi-t-57cce3.mp3
25. AI Post Transformers: Unembedding Matrices as Feature Lenses for Embeddings — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-06-08-unembedding-matrices-as-feature-lenses-f-4dc415.mp3
Interactive Visualization: TransactionGPT as a Payments Foundation Model
1. TransactionGPT — Yingtong Dou, Zhimeng Jiang, Tianyi Zhang, Mingzhi Hu, Zhichao Xu, Shubham Jain, Uday Singh Saini, Xiran Fan, Jiarui Sun, Menghai Pan, Junpeng Wang, Xin Dai, Liang Wang, Chin-Chia Michael Yeh, Yujie Fan, Yan Zheng, Vineeth Rakesh, Huiyuan Chen, Guanchu Wang, Mangesh Bendre, Zhongfang Zhuang, Xiaoting Li, Prince Aboagye, Vivian Lai, Minghua Xu, Hao Yang, Yiwei Cai, Mahashweta Das, Yuzhong Chen, 2025
http://arxiv.org/abs/2511.08939
2. TabDPT: Scaling Tabular Foundation Models on Real Data — Junwei Ma, Valentin Thomas, Rasa Hosseinzadeh, Alex Labach, Hamidreza Kamkari, Jesse C. Cresswell, Keyvan Golestan, Guangwei Yu, Anthony L. Caterini, Maksims Volkovs, 2024
https://arxiv.org/abs/2410.18164
3. Towards a Foundation Purchasing Model: Pretrained Generative Autoregression on Transaction Sequences — Piotr Skalski, David Sutton, Stuart Burrell, Iker Perez, Jason Wong, 2024
https://arxiv.org/abs/2401.01641
4. Open Banking Foundational Model: Learning Language Representations from Few Financial Transactions — Gustavo Polleti, Marlesson Santana, Eduardo Fontes, 2025
https://arxiv.org/abs/2511.12154
5. TransactionGPT — Yingtong Dou, Zhimeng Jiang, Tianyi Zhang, Mingzhi Hu, Zhichao Xu, Shubham Jain, Uday Singh Saini, Xiran Fan, Jiarui Sun, Menghai Pan, Junpeng Wang, Xin Dai, Liang Wang, Chin-Chia Michael Yeh, Yujie Fan, Vineeth Rakesh, Huiyuan Chen, Mangesh Bendre, Zhongfang Zhuang, Xiaoting Li, Prince Aboagye, Vivian Lai, Minghua Xu, Hao Yang, Yiwei Cai, Mahashweta Das, Yuzhong Chen, 2025
https://arxiv.org/abs/2511.08939
6. TabTransformer: Tabular Data Modeling Using Contextual Embeddings — Xin Huang, Ashish Khetan, Milan Cvitkovic, Zohar Karnin, 2020
https://arxiv.org/abs/2012.06678
7. Tabular Transformers for Modeling Multivariate Time Series — Inkit Padhi, Yair Schiff, Igor Melnyk, Mattia Rigotti, Youssef Mroueh, Pierre Dognin, Jerret Ross, Ravi Nair, Erik Altman, 2021
https://arxiv.org/abs/2011.01843
8. FATA-Trans: Field And Time-Aware Transformer for Sequential Tabular Data — Dongyu Zhang, Liang Wang, Xin Dai, Shubham Jain, Junpeng Wang, Yujie Fan, Chin-Chia Michael Yeh, Yan Zheng, Zhongfang Zhuang, Wei Zhang, 2023
https://arxiv.org/abs/2310.13818
9. Multi-modal Time Series Analysis: A Tutorial and Survey — Yushan Jiang, Kanghui Ning, Zijie Pan, Xuyang Shen, Jingchao Ni, Wenchao Yu, Anderson Schneider, Haifeng Chen, Yuriy Nevmyvaka, Dongjin Song, 2025
https://arxiv.org/abs/2503.13709
10. Credit card fraud detection using machine learning: A survey — Yvan Lucas, Johannes Jurgovsky, 2020
https://arxiv.org/abs/2010.06479
11. Interleaved Sequence RNNs for Fraud Detection — Bernardo Branco, Pedro Abreu, Ana Sofia Gomes, Mariana S. C. Almeida, João Tiago Ascensão, Pedro Bizarro, 2020
https://arxiv.org/abs/2002.05988
12. FraudTransformer: Time-Aware GPT for Transaction Fraud Detection — Gholamali Aminian, Andrew Elliott, Tiger Li, Timothy Cheuk Hin Wong, Victor Claude Dehon, Lukasz Szpruch, Carsten Maple, Christopher Read, Martin Brown, Gesine Reinert, Mo Mamouei, 2025
https://arxiv.org/abs/2509.23712
13. Enhancing Foundation Models in Transaction Understanding with LLM-based Sentence Embeddings — Xiran Fan, Zhimeng Jiang, Chin-Chia Michael Yeh, Yuzhong Chen, Yingtong Dou, Menghai Pan, Yan Zheng, 2025
https://scholar.google.com/scholar?q=Enhancing+Foundation+Models+in+Transaction+Understanding+with+LLM-based+Sentence+Embeddings
14. Self-Attentive Sequential Recommendation — Wang-Cheng Kang, Julian McAuley, 2018
https://scholar.google.com/scholar?q=Self-Attentive+Sequential+Recommendation
15. TabLLM: Few-shot Classification of Tabular Data with Large Language Models — Stefan Hegselmann, Alejandro Buendia, Hunter Lang, Monica Agrawal, Xiaoyi Jiang, David Sontag, 2022
https://scholar.google.com/scholar?q=TabLLM%3A+Few-shot+Classification+of+Tabular+Data+with+Large+Language+Models
16. TimeGPT-1 — Azul Garza, Cristian Challu, Max Mergenthaler-Canseco, 2023
https://scholar.google.com/scholar?q=TimeGPT-1
17. ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learning — Junguang Jiang et al., 2023
https://scholar.google.com/scholar?q=ForkMerge%3A+Mitigating+Negative+Transfer+in+Auxiliary-Task+Learning
18. Identification of Negative Transfers in Multitask Learning Using Surrogate Models — Dongyue Li, Huy L. Nguyen, Hongyang R. Zhang, 2023
https://scholar.google.com/scholar?q=Identification+of+Negative+Transfers+in+Multitask+Learning+Using+Surrogate+Models
19. Enriching Tabular Data with Contextual LLM Embeddings: A Comprehensive Ablation Study for Ensemble Classifiers — Gjergji Kasneci, Enkelejda Kasneci, 2024
https://scholar.google.com/scholar?q=Enriching+Tabular+Data+with+Contextual+LLM+Embeddings%3A+A+Comprehensive+Ablation+Study+for+Ensemble+Classifiers
20. LLM Embeddings Improve Test-time Adaptation to Tabular Y|X-Shifts — Yibo Zeng et al., 2024
https://scholar.google.com/scholar?q=LLM+Embeddings+Improve+Test-time+Adaptation+to+Tabular+Y%7CX-Shifts
21. LLM Embeddings for Deep Learning on Tabular Data — Boshko Koloski et al., 2025
https://scholar.google.com/scholar?q=LLM+Embeddings+for+Deep+Learning+on+Tabular+Data
22. TableGPT2: A Large Multimodal Model with Tabular Data Integration — Aofeng Su et al., 2024
https://scholar.google.com/scholar?q=TableGPT2%3A+A+Large+Multimodal+Model+with+Tabular+Data+Integration
23. AI Post Transformers: KumoRFM for In-Context Relational Learning — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-06-11-kumorfm-for-in-context-relational-learni-520d2b.mp3
24. AI Post Transformers: Relational Graph Transformer for Multi-Table Learning — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-06-11-relational-graph-transformer-for-multi-t-57cce3.mp3
25. AI Post Transformers: Unembedding Matrices as Feature Lenses for Embeddings — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-06-08-unembedding-matrices-as-feature-lenses-f-4dc415.mp3
Interactive Visualization: TransactionGPT as a Payments Foundation Model