Servus! 👋
Viet Pham
Researcher // AI in Medicine // Multimodal AI
PhD Candidate · Technical University of Munich
About
I'm a PhD candidate in Medical AI at the Institute for AI and Informatics in Medicine (AIIM), Technical University of Munich, working on multimodal clinical AI — with a focus on medical imaging and how it can be combined with other clinical data. I'm part of the GO-TWIN project, led by Dr. Jacqueline Lammert.
Before that, my work centered on medical robotics and computer vision. My path runs from data engineering at apic.ai, through surgical-robotics research at KIT's HERA Lab (under Prof. Franziska Mathis-Ullrich) and Carl Zeiss AG, to a visiting scholarship at Johns Hopkins (IMERSE Lab, Prof. Axel Krieger), where I built multimodal imitation-learning models for robotic partial nephrectomy.
I hold an M.Sc. in Computer Science from TUM (focus: AI in Medicine, Computer Aided Medical Procedures) and a B.Sc. from KIT.
Fun fact: my research runs on coffee ☕ — about 30 cups since I started my PhD, give or take.
News
- Sep 2026Started my PhD in Medical AI at the AIIM Institute, TUM, joining the GO-TWIN project.
- Jul 2026Completed my M.Sc. in Computer Science at TUM.
- Apr 2026The Open-H-Embodiment paper, a large-scale dataset for foundation models in medical robotics, is out on arXiv. I contributed surgical demonstration data from Johns Hopkins.
- Mar 2026Defended my Master's thesis on multi-modal imitation learning for robotic partial nephrectomy with a grade of 1.0 (the highest grade).
- Mar 2026Our paper on weakly supervised surgical instrument segmentation was published in Scientific Reports.
- Nov 2025Our Surgical Gaussian Surfels paper was published at WACV 2026 and received the Best Paper Award at the SPAR3D Workshop.
- Apr 2025Joined the IMERSE Lab at Johns Hopkins University (Prof. Axel Krieger) as a visiting graduate scholar for my Master's thesis, after almost two years in research & innovation at Carl Zeiss.
- Aug 2024Our paper on reinforcement learning for lens capsule tearing in cataract surgery was published at ICRA 2024.
Research Interests
- Multimodal clinical AI — fusing medical imaging with complementary clinical data for diagnosis and treatment decisions
- Surgical robotics — autonomous and semi-autonomous control for minimally invasive procedures
- Imitation & reinforcement learning — learning manipulation policies from demonstration and simulation
- Vision-language(-action) models — grounding natural-language instructions in robotic actions
- Computer vision for surgery — instrument segmentation, scene reconstruction, intra-operative perception
Publications
My name is shown in bold. See also my Google Scholar profile.
-
Open-H-Embodiment: A Large-Scale Dataset for Enabling Foundation Models in Medical Robotics
arXiv preprint · April 2026
@misc{openh2026embodiment, title = {Open-H-Embodiment: A Large-Scale Dataset for Enabling Foundation Models in Medical Robotics}, author = {{Open-H-Embodiment Consortium} and Nelson, Nigel and Chen, Juo-Tung and Haworth, Jesse and Chen, Xinhao and Zbinden, Lukas and Huang, Dianye and Abdelaal, Alaa Eldin and Arezzo, Alberto and Acar, Ayberk and Alambeigi, Farshid and Ammirati, Carlo Alberto and Ao, Yunke and Rodriguez, Pablo David Aranda and Atar, Soofiyan and Ballo, Mattia and Barnes, Noah and Barontini, Federica and Binkiewicz, Filip and Black, Peter and Bodenstedt, Sebastian and Borgioli, Leonardo and Budjak, Nikola and Calmé, Benjamin and Carrillo, Fabio and Cavalcanti, Nicola and Chen, Changwei and Chen, Haoxin and Chen, Sihang and Chen, Qihan and Chen, Zhongyu and Chen, Ziyang and Cheng, Shing Shin and Cheng, Meiqing and Cheng, Min and Chiu, Zih-Yun Sarah and Chu, Xiangyu and Correa-Gallego, Camilo and Dagnino, Giulio and Deguet, Anton and Delgado, Jacob and DeLong, Jonathan C. and Deng, Kaizhong and Dimitrakakis, Alexander and Ding, Qingpeng and Ding, Hao and Distefano, Giovanni and Donoho, Daniel and Duan, Anqing and Esposito, Marco and Farritor, Shane and Fayad, Jad and Fayad, Zahi and Ferradosa, Mario and Filicori, Filippo and Finn, Chelsea and Fürnstahl, Philipp and Ge, Jiawei and Giannarou, Stamatia and Ludevid, Xavier Giralt and Giraud, Frederic and Godbole, Aditya Amit and Goldberg, Ken and Goldenberg, Antony and Marana, Diego Granero and Guo, Xiaoqing and Haidegger, Tamás and Hailey, Evan and Hansen, Pascal and Hao, Ziyi and Hari, Kush and Hayashi, Kengo and Hawkins, Jonathon and Haworth, Shelby and Hellig, Ortrun and Herrell, S. Duke and Hong, Zhouyang and Howe, Andrew and Hu, Junlei and Hu, Zhaoyang Jacopo and Jain, Ria and Javazm, Mohammad Rafiee and Ji, Howard and Ji, Rui and Ji, Jianmin and Jiang, Zhongliang and Jones, Dominic and Jopling, Jeffrey and Jordan, Britton and Ju, Ran and Kam, Michael and Kang, Luoyao and Kang, Fausto and Kapuria, Siddhartha and Kazanzides, Peter and Kiehler, Sonika and Kilmer, Ethan and Kim, Ji Woong and Korzeniowski, Przemysław and Kuchi, Chandra and Kumar, Nithesh and Kuntz, Alan and Lavagno, Federico and Lee, Yu Chung and Lee, Hao-Chih and Li, Hang and Li, Zhen and Liang, Xiao and Lin, Xinxin and Lin, Jinsong and Liu, Chang and Liu, Fei and Liu, Pei and Liu, Yun-hui and Liuchen, Wanli and Lukács, Eszter and Mann, Sareena and Mannas, Miles and Marinelli, Brett and Martyniak, Sabina and Marzola, Francesco and Mazza, Lorenzo and Mei, Xueyan and Morais, Maria Clara and Muratore, Luigi and Narayanaswamy, Chetan Reddy and Naskręt, Michał and Navarro-Alarcon, David and Neary, Cyrus and Ng, Chi Kit and Nguan, Christopher and Noonan, David and Oh, Ki Hwan and Olesch, Tom Christian and Okamura, Allison M. and Opfermann, Justin and Pescio, Matteo and Pham, Doan Xuan Viet and Porras, Tito and Ren, Hongliang and Jimenez, Ariel Rodriguez and Baena, Ferdinando Rodriguez y and Salcudean, Septimiu E. and Sathya, Asmitha and Satish, Preethi and Seenivasan, Lalithkumar and Shao, Jiaqi and Shen, Yiqing and Sheng, Yu and Shi, Lucy XiaoYang and Soulé, Zoe and Speidel, Stefanie and Su, Mingwu and Su, Jianhao and Sunmola, Idris and Takács, Kristóf and Tang, Yunxi and Thornycroft, Patrick and Tian, Yu and Thompson, Jordan and Turkcan, Mehmet K. and Unberath, Mathias and Valdastri, Pietro and Vives, Carlos and Vuong, Quan and Wagner, Martin and Wang, Farong and Wang, Wei and Wang, Lidian and Wang, Chung-Pang and Wang, Guankun and Wang, Junyi and Wang, Erqi and Wang, Ziyi and Watts, Tanner and Wein, Wolfgang and Wu, Yimeng and Wu, Zijian and Wu, Hongjun and Wu, Luohong and Wu, Jie Ying and Wu, Junlin and Wu, Victoria and Wu, Kaixuan and Wójcikowski, Mateusz and Xiao, Yunye and Xiao, Nan and Xie, Wenxuan and Yang, Hao and Yang, Tianqi and Yang, Yinuo and Ye, Menglong and Yeung, Ryan S. and Yilmaz, Nural and Yin, Chim Ho and Yip, Michael and Younis, Rayan and Yu, Chenhao and Zaman, Sayem Nazmuz and Zefran, Milos and Zhang, Han and Zhang, Yuelin and Zhang, Yidong and Zhang, Yanyong and Zhang, Xuyang and Zhang, Yameng and Zhang, Joyce and Zhong, Ning and Zhou, Peng and Zhou, Haoying and Zuo, Xiuli and Navab, Nassir and Azizian, Mahdi and Huver, Sean D. and Krieger, Axel}, year = {2026}, eprint = {2604.21017}, archivePrefix = {arXiv}, url = {https://arxiv.org/abs/2604.21017} } -
Domain-agnostic weakly supervised surgical instrument segmentation
Scientific Reports (Nature Portfolio) · March 2026
@article{peter2026domain, title = {Domain-agnostic weakly supervised surgical instrument segmentation}, author = {Peter, Rebekka and Pham, Doan Xuan Viet and Matten, Philipp and Wu, Erik and Nienhaus, Jonas and Meissen, Felix and Menten, Martin J. and Tagliabue, Eleonora and Mathis-Ullrich, Franziska}, journal = {Scientific Reports}, volume = {16}, number = {1}, year = {2026}, publisher = {Springer Science and Business Media LLC}, doi = {10.1038/s41598-026-43054-1} } -
Surgical Gaussian Surfels: Highly Accurate Real-time Surgical Scene Rendering using Gaussian Surfels
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2026) · November 2025
Best Paper Award · SPAR3D Workshop
@misc{sunmola2025surgical, title = {Surgical Gaussian Surfels: Highly Accurate Real-time Surgical Scene Rendering using Gaussian Surfels}, author = {Sunmola, Idris O. and Zhao, Zhenjun and Schmidgall, Samuel and Wang, Yumeng and Scheikl, Paul Maria and Pham, Viet and Krieger, Axel}, year = {2025}, eprint = {2503.04079}, archivePrefix = {arXiv}, url = {https://arxiv.org/abs/2503.04079} } -
Lens Capsule Tearing in Cataract Surgery using Reinforcement Learning
IEEE International Conference on Robotics and Automation (ICRA 2024) · August 2024
@inproceedings{peter2024lens, title = {Lens Capsule Tearing in Cataract Surgery using Reinforcement Learning}, author = {Peter, Rebekka Charlotte and Peikert, Steffen and Haide, Ludwig and Pham, Doan Xuan Viet and Chettaoui, Tahar and Tagliabue, Eleonora and Scheikl, Paul Maria and Fauser, Johannes and Hillenbrand, Matthias and Neumann, Gerhard and Mathis-Ullrich, Franziska}, booktitle = {2024 IEEE International Conference on Robotics and Automation (ICRA)}, pages = {15501--15508}, year = {2024}, publisher = {IEEE}, doi = {10.1109/ICRA57147.2024.10611714} }
Experience
-
Sep 2026 — Present
PhD Candidate · Medical AI
TUM Klinikum Rechts der Isar · AIIM Institute · Munich, Germany
- Researching multimodal clinical AI within the GO-TWIN project (Gynecological Oncology – Targeting Women's Individual Needs), led by Dr. Jacqueline Lammert.
-
Apr 2025 — Sep 2025
Visiting Graduate Scholar
Johns Hopkins University · IMERSE Lab (Prof. Axel Krieger) · Baltimore, USA
- Developed multimodal imitation learning models for robotic partial nephrectomy, fusing visual observations with BERT-encoded language instructions and pre-operative anatomical context for action prediction.
- Contributed multimodal surgical demonstration data (robot trajectories, synchronized camera streams) to the Open-H-Embodiment dataset — a multi-institution open-source healthcare robotics initiative (NVIDIA, JHU, TUM).
-
Oct 2023 — Mar 2025
Working Student · Research & Innovation
Carl Zeiss AG · Karlsruhe, Germany
- Self-initiated interdisciplinary research collaboration with the AI in Medicine (AIM) Lab at TUM on instrument localization in microscope-integrated OCT images; implemented methodology and conducted experiments underlying the Scientific Reports 2026 publication.
- Extended and maintained reinforcement learning pipelines in medical simulations (SOFA); integrated new features and conducted follow-up experiments.
-
Jun 2023 — Sep 2023
Research & Innovation Intern
Carl Zeiss AG · Karlsruhe, Germany
- Designed and trained reinforcement learning agents for the capsulorhexis procedure in cataract surgery; implemented the SOFA-based RL training pipeline, designed the reward function, and conducted the experiments underlying the ICRA 2024 publication.
- Ran wet-lab experiments on a Mecademic single-arm robot using ex-vivo porcine tissue and phantoms, including software/hardware integration for closed-loop agent evaluation.
-
Jun 2022 — Mar 2023
Student Research Assistant
Karlsruhe Institute of Technology · HERA Lab · Karlsruhe, Germany
- Performed semantic image segmentation on surgical images using deep learning.
- Ported ETH Zurich's magnetic continuum robot (m-CR) SOFA simulation for reinforcement learning.
-
Jan 2020 — Mar 2022
Working Student · Software / Data Engineer
apic.ai GmbH · Karlsruhe, Germany
- Developed ETL data pipelines for AI-based video analysis, processing over 35 TB / year.
- Optimized Kubernetes clusters with up to 256 NVIDIA GPU instances (GKE, EKS).
Education
-
Oct 2023 — Jul 2026
M.Sc. Computer Science
Technical University of Munich · Munich, Germany
Focus: AI in Medicine, Computer Aided Medical Procedures, Computer Vision.
-
Apr 2025 — Sep 2025
Master's Thesis in Computer Science
Johns Hopkins University · Baltimore, USA
Multi-Modal Imitation Learning for Robotic Partial Nephrectomy · Grade: 1.0
-
Oct 2017 — Sep 2023
B.Sc. Computer Science (Minor: Electrical Engineering)
Karlsruhe Institute of Technology · Karlsruhe, Germany
Thesis: Reinforcement Learning for Laparoscopic Tissue Retraction · Grade: 1.0
Skills
- Tooling
- Python · C++ · C# · PyTorch · ROS · OpenCV · NumPy · SOFA · Docker · Kubernetes
- Methods
- Multimodal Learning · Medical Imaging · Reinforcement Learning · Imitation Learning · Vision-Language Models · Computer Vision · Medical Robotics
- Languages
- German (native) · Vietnamese (native) · English (fluent)
Get in touch
I'm always happy to talk about research and collaborations in multimodal clinical AI. Feel free to reach out via viet.x.pham [at] tum.de or save my contact card.