John Collomosse

Principal Scientist

London

Prof. John Collomosse is a principal scientist and research manager in Adobe Research’s Image & Language Lab (ILO) lab, and leads the Cross-modal Representation Learning (XRL) team.

John leads research for Adobe’s Content Authenticity Initiative (CAI) and is a core technical advisor to the initiative since his early involvement in its inception in 2019. Now with 3000+ members, CAI leads a cross-industry standards group (C2PA; Coalition for Content Provenance and Authenticity) where John chairs cross-industry task forces on watermarking, fingerprinting and Blockchain.  

John’s research intersects Artificial Intelligence (AI) and Distributed Ledger Technology (DLT), with focus on media provenance to fight misinformation and online harms, and on improving data integrity and attribution for responsible AI.  John’s content fingerprinting research is used to protect millions of images daily across Adobe’s platforms such as Photoshop, Lightroom and generative AI Firefly tools.  John has also pioneered several visual search technologies, such as style, sketch and pose based search.  These have been demoed at Adobe MAX and shipped in products such as Behance ‘More Like This’ style similarity search.  John has published over 130 papers and over 30 patents, and is an Adobe Distinguished Inventor.

John has over 20 years experience at the intersection of Computer Vision, Graphics and AI, completing his PhD on Image and Video Stylization at the University of Bath in 2004.  John is concurrently a full professor at the University of Surrey, Centre for Vision Speech and Signal Processing (CVSSP) where he is the founder and director of DECaDE, the UKRI Natioanl Research Centre for the Decentralized Creative Economy.  Notably, he led the ARCHANGEL project which pioneered use of AI and Blockchain for provenance to tamper-proof National Archives around the world and was called out as a highlight of the UK Science Council (EPSRC) Digital Economy research programme.

John has also spent time elsewhere working on industry R&D in Computer Vision/AI, including at HP Labs, Vodafone Munich and IBM Research Hursley.  He is a Fellow of the IET (FIET), a Senior member of the IEEE, and a Chartered Engineer (CEng).  He is a member of the EPSRC advisory team for Information & Communication Technologies (ICT) portfolio (2018-2024).

Publications

SegGuard: Defending Scene Segmentation against Adversarial Patch Attack

Gittings, T., Schneider, S., Collomosse, J. (Oct. 27, 2024)

International Conference on Image Processing (ICIP)

Thinking Outside the BBox: Unconstrained Generative Object Compositing

Tarres, G., Lin, Z., Zhang, Z., Zhang, J., Song, Y., Ruta, D., Gilbert, A., Collomosse, J., Kim, S. (Oct. 1, 2024)

European Conference on Computer Vision (ECCV)

FineMatch: Aspect-based Fine-grained Image and Text Mismatch Detection and Correction

Hua, H., Shi, J., Kafle, K., Jenni, S., Zhang, D., Collomosse, J., Cohen, S., Liu, J. (Oct. 1, 2024)

European Conference on Computer Vision (ECCV)

Self-supervised disentangled representation learning of artistic style through Neural Style Transfer

Ruta, D., Tarres, G., Black, A., Gilbert, A., Collomosse, J. (Oct. 1, 2024)

European Conference on Computer Vision (ECCVW) Workshop on Vision and Art (VISART)

NeAT: Neural Artistic Tracing for high resolution Style Transfer

Ruta, D., Gilbert, A., Collomosse, J., Shechtman, E., Kolkin, N. (Sep. 29, 2024)

European Conference on Computer Vision (ECCVW'24) Workshop on Vision for Art (VISART VII)

DIFF-NST: Diffusion Interleaving For deFormable Neural Style Transfer

Ruta, D., Tarres, G., Gilbert, A., Shechtman, E., Kolkin, N., Collomosse, J. (Sep. 29, 2024)

European Conference on Computer Vision (ECCVW'24) Workshop on Vision for Art (VISART VII)

ORAgen: Exploring the Design of Attribution through Media Tokenisation

Liddell, F., Tallyn, E., Morgan, E., Balan, K., Disley, M., Koterwas, T., Dixon, B., Moruzzi, C., Collomosse, J., Elsden, C. (Jul. 14, 2024)

ACM Designing Interactive Systems (DIS)

To Authenticity, and Beyond! Building Safe and Fair Generative AI upon the Three Pillars of Provenance

Parsons, A., Collomosse, J. (Jun. 30, 2024)

IEEE Computer Graphics and Applications (IEEE CG&A)

ProMark: Proactive Diffusion Watermarking for Causal Attribution

Asnani, V., Collomosse, J., Bui, T., Liu, X., Agarwal, S. (Jun. 17, 2024)

CVPR 2024

PARASOL: Parametric Style Control for Diffusion Image Synthesis

Tarres, G., Ruta, D., Bui, T., Collomosse, J. (Jun. 16, 2024)

Computer Vision and Pattern Recognition (CVPRW) Workshop on Women in Computer Vision (WiCV)

VIXEN: Visual Text Comparison Network for Image Difference Captioning

Black, A., Shi, J., Fan, Y., Bui, T., Collomosse, J. (Jan. 14, 2024)

AAAI Conference on Artificial Intelligence (AAAI)

DECORAIT: DECentralized Opt-in/out Registry for AI Training

Balan, K., Black, A., Jenni, S., Gilbert, A., Parsons, A., Collomosse, J. (Dec. 17, 2023)

Best paper award

European Conference on Computer Vision in Media Production (CVMP)

Scene Designer: a Unified Model for Scene Search and Synthesis from Sketch

Ribeiro, L., Bui, T., Collomosse, J., Ponti, M. (Dec. 1, 2023)

Journal on Multimedia Tools and Applications (MTAP)

EKILA: Synthetic Media Provenance and Attribution for Generative Art

Balan, K., Agarwal, S., Jenni, S., Parsons, A., Gilbert, A., Collomosse, J. (Jun. 21, 2023)

CVPR Workshop on Media Forensics (CVPRW)

SceneComposer: Any-Level Semantic Image Synthesis

Zeng, Y., Lin, Z., Zhang, J., Liu, Q., Collomosse, J., Kuen, J., Patel, V. (Jun. 20, 2023)

Computer Vision and Pattern Recognition (CVPR)

RoSteALS: Robust Steganography using Autoencoder Latent Space

Bui, T., Agarwal, S., Yu, N., Collomosse, J. (Jun. 18, 2023)

CVPR Workshop on Media Forensics (CVPRW)

Video Manipulations Beyond Faces: A Dataset with Human-Machine Analysis

Mittal, T., Sinha, R., Swaminathan, V., Collomosse, J., Manocha, D. (Jan. 4, 2023)

IEEE/CVF Winter Conf. on Applications of Computer Vision (WACV-W)

Audio-Visual Contrastive Learning with Temporal Self-Supervision

Jenni, S., Black, A., Collomosse, J., Collomosse, J. (Jan. 4, 2023)

AAAI Conference on Artificial Intelligence (AAAI)

Layout Representation Learning with Spatial and Structural Hierarchies

Bai, Y., Manandhar, D., Wang, Z., Collomosse, J., Fu, Y. (Jan. 4, 2023)

AAAI Conference on Artificial Intelligence (AAAI)

RepMix: Representation Mixing for Robust Attribution of Synthesized Images

Bui, T., Yu, N., Collomosse, J. (Oct. 23, 2022)

European Conference on Computer Vision (ECCV)

CoGS: Controllable Generation and Search from Sketch and Style

Ham, C., Tarres, G., Bui, T., Hays, J., Lin, Z., Collomosse, J. (Oct. 23, 2022)

European Conference on Computer Vision (ECCV)

StyleBabel: Artistic Style Tagging and Captioning

Ruta, D., Gilbert, A., Aggarwal, P., Marri, N., Kale, A., Briggs, J., Speed, C., Jin, H., Faieta, B., Filipkowski, A., Collomosse, J. (Oct. 23, 2022)

European Conference on Computer Vision (ECCV)

HyperNST: Hyper-Networks for Neural Style Transfer

Ruta, D., Gilbert, A., Motiian, S., Faeita, B., Lin, Z., Collomosse, J. (Oct. 22, 2022)

European Conference on Computer Vision Workshop (ECCVW) on Vision and Art (VISART)

ARIA: Adversarially Robust Image Attribution for Content Provenance

Andriushchenko, M., LI, R., Oxholm, G., Gittings, T., Bui, T., Flammarion, N., Collomosse, J. (Jun. 19, 2022)

CVPR 2022, Workshop on Media Forensics

VPN: Video Provenance Network for Robust Content Attribution

Black, A., Bui, T., Jenni, S., Swaminathan, V., Collomosse, J. (Dec. 6, 2021)

International Conference on Computer Vision in Media Production (CVMP)

Defending Object Detection Networks Against Adversarial Patch Attacks

Gittings, T., Schneider, S., Collomosse, J. (Oct. 17, 2021)

ICCV Workshop on Adversarial Robustness in the Real World (ICCVW)

Scene Designer: a Unified Model for Scene Search and Synthesis from Sketch

Ribeiro, L., Bui, T., Collomosse, J., Ponti, M. (Oct. 17, 2021)

ICCV Workshop on Sketching and Human Expressivity (ICCVW)

OSCAR-Net: Object-centric Scene Graph Attention for Image Attribution

Nguyen, E., Bui, T., Swaminathan, V., Collomosse, J. (Oct. 17, 2021)

International Conference on Computer Vision (ICCV)

ALADIN: All Layer Adaptive Instance Normalization for Fine-grained Style Similarity

Ruta, D., Motiian, S., Faeita, B., Lin, Z., Jin, H., Filipkowski, A., Gilbert, A., Collomosse, J. (Oct. 17, 2021)

International Conference on Computer Vision (ICCV)

Deep Image Comparator: Learning to Visualize Editorial Change

Black, A., Bui, T., Jin, H., Swaminathan, V., Collomosse, J. (Jun. 20, 2021)

CVPR Workshop on Media Forensics (CVPRW)

Magic Layouts: Structural Prior for Component Detection in User Interface Designs

Manandhar, D., Jin, H., Collomosse, J., Collomosse, J. (Jun. 6, 2021)

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Neural Architecture Search for Deep Image Prior

Ho, K., Gilbert, A., Jin, H., Collomosse, J. (Jan. 1, 2021)

Elsevier Computers & Graphics (CAG)

Compositional Sketch Search

Black, A., Bui, T., Mai, L., Jin, H., Collomosse, J. (Jan. 1, 2021)

International Conference on Image Processing (ICIP)

Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human Reconstruction

He, T., Collomosse, J., Jin, H., Soatto, S. (Dec. 1, 2020)

Advances in Neural Information Processing Systems (NeurIPS)

DeepVoxels++: Enhancing the Fidelity of Novel View Synthesis from 3D Voxel Embeddings

He, T., Collomosse, J., Jin, H., Soatto, S. (Dec. 1, 2020)

Asian Conference on Computer Vision (ACCV)

Vax-a-Net: Training-time Defence Against Adversarial Patch Attacks

Gittings, T., Schneider, S., Collomosse, J. (Dec. 1, 2020)

Asian Conference on Computer Vision (ACCV)

Semantic Estimation of 3D Body Shape and Pose using Minimal Cameras

Gilbert, A., Trumble, M., Hilton, A., Collomosse, J. (Sep. 9, 2020)

British Machine Vision Conference (BMVC)

Sketchformer: Transformer-based Representation for Sketched Structure

Ribeiro, L., Bui, T., Collomosse, J., Ponti, M. (Jun. 20, 2020)

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

TAPESTRY: A De-centralized Service for Trusted Interaction Online

Yang, Y., Cooper, D., Collomosse, J., Dragan, C., Manulis, M., Steane, J., Manohar, A., Briggs, J., Moncur, W., Jones, H. (May. 31, 2020)

IEEE Transactions on Services Computing (TSC)

Learning Structural Similarity of User Interface Layouts using Graph Networks

Manandhar, D., Ruta, D., Collomosse, J. (Mar. 1, 2020)

European Conference on Computer Vision (ECCV)

Tamper-proofing Video with Hierarchical Attention Autoencoder Hashing on Blockchain

Bui, T., Cooper, D., Collomosse, J., Bell, M., Green, A., Sheridan, J., Higgins, J., Das, A., Keller, J., Thereaux, O. (Jan. 31, 2020)

IEEE Transactions on Multimedia

Real-Time Multi-person Motion Capture from Multi-view Video

Malleson, C., Collomosse, J., Hilton, A. (Dec. 17, 2019)

International Journal on Computer Vision (IJCV)

An Internal Learning Approach to Video Inpainting

Zhang, H., Mai, L., Xu, N., Wang, Z., Collomosse, J., Jin, H. (Oct. 28, 2019)

International Conference on Computer Vision (ICCV) 2019

Robust Synthesis of Adversarial Visual Examples Using a Deep Image Prior

Gittings, T., Schneider, S., Collomosse, J. (Sep. 11, 2019)

British Machine Vision Conference (BMVC) 2019

LiveSketch: Query Perturbation for Guided Sketch-based Visual Search

Collomosse, J., Bui, T., Jin, H. (Jun. 18, 2019)

Conference on Computer Vision and Pattern Recognition (CVPR) 2019

ARCHANGEL: Tamper-proofing Video Archives using Temporal Content Hashes on the Blockchain

Bui, T., Cooper, D., Collomosse, J., Bell, M., Green, A., Sheridan, J., Higgins, J., Das, A., Keller, J., Thereaux, O., Brown, A. (Jun. 17, 2019)

Computer Vision and Pattern Recognition (CVPR) Workshops 2019. Computer Vision, AI and Blockchain.

Fusing Visual and Inertial Sensors with Semantics for 3D Human Pose Estimation

Gilbert, A., Trumble, M., Malleson, C., Hilton, A., Collomosse, J. (Apr. 1, 2019)

International Journal of Computer Vision (IJCV)

In-painting of Wide-baseline Multiple Viewpoint Video

Gilbert, A., Trumble, M., Hilton, A., Collomosse, J., Collomosse, J. (Dec. 24, 2018)

IEEE Transactions on Visualization and Computer Graphics (TVCG)

Deep Manifold Alignment for Mid-grain Sketch based Image Retrieval

Bui, T., Ribeiro, L., Ponti, M., Collomosse, J. (Dec. 2, 2018)

Asian Conference on Computer Vision (ACCV). 2018.

TAPESTRY: Visualizing Interwoven Identities for Trust Provenance

Yang, Y., Collomosse, J., Manohar, A., Briggs, J., Steane, J. (Oct. 22, 2018)

IEEE Symposium on Visualization for Cyber Security (VizSec)

Volumetric performance capture from minimal camera viewpoints

Trumble, M., Gilbert, A., Hilton, A., Collomosse, J. (Sep. 9, 2018)

European Conference on Computer Vision (ECCV). 2018.

Deep Autoencoder for Combined Human Pose Estimation and Body Model Upscaling

Trumble, M., Gilbert, A., Hilton, A., Collomosse, J. (Sep. 8, 2018)

European Conference on Computer Vision (ECCV). 2018.

ARCHANGEL: Trusted Archives of Digital Public Documents

Collomosse, J., Bui, T., Brown, A., Sheridan, J., Green, A., Bell, M., Fawcett, J., Higgins, J., Thereaux, O. (Aug. 29, 2018)

ACM Conference on Document Engineering (DocEng). 2018.

Disentangling Structure and Aesthetics for Content-aware Image Completion

Gilbert, A., Collomosse, J., Jin, H., Price, B. (Jun. 18, 2018)

Conference on Computer Vision and Pattern Recognition (CVPR) 2018

Sketching out the Details: Sketch-based Image Retrieval using Convolutional Neural Networks with Multi-stage Regression

Bui, T., Ribeiro, L., Ponti, M., Collomosse, J. (Apr. 1, 2018)

Elsevier Journal on Computers & Graphics (C&G)

Compact descriptors for sketch-based image retrieval using a triplet loss convolutional neural network

Bui, T., Ribeiro, L., Ponti, M., Collomosse, J. (Nov. 1, 2017)

Journal on Computer Vision and Image Understanding (CVIU)

Sketching with Style: Visual Search with Sketches and Aesthetic Context

Collomosse, J., Bui, T., Wilber, M., Fang, C., Jin, H. (Oct. 22, 2017)

International Conference on Computer Vision (ICCV). 2017.

BAM! The Behance Artistic Media Dataset for Recognition Beyond Photography

Wilber, M., Fang, C., Jin, H., Hertzmann, A., Collomosse, J., Belongie, S. (Oct. 22, 2017)

International Conference on Computer Vision (ICCV). 2017.

Real-time Full-Body Motion Capture from Video and IMUs

Malleson, C., Gilbert, A., Trumble, M., Collomosse, J., Hilton, A., Volino, M. (Oct. 10, 2017)

International Conference on 3D Vision (3DV). 2017.

Total Capture: 3D Human Pose Estimation Fusing Video and Inertial Sensors

Trumble, M., Gilbert, A., Malleson, C., Hilton, A., Collomosse, J. (Sep. 4, 2017)

British Machine Vision Conference (BMVC). 2017.

Comprehensible Video Thumbnails

Kim, J., Gray, C., Asente, P., Collomosse, J. (May. 1, 2015)

Computer Graphics Forum (Proc. Eurographics 2015)

A Particle Filtering Approach to Salient Video Object Localization

Gray, C., James, S., Collomosse, J., Asente, P. (Oct. 1, 2014)

IEEE International Conference on Image Processing (ICIP). 2014.

Special section on the 2011 joint symposium on computational aesthetics (CAe), non-photorealistic animation and rendering (NPAR), and sketch-based interfaces and modeling (SBIM)

Asente, P., Collomosse, J., Isenberg, T., Sezgin, T. (Oct. 1, 2012)

Computers & Graphics Volume 36, Issue 6 (October 2012)

NPAR 2011 – Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Non-Photorealistic Animation and Rendering

Collomosse, J., Asente, P., Spencer, S. (Aug. 5, 2011)

NPAR 2011 – Proceedings of the ACM/Eurographics Symposium on Non-Photorealistic Animation and Rendering

News