CLIPS: An Enhanced CLIP Framework for Learning with Synthetic …
2024年11月25日 · Abstract page for arXiv paper 2411.16828: CLIPS: An Enhanced CLIP Framework for Learning with Synthetic Captions Previous works show that noisy, web-crawled image-text pairs may limit vision-language pretraining like CLIP and propose learning with synthetic captions as a promising alternative.
jina-clip-v2: Multilingual Multimodal Embeddings for Text and …
2024年12月11日 · The resulting model, jina-clip-v2, outperforms its predecessor on text-only and multimodal tasks, while adding multilingual support, better understanding of complex visual documents and efficiency gains thanks to Matryoshka …
[2309.16671] Demystifying CLIP Data - arXiv.org
2023年9月28日 · Abstract page for arXiv paper 2309.16671: Demystifying CLIP Data Contrastive Language-Image Pre-training (CLIP) is an approach that has advanced research and applications in computer vision, fueling modern recognition systems and generative models.
A Sober Look at the Robustness of CLIPs to Spurious Features
2024年3月18日 · Abstract page for arXiv paper 2403.11497: A Sober Look at the Robustness of CLIPs to Spurious Features Large vision language models, such as CLIP, demonstrate impressive robustness to spurious features than single-modal models trained on ImageNet.
ConVIRT trained from scratch, which we call CLIP, for Con-trastive Language-Image Pre-training, is an efficient method of learning from natural language supervision. We study the scalability of CLIP by training a series of eight models spanning almost 2 orders of magnitude of compute and ob-serve that transfer performance is a smoothly predictable
GALIP: Generative Adversarial CLIPs for Text-to-Image Synthesis
2023年1月30日 · To enable high-quality, efficient, fast, and controllable text-to-image synthesis, we propose Generative Adversarial CLIPs, namely GALIP. GALIP leverages the powerful pretrained CLIP model both in the discriminator and generator. Specifically, we propose a CLIP-based discriminator.
[2311.00453] CLIP-AD: A Language-Guided Staged Dual-Path …
2023年11月1日 · Abstract: This paper considers zero-shot Anomaly Detection (AD), performing AD without reference images of the test objects. We propose a framework called CLIP-AD to leverage the zero-shot capabilities of the large vision-language model CLIP.
[2403.01560] Rethinking CLIP-based Video Learners in Cross …
2024年3月3日 · To answer this, we establish a CROSS-domain Open-Vocabulary Action recognition benchmark named XOV-Action, and conduct a comprehensive evaluation of five state-of-the-art CLIP-based video learners under various types of domain gaps.
[2410.13016] Interpreting and Analysing CLIP's Zero-Shot Image ...
2024年10月16日 · Abstract: Contrastive Language-Image Pretraining (CLIP) performs zero-shot image classification by mapping images and textual class representation into a shared embedding space, then retrieving the class closest to the image. This work provides a new approach for interpreting CLIP models for image classification from the lens of mutual ...
[2410.23698] Aggregate-and-Adapt Natural Language Prompts for ...
2024年10月31日 · Prompt learning offers a parameter-efficient finetuning framework that can adapt CLIP to downstream tasks even when limited annotation data are available. In this paper, we improve prompt learning by distilling the textual knowledge from natural language prompts (either human- or LLM-generated) to provide rich priors for those under-represented ...