Today, I studied a graph-to-graph generation algorithm from Tencent AI Lab. The paper was published on August 13, 2023, and the link is as follows: (https://arxiv.org/abs/2308.06721).
The project is named IP-Adapter, and the project page is: (https://ip-adapter.github.io/).
Unlike previous Text-to-Image solutions, the core function of IP-Adapter is Image-to-Image. In short, it is an efficient and lightweight adapter designed to add image prompting capabilities to pre-trained text-to-image Diffusion models.
Its core design concept is: decoupled cross-attention mechanism that independently processes the cross-attention layers for text and image features.
In addition, IP-Adapter has the following advantages:
Despite having only 22M parameters, it can match or even surpass the performance of fine-tuned image-prompting models. (The last column is can be applied to other models fine-tuned from the same base model. Supports controllable generation using existing tools. can elegantly combine image and text prompts to accomplish multimodal image generation.
The specific technical solution is as follows (image prompt is "Girl with a Pearl Earring", text prompt is "A girl with sunglasses").
