Small theta and the text sequence
#69
by liuliu87 - opened
Hi! Just want to call out that the implementation and training used a small theta (256) which is a good choice for image dimensions but not necessary for text dimension (as it will rotate back at 256 token interval). However, FLUX.1 uses all (0, 0, 0) encoding for text and works fine so it might not be a big issue. I will dig deeper to see if techniques post-stretch text sequence would be beneficial.