I am thinking about building an index (even a simple list stored in a file) of classes annotated with a certain annotation type at compile time with an annotation Processor in order to speed up annotated class retrieval at runtime.
So, is it a good practice? Are there any drawbacks? If it is as good as it seems to me now, why aren't there many libraries to do this in an easy way (the only one I have found is Class Index)? Instead for runtime processing there are so many?
I think the main disadvantages is that it's more complicated. Annotation processing is a whole new API and concept that many developers aren't familiar with. The Reflection API is easier and more well known. You can usually accomplish the same tasks at runtime.
If better startup performance is crucial (which is rarely the case) then maybe it's worth the added complexity.
I wouldn't trust the benchmarks though. They state "classpath size was set to 121MB" - an arbitrary value that makes any comparison to hard coded or compile time processing completely useless. Why would you want to scan the whole class path anyway? Scanning only the developers classes would be more reasonable in most cases.
Many frameworks use configuration files or have an API to limit the classes or packages that need to be scanned. This increases startup time significantly.
why aren't there many libraries to do this
Many OSGi tools/frameworks do this. Annotations are scanned at compile time and meta-data is written to the jar manifest file or they create more sophisticated meta-data files. I suspect the main reason for this is to keep compatibility with bnd and similar tools, which have been used to built and compile time analysis of OSGi components before annotations or annotation processing got more popular. Also, OSGi component have their own lifecycle and can come and go at any time. So this is a case where startup time does matter a bit more, as you can't only scan once at application start. You would need to scan for annotations whenever a component (re-)starts.
I wouldn't say it's a good nor bad practice. Use this technique when it fits your needs. I would avoid adding to much complexity for the sake of a few milliseconds startup time.