The GIFT (the GNU Image-Finding Tool) is a Content Based Image Retrieval System (CBIRS: http://en.wikipedia.org/wiki/CBIR). It enables you to do Query By Example (QBE: http://en.wikipedia.org/wiki/QBE) on images, giving you the opportunity to improve query results by relevance feedback. For processing your queries the program relies entirely on the content of the images, freeing you from the need to annotate all images before querying the collection.
The GIFT is an open framework for content-based image retrieval. We explicitly have taken into account the possibility of adding new ways of querying to the framework. Our communication protocol for client-server communication, MRML, is XML based and fully documented (http://www.mrml.net). This aims at promoting code reuse among researchers and application developers. Inquiries about APIs and how to integrate new clients/servers into the framework are welcome (send questions to http://mail.gnu.org/mailman/listinfo/help-gift or the maintainer mailto:Wolfgang.Mueller@wiai.uni-bamberg.de).
Also, see this review article on Query Based Image Retrieval in Medical Sciences