Fashion Search
With Fashion Search, your customers can take real-life pictures & find visually similar products in your clothing e-commerce (we call it "visual search"). Moreover, the Fashion Search service integrates three services that are often requested by our apparel & fashion customers - Fashion Tagging, Product Similarity and Dominant Colors.
How it works
- Your database is synchronized with a collection in Ximilar cloud (see below).
- You can get fashion tags for each product in your database (see below).
- You can get similar items to display alternative products to your customers (see below).
- You can send a real-life photo (e.g. from a cell phone), let the system detect apparel items on the photo and get the closest matching items from your collection (see below).
The API is on the following URL:
https://api.ximilar.com/similarity/fashion/v2/<method>
and it contains all methods from Photo & Product Similarity.
Python Library
If you are using Python programming language in your project you can query our API with our library. Our library is available on gitlab and through the pypi. For the Fashion Search, use class ximilar.client.search.SimilarityFashionClient.
Things to note
- For insert (/v2/insert) you can specify maximum of 10 records per request/batch size in
"records"field - Search (/v2/visualTagsKNN) is only possible with one record (photo) in field
"query_record" - We support a lot of customization as renaming/translation of tags, advanced filtering, ...
You store your product image database into a collection in Ximilar cloud in the same way you do with in Photo & Product Similarity. When a product photo is inserted into the collection, Ximilar process the image like this:
- detect apparel items on the photo,
- run Fashion Tagging on the largest detected item (see Apparel Detection & Tags),
- extract the visual information from this largest detected item,
- insert the image with the fashion tags to the collection.
Ximilar can do the sync
You can keep the collection synchronized yourself via the insert and delete API methods. The other option is let Ximilar do this for you - run a regular process (e.g. nightly) that:
- downloads your current database export (in any format agreed on) from a given URL,
- calculates the difference between the export and your collection
- inserts new products to the collection
- removes stale products from the collection.
Ximilar must know the format of your data and we can together define a mapping between your categories and Ximilar taxonomy and thus specify the top category (and category) of the main item during the insert operation (see above). We usually charge a small fee for implementation of the synchronization script (depending on complexity and format of the export).
Contact us at tech@ximilar.com and we will arrange this regular sync for you.
Sometimes it is difficult to correctly estimate the main item on the product image and its category. You can help Ximilar with this by sending over the information about the Top Category (and Category) of the product. You must use the categories according to Ximilar Fashion Taxonomy and put the information into each inserted image record, e.g.
{
"records":
[
{
"product_id": "ABC",
"_id": "ABC_1",
"_url": "https://example.com/product/image/url.png",
"Top Category": "Clothing",
"Category": "Clothing/Pants"
}
]
}
If you do this, the insert operation will not necessarily select the largest object detected but the largest object with the matching Top Category (and Category). In case there is no such object detected in the image, Ximilar considers the whole image as fashion item and uses Fashion Tagging method to determine its categories and tags.
Get fashion tags
As mentioned above, each product photo is processed by the Apparel Detection & Tags method. The response from this method is in the record stored in your Ximilar collection. There are two ways to get to this output:
- When you call the insert API operation, the response from this operation contains also the JSON response from the Apparel Detection & Tags method.
- You can use method /v2/get to get the fashion tags stored in individual records. See example of such response below.
CLICK TO SHOW JSON RESPONSE EXAMPLE
{
"records": [
{
"_url": "__URL_PATH__",
"_objects": [
{
"name": "Clothing",
"id": "8e6cf45d-d416-4560-a89a-1b83b80e003b",
"bound_box": [
266,
281,
544,
707
],
"prob": 0.9437555074691772,
"area": 0.20078531901041666,
"Top Category": "Clothing",
"_tags": {
"Category": [
{
"prob": 0.99295,
"name": "Clothing/Pants",
"id": "7e3a6f68-f13a-417c-9613-16e6045b4020"
}
],
"Color": [
{
"prob": 0.53784,
"name": "black",
"id": "ca5a002b-d206-45a3-b586-820539845a5e"
},
{
"prob": 0.46087,
"name": "dark blue and navy",
"id": "848d99a2-0f52-4ecf-8ca4-947ee55e1c36"
}
],
"Style": [
{
"prob": 1.0,
"name": "elegant",
"id": "8ec36318-6a71-41f6-90e6-6f8eba3dad2a"
}
],
"Subcategory": [
{
"prob": 0.99998,
"name": "suit trousers",
"id": "87ea03a2-040f-4b21-8b12-33e3b5f04261"
},
{
"name": "formal trousers",
"prob": 0.99295
}
],
"Pattern": [
{
"prob": 0.95253,
"name": "plain",
"id": "5065ee09-6fa1-48f3-834f-eced387f46a8"
}
],
"Gender": [
{
"prob": 0.38094,
"name": "men",
"id": "724afe83-bfb7-4f18-aeeb-a7511feeb9b1"
},
{
"prob": 0.32215,
"name": "women",
"id": "6d310c15-258f-40e7-9ced-d8be7059b3e9"
}
],
"Length": [
{
"prob": 0.97548,
"name": "long",
"id": "bfca0daa-2ef7-4eff-892d-2021087064a9"
}
],
"Fit": [
{
"prob": 0.74897,
"name": "slim",
"id": "ebb81e52-8a41-450e-988c-58003a714286"
}
],
"Top Category": [
{
"id": "13052be8-2cc7-48e4-a782-a00d658f997c",
"name": "Clothing",
"prob": 1.0
}
]
},
"_tags_simple": [
"elegant",
"long",
"Clothing/Pants",
"plain",
"slim",
"Clothing",
"formal trousers",
"suit trousers",
"black"
],
"Category": "Clothing/Pants"
},
{
"name": "Clothing",
"id": "8e6cf45d-d416-4560-a89a-1b83b80e003b",
"bound_box": [
208,
33,
588,
356
],
"prob": 0.886322557926178,
"area": 0.2080959743923611,
"Top Category": "Clothing",
"Category": "Clothing/Jackets and Coats"
}
]
}
],
"status": {
"code": 200,
"text": "OK",
"request_id": "fd114fb0-dff9-4851-842c-bf9bc4ce856d",
"proc_id": "55669f4d-79f8-4957-aaf5-0be301252708"
},
"statistics": {
"processing time": 2.2415950298309326
}
}
Get similar fashion products
You can get similar item recommendations in the same way as you would with image similarity using method
https://api.ximilar.com/similarity/fashion/v2/visualTagsKNN
The method combines the visual similarity with the similarity of tags from the fashion tagging (it is stored in field _tags_simple). Internally, the system also uses a default filter that is automatically internally added to the filter parameter specified in the API methods.
Currently this default filter is:
{ "filter": { "Top Category": "{TAKE_FROM_QUERY}", "Category": "{TAKE_FROM_QUERY}" } }
This default filter might change in future but it does not influence the overall behaviour of the search.
Visual search by real photo
Finally, let us see how to get visual search results like in this demo. It is done via the same method as getting similar collection items:
https://api.ximilar.com/similarity/fashion/v2/visualTagsKNN
This method returns:
- all fashion items detected in the query photo - it is in field
query_records[0]._objects, - and products from your collection that are similar to the largest item detected - standard fields
answer_recordsanddistances.
CLICK TO SHOW JSON RESPONSE EXAMPLE
{
"query_records": [
{
"_id": "49831",
"_url": "https://example.com/image.png",
"_objects": [
{
"area": 0.696093738079071,
"prob": 0.9520546793937683,
"Top Category": "Clothing",
"Category": "Clothing/Jackets and Coats",
"bound_box": [
49,
3,
346,
378
],
"_tags": {
"detailed tag list as returned from detect_tags": true
},
"_tags_simple": [
"simple tag list as returned from detect_tags"
]
},
{
"area": 0.376093738079071,
"prob": 0.9120546793937683,
"Top Category": "Clothing",
"Category": "Clothing/Pants",
"bound_box": [
349,
23,
646,
378
]
}
]
}
],
"answer_records": [
{
"_id": "294215",
"_url": "https://images.ximilar.com/fashion/Clothing/Pullover/Knit sweater/v70kjnwe53sz4may/v70kjnwe53sz4may_1.jpg"
},
{
"_id": "426897",
"_url": "https://images.ximilar.com/fashion/Clothing/Pullover/Cardigans/tjp1xeo84f67yu3m/tjp1xeo84f67yu3m_1.jpg"
},
{
"_id": "148430",
"_url": "https://images.ximilar.com/fashion/Clothing/Pullover/kebwxtnm84c7lhf0/kebwxtnm84c7lhf0_1.jpg"
},
{
"_id": "148232",
"_url": "https://images.ximilar.com/fashion/Clothing/Pants/Jumpsuits/pw369sxkearg08ot/pw369sxkearg08ot_1.jpg"
},
{
"_id": "1364490",
"_url": "https://images.ximilar.com/fashion/Clothing/Swimwear/Beach clothing/row0ztdqnj897li3/row0ztdqnj897li3_1.jpg"
}
],
"answer_distances": [
0.64635134,
0.65307707,
0.66936225,
0.6757167,
0.6777565
],
"answer_count": 5,
"status": {
"code": 200,
"text": "OK"
},
"statistics": {
"OperationTime": 19
}
}
If you want to get products similar to any other object than the main (largest) one, you can do it by simply passing the query object with the field _objects pre-filled with the secondary object. The query might then look like this:
$ curl --request POST \
--url https://api.ximilar.com/similarity/fashion/v2/visualTagsKNN \
--header 'authorization: Token __API_TOKEN__' \
--header 'collection-id: __COLLECTION_ID__' \
--header 'content-type: application/json' \
--data '{
"query_record": {
"_url": "https://example.com/image.png",
"_objects": [
{
"area": 0.376093738079071,
"prob": 0.9120546793937683,
"Top Category": "Clothing",
"Category": "Clothing/Pants",
"bound_box": [
349,
23,
646,
378
]
}
]
},
"fields_to_return": [ "_id", "_url", "product_id" ]
}'
from ximilar.client.search import SimilarityFashionClient
client = SimilarityFashionClient(token='__API_TOKEN__', collection='__COLLECTION_ID__')
result = client.search([{"_url": "__URL_PATH_TO_IMAGE__", "_objects": [
{
"area": 0.376093738079071,
"prob": 0.9120546793937683,
"Top Category": "Clothing",
"Category": "Clothing/Pants",
"bound_box": [
349,
23,
646,
378
]
}
]
}])
Just make sure that you copy the whole JSON record of the object (all fields) from the API response.
You can combine the Apparel Detection method and the visual search like this:
- call the Apparel Detection method to get all fashion items in the photo (without directly getting similar products),
- let your user select the item they are interested in,
- use the object JSON record in the subsequent visual search query.
Customization and special features
Following examples are for /v2/insert endpoint. If you want to use this in search (/v2/visualTagsKNN) please use "query_record" dictionary instead of "records" list.
Connect custom flow
Ximilar Custom Image Recognition allows you to train image categorization or tagging models. With Ximilar Flows you can connect these models to powerful complex vision systems with hierarchy. During the insert process to your collection or during the search/query you can call your trained models with Flows. This will add more knowledge/meta information to your product images. This is great if you want to recognize some special tags which are not included in our Fashion Tagging taxonomy. It is working by specifing custom_flow field in your request. There are several requirements for the flow which must be satisfied in order to run it successfully. Just write us and we will discuss the usage with you.
{
"records": [
{
"_url": "__URL_IMAGE__",
"_tags_custom": ["men"]
}
],
"custom_flow": "__FLOW_ID__"
}
Use custom profile
Would you like to rename some of our tags or do some other modifications? For example if our tagging system says that the product is 'Dress', 'black', 'mini' we can generate 'black mini dress' tag. This is done by creating custom profile for you. Just write us and we will discuss the usage with you.
{
"records": [
{
"_url": "__URL_IMAGE__",
"_tags_custom": ["men"]
}
],
"profile": "__FLOW_ID__"
}
Use custom tags and gender tags
Do you know if your product is for men, women, children or unisex? Then specify this in a record field _tags_custom. You can also add any your tag to _tags_custom.
{
"records": [
{
"_url": "__URL_IMAGE__",
"_tags_custom": ["men"]
}
]
}
Draw a bounding box in mobile app?
Do you want to use fashion search in your mobile app and let user to draw the object? Just specify it in following way. The format of bounding box is [xmin, ymin, xmax, ymax].
{
"records": [
{
"_url": "__URL_IMAGE__",
"_objects": [
{
"bound_box": [
701,
1760,
1155,
1952
]
}
]
}
]
}