Skip to content

Fashion Search

PRICING DEMO

With Fashion Search, your customers can take real-life pictures & find visually similar products in your clothing e-commerce (we call it "visual search"). It also supports multimodal & multilingual text to image search on your product database. Moreover, the Fashion Search service integrates three services that are often requested by our apparel & fashion customers - Fashion Tagging (Categorization), Product Similarity and Dominant Colors.

Screenshot

How it works

Before searching you will need to create a search collection via Ximilar platform. For this you will need to have at least a business plan activated. This will give you unique identifier of the collection that you need to use in your requests. After that you will need to first insert some images (records with metadata and id) into the collection.

The API is on the following URLs:

https://api.ximilar.com/similarity/fashion-new/v2/<method> https://api.ximilar.com/similarity/text/fashion/v2/text

and it contains all methods from Photo & Product Similarity.

Python Library

If you are using Python programming language in your project you can query our API with our library. Our library is available on gitlab and through the pypi. For the Fashion Search, use class ximilar.client.search.SimilarityFashionClient.

Things to note

  • For insert (/v2/insert) you can specify maximum of 10 records per request/batch size in "records" field
  • Search (/v2/visualKNN) is only possible with one record (photo) in field "query_record"
  • Text search is available via endpoint https://api.ximilar.com/similarity/text/fashion/v2/text

You store your product image database into a collection in Ximilar cloud in the same way you do with in Photo & Product Similarity. When a product photo is inserted (/v2/insert) into the collection, Ximilar process the image like this:

  1. Detect apparel (fashion) items on the photo.
  2. Optionally analyze colors of the product and get tags.
  3. Extract the visual information (embedding) from this largest detected item.
  4. Insert the embedding and meta-data for image to the collection.

You can keep the collection synchronized yourself via the insert, update and delete API methods.

Sometimes it is difficult to correctly estimate the main item on the product image and its category. You can help Ximilar with this by sending over the information about the Top Category (and Category) of the product. You must use the categories according to Ximilar Fashion Taxonomy and put the information into each inserted image record, e.g.

{
  "records": 
  [
    {
      "product_id": "ABC",
      "_id": "ABC_1",
      "_url": "https://example.com/product/image/url.png",
      "Top Category": "Clothing",
      "Category": "Clothing/Pants"        
    }
  ] 
}

If you do this, the insert operation will not necessarily select the largest object detected but the largest object with the matching Top Category (and Category). In case there is no such object detected in the image, Ximilar considers the whole image as fashion item and uses Fashion Tagging method to determine its categories and tags.

You can keep the collection synchronized yourself via the insert and delete API methods. The other option is let Ximilar do this for you - run a regular process (e.g. nightly) that:

  • downloads your current database export (in any format agreed on) from a given URL,
  • calculates the difference between the export and your collection
  • inserts new products to the collection
  • removes stale products from the collection.

Ximilar must know the format of your data and we can together define a mapping between your categories and Ximilar taxonomy and thus specify the top category (and category) of the main item during the insert operation (see above). We usually charge a small fee for implementation of the synchronization script (depending on complexity and format of the export).

Contact us at tech@ximilar.com and we will arrange this regular sync for you.

If you internally want to use also dominant colors and tags, just send the extract_colors and extract_tags flags in the request. You can specify your own tags in the field _tags_simple for records when calling insert or search method. If needed our Fashion Tagging can generate also tags for your images. However extracting colors and tags are additionally billed according to our pricing

{
    "records": [
        {
            "_id": "__IMAGE_ID__",
            "_url": "__IMAGE_URL__",
            "extract_colors": true,
            "extract_tags": true,
            "_tags_simple": ["yellow", "stilleto", "women"]
        }
    ]
}

If you wan to insert multple images per one product specify same product_id field for each record with the identification number of product. Specify this product_id field also in the collection settings via app.ximilar.com.

{
    "records": [
        {
            "_id": "__IMAGE_ID__",
            "_url": "__IMAGE_URL__",
            "product_id": "1",
        },
        {
          "_id_": "__IMAGE_ID__",
          "_url": "__IMAGE_URL__",
          "product_id": "1"
        },
        {
          "_id_": "__IMAGE_ID__",
          "_url": "__IMAGE_URL__",
          "product_id": "1"
        }
    ]
}

Get similar fashion products

You can get similar item recommendations in the same way as you would with image similarity using method visualKNN and visualTagsKNN:

https://api.ximilar.com/similarity/fashion-new/v2/visualKNN
https://api.ximilar.com/similarity/fashion-new/v2/visualTagsKNN

The method /v2/visualTagsKNN can combine the visual similarity with the similarity of tags and colors if they are present in stored and search records (tags and colors are optional).

Internally, the system also uses a default filter that is automatically internally added to the filter parameter specified in the API methods. Currently this default filter is:

{ 
  "filter": { 
    "Top Category": "{TAKE_FROM_QUERY}",
    "Category": "{TAKE_FROM_QUERY}"
  }
}

Visual search by real photo

Finally, let us see how to get visual search results like in this demo. It is done via the same method as getting similar collection items:

https://api.ximilar.com/similarity/fashion-new/v2/visualKNN

This method returns:

  • all fashion items detected in the query photo - it is in field query_records[0]._objects,
  • and products from your collection that are similar to the largest item detected - standard fields answer_records and distances.

CLICK TO SHOW JSON RESPONSE EXAMPLE 

 {
  "query_records": [
     {
       "_id": "49831",
       "_url": "https://example.com/image.png",
       "_objects": [
         {
           "area": 0.696093738079071,
           "prob": 0.9520546793937683,
           "Top Category": "Clothing",
           "Category": "Clothing/Jackets and Coats",
           "bound_box": [
             49,
             3,
             346,
             378
           ],
           "_tags": {
             "detailed tag list as returned from detect_tags": true
           },
           "_tags_simple": [
              "simple tag list as returned from detect_tags"
           ]
         },
         {
           "area": 0.376093738079071,
           "prob": 0.9120546793937683,
           "Top Category": "Clothing",
           "Category": "Clothing/Pants",
           "bound_box": [
             349,
             23,
             646,
             378
           ]
         }
       ]
     }
   ],
   "answer_records": [
     {
       "_id": "294215",
       "_url": "https://images.ximilar.com/fashion/Clothing/Pullover/Knit sweater/v70kjnwe53sz4may/v70kjnwe53sz4may_1.jpg"
     },
     {
       "_id": "426897",
       "_url": "https://images.ximilar.com/fashion/Clothing/Pullover/Cardigans/tjp1xeo84f67yu3m/tjp1xeo84f67yu3m_1.jpg"
     },
     {
       "_id": "148430",
       "_url": "https://images.ximilar.com/fashion/Clothing/Pullover/kebwxtnm84c7lhf0/kebwxtnm84c7lhf0_1.jpg"
     },
     {
       "_id": "148232",
       "_url": "https://images.ximilar.com/fashion/Clothing/Pants/Jumpsuits/pw369sxkearg08ot/pw369sxkearg08ot_1.jpg"
     },
     {
       "_id": "1364490",
       "_url": "https://images.ximilar.com/fashion/Clothing/Swimwear/Beach clothing/row0ztdqnj897li3/row0ztdqnj897li3_1.jpg"
     }
   ],
   "answer_distances": [
     0.64635134,
     0.65307707,
     0.66936225,
     0.6757167,
     0.6777565
   ],
   "answer_count": 5,
   "status": {
     "code": 200,
     "text": "OK"
   },
   "statistics": {
     "OperationTime": 19
   }
}

Visual search by text query

Performs multilingual text search (visual similarity/knn) on your collection of images (collection is specified in header) via text query, sometimes also called as neural text search or text-to-image search.

Parameters:

  • query_record: must contain a _text_data field with a sentence or word as text query
  • fields_to_return: list of fields (metadata) to return in results
  • filter: mongodb filtering on fields (metadata) that was stored with the images/records
  • k: number of results to return

Example:

curl --request POST \
  --url https://api.ximilar.com/similarity/text/fashion/v2/text \
  --header 'authorization: Token __APITOKEN__' \
  --header 'collection-id: __YOURCOLLECTIONID__' \
  --header 'content-type: application/json' \
  --data '{
    "query_record": {
        "_text_data": "a blue pullower with white stripes pattern",
    },
    "filter": {
        "supplierid": {
            "$lte": 600
        }
    },
    "fields_to_return": [
        "_id",
        "_url",
        "supplierid"
    ],
    "k": 3
}'

Returns:

CLICK TO SHOW JSON RESPONSE EXAMPLE 

{
    "status": {
        "code": 200,
        "text": "OK",
        "proc_id": "d820b7a9-4b93-4a85-a4ed-d96731bc471e"
    },
    "statistics": {
        "OperationTime": 244,
        "processing time": 0.31435346603393555
    },
    "answer_records": [
        {
            "_id": "65732555",
            "_url": "_URL_1_",
            "supplierid": 450
        },
        {
            "_id": "88182211",
            "_url": "_URL_2_",
            "supplierid": 451
        },
        {
            "_id": "106629458",
            "_url": "_URL_3_"
            "supplierid": 200
        }
    ],
    "answer_distances": [
        1.1622182,
        1.165117,
        1.1667057
    ],
    "answer_count": 3
}

Search on specific object

If you want to get products similar to any other object than the main (largest) one, you can do it by simply passing the query object with the field _objects pre-filled with the secondary object. The query might then look like this:

$ curl --request POST \
    --url https://api.ximilar.com/similarity/fashion-new/v2/visualKNN \
    --header 'authorization: Token __API_TOKEN__' \
    --header 'collection-id: __COLLECTION_ID__' \
    --header 'content-type: application/json' \
    --data '{
    "query_record": {
       "_url": "https://example.com/image.png",
       "_objects": [
         {
           "area": 0.376093738079071,
           "prob": 0.9120546793937683,
           "Top Category": "Clothing",
           "Category": "Clothing/Pants",
           "bound_box": [
             349,
             23,
             646,
             378
           ]
         }
       ]
    },
    "fields_to_return": [ "_id", "_url", "product_id" ]
  }'

from ximilar.client.search import SimilarityFashionClient

client = SimilarityFashionClient(token='__API_TOKEN__', collection='__COLLECTION_ID__')

result = client.search([{"_url": "__URL_PATH_TO_IMAGE__", "_objects": [ 
         {
           "area": 0.376093738079071,
           "prob": 0.9120546793937683,
           "Top Category": "Clothing",
           "Category": "Clothing/Pants",
           "bound_box": [
             349,
             23,
             646,
             378
           ]
         }
    ]
}])

Just make sure that you copy the whole JSON record of the object (all fields) from the API response.

You can combine the Fashion Detection method and the visual search like this:

  1. call the Fashion Detection method to get all fashion items in the photo (without directly getting similar products),
  2. let your user select the item they are interested in,
  3. use the object JSON record in the subsequent visual search query.

Draw a bounding box in mobile app?

Do you want to use fashion search in your mobile app and let user to draw the object? Just specify it in following way. The format of bounding box is [xmin, ymin, xmax, ymax].

{
    "records": [
        {
            "_url": "__URL_IMAGE__",
            "_objects": [
                {
                    "bound_box": [
                        701,
                        1760,
                        1155,
                        1952
                    ]
                }
            ]
        }
    ]
}