Skip to content

Smart Product Search

This page describes API of the Smart Product Search service which takes a real-life photo, detects (mostly fashion) products on this photo and finds visually similar images in the product collection.

The API follows the general rules of Ximilar API as described in Section First steps.

API is a set of HTTP REST services accepting JSON-formatted documents using POST and returning JSON documents. These services run at these URLs:

https://api.ximilar.com/smart/v2/<method>

Please, see page API Calls and Credits for exact prices for individual API calls. The Smart Product Search is a complex service where each API call typically consists of detection, tagging, feature extraction and similarity search and thus the calls cost more credits than our simple services.

Contact us before using this service

In order to get access to the Proto & Product Similarity services, please register at https://app.ximilar.com and then contact us at tech@ximilar.com to make the service accessible for your Ximilar account.

Overview of API Methods

You always work with your collection of images, which is specified in the collection header of each API call. Your collection(s) must be first created by Ximilar administrators.

There are also methods for management of the records already inserted into the system. The API is exactly the same as Ximilar Search API: Ximilar Search API.

The response of method /v2/product/insert returns the type of product for each record, e.g. "_obj_type": "clothing" (see below). There are separate indexes (at this point three) for each type of the object: clothing, footwear, watch and the collection management methods are available at separate URLs for each type:

https://api.ximilar.com/smart/<type_of_object>

where <type_of_object> is clothing, footwear, watch

Showing bounding box of the object

You minght want to display bounding boxes of the detected objects in your application. Here is an example of small python snippet with opencv library which will help you:

import cv2

# get bounding box of first record and first object of the record
bbox = result['records'][0]['_objects'][0]['bound_box']

# read the same image in opencv
image = cv2.imread('photo.jpg', 0)
image = cv2.rectangle(image, (bbox[0], bbox[1]), (bbox[2], bbox[3]), (255,0,0), 2)

cv2.imwrite('photo_bbox.jpg', image)

Detailed Descriptions of API Methods

/v2/product/insert

You call this insert method to fill the smart product index with your data. Unlike the standard /v2/insert, this method first detects objects on the image, cuts the rectangular area with the object out of the image, extracts visual descriptor from the area, assigns some tags and only then inserts the image into the index. The inserted records can also contain additional meta data for consequent filtering.

Parameters:​

  • records: A list of product photos to be inserted into the collection; each record
    • must contain either of ​_url, _base64
    • should contain field ​category ​that tells the product category; the values are from customer dictionary (Ximilar has mapping of customer categories to Ximilar products for each collection)

curl https://api.ximilar.com/products/v2/product/insert
  -H "Content-Type: application/json" 
  -H 'Authorization: Token 1af538baa90-----XXX-----baf83ff24'
  -H 'collection: mycoll'
  -d '{ 
    "records": [
      { "_id": "1", "_url": "http://url.to/product/photo.jpg", "product_id": "Zara T-Shirt 1234", "category": "TSHIRTS" },
      { "_id": "2", "_base64": "ABD432...", "product_id": "Adidas Shoes 1", "category": "RUNNING_SHOES" }
    ]
  }'

The response is very similar as the one of method /v2/insert. The returned records contain also the detected objects with their tags and also parameter "_obj_type" with possible values: "clothing", "footwear", "watch".

CLICK TO SHOW JSON RESULT

{
  "status": {"text": "OK", "code": 200},
  "records": [
    {
      "_id": "1",
      "_url": "http://url.to/product/photo.jpg", 
      "product_id": "Zara T-Shirt 1234",
      "_obj_type": "clothing",
      "_objects": [
        {
          "name": "Clothing/Upper",
          "prob": 0.99,
          "bound_box": [xmin, ymin, xmax, ymax]
        }
      ],
      "_width": 1200,
      "_height": 1100
    },
    { 
      # "_base64" field is omitted from the answer
      "_id": "2",
      "product_id": "Adidas Shoes 1",
      "_obj_type": "footwear",
      "_objects": [
        {
          "name": "footwear",
          "prob": 0.97,
          "bound_box": [xmin, ymin, xmax, ymax]
        }
      ],
      "_width": 900,
      "_height": 700
    }
  ],
  "statistics": {"processing time": 0.8932}
}

/v2/product/search

This is main method of the smart product search for finding most similar item from the picture. If more than 1 object is detected then search will return similar items for most dominant item (with largest area or probability, mode param).

Parameters:

  • records - a list that should contain only ONE real-life photo to find similar products for
  • mode - modes for searching (default: area)
    • area: find similar products for the detected object with largest area (and probability >= 0.9)
    • prob: find similar products for the detected object with the highest detection probability
  • k with default value 10
  • from with default value 0
  • fields_to_return additional fields to be returned for each records in the answer. By default only these fields are returned: [ _id, _url, _file, _objects ]
  • filter - additional filter to be applied during the search (besides collection and type of detected object); default value: null

See this Product & Product Similarity for details on the search params.

curl https://api.ximilar.com/smart/v2/product/search 
  -H "Content-Type: application/json" 
  -H "Authorization: Token 713-----XXXXXXXXXXXXXXXXX----0e19beb" 
  -H "collection: mycoll"
  -d '{
    "records": [ { "_url": "http://url.to/real_life/photo.jpg" } ],
    "mode": ["area"|"prob"],
    "k": 2, 
    "from": 0,
    "filter": { "category": "external_category" }, 
    "fields_to_return": [ "_id", "category", "_url", "_product_id", "_objects" ] 
  }

Similar items are present in answer_records field with distances to the query object in answer_distances.

CLICK TO SHOW JSON RESULT

{
  "status": {"text": "OK", "code": 200},
  "records": [
    {
      "_url": "http://url.to/real_life/photo.jpg", 
      "_objects": [
        {
          "name": "Clothing/Upper",
          "prob": 0.95,
          "bound_box": [xmin, ymin, xmax, ymax],
          "_tags": [
            {"prob": 0.99971, "name": "Sleeves long"}, 
            {"prob": 0.88038, "name": "Pullovers"}, 
            {"prob": 1.0, "name": "Women"}
          ], 
          "_tags_simple": ["Sleeves long", "Pullovers", "Women"]
        }
      ],
      "answer_records": [ 
        { 
          "_id": "12",
          "_url": "http://url.to/product/photo1.jpg", 
          "product_id": "12345", 
          "_objects": [ { "name": "Clothing/Upper", "prob": 0.99, "bound_box": [xmin, ymin, xmax, ymax] } ] 
        },
        { 
          "_id": "16",
          "_url": "http://url.to/product/photo2.jpg", 
          "product_id": "12346", 
          "_objects" [ ... ] 
        }
      ],
      "answer_distances": [ 0.73, 0.71 ]
    }
  "statistics": {"processing time": 0.8932}
}

/v2/product/detect

You can use this method to detect all product object on your real-life photo and get tags for these objects. Subsequently, you can pick one object from '_objects' field and put it into the request body of method for /v2/product/search_by_object

Parameters:

  • records: A list of real-life photos to detect product object on
    • must contain either of _url or _base64 field
curl https://api.ximilar.com/smart/v2/product/search 
  -H "Content-Type: application/json" 
  -H "Authorization: Token 713-----XXXXXXXXXXXXXXXXX----0e19beb" 
  -H "collection: mycoll"
  -d '{
    "records": [ { "_url": "http://url.to/real_life/photo.jpg" } ]
  }

Objects are present in the _objects field in every record. Every object contains name, probability prob, bound_box and additional information like about tags.

CLICK TO SHOW JSON RESULT

{
  "status": {"text": "OK", "code": 200},
  "records": [
    {
      "_url": "http://url.to/real_life/photo.jpg", 
      "_objects": [
        {
          "name": "Clothing/Upper",
          "prob": 0.99,
          "bound_box": [xmin, ymin, xmax, ymax],
          "_tags": [
            {"prob": 0.99971, "name": "Sleeves long"}, 
            {"prob": 0.88038, "name": "Pullovers"}, 
            {"prob": 1.0, "name": "Women"}
          ], 
          "_tags_simple": ["Sleeves long", "Pullovers", "Women"]
        },
        {
          "name": "footwear",
          "prob": 0.97,
          "bound_box": [xmin, ymin, xmax, ymax],
          "_tags": [
            {"prob": 0.971, "name": "Leather shoes"}, 
            {"prob": 0.838, "name": "Loafers"}
          ], 
          "_tags_simple": ["Leather shoes", "Loafers"]
        },
        {
          "name": "footwear",
          "prob": 0.91,
          "bound_box": [xmin, ymin, xmax, ymax],
          "_tags": [
            {"prob": 0.971, "name": "Leather shoes"}, 
            {"prob": 0.838, "name": "Loafers"}
          ], 
          "_tags_simple": ["Leather shoes", "Loafers"]          
        }
      ],
      "_width": 1200,
      "_height": 1100
    }
  ],
  "statistics": {"processing time": 0.8932}
}

/v2/product/search_by_object

If you already have the object (for example from /v2/product/detect) you can send it in the record to this method. The result is similar to /v2/product/search but without additional detections for your record. This method is suitable for your applications if you want to implement user interaction to let them select the object to find similarity products for.

Parameters (the same as for /v2/product/search):

  • records - a list that should contain only ONE real-life photo with the field _objects with one object as returned by /v2/product/detect;
  • k with default value 10
  • from with default value 0
  • fields_to_return additional fields to be returned for each records in the answer. By default only these fields are returned: [ _id, _url, _file, _objects ]
  • filter - additional filter to be applied during the search (besides collection and type of detected object); default value: null
curl https://api.ximilar.com/smart/v2/product/search 
  -H "Content-Type: application/json" 
  -H "Authorization: Token 713-----XXXXXXXXXXXXXXXXX----0e19beb" 
  -H "collection: mycoll"
  -d '{
    "records": [ { 
      "_url": "http://url.to/real_life/photo.jpg",
      "_objects": [
        {
          "name": "Clothing/Upper",
          "prob": 0.99,
          "bound_box": [xmin, ymin, xmax, ymax],
          "_tags": [
            {"prob": 0.99971, "name": "Sleeves long"}, 
            {"prob": 0.88038, "name": "Pullovers"}, 
            {"prob": 1.0, "name": "Women"}
          ], 
          "_tags_simple": ["Sleeves long", "Pullovers", "Women"]
        }       
    } ]
  }

The result is the same as for /v2/product/search.