Skip to content

Smart Product Search

This page describes API of the Smart Product Search service which takes a real-life photo, detects (mostly fashion) products on this photo and finds visually similar images in the product collection.

The API follows the general rules of Ximilar API as described in Section First steps.

API is a set of HTTP REST services accepting JSON-formatted documents using POST and returning JSON documents. These services run at these URLs:

https://api.ximilar.com/smart/v2/<method>

Please, see page API Calls and Credits for exact prices for individual API calls. The Smart Product Search is a complex service where each API call typically consists of detection, tagging, feature extraction and similarity search and thus the calls cost more credits than our simple services.

Contact us before using this service

In order to get access to the Proto & Product Similarity services, please register at https://app.ximilar.com and then contact us at tech@ximilar.com to make the service accessible for your Ximilar account.

Overview of API Methods

You always work with your "collection" of images, which is specified in the collection-id header of each API call. Your collection(s) must be first created by Ximilar administrators.

Deprecated

Instead of ID of the collection specified in collection-id header, you may also directly use the name of the collection in collection header. This usage assumes that you have only one collection with this name. Usage of collection header is deprecated and might be removed in future.

All API methods use POST, require a JSON record (JSON map) in the body of the request, and return answer as another JSON record. Here is an example of a communication with the API:

$ curl --request POST \
    --url https://api.ximilar.com/smart-dev/v2/product/search \
    --header 'authorization: Token 7e8-------------------4a89' \
    --header 'collection-id: 062b0721-4df6-4c60XXXXXXXXXX764ba94c' \
    --header 'content-type: application/json' \
    --data '{
    "fields_to_return": [ "_id", "_url", "product_id" ],
    "records": [
        {
            "_url": "http://vize.ai/fashion_examples/10.jpg"
        }
    ]
  }'

and example response:

{
  "status": {"text": "OK", "code": 200},
  "records": [
    {
      "_url": "http://url.to/real_life/photo.jpg", 
      "_objects": [
        {
          "name": "Clothing/Upper",
          "prob": 0.95,
          "bound_box": [120, 150, 270, 390],
          "_tags": [
            {"prob": 0.99971, "name": "Sleeves long"}, 
            {"prob": 0.88038, "name": "Pullovers"}, 
            {"prob": 1.0, "name": "Women"}
          ], 
          "_tags_simple": ["Sleeves long", "Pullovers", "Women"]
        }
      ]
     }
  ],
  "answer_records": [ 
    { 
     "_id": "12",
     "_url": "http://url.to/product/photo1.jpg", 
     "product_id": "12345" 
    },
    { 
      "_id": "16",
      "_url": "http://url.to/product/photo2.jpg", 
      "product_id": "12346" 
    }
  ],
  "answer_distances": [ 0.73, 0.71 ],
  "statistics": {"processing time": 0.8932}
}

There are also methods for management of the records already inserted into the system that are very similar to those described in Ximilar Search API: Ximilar Search API:

  • /v2/get - method to insert new product photo into your collection,
  • /v2/update - method to search the most similar products given a real-life image
  • /v2/delete - method to detect product objects on the photo

Showing bounding box of the object

You minght want to display bounding boxes of the detected objects in your application. Here is an example of small python snippet with opencv library which will help you:

import cv2

# get bounding box of first record and first object of the record
bbox = result['records'][0]['_objects'][0]['bound_box']

# read the same image in opencv
image = cv2.imread('photo.jpg', 0)
image = cv2.rectangle(image, (bbox[0], bbox[1]), (bbox[2], bbox[3]), (255,0,0), 2)

cv2.imwrite('photo_bbox.jpg', image)

Detailed Descriptions of API Methods

/v2/product/insert

You call this insert method to fill the smart product index with your data. Unlike the standard /v2/insert, this method first detects objects on the image, cuts the rectangular area with the object out of the image, extracts visual descriptor from the area, assigns some tags and only then inserts the image into the index. The inserted records can also contain additional meta data for consequent filtering.

Parameters:​

  • records: A list of product photos to be inserted into the collection; each record
    • must contain either of ​_url, _base64
    • should contain field ​category ​that tells the product category; the values are from customer dictionary (Ximilar has mapping of customer categories to Ximilar products for each collection)

curl https://api.ximilar.com/smart/v2/product/insert
  -H "Content-Type: application/json" 
  -H 'Authorization: Token 1af538baa90-----XXX-----baf83ff24'
  -H 'collection: mycoll'
  -d '{ 
    "records": [
      { "_id": "1", "_url": "http://url.to/product/photo.jpg", "product_id": "Zara T-Shirt 1234", "category": "TSHIRTS" },
      { "_id": "2", "_base64": "ABD432...", "product_id": "Adidas Shoes 1", "category": "RUNNING_SHOES" }
    ]
  }'

The response is very similar as the one of method /v2/insert. The returned records contain also the detected objects with their tags and also parameter "_obj_type" with possible values: "clothing", "footwear", "watch".

CLICK TO SHOW JSON RESULT

{
  "status": {"text": "OK", "code": 200},
  "records": [
    {
      "_id": "1",
      "_url": "http://url.to/product/photo.jpg", 
      "product_id": "Zara T-Shirt 1234",
      "_obj_type": "clothing",
      "_objects": [
        {
          "name": "Clothing/Upper",
          "prob": 0.99,
          "bound_box": [xmin, ymin, xmax, ymax]
        }
      ],
      "_width": 1200,
      "_height": 1100
    },
    { 
      # "_base64" field is omitted from the answer
      "_id": "2",
      "product_id": "Adidas Shoes 1",
      "_obj_type": "footwear",
      "_objects": [
        {
          "name": "footwear",
          "prob": 0.97,
          "bound_box": [xmin, ymin, xmax, ymax]
        }
      ],
      "_width": 900,
      "_height": 700
    }
  ],
  "statistics": {"processing time": 0.8932}
}

/v2/product/search

This is main method of the smart product search for finding most similar item from the picture. If more than 1 object is detected then search will return similar items for most dominant item (with largest area or probability, mode param).

Parameters:

  • records - a list that should contain only ONE real-life photo to find similar products for
  • mode - modes for searching (default: area)
    • area: find similar products for the detected object with largest area (and probability >= 0.9)
    • prob: find similar products for the detected object with the highest detection probability
  • k with default value 10
  • from with default value 0
  • fields_to_return additional fields to be returned for each records in the answer. By default only these fields are returned: [ _id, _url, _file, _objects ]
  • filter - additional filter to be applied during the search (besides collection and type of detected object); default value: null

See this Product & Product Similarity for details on the search params.

curl https://api.ximilar.com/smart/v2/product/search 
  -H "Content-Type: application/json" 
  -H "Authorization: Token 713-----XXXXXXXXXXXXXXXXX----0e19beb" 
  -H "collection: mycoll"
  -d '{
    "records": [ { "_url": "http://url.to/real_life/photo.jpg" } ],
    "mode": ["area"|"prob"],
    "k": 2, 
    "from": 0,
    "filter": { "category": "external_category" }, 
    "fields_to_return": [ "_id", "category", "_url", "_product_id", "_objects" ] 
  }

Similar items are present in answer_records field with distances to the query object in answer_distances.

CLICK TO SHOW JSON RESULT

{
  "status": {"text": "OK", "code": 200},
  "records": [
    {
      "_url": "http://url.to/real_life/photo.jpg", 
      "_objects": [
        {
          "name": "Clothing/Upper",
          "prob": 0.95,
          "bound_box": [xmin, ymin, xmax, ymax],
          "_tags": [
            {"prob": 0.99971, "name": "Sleeves long"}, 
            {"prob": 0.88038, "name": "Pullovers"}, 
            {"prob": 1.0, "name": "Women"}
          ], 
          "_tags_simple": ["Sleeves long", "Pullovers", "Women"]
        }
      ],
      "answer_records": [ 
        { 
          "_id": "12",
          "_url": "http://url.to/product/photo1.jpg", 
          "product_id": "12345", 
          "_objects": [ { "name": "Clothing/Upper", "prob": 0.99, "bound_box": [xmin, ymin, xmax, ymax] } ] 
        },
        { 
          "_id": "16",
          "_url": "http://url.to/product/photo2.jpg", 
          "product_id": "12346", 
          "_objects" [ ... ] 
        }
      ],
      "answer_distances": [ 0.73, 0.71 ]
    }
  "statistics": {"processing time": 0.8932}
}

/v2/product/detect

You can use this method to detect all product object on your real-life photo and get tags for these objects. Subsequently, you can pick one object from '_objects' field and put it into the request body of method for /v2/product/search_by_object

Parameters:

  • records: A list of real-life photos to detect product object on
    • must contain either of _url or _base64 field
curl https://api.ximilar.com/smart/v2/product/search 
  -H "Content-Type: application/json" 
  -H "Authorization: Token 713-----XXXXXXXXXXXXXXXXX----0e19beb" 
  -H "collection: mycoll"
  -d '{
    "records": [ { "_url": "http://url.to/real_life/photo.jpg" } ]
  }

Objects are present in the _objects field in every record. Every object contains name, probability prob, bound_box and additional information like about tags.

CLICK TO SHOW JSON RESULT

{
  "status": {"text": "OK", "code": 200},
  "records": [
    {
      "_url": "http://url.to/real_life/photo.jpg", 
      "_objects": [
        {
          "name": "Clothing/Upper",
          "prob": 0.99,
          "bound_box": [xmin, ymin, xmax, ymax],
          "_tags": [
            {"prob": 0.99971, "name": "Sleeves long"}, 
            {"prob": 0.88038, "name": "Pullovers"}, 
            {"prob": 1.0, "name": "Women"}
          ], 
          "_tags_simple": ["Sleeves long", "Pullovers", "Women"]
        },
        {
          "name": "footwear",
          "prob": 0.97,
          "bound_box": [xmin, ymin, xmax, ymax],
          "_tags": [
            {"prob": 0.971, "name": "Leather shoes"}, 
            {"prob": 0.838, "name": "Loafers"}
          ], 
          "_tags_simple": ["Leather shoes", "Loafers"]
        },
        {
          "name": "footwear",
          "prob": 0.91,
          "bound_box": [xmin, ymin, xmax, ymax],
          "_tags": [
            {"prob": 0.971, "name": "Leather shoes"}, 
            {"prob": 0.838, "name": "Loafers"}
          ], 
          "_tags_simple": ["Leather shoes", "Loafers"]          
        }
      ],
      "_width": 1200,
      "_height": 1100
    }
  ],
  "statistics": {"processing time": 0.8932}
}

/v2/product/search_by_object

If you already have the object (for example from /v2/product/detect) you can send it in the record to this method. The result is similar to /v2/product/search but without additional detections for your record. This method is suitable for your applications if you want to implement user interaction to let them select the object to find similarity products for.

Parameters (the same as for /v2/product/search):

  • records - a list that should contain only ONE real-life photo with the field _objects with one object as returned by /v2/product/detect;
  • k with default value 10
  • from with default value 0
  • fields_to_return additional fields to be returned for each records in the answer. By default only these fields are returned: [ _id, _url, _file, _objects ]
  • filter - additional filter to be applied during the search (besides collection and type of detected object); default value: null
curl https://api.ximilar.com/smart/v2/product/search 
  -H "Content-Type: application/json" 
  -H "Authorization: Token 713-----XXXXXXXXXXXXXXXXX----0e19beb" 
  -H "collection: mycoll"
  -d '{
    "records": [ { 
      "_url": "http://url.to/real_life/photo.jpg",
      "_objects": [
        {
          "name": "Clothing/Upper",
          "prob": 0.99,
          "bound_box": [xmin, ymin, xmax, ymax],
          "_tags": [
            {"prob": 0.99971, "name": "Sleeves long"}, 
            {"prob": 0.88038, "name": "Pullovers"}, 
            {"prob": 1.0, "name": "Women"}
          ], 
          "_tags_simple": ["Sleeves long", "Pullovers", "Women"]
        }       
    } ]
  }

The result is the same as for /v2/product/search.

/v2/get

Description: finds and returns given list of records (identified by _id) from the index

Example:

$ curl 'https://api.ximilar.com/similarity/photos/v2/get' -i -X POST \
    -H 'Content-Type: application/json;charset=UTF-8' \
    -H 'collection-id: mycoll_id' \
    -H 'Authorization: Token 1af538baa90-----XXX-----baf83ff24' \
    -d '{
  "fields_to_return" : [ "*" ],
  "records" : [ {
    "_id" : "1"
  }, {
    "_id" : "2"
  }, {
    "_id" : "3"
  } ]
}'
{
  "fields_to_return" : [ "*" ],
  "records" : [ {
    "_id" : "1"
  }, {
    "_id" : "2"
  }, {
    "_id" : "3"
  } ]
}
$ echo '{
  "fields_to_return" : [ "*" ],
  "records" : [ {
    "_id" : "1"
  }, {
    "_id" : "2"
  }, {
    "_id" : "3"
  } ]
}' | http POST 'https://api.ximilar.com/similarity/photos/v2/get' \
    'Content-Type:application/json;charset=UTF-8' \
    'collection-id:mycoll_id' \
    'Authorization:Token 1af538baa90-----XXX-----baf83ff24'

Request description:

Path Type Required Description
records Array Yes Records to be returned, identified by their _id
fields_to_return Array No Fields to be returned in every record, defaults to ["*"]

Example response:

{
  "status" : {
    "code" : 213,
    "text" : "some of the records not found"
  },
  "statistics" : {
    "OperationTime" : 9
  },
  "answer_records" : [ {
    "_url" : "http://mydomain.com/img.png",
    "_id" : "1"
  }, {
    "_file" : "/path/to/img.png",
    "_id" : "2"
  } ],
  "skipped_records" : [ {
    "_id" : "3",
    "_reason" : {
      "status" : {
        "code" : 404,
        "text" : "records not found"
      }
    }
  } ],
  "answer_count" : 2
}

Response description:

Path Type Description
answer_records Array Records found
skipped_records Array Records that couldn't be returned
answer_count Number Number of records found

Possible status values:

  • "status": {"code": 205, "text": "records found"} -- all records found and returned
  • "status": {"code": 206, "text": "some of the records not found"} -- some of the records are returned, the rest has not been found
  • "status": {"code": 404, "text": "records not found"} -- none of the requested records were found and empty answer is returned

/v2/update

Description: updates attributes of the given list of records (identified by _id) stored in the index. This method can only update additional attributes used for filtering (including tags), but NOT the image (use delete and re-insert if you want to change the image).

Example:

$ curl 'https://api.ximilar.com/similarity/photos/v2/update' -i -X POST \
    -H 'Content-Type: application/json;charset=UTF-8' \
    -H 'collection-id: mycoll_id' \
    -H 'Authorization: Token 1af538baa90-----XXX-----baf83ff24' \
    -d '{
  "fields_to_return" : [ "*" ],
  "records" : [ {
    "_id" : "1",
    "day" : "monday"
  }, {
    "_id" : "2",
    "day" : "tuesday"
  }, {
    "_id" : "3",
    "day" : "wednesday"
  } ]
}'
{
  "fields_to_return" : [ "*" ],
  "records" : [ {
    "_id" : "1",
    "day" : "monday"
  }, {
    "_id" : "2",
    "day" : "tuesday"
  }, {
    "_id" : "3",
    "day" : "wednesday"
  } ]
}
$ echo '{
  "fields_to_return" : [ "*" ],
  "records" : [ {
    "_id" : "1",
    "day" : "monday"
  }, {
    "_id" : "2",
    "day" : "tuesday"
  }, {
    "_id" : "3",
    "day" : "wednesday"
  } ]
}' | http POST 'https://api.ximilar.com/similarity/photos/v2/update' \
    'Content-Type:application/json;charset=UTF-8' \
    'collection-id:mycoll_id' \
    'Authorization:Token 1af538baa90-----XXX-----baf83ff24'

Request description:

Path Type Required Description
records Array Yes Records to be updated, identified by their _id
fields_to_return Array No Fields to be returned in every record, defaults to ["*"]

Example response:

{
  "status" : {
    "code" : 213,
    "text" : "some of the records not found"
  },
  "statistics" : {
    "OperationTime" : 1
  },
  "answer_records" : [ {
    "another-field" : "another-value",
    "_id" : "1",
    "day" : "monday"
  }, {
    "day" : "tuesday",
    "_id" : "2"
  } ],
  "skipped_records" : [ {
    "_id" : "3",
    "_reason" : {
      "status" : {
        "code" : 404,
        "text" : "records not found"
      }
    }
  } ],
  "answer_count" : 2
}

Response description:

Path Type Description
answer_records Array Updated records
skipped_records Array Records that couldn't be updated
answer_count Number Number of records updated

Possible status values (the same as /v2/get operation):

  • "status": {"code": 205, "text": "records found"} -- all records found and returned
  • "status": {"code": 206, "text": "some of the records not found"} -- some of the records are returned, the rest has not been found
  • "status": {"code": 404, "text": "records not found"} -- none of the requested records were found and empty answer is returned

/v2/delete

Description: deletes given list of records (identified by _id) from the index

Example:

$ curl 'https://api.ximilar.com/similarity/photos/v2/delete' -i -X POST \
    -H 'Content-Type: application/json;charset=UTF-8' \
    -H 'collection-id: mycoll_id' \
    -H 'Authorization: Token 1af538baa90-----XXX-----baf83ff24' \
    -d '{
  "records" : [ {
    "_id" : "1"
  }, {
    "_id" : "2"
  }, {
    "_id" : "3"
  } ]
}'
{
  "records" : [ {
    "_id" : "1"
  }, {
    "_id" : "2"
  }, {
    "_id" : "3"
  } ]
}
$ echo '{
  "records" : [ {
    "_id" : "1"
  }, {
    "_id" : "2"
  }, {
    "_id" : "3"
  } ]
}' | http POST 'https://api.ximilar.com/similarity/photos/v2/delete' \
    'Content-Type:application/json;charset=UTF-8' \
    'collection-id:mycoll_id' \
    'Authorization:Token 1af538baa90-----XXX-----baf83ff24'

Request description:

Path Type Required Description
records Array Yes Records to be deleted by their _id

Example response:

{
  "status" : {
    "code" : 213,
    "text" : "some of the records not found"
  },
  "statistics" : {
    "OperationTime" : 3
  },
  "answer_records" : [ {
    "_id" : "1"
  }, {
    "_id" : "2"
  } ],
  "skipped_records" : [ {
    "_id" : "3",
    "_reason" : {
      "status" : {
        "code" : 404,
        "text" : "records not found"
      }
    }
  } ],
  "answer_count" : 2
}

Response description:

Path Type Description
answer_records Array Successfully deleted records
skipped_records Array Records that couldn't be deleted
answer_count Number Number of deleted records

Possible status values:

  • "status": {"code": 220, "text": "records deleted"} -- all records deleted
  • "status": {"code": 206, "text": "some of the records not found"} -- some of the records were not deleted. Answer field answer_records contains list of records actually deleted and "skipped_records contains the list of records not found.
  • "status": {"code": 404, "text": "records not found"} -- none of the requested records were found and deleted

/v2/allIDs

Description: gets all IDs of records stored in the collection.

Example:

$ curl 'https://api.ximilar.com/smart/v2/allIDs' -i -X POST \
    -H 'Content-Type: application/json;charset=UTF-8' \
    -H 'collection-id: mycoll_id' \
    -H 'Authorization: Token 1af538baa90-----XXX-----baf83ff24' \
    -d '{}'

Example response:

{
  "status" : {
    "code" : 200,
    "text" : "OK"
  },
  "statistics" : {
    "OperationTime" : 3
  },
  "answer_records" : [ {
    "_id" : "1"
  }, {
    "_id" : "2"
  } ],
  "answer_count" : 2
}