Recognition as a Service

Quick and easy way to detect, classify and track objects real-time or non-real-time from images as well as video streams

Optimize your business processes

Automate processes with application integration. Shape your future project with visual data processing, object detection or image classification based on Artificial Intelligence and refine transaction-based functions. Centralize the data processing and validating capabilities of your business. Transform your app with computer vision – all through a REST API call.

Artificial Intelligence (AI) is a branch of computing that involves training computers to do things that normally require human intelligence. Machine Learning (ML) is a subset of AI which involves computers learning from and finding patterns in data in order to then be able to make predictions on new data by themselves.

Recognition Services™ can use for many scenarios, such as sentiment analysis, price prediction, product recommendation, sales forecasting, image classification, object detection and tracking, and more — it can easily integrate machine learning into your web, mobile, desktop, gaming, and IoT apps.

Run Recognition Services™ in the cloud or on-premises with containers — no third party services or infrastructure required. Apply it to diverse scenarios, like mobility or transport management related to image analysis, where data security and low latency are paramount.

The ability to process data all the way to the endpoints ensures maximum compliance with General Data Protection Regulation — GDPR — data protection standards and regulations, and the complex business processes built on this platform also meet these requirements.

Turn data into insights

Recognition Services™ provides services based on different, yet identical, computer vision technology and algorithmic backgrounds. Each technology solution can be used separately or nested. Hardware acceleration solutions – GPUs, TPUs – are needed to complete learning processes more efficiently and quickly.

Tensorflow

TensorFlow is an end-to-end open-source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries and community resources that lets researchers push the state-of-the-art in ML and CloudTrust developers easily build and deploy ML-powered applications.

Extended with TensorFlow & more

Microsoft Machine Learning for .NET, ML.NET has been designed as an extensible platform so that CloudTrust developers can consume other popular ML frameworks – TensorFlow, ONNX, OpenCV, Yolo (You Only Look Once) and more popular AI frameworks – and have access to even more machine learning scenarios, like image classification, object detection, and more.

Train models faster

CUDA® is a parallel computing platform and programming model developed by NVIDIA for general computing on graphical processing units – GPUs. With CUDA, CloudTrust developers are able to dramatically speed up computing applications by harnessing the power of GPUs. Pre-trained AI models give a faster path to trained and optimized AI networks, while containerized deployment to edge devices allows flexible and seamless updates.

Take advantage of Recognition Services™ to connect to SaaS applications

Recognition Services™ makes it possible to start centralized business processing from the Cloud, which allows you to take advantage of existing processes implemented, keep data within their on-premises environment, and integrate with cloud-based processes. A new adapter enables Recognition Services™ to invoke web services, which provides seamless connectivity to numerous SaaS applications.


This allows you to take advantage of cloud-based integration and keep data and processes entirely on-premises. Recognition Services™ features cloud-native support that lets developers build and deploy high-quality, software-defined features on embedded and edge devices.

Selected use cases

The Recognition Servies™ supports the following major use cases.

Object detection

Object detection is one of the classical problems in computer vision: recognize what the objects are inside a given image and also where they are in the image. For these cases, you can either use pre-trained models or train your own model to classify images specific to your custom domain.


Image classification

Image classification is a common case in many business scenarios. For these cases, you can either use pre-trained models or train your own model to classify images specific to your custom domain.


Both Object detection and Image classification components are capable of extracting results from image files (JPEG, GIF, PNG) and motion pictures (H264 over RTSP source, ONVIF compliant data stream) — extracted one-off images from video content. With the right infrastructure in place, a real-time system can be built.

Moving object detection

Moving object detection is a technique used in computer vision and image processing. Multiple consecutive frames from a video – FPS based sampling – are compared by various methods to determine if any moving object is detected. Recognition Servies™ based moving objects detection has been used for wide range of applications like video surveillance, activity recognition, real-time and non-real-time traffic counting, traffic analysis, etc – recognize the physical movement of an object in a given place or region.

Recognition Services™ API introduction

Requests

Any tool that is fluent in HTTP can communicate with the API simply by requesting the correct URI. Requests should be made using the HTTPS protocol so that traffic is encrypted. The interface responds to different methods depending on the action required.

Method Usage
GET For simple retrieval of information about your account, objects to be processed or environment, you should use the GET method. The information you request will be returned to you as a JSON object. The attributes defined by the JSON object can be used to form additional requests. Any request using the GET method is read-only and will not affect any of the objects you are querying.
POST To create a new object, your request should specify the POST method. The POST request includes all of the attributes necessary to create a new object. When you wish to create a new object, send a POST request to the target endpoint.
HEAD Finally, to retrieve metadata information, you should use the HEAD method to get the headers. This returns only the header of what would be returned with an associated GET request. Response headers contain some useful information about your API access and the results that are available for your request. For instance, the headers contain your current rate-limit value and the amount of time available until the limit resets. It also contains metrics about the total number of objects found, pagination information, and the total content length.

HTTP statuses

Along with the HTTP methods that the API responds to, it will also return standard HTTP statuses, including error codes. In the event of a problem, the status will contain the error code, while the body of the response will usually contain additional information about the problem that was encountered. In general, if the status returned is in the 200 range, it indicates that the request was fulfilled successfully and that no error was encountered. Return codes in the 400 range typically indicate that there was an issue with the request that was sent. Among other things, this could mean that you did not authenticate correctly, that you are requesting an action that you do not have authorization for, that the object you are requesting does not exist, or that your request is malformed. If you receive a status in the 500 range, this generally indicates a server-side problem. This means that we are having an issue on our end and cannot fulfill your request currently.

Responses

When a request is successful, a response body will typically be sent back in the form of a JSON object. An exception to this is when a DELETE request is processed, which will result in a successful HTTP 204 status and an empty response body. Inside of this JSON object, the resource root that was the target of the request will be set as the key. This will be the singular form of the word if the request operated on a single object, and the plural form of the word if a collection was processed. The value of these keys will generally be a JSON object for a request on a single object and an array of objects for a request on a collection of objects.

Curl examples

Throughout this document, some example API requests will be given using the curl command. This will allow us to demonstrate the various endpoints in a simple, textual format.

The first variable that you should set to get started is your OAuth authorization token. If you need access to the headers of a response through curl, you can pass the -i flag to display the header information along with the body. If you are only interested in the header, you can instead pass the -I flag, which will exclude the response body entirely.

Parameters

There are two different ways to pass parameters in a request with the API.

When passing parameters to create or update an object, parameters should be passed as a JSON object containing the appropriate attribute names and values as key-value pairs. When you use this format, you should specify that you are sending a JSON object in the header. This is done by setting the Content-Type header to application/json. This ensures that your request is interpreted correctly.

When passing parameters to filter a response on GET requests, parameters can be passed using standard query attributes. In this case, the parameters would be embedded into the URI itself by appending a ? to the end of the URI and then setting each attribute with an equal sign. Attributes can be separated with a &. Tools like curl can create the appropriate URI when given parameters and values; this can also be done using the -F flag and then passing the key and value as an argument. The argument should take the form of a quoted string with the attribute being set to a value with an equal sign.

Name Type Description
id string A unique ID that can be used to identify and reference an image.
name string A unique human-readable name referring to an image.
fingerprint string A unique identifier generated from the SHA-1 fingerprint of the image.
image string A base64 encoding of bytes representing the image data to be recognised.

Sample request

Curl example
curl -X GET -H "Content-Type: application/json" -H "Authorization: Bearer b7d03a6947b217efb6f3ec3bd3504582" "https://api.recognition.services/v1/objectdetection/person"

Request headers
Content-Type: application/json
Authorization: Bearer b7d03a6947b217efb6f3ec3bd3504582


Request body
{
  "id": 512190,
  "name": "My image of persons",
  "fingerprint": "1baa2c4ae9f668dc871ff6a970c82d16715693da",
  "image": "QUJDLTEyMw=="
}

Name Type Description
id string A unique ID that can be used to identify and reference an image.
name string A unique human-readable name referring to an image.
started_at string A time value given in ISO8601 combined date and time format that represents when the image processing was started.
finished_at string A time value given in ISO8601 combined date and time format that represents when the image processing was finished.
person_upper_left_coordinate string The top left coordinate of the person recognized in the image, in pixels relative to the top left (0, 0) coordinate of the original image.
person_bottom_right_coordinate string The bottom right coordinate of person recognized in the image, in pixels relative to the top left (0, 0) coordinate of the original image.
confidence string Confidence level of object recognition.

Sample response

Response headers
content-type: application/json; charset=utf-8
status: 200 OK


Response body for single object
{
  "object": {
   "id": 512190,
   "name": "My image of persons",
   "started_at": "2019-12-04T15:24:21Z",
   "finished_at": "2019-12-04T15:24:21Z",
   "person_upper_left_coordinate": "100,100",
   "person_bottom_right_coordinate": "500,300",
   "confidence": "0.673"
  }
}


Response body for an object collection
{
  "objects": [
   {
    "id": 512190,
    ...
   },
   {
    "id": 512190,
    ...
   }
  ]
}

Sample request

Curl example
curl -X GET -H "Content-Type: application/json" -H "Authorization: Bearer b7d03a6947b217efb6f3ec3bd3504582" "https://api.recognition.services/v1/objectdetection/licenseplatenumber"

Request headers
Content-Type: application/json
Authorization: Bearer b7d03a6947b217efb6f3ec3bd3504582


Request body
{
  "id": 512190,
  "name": "My image",
  "fingerprint": "1baa2c4ae9f668dc871ff6a970c82d16715693da",
  "image": "QUJDLTEyMw=="
}

Name Type Description
id string A unique ID that can be used to identify and reference an image.
name string A unique human-readable name referring to an image.
started_at string A time value given in ISO8601 combined date and time format that represents when the image processing was started.
finished_at string A time value given in ISO8601 combined date and time format that represents when the image processing was finished.
car_upper_left_coordinate string The top left coordinate of the car recognized in the image, in pixels relative to the top left (0, 0) coordinate of the original image.
car_bottom_right_coordinate string The bottom right coordinate of car recognized in the image, in pixels relative to the top left (0, 0) coordinate of the original image.
car_color string Color of the car recognised from image.
license_plate_upper_left_coordinate string The top left coordinate of the license plate recognized in the image, in pixels relative to the top left (0, 0) coordinate of the original image.
license_plate_bottom_right_coordinate string The bottom right coordinate of the license plate recognized in the image, in pixels relative to the top left (0, 0) coordinate of the original image.
license_plate_color string Color of the license plate recognised from image.
license_plate_number string Recognised license plate number from uploaded image.
license_plate_country_code string A country code based on the license plate, which can be identified from the strip on the left side of the license plate.

Sample response

Response headers
content-type: application/json; charset=utf-8
status: 200 OK


Response body for single object
{
  "object": {
   "id": 512190,
   "name": "My image",
   "started_at": "2019-12-04T15:24:21Z",
   "finished_at": "2019-12-04T15:24:21Z",
   "car_upper_left_coordinate": "100,100",
   "car_bottom_right_coordinate": "500,300",
   "car_color": "red",
   "license_plate_upper_left_coordinate": "200,200",
   "license_plate_bottom_right_coordinate": "300,250",
   "license_plate_color": "white",
   "license_plate_number": "PXO755",
   "license_plate_country_code": "S,Sweden"
  }
}


Response body for an object collection
{
  "objects": [
   {
    "id": 512190,
    ...
   },
   {
    "id": 512190,
    ...
   }
  ]
}

Upload image to Recognition Services™

Select an image for Recognition Services™. The selected image will be processed after upload.

The following demonstration allows you to test image classification and object recognition using a result of a real learning algorithm. Parameters of the learning algorithm: recognition of Hungarian license plates (including taxi and electric car plates), more than 1000 real training images, 360x240 image resolution, more than 6000 training iterations, NVIDIA GeForce GT-730 GPU support.

 

Steps to creating models and support low-power modes embedded edge computing

There are multiple steps in the process of getting answers from data using the Recognition Services™.

  1. Gather data
  2. Explore data
  3. Prepare data
  4. Build, train and evaluate a model
  5. Tune hyper-parameters
  6. Deploy model
Deploy models built and trained in the cloud and run them on-premises. For example, if you deploy a predictive model to an RTSP-capable factory camera to test for quality control and an issue is detected, edge device triggers an alert and processes the data locally or sends it to the cloud for further analysis.

Intelligent suggestions

AI is the capability of a machine to imitate intelligent human behaviour. Through AI, machines can analyze raw data, images, comprehend speech, interact in natural ways and make predictions using data.

Take advantage of Recognition Services™ AI's experience in evaluating data and developing prediction procedures.

Having the lowest latency possible between the data and the decision is critical. Rather than process your data in the cloud, edge processes it on the device itself, with the option of using low-power modes edge device hardware architecture. Recognition Services™ make real-time AI calculations, giving you the immediate insights you need to make critical business decisions.

Operating models and Recognition Services™

The Recognition Services™ can be used in different operating models.

On-premise model — In an on-premise operating model, a runtime environment is built into the customer's operating environment. The closed environment of the runtime environment does not move to an external service location due to data processing. The runtime environment always works with the latest model, only the model changes when updated.

Cloud model — In a cloud operating model, there is no runtime environment in the customer's operating environment. The data to be processed is sent to an external service location for processing. The cloud service always has the latest functionality.

Data travels over the HTTPS secure transport layer via Web service, Web socket or REST calls, and the results can be processed as a response to the service.


Hybrid model — In a hybrid operating model, a runtime environment is built into the client operating environment. In some cases, data from the closed environment of the runtime environment is sent to an external service location for processing. The runtime environment always works with the latest model and the cloud service always has the latest functionality.

In the case of a hybrid design, the services can be shared, on-premise and in a data processing center.

Operate your edge devices reliably and securely, even when they're offline or they have intermittent connectivity to the cloud. Remote Network Connection™ Edge device management automatically syncs the latest state of your devices after they're reconnected to ensure seamless operability. With our own solution, so-called Edge computing endpoints can be arranged in a virtual private network. This allows remote management or shared processing capability to be available at any time. For details on Remote Network Connection™, click here.

CloudTrust is a highly innovative company — born in the Cloud — that has set itself the task to develop and implement Cloud-based secure solutions and products. Clean and simple business models motivate us in our unique Software as a Service ("SaaS") solutions.

We believe that when you focus on your things like line-of-business assets, and data, you can dramatically improve efficiency and enable innovation, and ultimately fuel business transformation.

With decades of experience as a systems integrator and a commitment to providing and supporting superior technology, we’re able to build, deploy and support Microsoft and other leading enterprise platform solutions.  Our partnership and commitment to Microsoft technology have led us to develop products and services to support Microsoft technology and create smooth integration opportunities for our customers.