Crowler

Crowler is a model serving framework complete with local and SageMaker training paradigms, a front end application for mobile usage, rolling deployment options, and a convenient way to configure all kinds of computer vision and language based models.

Crowler is optimized for the reduction in cost for inference: from the model compilation process to the focus on CPU for compute.

In our example below, we are filtering detections and performing similarity queries on roughly 2 billion image embeddings on a lone CPU instance behind an Application Load Balancer.

Crowler Front End

Crowler’s front end is built for Android and iOS targets and whether the target is for detection, segmentation, simple classification, or visual inspection, it can be augmented to fit the deployment’s needs.

Hover Inspection

Hover highlighting and per-detection informational displays

Backend, Models, and Support

Training

local

python3 train_{detector, embedder, segmenter}.py

cloud

python3 train_sagemaker.py

Inference

def predict(self, data):
    """
    This method considers only one input data

    Args:
    -----
    data: Data is list of map
        format is
        [
        {
            "parameterName": name
            "parameterValue": data
        },
        {...}
        ]

    Returns
    -------
    response: list()
        contains the [uuid, mean-dist, xmin, ymin, xmax, ymax] of each detection
    """
    request_batch = self.read_data_server(data)
    request_batch = self.detect(request_batch)
    request_batch = self.recognize(request_batch)
    response = self.format(request_batch)
    return response

Architecture