Migration Guide

This guide will help you migrate from other frameworks to mosec.

From the Triton Inference Server

Both PyTriton and Triton Python Backend are using Triton Inference Server.

• mosec doesn’t require a specific client, you can use any HTTP client library

• dynamic batching is configured when calling the append_worker

• mosec doesn’t need to declare the inputs and outputs. If you want to validate the request, you can use the TypedMsgPackMixin (ref Validate Request)

Triton Python Backend

• change the TritonPythonModel class to a worker class that inherits mosec.Worker

• move the initialize method to the __init__ method in the new class

• move the execute method to the forward method in the new class

• if you still prefer to use the auto_complete_config method, you can merge it into the __init__ method

• mosec doesn’t have the corresponding finalize method as an unloading handler

• mosec doesn’t require any special model directories or configurations

• to run multiple replicas, configure the num in append_worker

PyTriton

• move the model loading logic to the __init__ method, since this happens in a different process

• move the infer_func function to the forward method