When we have a custom model imported into tensorflow serving, we also need to create custom client which will communicate with the model server.

I created reterival based model to work on ubuntu dataset, in very short this model used to predict or score the best response among a set of response for a question.

After saving it as "pb" file and importing to tensorflow model server, a client had to be created to communicate with the model via model server.

I wrote a client named : reterieval_based_model_client.py ( here it is named as grpc__client.py )

First I had it in the same directory as other model codes,

/home/cdpdemo/tensorflow/reterival-based-model/chatbot-retrieval

If i try to run it directly just like any python code, i got following error,

Fri May 19 05:13:50 UTC 2017:cdpdemo:/home/cdpdemo/tensorflow/reterival-based-model/chatbot-retrieval>python grpc_client.py
Traceback (most recent call last):
  File "grpc_client.py", line 4, in <module>
    from tensorflow_serving.apis import predict_pb2
ImportError: No module named tensorflow_serving.apis

This is because to use tensorflow_serving libraries we will have to build it via bazel.

I tried to build it via bazel directly,

bazel build /home/cdpdemo/tensorflow/reterival-based-model/chatbot-retrieval:grpc_client

Note: this command is to be executed from "serving" directory where tensorflow_serving was installed.

I get below error,

Fri May 19 06:41:20 UTC 2017:cdpdemo:/home/cdpdemo/PythonCode/serving>bazel build /home/cdpdemo/tensorflow/reterival-based-model/chatbot-retrieval:grpc_client
..........................
ERROR: not a valid absolute pattern (absolute target patterns must start with exactly two slashes): '/home/cdpdemo/tensorflow/reterival-based-model/chatbot-retrieval:grpc_client'.
INFO: Elapsed time: 0.690s
Fri May 19 06:42:08 UTC 2017:cdpdemo:/home/cdpdemo/PythonCode/serving>bazel build //home/cdpdemo/tensorflow/reterival-based-model/chatbot-retrieval:grpc_client
ERROR: no such package 'home/cdpdemo/tensorflow/reterival-based-model/chatbot-retrieval': BUILD file not found on package path.
INFO: Elapsed time: 0.171s

To solve this error , i moved the file "grpc_client.py" to folder "/home/cdpdemo/PythonCode/serving/tensorflow_serving/example/"

Now i ran the below command,

bazel build //tensorflow_serving/example:grpc_client

I get another error now, i have put that in next page.