Using Reranker Models

Reranker models are specialized models designed to improve the ranking of a list of items based on relevance to a given query. They are commonly used in information retrieval and search systems to refine initial search results, prioritizing items that are more likely to meet the user’s intent. Reranker models take the initial document list and reorder items to enhance precision in applications such as search engines, recommendation systems, and question-answering tasks.

In this tutorial, we will guide you through deploying and using reranker models in GPUStack.

Prerequisites

Before you begin, ensure that you have the following:

• GPUStack is installed and running. If not, refer to the Quickstart Guide.
• Access to Hugging Face for downloading the model files.

Step 1: Deploy the Model

Follow these steps to deploy the model from Hugging Face:

1. Navigate to the Models page in the GPUStack UI.
2. Click the Deploy Model button.
3. In the dropdown, select Hugging Face as the source for your model.
4. Enable the GGUF checkbox to filter models by GGUF format.
5. Use the search bar in the top left to search for the model name gpustack/bge-reranker-v2-m3-GGUF.
6. Leave everything as default and click the Save button to deploy the model.

After deployment, you can monitor the model's status on the Models page.

Step 2: Generate an API Key

We will use the GPUStack API to interact with the model. To do this, you need to generate an API key:

1. Navigate to the API Keys page in the GPUStack UI.
2. Click the New API Key button.
3. Enter a name for the API key and click the Save button.
4. Copy the generated API key. You can only view the API key once, so make sure to save it securely.

Step 3: Reranking

With the model deployed and an API key, you can rerank a list of documents via the GPUStack API. Here is an example script using curl:

export SERVER_URL=<your-server-url>
export GPUSTACK_API_KEY=<your-api-key>
curl $SERVER_URL/v1/rerank \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer $GPUSTACK_API_KEY" \
    -d '{
        "model": "bge-reranker-v2-m3",
        "query": "What is a panda?",
        "top_n": 3,
        "documents": [
            "hi",
            "it is a bear",
            "The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear or simply panda, is a bear species endemic to China."
        ]
    }' | jq

Replace <your-server-url> with the URL of your GPUStack server and <your-api-key> with the API key you generated in the previous step.

Example response:

{
  "model": "bge-reranker-v2-m3",
  "object": "list",
  "results": [
    {
      "document": {
        "text": "The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear or simply panda, is a bear species endemic to China."
      },
      "index": 2,
      "relevance_score": 1.951932668685913
    },
    {
      "document": {
        "text": "it is a bear"
      },
      "index": 1,
      "relevance_score": -3.7347371578216553
    },
    {
      "document": {
        "text": "hi"
      },
      "index": 0,
      "relevance_score": -6.157620906829834
    }
  ],
  "usage": {
    "prompt_tokens": 69,
    "total_tokens": 69
  }
}