Request LLM API key – LLMs as a Service

Experimental OpenAI-compatible local LLM inference instances of NHR@FAU – LLMs as a Service

Terms of use:

  • This experimental service is provided as-is for scientific research and development only.
  • The service may fail or be stopped at any time without prior notice.
  • The models provided are subject to change at any time at NHR@FAU’s sole discretion.
  • NHR@FAU may introduce rate limiting as needed once too many requests come in.
  • There is no right or entitlement to an API key. NHR@FAU may refuse or revoke API keys at any time at its sole discretion.
  • Usage per API key is accounted (i.e. the number of tokens transferred and the model used), but queries&answers are never stored.
  • Using the local LLM inference instances of NHR@FAU, you agree to provide once per year a short scientific report on its usage upon request.

With regard to the EU AI Act:

  • The EU AI Act is not applicable owing to the exception of scientific research and development according to Art. 2 (6).
  • We provide Open Weight models as-is; detailed model cards for all provided models are available on Huggingface. That’s also the place where we download the pre-trained models.
  • We typically use vLLM as inference software. We always run the inference on-premis on our data center hardware.
  • Customers of the API endpoint are solely responsible for the risk classification and for complying with the obligations of the EU AI Act within their specific use case.
  • We have no knowledge of your concrete deployment scenario and therefore cannot perform any end-user risk assessment.
  • The usage for all kinds of high-risk applications is forbidden.

Application

  •  


Usage examples

Once you got your personal API key, you can use the LLM inference instances as follows (assuming you make the API key available to the code by setting the environment variable LLMAPI_KEY):

Show available models with curl:

curl -s -H "Authorization: Bearer $LLMAPI_KEY" \
         https://hub.nhr.fau.de/api/llmgw/v1/models | jq .

Simple chat using the OpenAI python module:

from openai import OpenAI
import os

# Initialize client with private endpoint URL and API key
client = OpenAI(
    # This is the default and can be omitted
    api_key=os.getenv("LLMAPI_KEY"),
    base_url="https://hub.nhr.fau.de/api/llmgw/v1"
)

# Create a chat completion request
response = client.chat.completions.create(
    model="gpt-oss-120b", # Replace with a model name available to you!
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Help me using the OpenAI API! Keep in mind that I have to use https://hub.nhr.fau.de/api/llmgw/v1 as base URL for the API and consider that OpenAI changed its API in version 1.0.0. Simple curl command lines are also welcome."}
    ],
    temperature=0.7 # Optional parameter
)

# Print the response
print(response.choices[0].message.content)