AI server

Our lab has an AI server (living in Chris's office) which you are welcome to use for educational or research purposes.

The server has 700GB memory, but this memory is not parallelized in a GPU. This means the server is capable of running frontier open models, but they will run very slowly. Therefore the server is most useful for your own learning (tinkering), for evaluating models, and for batch processing (e.g. generating embeddings for use with qualitative coding).

To access the server, you must be on UB's campus network or UB's VPN, and you need an API key. Please contact Chris if you would like a key, or for help getting set up.

Demo

This probably won't work for you 😔

The AI server currently runs over http, and you are likely viewing this page over a https connection, so the widget below won't work. The easiest workaround is to save this webpage as a file on your computer and then open the saved copy.

API Key

Returns all models currently available on the server (GET /api/v1/models).

Load a model into memory (POST /api/v1/models/load).

Model

TTL — seconds before auto-unload (optional)

Unload a model from memory (POST /api/v1/models/unload).

Model

Start downloading a model to the server (POST /api/v1/models/download). Returns a job_id.

Model ID

Model

Select a model above and say something.

Response

→ Enter your API key and select an operation above.

Connecting to the API

The server runs LMStudio (v0.4.10) to host models. LMStudio has its own API, and also supports endpoints compapatible with OpenAI and Anthropic.

The server's URL is gseai.gse.buffalo.edu, port 11434. You can connect directly via HTTP requests (here we use the httpie command-line tool):

% http gseai.gse.buffalo.edu:11434/api/v1/models "Authorization: Bearer sk-lm-..."

However, we suggest using the gseai package (documentation) to access the GSE AI server with code. Here's a quick example:

from gseai import GSEAIServer

server = GSEAIServer("sk-lm-...")
models = server.list_models()
response = server.chat("model-id", "What is machine learning?")

You can also connect using the OpenAI Python client:

import openai
client = OpenAI(
    base_url="http://gseai.gse.buffalo.edu:11434/v1", 
    api_key="sk-lm-..."
)
messages = [
    {"role": "system", "content": "You are a mischevious leprechaun."}
]
response = client.chat.completions.create(model="model-id", messages=messages)