How to create a website using GPT-J with Pipeline API

This tutorial will assume only a basic understanding of Python, HTML, CSS and javascript. We will be focused on using Flask to create a backend service to communicate with the Pipeline API and a super simple static website with a text input and submit button that communicates with the backend service.

Flask is a lightweight web framework that we will use for our backend server. Flask is great for development since it comes with a development WSGI server and has a nice no-frills set up. For production applications, with their increased demands on performance and security, you would need to run the Flask application with a more robust web server like Gunicorn or an ASGI set up with FastAPI and Uvicorn.

I'll start by creating a conda environment called pipeline-website with mamba. I will also include the packages flask and flask-cors

1$ mamba create -n pipeline-website python=3.9 flask flask-cors

Since the pipeline-ai package is not (yet) included in conda-forge, we need to install it separately with pip in the activated environment:

1$ mamba activate pipeline-website
2(pipeline-website) $ pip install pipeline-ai

Now that our environment is set up, create a directory for your project, here I will call it my_website. In here, create frontend and backend directories. In backend create a file app.py. This will be the only file in this directory!

Let's create our Flask app with a basic api route that we can use to test our set up:

1# backend/app.py
2
3from flask import Flask
4
5\
6app = Flask(__name__)
7
8\
9@app.route("/hello")
10def hello_world():
11	return "Hello, world!"

Let's load up the flask development server and test our route. Activate your python environment (if using mamba, mamba activate pipeline-website) and run

1(pipeline-website) my_website/backend $ flask --app app run
2 * Serving Flask app 'app'
3 * Debug mode: off
4
5WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
6 * Running on http://127.0.0.1:5000

As it explains, this starts a development server. Let's see it working by opening http://127.0.0.1:5000/hello. You should see Hello, World!.

Now, let's get started with using the Pipeline API! The python pipeline-ai package comes with a PipelineCloud object that will be our interface with the API (instead of using requests directly, although the code would look very similar). The PipelineCloud object takes a token keyword argument. This must contain a valid API token that you can generate in your dashboard on pipeline.ai. Our updated app.py now looks like this

1# backend/app.py
2
3from pipeline import PipelineCloud
4
5from flask import Flask
6
7\
8app = Flask(__name__)
9pc = PipelineCloud(token="YOUR API TOKEN")
10
11\
12@app.route("/hello")
13...

Now when we restart our Flask server, we should see an authentication message to confirm we're connected to Pipeline API with a valid API token:

1(pipeline-website) my_website/backend $ flask --app app run
2
3Authenticating
4
5Succesfully authenticated with the Pipeline API (https://api.pipeline.ai)
6...

Our API will have only one endpoint for running GPT-J, /gptj. This will accept only POST requests sent to it with only one field in the payload, prompt. We won't include verification for the request payload but this is something you would want to do in production.

1# backend/app.py
2
3from flask import Flask, jsonify, request
4...
5@app.route("/gptj", methods=["POST"])
6def generate_gptj():
7	prompt = request.json["prompt"]
8	run = pc.run_pipeline(
9		"pipeline_6908d8fb68974c288c69ef45454c8475",
10		[
11			prompt,
12			{
13			"response_length": 64, # The number of tokens to generate
14			"include_input": False,
15			"temperature": 1.0,
16			"top_k": 50
17			},
18		],
19	)
20	response = jsonify({"result": run["result_preview"][0][0]})
21	return response

When a POST request is received to /gptj, Flask runs the function generate_gptj() with request being the request object that triggered the function. We can access the contents of the request's json and extract the prompt with request.json["prompt"]. Each pipeline available on pipeline.ai has a different set of parameters that change its behaviour. This request is almost identical to the example request found on the GPT-J page on your Pipeline dashboard.

Let's restart the server and try out our new endpoint with a cURL request

1$ curl http://127.0.0.1:5000/gptj -X POST -H "Content-Type: application/json" -d '{"prompt":"War is peace, "}'
2{"result":" \nViolence is justice.\n\nToward the end of the Great War, a former artillery officer stood next to King George and explained to his courtiers why he had decided to become a pacifist.\n\n\"Once I was a soldier and fought on the continent of Europe with my fellow soldiers. Now"}

It works! Now to make the website. In our frontend directory create the file index.html

1<html>
2<head>
3	<title>GPT-J Website</title>
4</head>
5<body>
6	<h1>GPT-J</h1>
7	<input placeholder="Among the most myserious things known to humans"  type="text" id="textInput">
8	<button type="button">Get response</button>
9</body>
10</html>

Now if you open this in a browser you my feel ill: that's called nostalgia. We'll get to prettifying it in a bit but first we need to make it do something.

Create a file main.js in the same directory as index.html

1async function fetchText(request_text) {
2	let request_dict = {
3		method: "POST",
4		headers: { "Content-Type":  "application/json" },
5		body: JSON.stringify({ prompt: request_text }),
6	}
7	let response = await fetch("http://127.0.0.1:5000/gptj", request_dict);
8	return response.text();
9}
10
11async function getInput() {
12	let inputVal = document.getElementById("textInput").value;
13	if (inputVal.length === 0) {
14		inputVal = document.getElementById("textInput").placeholder;
15	}
16	let response = await fetchText(inputVal);
17	response_text = JSON.parse(response)["result"];
18	
19	const output = document.createElement("p");
20	output.textContent = response_text;
21	document.body.appendChild(output);
22}

And add the script at the end of our html <body> and update our button to run getInput() when a user clicks it.

1
2<button  type="button"  onClick="getInput();">Get response</button>
3
4<script src="main.js"></script>
5

Now let's test our new functionality. It is good practice to set up a basic server to host our website to simulate a more realistic environment. We can do so by running a python http.server.

Navigate to your frontend directory and run

1frontend $ python -m http.server
2
3Serving HTTP on :: port 8000 (http://[::]:8000/) ...

Now open up an incognito tab (so we don't cache the website which causes problems seeing changes) and go to http://127.0.0.1:8000/. You should see the same website as before but now our button should work... Maybe not. Checking out the browser console we see

1Access to fetch at 'http://127.0.0.1:5000/gptj' from origin 'http://127.0.0.1:8000' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: 
2No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

Oh no!

This message means that our flask server is not happy with our origin http://127.0.0.1:8000. Our flask server is trying to protect us from this request being fired from a rogue website (imagine that we were on a spoofed login page for a bank. The server holding our account doesn't recognise this page tries to block us from sending requests to it from scripts loaded on this login page). We know, however, that http://127.0.0.1:8000 is not a rogue website, so we should allow this origin.

To do so we use the Flask plugin flask_cors. Let's update our app.py

1
2from flask_cors import CORS
3
4
5app = Flask(__name__)
6CORS(app, origins=["http://127.0.0.1:8000"])
7

Now restart the Flask server and our button should work as expected.

Remember to start both our servers, open two terminals and run

1backend $ mamba activate pipeline-website
2(pipeline-website) backend $ flask --app app run

1frontend $ python -m http.server 

Just in case something was missed, or to just clone all the code, here's the github repo.

#Wrap up

The key points to remember when using the API:

ABOUT PIPELINE.AI

Pipeline AI makes it easy to work with ML models and to deploy AI at scale. The self-serve platform provides a fast pay-as-you-go API to run pretrained or proprietory models in production. If you are looking to deploy a large product and would like to sign up as an Enterprise customer please get in touch.