
5 minutes read
September 3, 2022
How to create a website using GPT-J with Pipeline API
TUTORIAL: How to get up and running with GPT-J in your app or website
This tutorial will assume only a basic understanding of Python, HTML, CSS and javascript. We will be focused on using Flask to create a backend service to communicate with the Pipeline API and a super simple static website with a text input and submit button that communicates with the backend service.
Flask is a lightweight web framework that we will use for our backend server. Flask is great for development since it comes with a development WSGI server and has a nice no-frills set up. For production applications, with their increased demands on performance and security, you would need to run the Flask application with a more robust web server like Gunicorn or an ASGI set up with FastAPI and Uvicorn.
I'll start by creating a conda environment called pipeline-website
with mamba. I will also include the packages flask
and flask-cors
1$ mamba create -n pipeline-website python=3.9 flask flask-cors
Since the pipeline-ai
package is not (yet) included in conda-forge, we need to install it separately with pip in the activated environment:
1$ mamba activate pipeline-website
2(pipeline-website) $ pip install pipeline-ai
Now that our environment is set up, create a directory for your project, here I will call it my_website
. In here, create frontend
and backend
directories. In backend
create a file app.py
. This will be the only file in this directory!
Let's create our Flask app with a basic api route that we can use to test our set up:
1# backend/app.py
2
3from flask import Flask
4
5\
6app = Flask(__name__)
7
8\
9@app.route("/hello")
10def hello_world():
11 return "Hello, world!"
Let's load up the flask development server and test our route. Activate your python environment (if using mamba, mamba activate pipeline-website
) and run
1(pipeline-website) my_website/backend $ flask --app app run
2 * Serving Flask app 'app'
3 * Debug mode: off
4
5WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
6 * Running on http://127.0.0.1:5000
As it explains, this starts a development server. Let's see it working by opening http://127.0.0.1:5000/hello. You should see Hello, World!
.
Now, let's get started with using the Pipeline API! The python pipeline-ai
package comes with a PipelineCloud
object that will be our interface with the API (instead of using requests directly, although the code would look very similar). The PipelineCloud
object takes a token
keyword argument. This must contain a valid API token that you can generate in your dashboard on pipeline.ai.
Our updated app.py
now looks like this
1# backend/app.py
2
3from pipeline import PipelineCloud
4
5from flask import Flask
6
7\
8app = Flask(__name__)
9pc = PipelineCloud(token="YOUR API TOKEN")
10
11\
12@app.route("/hello")
13...
Now when we restart our Flask server, we should see an authentication message to confirm we're connected to Pipeline API with a valid API token:
1(pipeline-website) my_website/backend $ flask --app app run
2
3Authenticating
4
5Succesfully authenticated with the Pipeline API (https://api.pipeline.ai)
6...
Our API will have only one endpoint for running GPT-J, /gptj
. This will accept only POST
requests sent to it with only one field in the payload, prompt
. We won't include verification for the request payload but this is something you would want to do in production.
1# backend/app.py
2
3from flask import Flask, jsonify, request
4...
5@app.route("/gptj", methods=["POST"])
6def generate_gptj():
7 prompt = request.json["prompt"]
8 run = pc.run_pipeline(
9 "pipeline_6908d8fb68974c288c69ef45454c8475",
10 [
11 prompt,
12 {
13 "response_length": 64, # The number of tokens to generate
14 "include_input": False,
15 "temperature": 1.0,
16 "top_k": 50
17 },
18 ],
19 )
20 response = jsonify({"result": run["result_preview"][0][0]})
21 return response
When a POST
request is received to /gptj
, Flask runs the function generate_gptj()
with request
being the request object that triggered the function. We can access the contents of the request's json and extract the prompt with request.json["prompt"]
.
Each pipeline available on pipeline.ai has a different set of parameters that change its behaviour. This request is almost identical to the example request found on the GPT-J page on your Pipeline dashboard.
Let's restart the server and try out our new endpoint with a cURL request
1$ curl http://127.0.0.1:5000/gptj -X POST -H "Content-Type: application/json" -d '{"prompt":"War is peace, "}'
2{"result":" \nViolence is justice.\n\nToward the end of the Great War, a former artillery officer stood next to King George and explained to his courtiers why he had decided to become a pacifist.\n\n\"Once I was a soldier and fought on the continent of Europe with my fellow soldiers. Now"}
It works! Now to make the website. In our frontend
directory create the file index.html
1<html>
2<head>
3 <title>GPT-J Website</title>
4</head>
5<body>
6 <h1>GPT-J</h1>
7 <input placeholder="Among the most myserious things known to humans" type="text" id="textInput">
8 <button type="button">Get response</button>
9</body>
10</html>
Now if you open this in a browser you my feel ill: that's called nostalgia. We'll get to prettifying it in a bit but first we need to make it do something.
Create a file main.js
in the same directory as index.html
1async function fetchText(request_text) {
2 let request_dict = {
3 method: "POST",
4 headers: { "Content-Type": "application/json" },
5 body: JSON.stringify({ prompt: request_text }),
6 }
7 let response = await fetch("http://127.0.0.1:5000/gptj", request_dict);
8 return response.text();
9}
10
11async function getInput() {
12 let inputVal = document.getElementById("textInput").value;
13 if (inputVal.length === 0) {
14 inputVal = document.getElementById("textInput").placeholder;
15 }
16 let response = await fetchText(inputVal);
17 response_text = JSON.parse(response)["result"];
18
19 const output = document.createElement("p");
20 output.textContent = response_text;
21 document.body.appendChild(output);
22}
And add the script at the end of our html <body>
and update our button to run getInput()
when a user clicks it.
1
2<button type="button" onClick="getInput();">Get response</button>
3
4<script src="main.js"></script>
5
Now let's test our new functionality. It is good practice to set up a basic server to host our website to simulate a more realistic environment. We can do so by running a python http.server
.
Navigate to your frontend
directory and run
1frontend $ python -m http.server
2
3Serving HTTP on :: port 8000 (http://[::]:8000/) ...
Now open up an incognito tab (so we don't cache the website which causes problems seeing changes) and go to http://127.0.0.1:8000/. You should see the same website as before but now our button should work... Maybe not. Checking out the browser console we see
1Access to fetch at 'http://127.0.0.1:5000/gptj' from origin 'http://127.0.0.1:8000' has been blocked by CORS policy: Response to preflight request doesn't pass access control check:
2No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
Oh no!
This message means that our flask server is not happy with our origin http://127.0.0.1:8000
. Our flask server is trying to protect us from this request being fired from a rogue website (imagine that we were on a spoofed login page for a bank. The server holding our account doesn't recognise this page tries to block us from sending requests to it from scripts loaded on this login page). We know, however, that http://127.0.0.1:8000
is not a rogue website, so we should allow this origin.
To do so we use the Flask plugin flask_cors
. Let's update our app.py
1
2from flask_cors import CORS
3
4
5app = Flask(__name__)
6CORS(app, origins=["http://127.0.0.1:8000"])
7
Now restart the Flask server and our button should work as expected.
Remember to start both our servers, open two terminals and run
1backend $ mamba activate pipeline-website
2(pipeline-website) backend $ flask --app app run
1frontend $ python -m http.server
Just in case something was missed, or to just clone all the code, here's the github repo.
#Wrap up
The key points to remember when using the API:
> If you run into issues, the Pipeline team are always willing to help in any way. Contact us here.> You will get CORS errors if you naively try to run requests from a browser. This is a good thing. If you are using the Pipeline API to provide a service, you must keep your API token secret on a server and only send requests from the server (which doesn't care about CORS).