On-the-Fly Auto translation&Auto summary by ChatGPT

jsurfer's language selection page >
jsurfer's homepage >
Web Apps >

On-the-Fly Auto translation&Auto summary by ChatGPT
Created on: September 25, 2023
Last updated on: September 25, 2023

Web App which translates contents of web page and also summarizes it to the target language on-the-fly by using OpenAI API(ChatGPT).

Below are the supported languages but you can add more languages easily:

English
German
Spanish
French
Italian
Dutch
Brazilian Portugese
Swedish
Japanese
Simplified Chinese
Traditional Chinese
Korean

What you will need:

1) OpenAI API Key (Only paid users can get this API Key)

2) Python (I used Python3 on my MacBook Pro 16-inch, 2019. As of September 25, 2023, the latest MacOS (Ventura, Version 13.6) has only Python3 so if you have the lastet MacOS, use 'python3' command instead of 'python' command and also use 'pip3' command instead of 'pip' command or else you will get an error.)

3) requests (Library to get Web page data)

4) Flask (Web Application Framework for Python)

5) langdetect (Library to detect language)

How this Web App works:

The user enters URL to translate on the Web form(index.html)

The server gets the URL and contents of the page(app.py)

Detect what language is being used(app.py)

ChatGPTtranslates to the Target language(app.py)

Display translated contents and summary in a new web page(translated.html)

==========

If you just want to see the Web App working, just read this section.

1. Download the zipped file

2. Unzip the zipped file and make sure the directory structure is this:

3. Open 'app.py' with any text editor app and replace "ENTER-YOUR-API-KEY-HERE" with your API key and save.

4. Launch Terminal and run the next 4 commands to install 'flask', 'openai', 'requests' and 'langdetect' packages. If you already have them installed, you can skip this step though it will not hurt to run:

5. Run this line to change the current directory to 'myFlaskApp':

6. Run this line to launch 'app.py' (If you are using 'python3', replace 'python' with 'python3'):

If you see this in your Terminal window, the Web App is running as expected:

Now you can just open 'http://127.0.0.1:5000/' from your web browser to see the URL entry form:

Let's have it translate and summarize one of my personal web pages. The URL is http://cheezsj.com/ios/ios_en.html:

.
.
.

Paste the URL, select "Japanese" from the list, select "Yes", and then click on the [Translate] button:

And this is the result. You can see that the page is translated and summarized:

==========

Just for your info, below is what I did to develop this Web App:

1. Run these 4 commands to install packages(If you are using python3, replace 'pip' command with 'pip3' command):

$pip install flask

$pip install openai

$pip install requests

$pip install langdetect

2. Create python script and save the it with filename 'app.py' (The actual files are in this zipped file)

from flask import Flask, request, render_template
import requests
import openai
from bs4 import BeautifulSoup
from langdetect import detect

openai.api_key = "ENTER-YOUR-API-KEY-HERE"

app = Flask(__name__)

@app.route('/', methods=['GET', 'POST'])
def index():
    if request.method == 'POST':
        # Get URL from the user's input
        url = request.form.get('url')
        chosen_language = request.form.get('language')
        # Get user's choice on summarize or not
        summarize_choice = request.form.get('summarize', 'no')

        # Set prompt based on the language
        language_mapping = {
            "en": "English",
            "jp": "日本語",
            "fr": "Français",
            "es": "Español",
            "it": "Italiano",
            "de": "Deutsch",
            "sv": "Svenska",
            "nl": "Nederlands",
            "zh_s": "简体中文",
            "zh_t": "繁體中文",
            "ko": "한국어",
            "pt_br": "Português (Brasil)"
            # Add more lang option here in the future
        }

        target_language = language_mapping.get(chosen_language, "English")

        # Get the content of the Web page
        response = requests.get(url)
        page_content = response.text

        # Get rid of HTML tags
        soup = BeautifulSoup(page_content, 'html.parser')
        text_only = soup.get_text()

        # Detect the language from the content of the Web page
        detected_lang = detect(text_only)

        # Translate to the target language
        if detected_lang == chosen_language:
            # Do not attemp to translate of target lang is same as detected lang
            translated_content = text_only
            # Translate and also trim the text to the first 3000 characters to be retrieved from the web page
            summary_prompt = "Summarize this text:\n" + text_only[:3000]
        else:
            # Translate and also trim the text to the first 3000 characters to be retrieved from the web page
            prompt_text = f"Translate from {detected_lang} to {target_language}:\n" + text_only[:3000]
            translation = openai.Completion.create(
                model="text-davinci-003",
                prompt=prompt_text,
                max_tokens=2000, # The model limits the generated text to 2000 tokens
                n=1,
                stop=None,
                temperature=0.5
            )
            translated_content = translation.choices[0].text.strip()
            summary_prompt = "Summarize this text in {target_language}:\n" + translated_content



        translated_content = translated_content.replace("\n", "<br>")  # Replace new line with "br"

        if summarize_choice == 'yes':
            # Summarize the web page
            summary = openai.Completion.create(
                model="text-davinci-003",
                prompt=summary_prompt,
                max_tokens=500,
                n=1,
                stop=None,
                temperature=0.5
            )
            summarized_content = summary.choices[0].text.strip()
            print(f"DEBUG: Summarized content is: {summarized_content}")  # Display info for debugging
        else:
            summarized_content = "Summary not requested."



        # Display the translation and the summary
        return render_template('translated.html', content=translated_content, summary=summarized_content)

    # For GET requests, display a form to input the URL
    return render_template('index.html')

if __name__ == "__main__":
    app.run(debug=True)

3. Create html file for the user to enter URL and save it with filename 'index.html' under templates folder.

<!DOCTYPE html>
<html>
<head>
  <title>Translation Service</title>
</head>
<body>

  <h1>Webpage Translator</h1>

  <form action="/" method="post">
    <br>
    <label for="url">1) Enter the URL to translate/summarize:</label><br>
    <input type="text" id="url" name="url"><br><br>

    <label for="language">2) Choose the target language:</label><br>
    <select id="language" name="language">
      <option value="en">English</option>
      <option value="de">Deutsch</option>
      <option value="es">Español</option>
      <option value="fr">Français</option>
      <option value="it">Italiano</option>
      <option value="nl">Nederlands</option>
      <option value="pt_br">Português (Brasil)</option>
      <option value="sv">Svenska</option>
      <option value="jp">日本語</option>
      <option value="zh_s">简体中文</option>
      <option value="zh_t">繁體中文</option>
      <option value="ko">한국어</option>
    </select><br><br>
      
    <label>3) Summarize contents？</label><br>
    <input type="radio" name="summarize" value="yes" checked>Yes
    <input type="radio" name="summarize" value="no">No<br><br><br>
    <input type="submit" value="Translate">
  </form>

</body>
</html>

Also, create html file to display translated context and to display summary, and save it with filename 'translated.html' under templates folder.

<!DOCTYPE html>
<html>
<head>
    <title>Translation</title>
</head>
<body>
    <h1>Translated Content</h1>
    <div>{{ content|safe }}</div>
    <h2>Summary</h2>
    <div>{{ summary }}</div>
</body>
</html>

4. Make sure the directory structure is this:

5. Run the actual Web App by running this line (If you are using python3, replace 'python' command with 'python3' command):

If you see this in your Terminal window, the Web App is running as expected:

Now just open a web browser and point to http://127.0.0.1:5000/ and you should see URL entry form. Just enter the URL of the page you want to translate and summarize, select one of the languages from the list, select 'Yes', and then click on the 'Translate' button to see the translation and summary.